Re: URL Dispatcher

2015-09-09 Thread James Addison
Marten, good to have you back in the fold. I'll leave the effort in your
hands (phew!) - as before, let me know if I can help in any way. I'm fairly
new to Django internals, so don't expect much!

I can test, however, and help with docs somewhat (based on limited
knowledge).

Cheers,
James
On Sep 9, 2015 2:52 AM, "Marten Kenbeek" <marten.k...@gmail.com> wrote:

> Hi James,
>
> Thanks for keeping this alive while I had some personal issues to sort
> out. I'll be working on this again, though I can use all the help I can
> get.
>
> As for the work that still needs to be done:
>
>- 3.1 is done, mostly as a proof-of-concept that can be found at
>https://github.com/knbk/django/tree/url_dispatcher
>- 3.2 is done. I've worked on this in separate PR's, which have all
>been merged by now.
>- 3.3:
>   - Legacy functionality has been implemented, namely through the
>   RegexPattern, LocalizedRegexPattern and LocalePrefix constraints.
>   - I've kept a lot of functional tests, and I believe the unit tests
>   have decent coverage. It would be good to have more tests, but I don't
>   think it's a necessity.
>   - Yes, pretty much all the documentation must still be written.
>   - These could be implemented pretty straight-forward in separate
>   patches after the main project has been merged.
>
> The resolver is not quite as fast as I'd hoped, especially `reverse()`.
> There are some trade-offs we can make, but we'll have to decide which are
> worth it.
> I think that is the main problem we need to tackle before we push for a
> merge.
>
> Marten
>
> Op dinsdag 25 augustus 2015 20:57:14 UTC+2 schreef Tim Graham:
>>
>> Thanks for taking this on, James. I do think it would be best to defer
>> this to Django 1.10 rather than try to rush it into Django 1.9 at this
>> point. Also, there are several URL related deprecations scheduled to
>> complete in 1.10 which should simplify things (I've started on those
>> removals here: https://github.com/django/django/pull/5143). Absent
>> another interested person, I should be able to mentor you on this during
>> the Django 1.10 release cycle (which can start once we cut the stable/1.9.x
>> branch after 1.9 alpha around September 21). In the meantime, if you want
>> take a look for some simpler tickets to tackle to get a feel for our
>> contribution process, I think that would be helpful.
>>
>> On Tuesday, August 25, 2015 at 2:25:07 PM UTC-4, James Addison wrote:
>>>
>>> In the interests of keeping the excellent work done by Marten Kenbeek
>>> (knbk on IRC/Github) on the GSOC 2015 URL Dispatcher project [1] current
>>> and moving along, I've created a new branch [2] and rebased it against
>>> current master. (I am doing this because it seems like Marten is no longer
>>> able to continue working on the effort. *If this should change or is
>>> incorrect, Marten, please let me/us know!*)
>>>
>>> To try and find how far along the project is, I've quickly compared the
>>> original GSOC outline/timeline [1] against my updated branch [2] to see
>>> what is done/remaining:
>>>
>>>- section 3.1 (Design phase) I believe was done, but am unable to
>>>find anything other than a few posts on the django-developer mailing 
>>> list - *is
>>>there a source for this?*
>>>- section 3.2 (Namespace overhaul) hasn't been addressed that I can
>>>tell - *I'd recommend dropping this as it isn't core to the effort*
>>>- Items completed in section 3.3 (Dispatcher API): 3.3.[1-4]
>>>- *Items remaining* in section 3.3: (Dispatcher API): 3.3.[5-9] -
>>>some highlights:
>>>   - re-implementing some legacy functionality for backward
>>>   compatibility (ie. RegexURLPattern)
>>>   - additional tests
>>>   - lots of documentation!
>>>   - additional resolvers (ie. scheme, sub-domain, etc)
>>>
>>>
>>>
>>> Please keep in mind that I'm mostly unfamiliar with the internals of
>>> Django core elements, so I may have easily overlooked/mistook relevant
>>> information. I'd appreciate any other review/input as well. Here is the
>>> branch comparison [3] on Github.
>>>
>>> Based on the Django 1.9 release schedule, I have doubts that this will
>>> be merged in for feature freeze, which is unfortunate. I would like to
>>> think that I am up to the task, but the URL functionality in Django is
>>> about as 'core' as it gets and I would not want to bite off more than I can
>>> chew without backup. 

Re: URL Dispatcher

2015-09-09 Thread Marten Kenbeek
Hi James,

Thanks for keeping this alive while I had some personal issues to sort out. 
I'll be working on this again, though I can use all the help I can get. 

As for the work that still needs to be done:

   - 3.1 is done, mostly as a proof-of-concept that can be found 
   at https://github.com/knbk/django/tree/url_dispatcher
   - 3.2 is done. I've worked on this in separate PR's, which have all been 
   merged by now. 
   - 3.3: 
  - Legacy functionality has been implemented, namely through the 
  RegexPattern, LocalizedRegexPattern and LocalePrefix constraints.
  - I've kept a lot of functional tests, and I believe the unit tests 
  have decent coverage. It would be good to have more tests, but I don't 
  think it's a necessity. 
  - Yes, pretty much all the documentation must still be written. 
  - These could be implemented pretty straight-forward in separate 
  patches after the main project has been merged. 
   
The resolver is not quite as fast as I'd hoped, especially `reverse()`. 
There are some trade-offs we can make, but we'll have to decide which are 
worth it. 
I think that is the main problem we need to tackle before we push for a 
merge. 

Marten

Op dinsdag 25 augustus 2015 20:57:14 UTC+2 schreef Tim Graham:
>
> Thanks for taking this on, James. I do think it would be best to defer 
> this to Django 1.10 rather than try to rush it into Django 1.9 at this 
> point. Also, there are several URL related deprecations scheduled to 
> complete in 1.10 which should simplify things (I've started on those 
> removals here: https://github.com/django/django/pull/5143). Absent 
> another interested person, I should be able to mentor you on this during 
> the Django 1.10 release cycle (which can start once we cut the stable/1.9.x 
> branch after 1.9 alpha around September 21). In the meantime, if you want 
> take a look for some simpler tickets to tackle to get a feel for our 
> contribution process, I think that would be helpful.
>
> On Tuesday, August 25, 2015 at 2:25:07 PM UTC-4, James Addison wrote:
>>
>> In the interests of keeping the excellent work done by Marten Kenbeek 
>> (knbk on IRC/Github) on the GSOC 2015 URL Dispatcher project [1] current 
>> and moving along, I've created a new branch [2] and rebased it against 
>> current master. (I am doing this because it seems like Marten is no longer 
>> able to continue working on the effort. *If this should change or is 
>> incorrect, Marten, please let me/us know!*)
>>
>> To try and find how far along the project is, I've quickly compared the 
>> original GSOC outline/timeline [1] against my updated branch [2] to see 
>> what is done/remaining:
>>
>>- section 3.1 (Design phase) I believe was done, but am unable to 
>>find anything other than a few posts on the django-developer mailing list 
>> - *is 
>>there a source for this?*
>>- section 3.2 (Namespace overhaul) hasn't been addressed that I can 
>>tell - *I'd recommend dropping this as it isn't core to the effort*
>>- Items completed in section 3.3 (Dispatcher API): 3.3.[1-4]
>>- *Items remaining* in section 3.3: (Dispatcher API): 3.3.[5-9] - 
>>some highlights:
>>   - re-implementing some legacy functionality for backward 
>>   compatibility (ie. RegexURLPattern)
>>   - additional tests
>>   - lots of documentation!
>>   - additional resolvers (ie. scheme, sub-domain, etc)
>>
>>
>>
>> Please keep in mind that I'm mostly unfamiliar with the internals of 
>> Django core elements, so I may have easily overlooked/mistook relevant 
>> information. I'd appreciate any other review/input as well. Here is the 
>> branch comparison [3] on Github.
>>
>> Based on the Django 1.9 release schedule, I have doubts that this will be 
>> merged in for feature freeze, which is unfortunate. I would like to think 
>> that I am up to the task, but the URL functionality in Django is about as 
>> 'core' as it gets and I would not want to bite off more than I can chew 
>> without backup. Any thoughts on the continued viability of this effort? Is 
>> this something that someone inexperienced with Django internals should take 
>> on? If so, I'd definitely need coaching!
>>
>> Lastly, I want to mention that I do have a new project in which I had 
>> been using Marten's branch (and am now using my own branch [2]), and it has 
>> been functionally solid for the scheme, (sub) domain and url resolving and 
>> reversing use cases. See the the urls.py [4] from that project to whet 
>> your appetites as to the possibilities of all this. :)
>>
>> Previous discussions on the django-developers mailing list [

Re: URL Dispatcher

2015-08-25 Thread Tim Graham
Thanks for taking this on, James. I do think it would be best to defer this 
to Django 1.10 rather than try to rush it into Django 1.9 at this point. 
Also, there are several URL related deprecations scheduled to complete in 
1.10 which should simplify things (I've started on those removals here: 
https://github.com/django/django/pull/5143). Absent another interested 
person, I should be able to mentor you on this during the Django 1.10 
release cycle (which can start once we cut the stable/1.9.x branch after 
1.9 alpha around September 21). In the meantime, if you want take a look 
for some simpler tickets to tackle to get a feel for our contribution 
process, I think that would be helpful.

On Tuesday, August 25, 2015 at 2:25:07 PM UTC-4, James Addison wrote:
>
> In the interests of keeping the excellent work done by Marten Kenbeek 
> (knbk on IRC/Github) on the GSOC 2015 URL Dispatcher project [1] current 
> and moving along, I've created a new branch [2] and rebased it against 
> current master. (I am doing this because it seems like Marten is no longer 
> able to continue working on the effort. *If this should change or is 
> incorrect, Marten, please let me/us know!*)
>
> To try and find how far along the project is, I've quickly compared the 
> original GSOC outline/timeline [1] against my updated branch [2] to see 
> what is done/remaining:
>
>- section 3.1 (Design phase) I believe was done, but am unable to find 
>anything other than a few posts on the django-developer mailing list - *is 
>there a source for this?*
>- section 3.2 (Namespace overhaul) hasn't been addressed that I can 
>tell - *I'd recommend dropping this as it isn't core to the effort*
>- Items completed in section 3.3 (Dispatcher API): 3.3.[1-4]
>- *Items remaining* in section 3.3: (Dispatcher API): 3.3.[5-9] - some 
>highlights:
>   - re-implementing some legacy functionality for backward 
>   compatibility (ie. RegexURLPattern)
>   - additional tests
>   - lots of documentation!
>   - additional resolvers (ie. scheme, sub-domain, etc)
>
>
>
> Please keep in mind that I'm mostly unfamiliar with the internals of 
> Django core elements, so I may have easily overlooked/mistook relevant 
> information. I'd appreciate any other review/input as well. Here is the 
> branch comparison [3] on Github.
>
> Based on the Django 1.9 release schedule, I have doubts that this will be 
> merged in for feature freeze, which is unfortunate. I would like to think 
> that I am up to the task, but the URL functionality in Django is about as 
> 'core' as it gets and I would not want to bite off more than I can chew 
> without backup. Any thoughts on the continued viability of this effort? Is 
> this something that someone inexperienced with Django internals should take 
> on? If so, I'd definitely need coaching!
>
> Lastly, I want to mention that I do have a new project in which I had been 
> using Marten's branch (and am now using my own branch [2]), and it has been 
> functionally solid for the scheme, (sub) domain and url resolving and 
> reversing use cases. See the the urls.py [4] from that project to whet 
> your appetites as to the possibilities of all this. :)
>
> Previous discussions on the django-developers mailing list [5].
>
> [1] 
> https://www.google-melange.com/gsoc/project/details/google/gsoc2015/knbk/5668600916475904
> [2] https://github.com/jaddison/django/tree/gsoc2015_url_dispatcher
> [3] 
> https://github.com/django/django/compare/master...jaddison:gsoc2015_url_dispatcher
> [4] https://gist.github.com/jaddison/18dea1bf93767f326aa5
> [5] 
> https://groups.google.com/forum/#!searchin/django-developers/marten/django-developers/IoivPLHJ_nw
>  
> and 
> https://groups.google.com/forum/#!searchin/django-developers/marten/django-developers/WQoJN_MGTOg
>  
> (most recent)
>
> Cheers,
> James Addison
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/5c0ff7fb-28ec-41c2-bb1a-8f76caf35ddc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


URL Dispatcher

2015-08-25 Thread James Addison
In the interests of keeping the excellent work done by Marten Kenbeek (knbk 
on IRC/Github) on the GSOC 2015 URL Dispatcher project [1] current and 
moving along, I've created a new branch [2] and rebased it against current 
master. (I am doing this because it seems like Marten is no longer able to 
continue working on the effort. *If this should change or is incorrect, 
Marten, please let me/us know!*)

To try and find how far along the project is, I've quickly compared the 
original GSOC outline/timeline [1] against my updated branch [2] to see 
what is done/remaining:

   - section 3.1 (Design phase) I believe was done, but am unable to find 
   anything other than a few posts on the django-developer mailing list - *is 
   there a source for this?*
   - section 3.2 (Namespace overhaul) hasn't been addressed that I can tell 
   - *I'd recommend dropping this as it isn't core to the effort*
   - Items completed in section 3.3 (Dispatcher API): 3.3.[1-4]
   - *Items remaining* in section 3.3: (Dispatcher API): 3.3.[5-9] - some 
   highlights:
  - re-implementing some legacy functionality for backward 
  compatibility (ie. RegexURLPattern)
  - additional tests
  - lots of documentation!
  - additional resolvers (ie. scheme, sub-domain, etc)
   

   
Please keep in mind that I'm mostly unfamiliar with the internals of Django 
core elements, so I may have easily overlooked/mistook relevant 
information. I'd appreciate any other review/input as well. Here is the 
branch comparison [3] on Github.

Based on the Django 1.9 release schedule, I have doubts that this will be 
merged in for feature freeze, which is unfortunate. I would like to think 
that I am up to the task, but the URL functionality in Django is about as 
'core' as it gets and I would not want to bite off more than I can chew 
without backup. Any thoughts on the continued viability of this effort? Is 
this something that someone inexperienced with Django internals should take 
on? If so, I'd definitely need coaching!

Lastly, I want to mention that I do have a new project in which I had been 
using Marten's branch (and am now using my own branch [2]), and it has been 
functionally solid for the scheme, (sub) domain and url resolving and 
reversing use cases. See the the urls.py [4] from that project to whet your 
appetites as to the possibilities of all this. :)

Previous discussions on the django-developers mailing list [5].

[1] 
https://www.google-melange.com/gsoc/project/details/google/gsoc2015/knbk/5668600916475904
[2] https://github.com/jaddison/django/tree/gsoc2015_url_dispatcher
[3] 
https://github.com/django/django/compare/master...jaddison:gsoc2015_url_dispatcher
[4] https://gist.github.com/jaddison/18dea1bf93767f326aa5
[5] 
https://groups.google.com/forum/#!searchin/django-developers/marten/django-developers/IoivPLHJ_nw
 
and 
https://groups.google.com/forum/#!searchin/django-developers/marten/django-developers/WQoJN_MGTOg
 
(most recent)

Cheers,
James Addison

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/8fe229a5-4418-4d14-8f41-1bf8613e32c9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: URL dispatcher API

2015-08-24 Thread Tim Graham
You would have to take a look at the pull request and figure out the 
status. As far as I know, there aren't any status updates elsewhere.

On Monday, August 24, 2015 at 11:52:49 AM UTC-4, James Addison wrote:
>
> Assuming 
> https://www.google-melange.com/gsoc/project/details/google/gsoc2015/knbk/5668600916475904
>  
> is the canonical source for the project details, what is the status of each 
> of its goals? From a high level, the project seems well described on that 
> page, but everything after (ie. status) seems opaque - unless there is 
> another resource?
>
> It would be a shame for something so powerful to fall by the wayside. I'm 
> sure I'm not the only one to see such value in this addition.
>
> On Monday, August 24, 2015 at 10:36:53 AM UTC-4, Tim Graham wrote:
>>
>> I don't think so. Marten has been MIA unfortunately. Someone else could 
>> try to pick it up, but it seems a bit ambitious to complete in the time 
>> remaining before alpha.
>>
>> On Monday, August 24, 2015 at 10:02:38 AM UTC-4, James Addison wrote:
>>>
>>> Hi Marten,
>>>
>>> I'm just curious if this is still on track for inclusion within the Sept 
>>> 21 Apha feature deadline? I've been using your Django `dispatcher_api` 
>>> branch for a new project since I learned of your proposed changes and am 
>>> excited to see them in stable Django. There hasn't been any changes in 
>>> about a month on your branch, so I'm (hopefully understandably) curious as 
>>> to its future?
>>>
>>> Cheers,
>>> James
>>>
>>> On Friday, July 10, 2015 at 4:55:22 PM UTC-4, Marten Kenbeek wrote:

 Hi James,

 Thanks for taking a look at this, I really appreciate it. 

 *Major bug:* the `request` object needs to be passed into `resolve()` 
> here: 
> https://github.com/knbk/django/blob/4a9d2a05bb76c9ad996921b9efadd8dad877540e/django/core/handlers/base.py#L134
>  
> - otherwise host and scheme constraints cannot work. I believe there are 
> other instances of `resolve()` not getting `request`, but mostly in 
> tests. 
> Is there a way for `request` to be a required parameter instead of 
> defaulting to `None`?
>

 Damn... I'll fix this asap. I haven't implemented any request-based 
 constraints yet, which allowed this to slip through without any test 
 failures. I'm afraid I can't just make `request` a required parameter, as 
 it needs to maintain backwards compatibility. With that said, the current 
 function signature of `resolve()` is just... awkward. One option is to 
 change it to `resolve(request_or_path, urlconf=None)`, and get the 
 path_info from the request if a request is supplied. `resolve()` might 
 still be called in a context where there is not request available, so I 
 think we need to maintain the ability to resolve just the path. 
  

> *Minor bug:* Given two subdomains, my.example.com and localhost:8000, 
> going to a url using the 'localhost:8000' subdomain that only exists 
> on the 'my' subdomain (ie. http://my.example.com/test3/ exists, but 
> you try to go to http://localhost:8000/test3/), the debug mode 
> 'patterns tried list' is a series of blank lines. See image below:
>

 I would've expected at least the regexes here. I'll look into this. 

 *​Nice to have:* When attempting to use multiple constraints (ie. list 
> or tuple of constraints), using `RegexPattern` seems to be required 
> when doing pattern matching - otherwise it messes up while calling 
> `construct`. First glance says this is by design? I think it would be 
> nice 
> to still be able to use the old r'' (without wrapping in 
> `RegexPattern`) syntax as well. Not critical, as the multi-constraints is 
> NOT breaking behaviour, just new.
>

 Yes, this is more or less by design. While in the case of a single 
 constraint it's nice and concise. Once you use a list of constraints 
 you'll 
 have to start counting brackets to see if that string is a RegexPattern or 
 a parameter to another constraint, or you'll put each constraint on a new 
 line and it doesn't matter as much imo. It isn't set in stone, so if you 
 have any compelling arguments to change this, let me know. 
  

> *Nice to have:* I realise this isn't likely to happen at all, but it 
> would be nice if when `reverse()` and `{% url %}` are called, it would be 
> good to be able to automatically drop the scheme and host when 
> reconstituting an absolute URL if the scheme and host of the current 
> request match.  However, I'm pretty sure that this is not possible, given 
> the various scenarios in which these methods can be called. Obviously, 
> this 
> is not required, as the resulting URL (with scheme/host or without when 
> matching) will still work regardless!
>

 I'll probably implement something like this before it is merged. I've 
 

Re: URL dispatcher API

2015-08-24 Thread James Addison
Assuming 
https://www.google-melange.com/gsoc/project/details/google/gsoc2015/knbk/5668600916475904
 
is the canonical source for the project details, what is the status of each 
of its goals? From a high level, the project seems well described on that 
page, but everything after (ie. status) seems opaque - unless there is 
another resource?

It would be a shame for something so powerful to fall by the wayside. I'm 
sure I'm not the only one to see such value in this addition.

On Monday, August 24, 2015 at 10:36:53 AM UTC-4, Tim Graham wrote:
>
> I don't think so. Marten has been MIA unfortunately. Someone else could 
> try to pick it up, but it seems a bit ambitious to complete in the time 
> remaining before alpha.
>
> On Monday, August 24, 2015 at 10:02:38 AM UTC-4, James Addison wrote:
>>
>> Hi Marten,
>>
>> I'm just curious if this is still on track for inclusion within the Sept 
>> 21 Apha feature deadline? I've been using your Django `dispatcher_api` 
>> branch for a new project since I learned of your proposed changes and am 
>> excited to see them in stable Django. There hasn't been any changes in 
>> about a month on your branch, so I'm (hopefully understandably) curious as 
>> to its future?
>>
>> Cheers,
>> James
>>
>> On Friday, July 10, 2015 at 4:55:22 PM UTC-4, Marten Kenbeek wrote:
>>>
>>> Hi James,
>>>
>>> Thanks for taking a look at this, I really appreciate it. 
>>>
>>> *Major bug:* the `request` object needs to be passed into `resolve()` 
 here: 
 https://github.com/knbk/django/blob/4a9d2a05bb76c9ad996921b9efadd8dad877540e/django/core/handlers/base.py#L134
  
 - otherwise host and scheme constraints cannot work. I believe there are 
 other instances of `resolve()` not getting `request`, but mostly in tests. 
 Is there a way for `request` to be a required parameter instead of 
 defaulting to `None`?

>>>
>>> Damn... I'll fix this asap. I haven't implemented any request-based 
>>> constraints yet, which allowed this to slip through without any test 
>>> failures. I'm afraid I can't just make `request` a required parameter, as 
>>> it needs to maintain backwards compatibility. With that said, the current 
>>> function signature of `resolve()` is just... awkward. One option is to 
>>> change it to `resolve(request_or_path, urlconf=None)`, and get the 
>>> path_info from the request if a request is supplied. `resolve()` might 
>>> still be called in a context where there is not request available, so I 
>>> think we need to maintain the ability to resolve just the path. 
>>>  
>>>
 *Minor bug:* Given two subdomains, my.example.com and localhost:8000, 
 going to a url using the 'localhost:8000' subdomain that only exists 
 on the 'my' subdomain (ie. http://my.example.com/test3/ exists, but 
 you try to go to http://localhost:8000/test3/), the debug mode 
 'patterns tried list' is a series of blank lines. See image below:

>>>
>>> I would've expected at least the regexes here. I'll look into this. 
>>>
>>> *​Nice to have:* When attempting to use multiple constraints (ie. list 
 or tuple of constraints), using `RegexPattern` seems to be required 
 when doing pattern matching - otherwise it messes up while calling 
 `construct`. First glance says this is by design? I think it would be nice 
 to still be able to use the old r'' (without wrapping in 
 `RegexPattern`) syntax as well. Not critical, as the multi-constraints is 
 NOT breaking behaviour, just new.

>>>
>>> Yes, this is more or less by design. While in the case of a single 
>>> constraint it's nice and concise. Once you use a list of constraints you'll 
>>> have to start counting brackets to see if that string is a RegexPattern or 
>>> a parameter to another constraint, or you'll put each constraint on a new 
>>> line and it doesn't matter as much imo. It isn't set in stone, so if you 
>>> have any compelling arguments to change this, let me know. 
>>>  
>>>
 *Nice to have:* I realise this isn't likely to happen at all, but it 
 would be nice if when `reverse()` and `{% url %}` are called, it would be 
 good to be able to automatically drop the scheme and host when 
 reconstituting an absolute URL if the scheme and host of the current 
 request match.  However, I'm pretty sure that this is not possible, given 
 the various scenarios in which these methods can be called. Obviously, 
 this 
 is not required, as the resulting URL (with scheme/host or without when 
 matching) will still work regardless!

>>>
>>> I'll probably implement something like this before it is merged. I've 
>>> been struggling to implement the `URL` class in a way that provided such 
>>> methods in a way that fully conformed RFC 3986, but it was taking up too 
>>> much of my time and I dropped it for now. I'm not sure I'll get it to 
>>> conform to RFC 3986, but I'll surely write some basic implementation to 
>>> output urls relative to the 

Re: URL dispatcher API

2015-08-24 Thread Tim Graham
I don't think so. Marten has been MIA unfortunately. Someone else could try 
to pick it up, but it seems a bit ambitious to complete in the time 
remaining before alpha.

On Monday, August 24, 2015 at 10:02:38 AM UTC-4, James Addison wrote:
>
> Hi Marten,
>
> I'm just curious if this is still on track for inclusion within the Sept 
> 21 Apha feature deadline? I've been using your Django `dispatcher_api` 
> branch for a new project since I learned of your proposed changes and am 
> excited to see them in stable Django. There hasn't been any changes in 
> about a month on your branch, so I'm (hopefully understandably) curious as 
> to its future?
>
> Cheers,
> James
>
> On Friday, July 10, 2015 at 4:55:22 PM UTC-4, Marten Kenbeek wrote:
>>
>> Hi James,
>>
>> Thanks for taking a look at this, I really appreciate it. 
>>
>> *Major bug:* the `request` object needs to be passed into `resolve()` 
>>> here: 
>>> https://github.com/knbk/django/blob/4a9d2a05bb76c9ad996921b9efadd8dad877540e/django/core/handlers/base.py#L134
>>>  
>>> - otherwise host and scheme constraints cannot work. I believe there are 
>>> other instances of `resolve()` not getting `request`, but mostly in tests. 
>>> Is there a way for `request` to be a required parameter instead of 
>>> defaulting to `None`?
>>>
>>
>> Damn... I'll fix this asap. I haven't implemented any request-based 
>> constraints yet, which allowed this to slip through without any test 
>> failures. I'm afraid I can't just make `request` a required parameter, as 
>> it needs to maintain backwards compatibility. With that said, the current 
>> function signature of `resolve()` is just... awkward. One option is to 
>> change it to `resolve(request_or_path, urlconf=None)`, and get the 
>> path_info from the request if a request is supplied. `resolve()` might 
>> still be called in a context where there is not request available, so I 
>> think we need to maintain the ability to resolve just the path. 
>>  
>>
>>> *Minor bug:* Given two subdomains, my.example.com and localhost:8000, 
>>> going to a url using the 'localhost:8000' subdomain that only exists on 
>>> the 'my' subdomain (ie. http://my.example.com/test3/ exists, but you 
>>> try to go to http://localhost:8000/test3/), the debug mode 'patterns 
>>> tried list' is a series of blank lines. See image below:
>>>
>>
>> I would've expected at least the regexes here. I'll look into this. 
>>
>> *​Nice to have:* When attempting to use multiple constraints (ie. list 
>>> or tuple of constraints), using `RegexPattern` seems to be required 
>>> when doing pattern matching - otherwise it messes up while calling 
>>> `construct`. First glance says this is by design? I think it would be nice 
>>> to still be able to use the old r'' (without wrapping in 
>>> `RegexPattern`) syntax as well. Not critical, as the multi-constraints is 
>>> NOT breaking behaviour, just new.
>>>
>>
>> Yes, this is more or less by design. While in the case of a single 
>> constraint it's nice and concise. Once you use a list of constraints you'll 
>> have to start counting brackets to see if that string is a RegexPattern or 
>> a parameter to another constraint, or you'll put each constraint on a new 
>> line and it doesn't matter as much imo. It isn't set in stone, so if you 
>> have any compelling arguments to change this, let me know. 
>>  
>>
>>> *Nice to have:* I realise this isn't likely to happen at all, but it 
>>> would be nice if when `reverse()` and `{% url %}` are called, it would be 
>>> good to be able to automatically drop the scheme and host when 
>>> reconstituting an absolute URL if the scheme and host of the current 
>>> request match.  However, I'm pretty sure that this is not possible, given 
>>> the various scenarios in which these methods can be called. Obviously, this 
>>> is not required, as the resulting URL (with scheme/host or without when 
>>> matching) will still work regardless!
>>>
>>
>> I'll probably implement something like this before it is merged. I've 
>> been struggling to implement the `URL` class in a way that provided such 
>> methods in a way that fully conformed RFC 3986, but it was taking up too 
>> much of my time and I dropped it for now. I'm not sure I'll get it to 
>> conform to RFC 3986, but I'll surely write some basic implementation to 
>> output urls relative to the current host and scheme. Of course it won't 
>> cover 100% of the cases as the "current" host and scheme are not always 
>> available, but in the basic cases it should be doable. 
>>
>> Op vrijdag 10 juli 2015 15:21:36 UTC+2 schreef Tim Graham:
>>>
>>> Marten, did you consider putting the new API in `django.urls` instead of 
>>> `django.core.urls`? I don't need there's a need to namespace it under 
>>> "core", but others might be able to confirm the philosophy behind this.
>>>
>>
>> That's certainly worth considering, in fact, I quite like the idea. If 
>> there are no objections (philosophical or otherwise) I'll move it to 
>> `django.urls`. 

Re: URL dispatcher API

2015-08-24 Thread James Addison
Hi Marten,

I'm just curious if this is still on track for inclusion within the Sept 21 
Apha feature deadline? I've been using your Django `dispatcher_api` branch 
for a new project since I learned of your proposed changes and am excited 
to see them in stable Django. There hasn't been any changes in about a 
month on your branch, so I'm (hopefully understandably) curious as to its 
future?

Cheers,
James

On Friday, July 10, 2015 at 4:55:22 PM UTC-4, Marten Kenbeek wrote:
>
> Hi James,
>
> Thanks for taking a look at this, I really appreciate it. 
>
> *Major bug:* the `request` object needs to be passed into `resolve()` 
>> here: 
>> https://github.com/knbk/django/blob/4a9d2a05bb76c9ad996921b9efadd8dad877540e/django/core/handlers/base.py#L134
>>  
>> - otherwise host and scheme constraints cannot work. I believe there are 
>> other instances of `resolve()` not getting `request`, but mostly in tests. 
>> Is there a way for `request` to be a required parameter instead of 
>> defaulting to `None`?
>>
>
> Damn... I'll fix this asap. I haven't implemented any request-based 
> constraints yet, which allowed this to slip through without any test 
> failures. I'm afraid I can't just make `request` a required parameter, as 
> it needs to maintain backwards compatibility. With that said, the current 
> function signature of `resolve()` is just... awkward. One option is to 
> change it to `resolve(request_or_path, urlconf=None)`, and get the 
> path_info from the request if a request is supplied. `resolve()` might 
> still be called in a context where there is not request available, so I 
> think we need to maintain the ability to resolve just the path. 
>  
>
>> *Minor bug:* Given two subdomains, my.example.com and localhost:8000, 
>> going to a url using the 'localhost:8000' subdomain that only exists on 
>> the 'my' subdomain (ie. http://my.example.com/test3/ exists, but you try 
>> to go to http://localhost:8000/test3/), the debug mode 'patterns tried 
>> list' is a series of blank lines. See image below:
>>
>
> I would've expected at least the regexes here. I'll look into this. 
>
> *​Nice to have:* When attempting to use multiple constraints (ie. list or 
>> tuple of constraints), using `RegexPattern` seems to be required when 
>> doing pattern matching - otherwise it messes up while calling `construct`. 
>> First glance says this is by design? I think it would be nice to still be 
>> able to use the old r'' (without wrapping in `RegexPattern`) 
>> syntax as well. Not critical, as the multi-constraints is NOT breaking 
>> behaviour, just new.
>>
>
> Yes, this is more or less by design. While in the case of a single 
> constraint it's nice and concise. Once you use a list of constraints you'll 
> have to start counting brackets to see if that string is a RegexPattern or 
> a parameter to another constraint, or you'll put each constraint on a new 
> line and it doesn't matter as much imo. It isn't set in stone, so if you 
> have any compelling arguments to change this, let me know. 
>  
>
>> *Nice to have:* I realise this isn't likely to happen at all, but it 
>> would be nice if when `reverse()` and `{% url %}` are called, it would be 
>> good to be able to automatically drop the scheme and host when 
>> reconstituting an absolute URL if the scheme and host of the current 
>> request match.  However, I'm pretty sure that this is not possible, given 
>> the various scenarios in which these methods can be called. Obviously, this 
>> is not required, as the resulting URL (with scheme/host or without when 
>> matching) will still work regardless!
>>
>
> I'll probably implement something like this before it is merged. I've been 
> struggling to implement the `URL` class in a way that provided such methods 
> in a way that fully conformed RFC 3986, but it was taking up too much of my 
> time and I dropped it for now. I'm not sure I'll get it to conform to RFC 
> 3986, but I'll surely write some basic implementation to output urls 
> relative to the current host and scheme. Of course it won't cover 100% of 
> the cases as the "current" host and scheme are not always available, but in 
> the basic cases it should be doable. 
>
> Op vrijdag 10 juli 2015 15:21:36 UTC+2 schreef Tim Graham:
>>
>> Marten, did you consider putting the new API in `django.urls` instead of 
>> `django.core.urls`? I don't need there's a need to namespace it under 
>> "core", but others might be able to confirm the philosophy behind this.
>>
>
> That's certainly worth considering, in fact, I quite like the idea. If 
> there are no objections (philosophical or otherwise) I'll move it to 
> `django.urls`. 
>
> Marten
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to 

Re: URL dispatcher API

2015-07-10 Thread Marc Tamlyn
+100 for django.urls. resolving and reversing functions should be located
in the same namespace.

M
On 10 Jul 2015 15:55, "Marten Kenbeek"  wrote:

> Hi James,
>
> Thanks for taking a look at this, I really appreciate it.
>
> *Major bug:* the `request` object needs to be passed into `resolve()`
>> here:
>> https://github.com/knbk/django/blob/4a9d2a05bb76c9ad996921b9efadd8dad877540e/django/core/handlers/base.py#L134
>> - otherwise host and scheme constraints cannot work. I believe there are
>> other instances of `resolve()` not getting `request`, but mostly in tests.
>> Is there a way for `request` to be a required parameter instead of
>> defaulting to `None`?
>>
>
> Damn... I'll fix this asap. I haven't implemented any request-based
> constraints yet, which allowed this to slip through without any test
> failures. I'm afraid I can't just make `request` a required parameter, as
> it needs to maintain backwards compatibility. With that said, the current
> function signature of `resolve()` is just... awkward. One option is to
> change it to `resolve(request_or_path, urlconf=None)`, and get the
> path_info from the request if a request is supplied. `resolve()` might
> still be called in a context where there is not request available, so I
> think we need to maintain the ability to resolve just the path.
>
>
>> *Minor bug:* Given two subdomains, my.example.com and localhost:8000,
>> going to a url using the 'localhost:8000' subdomain that only exists on
>> the 'my' subdomain (ie. http://my.example.com/test3/ exists, but you try
>> to go to http://localhost:8000/test3/), the debug mode 'patterns tried
>> list' is a series of blank lines. See image below:
>>
>
> I would've expected at least the regexes here. I'll look into this.
>
> *​Nice to have:* When attempting to use multiple constraints (ie. list or
>> tuple of constraints), using `RegexPattern` seems to be required when
>> doing pattern matching - otherwise it messes up while calling `construct`.
>> First glance says this is by design? I think it would be nice to still be
>> able to use the old r'' (without wrapping in `RegexPattern`)
>> syntax as well. Not critical, as the multi-constraints is NOT breaking
>> behaviour, just new.
>>
>
> Yes, this is more or less by design. While in the case of a single
> constraint it's nice and concise. Once you use a list of constraints you'll
> have to start counting brackets to see if that string is a RegexPattern or
> a parameter to another constraint, or you'll put each constraint on a new
> line and it doesn't matter as much imo. It isn't set in stone, so if you
> have any compelling arguments to change this, let me know.
>
>
>> *Nice to have:* I realise this isn't likely to happen at all, but it
>> would be nice if when `reverse()` and `{% url %}` are called, it would be
>> good to be able to automatically drop the scheme and host when
>> reconstituting an absolute URL if the scheme and host of the current
>> request match.  However, I'm pretty sure that this is not possible, given
>> the various scenarios in which these methods can be called. Obviously, this
>> is not required, as the resulting URL (with scheme/host or without when
>> matching) will still work regardless!
>>
>
> I'll probably implement something like this before it is merged. I've been
> struggling to implement the `URL` class in a way that provided such methods
> in a way that fully conformed RFC 3986, but it was taking up too much of my
> time and I dropped it for now. I'm not sure I'll get it to conform to RFC
> 3986, but I'll surely write some basic implementation to output urls
> relative to the current host and scheme. Of course it won't cover 100% of
> the cases as the "current" host and scheme are not always available, but in
> the basic cases it should be doable.
>
> Op vrijdag 10 juli 2015 15:21:36 UTC+2 schreef Tim Graham:
>>
>> Marten, did you consider putting the new API in `django.urls` instead of
>> `django.core.urls`? I don't need there's a need to namespace it under
>> "core", but others might be able to confirm the philosophy behind this.
>>
>
> That's certainly worth considering, in fact, I quite like the idea. If
> there are no objections (philosophical or otherwise) I'll move it to
> `django.urls`.
>
> Marten
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-developers/a8174a99-2364-4e52-9368-320e94810c62%40googlegroups.com
> 

Re: URL dispatcher API

2015-07-10 Thread Marten Kenbeek
Hi James,

Thanks for taking a look at this, I really appreciate it. 

*Major bug:* the `request` object needs to be passed into `resolve()` here: 
> https://github.com/knbk/django/blob/4a9d2a05bb76c9ad996921b9efadd8dad877540e/django/core/handlers/base.py#L134
>  
> - otherwise host and scheme constraints cannot work. I believe there are 
> other instances of `resolve()` not getting `request`, but mostly in tests. 
> Is there a way for `request` to be a required parameter instead of 
> defaulting to `None`?
>

Damn... I'll fix this asap. I haven't implemented any request-based 
constraints yet, which allowed this to slip through without any test 
failures. I'm afraid I can't just make `request` a required parameter, as 
it needs to maintain backwards compatibility. With that said, the current 
function signature of `resolve()` is just... awkward. One option is to 
change it to `resolve(request_or_path, urlconf=None)`, and get the 
path_info from the request if a request is supplied. `resolve()` might 
still be called in a context where there is not request available, so I 
think we need to maintain the ability to resolve just the path. 
 

> *Minor bug:* Given two subdomains, my.example.com and localhost:8000, 
> going to a url using the 'localhost:8000' subdomain that only exists on 
> the 'my' subdomain (ie. http://my.example.com/test3/ exists, but you try 
> to go to http://localhost:8000/test3/), the debug mode 'patterns tried 
> list' is a series of blank lines. See image below:
>

I would've expected at least the regexes here. I'll look into this. 

*​Nice to have:* When attempting to use multiple constraints (ie. list or 
> tuple of constraints), using `RegexPattern` seems to be required when 
> doing pattern matching - otherwise it messes up while calling `construct`. 
> First glance says this is by design? I think it would be nice to still be 
> able to use the old r'' (without wrapping in `RegexPattern`) 
> syntax as well. Not critical, as the multi-constraints is NOT breaking 
> behaviour, just new.
>

Yes, this is more or less by design. While in the case of a single 
constraint it's nice and concise. Once you use a list of constraints you'll 
have to start counting brackets to see if that string is a RegexPattern or 
a parameter to another constraint, or you'll put each constraint on a new 
line and it doesn't matter as much imo. It isn't set in stone, so if you 
have any compelling arguments to change this, let me know. 
 

> *Nice to have:* I realise this isn't likely to happen at all, but it 
> would be nice if when `reverse()` and `{% url %}` are called, it would be 
> good to be able to automatically drop the scheme and host when 
> reconstituting an absolute URL if the scheme and host of the current 
> request match.  However, I'm pretty sure that this is not possible, given 
> the various scenarios in which these methods can be called. Obviously, this 
> is not required, as the resulting URL (with scheme/host or without when 
> matching) will still work regardless!
>

I'll probably implement something like this before it is merged. I've been 
struggling to implement the `URL` class in a way that provided such methods 
in a way that fully conformed RFC 3986, but it was taking up too much of my 
time and I dropped it for now. I'm not sure I'll get it to conform to RFC 
3986, but I'll surely write some basic implementation to output urls 
relative to the current host and scheme. Of course it won't cover 100% of 
the cases as the "current" host and scheme are not always available, but in 
the basic cases it should be doable. 

Op vrijdag 10 juli 2015 15:21:36 UTC+2 schreef Tim Graham:
>
> Marten, did you consider putting the new API in `django.urls` instead of 
> `django.core.urls`? I don't need there's a need to namespace it under 
> "core", but others might be able to confirm the philosophy behind this.
>

That's certainly worth considering, in fact, I quite like the idea. If 
there are no objections (philosophical or otherwise) I'll move it to 
`django.urls`. 

Marten

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/a8174a99-2364-4e52-9368-320e94810c62%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: URL dispatcher API

2015-07-10 Thread Tim Graham
Marten, did you consider putting the new API in `django.urls` instead of 
`django.core.urls`? I don't need there's a need to namespace it under 
"core", but others might be able to confirm the philosophy behind this.

On Thursday, July 9, 2015 at 7:19:11 AM UTC-4, James Addison wrote:
>
> Marten,
>
> Using your *dispatcher_api* branch, I found a few issues - I believe this 
> is an acceptable place to post them. My apologies if this is not the case.
>
> My use case is this: Using a single Django codebase, I plan to have 
> www.example.com and my.example.com. 'my' will be https, while 'www' will 
> be http. 'my' will be for logged in actions (including signup/logging in), 
> and 'www' will be largely anonymous usage and cached using various 
> mechanisms. I've figured out how to do scheme and host based constraints 
> easily enough, thanks to a code sample you provided to me.  This is what 
> I've developed it into, and it works pretty well (once I got around the 
> *Major* issue listed below):
>
> from django.conf.urls import url, include
> from django.core.urls import Constraint, Resolver404
>
>
> class ConstraintBase(Constraint):
> """
> This exists purely as a way to ensure MRO mixin inheritance will work
> """
> def match(self, path, request=None):
> return path, (), {}
>
> def construct(self, url, *args, **kwargs):
> return url, args, kwargs
>
>
> class SchemeConstraintMixin(object):
> scheme = ''
>
> def match(self, path, request=None):
> if request and request.scheme == self.scheme:
> return super(SchemeConstraintMixin, self).match(path, request)
> raise Resolver404()
>
> def construct(self, url, *args, **kwargs):
> url.scheme = self.scheme
> return super(SchemeConstraintMixin, self).construct(url, *args, 
> **kwargs)
>
>
> class HostConstraintMixin(object):
> host = ''
>
> def match(self, path, request=None):
> if request and request.get_host() == self.host:
> return super(HostConstraintMixin, self).match(path, request)
> raise Resolver404()
>
> def construct(self, url, *args, **kwargs):
> url.host = self.host
> return super(HostConstraintMixin, self).construct(url, *args, 
> **kwargs)
>
>
> class SchemeHostConstraint(SchemeConstraintMixin, HostConstraintMixin, 
> ConstraintBase):
> def __init__(self, scheme, host):
> self.scheme = scheme
> self.host = host
>
>
> www_constraint = SchemeHostConstraint('http', 'localhost:8000')
> my_constraint = SchemeHostConstraint('https', 'my.example.com')
>
> urlpatterns = [
> url(www_constraint, include([
> url(r'^$', 'testviews.views.testview', {'template': 
> 'testview1.html'}, 'testview1'),
> url(r'^test2/$', 'testviews.views.testview', {'template': 
> 'testview2.html'}, 'testview2')
> ])),
> url(my_constraint, include([
> url(r'^test3/$', 'testviews.views.testview', {'template': 
> 'testview3.html'}, 'testview3'),
> url(r'^test4/$', 'testviews.views.testview', {'template': 
> 'testview3.html'}, 'testview4')
> ]))
> ]
>
> The view simply renders a template. The templates simply display the `{% 
> url %}` output of the available url patterns mentioned above. With that 
> preamble done, I'll mention my findings below:
>
> *Major bug:* the `request` object needs to be passed into `resolve()` 
> here: 
> https://github.com/knbk/django/blob/4a9d2a05bb76c9ad996921b9efadd8dad877540e/django/core/handlers/base.py#L134
>  
> - otherwise host and scheme constraints cannot work. I believe there are 
> other instances of `resolve()` not getting `request`, but mostly in tests. 
> Is there a way for `request` to be a required parameter instead of 
> defaulting to `None`?
>
> *Minor bug:* Given two subdomains, my.example.com and localhost:8000, 
> going to a url using the 'localhost:8000' subdomain that only exists on 
> the 'my' subdomain (ie. http://my.example.com/test3/ exists, but you try 
> to go to http://localhost:8000/test3/), the debug mode 'patterns tried 
> list' is a series of blank lines. See image below:
>
> 
>
>
> *​Nice to have:* When attempting to use multiple constraints (ie. list or 
> tuple of constraints), using `RegexPattern` seems to be required when 
> doing pattern matching - otherwise it messes up while calling `construct`. 
> First glance says this is by design? I think it would be nice to still be 
> able to use the old r'' (without wrapping in `RegexPattern`) 
> syntax as well. Not critical, as the multi-constraints is NOT breaking 
> behaviour, just new.
>
> *Nice to have:* I realise this isn't likely to happen at all, but it 
> would be nice if when `reverse()` and `{% url %}` are called, it would be 
> good to be able to automatically drop the scheme and host when 
> reconstituting an 

Re: URL dispatcher API

2015-07-09 Thread James Addison
Marten,

Using your *dispatcher_api* branch, I found a few issues - I believe this 
is an acceptable place to post them. My apologies if this is not the case.

My use case is this: Using a single Django codebase, I plan to have 
www.example.com and my.example.com. 'my' will be https, while 'www' will be 
http. 'my' will be for logged in actions (including signup/logging in), and 
'www' will be largely anonymous usage and cached using various mechanisms. 
I've figured out how to do scheme and host based constraints easily enough, 
thanks to a code sample you provided to me.  This is what I've developed it 
into, and it works pretty well (once I got around the *Major* issue listed 
below):

from django.conf.urls import url, include
from django.core.urls import Constraint, Resolver404


class ConstraintBase(Constraint):
"""
This exists purely as a way to ensure MRO mixin inheritance will work
"""
def match(self, path, request=None):
return path, (), {}

def construct(self, url, *args, **kwargs):
return url, args, kwargs


class SchemeConstraintMixin(object):
scheme = ''

def match(self, path, request=None):
if request and request.scheme == self.scheme:
return super(SchemeConstraintMixin, self).match(path, request)
raise Resolver404()

def construct(self, url, *args, **kwargs):
url.scheme = self.scheme
return super(SchemeConstraintMixin, self).construct(url, *args, 
**kwargs)


class HostConstraintMixin(object):
host = ''

def match(self, path, request=None):
if request and request.get_host() == self.host:
return super(HostConstraintMixin, self).match(path, request)
raise Resolver404()

def construct(self, url, *args, **kwargs):
url.host = self.host
return super(HostConstraintMixin, self).construct(url, *args, 
**kwargs)


class SchemeHostConstraint(SchemeConstraintMixin, HostConstraintMixin, 
ConstraintBase):
def __init__(self, scheme, host):
self.scheme = scheme
self.host = host


www_constraint = SchemeHostConstraint('http', 'localhost:8000')
my_constraint = SchemeHostConstraint('https', 'my.example.com')

urlpatterns = [
url(www_constraint, include([
url(r'^$', 'testviews.views.testview', {'template': 
'testview1.html'}, 'testview1'),
url(r'^test2/$', 'testviews.views.testview', {'template': 
'testview2.html'}, 'testview2')
])),
url(my_constraint, include([
url(r'^test3/$', 'testviews.views.testview', {'template': 
'testview3.html'}, 'testview3'),
url(r'^test4/$', 'testviews.views.testview', {'template': 
'testview3.html'}, 'testview4')
]))
]

The view simply renders a template. The templates simply display the `{% 
url %}` output of the available url patterns mentioned above. With that 
preamble done, I'll mention my findings below:

*Major bug:* the `request` object needs to be passed into `resolve()` here: 
https://github.com/knbk/django/blob/4a9d2a05bb76c9ad996921b9efadd8dad877540e/django/core/handlers/base.py#L134
 
- otherwise host and scheme constraints cannot work. I believe there are 
other instances of `resolve()` not getting `request`, but mostly in tests. 
Is there a way for `request` to be a required parameter instead of 
defaulting to `None`?

*Minor bug:* Given two subdomains, my.example.com and localhost:8000, going 
to a url using the 'localhost:8000' subdomain that only exists on the 'my' 
subdomain (ie. http://my.example.com/test3/ exists, but you try to go 
to http://localhost:8000/test3/), the debug mode 'patterns tried list' is a 
series of blank lines. See image below:



*​Nice to have:* When attempting to use multiple constraints (ie. list or 
tuple of constraints), using `RegexPattern` seems to be required when doing 
pattern matching - otherwise it messes up while calling `construct`. First 
glance says this is by design? I think it would be nice to still be able to 
use the old r'' (without wrapping in `RegexPattern`) syntax as 
well. Not critical, as the multi-constraints is NOT breaking behaviour, 
just new.

*Nice to have:* I realise this isn't likely to happen at all, but it would 
be nice if when `reverse()` and `{% url %}` are called, it would be good to 
be able to automatically drop the scheme and host when reconstituting an 
absolute URL if the scheme and host of the current request match.  However, 
I'm pretty sure that this is not possible, given the various scenarios in 
which these methods can be called. Obviously, this is not required, as the 
resulting URL (with scheme/host or without when matching) will still work 
regardless!

I hope this was clear. If there is a way I can be of more assistance, 
please let me know!  Nice work, by the way. I'm stoked to have this in 1.9.

James

-- 
You received this message 

Re: URL dispatcher API

2015-06-28 Thread Marten Kenbeek
ity anyway), and the decorator 
> functionality has been moved over there. The resolving functionality has 
> been moved to the Resolver class. This does feel quite a bit cleaner :)
>
> One important thing to keep in mind is reverse() performance.
>
>
> My (very limited) tests leave me optimistic. With some optimized 
> constraints, reversing could be a lot faster than the current 
> implementation, I believe.
>
> I am a little less acquainted with the internals of resolving and 
>> reversing than the people above, and I must say that I have a hard time 
>> understanding your new syntax when looking at the example gist. That might 
>> also have to do with the indentation style. Would the (blog_detail, 
>> blog_index) -> blog_resolver -> patterns be the replacement of the 
>> urlpatterns below? Because I think the latter is a lot more readable. 
>
>
> The example configuration is just to have a working example at hand. The 
> final patch will have url() and include() functions, just like the current 
> implementation. In reality you would use the urlpatterns at the bottom, not 
> the patterns at the top. You do have a point, I've tried to clean up the 
> example to be a bit more readable, and to work with the updated branch.
>
> Do you have an example of something that is easy to do with this API that 
>> is hard or impossible with the current API?
>
>
> Domain-based resolving, for one. Another, non-obvious advantage is that a 
> Resolver subclass can dynamically resolve or reverse patterns. A lot of 
> apps (think flatpages, a CMS, or the sitemaps app) have a semi-static set 
> of urls. The current approaches are limited to a catch-all pattern that 
> further resolves in the view, or a static url for each option. The first 
> approach has the obvious disadvantage that it can only ever be used once in 
> a single urlconf, and the view has to solve problems that the url 
> dispatcher should handle. With the second approach the old url dispatcher 
> makes it virtually impossible to make runtime changes. I'm sure that's not 
> all ;)
>
> I will probably create a separate thread to discuss the mess that is url 
> namespaces. 
>
> Marten
>
> Op zondag 17 mei 2015 16:27:32 UTC+2 schreef Tino de Bruijn:
>>
>> Hi Marten,
>>
>> I am a little less acquainted with the internals of resolving and 
>> reversing than the people above, and I must say that I have a hard time 
>> understanding your new syntax when looking at the example gist. That might 
>> also have to do with the indentation style. Would the (blog_detail, 
>> blog_index) -> blog_resolver -> patterns be the replacement of the 
>> urlpatterns below? Because I think the latter is a lot more readable. 
>>
>> Do you have an example of something that is easy to do with this API that 
>> is hard or impossible with the current API?
>>
>> Thanks,
>>
>>
>> Tino
>>
>> On Sun, May 17, 2015 at 2:35 PM, Anssi Kääriäinen <akaa...@gmail.com> 
>> wrote:
>>
>>> One important thing to keep in mind is reverse() performance.
>>>
>>> While it isn't important that the prototype is fast, making sure the API 
>>> doesn't limit performance for the most important operations must be checked.
>>>
>>> At least my experience is that URL resolving performance isn't that 
>>> important, but having fast reverse() is really important. The reason is 
>>> that single request typically has just one resolve operation, and thus the 
>>> amount of time used for resolving tends to end up in the noise. But large 
>>> list/table views can have up to thousands of reverse() operations, when 
>>> there are multiple urls per row.
>>>
>>> I believe we can have clean, fast and extensible URL resolving system, 
>>> so I'm not too worried about this. Still, lets make sure performance won't 
>>> be a problem.
>>>
>>>  - Anssi
>>>
>>>
>>> On Monday, May 4, 2015, Marten Kenbeek <marte...@gmail.com> wrote:
>>>
>>>> Hi all,
>>>>
>>>> I'd like to discuss the API I'm proposing for the new url dispatcher 
>>>> I'm working on. I'll try to explain the API and some of the rationale 
>>>> behind it.
>>>>
>>>> There is a working proof-of-concept at 
>>>> https://github.com/knbk/django/tree/url_dispatcher. Currently, all the 
>>>> names are chosen as not to conflict with the old implementation. A short 
>>>> description can be found at 
>>>> https://gist.github.com/knbk/96999abaab4ad4e5f8f9, which also contains 

Re: URL dispatcher API

2015-05-23 Thread Marten Kenbeek
 be used once in 
a single urlconf, and the view has to solve problems that the url 
dispatcher should handle. With the second approach the old url dispatcher 
makes it virtually impossible to make runtime changes. I'm sure that's not 
all ;)

I will probably create a separate thread to discuss the mess that is url 
namespaces. 

Marten

Op zondag 17 mei 2015 16:27:32 UTC+2 schreef Tino de Bruijn:
>
> Hi Marten,
>
> I am a little less acquainted with the internals of resolving and 
> reversing than the people above, and I must say that I have a hard time 
> understanding your new syntax when looking at the example gist. That might 
> also have to do with the indentation style. Would the (blog_detail, 
> blog_index) -> blog_resolver -> patterns be the replacement of the 
> urlpatterns below? Because I think the latter is a lot more readable. 
>
> Do you have an example of something that is easy to do with this API that 
> is hard or impossible with the current API?
>
> Thanks,
>
>
> Tino
>
> On Sun, May 17, 2015 at 2:35 PM, Anssi Kääriäinen <akaa...@gmail.com 
> > wrote:
>
>> One important thing to keep in mind is reverse() performance.
>>
>> While it isn't important that the prototype is fast, making sure the API 
>> doesn't limit performance for the most important operations must be checked.
>>
>> At least my experience is that URL resolving performance isn't that 
>> important, but having fast reverse() is really important. The reason is 
>> that single request typically has just one resolve operation, and thus the 
>> amount of time used for resolving tends to end up in the noise. But large 
>> list/table views can have up to thousands of reverse() operations, when 
>> there are multiple urls per row.
>>
>> I believe we can have clean, fast and extensible URL resolving system, so 
>> I'm not too worried about this. Still, lets make sure performance won't be 
>> a problem.
>>
>>  - Anssi
>>
>>
>> On Monday, May 4, 2015, Marten Kenbeek <marte...@gmail.com > 
>> wrote:
>>
>>> Hi all,
>>>
>>> I'd like to discuss the API I'm proposing for the new url dispatcher I'm 
>>> working on. I'll try to explain the API and some of the rationale behind it.
>>>
>>> There is a working proof-of-concept at 
>>> https://github.com/knbk/django/tree/url_dispatcher. Currently, all the 
>>> names are chosen as not to conflict with the old implementation. A short 
>>> description can be found at 
>>> https://gist.github.com/knbk/96999abaab4ad4e5f8f9, which also contains 
>>> a link to an example url configuration. 
>>>
>>> My main goal was to create a simple, extensible API. In the old API, 
>>> about 90% of the work was done in the `resolve()`, `_reverse_with_prefix()` 
>>> and `populate()` of the RegexURLResolver. This made for a tightly coupled 
>>> design that was almost impossible to extend.
>>>
>>> My starting point was the RegexURLResolver and RegexURLPattern. Both 
>>> have a regex constraint that can match the current path and extract 
>>> arguments. The former can then pass the remainder of the path to its list 
>>> of resolvers and patterns; the latter can return a ResolverMatch containing 
>>> the callback specified by that pattern. I've kept this distinction, with 
>>> the `Resolver` and `View` classes. The change of name from `Pattern`  to 
>>> `View` is because the `View` object has a bit more logic that determines 
>>> how the view is called. It is a thin, callable wrapper that can optionally 
>>> decorate the actual view function with a set of decorators passed down from 
>>> parent resolvers, and when overwritten, can do some more processing before 
>>> or after the view function is called. 
>>>
>>> The hard-coded dependence on a regex pattern has been abstracted to a 
>>> Constraint object. Each Resolver and View has a (set of) Constraint 
>>> object(s), that can match the current url and extract arguments. A 
>>> RegexPattern that simply mimics the old behaviour will be available, but 
>>> other implementations, such as a Constraint that matches the request's host 
>>> or method, are easily provided. A Constraint can also reverse a set of 
>>> arguments to a partial url. That means that the full set of constraints 
>>> used to match an url to a view, together with a suitable set of arguments, 
>>> can be reversed to the url itself.
>>>
>>> The main strength of a Constraint is that it can contain very specific 
>>> logic about its arguments.

Re: URL dispatcher API

2015-05-17 Thread Tino de Bruijn
Hi Marten,

I am a little less acquainted with the internals of resolving and reversing
than the people above, and I must say that I have a hard time understanding
your new syntax when looking at the example gist. That might also have to
do with the indentation style. Would the (blog_detail, blog_index) ->
blog_resolver -> patterns be the replacement of the urlpatterns below?
Because I think the latter is a lot more readable.

Do you have an example of something that is easy to do with this API that
is hard or impossible with the current API?

Thanks,


Tino

On Sun, May 17, 2015 at 2:35 PM, Anssi Kääriäinen <akaar...@gmail.com>
wrote:

> One important thing to keep in mind is reverse() performance.
>
> While it isn't important that the prototype is fast, making sure the API
> doesn't limit performance for the most important operations must be checked.
>
> At least my experience is that URL resolving performance isn't that
> important, but having fast reverse() is really important. The reason is
> that single request typically has just one resolve operation, and thus the
> amount of time used for resolving tends to end up in the noise. But large
> list/table views can have up to thousands of reverse() operations, when
> there are multiple urls per row.
>
> I believe we can have clean, fast and extensible URL resolving system, so
> I'm not too worried about this. Still, lets make sure performance won't be
> a problem.
>
>  - Anssi
>
>
> On Monday, May 4, 2015, Marten Kenbeek <marten.k...@gmail.com> wrote:
>
>> Hi all,
>>
>> I'd like to discuss the API I'm proposing for the new url dispatcher I'm
>> working on. I'll try to explain the API and some of the rationale behind it.
>>
>> There is a working proof-of-concept at
>> https://github.com/knbk/django/tree/url_dispatcher. Currently, all the
>> names are chosen as not to conflict with the old implementation. A short
>> description can be found at
>> https://gist.github.com/knbk/96999abaab4ad4e5f8f9, which also contains a
>> link to an example url configuration.
>>
>> My main goal was to create a simple, extensible API. In the old API,
>> about 90% of the work was done in the `resolve()`, `_reverse_with_prefix()`
>> and `populate()` of the RegexURLResolver. This made for a tightly coupled
>> design that was almost impossible to extend.
>>
>> My starting point was the RegexURLResolver and RegexURLPattern. Both have
>> a regex constraint that can match the current path and extract arguments.
>> The former can then pass the remainder of the path to its list of resolvers
>> and patterns; the latter can return a ResolverMatch containing the callback
>> specified by that pattern. I've kept this distinction, with the `Resolver`
>> and `View` classes. The change of name from `Pattern`  to `View` is because
>> the `View` object has a bit more logic that determines how the view is
>> called. It is a thin, callable wrapper that can optionally decorate the
>> actual view function with a set of decorators passed down from parent
>> resolvers, and when overwritten, can do some more processing before or
>> after the view function is called.
>>
>> The hard-coded dependence on a regex pattern has been abstracted to a
>> Constraint object. Each Resolver and View has a (set of) Constraint
>> object(s), that can match the current url and extract arguments. A
>> RegexPattern that simply mimics the old behaviour will be available, but
>> other implementations, such as a Constraint that matches the request's host
>> or method, are easily provided. A Constraint can also reverse a set of
>> arguments to a partial url. That means that the full set of constraints
>> used to match an url to a view, together with a suitable set of arguments,
>> can be reversed to the url itself.
>>
>> The main strength of a Constraint is that it can contain very specific
>> logic about its arguments. For example, a Constraint may know that it
>> resolves to a Model's primary key. If it then receives a Model instance of
>> that particular type, it will know how to reverse that model instance to a
>> valid string-based partial url, so that it can later be resolved to match
>> the same object. It could also e.g. infer the regex pattern from the
>> field's type.
>>
>> There's one final piece to the puzzle: the URL object. This is the state
>> of the url in the process of resolving or reversing an url. It's a two-way
>> street: when resolving, it starts out as a full path, and the Constraints
>> chip away at the path, while the set of constraints and extracted argument
>> grows. When reversing, it starts out as a set o

Re: URL dispatcher API

2015-05-17 Thread Anssi Kääriäinen
One important thing to keep in mind is reverse() performance.

While it isn't important that the prototype is fast, making sure the API
doesn't limit performance for the most important operations must be checked.

At least my experience is that URL resolving performance isn't that
important, but having fast reverse() is really important. The reason is
that single request typically has just one resolve operation, and thus the
amount of time used for resolving tends to end up in the noise. But large
list/table views can have up to thousands of reverse() operations, when
there are multiple urls per row.

I believe we can have clean, fast and extensible URL resolving system, so
I'm not too worried about this. Still, lets make sure performance won't be
a problem.

 - Anssi

On Monday, May 4, 2015, Marten Kenbeek <marten.k...@gmail.com> wrote:

> Hi all,
>
> I'd like to discuss the API I'm proposing for the new url dispatcher I'm
> working on. I'll try to explain the API and some of the rationale behind it.
>
> There is a working proof-of-concept at
> https://github.com/knbk/django/tree/url_dispatcher. Currently, all the
> names are chosen as not to conflict with the old implementation. A short
> description can be found at
> https://gist.github.com/knbk/96999abaab4ad4e5f8f9, which also contains a
> link to an example url configuration.
>
> My main goal was to create a simple, extensible API. In the old API, about
> 90% of the work was done in the `resolve()`, `_reverse_with_prefix()` and
> `populate()` of the RegexURLResolver. This made for a tightly coupled
> design that was almost impossible to extend.
>
> My starting point was the RegexURLResolver and RegexURLPattern. Both have
> a regex constraint that can match the current path and extract arguments.
> The former can then pass the remainder of the path to its list of resolvers
> and patterns; the latter can return a ResolverMatch containing the callback
> specified by that pattern. I've kept this distinction, with the `Resolver`
> and `View` classes. The change of name from `Pattern`  to `View` is because
> the `View` object has a bit more logic that determines how the view is
> called. It is a thin, callable wrapper that can optionally decorate the
> actual view function with a set of decorators passed down from parent
> resolvers, and when overwritten, can do some more processing before or
> after the view function is called.
>
> The hard-coded dependence on a regex pattern has been abstracted to a
> Constraint object. Each Resolver and View has a (set of) Constraint
> object(s), that can match the current url and extract arguments. A
> RegexPattern that simply mimics the old behaviour will be available, but
> other implementations, such as a Constraint that matches the request's host
> or method, are easily provided. A Constraint can also reverse a set of
> arguments to a partial url. That means that the full set of constraints
> used to match an url to a view, together with a suitable set of arguments,
> can be reversed to the url itself.
>
> The main strength of a Constraint is that it can contain very specific
> logic about its arguments. For example, a Constraint may know that it
> resolves to a Model's primary key. If it then receives a Model instance of
> that particular type, it will know how to reverse that model instance to a
> valid string-based partial url, so that it can later be resolved to match
> the same object. It could also e.g. infer the regex pattern from the
> field's type.
>
> There's one final piece to the puzzle: the URL object. This is the state
> of the url in the process of resolving or reversing an url. It's a two-way
> street: when resolving, it starts out as a full path, and the Constraints
> chip away at the path, while the set of constraints and extracted argument
> grows. When reversing, it starts out as a set of constraints and arguments,
> and reconstructs the partial urls from those constraints and arguments
> until a full url path is reconstructed. It shifts some of the logic from
> the Resolver to the URL, so that it is easier to extend the Resolver. It is
> also a simple container that allows any Constraint access to the full
> request. Last but not least, it allows to dynamically build an url against
> the current request. This is useful if e.g. a constraint matches a
> different subdomain than the current request, so that a link automatically
> points to the right domain.
>
> I'm looking forwards to your feedback.
>
> Thanks,
> Marten
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+u

Re: URL dispatcher API

2015-05-17 Thread Aymeric Augustin
Hi Marten,

I had a look at your proposal today. You've done good work there.

My comments revolve around making the API obvious, simplifying it until
someone who discovers it understands immediately what the moving pieces are,
what they do, and how they relate to one another.

Here are my two main reactions.


1) The interplay between URL and Constraint is unclear.

In particular I'm not sure which one is designed to be customized, perhaps by
subclassing. To what extent are constraints expected to poke into the URL's
internals?

The documentation shows that URL has methods that accept Constraint in
argument and Constraint has methods that accept URL in argument. Often such
dependency loops indicate that responsibilities should be split differently
between the two classes.

I also share Carl's concerns with regards to URL being mutable.

I don't have a ready-made solution to offer but two suggestions:

a. Try to organize classes in a tree where parent classes use their children's
  API but children don't known anything about their parents.

b. Try to clarify further the responsibilities of each class and make sure
   they don't know more than they need to achieve this purpose.

For example, having Constraint.match() accept a string in argument -- the
current URL path -- instead of an URL object would prevent Constraint from
depending on URL's API.


2) The Resolver / View API can be simplified.

Is there a fundamental reason to have two different classes here? If so, is
the split in responsibilities optimal?

As far as I can tell, there's some overlap between the two classes, especially
in `resolve()`, but:

- Resolver has additional code for resolving further
- View has additional code for calling a view

My gut feeling is that View should be subsumed into Resolver.


Finally, if you haven't already, you should read about Simon Willison's theory
of "Turtles all the way down":

https://web.archive.org/web/20121013085920/http://simonwillison.net/2009/May/19/djng/

The general idea is that at any point in the Django stack (down to the view
callable) you have stuff that sees an incoming request and will eventually
return a response. I don't think it's directly relevant to your work but it
can provide some guiding priciples.


I hope this helps!

-- 
Aymeric.



> On 4 mai 2015, at 23:04, Carl Meyer <c...@oddbird.net> wrote:
> 
> Hi Marten,
> 
> On 05/04/2015 01:45 PM, Marten Kenbeek wrote:
>> I'd like to discuss the API I'm proposing for the new url dispatcher I'm
>> working on. I'll try to explain the API and some of the rationale behind it.
> 
> Thanks for seeking comment early, and providing working PoC code, prose
> description, and API example!
> 
>> There is a working proof-of-concept
>> at https://github.com/knbk/django/tree/url_dispatcher. Currently, all
>> the names are chosen as not to conflict with the old implementation. A
>> short description can be found
>> at https://gist.github.com/knbk/96999abaab4ad4e5f8f9, which also
>> contains a link to an example url configuration. 
>> 
>> My main goal was to create a simple, extensible API. In the old API,
>> about 90% of the work was done in the `resolve()`,
>> `_reverse_with_prefix()` and `populate()` of the RegexURLResolver. This
>> made for a tightly coupled design that was almost impossible to extend.
>> 
>> My starting point was the RegexURLResolver and RegexURLPattern. Both
>> have a regex constraint that can match the current path and extract
>> arguments. The former can then pass the remainder of the path to its
>> list of resolvers and patterns; the latter can return a ResolverMatch
>> containing the callback specified by that pattern. I've kept this
>> distinction, with the `Resolver` and `View` classes. The change of name
>> from `Pattern`  to `View` is because the `View` object has a bit more
>> logic that determines how the view is called. It is a thin, callable
>> wrapper that can optionally decorate the actual view function with a set
>> of decorators passed down from parent resolvers, and when overwritten,
>> can do some more processing before or after the view function is called. 
> 
> This is a minor point, but I don't believe that the name `View` works
> for this class; it's too ambiguous when we already have a `View` class
> at the root of the class-based-views hierarchy. This class is not a
> view: it wraps one or calls one, but it is not itself the view.
> 
> I'm also curious to learn more about this "set of decorators" that can
> be applied to a view by the url resolver - that feature doesn't seem to
> be documented in the linked gist or in the API example.
> 
>> The hard-coded dependence on a regex pattern has been abstracted to a
>> Constraint object. Each Resolver and View has a (s

Re: URL dispatcher API

2015-05-04 Thread Carl Meyer
Hi Marten,

On 05/04/2015 01:45 PM, Marten Kenbeek wrote:
> I'd like to discuss the API I'm proposing for the new url dispatcher I'm
> working on. I'll try to explain the API and some of the rationale behind it.

Thanks for seeking comment early, and providing working PoC code, prose
description, and API example!

> There is a working proof-of-concept
> at https://github.com/knbk/django/tree/url_dispatcher. Currently, all
> the names are chosen as not to conflict with the old implementation. A
> short description can be found
> at https://gist.github.com/knbk/96999abaab4ad4e5f8f9, which also
> contains a link to an example url configuration. 
> 
> My main goal was to create a simple, extensible API. In the old API,
> about 90% of the work was done in the `resolve()`,
> `_reverse_with_prefix()` and `populate()` of the RegexURLResolver. This
> made for a tightly coupled design that was almost impossible to extend.
> 
> My starting point was the RegexURLResolver and RegexURLPattern. Both
> have a regex constraint that can match the current path and extract
> arguments. The former can then pass the remainder of the path to its
> list of resolvers and patterns; the latter can return a ResolverMatch
> containing the callback specified by that pattern. I've kept this
> distinction, with the `Resolver` and `View` classes. The change of name
> from `Pattern`  to `View` is because the `View` object has a bit more
> logic that determines how the view is called. It is a thin, callable
> wrapper that can optionally decorate the actual view function with a set
> of decorators passed down from parent resolvers, and when overwritten,
> can do some more processing before or after the view function is called. 

This is a minor point, but I don't believe that the name `View` works
for this class; it's too ambiguous when we already have a `View` class
at the root of the class-based-views hierarchy. This class is not a
view: it wraps one or calls one, but it is not itself the view.

I'm also curious to learn more about this "set of decorators" that can
be applied to a view by the url resolver - that feature doesn't seem to
be documented in the linked gist or in the API example.

> The hard-coded dependence on a regex pattern has been abstracted to a
> Constraint object. Each Resolver and View has a (set of) Constraint
> object(s), that can match the current url and extract arguments. A
> RegexPattern that simply mimics the old behaviour will be available, but
> other implementations, such as a Constraint that matches the request's
> host or method, are easily provided. A Constraint can also reverse a set
> of arguments to a partial url. That means that the full set of
> constraints used to match an url to a view, together with a suitable set
> of arguments, can be reversed to the url itself.

I like the general idea of the Constraint object! It reminds me of
Pyramid's view predicates system, which I think are a good model that it
may be worth spending some time reviewing for inspiration:
http://docs.pylonsproject.org/projects/pyramid/en/latest/narr/viewconfig.html

> The main strength of a Constraint is that it can contain very specific
> logic about its arguments. For example, a Constraint may know that it
> resolves to a Model's primary key. If it then receives a Model instance
> of that particular type, it will know how to reverse that model instance
> to a valid string-based partial url, so that it can later be resolved to
> match the same object. It could also e.g. infer the regex pattern from
> the field's type. 

I am not at all sure that it is a good idea to mix layers in this
suggested way, or what specific use case it improves. If the argument
that a constraint matches is an integer, why is it (in practice) useful
for the constraint to know that this integer is supposed to be the ID of
some particular model class? Why is it good for that constraint, when
reversing, to be able to take a model instance instead of an ID? To me
this sounds like the sort of implicit type coercion that makes APIs
harder to use and understand, not easier.

> There's one final piece to the puzzle: the URL object. This is the state
> of the url in the process of resolving or reversing an url. It's a
> two-way street: when resolving, it starts out as a full path, and the
> Constraints chip away at the path, while the set of constraints and
> extracted argument grows. When reversing, it starts out as a set of
> constraints and arguments, and reconstructs the partial urls from those
> constraints and arguments until a full url path is reconstructed. It
> shifts some of the logic from the Resolver to the URL, so that it is
> easier to extend the Resolver. It is also a simple container that allows
> any Constraint access to the full request. Last but not least, it allows
> t

URL dispatcher API

2015-05-04 Thread Marten Kenbeek
Hi all,

I'd like to discuss the API I'm proposing for the new url dispatcher I'm 
working on. I'll try to explain the API and some of the rationale behind it.

There is a working proof-of-concept 
at https://github.com/knbk/django/tree/url_dispatcher. Currently, all the 
names are chosen as not to conflict with the old implementation. A short 
description can be found 
at https://gist.github.com/knbk/96999abaab4ad4e5f8f9, which also contains a 
link to an example url configuration. 

My main goal was to create a simple, extensible API. In the old API, about 
90% of the work was done in the `resolve()`, `_reverse_with_prefix()` and 
`populate()` of the RegexURLResolver. This made for a tightly coupled 
design that was almost impossible to extend.

My starting point was the RegexURLResolver and RegexURLPattern. Both have a 
regex constraint that can match the current path and extract arguments. The 
former can then pass the remainder of the path to its list of resolvers and 
patterns; the latter can return a ResolverMatch containing the callback 
specified by that pattern. I've kept this distinction, with the `Resolver` 
and `View` classes. The change of name from `Pattern`  to `View` is because 
the `View` object has a bit more logic that determines how the view is 
called. It is a thin, callable wrapper that can optionally decorate the 
actual view function with a set of decorators passed down from parent 
resolvers, and when overwritten, can do some more processing before or 
after the view function is called. 

The hard-coded dependence on a regex pattern has been abstracted to a 
Constraint object. Each Resolver and View has a (set of) Constraint 
object(s), that can match the current url and extract arguments. A 
RegexPattern that simply mimics the old behaviour will be available, but 
other implementations, such as a Constraint that matches the request's host 
or method, are easily provided. A Constraint can also reverse a set of 
arguments to a partial url. That means that the full set of constraints 
used to match an url to a view, together with a suitable set of arguments, 
can be reversed to the url itself.

The main strength of a Constraint is that it can contain very specific 
logic about its arguments. For example, a Constraint may know that it 
resolves to a Model's primary key. If it then receives a Model instance of 
that particular type, it will know how to reverse that model instance to a 
valid string-based partial url, so that it can later be resolved to match 
the same object. It could also e.g. infer the regex pattern from the 
field's type. 

There's one final piece to the puzzle: the URL object. This is the state of 
the url in the process of resolving or reversing an url. It's a two-way 
street: when resolving, it starts out as a full path, and the Constraints 
chip away at the path, while the set of constraints and extracted argument 
grows. When reversing, it starts out as a set of constraints and arguments, 
and reconstructs the partial urls from those constraints and arguments 
until a full url path is reconstructed. It shifts some of the logic from 
the Resolver to the URL, so that it is easier to extend the Resolver. It is 
also a simple container that allows any Constraint access to the full 
request. Last but not least, it allows to dynamically build an url against 
the current request. This is useful if e.g. a constraint matches a 
different subdomain than the current request, so that a link automatically 
points to the right domain. 

I'm looking forwards to your feedback.

Thanks,
Marten

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/3cb36c11-16ee-4702-92a3-f9afb177bbca%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Extending the URL Dispatcher (GSoC 2015 proposal)

2015-03-26 Thread Marten Kenbeek
I've updated my proposal with a few more changes. Specifically, I've added 
a small part about reversing namespaced urls without specifying the 
namespace. In some circumstances this can provide looser coupling between 
third-party apps and arbitrary, reversible objects. It can also ease the 
transition from non-namespaced urls to namespaced urls by providing an 
intermediate step where an url name can match both. This step is absolutely 
necessary if third-party apps want to switch to namespaced urls but also 
want to provide backwards compatibility with previous versions. 

Additionally, I've included a timeline. Not my strongest point, but I think 
it's a decent guideline of what I'll be working on during the project. 

The up-to-date proposal can be found 
at 
http://www.google-melange.com/gsoc/proposal/public/google/gsoc2015/knbk/5629499534213120.
 
Any additional feedback before (or after) the deadline is greatly 
appreciated. If something in the proposal is not clear, let me know and 
I'll try to explain it better in my proposal. 

There are still a few design decisions to make between now and the start of 
GSoC, but I'll probably create a separate thread for those. 

Thanks,
Marten

Op maandag 2 maart 2015 17:57:24 UTC+1 schreef Marten Kenbeek:
>
> Hey all,
>
> I'm working on a proposal to extend the URL dispatcher. Here, I'd like to 
> provide a quick overview of the features I propose. 
>
> I'd like to:
> - Allow matching based on request attributes such as the subdomain or 
> protocol, and business logic such as the existence of a database object.
> - Make middleware configurable for a subset of views. It should be easy to 
> add, reorder or replace middleware at any level in the (currently 
> recursive) matching algorithm. 
> - Provide conventions for common patterns, such as an easy-to-configure 
> URL router for all generic Model views. For generic views, this should be a 
> one-liner. For custom views and other non-default options, this should 
> still be relatively easy to configure compared to writing out all patterns. 
>
> In the process, I'd like to formalize some classes used in the dispatcher. 
> Currently, the RegexURLPattern and RegexURLResolver classes provide most of 
> the functionality of the URL dispatcher. By abstracting these classes, and 
> factoring out the loading mechanism and some other internals, I hope to 
> provide an extensible dispatching framework for third-party apps.
>
> The full, yet incomplete proposal can be found at 
> https://gist.github.com/knbk/325d415baa92094f1e93 if you want more 
> details. It currently contains a slightly more in-depth discussion of the 
> current dispatcher and the proposed features, and a start on the redesign 
> of the dispatcher. 
>
> I'm mostly looking for some feedback on the general direction of these 
> features, though any feedback is welcome. I'm still working on it, so 
> details might change based on new insights.
>
> Thanks,
> Marten
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/b0df1d73-5ce4-4417-85f5-214faca6c3fc%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Extending the URL Dispatcher (GSoC 2015 proposal)

2015-03-12 Thread Marten Kenbeek
I came across an app named django-url-namespaces[1]. It provides support 
for declarative style url patterns, very similar to the declarative style 
of models and forms. I'm not particularly for or against this style or the 
current style, but I wanted to put this out here for discussion. It can 
simplify namespaces as well: the app name is nothing more than the class 
name (if not overridden), and the namespace is nothing more than the 
attribute name of the including class (again, if not overridden). This 
project is going towards a more class-based approach anyway, so it might 
fit the new design.

Any strong feelings or convincing arguments for one or the other? I'm 
slightly in favour of the declarative class style, mostly because it's in 
line with many other parts in Django, though there is practically no 
difference between classes and their instances, unlike forms and models. 
You might treat the class as a configuration and an instance as a resolver 
match, though, which would give clear, distinctive semantics to classes and 
instances. The instance would work both for resolving and reversing, as it 
knows all about the view, the url and the arguments. 

Anyway, I have an updated proposal available 
at https://gist.github.com/knbk/cd0d339e1d3fa127cf7a. 

I've intentionally left out a `ContinueResolving` exception. This is 
nothing that cannot easily be implemented with a custom resolver. What is 
harder to implement with a custom resolver is a `StopResolving` 
(`AbortResolving`? need to work on the name) kind of exception, basically a 
`Resolver404` that breaks through its current recursion level. I feel it is 
warranted that this is included in the proposal. Think e.g. of a 
`SubdomainResolver`. You usually don't want to include all the urls for the 
main domain into your subdomain, so this allows you to raise 
`StopResolving` if none of the subdomain urls match. 

[1] https://github.com/fish2000/django-url-namespaces

Op maandag 9 maart 2015 15:01:02 UTC+1 schreef Marten Kenbeek:
>
> After all the feedback I got here and at the sprint, I think the core of 
> my proposal will be the revamping of the current dispatcher and creating a 
> public API. I'll keep in mind the other features and in general the 
> extendibility of the new dispatcher, and if there is time left I'll start 
> implementing some of them.
>
> One interesting note is how content management systems try to work with 
> the url dispatcher. Most systems simply use a catch-all pattern. This often 
> includes custom machinery to resolve and reverse url patterns for pages, 
> blog posts, and other content types or plugins. Django's url dispatcher is 
> completely static in that it doesn't provide any public API to change the 
> url configuration after it has been loaded. This can be problematic with 
> the dynamic nature of CMS's, hence the custom machinery. Bas (bpeschier) 
> had to take it to a new level by routing certain content entries to a 
> custom view. If you want to avoid a "router" view (which is the url 
> dispatcher's job after all), you'd need to dig into the internals of the 
> url dispatcher to have any kind of dynamic updating of the configuration.
>
> I'd like to keep this dynamic nature in mind when designing the new API, 
> and in time implement a public API for this as well (e.g. a simple 
> `register` and `unregister`). This would avoid the need for either a router 
> view or unique url prefixes for each content type as well. It should 
> certainly allow for granular control, I believe reloading the complete url 
> dispatcher can take quite some time (I should probably test that). 
>
> I'm still in doubt on whether I should implement a refactor of url 
> namespaces and middleware. Url namespacing is overly complex and I'm not 
> too sure what the exact goal of the mechanism is. It obviously needs to 
> differentiate multiple instances of the same url configuration, and it is 
> also used to differentiate url configurations as well as to provide a 
> default instance for an url configuration. I'm not too sure what is 
> absolutely needed and what just makes it more complicated than necessary. 
> However, as namespaces are such an integral part of the dispatcher, it is 
> worth looking into and it might be necessary to keep in mind with the new 
> API. 
>
> As for middleware, I'm inclined to only support per-include decorators. 
> Users can always use `decorator_from_middleware` to define middleware for a 
> subset of views. While middleware certainly needs a revamp, I'm not too 
> familiar with its current issues, and I feel this is slightly out of this 
> project's scope. 
>
> On Friday, March 6, 2015 at 5:21:01 PM UTC+1, Tom Christie wrote:
>>
>> > E.g., flask uses a simple `` to match an integer and capture it 
>> in the `id` parameter. Support to 

Re: Extending the URL Dispatcher (GSoC 2015 proposal)

2015-03-11 Thread Alexandr Shurigin
Hi all.

You can take a look of http://phpdude.github.io/django-macros-url/. 


-- 
Alexandr Shurigin
Sent with Airmail

Включено 11 марта 2015 г. в 16:57:25, Marten Kenbeek (marten.k...@gmail.com) 
написал:

I came across an app named django-url-namespaces[1]. It provides support for 
declarative style url patterns, very similar to the declarative style of models 
and forms. I'm not particularly for or against this style or the current style, 
but I wanted to put this out here for discussion. It can simplify namespaces as 
well: the app name is nothing more than the class name (if not overridden), and 
the namespace is nothing more than the attribute name of the including class 
(again, if not overridden). This project is going towards a more class-based 
approach anyway, so it might fit the new design.

Any strong feelings or convincing arguments for one or the other? I'm slightly 
in favour of the declarative class style, mostly because it's in line with many 
other parts in Django, though there is practically no difference between 
classes and their instances, unlike forms and models. You might treat the class 
as a configuration and an instance as a resolver match, though, which would 
give clear, distinctive semantics to classes and instances. The instance would 
work both for resolving and reversing, as it knows all about the view, the url 
and the arguments. 

Anyway, I have an updated proposal available at 
https://gist.github.com/knbk/cd0d339e1d3fa127cf7a. 

I've intentionally left out a `ContinueResolving` exception. This is nothing 
that cannot easily be implemented with a custom resolver. What is harder to 
implement with a custom resolver is a `StopResolving` (`AbortResolving`? need 
to work on the name) kind of exception, basically a `Resolver404` that breaks 
through its current recursion level. I feel it is warranted that this is 
included in the proposal. Think e.g. of a `SubdomainResolver`. You usually 
don't want to include all the urls for the main domain into your subdomain, so 
this allows you to raise `StopResolving` if none of the subdomain urls match. 

[1] https://github.com/fish2000/django-url-namespaces

Op maandag 9 maart 2015 15:01:02 UTC+1 schreef Marten Kenbeek:
After all the feedback I got here and at the sprint, I think the core of my 
proposal will be the revamping of the current dispatcher and creating a public 
API. I'll keep in mind the other features and in general the extendibility of 
the new dispatcher, and if there is time left I'll start implementing some of 
them.

One interesting note is how content management systems try to work with the url 
dispatcher. Most systems simply use a catch-all pattern. This often includes 
custom machinery to resolve and reverse url patterns for pages, blog posts, and 
other content types or plugins. Django's url dispatcher is completely static in 
that it doesn't provide any public API to change the url configuration after it 
has been loaded. This can be problematic with the dynamic nature of CMS's, 
hence the custom machinery. Bas (bpeschier) had to take it to a new level by 
routing certain content entries to a custom view. If you want to avoid a 
"router" view (which is the url dispatcher's job after all), you'd need to dig 
into the internals of the url dispatcher to have any kind of dynamic updating 
of the configuration.

I'd like to keep this dynamic nature in mind when designing the new API, and in 
time implement a public API for this as well (e.g. a simple `register` and 
`unregister`). This would avoid the need for either a router view or unique url 
prefixes for each content type as well. It should certainly allow for granular 
control, I believe reloading the complete url dispatcher can take quite some 
time (I should probably test that). 

I'm still in doubt on whether I should implement a refactor of url namespaces 
and middleware. Url namespacing is overly complex and I'm not too sure what the 
exact goal of the mechanism is. It obviously needs to differentiate multiple 
instances of the same url configuration, and it is also used to differentiate 
url configurations as well as to provide a default instance for an url 
configuration. I'm not too sure what is absolutely needed and what just makes 
it more complicated than necessary. However, as namespaces are such an integral 
part of the dispatcher, it is worth looking into and it might be necessary to 
keep in mind with the new API. 

As for middleware, I'm inclined to only support per-include decorators. Users 
can always use `decorator_from_middleware` to define middleware for a subset of 
views. While middleware certainly needs a revamp, I'm not too familiar with its 
current issues, and I feel this is slightly out of this project's scope. 

On Friday, March 6, 2015 at 5:21:01 PM UTC+1, Tom Christie wrote:
> E.g., flask uses a simple `` to match an integer and capture it in 
>the `id` parameter. Support to check for conflicts would 

Re: Extending the URL Dispatcher (GSoC 2015 proposal)

2015-03-09 Thread Marten Kenbeek
After all the feedback I got here and at the sprint, I think the core of my 
proposal will be the revamping of the current dispatcher and creating a 
public API. I'll keep in mind the other features and in general the 
extendibility of the new dispatcher, and if there is time left I'll start 
implementing some of them.

One interesting note is how content management systems try to work with the 
url dispatcher. Most systems simply use a catch-all pattern. This often 
includes custom machinery to resolve and reverse url patterns for pages, 
blog posts, and other content types or plugins. Django's url dispatcher is 
completely static in that it doesn't provide any public API to change the 
url configuration after it has been loaded. This can be problematic with 
the dynamic nature of CMS's, hence the custom machinery. Bas (bpeschier) 
had to take it to a new level by routing certain content entries to a 
custom view. If you want to avoid a "router" view (which is the url 
dispatcher's job after all), you'd need to dig into the internals of the 
url dispatcher to have any kind of dynamic updating of the configuration.

I'd like to keep this dynamic nature in mind when designing the new API, 
and in time implement a public API for this as well (e.g. a simple 
`register` and `unregister`). This would avoid the need for either a router 
view or unique url prefixes for each content type as well. It should 
certainly allow for granular control, I believe reloading the complete url 
dispatcher can take quite some time (I should probably test that). 

I'm still in doubt on whether I should implement a refactor of url 
namespaces and middleware. Url namespacing is overly complex and I'm not 
too sure what the exact goal of the mechanism is. It obviously needs to 
differentiate multiple instances of the same url configuration, and it is 
also used to differentiate url configurations as well as to provide a 
default instance for an url configuration. I'm not too sure what is 
absolutely needed and what just makes it more complicated than necessary. 
However, as namespaces are such an integral part of the dispatcher, it is 
worth looking into and it might be necessary to keep in mind with the new 
API. 

As for middleware, I'm inclined to only support per-include decorators. 
Users can always use `decorator_from_middleware` to define middleware for a 
subset of views. While middleware certainly needs a revamp, I'm not too 
familiar with its current issues, and I feel this is slightly out of this 
project's scope. 

On Friday, March 6, 2015 at 5:21:01 PM UTC+1, Tom Christie wrote:
>
> > E.g., flask uses a simple `` to match an integer and capture it 
> in the `id` parameter. Support to check for conflicts would be a lot 
> simpler with those patterns. It would definitely be a best-effort feature.
>
> From my point of view, this by itself would make for a really 
> nicely-scoped GSoC project.
> Being able to demonstrate an API that allowed the user to switch to a URL 
> resolver that used that simpler style would be a really, really nice 
> feature,
> and also feels like it might actually be a manageable amount of work.
> This wouldn't *necessarily* need to allow decorator style routing, instead 
> of the current URLConf style, but that might also be a nice addition. 
> Personally though I would consider tackling that as an incremental 
> improvement.
>
> Things I'd be wary of:
>
> * Anything around "continue resolving" exceptions or object inspection 
> during routing, both of which sound like an anti-pattern to me.
> * Method based routing. Feel less strongly about this, but still not 
> convinced that it's a good style.
> * Generic views / model routes / automatic routing. Too broadly defined, 
> and with no clear best answer.
>
> Anyways, interesting stuff Marten, look forward to hearing more.
>
>   Tom
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/0bf15653-717c-4e24-8406-14fa71bf5bf9%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Extending the URL Dispatcher (GSoC 2015 proposal)

2015-03-06 Thread Tom Christie
> E.g., flask uses a simple `` to match an integer and capture it 
in the `id` parameter. Support to check for conflicts would be a lot 
simpler with those patterns. It would definitely be a best-effort feature.

>From my point of view, this by itself would make for a really nicely-scoped 
GSoC project.
Being able to demonstrate an API that allowed the user to switch to a URL 
resolver that used that simpler style would be a really, really nice 
feature,
and also feels like it might actually be a manageable amount of work.
This wouldn't *necessarily* need to allow decorator style routing, instead 
of the current URLConf style, but that might also be a nice addition. 
Personally though I would consider tackling that as an incremental 
improvement.

Things I'd be wary of:

* Anything around "continue resolving" exceptions or object inspection 
during routing, both of which sound like an anti-pattern to me.
* Method based routing. Feel less strongly about this, but still not 
convinced that it's a good style.
* Generic views / model routes / automatic routing. Too broadly defined, 
and with no clear best answer.

Anyways, interesting stuff Marten, look forward to hearing more.

  Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/bc559676-60d9-48e1-bbe0-7abb52397184%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: Extending the URL Dispatcher (GSoC 2015 proposal)

2015-03-05 Thread Marten Kenbeek
To respond to the other issues raised:

On 2 March 2015 at 17:14, Tim Graham <timog...@gmail.com> wrote:

> I think it would be helpful to motivate this with some pseudocode of 
> specific use cases you are aiming to solve. Have you looked into whether 
> there are any related third-party projects related to your ideas from which 
> you could draw inspiration?
>

There isn't too much out there for Django, the few apps that are out there 
usually work around the url dispatcher, e.g. with additional middleware. 
I've mostly looked at other frameworks like flask and ruby-on-rails for 
features. My previous post has some (pseudo)code that covers most features 
I'm planning on implementing. 

On Monday, March 2, 2015 at 8:20:20 PM UTC+1, Marc Tamlyn wrote:
>
> A collection of thoughts:
>
> I think allowing the url dispatcher to inspect the database for the 
> existence of certain objects is potentially somewhat dangerous. However, 
> good support for a view raising a "continue resolving" exception along the 
> lines of https://github.com/jacobian-archive/django-multiurl might be 
> interesting.
>

How so? I guess the potential to do other things than just fetching the 
object can be dangerous. If overused it can certainly cause performance 
issues, so matching against an object should generally be the last step in 
the url resolver. 

I prefer to think of it as a view preprocessor: if the url matches, you 
convert the string arguments to the values that are actually used, with an 
option to continue resolving. Maybe the implementation should clearly 
separate these two steps. A "continue resolving" exception in the view, if 
carelessly used, opens up the possibility that a view is partly processed 
and then control is passed to another view. Not to mention the view 
middleware, which received the first view. It just feels to me like a 
"continue resolving" exception should be part of the resolving process, not 
of the view. 

I also don't want to separate it into too many steps, that's why I wanted 
to include this step in the url resolver. However, it might warrant its own 
step in the whole handling process. 

Related to this, a check for potentially conflicting url mappings could be 
> interesting.


I could definitely use some regex patterns to check for potentially 
conflicting regex patterns (maybe I should use jQuery as well) :P Kidding 
aside, regexes are hard to compare by nature. We could factor out static 
prefixes, and maybe check for common patterns. It would be easier with 
alternate matching schemes. E.g., flask uses a simple `` to match 
an integer and capture it in the `id` parameter. Support to check for 
conflicts would be a lot simpler with those patterns. It would definitely 
be a best-effort feature. 

Middleware currently has complex and unintuitive behaviour in the event of 
> exceptions. You talk about the middleware/decorator split, but not how to 
> make either make sense.
>

I guess I'm not too familiar with real-world usage of middleware. Could you 
elaborate on this complex and unintuitive behaviour? Initially I simply 
thought of middleware and decorators as site-wide and per-view wrappers, 
but further investigation reminded me that there are more distinctions. The 
current middleware doesn't fully "fit in" with what I had in mind, 
especially the fact that request middleware is executed before the url is 
resolved, so this can't even be changed in the resolver. The middleware 
requires some more thought and discussion. 

Supporting generic sets of views has some logic, although in my experience 
> it is extremely rare that you can sensibly use a generic view with no 
> alterations at all - it almost always needs extra context or some other 
> tweaks. I'm not really convinced that a one liner to get CRUD for a 
> particular model will actually be that useful in the wild - you're likely 
> to end up changing too many things. I don't find the "one line in a urlconf 
> for each view" convention to be particularly problematic, however writing 
> all the regexes is potentially more prone to problems.
>

Yeah, the generic views should be more of a fallback. The fact that it is a 
one-liner for generic views is more of an added nicety, I should've put 
that differently. As I elaborated in my previous post. the real power is 
that you can reverse urls based on models or model instances, rather than 
having to specify the parameters in the exact way the url expects. One huge 
benefit (can't believe I only thought of this just now) is that, if you 
decide to change the url structure, e.g. to use the slug instead of the pk, 
you don't have to run though your entire codebase to change every instance 
where you reverse that url. That might be an even larger benefit than 
url-agnostic reversing for third-party apps. No more regex duplication? Not 
bad either

Re: Extending the URL Dispatcher (GSoC 2015 proposal)

2015-03-03 Thread Marten Kenbeek
First of all, thanks for the feedback. There are some good points here that 
I hadn't thought about.

Behaviour similar to a `ContinueResolving` exception is one of the things I 
was aiming at. However, I can't see how to raise this in a view while 
maintaining backwards compatibility with view middleware. E.g. the 
CsrfViewMiddleware might have bailed out before the first view was 
executed, while the second view might be csrf exempt. Executing the view 
middleware twice might cause problems as well.

As for alternate url resolvers: they'll provide the same functions 
(`resolve` and `reverse`) that are required for any new-style resolvers, so 
they can be freely mixed and replaced, for example:

urlpatterns = patterns('',
SubdomainResolver('accounts.', include('some.url.config.urls'), 
decorators=[login_required]),
url(r'^$', 'my.view.function'),
ModelRouter(r'^mymodel/', model=MyModel, field='some_field',
views = {
'detail': MyDetailView.as_view(),
'list':   MyListView.as_view(),
'edit':   MyUpdateView.as_view(),
'new':MyCreateView.as_view(),
'delete': None,
}
),
)

As for the model router: a real-world example can be as simple as this. You 
can either specify a field and based on the field, `detail` and `edit` will 
accept only numbers or strings or whatever, or you can specify your own 
regex that is repeated for all single-object views. The magic is that the 
model router will automatically reverse the url based on a model or model 
instance if that model uses a model router, similar to how 
`get_absolute_url` provides the same functionality for a single view per 
model.

However, looking at ruby-on-rails and django-rest-framework, routers are 
tightly coupled with controllers/viewsets, which provide all the views for 
a specific model. Django's own `ModelAdmin` combines the two, though it is 
more of a router-and-controller-and-much-more in one. Controllers and 
viewsets are both tightly coupled with the routers, so it might be 
warranted that a possible viewset simply implements the `resolve` and 
`reverse` methods for its contained views.

The `decorators` parameter above also shows what I had in mind in that 
regard. After looking for concrete examples, I can't seem to think of a 
proper use case for the same behaviour for middleware, at least not with 
Django's built-in middleware. For decorators, though, it's a great 
addition, if there is a proper way to add, reorder or remove inherited 
decorators (reorder mostly because of csrf_exempt).

That's it for today.

Marten

On Tuesday, March 3, 2015 at 2:56:39 AM UTC+1, Curtis Maloney wrote:
>
>
>
> On 3 March 2015 at 03:57, Marten Kenbeek <marte...@gmail.com 
> > wrote:
>
>> Hey all,
>>
>> I'm working on a proposal to extend the URL dispatcher. Here, I'd like to 
>> provide a quick overview of the features I propose. 
>>
>> I'd like to:
>> - Allow matching based on request attributes such as the subdomain or 
>> protocol, and business logic such as the existence of a database object.
>>
>
> There was a "continue resolving" sort of exception proposed/implemented 
> that would obviate this, allowing the logic to remain in views [or view 
> decorators]... a much simpler solution, IMHO.
>  
>
>> - Make middleware configurable for a subset of views. It should be easy 
>> to add, reorder or replace middleware at any level in the (currently 
>> recursive) matching algorithm. 
>>
>
> This has certainly been on the "wanted" list for many years now, however I 
> expect it would require the middleware re-design that so far has proven too 
> invasive to land.
>
> That said, providing the "new" middleware-as-wrapper interface around url 
> patterns lists could be a good stepping stone to eventually removing the 
> existing middleware API.
>  
>
>> - Provide conventions for common patterns, such as an easy-to-configure 
>> URL router for all generic Model views. For generic views, this should be a 
>> one-liner. For custom views and other non-default options, this should 
>> still be relatively easy to configure compared to writing out all patterns. 
>>
>
> Are you talking about pre-cooked url patterns for the various CBV?  Or 
> plugin routers for groups of CBV?  I'm certainly in favour of some tool 
> that makes it easier to express "common" regex matches [satisfying the 
> "protect from the tedium" rule of frameworks]
>  
>
> In the process, I'd like to formalize some classes used in the dispatcher. 
>> Currently, the RegexURLPattern and RegexURLResolver classes provide most of 
>> the functionality of the URL dispatcher. By abstracting these classes, and 
>> factoring out the loading mecha

Re: Extending the URL Dispatcher (GSoC 2015 proposal)

2015-03-02 Thread Curtis Maloney
On 3 March 2015 at 03:57, Marten Kenbeek <marten.k...@gmail.com> wrote:

> Hey all,
>
> I'm working on a proposal to extend the URL dispatcher. Here, I'd like to
> provide a quick overview of the features I propose.
>
> I'd like to:
> - Allow matching based on request attributes such as the subdomain or
> protocol, and business logic such as the existence of a database object.
>

There was a "continue resolving" sort of exception proposed/implemented
that would obviate this, allowing the logic to remain in views [or view
decorators]... a much simpler solution, IMHO.


> - Make middleware configurable for a subset of views. It should be easy to
> add, reorder or replace middleware at any level in the (currently
> recursive) matching algorithm.
>

This has certainly been on the "wanted" list for many years now, however I
expect it would require the middleware re-design that so far has proven too
invasive to land.

That said, providing the "new" middleware-as-wrapper interface around url
patterns lists could be a good stepping stone to eventually removing the
existing middleware API.


> - Provide conventions for common patterns, such as an easy-to-configure
> URL router for all generic Model views. For generic views, this should be a
> one-liner. For custom views and other non-default options, this should
> still be relatively easy to configure compared to writing out all patterns.
>

Are you talking about pre-cooked url patterns for the various CBV?  Or
plugin routers for groups of CBV?  I'm certainly in favour of some tool
that makes it easier to express "common" regex matches [satisfying the
"protect from the tedium" rule of frameworks]


In the process, I'd like to formalize some classes used in the dispatcher.
> Currently, the RegexURLPattern and RegexURLResolver classes provide most of
> the functionality of the URL dispatcher. By abstracting these classes, and
> factoring out the loading mechanism and some other internals, I hope to
> provide an extensible dispatching framework for third-party apps.
>

As mentioned elsewhere, I would very much like to see a resolver system
based on the "parse" library [essentially, the inverse of str.format -
https://pypi.python.org/pypi/parse], and to do so would indeed require some
formal analysis / documentation of the existing resolver architecture.

--
Curtis

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/CAG_XiSBaPz%2BJJs-CGLbZzdbpvSRo3vjpeWnpcxhVVq_tfs4gaQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.


Re: Extending the URL Dispatcher (GSoC 2015 proposal)

2015-03-02 Thread Marc Tamlyn
A collection of thoughts:

I think allowing the url dispatcher to inspect the database for the
existence of certain objects is potentially somewhat dangerous. However,
good support for a view raising a "continue resolving" exception along the
lines of https://github.com/jacobian-archive/django-multiurl might be
interesting. Related to this, a check for potentially conflicting url
mappings could be interesting.

Middleware currently has complex and unintuitive behaviour in the event of
exceptions. You talk about the middleware/decorator split, but not how to
make either make sense.

Supporting generic sets of views has some logic, although in my experience
it is extremely rare that you can sensibly use a generic view with no
alterations at all - it almost always needs extra context or some other
tweaks. I'm not really convinced that a one liner to get CRUD for a
particular model will actually be that useful in the wild - you're likely
to end up changing too many things. I don't find the "one line in a urlconf
for each view" convention to be particularly problematic, however writing
all the regexes is potentially more prone to problems.

If you are intending on introducing alternative URL resolvers, some example
ideas would be needed. The lack of a consistent way to reverse a slug for
example is a good idea to address, but we need to establish how.

How are you intending to support different resolvers in the same project?
It seems to me that it is rather inefficient in large projects to loop
through all resolvers for all urls.

Namespacing urls is currently over complex for the 90% use case, and the
docs are hard to understand as a result. Alternative designs in this area
could be interesting.

Overall there are lots of interesting starts of ideas here, but I feel one
or two dead ends. It's a potentially very varied project and the crux of
the proposal needs to focus on ensuring that some specific tasks are well
designed and achievable, with others being extensions later on.

Marc

On 2 March 2015 at 17:14, Tim Graham <timogra...@gmail.com> wrote:

> Hi Marten,
>
> I think it would be helpful to motivate this with some pseudocode of
> specific use cases you are aiming to solve. Have you looked into whether
> there are any related third-party projects related to your ideas from which
> you could draw inspiration?
>
> Tim
>
>
> On Monday, March 2, 2015 at 11:57:24 AM UTC-5, Marten Kenbeek wrote:
>>
>> Hey all,
>>
>> I'm working on a proposal to extend the URL dispatcher. Here, I'd like to
>> provide a quick overview of the features I propose.
>>
>> I'd like to:
>> - Allow matching based on request attributes such as the subdomain or
>> protocol, and business logic such as the existence of a database object.
>> - Make middleware configurable for a subset of views. It should be easy
>> to add, reorder or replace middleware at any level in the (currently
>> recursive) matching algorithm.
>> - Provide conventions for common patterns, such as an easy-to-configure
>> URL router for all generic Model views. For generic views, this should be a
>> one-liner. For custom views and other non-default options, this should
>> still be relatively easy to configure compared to writing out all patterns.
>>
>> In the process, I'd like to formalize some classes used in the
>> dispatcher. Currently, the RegexURLPattern and RegexURLResolver classes
>> provide most of the functionality of the URL dispatcher. By abstracting
>> these classes, and factoring out the loading mechanism and some other
>> internals, I hope to provide an extensible dispatching framework for
>> third-party apps.
>>
>> The full, yet incomplete proposal can be found at
>> https://gist.github.com/knbk/325d415baa92094f1e93 if you want more
>> details. It currently contains a slightly more in-depth discussion of the
>> current dispatcher and the proposed features, and a start on the redesign
>> of the dispatcher.
>>
>> I'm mostly looking for some feedback on the general direction of these
>> features, though any feedback is welcome. I'm still working on it, so
>> details might change based on new insights.
>>
>> Thanks,
>> Marten
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers (Contributions to Django itself)" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/django-deve

Re: Extending the URL Dispatcher (GSoC 2015 proposal)

2015-03-02 Thread Tim Graham
Hi Marten,

I think it would be helpful to motivate this with some pseudocode of 
specific use cases you are aiming to solve. Have you looked into whether 
there are any related third-party projects related to your ideas from which 
you could draw inspiration?

Tim

On Monday, March 2, 2015 at 11:57:24 AM UTC-5, Marten Kenbeek wrote:
>
> Hey all,
>
> I'm working on a proposal to extend the URL dispatcher. Here, I'd like to 
> provide a quick overview of the features I propose. 
>
> I'd like to:
> - Allow matching based on request attributes such as the subdomain or 
> protocol, and business logic such as the existence of a database object.
> - Make middleware configurable for a subset of views. It should be easy to 
> add, reorder or replace middleware at any level in the (currently 
> recursive) matching algorithm. 
> - Provide conventions for common patterns, such as an easy-to-configure 
> URL router for all generic Model views. For generic views, this should be a 
> one-liner. For custom views and other non-default options, this should 
> still be relatively easy to configure compared to writing out all patterns. 
>
> In the process, I'd like to formalize some classes used in the dispatcher. 
> Currently, the RegexURLPattern and RegexURLResolver classes provide most of 
> the functionality of the URL dispatcher. By abstracting these classes, and 
> factoring out the loading mechanism and some other internals, I hope to 
> provide an extensible dispatching framework for third-party apps.
>
> The full, yet incomplete proposal can be found at 
> https://gist.github.com/knbk/325d415baa92094f1e93 if you want more 
> details. It currently contains a slightly more in-depth discussion of the 
> current dispatcher and the proposed features, and a start on the redesign 
> of the dispatcher. 
>
> I'm mostly looking for some feedback on the general direction of these 
> features, though any feedback is welcome. I'm still working on it, so 
> details might change based on new insights.
>
> Thanks,
> Marten
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/34ebf9ed-9961-4b33-9f49-5e6a4f9c6469%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Extending the URL Dispatcher (GSoC 2015 proposal)

2015-03-02 Thread Marten Kenbeek
Hey all,

I'm working on a proposal to extend the URL dispatcher. Here, I'd like to 
provide a quick overview of the features I propose. 

I'd like to:
- Allow matching based on request attributes such as the subdomain or 
protocol, and business logic such as the existence of a database object.
- Make middleware configurable for a subset of views. It should be easy to 
add, reorder or replace middleware at any level in the (currently 
recursive) matching algorithm. 
- Provide conventions for common patterns, such as an easy-to-configure URL 
router for all generic Model views. For generic views, this should be a 
one-liner. For custom views and other non-default options, this should 
still be relatively easy to configure compared to writing out all patterns. 

In the process, I'd like to formalize some classes used in the dispatcher. 
Currently, the RegexURLPattern and RegexURLResolver classes provide most of 
the functionality of the URL dispatcher. By abstracting these classes, and 
factoring out the loading mechanism and some other internals, I hope to 
provide an extensible dispatching framework for third-party apps.

The full, yet incomplete proposal can be found at 
https://gist.github.com/knbk/325d415baa92094f1e93 if you want more details. 
It currently contains a slightly more in-depth discussion of the current 
dispatcher and the proposed features, and a start on the redesign of the 
dispatcher. 

I'm mostly looking for some feedback on the general direction of these 
features, though any feedback is welcome. I'm still working on it, so 
details might change based on new insights.

Thanks,
Marten

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers  (Contributions to Django itself)" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/django-developers/1555e45b-8a98-4d36-bc7b-00c7136e782d%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.


Re: URL dispatcher fallthrough?

2013-04-03 Thread Jacob Kaplan-Moss
I think it's esoteric enough that mentioning it in the docs doesn't
seem like it's worth the distraction. But the wiki's, um, a wiki... so
feel free to edit!

Jacob

On Wed, Apr 3, 2013 at 9:07 AM, Felipe Prenholato  wrote:
> Jacob, I know that Django don't refer to third party packages in docs, but
> is possible to have something in doc or wiki (and doc linking to wiki) about
> this url helper (and possible others)? I ask because I see functionality
> proposed by this thread and by your app good enough to be cited in some
> place close to official docs.
>
> Felipe 'chronos' Prenholato.
> Linux User nº 405489
> Home page: http://devwithpassion.com | http://chronosbox.org/blog
> GitHub: http://github.com/chronossc/ | Twitter: http://twitter.com/chronossc
>
>
> 2013/4/2 Jacob Kaplan-Moss 
>>
>> On Tue, Apr 2, 2013 at 4:49 AM, David Danier 
>> wrote:
>> > This is somethign that does not need to be inside Django core. So why
>> > not just start an thirt party app implementing the proposal?
>>
>> I did just that: https://pypi.python.org/pypi/django-multiurl.
>>
>> Turns out it takes a fair bit of spelunking inside the guts of the
>> urlresolver, but ends up being a fairly short bit of code. Give it a
>> shot, let me know if it works for your usecase(s).
>>
>> Jacob
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to django-developers+unsubscr...@googlegroups.com.
>> To post to this group, send email to django-developers@googlegroups.com.
>> Visit this group at
>> http://groups.google.com/group/django-developers?hl=en.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-04-03 Thread Felipe Prenholato
Jacob, I know that Django don't refer to third party packages in docs, but
is possible to have something in doc or wiki (and doc linking to wiki)
about this url helper (and possible others)? I ask because I see
functionality proposed by this thread and by your app good enough to be
cited in some place close to official docs.

Felipe 'chronos' Prenholato.
Linux User nº 405489
Home page: http://devwithpassion.com | http://chronosbox.org/blog
GitHub: http://github.com/chronossc/ | Twitter: http://twitter.com/chronossc


2013/4/2 Jacob Kaplan-Moss 

> On Tue, Apr 2, 2013 at 4:49 AM, David Danier 
> wrote:
> > This is somethign that does not need to be inside Django core. So why
> > not just start an thirt party app implementing the proposal?
>
> I did just that: https://pypi.python.org/pypi/django-multiurl.
>
> Turns out it takes a fair bit of spelunking inside the guts of the
> urlresolver, but ends up being a fairly short bit of code. Give it a
> shot, let me know if it works for your usecase(s).
>
> Jacob
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-04-02 Thread Jacob Kaplan-Moss
On Tue, Apr 2, 2013 at 4:49 AM, David Danier  wrote:
> This is somethign that does not need to be inside Django core. So why
> not just start an thirt party app implementing the proposal?

I did just that: https://pypi.python.org/pypi/django-multiurl.

Turns out it takes a fair bit of spelunking inside the guts of the
urlresolver, but ends up being a fairly short bit of code. Give it a
shot, let me know if it works for your usecase(s).

Jacob

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-04-02 Thread David Danier
> David: The slugs wouldn't be overlapping if they inherited from some
> sort of "Organization" model with unique slug. The user could also add
> validation code to prevent company and schools having same slugs.

If you have a common base model this sounds like some polymorphic model
problem, which does not need to be solved in the URLs (and probably
should not). There are existing third party solutions, which may help
you get the right model back, when fetching one "Organization".

An user validation is only useful if you put the logic into an unrelated
app, as my proposed alias app. Otherwise you will need to mix your
models, at least when doing validation. This may not follow the loose
coupling philosophy
(https://docs.djangoproject.com/en/dev/misc/design-philosophies/) of
Django. If you have this app the name resolution can be done without
doing fallbacks anywhere in your code.

Anyways, about your proposal:
This is somethign that does not need to be inside Django core. So why
not just start an thirt party app implementing the proposal?

David

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-28 Thread Michael Manfre
On Thu, Mar 28, 2013 at 10:28 PM, meric <meric...@gmail.com> wrote:
>
> Michael: How does the middleware work with other URLs that do not need to
> have fallthrough? Do you have to add them to excluded_path_patterns?
> Would like a more general description of how it works.
>

A ContentMiddleware behaves basically the same as a standard 404
middleware. If no other middleware or view handles the request, then a
ContentMiddleware will get a chance to match during the process_response
phase. Most of these middleware are a shim to a view function, but some
include logic to make faster (than the view) determinations about the user,
type of asset, the request, and whether or not the underlying views could
possibly handle request. Most of our content is subject to record level
authorization checks that result in heavy queries and reducing response
time was one of the main goals of this implementation.

There are three possible types of matches that can happen for a middleware
and a request; an explicit match, an explicit miss, and an implicit match.
An explicit miss only happens when the middleware's process() raises
Http404. An explicit match is when a response is returned, either of the
ContentHandled exceptions are raised, or a permission error is raised. The
ContentHandled exceptions allow the implementing middleware to pass back
knowledge of owned URL spaces or assets that could exist at some point in
the future. For our site, when a permission check fails, it means the
assets exists. The implicit matching happens on all other exceptions.

The type of match dictates how long to cache the knowledge about whether or
not the request was handled by the middleware. Each middleware dictates
what it means for it to match and this is often more than just the URL
pattern.

During the middleware process_request phase, any previously matched request
is immediately processed by the middleware. This short circuits the URL
dispatcher and reduces the overall response time for handling the request.
On our system, users are able to arbitrarily place assets in the URL
structure and these often overlap with the URL patterns of our views. Some
of the overlapped views have heavy look ups that could take a few hundred
ms before finding out it is and probably always will be a 404.

The excluded_path_patterns provides an easy way of explicitly missing based
upon the URL without needing to check the database. This also yields a
faster response time by avoiding any lookups when the middleware attempts
to determine "is this mine" for a new request. We also use
excluded_path_patterns to help enforce business rules about the URL space
where assets may reside. The example ManualPageMiddleware explicitly
prevents any static pages from being found under our image folder. The
underlying ManualPage model is generic and has no such restrictions for its
URL.

Also, couldn't the caching logic, (ignoring fallthrough urls for a minute),
> be implemented in a view decorator (which is also what how we do it)?
>

Basic caching logic could be worked in to a view decorator, but it will not
provide responses as quickly as my middleware. The decorated views would be
processed in order and you could end up with a request chasing just behind
expiring cache entries and having to recheck them all. My middleware is
able to check the "is this handled" cache for all of the middleware before
needing to go through the URL dispatcher and then trying each of the
middleware. You could probably get similar behavior with a decorator
wrapping several entries in urls, but it would be a bit ugly and less
flexible with cache timeouts.

Regards,
Michael Manfre

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-28 Thread meric
In this mailing list alone, Michael, Tom, David, Felipe, julianb, Ian and I 
have all proposed different solutions. 7 different solutions to what 
appears to be a very common problem. :(

Tom: 

As you said the get_instance_or_404 only works for simple cases.

The switcher view only works if the delegates have no kwargs, or if they 
do, all their kwargs are named the same way. The problem remains: URL 
structure forcing a particular design onto user's view definitions.

Good point about the middleware:

The view middleware can't be called just once with the 'correct' view, 
> because there's not way of determining up front if a view will raise a 
> `UrlNotMatched` without calling into it.
>
>
Currently in my pull request they do get called again and I see it is a 
problem.

David: The slugs wouldn't be overlapping if they inherited from some sort 
of "Organization" model with unique slug. The user could also add 
validation code to prevent company and schools having same slugs.

Michael: How does the middleware work with other URLs that do not need to 
have fallthrough? Do you have to add them to excluded_path_patterns? Would 
like a more general description of how it works. Also, couldn't the caching 
logic, (ignoring fallthrough urls for a minute), be implemented in a view 
decorator (which is also what how we do it)?

I have thought about some of everyone's proposed solutions, especially 
Ian's and Tom's. What about a new function for use when declaring 
urlpatterns, called "urlrouter_view" which you wrap around URLs to 
explicitly mark them as having fallthrough enabled. When calling it, it 
will look at its arguments to make sure the patterns are all identical. It 
creates an anonymous view that would route to each view one by one until 
one resolves without raising the DoesNotResolve exception. If no view could 
handle the URL, it will raise an Http404. (Since the number of URLs are 
more limited, maybe we can do away with DoesNotResolve and use a Http404 as 
a signal to continue resolving.)  Any view middleware would only have been 
run once for the URL, given the anonymous view as the argument and any 
keyword arguments are converted to list argument format for the middleware. 

Here is an example (forgive my poorly named views):

urlpatterns += patterns('',
# ... other normally managed urls.
urlrouter_view(
url(r'^(?P[_a-z0-9-]+)/(?P[_a-z0-9-]+)/$', 
'list_activity'),
url(r'^(?P[_a-z0-9-]+)/(?P[_a-z0-9-]+)/$', 
'list_company'),
url(r'^(?P[_a-z0-9-]+)/(?P[_a-z0-9-]+)/$', 
'list_activity'),
),
# ... other normally managed urls.
)

So basically my proposal as it stands but its abilities have to explicitly 
marked in urls.py. I have not thought about how this can be implemented 
yet, especially in regards to url includes and exposing the url names and 
kwargs to reverse, but it addresses the problem with view_middleware.

Alternative syntax (names can be improved), removes repetition on regex:

urlpatterns += patterns('',
# ... other normally managed urls.
urlrouter_view('^([_a-z0-9-]+)/([_a-z0-9-]+)/$'
view('list_activity', kwargs=['country', 'company'], 
name="list_activity),
view('list_company', kwargs=['country', 'industry'], name="
list_company),
view('list_activity', kwargs=['company', 'industry'], 
name="list_activity),
),
# ... other normally managed urls.
)

Any thoughts? 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-28 Thread Michael Manfre
On Thu, Mar 28, 2013 at 8:21 AM, Loic Bistuer wrote:

> What I would like to see:
>
> urlpatterns = patterns('',
> # Standard patterns
>  url(r'^admin/', include(admin.site.urls)),
>
> # Fallthrough patterns
> url('', include('myapp.urls'),
> url('', include('django.contrib.flatpages.urls'),
> url('', include('django.contrib.redirects.urls'),
> )
>
> Each app minds its own business and no more middleware that juggle with
> Http404.
>

Fall through URL patterns make it less clear about what view is handling a
URL. It's already known and established that you need to check middleware
for arbitrary URL handling, adding this to the URL patterns as well doesn't
feel like a step in the right direction.

Middlewares that juggle Http404 are not all bad and have the potential of
being a bit saner and faster than a URL fall through. If the dispatcher
falls through, then every URL that matches the pattern would need to hit
the database and/or include its own caching to determine if the URL is a
hit for a specific app/view. That leads to the same bit of code duplicated
in the top of almost every one of these fall through views.

With middleware, you can put all of this shared logic in a base class and
provide the ability to completely short circuit the dispatcher for known
matches. For src.org, we have a few different models with staff defined
URLs. The traditional Http404 middleware pattern was a bit repetitive and I
solved the problem with this middleware (
https://gist.github.com/manfre/5262782). The heart of the middlware is
ContentMiddleware._process(), which contains the "was this handled" logic.
The Gist includes one of the simple arbitrary URL model handlers, but we
have quite a few more.


Regards,
Michael Manfre

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-28 Thread David Danier
I'm with Tom here, this just feels wrong. The whole point of urls.py is
to have a clean mapping of URLs to views.

Of course I understand your problem, so let's look at the details.

> //  # front page for country
> // / # list of schools and companies with activities
> in that industry, in that country
> /// # list of industries company has activities in, in
> that country

This breaks if an industry with the same pk/slug as a company exists
(/de/foo/ as industry and /de/foo/ as company). You would never be able
to call the company again, as the industry just overlaps. This schema
just is badly designed and not able to handle this case in a sane way.
This is true for router-views, throwing UrlNotMatched or any other way.
You basicaly loose the clean mapping of your URLs. It could even happen,
that you overlap a valid company url by adding a new industry to your
database, moving that URL to a completely new content.

(Sidenote: I think this is a problem even the Django admin has. Just add
an Model with an CharField as primary key and insert "add" into this
field. You will neven be able too edit this in the admin, as the
add_view will be called instead.)

You could of course make sure the URLs never overlap by making the slug
unique. One easy way would be to add a prefix, but then again you could
implement this in your urls.py without any hassle
("//industry-/").

Another solution might be to add a router-view, which is based on
aliases. So you could have a central db-table which stores alias to view
releations (meaning: /de/foo/ -> (industry_view, slug=foo), /de/foo-2/
-> (company_view, slug=foo), stored inside the database). This way you
could avoid overlaying and even create fallback-names for duplicate
slugs. But you need a central storage to accomplish that. (Drupal works
this way.)

Anyway, I don't think throwing UrlNotMatched is the right solution here.

David

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-28 Thread Loic Bistuer
> urlpatterns = patterns('',
> (r'/([^/]+)/$', Switcher.as_view(delegates=[...])),
> (r'/([^/]+)/([^/]+)/$', Switcher.as_view(delegates=[...])),
> (r'/([^/]+)/([^/]+)/([^/]+)/$', Switcher.as_view(delegates=[...]))
> )

I don't see this as an improvement because you still bind together multiple 
apps into a single urlpattern. Also wrapper views might be ugly but at least 
there are explicit and easy to understand.

What I would like to see:

urlpatterns = patterns('',
# Standard patterns
 url(r'^admin/', include(admin.site.urls)),

# Fallthrough patterns
url('', include('myapp.urls'),
url('', include('django.contrib.flatpages.urls'),
url('', include('django.contrib.redirects.urls'),
)

Each app minds its own business and no more middleware that juggle with Http404.

> The view middleware question seems like another really good argument against 
> the proposed change...

As far as I'm concerned, Django middleware should have been deprecated 
altogether, that's certainly too late to do it now but let's not use them as an 
excuse to prevent improving the flexibility of the URL dispatcher.

-- 
Loic

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-28 Thread Tom Christie

>
> What is the reasonable alternative? The only way I can think how a 
> wrapping view will be able to handle this case, is to write your own 
> router...
>
 

...In each case you'll have lots of ifs, try..excepts, for each model, and 
> then to appropriate view. For the second and third router_views you'd have 
> to have the same thing repeated, but this time it is nested.


One pattern you could use to avoid ugly nesting would be to write a helper 
function that attempts a queryset lookup against multiple models, for 
example...

obj = get_instance_or_404(Foo, Bar, Baz, pk=pk)
if isinstance(obj, Foo):
...
elif isinstance(obj, Bar):
...
elif isinstance(obj, Baz):
...

Something like that would probably be fine for the simpler cases.

For more complicated cases you can implement pretty much the same API 
you're looking for but do it in view code.  Something like this...

class Switcher(View):
"""
Delegate the handling of the request to one of a list of delegates.

Any delegate may raise an IgnoreRequest exception to pass handling
over to the next delegate in the list.
"""
delegates = []

def dispatch(self, request, *args, **kwargs):
for delegate in self.delegates:
try
return delegate(request, *args, **kwargs)
except IgnoreRequest:
pass
raise Http404

urlpatterns = patterns('',
(r'/([^/]+)/$', Switcher.as_view(delegates=[...])),
(r'/([^/]+)/([^/]+)/$', Switcher.as_view(delegates=[...])),
(r'/([^/]+)/([^/]+)/([^/]+)/$', Switcher.as_view(delegates=[...]))
)

There's also the question of view_middleware not being applied for the 
> appropriate view unless the user specifically looks into how middleware 
> gets called and manages handling view middleware themselves in the 
> different router_views.


The view middleware question seems like another really good argument 
against the proposed change...

The view middleware can't be called just once with the 'correct' view, 
because there's not way of determining up front if a view will raise a 
`UrlNotMatched` without calling into it.

So, what happens when `UrlNotMatched` is raised - do all the middleware 
`process_view` methods get called again?

Cheers,

  Tom :)


On Wednesday, 27 March 2013 23:27:44 UTC, meric wrote:
>
> Tom, you're right about the second and third points though. If the user 
> perform any operation with side effect that requires writing to the 
> database before the view checks whether the keyword arguments are 
> appropriate and decide to raise DoesNotResolve, it can definitely be a 
> source of non-obvious bugs and surprising behaviour. For example, a counter 
> increment in the view to count how many times the view has been visited, 
> placed before checking for whether to raise DoesNotResolve. Multiple such 
> views get executed, multiple view counters incremented for one HTTP 
> connection.
>
> I can only think of adding a stern warning to the documentation something 
> along the lines of "Must not perform any operation requiring a database 
> write or any other operation with side effects before the check for 
> DoesNotResolve is made.".
>
> Eric
>
> On Thursday, March 28, 2013 3:28:10 AM UTC+11, Tom Christie wrote:
>>
>> * A `UrlNotMatched` exception sounds like the potential source of 
>> incredibly non-obvious bugs and surprising behavior.
>> * Allow a single HTTP call to end up calling into multiple views just 
>> seems like it would a fundamentally bad design decision.
>>
>> I really don't see the trade-off of allowing this type of behavior to be 
>> worth the cost of breaking the current mental model of what a view is and 
>> does.
>>
>> Just my personal opinion of course :)
>>
>>   Tom
>>
>>>  
>>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-27 Thread meric
Tom, you're right about the second and third points though. If the user 
perform any operation with side effect that requires writing to the 
database before the view checks whether the keyword arguments are 
appropriate and decide to raise DoesNotResolve, it can definitely be a 
source of non-obvious bugs and surprising behaviour. For example, a counter 
increment in the view to count how many times the view has been visited, 
placed before checking for whether to raise DoesNotResolve. Multiple such 
views get executed, multiple view counters incremented for one HTTP 
connection.

I can only think of adding a stern warning to the documentation something 
along the lines of "Must not perform any operation requiring a database 
write or any other operation with side effects before the check for 
DoesNotResolve is made.".

Eric

On Thursday, March 28, 2013 3:28:10 AM UTC+11, Tom Christie wrote:
>
> * A `UrlNotMatched` exception sounds like the potential source of 
> incredibly non-obvious bugs and surprising behavior.
> * Allow a single HTTP call to end up calling into multiple views just 
> seems like it would a fundamentally bad design decision.
>
> I really don't see the trade-off of allowing this type of behavior to be 
> worth the cost of breaking the current mental model of what a view is and 
> does.
>
> Just my personal opinion of course :)
>
>   Tom
>
>>  
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-27 Thread meric
Hi Tom, interested to see why you think a single wrapping view is a 
reasonable alternative for the example I showed above, where you have 
following list of URLs:

//  # front page for country
// / # list of schools and companies with activities in 
that industry, in that country
/// # list of industries company has activities in, in 
that country
 # list of activities company has in that 
industry, in that country
/// # list of activities school has in that country
 # list of activities school has, in that 
industry
/ / # list of schools and companies with activities in that 
industry, in all countries
// # list of industries company has, globally
/// # list of activities company has in that industry, 
globally
// # list of industries the school is involved in, globally
///  # list of activities school has in that industry, 
globally

What is the reasonable alternative? The only way I can think how a wrapping 
view will be able to handle this case, is to write your own router like 

router_view1(slug1=None)
router_view2(slug1=None, slug2=None)
router_view3(slug1=None, slug2=None, slug3=None)

In each case you'll have lots of ifs, try..excepts, for each model, and 
then to appropriate view. For the second and third router_views you'd have 
to have the same thing repeated, but this time it is nested.

Yet, in urls.py, all you have is:

(r'/')
(r'//')
(r'///')

Forcing the user to handle complex logic like this (each user probably 
needs to do it differently), which is not easy to test due to many branches 
involved, IMHO is a big cost.

There's also the question of view_middleware not being applied for the 
appropriate view unless the user specifically looks into how middleware 
gets called and manages handling view middleware themselves in the 
different router_views.

Perhaps you have a better way to implement this than what I can think of? 
This is how we do it in our company, and it'd be great if this can be 
improved.

On Thursday, March 28, 2013 3:28:10 AM UTC+11, Tom Christie wrote:
>
> For what it's worth I'd be against this proposal as it stands.
>
> * I haven't seen anything anything that convinces me that a single 
> wrapping view isn't a reasonable alternative in the examples cited.
> * A `UrlNotMatched` exception sounds like the potential source of 
> incredibly non-obvious bugs and surprising behavior.
> * Allow a single HTTP call to end up calling into multiple views just 
> seems like it would a fundamentally bad design decision.
>
> I really don't see the trade-off of allowing this type of behavior to be 
> worth the cost of breaking the current mental model of what a view is and 
> does.
>
> Just my personal opinion of course :)
>
>   Tom
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-27 Thread Tom Christie
For what it's worth I'd be against this proposal as it stands.

* I haven't seen anything anything that convinces me that a single wrapping 
view isn't a reasonable alternative in the examples cited.
* A `UrlNotMatched` exception sounds like the potential source of 
incredibly non-obvious bugs and surprising behavior.
* Allow a single HTTP call to end up calling into multiple views just seems 
like it would a fundamentally bad design decision.

I really don't see the trade-off of allowing this type of behavior to be 
worth the cost of breaking the current mental model of what a view is and 
does.

Just my personal opinion of course :)

  Tom

On Tuesday, 26 March 2013 17:49:07 UTC, Val Neekman wrote:
>
> +1 here! 
>
> It might also be a great feature to run the 404 through before sending 
> that email out. 
> A replacement and/or complement to IGNORABLE_404_URLS 
>
> Val 
>
>
> On Tue, Mar 26, 2013 at 1:25 PM, Loic Bistuer 
> <loic.b...@sixmedia.com> 
> wrote: 
> > +1 for me. 
> > 
> > Having a catchall view under a single URL pattern is a tightly coupled 
> > system, more so than allowing independent views to "test the water". 
> > 
> > Django core relies on middleware hacks because the URL dispatcher is 
> missing 
> > this very feature. Having this in core would allow a cleaner 
> implementation 
> > of the fallback mechanisms in contrib.flatpages and contrib.redirects. 
> > 
> > -- 
> > Loic 
> > 
> > On Mar 19, 2013, at 11:18 PM, Adrian Holovaty 
> > <adr...@holovaty.com> 
> wrote: 
> > 
> > I'd rather not add this to the framework, as it's already possible 
> > with a "wrapper" view (as others have suggested). And on a more 
> > theoretical level, it introduces more coupling between URL patterns 
> > and views. 
> > 
> > 
> > -- 
> > You received this message because you are subscribed to the Google 
> Groups 
> > "Django developers" group. 
> > To unsubscribe from this group and stop receiving emails from it, send 
> an 
> > email to django-develop...@googlegroups.com . 
> > To post to this group, send email to 
> > django-d...@googlegroups.com. 
>
> > Visit this group at 
> http://groups.google.com/group/django-developers?hl=en. 
> > For more options, visit https://groups.google.com/groups/opt_out. 
> > 
> > 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-26 Thread Val Neekman
+1 here!

It might also be a great feature to run the 404 through before sending
that email out.
A replacement and/or complement to IGNORABLE_404_URLS

Val


On Tue, Mar 26, 2013 at 1:25 PM, Loic Bistuer <loic.bist...@sixmedia.com> wrote:
> +1 for me.
>
> Having a catchall view under a single URL pattern is a tightly coupled
> system, more so than allowing independent views to "test the water".
>
> Django core relies on middleware hacks because the URL dispatcher is missing
> this very feature. Having this in core would allow a cleaner implementation
> of the fallback mechanisms in contrib.flatpages and contrib.redirects.
>
> --
> Loic
>
> On Mar 19, 2013, at 11:18 PM, Adrian Holovaty <adr...@holovaty.com> wrote:
>
> I'd rather not add this to the framework, as it's already possible
> with a "wrapper" view (as others have suggested). And on a more
> theoretical level, it introduces more coupling between URL patterns
> and views.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-26 Thread Loic Bistuer
+1 for me.

Having a catchall view under a single URL pattern is a tightly coupled system, 
more so than allowing independent views to "test the water".

Django core relies on middleware hacks because the URL dispatcher is missing 
this very feature. Having this in core would allow a cleaner implementation of 
the fallback mechanisms in contrib.flatpages and contrib.redirects.

-- 
Loic

On Mar 19, 2013, at 11:18 PM, Adrian Holovaty <adr...@holovaty.com> wrote:

> I'd rather not add this to the framework, as it's already possible
> with a "wrapper" view (as others have suggested). And on a more
> theoretical level, it introduces more coupling between URL patterns
> and views.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-26 Thread Andre Terra
On Tue, Mar 26, 2013 at 10:44 AM, meric  wrote:

> Thinking through Adrian's post and mine above, it appears to be a trade
> off between coupling in the framework and increased responsibility for user
> views code. Personally I would opt for the former, because IMHO the idea of
> a framework is to reduce responsibility of user code and promoting same
> sets of design patterns for common problems (rather than each similar
> program having different adhoc ways to deal with the same problems I.e
> rather than having different ways people would implement a catch all view,
> have views send the same signal if the given arguments were not
> interpretable.) first before anything else, otherwise what else can a
> framework provide?.


Even though I suggested the use of a catch-all view for handling multiple
objects in the beginning of the thread, I now see the value of moving this
to urls.py. For more complex websites and scenarios, some patterns may
emerge that lead to such a number of possibilities that at some point, from
a design perspective, you start wondering if you're not packing too much
routing into the views (with a catch-all view) when they should be handled
by the URL routing module.

Previously -1, now +1 on this idea.


Cheers,
AT

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-26 Thread meric
Thinking through Adrian's post and mine above, it appears to be a trade off 
between coupling in the framework and increased responsibility for user views 
code. Personally I would opt for the former, because IMHO the idea of a 
framework is to reduce responsibility of user code and promoting same sets of 
design patterns for common problems (rather than each similar program having 
different adhoc ways to deal with the same problems I.e rather than having 
different ways people would implement a catch all view, have views send the 
same signal if the given arguments were not interpretable.) first before 
anything else, otherwise what else can a framework provide?.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-25 Thread meric
Previous discussion (which I've read before): 
https://code.djangoproject.com/ticket/16774 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-25 Thread meric
I've previously raised this idea and created a pull request.

https://github.com/django/django/pull/378

The problem with creating with a catch all view:

You have following models:

Country, Industry, Company, School.

You want to have the following kinds of urls:

//
// /
///

///

/ /
//
///
//
///

It can get extremely cumbersome to use catch-all views to manage all of 
these URLs, rather than a single view with optional keyword-arguments.

In my example I have only listed 4 models. It wouldn't be implausible to 
suggest there are situations when there could be more.

I suggest it is better for the framework to handle this complex URL routing 
on the urls.py level, because it really isn't the view's responsibility to 
think about URLs; Since URLs already introspect into view's arguments, and 
in some cases, even instantiate class based views, especially generic 
views, it would be better to keep urlresolver's relationship with views 
instead of introducing a new relationship where views must now take care of 
URLs too.

I think the Inversion of Control principle would be good to 
follow: http://en.wikipedia.org/wiki/Inversion_of_control








On Tuesday, March 19, 2013 2:23:41 AM UTC+11, julianb wrote:
>
> Hi,
>
> imagine the following use case:
>
> You build an online store where you have sorted products into several 
> categories and maybe associated an occasion. Now you want to build URLs. So 
> the URL schema that all of the store's owners agree on is:
>
> //
> //
> //
>
> Look simple.
>
> Because product slugs should generally be different to category slugs and 
> occasions, you expect no clashes here. And you don't want to bloat the URLs 
> by putting product/ and category/ in there. And you also do not want to 
> resort making it look cryptic by having just /p// and so on.
> Then it comes to Django. How do you do that?
>
> Well, at the moment, as far as I am aware, you can't. The first URL will 
> match everything all the time, not giving the other views a chance to kick 
> in.
>
> So I propose some kind of URL fallthrough. The view could do
>
> raise UrlNotMatched
>
> and the dispatcher goes to the next possible matching view.
>
> Would that be good? Any thoughts?
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: An idea for the URL dispatcher

2013-03-21 Thread Carl Meyer
Hi Daniele,

On 03/21/2013 02:07 AM, Daniele Procida wrote:
> I think it would at minimum require hooks so that the URL dispatcher
> could call other functions, not just views.
> 
> So:
> 
> * the URL dispatcher could be invoked with a special argument when
> you want it not to call views but perform some other function
> 
> * the URL confs could take this argument, and call the function it
> specifies instead of the usual view, failing gracefully if the
> function doesn't exist
> 
> 
> I guess it would have to be a keyword argument such as:
> 
> { 'alternative_function': 'some.alternative.function'}

I don't think this is necessary. The url resolver module does not call
anything; it just returns ResolverMatch objects with "func", "args", and
"kwargs" attributes (where "func" is a view function). I think it would
make more sense to leave the url-resolver alone, and build your system
around it; if you have a way to map a view-func plus args/kwargs to a
model instance (which is more or less what you'll need in any
implementation of this idea), then you can get the ResolverMatch from
the url resolver and resolve that to a model.

Carl



signature.asc
Description: OpenPGP digital signature


Re: An idea for the URL dispatcher

2013-03-21 Thread Daniele Procida
On Wed, Mar 20, 2013, Carl Meyer <c...@oddbird.net> wrote:

>> Often - not always but often - a Django URL will map to a model
>> instance via a URLconf and a view

>I think this is an interesting idea for some specific use cases, but I
>don't think its implementation belongs in the core url-resolver code,
>for a couple reasons

>If you work on implementing this outside Django and find that you need
>hooks that are not currently available, those hooks would making
>interesting feature proposals for Django.

Thanks for the feedback.

I think it would at minimum require hooks so that the URL dispatcher could call 
other functions, not just views.

So:

* the URL dispatcher could be invoked with a special argument when you want it 
not to call views but perform some other function

* the URL confs could take this argument, and call the function it specifies 
instead of the usual view, failing gracefully if the function doesn't exist


I guess it would have to be a keyword argument such as:

{ 'alternative_function': 'some.alternative.function'}

In fact this would allow you to use the URL dispatcher for other URL-related 
things, not just my URL-to-model functionality (though I can't think of 
anything very useful just at this moment).

Then as you say the actual functionality would be built outside Django itself.

How does that sound?

Daniele

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: An idea for the URL dispatcher

2013-03-20 Thread Carl Meyer
Hi Daniele,

On 03/20/2013 05:46 AM, Daniele Procida wrote:
> I have an idea for the URL dispatcher, and I'd like to know what your
> thoughts are.
> 
> Often - not always but often - a Django URL will map to a model
> instance via a URLconf and a view.
> 
> I can think of a few uses for a feature that used Django's URLconfs
> to return the model instance, if there is one, mapped to by a
> particular URL.
> 
> In other words, one would feed a URL to the system, which would
> return a model instance, if one could be uniquely associated with
> that URL.

I think this is an interesting idea for some specific use cases, but I
don't think its implementation belongs in the core url-resolver code,
for a couple reasons:

1) The pattern is simply not common enough to require an implementation
in core Django.

2) It couples the URL resolver to the ORM. These are separate modules
that should not need to depend on one another, in either direction.

If you work on implementing this outside Django and find that you need
hooks that are not currently available, those hooks would making
interesting feature proposals for Django.

Carl



signature.asc
Description: OpenPGP digital signature


An idea for the URL dispatcher

2013-03-20 Thread Daniele Procida
I have an idea for the URL dispatcher, and I'd like to know what your thoughts 
are.

Often - not always but often - a Django URL will map to a model instance via a 
URLconf and a view.

I can think of a few uses for a feature that used Django's URLconfs to return 
the model instance, if there is one, mapped to by a particular URL.

In other words, one would feed a URL to the system, which would return a model 
instance, if one could be uniquely associated with that URL. 

Examples of the utility of this idea:

* applications that handle links to other Django models using generic content 
types often make the admin user select the object's class and instance; this 
feature would allow them also to enter a URL into a form, and the system would 
then select the object's class and instance, if a match can be found

* web server logs and analytics reports list URLs visited, that then have to be 
interpreted; from this one could obtain a report listing actual things that 
visitors were interested in

* it could be an easy way to get to the admin URL for any model instance from 
its URL

Currently, Django's URLconfs are used to call views. But, I think it would be 
feasible to amend them so that they could also return an object, using optional 
arguments to ask it to do that instead of calling the view.

As I understand it, namespacing like:

(r'^help/', include('apps.help.urls', namespace='foo', app_name='bar')),

already helps map URLs and model instances. Then, if each particular URLconf 
had an optional argument:

mapper = some_mapping_function

then as long as there is some way to tell the URL dispatcher that it should not 
call the view, but instead call the mapper, it will return the model instance 
if it finds one.

So my questions are:

* would this functionality be valuable
* is the concept viable
* and would it be worth the effort to develop it?

I'd certainly have a lot of use for it myself, but I don't know whether it 
would be more widely useful, and I don't fully understand what would be 
involved in implementing it.

Daniele

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-19 Thread Ian Kelly
On Tue, Mar 19, 2013 at 12:26 PM, Felipe Prenholato
 wrote:
> Hi Julian. My 2 cents.
>
> What you want is:
>
> urlpatterns = patterns('',
> url_fallthrought(r'^(?P[-\w]+)$',
> views=[(ProductView.as_view(),
>CategorView.as_view(),
>OccasionView.as_view())], name="my_super_url"),)

def view_chain(views):
def inner(*args, **kwargs):
for view in views:
try:
return view(*args, **kwargs)
except Http404:
pass
raise Http404
return inner

urlpatterns = patterns('',
(r'^(?P[-\w]+)$',
 view_chain([ProductView.as_view(),
 CategoryView.as_view(),
 OccasionView.as_view()]), name="my_super_url"),
)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-19 Thread Felipe Prenholato
Hi Julian. My 2 cents.

What you want is:

urlpatterns = patterns('',
url_fallthrought(r'^(?P[-\w]+)$',
views=[(ProductView.as_view(),
   CategorView.as_view(),
   OccasionView.as_view())], name="my_super_url"),)

But you know that we don't have it. To make it works you will need to do
some loop on views and answer with first that not raises 404, and than
return it response. To be better you just run get_object() of view and
after return get() or post(). It should be done on object (wrapper view?)
that url_fallthrought returns. This will be very very similar to what you
will do if use a wrapper view... to not say same code.

This functionality will be used mostly by Django based CRMs, Blogs and
E-commerce solutions, lot of people of course, but in my opinion is so
simple to do that don't need to be in Django itself.

Att,

Felipe 'chronos' Prenholato.
Linux User nº 405489
Home page: http://devwithpassion.com | http://chronosbox.org/blog
GitHub: http://github.com/chronossc/ | Twitter: http://twitter.com/chronossc


2013/3/19 Adrian Holovaty 

> On Mon, Mar 18, 2013 at 10:23 AM, julianb  wrote:
> > Well, at the moment, as far as I am aware, you can't. The first URL will
> > match everything all the time, not giving the other views a chance to
> kick
> > in.
> >
> > So I propose some kind of URL fallthrough. The view could do
> >
> > raise UrlNotMatched
> >
> > and the dispatcher goes to the next possible matching view.
> >
> > Would that be good? Any thoughts?
>
> Hi Julian,
>
> I'd rather not add this to the framework, as it's already possible
> with a "wrapper" view (as others have suggested). And on a more
> theoretical level, it introduces more coupling between URL patterns
> and views.
>
> Adrian
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-19 Thread Adrian Holovaty
On Mon, Mar 18, 2013 at 10:23 AM, julianb  wrote:
> Well, at the moment, as far as I am aware, you can't. The first URL will
> match everything all the time, not giving the other views a chance to kick
> in.
>
> So I propose some kind of URL fallthrough. The view could do
>
> raise UrlNotMatched
>
> and the dispatcher goes to the next possible matching view.
>
> Would that be good? Any thoughts?

Hi Julian,

I'd rather not add this to the framework, as it's already possible
with a "wrapper" view (as others have suggested). And on a more
theoretical level, it introduces more coupling between URL patterns
and views.

Adrian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-19 Thread Tom Evans
On Mon, Mar 18, 2013 at 3:23 PM, julianb  wrote:
> Hi,
>
> imagine the following use case:
>
> You build an online store where you have sorted products into several
> categories and maybe associated an occasion. Now you want to build URLs. So
> the URL schema that all of the store's owners agree on is:
>
> //
> //
> //
>

Really straying into user talk now...

If you have a heterogeneous hierarchy of objects that have slugs, and
you wish to look up arbitrary objects by slug, perhaps just go the
whole hog, add a SluggedItem model with a generic foreign key, and a
view that fetches items from their slugs and dispatches to the
appropriate view.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-19 Thread julianb
A single pattern with a catchall view is certainly a way to go. However, I 
really does not help to see what the views do. And if you have different 
views with different arguments, you cannot do your regex explicitly.
You are right that it is non-obvious when the view itself has to invoke the 
fallthrough. So I'm thinking it would be better in the urlpatterns itself. 
Then it is explicitly clear, rather than having some catchall magic do-all 
views and patterns. Right?

On Monday, March 18, 2013 4:44:19 PM UTC+1, Andrew Ingram wrote:
>
> The approach we take at work is to have a view that wraps around the 
> product, category and occasion views and takes care of the fallthrough 
> logic. I'm not a fan of this approach, because it means that you can't just 
> look at the urlconf and see which pattern maps to which view function. On 
> the other hand, I don't think your solution would solve this either, you'd 
> end up having 3 patterns that could all match, making it non-obvious what 
> the behaviour is.
>
> Obviously the ideal solution is to scope URLs correctly to avoid all this 
> in the first place, but I've found that in the real world this is easier 
> said than done. So it would be nice to have an idiomatic solution.
>
> An alternative might be a kwarg on the url function, such as 
> 'fallthrough_on_404'. But it all feels a bit hackish to me :/
>
> Regards,
> Andy
>
>
> On 18 March 2013 15:23, julianb  wrote:
>
>> Hi,
>>
>> imagine the following use case:
>>
>> You build an online store where you have sorted products into several 
>> categories and maybe associated an occasion. Now you want to build URLs. So 
>> the URL schema that all of the store's owners agree on is:
>>
>> //
>> //
>> //
>>
>> Look simple.
>>
>> Because product slugs should generally be different to category slugs and 
>> occasions, you expect no clashes here. And you don't want to bloat the URLs 
>> by putting product/ and category/ in there. And you also do not want to 
>> resort making it look cryptic by having just /p// and so on.
>> Then it comes to Django. How do you do that?
>>
>> Well, at the moment, as far as I am aware, you can't. The first URL will 
>> match everything all the time, not giving the other views a chance to kick 
>> in.
>>
>> So I propose some kind of URL fallthrough. The view could do
>>
>> raise UrlNotMatched
>>
>> and the dispatcher goes to the next possible matching view.
>>
>> Would that be good? Any thoughts?
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Django developers" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to django-develop...@googlegroups.com .
>> To post to this group, send email to 
>> django-d...@googlegroups.com
>> .
>> Visit this group at 
>> http://groups.google.com/group/django-developers?hl=en.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>  
>>  
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-18 Thread Chris Wilson

Hi Julian,

On Mon, 18 Mar 2013, julianb wrote:


imagine the following use case:

You build an online store where you have sorted products into several 
categories and maybe associated an occasion. Now you want to build URLs. So the 
URL schema that all of the
store's owners agree on is:

//
//
//

Look simple.

Because product slugs should generally be different to category slugs and 
occasions, you expect no clashes here. And you don't want to bloat the URLs by 
putting product/ and
category/ in there. And you also do not want to resort making it look cryptic by 
having just /p// and so on.
Then it comes to Django. How do you do that?


I put the type in the URL to make it obvious. That doesn't harm SEO and 
makes your URL routing clearer. For example:


/product//
/category//
/occasion//

But that's getting into django-users territory.

Cheers, Chris.
--
Aptivate | http://www.aptivate.org | Phone: +44 1223 967 838
Future Business, Cam City FC, Milton Rd, Cambridge, CB4 1UY, UK

Aptivate is a not-for-profit company registered in England and Wales
with company number 04980791.

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-18 Thread Andre Terra
Hi, Julian

I think this problem can already be addressed by having a single catchall
// URL pattern, and writing a SlugHandlerView which would try
different options  within the view code, rather than in the URL dispatching.

Then it would just be a matter or catching exceptions for your .get()
methods, and raising a final 404 in case nothing is found, which would in
turn make this a django-users sort of question from here on.

Unless I'm missing something, this behavior can be achieved with what's
already in core today.


Cheers,
AT


On Mon, Mar 18, 2013 at 12:23 PM, julianb  wrote:

> Hi,
>
> imagine the following use case:
>
> You build an online store where you have sorted products into several
> categories and maybe associated an occasion. Now you want to build URLs. So
> the URL schema that all of the store's owners agree on is:
>
> //
> //
> //
>
> Look simple.
>
> Because product slugs should generally be different to category slugs and
> occasions, you expect no clashes here. And you don't want to bloat the URLs
> by putting product/ and category/ in there. And you also do not want to
> resort making it look cryptic by having just /p// and so on.
> Then it comes to Django. How do you do that?
>
> Well, at the moment, as far as I am aware, you can't. The first URL will
> match everything all the time, not giving the other views a chance to kick
> in.
>
> So I propose some kind of URL fallthrough. The view could do
>
> raise UrlNotMatched
>
> and the dispatcher goes to the next possible matching view.
>
> Would that be good? Any thoughts?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher fallthrough?

2013-03-18 Thread Andrew Ingram
The approach we take at work is to have a view that wraps around the
product, category and occasion views and takes care of the fallthrough
logic. I'm not a fan of this approach, because it means that you can't just
look at the urlconf and see which pattern maps to which view function. On
the other hand, I don't think your solution would solve this either, you'd
end up having 3 patterns that could all match, making it non-obvious what
the behaviour is.

Obviously the ideal solution is to scope URLs correctly to avoid all this
in the first place, but I've found that in the real world this is easier
said than done. So it would be nice to have an idiomatic solution.

An alternative might be a kwarg on the url function, such as
'fallthrough_on_404'. But it all feels a bit hackish to me :/

Regards,
Andy


On 18 March 2013 15:23, julianb  wrote:

> Hi,
>
> imagine the following use case:
>
> You build an online store where you have sorted products into several
> categories and maybe associated an occasion. Now you want to build URLs. So
> the URL schema that all of the store's owners agree on is:
>
> //
> //
> //
>
> Look simple.
>
> Because product slugs should generally be different to category slugs and
> occasions, you expect no clashes here. And you don't want to bloat the URLs
> by putting product/ and category/ in there. And you also do not want to
> resort making it look cryptic by having just /p// and so on.
> Then it comes to Django. How do you do that?
>
> Well, at the moment, as far as I am aware, you can't. The first URL will
> match everything all the time, not giving the other views a chance to kick
> in.
>
> So I propose some kind of URL fallthrough. The view could do
>
> raise UrlNotMatched
>
> and the dispatcher goes to the next possible matching view.
>
> Would that be good? Any thoughts?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to django-developers+unsubscr...@googlegroups.com.
> To post to this group, send email to django-developers@googlegroups.com.
> Visit this group at http://groups.google.com/group/django-developers?hl=en
> .
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




URL dispatcher fallthrough?

2013-03-18 Thread julianb
Hi,

imagine the following use case:

You build an online store where you have sorted products into several 
categories and maybe associated an occasion. Now you want to build URLs. So 
the URL schema that all of the store's owners agree on is:

//
//
//

Look simple.

Because product slugs should generally be different to category slugs and 
occasions, you expect no clashes here. And you don't want to bloat the URLs 
by putting product/ and category/ in there. And you also do not want to 
resort making it look cryptic by having just /p// and so on.
Then it comes to Django. How do you do that?

Well, at the moment, as far as I am aware, you can't. The first URL will 
match everything all the time, not giving the other views a chance to kick 
in.

So I propose some kind of URL fallthrough. The view could do

raise UrlNotMatched

and the dispatcher goes to the next possible matching view.

Would that be good? Any thoughts?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to django-developers+unsubscr...@googlegroups.com.
To post to this group, send email to django-developers@googlegroups.com.
Visit this group at http://groups.google.com/group/django-developers?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.




Re: URL dispatcher slow?

2012-10-16 Thread Moonlight
static[0]   

 138017 function calls in 0.818 seconds

   Ordered by: internal time
   List reduced from 79 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
 10000.0620.0000.4900.000 base.py:72(get_response)
 20000.0470.0000.1290.000 base.py:240(get_script_name)
 10000.0420.0000.7580.001 wsgi.py:210(__call__)
 10000.0400.0000.2470.000 urlresolvers.py:293(resolve)
 10000.0390.0000.1360.000 wsgi.py:129(__init__)
 20000.0310.0000.0520.000 
trans_real.py:212(get_language)
 70000.0310.0000.0440.000 functional.py:182(inner)
 30000.0290.0000.0880.000 encoding.py:54(force_unicode)
140020.0290.0000.0290.000 {isinstance}
 10000.0270.0000.0810.000 __init__.py:564(__init__)


static[-1]  

 1929017 function calls in 10.654 seconds

   Ordered by: internal time
   List reduced from 80 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   2010002.9980.0004.9280.000 
trans_real.py:212(get_language)
   201.2880.0008.3780.000 urlresolvers.py:195(resolve)
 10001.1750.0019.9730.010 urlresolvers.py:293(resolve)
   4080001.0850.0001.0850.000 {getattr}
   2060000.8790.0001.2310.000 functional.py:182(inner)
   2010000.8780.0006.5770.000 urlresolvers.py:153(regex)
   2010000.7550.0005.6820.000 __init__.py:128(get_language)
   2010000.5470.0000.5470.000 {method 'search' of 
'_sre.SRE_Pattern' objects}
   1990000.2900.0000.2900.000 {method 'append' of 'list' 
objects}
 10000.0940.000   10.2720.010 base.py:72(get_response)


dynamic[0]

 3745017 function calls in 20.459 seconds

   Ordered by: internal time
   List reduced from 80 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   4020006.0060.0009.8660.000 
trans_real.py:212(get_language)
   4010002.5280.000   16.7000.000 urlresolvers.py:195(resolve)
 10002.3010.002   19.7330.020 urlresolvers.py:293(resolve)
   812.1620.0002.1620.000 {getattr}
   4020001.7350.000   13.1070.000 urlresolvers.py:153(regex)
   4070001.7310.0002.4240.000 functional.py:182(inner)
   4020001.4890.000   11.3550.000 __init__.py:128(get_language)
   4020001.0980.0001.0980.000 {method 'search' of 
'_sre.SRE_Pattern' objects}
   400.5710.0000.5710.000 {method 'append' of 'list' 
objects}
 10000.1370.000   20.0760.020 base.py:72(get_response)


dynamic[-1]

 3916017 function calls in 29.633 seconds

   Ordered by: internal time
   List reduced from 80 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   4210008.6500.000   14.2670.000 
trans_real.py:212(get_language)
   423.6930.000   24.2790.000 urlresolvers.py:195(resolve)
 10003.2990.003   28.6060.029 urlresolvers.py:293(resolve)
   8480003.1120.0003.1120.000 {getattr}
   4260002.5520.0003.5920.000 functional.py:182(inner)
   4210002.5270.000   18.9930.000 urlresolvers.py:153(regex)
   4210002.1760.000   16.4430.000 __init__.py:128(get_language)
   4210001.6390.0001.6390.000 {method 'search' of 
'_sre.SRE_Pattern' objects}
   4190000.8050.0000.8050.000 {method 'append' of 'list' 
objects}
 10000.2110.000   29.1030.029 base.py:72(get_response)


seo[0]

 1941017 function calls in 14.737 seconds

   Ordered by: internal time
   List reduced from 80 to 10 due to restriction <10>

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
   2020004.1620.0006.8550.000 
trans_real.py:212(get_language)
   2010001.7870.000   11.5990.000 urlresolvers.py:195(resolve)
 10001.5940.002   13.7970.014 urlresolvers.py:293(resolve)
   411.4970.0001.4970.000 {getattr}
   2070001.2430.0001.7470.000 functional.py:182(inner)
   2020001.2300.0009.1640.000 urlresolvers.py:153(regex)
   2020001.0560.0007.9110.000 __init__.py:128(get_language)
   2020000.6930.0000.6930.000 {method 'search' of 
'_sre.SRE_Pattern' objects}
   200.4020.0000.4020.000 {method 'append' of 'list' 
objects}
 10000.1430.000   14.2210.014 base.py:72(get_response)


seo[-1]

 2022017 function calls in 15.401 seconds

   Ordered by: internal time
   List reduced from 80 to 10 due to restriction <10>

   ncalls  tottime  

Re: URL dispatcher slow?

2012-10-12 Thread Tom Evans
On Thu, Oct 11, 2012 at 10:58 PM, Łukasz Rekucki  wrote:
> On 11 October 2012 10:20, Russell Keith-Magee  wrote:
>>
>> And don't just say "Why are Django's URL resolvers slow?". Do some
>> profiling, and come back with an analysis of where the time is being
>> spent and/or wasted.
>
> FWIW, here's a link to a cProfile result for the mentioned
> benchmark[1] on Django 1.4.1 and CPython 2.7.3. A quick look shows
> that we're calling get_language() 1.5mln times (read: for every
> pattern), so that's definitely going to slow down things.
>
> I'll try running the same with 1.3 and maybe 1.4 without the locale
> mixin and post if I find anything interesting.
>
> [1]: https://gist.github.com/3875701
>
>

It definitely doesn't help; ideally language code should be calculated
once per call to resolve, and the same value used throughout.
Hard-coding a value of 'en' for language code in
LocaleRegexProvider.regex gives an idea of what gains would be:

Stock django:

static[0] msecrps  tcalls  funcs
django1583   6318 143 69

static[-1]msecrps  tcalls  funcs
django   208794791934 70


Hard coded 'en' in LocaleRegexProvider.regex:

static[0] msecrps  tcalls  funcs
django1424   7023 133 67

static[-1]msecrps  tcalls  funcs
django4972   2011 929 68

static[0] represents tests matching the first URL in the urlconf,
static[-1] represents tests matching the approximately 100th URL in
the urlconf.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-12 Thread Moonlight


> FWIW, here's a link to a cProfile result for the mentioned 
> benchmark[1] on Django 1.4.1 and CPython 2.7.3. A quick look shows 
> that we're calling get_language() 1.5mln times (read: for every 
> pattern), so that's definitely going to slow down things. 
>

Happy to see we moved it a bit forward! Well done!

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/ZnKPiKqj-tgJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-12 Thread Tom Evans
On Thu, Oct 11, 2012 at 2:53 PM, Tom Evans <tevans...@googlemail.com> wrote:
> On Wed, Oct 10, 2012 at 7:52 AM, Moonlight <moonlight_13_...@yahoo.com> wrote:
>> Here is an article comparing various URL dispatchers:
>>
>> http://mindref.blogspot.com/2012/10/python-web-routing-benchmark.html
>>
>> What cause django URL dispatcher that much... slow?
>>
>
> Now that I've looked in detail at the test, it is because the test is
> nonsensical. Each time it tests the URLs, it constructs a fresh WSGI
> application. Each fresh application has to compile each URL in the
> urlconf before using it. It then destroys the application, and starts
> another one up.
>
> In a normal django application, you do not start from fresh on each
> request. This explains why the performance degradation from
> "static[0]" to "static[-1] " - which is the difference between testing
> the first url in the urlconf and the last url in the urlconf - is so
> pathological, each time it is run Django is recompiling each and every
> URL regexp in the urlconf (as well as _all_ other work django does at
> server start).
>
> This is testing something, but it is not testing django as a web
> application, or as Django is intended to be run.
>
> Cheers
>
> Tom

I was wrong, the WSGI app is initialized correctly just once.
Something pathological is happening with this test case, the
performance of the resolver looks decidedly dicey as number of
patterns grows.

Something for the weekend…

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Łukasz Rekucki
On 11 October 2012 10:20, Russell Keith-Magee  wrote:
>
> And don't just say "Why are Django's URL resolvers slow?". Do some
> profiling, and come back with an analysis of where the time is being
> spent and/or wasted.

FWIW, here's a link to a cProfile result for the mentioned
benchmark[1] on Django 1.4.1 and CPython 2.7.3. A quick look shows
that we're calling get_language() 1.5mln times (read: for every
pattern), so that's definitely going to slow down things.

I'll try running the same with 1.3 and maybe 1.4 without the locale
mixin and post if I find anything interesting.

[1]: https://gist.github.com/3875701


-- 
Łukasz Rekucki

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Sean Bleier
JKM, I'm wondering if it would benefit the community to house
djangobench under https://github.com/django to give it more
visibility.  Just a thought.

On Thu, Oct 11, 2012 at 9:45 AM, ptone  wrote:
>
>
> On Thursday, October 11, 2012 1:21:09 AM UTC-7, Russell Keith-Magee wrote:
>>
>>
>> That said - could Django be faster? Sure. *Anything* can be improved.
>>
>> So, if you've got a suggestion, make it. If you think URL resolving is
>> the source of the problem, propose a way to improve the speed of URL
>> resolvers. If you think the template language is the problem, propose
>> a fix.
>
>
> To do my part to increase the signal:noise.
>
> I'll point out that just as writing a failing test is a great way to augment
> a bug report - contributing or improving a benchmark for:
>
> https://github.com/jacobian/djangobench
>
> would be a good way to contribute to a discussion around a particular
> performance improvement you're interested in seeing happen.
>
> -Preston
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/django-developers/-/kI7jNl7gYwEJ.
>
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Marco Paolini

On 11/10/2012 15:53, Tom Evans wrote:

On Wed, Oct 10, 2012 at 7:52 AM, Moonlight <moonlight_13_...@yahoo.com> wrote:

Here is an article comparing various URL dispatchers:

http://mindref.blogspot.com/2012/10/python-web-routing-benchmark.html

What cause django URL dispatcher that much... slow?



Now that I've looked in detail at the test, it is because the test is
nonsensical. Each time it tests the URLs, it constructs a fresh WSGI
application. Each fresh application has to compile each URL in the
urlconf before using it. It then destroys the application, and starts
another one up.

no, the wsgi handler is instatiated only once and outside the timeit call


--
markopaol...@gmail.com

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Aymeric Augustin
2012/10/11 Tom Evans 

> Now that I've looked in detail at the test, it is because the test is
> nonsensical. Each time it tests the URLs, it constructs a fresh WSGI
> application. Each fresh application has to compile each URL in the
> urlconf before using it. It then destroys the application, and starts
> another one up.
>

Hi Tom,

The performance gap was really huge, roughly two orders of magnitude.
I had a gut feeling that the implementation of the URL resolver couldn't
explain it fully. This makes much more sense.

So, thank you for checking the benchmark's source code!

(I have to admit that I was too lazy and didn't care enough to do it
myself.)

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Michael
This conversation is getting nasty. Django's URL dispatcher is what it is.
If you can get the flexibility that the current URL dispatcher provides
while making it faster, I imagine everyone on this thread would be
impressed.

There is no reason to use Django's URL dispatcher. It is quite easy to map
a WSGI path to the view functions. Heck, you can even write your own URL
dispatcher and let us all see your method. Maybe it will get traction.

I have not seen performance bottlenecks on templates or URL dispatchers. My
systems tend to bail due to the database or IO first. So benchmarks are
great and something to keep an eye on while adding new features or making
changes, but I am not going to worry about the abstract performance of URL
Dispatchers until it is clear that it accounts for a clear problem in the
request/response cycle.

Other than that we are seeing problems that are not actually there and
nothing will ever get done.


On Thu, Oct 11, 2012 at 10:01 AM, Alex Gaynor <alex.gay...@gmail.com> wrote:

>
>
> On Thu, Oct 11, 2012 at 6:52 AM, Daniel Sokolowski <
> daniel.sokolow...@klinsight.com> wrote:
>
>>   I absolutely agree with: if we were looking for speed we wouldn't use
>> python at all (period).
>>
>>
> Speak for yourself.
>
> Alex
>
> --
> "I disapprove of what you say, but I will defend to the death your right
> to say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
> "The people's good is the highest law." -- Cicero
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Daniel Sokolowski
Was that necessary?  I am tuning out of this conversion, it is becoming 
hostile. 
From: Alex Gaynor 
Sent: Thursday, October 11, 2012 10:01 AM
To: django-developers@googlegroups.com 
Subject: Re: URL dispatcher slow?




On Thu, Oct 11, 2012 at 6:52 AM, Daniel Sokolowski 
<daniel.sokolow...@klinsight.com> wrote:

  I absolutely agree with: if we were looking for speed we wouldn't use python 
at all (period).


Speak for yourself.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your right to say 
it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Alex Gaynor
On Thu, Oct 11, 2012 at 6:52 AM, Daniel Sokolowski <
daniel.sokolow...@klinsight.com> wrote:

>   I absolutely agree with: if we were looking for speed we wouldn't use
> python at all (period).
>
>
Speak for yourself.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Tom Evans
On Wed, Oct 10, 2012 at 7:52 AM, Moonlight <moonlight_13_...@yahoo.com> wrote:
> Here is an article comparing various URL dispatchers:
>
> http://mindref.blogspot.com/2012/10/python-web-routing-benchmark.html
>
> What cause django URL dispatcher that much... slow?
>

Now that I've looked in detail at the test, it is because the test is
nonsensical. Each time it tests the URLs, it constructs a fresh WSGI
application. Each fresh application has to compile each URL in the
urlconf before using it. It then destroys the application, and starts
another one up.

In a normal django application, you do not start from fresh on each
request. This explains why the performance degradation from
"static[0]" to "static[-1] " - which is the difference between testing
the first url in the urlconf and the last url in the urlconf - is so
pathological, each time it is run Django is recompiling each and every
URL regexp in the urlconf (as well as _all_ other work django does at
server start).

This is testing something, but it is not testing django as a web
application, or as Django is intended to be run.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Daniel Sokolowski
I absolutely agree with: if we were looking for speed we wouldn't use python at 
all (period).

From: Florian Apolloner 
Sent: Wednesday, October 10, 2012 6:29 PM
To: django-developers@googlegroups.com 
Subject: Re: URL dispatcher slow?

Oh cmon,

please stop playing a socket puppet for the wheezy.web author. A web framework 
consists of more than just a win in speed (an the author of wheezy.web can 
argue whatever he wants that "basic" stuff stuff should be fast) -- if we were 
looking for speed we wouldn't use python at all (period). The goal is to be 
fast enough and be able to scale out horizontally, everything else is not 
really interesting in the case of web development. While I won't deny that 
Django isn't the fastest framework out there it certainly gets it's job done 
more than good enough and that's all it matters. I don't know what your goal 
is, but please stop posting such meaningless comparisons on django-dev…

That said, if you really think Django's urlresolver is to slow, go ahead, 
profile it and improve it, we certainly won't say no to speed improvements…

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/DgNcKfL2AeUJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Daniel Sokolowski
http://webdesign.danols.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Yo-Yo Ma
I only now realized that this thread had started in a bit of a trolling 
fashion, and that there was a similar thread this week. That helps to explain 
the slightly more defensive stance.

Django can survive (and thrive) just fine in its current, reasonably performant 
state, but performance should have a priority, rather than being in a sort of 
"its fine as-is" stasis. 

Benchmarks like the ones posted aren't that helpful, but it doesn't change the 
fact that there tends to be an anti-performance sentiment in this group. When 
you look at the latest Python 3.3.x release, you see that there are many 
performance improvements that are clearly not due to incidental changes. I'm 
merely suggesting that an initiative to better the performance of Django, 
perhaps by benchmarking to find some of the weakest points, determining the 
lowest hanging fruit, and creating tickets in trac for them. I would love to 
help optimize any given part of Django and submit a patch, but not if I'm 
afraid that it will never be applied and/or the ticket will be marked as "works 
for me", etc. Help give us who care about performance the most a real chance to 
make some improvements. 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/bxfyOKgoRogJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Fw: URL dispatcher slow?

2012-10-11 Thread Kkk Kkk
somehow it went to wrong address... sorry, here it is.

- Forwarded Message -


>This is why I disagree with your conjecture that because Django is
>slower than another framework, therefore Django is slow. That is not
>what is shown, only that the other frameworks do certain things
> faster.

I think we have the same standpoint of view here. May be it is faster somewhere 
else... but where? If it is slower here and there... it is probaly just slow, 
isn't it? May be the word 'slow' is not correct... if the framework does more 
for me in URL dispatch I want to know what that more means... I know it can do 
reverse url but that feature is not used in url dispatch.

>Open source is wonderful, because you are allowed the choice of
>framework. You can choose full featured, and accept that those
>features will cost time, or you can choose lightweight, and not use
>those features. There is no magical mid-way point where you get all
>the features and all the speed.

Agreed. We are free to talk about this and that is good. It would be better if 
someone also listen. If the difference between lightweight and full featured 
impacts performance that much I want to see which features cause that. Let 
itemize the list. Honesty I tried... but just can not related them to the facts 
faced.




 From: Tom Evans <tevans...@googlemail.com>
To: Moonlight <moonlight_13_...@yahoo.com> 
Sent: Thursday, October 11, 2012 3:46 PM
Subject: Re: URL dispatcher slow?
 
On Thu, Oct 11, 2012 at 1:07 PM, Moonlight <moonlight_13_...@yahoo.com> wrote:
>> So, the benchmarks are interesting. They tell us which stacks are
>> fully featured, and which stacks are very lightweight. Apart from
>> that, they don't tell us much at all - is Django's template engine
>> slow, or is it about right for the work it does? This benchmark
>> doesn't tell us that, it only says it is slower than a bare bones
>> template engine, which is unsurprising, and shouldn't be a cause for
>> concern.
>
>
> It is sort of no sense. Django is fully featured and we do not care... some
> other are fast because they are lightweight... tell me which 'full
 featured'
> feature prevents it from become better. I guess for the #1 framework it is
> important to be leader... hmm... not sure what you mean by 'right for the
> work it does'... who needs a template that doesn't do what you need?

You are absolutely right - personally, I do not care one iota how fast
Django routes requests. I know that it does it fast enough, and that
improving the speed of that tiny subsection of code is not going to
improve the performance of my web application. This part of code makes
up a disproportionally small part of the request cycle, even if it was
twice as fast, the overall improvement would be negligible.

This is why I disagree with your conjecture that because Django is
slower than another framework, therefore Django is slow. That is not
what is shown, only that the other frameworks do certain things
faster.

>From your previous post, talking about
 urlresolver:

> I definitely would appreciate if someone from core team I guess or in the 
> community finally have a look there

Django's url resolver _is_ fast. It does a lot more work than the
other frameworks under test, which is why it is subjectively slower
than them in that test, but that is not relevant. It could be made
faster, if it stopped doing the useful features that we rely on to
build our projects.

Open source is wonderful, because you are allowed the choice of
framework. You can choose full featured, and accept that those
features will cost time, or you can choose lightweight, and not use
those features. There is no magical mid-way point where you get all
the features and all the speed.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Moonlight

>
> So, the benchmarks are interesting. They tell us which stacks are 
> fully featured, and which stacks are very lightweight. Apart from 
> that, they don't tell us much at all - is Django's template engine 
> slow, or is it about right for the work it does? This benchmark 
> doesn't tell us that, it only says it is slower than a bare bones 
> template engine, which is unsurprising, and shouldn't be a cause for 
> concern.
>
 
It is sort of no sense. Django is fully featured and we do not care... some 
other are fast because they are lightweight... tell me which 'full 
featured' feature prevents it from become better. I guess for the #1 
framework it is important to be leader... hmm... not sure what you mean by 
'right for the work it does'... who needs a template that doesn't do what 
you need?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/YAcwku7WouYJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Moonlight
Well I am not that good to get it fixed it django... it quite easy get 
lost there. But I definitely would appreciate if someone from core team I 
guess or in the community finally have a look there...

That said, if you really think Django's urlresolver is to slow, go ahead, 
> profile it and improve it, we certainly won't say no to speed improvements…
>
> Cheers,
> Florian
>

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/CZfjypvjpPQJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Tom Evans
On Thu, Oct 11, 2012 at 7:02 AM, Yo-Yo Ma  wrote:
> Why does every conversation about Django's performance met with "GTFO we 
> don't care"? (that was a rhetorical question :). I'd venture to guess that 
> most "It's fast enough for me!" responses are predicated on experiences that 
> can be likened to personal blog development, rather than large scale, 10+ 
> server deployments (e.g., Disqus, et al).

At $JOB, we used to write our websites in C++. No, really, we used to
write all our web apps as part of custom apache modules that used our
own C++ framework. If I compared our C++ url routing code against any
of those python stacks tested, it would absolutely murder them. Yet we
don't write new websites in C++ - why?

The expensive parts of a website are not routing requests, it is DB
queries and waiting to write responses to clients. When you look at it
like that, the speed of "grunt" parts of your framework is simply not
relevant. Django's template system is considerably faster than our
XSLT templates we used in C++ (that doesn't paint XSLT in a good
light).

So, the benchmarks are interesting. They tell us which stacks are
fully featured, and which stacks are very lightweight. Apart from
that, they don't tell us much at all - is Django's template engine
slow, or is it about right for the work it does? This benchmark
doesn't tell us that, it only says it is slower than a bare bones
template engine, which is unsurprising, and shouldn't be a cause for
concern.

Cheers

Tom

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Russell Keith-Magee
On Thu, Oct 11, 2012 at 2:02 PM, Yo-Yo Ma  wrote:
> Why does every conversation about Django's performance met with "GTFO we 
> don't care"? (that was a rhetorical question :).

Ok - If this is where the conversation is starting, it's going to go
downhill *very* fast.

Consider this a warning to *everyone* on this thread: we expect *all*
participants to maintain a civil and professional tone. Anyone
engaging in ad hominem attacks, straw man arguments, or any other sort
of offensive behaviour will be banned from posting. I'm not directing
this at anyone in particular - yet. But this is the second thread in a
week that has started down this path, and I want to make the rules
*absolutely* clear before anything gets out of control.

Back to the topic at hand:

We are interested in any concrete proposal that serves to improve
Django's performance.

However, we have no interest in optimisations for the sake of beating
some arbitrary benchmark score.

In addition, any claim that Django is "too slow" for practical
purposes is *demonstrably* false. There are *thousands* of sites in
production that are quite happily using Django. Many of them are
*very* large sites, serving *enormous* amounts of traffic. So, let's
dispense with the hyperbole that implies Django performance is so bad
it is only suitable for toy sites.

That said - could Django be faster? Sure. *Anything* can be improved.

So, if you've got a suggestion, make it. If you think URL resolving is
the source of the problem, propose a way to improve the speed of URL
resolvers. If you think the template language is the problem, propose
a fix.

And don't just say "Why are Django's URL resolvers slow?". Do some
profiling, and come back with an analysis of where the time is being
spent and/or wasted. Come up with a proposal for how Django's URL
resolvers could be made pluggable so that you can swap in a faster
resolver as an option.

We'll happily discuss concrete suggestions. We won't engage in fishing
expeditions.

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Florian Apolloner
Hi,

I won't answer your rheotircal questions ;)

On Thursday, October 11, 2012 8:02:19 AM UTC+2, Yo-Yo Ma wrote:
>
> BUT... Django is NOT that fast. 
>

We do know that (you know that and probably everyone else too) and I 
already said in my post that Django isn't the fastest framework out there.

The list goes on. What is the harm in discussing the weak points of Django? 
> Performance is probably the only major weak point in Django right now 
> (aside from the lack of schema migration in core, which is coming soon 
> anyway). 
>

There is no harm in discussing weak points in Django, we do it all the 
time. I just don't think that copy pasting a link (and that more than once) 
to a benchmark which is (imo) just there as a marketing strategy for 
another framework (note: I am not saying the benchmarks are a fake, I am 
just saying they probably aren't representive) adds real value to the 
discussion.

I have no solution or patches in my pocket, but I can say with absolute 
> certainty that performance will never, ever get better when discourse is in 
> a monologue format. 
>

I agree to some extend, but I think you shouldn't forget that many core 
devs use Django on somewhat large-scale deployments (read: definetely way 
over a blog) and performance issues are fixed as needed. But yes, as you 
said stuff like templates are probably slower than they should be :/

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/iVOJwI_v07kJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Alex Ogier
On Wed, Oct 10, 2012 at 6:29 PM, Florian Apolloner wrote:

> That said, if you really think Django's urlresolver is to slow, go ahead,
> profile it and improve it, we certainly won't say no to speed improvements…
>

That's not really fair. Django core *will* say no to speed improvements if
they necessitate any backwards incompatibilities, which due to early design
decisions they often will.

The main reason Django's url resolver is slow is that it was designed to
allow some very flexible patterns. Just like middleware is slow because it
is globally configured and called on every request. And the template
language is slow because it allows arbitrary textual replacements and has a
flexible import system that defeats compilation.

Django has committed itself to backwards compatibility, which means it
tends to get larger and heavier over time, and decisions that were made
long ago have lasting impact. Speed is a real feature even in python, but
Django has very consciously chosen to prioritize other things. I consider
it a valid argument to say, "Django cares more about developer productivity
and backwards compatibility than framework speed," but "Fix it yourself" is
a hackneyed open-source truism, and "Speed doesn't matter" is more or less
self-evidently false -- especially to someone evaluating frameworks, who
may not yet have a strong opinion on what features are game-changers for
his application.

Best,
Alex Ogier

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-11 Thread Yo-Yo Ma
Why does every conversation about Django's performance met with "GTFO we don't 
care"? (that was a rhetorical question :). I'd venture to guess that most "It's 
fast enough for me!" responses are predicated on experiences that can be 
likened to personal blog development, rather than large scale, 10+ server 
deployments (e.g., Disqus, et al). 

If you had great abs, nice pecs, ripped legs, a nice deltoids, and 4" 
circumference biceps, you'd *probably* start to care when people said, "hey, 
what about those biceps...", no?

Are CPUs, RAM, and electricity free? (that too was a rhetorical question :)

Django is an *epic* (not in the literal sense :) framework, which offers a 
tremendous amount of functionality with nary an inkling of logical restraint. 
You can accomplish basically anything with Django without running into 
arbitrary limitations (e.g., have a gander at the limitations in most 
frameworks' URL dispatchers, just as a relevant example). To shorten this, from 
a features / ease of use standpoint, Django is just plain awesome.

BUT... Django is NOT that fast.

1) Django templates are unbearably slow (doesn't matter for a blog or something 
simple, but try something with lots of inline model formsets in a page and 
quite a few includes, and before you know it, you're wasting a decent 
percentage of your CPU on templates).

2) URL dispatching

3) Middleware is applied liberally (vs the a la carté, more efficient decorator 
approach to AOS)

The list goes on. What is the harm in discussing the weak points of Django? 
Performance is probably the only major weak point in Django right now (aside 
from the lack of schema migration in core, which is coming soon anyway).

I have no solution or patches in my pocket, but I can say with absolute 
certainty that performance will never, ever get better when discourse is in a 
monologue format.

That's all.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/uZ9r1EtZnj4J.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-10 Thread Florian Apolloner
Oh cmon,

please stop playing a socket puppet for the wheezy.web author. A web 
framework consists of more than just a win in speed (an the author of 
wheezy.web can argue whatever he wants that "basic" stuff stuff should be 
fast) -- if we were looking for speed we wouldn't use python at all 
(period). The goal is to be fast enough and be able to scale out 
horizontally, everything else is not really interesting in the case of web 
development. While I won't deny that Django isn't the fastest framework out 
there it certainly gets it's job done more than good enough and that's all 
it matters. I don't know what your goal is, but please stop posting such 
meaningless comparisons on django-dev…

That said, if you really think Django's urlresolver is to slow, go ahead, 
profile it and improve it, we certainly won't say no to speed improvements…

Cheers,
Florian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/DgNcKfL2AeUJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-10 Thread Daniel Sokolowski
Yes, wheezy.web is much more bare metal compared to Django : 
http://packages.python.org/wheezy.web/tutorial.html

From: Daniel Sokolowski 
Sent: Wednesday, October 10, 2012 10:32 AM
To: django-developers@googlegroups.com 
Subject: Re: URL dispatcher slow?

The middlewares appear to be disabled and the test bypasses the template system 
too: 
https://bitbucket.org/akorn/helloworld/src/c3f2d44dfca7/02-routing/django/helloworld/settings.py?at=default
 but yes Aymeric is right that Django provides a lot of convenience by default. 
In my opinion I rather take convenience as speed has not been a issue to me,  
but one thing though is that these benchmarks are  transparent enough to give 
them some validity. 

From: Aymeric Augustin 
Sent: Wednesday, October 10, 2012 9:30 AM
To: django-developers@googlegroups.com 
Subject: Re: URL dispatcher slow?

2012/10/10 Moonlight <moonlight_13_...@yahoo.com>

  Here is an article comparing various URL dispatchers:

  http://mindref.blogspot.com/2012/10/python-web-routing-benchmark.html

  What cause django URL dispatcher that much... slow?

Django's URL dispatcher is more complex than others. It provides advances 
features such as reversing, automatic i18n, and namespaces. Of course, these 
come at the expense of speed.

Note that this benchmark doesn't test URL dispatchers; it tests full stack 
requests. With Django's default settings, lots of convenient features such as 
middleware are enabled. This is likely to skew the results.

It'd be interesting to analyze what parts of the URL resolver are slow. If you 
really want to know what going on, go ahead and profile it. I'd be interested 
in the results (even though Django's performance has absolutely never been a 
problem for me).

As a side note, I read these benchmarks with a grain of salt. They're developed 
by the author of wheezy.web and consistently show it as a clear winner. Where's 
the line between testing and marketing?

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Daniel Sokolowski
http://webdesign.danols.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-10 Thread Daniel Sokolowski
The middlewares appear to be disabled and the test bypasses the template system 
too: 
https://bitbucket.org/akorn/helloworld/src/c3f2d44dfca7/02-routing/django/helloworld/settings.py?at=default
 but yes Aymeric is right that Django provides a lot of convenience by default. 
In my opinion I rather take convenience as speed has not been a issue to me,  
but one thing though is that these benchmarks are  transparent enough to give 
them some validity. 

From: Aymeric Augustin 
Sent: Wednesday, October 10, 2012 9:30 AM
To: django-developers@googlegroups.com 
Subject: Re: URL dispatcher slow?

2012/10/10 Moonlight <moonlight_13_...@yahoo.com>

  Here is an article comparing various URL dispatchers:

  http://mindref.blogspot.com/2012/10/python-web-routing-benchmark.html

  What cause django URL dispatcher that much... slow?

Django's URL dispatcher is more complex than others. It provides advances 
features such as reversing, automatic i18n, and namespaces. Of course, these 
come at the expense of speed.

Note that this benchmark doesn't test URL dispatchers; it tests full stack 
requests. With Django's default settings, lots of convenient features such as 
middleware are enabled. This is likely to skew the results.

It'd be interesting to analyze what parts of the URL resolver are slow. If you 
really want to know what going on, go ahead and profile it. I'd be interested 
in the results (even though Django's performance has absolutely never been a 
problem for me).

As a side note, I read these benchmarks with a grain of salt. They're developed 
by the author of wheezy.web and consistently show it as a clear winner. Where's 
the line between testing and marketing?

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Daniel Sokolowski
http://webdesign.danols.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



Re: URL dispatcher slow?

2012-10-10 Thread Aymeric Augustin
2012/10/10 Moonlight <moonlight_13_...@yahoo.com>

> Here is an article comparing various URL dispatchers:
>
> http://mindref.blogspot.com/2012/10/python-web-routing-benchmark.html
>
> What cause django URL dispatcher that much... slow?


Django's URL dispatcher is more complex than others. It provides advances
features such as reversing, automatic i18n, and namespaces. Of course,
these come at the expense of speed.

Note that this benchmark doesn't test URL dispatchers; it tests full stack
requests. With Django's default settings, lots of convenient features such
as middleware are enabled. This is likely to skew the results.

It'd be interesting to analyze what parts of the URL resolver are slow. If
you really want to know what going on, go ahead and profile it. I'd be
interested in the results (even though Django's performance has absolutely
never been a problem for me).

As a side note, I read these benchmarks with a grain of salt. They're
developed by the author of wheezy.web and consistently show it as a clear
winner. Where's the line between testing and marketing?

-- 
Aymeric.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.



URL dispatcher slow?

2012-10-10 Thread Moonlight
Here is an article comparing various URL dispatchers:

http://mindref.blogspot.com/2012/10/python-web-routing-benchmark.html

What cause django URL dispatcher that much... slow?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/xJFGtYmJsBYJ.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.