Re: #6735 -- Class based generic views: call for comment
On Mon, 2010-10-04 at 19:37 +0200, Łukasz Rekucki wrote: > The only problem is decorators: You can't just simply apply > login_required() to the class or the dispatch() method. Otherwise, I > like this approach. It works fine: http://bitbucket.org/spookylukey/django-class-views-experiments/src/tip/classmethod.py#cl-40 (I have 'as_view' instead of 'dispatch'). My MyView.dispatch method could have decorators applied inside the actual class (as long as they are adjusted for being method decorators and not function decorators, in common with all other solutions). Luke -- "Doubt: In the battle between you and the world, bet on the world." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: #6735 -- Class based generic views: call for comment
On Mon, 2010-10-04 at 13:08 -0400, Alex Gaynor wrote:
> Given that the primary complain against the __new__ solution is that
> it's unintuitive to Python programmers that instantiating their class
> results in some other class, why not simply go with a more explicit
> classmethod. Simply used as:
>
> url("^articles/$", ArticlesView.dispatch).
>
> It seems to address all concerns: it's explicit, safe to assign
> attributes, Pythonic, and fails loudly if used improperly (using
> AritlcesView would result in an Exception being throw since it doesn't
> return an HttpResponse, and using, and ArticlesView() an error about
> not being callable).
>
> Last idea, I swear,
> Alex
>
I've put together a repos with a bunch of scripts that demonstrate in
about a page of code the essence of what we are trying to achieve
(without all the detail about the actual dispatch to HTTP verbs etc).
Start with naive.py:
http://bitbucket.org/spookylukey/django-class-views-experiments/src/tip/naive.py
(Just run as 'python naive.py' to see the output). Browse up a level to
see the other variants.
I implemented Alex's latest idea in classmethod.py (with 'as_view'
instead of 'dispatch', to avoid name clashes). It suffers from the same
limitation as the __new__ in that you can't pass configuration data to
the init. But you can do a 'configure' class method, as you can also do
with __new__
Personally, I now prefer these: first place:
http://bitbucket.org/spookylukey/django-class-views-experiments/src/tip/classmethod.py
Close second:
http://bitbucket.org/spookylukey/django-class-views-experiments/src/tip/magicnew_with_configure.py
Given the limitations, I'm no longer that bothered about the fact that
the __new__ method returns something completely unexpected, but Alex's
classmethod solution seems like a nice improvement on it, though it
makes a URLconf very slightly more verbose.
Luke
--
"Doubt: In the battle between you and the world, bet on the world."
(despair.com)
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: #6735 -- Class based generic views: call for comment
On Sat, 2010-10-02 at 09:20 -0700, George Sakkis wrote: > Having said that, I'm in favour of the first approach mentioned in the > wiki (store state on request) and I'm surprised it doesn't seem to > have any traction in this thread. The only argument against is "it's > unusual to have a class where you can't store state on self" but even > that can be overcome with an appropriate __setattr__/__getattr__ that > access self.request.state under the hood. Unless someone tries > self.__dict__[attr], it will look as if the state is stored on the > view although it's actually stored in the request (or better > request.state to avoid name clashes). Perhaps I am missing something > but this seems to me foolproof and much cleaner than the alternatives. This breaks if you set any values in __init__ - since the request object doesn't exist to store state on. Luke -- "DO NOT DISTURB. I'm disturbed enough already." Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: #6735 -- Class based generic views: call for comment
On Fri, 2010-10-01 at 17:31 +0100, Luke Plant wrote:
> Or, more drastically, we could just do this:
>
> class View(object):
> def __call__(self, request, *args, **kwargs):
> self.request = request
> self.args = args
> self.kwargs = kwargs
> resp = self.dispatch(request, *args, **kwargs)
> self.__dict__ = {} # goodbye state!
> return resp
>
Please ignore this suggestion, and my previous one, if you didn't
already; I don't know what I was thinking, because they don't address
the multi-threaded problem at all.
Luke
--
"DO NOT DISTURB. I'm disturbed enough already."
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Potential very minor security issue
On Fri, 2010-10-01 at 10:59 -0700, Yo-Yo Ma wrote: > Form wizard docs mention that MD5 hashes are made using > settings.SECRET_KEY - > http://docs.djangoproject.com/en/dev/ref/contrib/formtools/form-wizard/#django.contrib.formtools.wizard.FormWizard.security_hash > > If you give me a value, I give you a hash back, and you're aware that > I'm using Django (think BitBucket which everyone knows is using > Django), couldn't I use those values to learn your secret key. I > understand this would be difficult and time consuming to try to > accomplish, and I apologize if I'm raising unwarranted alarms, but I > thought it was worth mentioning. Um, in theory, yes, *all* cryptographic measures can be broken given some information and enough time. You somehow missed the part where you explained what kind of attack you would do, and showed roughly how long it would take to do this brute force attack, given the typical secret keys that a Django 'startproject' command generates, and why we should therefore be worried. (BTW, if you actually do this, and you discover there is a problem, the right place to report it is secur...@djangoproject.com, not here). > Also, why MD5? I thought Django was switching to SHA1 for security > reasons. If we switch we will break compatibility with existing hashes. We'd like to switch to HMAC-SHA1 (it's on my TODO list actually), but only if we can do so without causing lots of breakage. Luke -- "DO NOT DISTURB. I'm disturbed enough already." Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Sites Framework: RequestSite and get_current
On Fri, 2010-10-01 at 12:08 -0700, Gabriel Hurley wrote: > I can see why you might want control at the middleware level, but I'm > not sure that's the only/best way to accomplish that goal, and it > seems like the subject of a totally separate ticket to me. I'd rather > see this set of redundant problems solved in a simple fashion rather > than get into a debate over implementing a different feature... Absolutely right, go ahead with your patch. If someone really cares about the other feature (which seems a bit obscure to me), then they will do the normal things required. Thanks, Luke -- "DO NOT DISTURB. I'm disturbed enough already." Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: #6735 -- Class based generic views: call for comment
Or, more drastically, we could just do this:
class View(object):
def __call__(self, request, *args, **kwargs):
self.request = request
self.args = args
self.kwargs = kwargs
resp = self.dispatch(request, *args, **kwargs)
self.__dict__ = {} # goodbye state!
return resp
This would kill the ability to use __init__ to store anything on the
instance, since it would only work for the first request, and so would
enforce subclassing as the only means of customising behaviour. It would
be more effective at dealing with thread safety concerns than a
'copy()', but breaking __init__() like that is fairly evil. I'm not sure
if this is a serious suggestion or not...
Luke
--
"Despair: It's always darkest just before it goes pitch black."
(despair.com)
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: #6735 -- Class based generic views: call for comment
On Fri, 2010-10-01 at 12:16 +0200, Johannes Dollinger wrote: > Am 01.10.2010 um 07:26 schrieb Russell Keith-Magee: > > I've just added a summary of the last thread on class-based views > [1]. > > This summary isn't 100% complete -- any contributions from > interested > > parties are welcome. Try to keep opinions to a minimum; this page is > > about documenting the strengths and weaknesses of various > approaches, > > not about expressing opinions. In the same way that CSRF [2] and > > session-messages [3] have good wiki pages describing the design > > considerations, we need to be able to explain to future generations > > why class-based views are the way they are. > > Could you (or anyone knowledgable) add a section, that explains why > each request should have its own view instance? > The thread-safety argument alone does not suffice: if all _request_ > state would be stored on request instead of the view, you wouldn't > need new instances per request. You could also pass around state > explicitly - which admittedly gets messy quickly. > So is this purely about preventing users from shooting themselves in > the foot? (hyperbole: Will there be protection from globals in django > 1.4?) It's not just about stopping users from shooting themselves in the foot, it's about helping them to do the right thing easily. Without this kind of protection, it will be harder for *anyone*, including experienced developers who are aware of the problem, to do things correctly. It's like the autoescaping in templates - I *know* that I should use the 'escape' filter, but without autoescaping it is hard for anyone to do it right all the time. One alternative that has been suggested is to pass state around explicitly, which is definitely best practice, but in many cases might not be possible (e.g. if you are working with someone else's base class, and in one overridden method you want to set up some objects which ought to be available to other methods). You could also attach stateful data to the request object, but attaching random bits of data to the request is surely just as ugly a solution as the self.copy() call, especially if you need to avoid names clashes with all the other attributes attached to request. With regards to doing a shallow copy, it should be noted that at the point the copy is made, the only data attached to the object is data that has been attached in the constructor. It is of course possible that methods within the view might mutate such data, but it seems more likely that it will be used as immutable configuration data. However, I have already seen some example code that might fall foul of this problem - someone gave the example of storing a queryset on the object in the __init__. This will get implicitly mutated (the cache is filled) when it is used the first time. This could lead to frustrating problems that wouldn't be found in testing, and doing copy() on the instance won't help. So, in light of the fact that developers *will* need to be aware of this issue, I'd like to tentatively suggest an explicit solution which is nonetheless easy to use and get right. We could have an explicit 'state' object that is thrown away for every new request, something like this: class State(object): pass class View(object): def __call__(self, request, *args, **kwargs): """ Main entry point for a request-response process. """ self.state = State() self.request = request self.args = args self.kwargs = kwargs resp = self.dispatch(request, *args, **kwargs) # let any state get GC'd immediately: del self.state del self.request del self.args del self.kwargs return resp We document the issue, warn people not to store state on the instance itself, but tell them that if they must have stateful data, it should be attached to self.state instead of self, and they will be OK. They might still be bitten if they put mutable configuration data into the __init__ and it gets mutated, but we have made it easy to do the right thing - doing 'self.state.x' is only slightly harder than 'self.x' Thoughts? Luke -- "Despair: It's always darkest just before it goes pitch black." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Sites Framework: RequestSite and get_current
On Thu, 2010-09-30 at 23:48 -0700, Gabriel Hurley wrote: > I went to triage a few tickets tonight, and noticed that #8960, > #10235, #10608 and #13814 have all arrived at essentially the same > conclusion: there needs to be a single idiomatic way to get either the > current Site object if contrib.sites is installed, or a RequestSite > object if not. All four tickets use the same bit of code, the argument > really only lies in where to put it. > > #10235 adds it as a utility function in contrib.sites.models, #13814 > as a separate method on SiteManager, and #8960 and #10608 basically > just copy and paste where needed. I'm of the opinion it should have > its own home in contrib.sites, but where, and under what name? I think a stand-alone function in contrib.sites.models is fine, called `get_current_site` (or some other sensible colour of your choosing). Since both Site and RequestSite already live there, it seems a good place. Thanks for your work on this, Luke -- "Despair: It's always darkest just before it goes pitch black." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: #6735 -- Class based generic views: call for comment
On Fri, 2010-10-01 at 11:12 +0200, Hanne Moa wrote: > On 1 October 2010 07:26, Russell Keith-Magee wrote: > > I already have one specific piece of API feedback: the current > > implementation requires that all view logic is contained in GET(), > > POST() etc style views. This is fine, except for the fact that there > > are occasions where GET and POST share a lot of common logic. > > This! is! Python! Any shared logic can be in another method, or > function for that matter, and need not be canonicalized in an API. All > that would be needed is an example in the docs: > > class TheView(views.ClassView): > > def _common(self, bla): > # set up models, forms, other preprocessing here > # return models/forms or set them on class > # could be run from __init__ or __new__ or whatever we wind up with > > def GET(self, bla): > self._common(bla) > > def POST(self, bla): > self._common(bla) Passing things around between '_common' and GET and POST makes a simple view much more complex than it needs to be, especially when you have various local variables that you now have to assign in some way. In the end you will end up just routing it all to the one method: def GET(self, *args, **kwargs): return self._common(*args, **kwargs) def POST(self, *args, **kwargs): return self._common(*args, **kwargs) This is just 4 lines of annoying boilerplate to undo the dispatching that was 'helpfully' done for you. I would definitely support a 'request' method (perhaps call it 'dispatch'?) that allows us to avoid that logic. I suspect that every view I've created that used a form would be hampered by having dispatch based on HTTP verb done first. Luke -- "Despair: It's always darkest just before it goes pitch black." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Need opinion for an object permission rule backend
On Thu, 2010-09-30 at 13:29 +0200, Miguel Araujo wrote: > Hi everyone, > > > I have been recently thinking about an object permission system. After > reviewing Florian Apolloner (apollo13) patch for ticket #11010 and > reading his article at Django Advent. I though about creating an > Object Permission Rule Backend. The purpose of this message is explain > you my idea, so I can receive feedback from Django developers with > better understanding of the internals of the Framework. This way I > would like to discern if it's worth coding it or if it's a good > approach to a reusable solution. Since you are not suggesting something for inclusion in Django itself, this is not the right mailing list for this kind of post. You don't need the opinion of the core Django developers for this - the publicly documented APIs and normal Python/programming practices give you everything you need. Please ask on django-users. Thanks, Luke -- "Despair: It's always darkest just before it goes pitch black." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: ANN: Improving our decision-making and committer process
On Thu, 2010-09-30 at 01:32 +0400, Ivan Sagalaev wrote: > My suggestion is about this unfortunate ticket status -- 'Accepted'. > This now works as a sort of a dusty shelf: when anyone of the core team > looks at the patch and decides that there's nothing wrong with it he > puts it on that shelf where the ticket has all the chances to lie for > months or even years. And if the author of the patch tries from time to > time pitching it to get it into trunk he can easily fall into all sorts > of "not-a-good-times": conferences, feature freezes, hot discussions on > other topics etc. > > My proposal is simple: 'Accepted' status shouldn't exist. If the patch > is good it should be committed right away. If it's not there have to be > an explanation why it needs improvement or why it's going to be > wontfixed. Simple waiting doesn't really improve quality of the patch. > > What do you think? This doesn't account for these facts: 1) Accepted != Ready for checkin. 2) A lot of triage is done by people without commit access. Sometimes a ticket stays at 'Accepted' for a long time because it doesn't actually have anyone motivated enough to write even a patch, or tests etc, which means that it is de-facto low priority, and we shouldn't feel guilty about this kind. The ones in the 'Ready for checkin' queue are the ones that deserve to be checked in, and it currently has only 35 in it, compared to 1226 in 'Accepted'. However, there are many 'Accepted' tickets which actually ought to be in 'Ready for checkin', or need some feedback to say what should be done to move them on. This is not something that can only be done by committers, but how to get help with this massive task is still a big problem. It's quite possible for someone using Django to submit a completely valid bug report, with a good patch and tests etc., and for it not to get reviewed because no-one else in the community cares about that particular bug, and I for one don't know what to do about that. The core team do agree that the "it's not a good time" response has been an unhelpful sticking point, and we probably need to make it easier for people to discuss things at any point in time. Luke -- "Defeat: For every winner, there are dozens of losers. Odds are you're one of them." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: #12012 Logging: request for comments
On Sat, 2010-09-25 at 14:16 +0800, Russell Keith-Magee wrote:
> * The default logging configuration. Have I got the
> propagate/override options right for sensible defaults (both in global
> and new-project settings)?
I just noticed that the project template settings and the global
settings are different. The global settings use a null handler for debug
logs, while the project template config would send them to stderr. I
can see the value of having debug calls appear on the console while
running the development server, but most people will take the template
settings.py as a good starting point for deployment. Since we have
already defined DEBUG in the template settings.py, how about changing
the project template to have this:
LOGGING = {
'version': 1,
'disable_existing_loggers': False,
'handlers': {
'null': {
'level':'DEBUG',
'class':'django.utils.log.NullHandler',
},
'console':{
'level':'DEBUG',
'class':'logging.StreamHandler',
},
'mail_admins': {
'level': 'ERROR',
'class': 'django.utils.log.AdminEmailHandler'
}
},
'loggers': {
'django': {
'handlers':[DEBUG and 'console' or 'null'],
'propagate': True,
'level':'INFO',
},
'django.request':{
'handlers': ['mail_admins'],
'level': 'ERROR',
'propagate': True,
},
}
}
That will give a settings.py that is sensible for both development and
deployment.
Luke
--
"Cross country skiing is great if you live in a small country."
(Steven Wright)
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Changes to LazySettings
Hi all, Ticket #14297 [1] suggests changes to LazySettings for performance improvements. The best solution AFAIC is that __getattr__ triggers a setup routine on first access, which loads *all* the settings into the LazySetting object itself. This eliminates the __getattr__ overhead for all subsequent retrieval of settings (apart from when the setting doesn't exist, in which case you still get a __getattr__ call). This should be a fully backwards compatible change. There were some test failures, but these were all caused by test code relying on the internal implementation of LazySettings, i.e. the '_wrapped' object attribute (when in fact there was no need to do this after r11824 [2]) But can anyone else think of any gotchas with this change before I commit it? It produces a 30% improvement for the benchmark that relates to the ticket [3]. Thanks, Luke [1] http://code.djangoproject.com/ticket/14297 [2] http://code.djangoproject.com/changeset/11824 [3] http://github.com/spookylukey/djangobench/tree/master/djangobench/benchmarks/l10n_render/ -- "Christ Jesus came in to the world to save sinners" (1 Timothy 1:15) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: proposal for lazy foreignkeys
On Mon, 2010-09-27 at 16:46 +0100, Luke Plant wrote: > Some notes: > 1) it seems like line 15 in abstract.py should say 'abstract':'False', > not 'True' - did I miss something? Cancel that one, I was missing the fact that we are still inheriting from the generated class, not doing: MyCategory = CategoryFactory.construct() Thanks, Luke -- "Christ Jesus came in to the world to save sinners" (1 Timothy 1:15) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: proposal for lazy foreignkeys
On Mon, 2010-09-27 at 11:36 +0200, Patryk Zawadzki wrote: > > With the risk of being ignored once again, I dare to link to a working > solution that does not need any changed to the framework itself (other > than perhaps including the factory class): > > http://gist.github.com/584106 This looks rather good to me. It may have been ignored before because it has no comments and some things are not immediately obvious. For example, you are basically proposing that the concrete models are passed into view functions via URLconf, and from there are passed into any functions which need them, and so they would never actually need to be imported by the app that defines the abstract model. I for one would be much happier to not add any more machinery via Meta options. With some cleanup, and some documentation of this pattern, and possibly a better name, I think the AbstractMixin class you propose could be a good candidate for inclusion in core. Some notes: 1) it seems like line 15 in abstract.py should say 'abstract':'False', not 'True' - did I miss something? 2) there would need to be some way of merging the concrete class's own Meta options with the abstract class's Meta options 3) why do we need the _classcache? Is the key used specific enough - what if two different apps both create 'MyCategory' based on 'CategoryFactory', using them in different situations? Thanks, Luke -- "Christ Jesus came in to the world to save sinners" (1 Timothy 1:15) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: #12012 Logging: request for comments
Hi Russell, On Sat, 2010-09-25 at 14:16 +0800, Russell Keith-Magee wrote: > At this point, I'm calling for feedback, particularly on the following: > > * logging config as the last stage of settings loading. Is this the > right place? Can anyone think of a better place? In the context of running a Django app, this means it will definitely get called before the first request is dispatched, due to the need to find the URLconf. From the context of command line apps, it is less predictable in general, but I guess the command line app has to be responsible for ensuring that logging gets set up first. Anyway, this seems good to me, and I can't think of a better place. For the probably rare case of putting logging calls in your settings.py (like Hanne Moa does), there is nothing to stop you setting up logging yourself inside settings.py, which Hanne must presumably do already. The only problem is that logging could be set up twice. For this case, I guess we could support some convention like 'LOGGING_CONFIG = None' to stop Django configuring logging altogether and add a note in the docs. This might also be useful for the command line app situation. > * The default logging configuration. Have I got the > propagate/override options right for sensible defaults (both in global > and new-project settings)? I don't really have much experience with the logging module, but they seemed sensible to me. > * The logging calls that have been introduced. Is this too much or > too little? I'd rather err on the side of caution here (we can always > add extra logging later), but I'd like suggestions on anything else > people think should be logged. Also, are they logged at the right > level (e.g., is a 404 a warning?). These all look good to me, and cover the obvious cases. Warnings for 4XX codes seems good, since they could be a sign of something "going wrong" - dead links or some kind of attack - while not necessarily being an actual problem in the code, and it is useful to filter that category. > * The startup.py convention (and implementation). Is this necessary? > Does it address (or not address) any other prominent "on startup" use > cases? There's also a crossover here with Arthur's GSoC work; is the > startup code sufficiently important that we need it anyway, or can we > defer until the app refactor lands? I'm not sure what use cases this specifically targets. It seems that it depends on 'AppCache._populate' being called. From what I can tell, that isn't guaranteed to be called until some database query is made, or it is called by an app like the admin. Without the admin I can create an app that never populates AppCache but can serve responses. I don't see anything that depends on this startup proposal though, so I'd propose dropping that part until the other options have been discussed. Regards, Luke -- "Christ Jesus came in to the world to save sinners" (1 Timothy 1:15) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: #717 (If-Modified-Since checked for exact match) should be fixed
On Tue, 2010-09-21 at 11:40 -0700, julianb wrote: > Uuh, so what's with this? Nothing happened with the ticket and no > further comments here either... :( I accepted it on Trac, please see my comments there. Also, you mentioned about the 'condition decorator' that also needs updating - I'm not sure what you meant, but if that isn't the same code path, then it also needs to be updated (without duplicating the logic - not a copy/paste job). Cheers! Luke -- As Ralph Waldo Emerson once said, "I hate quotations." Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: [RFC] HttpResponse: freeze API, read_once
Hi Forest, you wrote: > > I'd like to return to the problem of middleware, view decorators, etc. > affecting > responses in negative ways. Thanks so much for all your hard work and persistence on this issue. Although I haven't gone through this in detail, I think this is arriving at a good solution now. I massively prefer the explicit 'freeze' API on HttpResponse to subclasses that do different things. It would be extremely useful to me if you could go through the use cases linked on the previous thread and see if your solution addressed them all. > I am more than happy to provide an implementation of the above, but I'd like > to > get an idea of what chance it has of being accepted before I start working on > it. I am also happy to maintain it outside of Django svn for a while so that > people can test it out. >From my perspective that would be great. One thing that would be brilliant whatever the eventual solution would be a set of use cases encoded into some tests - I suspect that whatever solution/API is chosen it would be worth doing the tests first so that you can see if it is going to work in practice, and so that other people can see how to use it. Many thanks. Luke -- When you're great, people often mistake candor for bragging. -- Calvin and Hobbes Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Have a submission to enforce password policy in the administrative interface
On Sat, 2010-09-11 at 08:27 -0700, dremspider wrote: > Django developers, >Unless I am missing something I haven't found a good way to enforce > password policies (minimum length, number of UPPER and lower case, > etc). I need something that enforced these settings as we have > password requirement policies to adhere to. I modified django/ > contrib/auth/forms.py to support this ability. The user sets in their > settings.py: > > MIN_PASSWORD_LENGTH=11 > MIN_NUM_LOWERCASE=2 > MIN_NUM_UPPERCASE=2 > MIN_NUM_SYMBOLS=2 Thanks so much for your interest in contributing to Django. Here are my suggestions - I'm afraid there are a number of them and quite big, but don't be discouraged! 1) In general, we are trying not to add more settings to Django, because they almost invariably cause problems (like when you want more than one instance of a component which behave in different ways). For example, in your patch, the admin form has the same requirements as the other forms, but that might not fit some people's requirements. Parameters to functions/classes are much more useful in general. 2) When we add something to core, we want to be sure that it will be as useful as possible to a very large number of people. These particular settings are quite good for *your* password policies, but as you recognise in what you said, there are lots of things that these *don't* cater for, like checking against a dictionary. These requirements are likely to be very custom in nature, and having settings for each one simply does not scale. So, what we don't want to do is add settings, or even keyword arguments to functions/classes, for every conceivable password requirement. Rather, a single keyword argument would do - you would specify a callable that will check whether the password meets requirements. In fact, in this case we already have a mechanism for extending the behaviour of a form - subclassing the form. And you can also subclass UserAdmin and change the 'change_password_form', and then swap out the existing UserAdmin for your own one e.g. see http://stackoverflow.com/questions/471550/customizing-an-admin-form-in-django-while-also-using-autodiscover So, all the hooks to add your requirements are in Django already, so the odds are stacked against getting specific support for this into Django. It's difficult to come up with ways of making these hooks easier to use without introducing settings, which is something we will be very reluctant to do. Once you've worked these out, I think the best way to contribute to the Django community would be to put your results up on djangosnippets.org 3) Unified diffs (diff -u) are much easier to read than context diffs, and usually they are much better attached to the e-mail (or put on a Trac ticker) rather than inline. 4) The way you have implemented verifyPasswordComplexity could be improved a lot: 1. Using a class is strange when it is basically just a function, and has no benefit to being a class. 2. You throw 'Exception' and then have to catch it, which will also catch and hide any other errors, such as programming errors. 3. Counting lower case letters could be done much more easily e.g. len([c for c in password if c.islower()]) Regards, Luke -- He who is in love with himself has at least this advantage -- he won't encounter many rivals. (Georg Lichtenberg) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: A prompt 1.2.3 release
On Thu, 2010-09-09 at 18:47 -0700, Paul McMillan wrote: > > Unfortunately we didn't catch this failure in four weeks, the fact > > that the buildbot > > isn't running tests for the 1.2.X branch doeesn't help either. > > The buildbot actually is running the 1.2.x tests, it is just > unfortunately named (and apparently re-naming it is a bit of a pain so > it hasn't happened yet). Look at the actual logs of any given build to > see what's actually checked out. Which buildbot? http://buildbot.djangoproject.com/waterfall doesn't seem to be doing 1.2.X - from the logs for the ones marked 1.1.X: /usr/bin/svn checkout --revision 13706 --non-interactive --no-auth-cache http://code.djangoproject.com/svn/django/branches/releases/1.1.X build I can't find one that is checking out branches/releases/1.2.X There was a hudson server running IIRC, but http://hudson.djangoproject.com/ is not responding to me. Luke -- There was a young man from Peru Whose limericks stopped at line two. Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
A prompt 1.2.3 release
Hi all, It looks like we'll be needing a fairly prompt 1.2.3 release. So far we've got: One major/critical bug (depending on how many people are using the deprecated CsrfResponseMiddleware): http://code.djangoproject.com/ticket/14235 One significant regression: http://code.djangoproject.com/ticket/14242 Packaging error(s): http://code.djangoproject.com/ticket/14245 Many core devs are at Djangocon and are just starting sprints, I'm not sure whether that will help or hinder. Luke -- A common mistake that people make when trying to design something completely foolproof is to underestimate the ingenuity of complete fools. -- Douglas Adams Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CSRF Middleware/SSL/Firefox 3.6.8 bug
On Wed, 2010-09-08 at 10:22 +0200, Patryk Zawadzki wrote: > Again, it's not there to counter attacks. Think of it as the > equivalent of the "csrftoken" cookie which could be read in the exact > same way. I just wanted a couple of strings that are not likely to > change between form generation and submission. The csrftoken cookie *cannot* be read in this situation by an attacker, since it is a cookie and will *not* be sent to the third party site, so they cannot create a form with a csrftoken which matches the user's cookie. (We are not talking about a MitM attack here, just a standard third party site that your browse to). Luke -- A mosquito cried out in pain: "A chemist has poisoned my brain!" The cause of his sorrow was para-dichloro- diphenyltrichloroethane Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CSRF Middleware/SSL/Firefox 3.6.8 bug
On Tue, 2010-09-07 at 03:31 +0100, Luke Plant wrote: > 1) The attacker can have the same (public) IP address as the victim if > they are both behind the same firewall, and this will in fact be one of > the most practical ways to launch a MitM + CSRF attack on HTTPS. Here and in my other message I meant 'NAT router', not 'firewall' (although they might be the same thing). Sorry for any confusion. Luke -- Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CSRF Middleware/SSL/Firefox 3.6.8 bug
On Mon, 2010-09-06 at 22:39 +0200, Patryk Zawadzki wrote:
> Another approach would be not to use a cookie at all. For each {%
> csrf_token %} use a slightly modified variant of the above
> encode_cookie function with:
>
> values = {
> 'host: request.META['HTTP_HOST'],
> 'scheme': request.is_secure(),
> 'user_ip': request.META['REMOTE_ADDR'],
> 'user_agent': request.META['HTTP_USER_AGENT'],
> 'ttl': time.time() + 30*60,
> }
>
> Then when handling a POST request, decipher the token and compare each
> META field with the ones from the request and validate ttl against
> time.time(). I believe it's not less secure than the current
> implementation and solves two problems:
>
> 1) each form served gets its own ttl, an attacker can't keep pinging
> the server to keep the token alive
> 2) each token serves for a single use and will inevitably timeout in
> 30 minutes while still allowing you to open two forms in two browser
> tabs and submit each of them separately
Your method is quite flawed:
1) Use of IP address - a bad idea for the reasons I mentioned in my
other message.
2) The use of user agent does nothing to stop an attacker, even for an
attacker who isn't a MitM:
Consider an attacker who lures you to his site e.g.:
http://evil.com/somepage.php
somepage.php can read your user agent, and make a request to
http://target.com/some-page-with-form/ with the same user agent to get a
valid CSRF token. (If he is behind the same firewall as you, he will
automatically have the same public IP, so adding the IP address is not a
perfect cure for this, even if we could do it, which we can't).
somepage.php then returns a page which has a form which targets
http://target.com/some-page-with-form/ and includes the CSRF token. The
token is valid and unused, and the attacker can proceed.
I don't think that adding a timeout will really ever help with CSRF.
The nature of CSRF attacks means that the attacker is massively more
likely to be able to attack 'now' (which is within a few seconds, the
time for a few HTTP requests to complete) rather than 'later', so
timeouts just don't help. In addition, they add nuisance for the user -
it's quite possible for someone to leave their machine for 30 minutes
and come back to it and want to carry on what they were doing.
Luke
--
A mosquito cried out in pain:
"A chemist has poisoned my brain!"
The cause of his sorrow
was para-dichloro-
diphenyltrichloroethane
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: CSRF Middleware/SSL/Firefox 3.6.8 bug
On Mon, 2010-09-06 at 21:11 +0200, Patryk Zawadzki wrote: > See above, I'm concerned about the case when the attacker is able to > perform a replay attack. If you validate CSRF against the client's IP > (and possibly against a TTL field¹), you effectively prevent that type > of attack. If the CSRF tokens were stored in the database (much like > sessions are), you'd gain additional security of all tokens expiring > after one use (or TTL reached). Replay attacks? 1) You didn't mention this before 2) This is out of scope for what the CSRF defences are about 3) SSL protects us: http://stason.org/TULARC/security/ssl-talk/4-1-Does-SSL-protect-users-from-replay-attack-by-eavesdropp.html If you are talking about the session cookie being stolen because it is leaked over HTTP, that again is out of scope for CSRF - it can only be addressed by using a secure cookie (SESSION_COOKIE_SECURE). > > A MitM cannot tamper with requests that are sent over SSL. The only > > exception to this is that if HTTP -> HTTPS POST requests are allowed, > > the MitM can indeed generate any request they like and get the user's > > browser to send it, with potentially damaging consequences. > > The MitM can also trick the user into POSTing same data over HTTP and > forge a HTTPS request of his own. That's why I propose using a > separate, secure CSRF cookie (using a different name) for HTTPS > requests. It doesn't matter what the name of the cookie is, the attacker can still set a cookie with that name. > > To stop > > this we, we simply refuse HTTP -> HTTPS POST requests. In the context of > > an HTTPS request from a browser, we can indeed trust the Referer header > > since no-one can tamper with it in transit, and the MitM will not be > > able to convincingly send the user's browser a page that looks like it > > comes from https://example.com. > > I believe you can use mere cookies to prevent this (see above) and > keep the site working for people who don't and can't send Referer > headers. I don't think so, for reasons given below. > > ¹ Consider the following example. It was based on the code I wrote to > handle the new Facebook API so it might be a bit of an overkill but it > shows it's possible. A simpler solution would be to return (TTL + ':' > + md5(token + user_ip + secret_key + TTL)). > You are proposing adding IP address as a means of identifying a user, but that is a bad idea on two counts: 1) The attacker can have the same (public) IP address as the victim if they are both behind the same firewall, and this will in fact be one of the most practical ways to launch a MitM + CSRF attack on HTTPS. 2) There is no guarantee that a user has a single IP addresses for the duration of his session. My mobile phone's IP address can change quite quickly. Once you've removed the IP address from the equation, the only barrier you are adding is a TTL. However, that is a very small barrier - any view that returns a POST form can easily be parsed to retrieve a valid CSRF token - we hand out valid CSRF tokens very easily, whether over HTTP or HTTPS. The attacker can use a token retrieved in that way. There is nothing to connect the CSRF token to the user, and this is *by design*, because: 1) In Django we have no way to identify a user. We don't want to rely on the session framework, because a site might not be using it. 2) Even if a site is using the session framework, we've had issues in the past with stale CSRF tokens caused by session cycling. Almost everything I've said to this point is explained in the document I linked before. Luke -- A mosquito cried out in pain: "A chemist has poisoned my brain!" The cause of his sorrow was para-dichloro- diphenyltrichloroethane Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CSRF Middleware/SSL/Firefox 3.6.8 bug
On Mon, 2010-09-06 at 18:46 +0200, Patryk Zawadzki wrote: > On Mon, Sep 6, 2010 at 6:34 PM, Luke Plant wrote: > > OK, that does it. I call 'troll'. If you're not trolling, my > > apologies, but I have run out of energy trying to explain this to you. > > I'm still waiting for you to explain anything. > You said you were afraid of a third-party injecting cookies over HTTP > that would be then sent over HTTPS. > > I replied by telling you that currently it does not have to do any > injecting as it can just forge both the cookie and the token when > doing a malicious POST (attaching any other cookies and post data > needed to DoEvil™) -- current CSRF implementation does not stop such > attacks at all. > > Moreover I proposed a more secure cookie header that is both immune to > stealing (as it's calculated and checked using your IP address) and > forging (as you no longer ask the client to send you the same string > twice, both post payload and headers being in plaintext). > > The curl example was there to illustrate the problem of the current > implementation: a malicious client or proxy is free to replace both > the cookie and post payload and CSRF will validate. If I can capture a > HTTP request (the MitM case) and read the headers, it's just a matter > of taking the cookies, replacing the CSRF token cookie with "foo" and > I am free to make any request on the site I want for as long as I > want. Requiring me to send a "Referer" header is a mild inconvenience > at most (just send "https://domain.com/";). Almost everything in this discussion and all your misconceptions are covered here: http://code.djangoproject.com/wiki/CsrfProtection For HTTP (not HTTPS), MitM attacks are out of scope for our CSRF protection, because a MitM sees everything in the clear and will easily be able to defeat any defences we put up. Your suggested alterations do nothing to stop that. In the context of HTTPS, the Referer header check does indeed add a necessary and effective protection. The nature of a CSRF attack is that the attacker has to gets the *user's* browser to make the request, not the attacker's. The curl command line says nothing at all - I'm well aware that the attacker can do that from their own machine if they want to, but it won't do them any good! I could not understand how you thought that showing a curl command line is at all relevant. A MitM cannot tamper with requests that are sent over SSL. The only exception to this is that if HTTP -> HTTPS POST requests are allowed, the MitM can indeed generate any request they like and get the user's browser to send it, with potentially damaging consequences. To stop this we, we simply refuse HTTP -> HTTPS POST requests. In the context of an HTTPS request from a browser, we can indeed trust the Referer header since no-one can tamper with it in transit, and the MitM will not be able to convincingly send the user's browser a page that looks like it comes from https://example.com. Please distinguish between the HTTP/HTTPS context if you've still got questions. Luke -- A mosquito cried out in pain: "A chemist has poisoned my brain!" The cause of his sorrow was para-dichloro- diphenyltrichloroethane Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CSRF Middleware/SSL/Firefox 3.6.8 bug
On Mon, 2010-09-06 at 18:14 +0200, Patryk Zawadzki wrote:
> On Mon, Sep 6, 2010 at 5:56 PM, Patryk Zawadzki wrote:
> > In fact current implementation is weaker than the old one. You are now
> > depending on the client delivering the challenge and the response,
> > both being sent over HTTP. An attacker no longer has to steal a valid
> > session cookie, it's enough to pick any string ("foo" will suffice)
> > and send it both in cookie headers and in POST payload.
>
> Mandatory demo attack:
>
> $ curl --cookie \
> csrftoken=foo \
> --form csrfmiddlewaretoken=foo \
> --referer https://example.com/innocent
> https://example.com/do/something/evil
>
> :)
OK, that does it. I call 'troll'. If you're not trolling, my
apologies, but I have run out of energy trying to explain this to you.
Luke
--
A mosquito cried out in pain:
"A chemist has poisoned my brain!"
The cause of his sorrow
was para-dichloro-
diphenyltrichloroethane
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: CSRF Middleware/SSL/Firefox 3.6.8 bug
On Mon, 2010-09-06 at 16:42 +0200, Patryk Zawadzki wrote: > If you use a separate *secure* cookie for CSRF over SSL then it's not > possible to submit data from an unsafe page to a safe processor. > Secure cookies are only sent to SSL targets so it's not possible to > intercept the cookie by any means. No - the 'secure' flag for cookies only stops the cookie being sent over a non-secure connection. It *cannot* be used to stop them being *set* over non-secure connections, which is what we need. And cookies without the secure flag will still be sent over a secure connection. And, once again, this is not about the cookie being intercepted, it is about it being *set* over an HTTP connection and then being *sent* over HTTPS. Luke -- A mosquito cried out in pain: "A chemist has poisoned my brain!" The cause of his sorrow was para-dichloro- diphenyltrichloroethane Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CSRF Middleware/SSL/Firefox 3.6.8 bug
On Sun, 2010-09-05 at 19:49 +0200, Patryk Zawadzki wrote: > As for the vulnerability -- it's only there if you implement it > yourself. If you send the initial login form over SSL (we do it this > way for various reasons), the cookies are never prone to be > intercepted. No, the vulnerability we are talking about is there even if *you* don't send the forms over HTTP, because the MitM can insert the forms themselves (and send a matching CSRF cookie) into any page served over HTTP, and have those forms target HTTPS connections, and submit them automatically by javascript. It's not about cookies being intercepted, it's about them being set by a MitM over HTTP. There may be mitigating factors that limit the damage (like if you have sent your session cookies with 'secure only'), but our CSRF protection aims to be generic and not depend on those details, and also to provide protection against login CSRF (even if you are not using Django's session framework). Luke -- A mosquito cried out in pain: "A chemist has poisoned my brain!" The cause of his sorrow was para-dichloro- diphenyltrichloroethane Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CSRF Middleware/SSL/Firefox 3.6.8 bug
On Fri, 2010-09-03 at 12:56 -0700, Paul McMillan wrote: > I've only a small voice in this matter, but I'd like to chime in on > the side of figuring out a way to solve this without requiring the > Referer header. > > Regardless of what the RFC may say, I know from inspecting my logs > that I (and I assume many others) get a lot of traffic without the > header. The traffic in question isn't hand-entered URLS. People who > operate sites that appeal to certain subset of the internet population > see much higher instances of this. On some sites, my numbers show that > nearly 15% of my visitors never send a Referer. Barth, Jackson and Mitchell [1] collected some data that said that for same-domain HTTPS POST requests, the header is missing in only 0.05% to 0.22% of cases. They've also got strong evidence that the header is suppressed in the network, not by the browser. If you've got very different data, for same-domain HTTPS POST requests, and *not* including other types of requests, because they are not relevant, then we will probably have to re-evaluate. Would you be able to collect such data? Thanks, Luke [1] http://www.adambarth.com/papers/2008/barth-jackson-mitchell-b.pdf -- A mosquito cried out in pain: "A chemist has poisoned my brain!" The cause of his sorrow was para-dichloro- diphenyltrichloroethane Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CSRF Middleware/SSL/Firefox 3.6.8 bug
On Fri, 2010-09-03 at 17:26 +0100, Tom Evans wrote: > On Fri, Sep 3, 2010 at 4:02 PM, Luke Plant wrote: > > Hi Tom, > > > > On Thu, 2010-09-02 at 11:34 +0100, Tom Evans wrote: > > > >> Any sort of Referer checking is broken by design, since that header is > >> clearly optional. RFC 2616 makes explicitly clear that applications > >> should not rely on Referer being sent. Any argument that sending it > >> "is virtually never a privacy concern" is moot; you must not rely on > >> the existence of that header, even if it is a useful tool in 99% of > >> cases. > > > > Thanks for your input. I've got 3 main responses: > > > > 1) Lots of things are "broken by design" on the internet. The fact that > > HTTPS and HTTP cookies are not completely segregated from each other in > > browsers is one example we've mentioned recently - it is completely > > brain dead, and punches a huge hole in the security of many HTTPS > > applications. > > > > In fact, it is difficult to determine exactly whether it is "by design" > > or not since there is essentially no spec for cookies. In theory, we > > should be using RFC 2109, but no-one else is really doing that - they > > are using something half way between 'Netscape' cookies and RFC 2109. > > Actually, we all ought to be using RFC 2965, which advocates Set-Cookie2 > > which fixes and obsoletes 2109, but virtually no-one is doing that > > either. > > I agree; cookies are poorly specified and have issues. I don't see the > relevance to this though. I'm saying that RFCs are not specs, they are not perfect, and they rarely followed perfectly in practice. While I think we should follow relevant RFCs as closely as we can, we should bear in mind the definition of SHOULD NOT: 4. SHOULD NOT This phrase, or the phrase "NOT RECOMMENDED" mean that there may exist valid reasons in particular circumstances when the particular behavior is acceptable or even useful, but the full implications should be understood and the case carefully weighed before implementing any behavior described with this label. > Relying on it's existence allows for unpredictable behaviour, and > allows the decisions made by some sys admin to deny our clients access > to our sites, which never looks good. I don't want ever to have to > explain that FooGlobal clients couldn't log in because their sysadmin > doesn't like Referer headers, and our chosen framework can't handle > that. That is pretty unlikely to happen. Over HTTPS, Referer headers cannot be suppressed over the network, only via browser settings. Of course the sysadmin might install some browser plugin/default settings for Firefox, but even on Windows the user will usually be able to change their own browser settings. The docs for the relevant setting in Firefox [1] clearly say that for best compatibility it should be left at its default setting. But if sysadmins are installing browser mods like that, then there are lots of other ways that they could block access to a Django site. [1] http://kb.mozillazine.org/Network.http.sendRefererHeader > > In fact, a note in section 13.5.2 or RFC 2616 says: > > > > Warning: unnecessary modification of end-to-end headers might > > cause authentication failures if stronger authentication > > mechanisms are introduced in later versions of HTTP. Such > > authentication mechanisms MAY rely on the values of header fields > > not listed here. > > > > Which I think gives us explicit permission to do what we are doing. > > > > Are you sure? 13.5 discusses how transparent caches may reconstruct > responses from what they have cached; the section you have quoted from > 13.5.2 is talking about the issues that may arise with proxies that > modify/remove headers, so I don't think it readily applies in this > scenario. > > Also, unless I've very much missed something, there have been no later > versions of HTTP. The section it is in doesn't really matter - the *reason* it gives for not altering end-to-end headers (which includes the Referer header) for future compatibility applies generally. I know that a later version of HTTP has not arrived, but things like HTTP evolve gradually, and often standardise real world implementations. All I'm saying is that the RFC is aware that future needs may promote currently optional headers into something more important. I think that using the Referer header the way we are doing is far from ideal, but we have carefully weighed the consequences, and this seems to be the best of a less than ideal set of options. BTW, you might be interested in
Re: CSRF Middleware/SSL/Firefox 3.6.8 bug
Hi Tom, On Thu, 2010-09-02 at 11:34 +0100, Tom Evans wrote: > Any sort of Referer checking is broken by design, since that header is > clearly optional. RFC 2616 makes explicitly clear that applications > should not rely on Referer being sent. Any argument that sending it > "is virtually never a privacy concern" is moot; you must not rely on > the existence of that header, even if it is a useful tool in 99% of > cases. Thanks for your input. I've got 3 main responses: 1) Lots of things are "broken by design" on the internet. The fact that HTTPS and HTTP cookies are not completely segregated from each other in browsers is one example we've mentioned recently - it is completely brain dead, and punches a huge hole in the security of many HTTPS applications. In fact, it is difficult to determine exactly whether it is "by design" or not since there is essentially no spec for cookies. In theory, we should be using RFC 2109, but no-one else is really doing that - they are using something half way between 'Netscape' cookies and RFC 2109. Actually, we all ought to be using RFC 2965, which advocates Set-Cookie2 which fixes and obsoletes 2109, but virtually no-one is doing that either. Anyway, RFC 2616 was written 10 years ago, when no-one had even heard of CSRF. Allowing a Man-in-the-middle to go ahead and make a CSRF attack on an HTTPS connection is, technically speaking, RFC compliant. But that would be far more "broken by design". 2) If you are going for a RFC based approach, you need to be more careful about the use of "SHOULD NOT" and "MUST NOT", and "OPTIONAL" - see RFC 2119, which says that we MUST interoperate with an implementation which does not provide an OPTIONAL item, "though perhaps with reduced functionality". I can't find anywhere that says that we MUST NOT rely on an OPTIONAL header for some of our functionality. In fact, a note in section 13.5.2 or RFC 2616 says: Warning: unnecessary modification of end-to-end headers might cause authentication failures if stronger authentication mechanisms are introduced in later versions of HTTP. Such authentication mechanisms MAY rely on the values of header fields not listed here. Which I think gives us explicit permission to do what we are doing. 3) What alternative do you propose, given the real world security problem that the HTTPS Referer check solves and the constraints that we are working under? Thanks, Luke -- A mosquito cried out in pain: "A chemist has poisoned my brain!" The cause of his sorrow was para-dichloro- diphenyltrichloroethane Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: a new template algorithm
On Tue, 2010-08-31 at 10:52 +0700, burc...@gmail.com wrote: > >From the engineering point of view, you did nothing important. In > fact, I think, you did a bad thing to Django: > By increasing the number of incompatible template engines for django, > you fraction the user base of each one, so you decrease the overall > quality of Django solutions, and Django plugins won't be that good (if > they provide any templates). I believe, new template engine should be > made only if it will be much better than old one (i.e, look at Razor > template engine from Microsoft ASP.NET team). > Your work also makes me think you haven't ever heard of Jinja2 and > Mako, both of which already do what you need (python programming in > templates), but have good documentation and have proper way to connect > to Django. > > >From the teenager point of view, look, you created new cool template > engine! Wow! You rock! I think this is put a bit harshly. There is absolutely no problem with people developing alternative template systems. If we bundle several with Django, that would indeed create problems of splitting the community, but allowing people to use alternatives has always been part of the philosophy of Django. This particular template system doesn't fit with our philosophy regarding templating, and also would have *very* big consequences for backwards compatibility. The only real criticism here is that Gabriele's work would be potentially much more useful to other people if it was developed as a standalone project rather than as a patch to Django. Regards, Luke -- "I don't want to achieve immortality through my work. I want to achieve it by not dying." -- Woody Allen Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: CSRF Middleware/SSL/Firefox 3.6.8 bug
On Fri, 2010-08-27 at 11:20 +0100, Mat Clayton wrote: > I forgot to mention that as well, not to mention that they can be > spooked pretty easily as well, although in this case thats not really > a concern. Any chance of getting some of the core dev's to think about > removing additional SSL checks, as FF 3.6.8 is a pretty major browser > to support and this is likely to only get worse as adoption increases. > Is the additional security worth the trade off? The additional check is there to stop an active network attack that can otherwise be launched against an HTTPS connection. It goes like this: - MitM intercepts an HTTP request and sends a CSRF cookie and a forged page with the corresponding form token to the user over HTTP. - the page causes the browser to make a request to an HTTPS target. Since it has a token and a matching cookie, it gets past our CSRF defences just fine. The additional security that SSL ought to provide against active network attacks has been removed. The basic flaw is that cookies set over unsecured connections (HTTP) are applied to secured connections (HTTPS). Until all popular browsers treat HTTP and HTTPS cookies completely separately (and I'm not aware of *any* that do this), we are stuck with this problem. So we need this check, otherwise there is a huge hole in HTTPS security. An alternative is to tie the CSRF token to the session in some way, like we used to do in Django 1.0 and 1.1. The reasons we moved away from doing that are documented here: http://code.djangoproject.com/wiki/CsrfProtection BTW, the problem that some users are having is almost certainly not related to firewalls, because they wouldn't be able to tamper with the contents of the HTTPS requests (and we don't do the checks for HTTP). But something running in the browser could do it. Finally, the only time we need Referer headers sent is for same origin requests (POST requests to be exact). Sending the Referer header in this case is virtually never a privacy concern, since the site will already be able to track what other pages you have visited on their site. So, if this turns out to be a problem, we could possibly ask Mozilla (and other browsers) to add special casing for this (e.g. make the 'sendRefererHeader' option only apply to cross domain requests). By the way, we are not the only ones doing this. Other people have suggested that strict Referer checking under HTTPS is a very effective and simple way to combat CSRF [1]. Luke [1] http://www.adambarth.com/papers/2008/barth-jackson-mitchell-b.pdf -- Environmentalists are much too concerned with planet earth. Their geocentric attitude prevents them from seeing the greater picture -- lots of planets are much worse off than earth is. Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Unit tests
On Wed, 2010-08-18 at 14:24 -0700, Tom wrote: > I'm interested in writing a unit test suite that does not require all > the components of Django to be bootstrapped. > Correct me if I'm wrong, there are only integration tests bundled with > Django. There are lots of tests that would certainly classify as unit rather than integration tests, and where the test cases inherit from unittest.TestCase rather than using any of the machinery in Django's TestCase. For example, in tests/regressiontests/templates/tests.py, regressiontests/httpwrappers/tests.py and lots of other places. Luke -- Environmentalists are much too concerned with planet earth. Their geocentric attitude prevents them from seeing the greater picture -- lots of planets are much worse off than earth is. Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Suggestion: Better handling of HTML entities by slugify()
On Wed, 2010-07-07 at 21:27 +0800, Russell Keith-Magee wrote: > On Wed, Jul 7, 2010 at 4:00 AM, C. Alan Zoppa wrote: > > I occasionally enter special characters as HTML entities (e.g. “, > > ®, etc.) in an object's title. I feel that slugify() would be more > > useful if these were removed entirely from the returned slug. For example: > > At the moment, a title of “Object Title ” returns the slug > > "ldquoobject-titlerdquo." It is my opinion that "object-title" would be more > > useful in this situation. > > I can't think of any way that ldquo et al could be considered helpful > in a slug, so yes -- this sounds like a bug to me. > Really? I thought Jerome was right here - slugify should be applied *before* escaping. Alan seems to be storing HTML in the model itself, if I understand correctly, and wanting the HTML escaped string to produce nice slugs with slugify. That would require slugify to un-HTML-escape first, which would strictly speaking be incorrect for some people. (e.g. the pathological case - a blog post with the title "Why you see < and > in RSS feeds" - the slug would be why-you-see-lt-and-gt-in-rss-feeds, not why-you-see-and-in-rss-feeds) The normal way to do this is to store special characters as unicode characters, and apply slugify to that, which works fine. Luke -- "Where a person wishes to attract, they should always be ignorant." (Jane Austen) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: MySQL index hints
On Mon, 2010-07-05 at 00:59 -0700, Simon Litchfield wrote: > I'd be happy to use raw(); but then you lose len(), slicing, > pagination, filter chaining, etc. My problem came about because admin > change lists were unusably slow on large tables. With_hints allowed a > simple monkey patch to admin, dropping 2s per page down to ~0.01s. So the underlying problem is: how to rewrite a query (whether generated by my code, someone else's code, or a mixture) so that it performs better on my database. There will always be third party code which generates queries that are really bad for performance, depending on your project or your database. This also applies to tickets like http://code.djangoproject.com/ticket/11604 This makes me think that what we need is a mechanism to do an equivalent to your monkey patching that is easier and more general. Some kind of "replace the auto-generated SQL with this SQL" method could be really useful. Could this be solved by writing your own cursor wrapper which checks the SQL against a list and rewrites as necessary? (You could set it up quickly like this: http://chris-lamb.co.uk/2010/06/01/appending-request-url-sql-statements-django/ That still involves monkey patching, but it is possible to do the same thing by writing a database backend). The major drawback is fragility - doing a replacement at this level could easily break, for example if you added a field to a model. Adequate testing could catch this if your cursor wrapper had a debug mode that could report what was replaced. If we tried to do the replacement at a higher level, then you run into the problem of how to recognise a query and replace it, which could be a much more expensive operation if we are trying to compare objects of type django.db.models.sql.query.Query, for example, and you also might have the problem of not having full access to SQL statements. However, if this method does work, if would be good to document how to do it, and provide some better builtin hooks if necessary. Luke -- "We may not return the affection of those who like us, but we always respect their good judgement." -- Libbie Fudim Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: documentation re-factor for 1.3?
On Sun, 2010-07-04 at 18:41 -0700, dffdgsdfgsdfhjhtre wrote: > On Jul 4, 8:40 pm, Luke Plant wrote: > > Unlike PHP, we have namespaces, which means that providing a list of > > every function/class is a different matter. > > So? Matplotlib has similar documentation to PHP's reference, and it's > fairly well regarded. The one thing that matplotlib doesn't do well is > the topical stuff which django does do well. Namespaces mean that the same name can refer to different things - like 'Field', all the different 'utils' modules etc. So you can't have a shortcut that finds a single thing unless you also know which module it belongs to, but that seemed to be the major problem you were highlighting. > > Also, in the index, the location of each class/function is defined, so > > that if you are looking for something specific, you can indeed find > > where it lives pretty easily. (Again, it is not correct for > > RegexValidator at the moment because of the lack of module directive). > > I did not know this... Maybe we should have documentation for the > documentation Well, the front page says this in a box right at the top: Looking for specific information? Try the Index, Module Index or the detailed table of contents. and it's the Index you need in this case. > Also, another great thing about PHP's documentation is that each and > every function page is structured the exact same way. You may call it > 'boilerplate', but I call it 'consistent structure'. With PHP's > documentation, If you know exactly what you're looking for, you can > find it very quickly. For instance if you want to look up the order > of parameters the preg_replace function takes, you type "preg_replace" > into the search box, then move your eyes down to the first pink box. > That takes 2 seconds max. For django, if you wanted to know the order > of parameters the reverse() function takes, it will take you much > longer. That speed is because PHP has no namespaces (and makes little use of classes). In this case, when you say "reverse" do you mean django.core.urlresolvers.reverse or django.db.models.QuerySet.reverse? So in Django there *has* to be at least one extra step. Equivalent in Django docs: 1) type 'reverse' in the search box. Click on first hit, look at 'Contents' pane down the right, click 'reverse'. 2) Go to the General Index, click 'R', scroll to 'reverse' and choose the one you want. (or, go to General Index and use your browser search functionality e.g. type /reverse in Firefox.) I guess what you are really asking for is a search box that returns only exact matches from the index. > I'm not saying we should make the documentation longer. I'm proposing > we reorganize what we already have into two supersections; reference, > and topical, since they are aimed at separate audiences and serve > different purposes. We *already* have this - the Module Index and Index (clearly linked from the front page) are the reference sections, and most of the rest is topical, with links to the reference sections thrown in. > > Finally, some things do not have documentation regarding their import > > paths because they are not considered public. Even slugify is only > > 'public' as a template filter, and for that purpose it is a builtin and > > does not need to be imported. We currently make no guarantee that "from > > django.template.defaultfilters import slugify" will work in future > > AFAIK, or that it will be suitable for use outside the template system. > > (It already has some things, related to auto-escaping, that might make > > it behave 'funny' depending on what exactly you pass it). > > Then that should all should be documented! The fact that only documented functions are considered public *is* documented: http://docs.djangoproject.com/en/dev/misc/api-stability/ (This is part of the meta-documentation section) But the details of how the slugify function works should *not* be documented. It is a *private* implementation detail (at the moment at least). People who want to use it do so at their own risk, and should read the source. If you want to propose that these functions be considered public *as functions*, that is another matter. There are always various things that ought to become public if they are sufficiently useful outside of Django internals. I don't happen to think django.template.defaultfilters is in that category as it stands - the way those functions work is full of things that are very specific to the template system. > There has to be lots of people out there who use slugify and other > template functions outside templates. I do
Re: documentation re-factor for 1.3?
find function that are not publicly documented *is* expected to know how to use grep (or 'ack', which is a great replacement, BTW). Regards, Luke Regards, Luke -- "Underachievement: The tallest blade of grass is the first to be cut by the lawnmower." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: A setting that skips tests for thirdparty applications
On Fri, 2010-07-02 at 14:48 -0700, lakin wrote: > http://code.djangoproject.com/ticket/13873 > > As part of a normal testing routine, I find it annoying that django > tests ALL of the applications listed in INSTALLED_APPS. While I do > want it to test most of them, I rarely want it to test third party > applications. This ticket has a patch that adds a TEST_SKIP_APP_TESTS > setting that allows for a developer to explicitly request that certain > apps are not tested when one runs ./manage.py test > > The tests for applications listed in TEST_SKIP_APP_TESTS doesn't stop > one from asking for that application to be tested by passing it's app > label to ./manage.py test as a command line argument. > > In any case, I was hoping to get a review/feedback on the patch. I'm > more than willing to improve it in order to get it included. The > original idea came from z...@nox.cx Thanks for the ticket and patch. However, I for one am struggling to understand why it is so painful to do: ./manage.py test app1 app2 app3 app4 ...especially if you create an alias or a shell script wrapper e.g.: alias foo_project_default_tests=./manage.py test app1 app2 app3 app4 (For myself I've just learnt to use the history features of bash and that's just as fast [1]). I also wonder whether this would be an appropriate use of a setting - it feels more appropriate as command line option for manage.py than a setting, since settings are things that 'belong' to a project, or to a project/machine combination, but this is more of a 'developer preference'. Once you make it a command line option, then I imagine you are back in the realm of wanting to make an alias/shell script wrapper so that you don't have to remember it, in which case it's of dubious usefulness IMO. Just my 2¢ Luke [1] http://mike-austin.com/blog/2007/03/history-search-backward.html -- "The truth will set your teeth free." (Calvin and Hobbes) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Feedback for #13849 (csrf referer checking)
On Wed, 2010-06-30 at 11:08 -0400, Paul McLanahan wrote: > I agree that not allowing them by default is good, but I think that > having the option to allow them is also good. I like the default > behavior as it is, as it's inline with our "secure by default" > philosophy. But in our situation, I'm just not as concerned with this > because our only other alternative is to turn off CSRF checking for > these views, which I think is an even worse idea. > > I will say that I'm not sure our situation even benefits from CSRF > protection. We're doing this on a login page (so they shouldn't have a > session at all yet), and on other pages on which we ask users to > submit their password (password change, username change, etc.). The > later all result in confirmation emails, which should prevent fraud. > We're still hesitant to remove CSRF protection from these views though > just in case we've done something else that may open a hole that the > CSRF mechanism will close. Like you said, we're trying to "err on the > side of caution." With Django's sessions and login method, you *do* have a session on the login page itself i.e. before the actual login step. So you need to worry about session fixation etc. Even if not, and you are using your own login method, there is still "login CSRF" to worry about [1]. Without the check for a HTTPS referer, you are wide open to a MITM attacker doing CSRF on your HTTPS connections. The check isn't "erring on the side of caution" — that was what I *previously* thought it was, having not thought through the possible attacks, but in fact it is *absolutely essential*. Without it, you should consider your site as having the same level of protection as an HTTP site. Yes, there are still some benefits to HTTPS (passwords not sent in plaintext), but you certainly don't have the kind of protection that would be expected in HTTPS, and in the general case you might find that all benefits are destroyed by what a successful attacker might be able to achieve (setting passwords etc.) unless you are extremely careful and have closed every possible loophole. For these reasons, changing this behaviour — or even allowing it as an option — would be craziness for a general purpose framework. An option is a bad idea in general, because it is global — and we certainly do *not* want this hole punched in the admin site. If you can really live with the security holes you want to make, that is absolutely up to you. You can write your own middleware, and ensure its suitability for your needs. But I cannot see an argument for supporting this as an option out of the box. Regards, Luke [1] http://www.adambarth.com/papers/2008/barth-jackson-mitchell-b.pdf -- "The only skills I have the patience to learn are those that have no real application in life." (Calvin and Hobbes) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Feedback for #13849 (csrf referer checking)
On Tue, 2010-06-29 at 13:10 -0400, Paul McLanahan wrote: > On Tue, Jun 29, 2010 at 11:40 AM, TiNo wrote: > > This is supposedly potentially insecure, see: > > http://samsclass.info/defcon.html (bottom of the page) > > or http://vimeo.com/7618090 from 2:45' > > Just my 2 cents, > > Tino > > Thanks for the reply. I realize that this attack is an outside > possibility. Were I running Facebook I would be concerned. If this is > a larger threat than it seems then this should definitely stay in > Django to discourage people like us. However, poisoning an arp cache > to insert an attacker machine as a proxy which can then rewrite > "https" to "http" in our page code isn't trivial, and if someone is > doing it, I'd imagine they've got a larger and more lucrative target > in mind. That said, we'll definitely keep it in mind and look at > securing the pages containing the forms as well. For now though, I > think the option of submitting a form over SSL from a non-SSL page > would be nice to have. I wrote the original code and the comment about whether the checking was too strict or not, because, without thinking through all the implications, I wanted to err on the side of caution, and now you are forcing the issue - that's good! In the context of the CSRF code, the whole point of the additional strict Referer checking for HTTPS is to defeat a man-in-the-middle attack: - user browses to http://example.com/ - a MITM modifies the page that is returned, so that is has a POST form which targets https://example.com/detonate-bomb/ . The MITM has to include a CSRF token, but that's not a problem because he can invent one and send a CSRF cookie to match. - the POST form is submitted by javascript from the user's browser and so includes the CSRF cookie, a matching CSRF token and the user's session cookie, and so will be accepted. Thus the CSRF token method fails if you have a MITM. We mitigate this by adding the strict Referer checking, but it only works if it treats an HTTP Referer as completely untrusted, just like an external site, which it is in this context. In the context of https://example.com/ , http://example.com/ is a completely unknown quantity, because with the SSL threat model we are allowing for the possibility of MITM. So if we loosen this check, we destroy the whole reason for it being there, and open up a big security hole. In addition, having looked at the links that Tino sent, I now think there is a very big advantage to not allowing HTTP -> HTTPS POST requests by default, even ignoring the CSRF issue. BTW, if you are using Django's sessions over HTTPS, then you need to have SESSION_COOKIE_SECURE = True, otherwise you are wide open to an HTTP-snooper hijacking HTTPS sessions. However, with SESSION_COOKIE_SECURE = True, sessions will not work over HTTP. This means you need to serve your whole site over HTTPS. I only just realised this for one of my own sites. We probably need to document that more clearly, but I'm not sure how. Regards, Luke -- "The only skills I have the patience to learn are those that have no real application in life." (Calvin and Hobbes) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: proposal: abstract file upload/download handling
On Thu, 2010-06-24 at 08:40 +0200, Waldemar Kornewald wrote: > The boolean is sufficient because those permission checks should be > done in the download view (or a router backend): > > if request.user.is_authenticated: > return file.serve() > else: > # user has no permissions This seems like an arbitrary restriction - why do anonymous users have no permissions? We already have a mechanism for making this decision - user.has_perm(), and that works for anonymous users as well as authenticated users. Luke -- Sometimes I wonder if men and women really suit each other. Perhaps they should live next door and just visit now and then. (Katherine Hepburn) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Class based generic views in 1.3?
On Thursday 03 June 2010 08:59:31 Roald wrote: > > One reason against this is it makes it harder to re-use other > > views from within the View. > > I don't really understand what you mean. Do you mean re-using > 'good' old function-views? Yes, or new ones, or any callable that returns an HttpResponse that I want to return. It is quite a common pattern to have one view wrap another or *several* others, and also to use standard response objects in some utility function or 'sub view'. e.g. def my_view(request, *args): if some_condition(): return some_view(request, *args) elif other_condition(): return some_standard_response(request.user) else: ... return render_to_response(...) def some_standard_response(user) return HttpResponseRedirect(...) This would be massively uglified, at best, if we make View a subclass of HttpResponse. Regards, Luke -- "Oh, look. I appear to be lying at the bottom of a very deep, dark hole. That seems a familiar concept. What does it remind me of? Ah, I remember. Life." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Class based generic views in 1.3?
On Tuesday 01 June 2010 11:43:30 henning.schroe...@gmail.com wrote: > On May 30, 7:24 am, Waldemar Kornewald wrote: > > Maybe I missed something, but why don't you use __new__ instead > > of copying the instance? > > Here is an example where I used __new__ for class based views in my > project: > http://djangosnippets.org/snippets/2046/ > No __call__ or as_view is needed with this approach. > I can easily replace a function with a class without changing the > url configuration. This is an interesting approach. The only downside I can think of at the moment is that implementing __new__() like that really violates expectations - Python programmers have a fairly strong expectation that if you do 'x = SomeClass()', where SomeClass is defined as a class, then x will be an instance of SomeClass. Luke -- "Oh, look. I appear to be lying at the bottom of a very deep, dark hole. That seems a familiar concept. What does it remind me of? Ah, I remember. Life." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Class based generic views in 1.3?
On Wednesday 02 June 2010 16:08:24 Roald wrote: > Maybe I've missed the reason, or it's just too late to change, but > why not using a class itself (so basically its __init__ method) as > a view. I'm using something like this in my projects (as a base > class): > > class View(HttpRequest): > def __init__(self, request, *args, **kwargs): > ... > super(View, self).__init__(self.content()) > ... > You mean: class View(HttpResponse): ... One reason against this is it makes it harder to re-use other views from within the View. You are forced to mutate the 'self' instance of HttpResponse (or else use some nasty hack), rather than being able to simply return the HttpResponse that might be returned from a utility function or a sub-view. Luke -- "Oh, look. I appear to be lying at the bottom of a very deep, dark hole. That seems a familiar concept. What does it remind me of? Ah, I remember. Life." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Class based generic views in 1.3?
On Friday 28 May 2010 15:51:32 Jacob Kaplan-Moss wrote:
> My only real objection here is that `as_view` is just a bunch of
> boilerplate::
>
> urlpatterns = patterns('',
> ('one/', SomeView.as_view()),
> ('two/', SomeOtherView.as_view()),
> ('three', YourView.as_view())
> )
>
> I just really don't like the way that looks.
Agreed. I also think that if you have a mixture of normal views and
class based view, this is ugly:
urlpatterns = patterns('app.views',
('one/', 'some_view_function',
('two/', SomeOtherView),
('three/', YourView)
)
and it would be nicer to spell it:
urlpatterns = patterns('app.views',
('one/', 'some_view_function'),
('two/', 'some_other_view'),
('three/', 'your_view')
)
and have these additional lines in 'app.views':
some_other_view = SomeOtherView.as_view()
your_view = YourView.as_view()
I know that is just moving the boilerplate around, but it is giving a
consistent interface. If some code in a re-usable app moves from
normal views to class based views, they will have to do something like
this *anyway* for backwards compatibility.
But I can see that if subclassing is the default way of re-using, then
exporting these functions gives multiple options about how they should
be re-used, which you are wanting to avoid.
> > There is also the issue of what to do when you need to add a
> > decorator to someone else's view function.
>
> Again, I want to encourge "subclass it" as the correct answer here.
In that case, I guess it would be good to make this hard to do wrong,
by providing helpers that add the decorator to the right end of the
list of decorators.
Regards,
Luke
--
"Oh, look. I appear to be lying at the bottom of a very deep, dark
hole. That seems a familiar concept. What does it remind me of? Ah,
I remember. Life." (Marvin the paranoid android)
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Class based generic views in 1.3?
On Friday 28 May 2010 09:03:02 David Larlet wrote: > > This will solve the thread safety issue, and doesn't require > > another import for a decorator, as Jari pointed out. > > > > We could decide whether to apply decorators inside as_view() or > > __call__(). The argument for putting it in 'as_view' is that > > __call__ does some processing that you might want to skip > > entirely (e.g. the get_object() call - if we applied a > > 'cache_page' decorator, we wouldn't want the get_object() call > > to be made at all if there was a cache hit). > > We discussed that point with Jacob and it looks like it's easier to > modify urlresolvers to create a new instance of the class when we > detect it's a class, this should solve the thread safety issue. Do > you see any drawback to this approach? I'm sure that will work. My main reluctance is using isinstance() when it isn't necessary, and having to change the definition of what a 'view' is when it isn't strictly necessary. (A view is no longer simply "a callable that takes a request and returns a response"). I guess this is partly about aesthetics, and I know practicality beats purity, but: There might also be the following practical disadvantages when it comes to the API we encourage. If the recommended usage is to do: urlpatterns = ... (r'something/', MyClassView) ... then it would be easy to accidentally do `MyClassView()`, and it would appear to work fine, until you later get thread safety issues. Also, if you want to pass arguments to the constructor, you would most naturally do `MyClassView(somearg=something)`, and again it would appear to work. If, however, we encouraged `MyClassView.as_view()` from the start, we can cope with constructor arguments more easily - it changes to `MyClassView.as_view(somearg=something)`. (Either way, it would still be possible to pass in MyClassView() and get thread safety problems, but it's about making mistakes less natural). Of course, if we want to use dotted path syntax in URLconfs - "path.to.MyClassView" - rather than imports, then we would *need* to have the isinstance() check to be able to cope with it at all. My response to that is: my preference would be that the use of class based views should remain an implementation detail, and as far as the URLconf is concerned, views.py should be exporting a ready-to-use callable not the class. (The class is exported for the purpose of other people subclassing, or passing their own constructor arguments, not for direct use in the URLconf). There is also the issue of what to do when you need to add a decorator to someone else's view function. As a developer, I shouldn't have to know that class based views even *exist*, let alone how to use them, if I'm just a client - I should *always* be able to say: from someapp.views import some_view_function my_view_function = login_required(some_view_function) But if someapp.views has only exported SomeClassView and not some_view_function, I now have to do something different. Given that, for many Django apps, the view functions are part of the public API that needs to be exported (i.e. hooked into people's URLconfs or wrapped for re-use), I think it would be good to encourage practices which will lead to stable and consistent APIs, and so *not* allow or encourage classes to be used directly in the URLconf. Regards, Luke -- "Oh, look. I appear to be lying at the bottom of a very deep, dark hole. That seems a familiar concept. What does it remind me of? Ah, I remember. Life." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Class based generic views in 1.3?
On Thursday 27 May 2010 17:29:28 David Larlet wrote: > Hello, > > We're working on this with Ben during djangocon.eu sprint [1]. Any > comment appreciated, still a work in progress though. Looks cool. I think Jari's idea of a class method to instantiate the classes would be a great pattern to establish - something like an 'as_view' classmethod on the base class which returns a view function that uses the class: @classmethod def as_view(cls, *initargs, **initkwargs): def view(*args, **kwargs): obj = cls(*initargs, **initkwargs) return obj(*args, **kwargs) return view The developer can choose whether to write 'MyClassView.as_view()' directly into the URLconf, or put a line into views.py like 'my_view = MyClassView.as_view()' This will solve the thread safety issue, and doesn't require another import for a decorator, as Jari pointed out. We could decide whether to apply decorators inside as_view() or __call__(). The argument for putting it in 'as_view' is that __call__ does some processing that you might want to skip entirely (e.g. the get_object() call - if we applied a 'cache_page' decorator, we wouldn't want the get_object() call to be made at all if there was a cache hit). On that point, I think 'GenericView' should be split into 'ClassView' and 'GenericView'. The get_object() call is probably not a good thing to have on a base class. Regards, Luke -- "Oh, look. I appear to be lying at the bottom of a very deep, dark hole. That seems a familiar concept. What does it remind me of? Ah, I remember. Life." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Static media handling proposal - ticket 12323
On Thursday 27 May 2010 17:09:58 Brian Rosner wrote: > > = Options = > > > > We could: > > 1) add 'STATIC_URL' and use it for widgets and all other media. > > 2) add 'USER_MEDIA_URL' and 'USER_MEDIA_ROOT' and use them for > > file storage. > > I am torn between these two. I am in favor of both to some extent. > I like the direction of being more clear about which is used by > FileFields. On the other hand calling the static resources for a > site STATIC_* seems the most correct to me. > > Of course Pinax had to take some direction and since we couldn't go > changing all the FileFields of the world we had to introduce new > settings. My stance would definitely favor option 2 for the added > clarity it provides. Pinax could end up with best of both worlds > ;-) Thanks for your feedback, it's good to have some input from Pinax developers on this. I knew you had adopted method 1), but I don't think Django doing 2) causes you much pain - you just need to set STATIC_ROOT = MEDIA_ROOT, STATIC_URL = MEDIA_URL in your settings, and add USER_MEDIA_* and nothing else would need to change. > > Finally, once these things are sorted out, is this a small enough > > change that I should go ahead and commit it, or should I wait for > > voting on Django 1.3 features? > > I suppose this is dependent on whether we want to introduce app > media handling or if we want this to separate from that. In my > opinion I can see them as two different bits that be done > independently. Agreed. It could be done separately, but that might some confusion. I'll probably make a branch in my bitbucket repository, and wait and see if anyone proposes app media handling for 1.3. Cheers, Luke -- "Oh, look. I appear to be lying at the bottom of a very deep, dark hole. That seems a familiar concept. What does it remind me of? Ah, I remember. Life." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Class based generic views in 1.3?
On Thursday 13 May 2010 12:08:26 Luke Plant wrote: > 3) Add the decorators when you call 'instantiator' (which, by the > way, I would rather call 'make_view' or something more specific). > You would then have, *in views.py*: > > 3a) my_view = some_decorator(make_view(MyViewClass)) > > or possibly (though I don't think this adds much): > > 3b) my_view = make_view(MyViewClass, > decorators=[some_decorator]) > I had some more thoughts about this. One problem with applying the decorators after is if someone is re-using the class (e.g. subclassing), but forgets to apply some *necessary* decorators when creating the view from it. One work-around is that 'make_view' checks the class itself for a 'decorators' attribute, and applies them: def make_view(viewclass): def view(request, *args, **kwargs): obj = viewclass() return obj(request, *args, **kwargs) for f in getattr(viewclass, 'decorators', []): view = f(view) return view This would make it harder to forget, but would still allow subclasses to adjust the decorators that are used by default. It won't help in the case where people use the class directly in the URLconf, but I don't think that should be encouraged anyway. Luke -- "Oh, look. I appear to be lying at the bottom of a very deep, dark hole. That seems a familiar concept. What does it remind me of? Ah, I remember. Life." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Static media handling - ticket 12323
A Django 1.3 proposal. It is a pretty small feature/change in some ways, but needs some discussion. = Motivation = This is intended to make it easier to cope with the distinction between user provided media and developer provided media. This is a significant pain point when it comes to source control/deployment/backup. For example, it would be much better when uploading my source if I could do 'rsync --delete', so that the files on the server are exactly what I expect - extra files can cause all kinds of bugs. But I can't do that, as it would delete all the user uploaded media (or at least the symlinks to it). Nor can I simply upload to a fresh directory - I would still need to mess around with symlinks or moving files to get user uploaded media where it is expected to be. Also, for security it would be good to have the whole source code directory (including templates, javascript etc) to be write protected from the point of the webserver. But user uploaded files mess that up. = Options = We could: 1) add 'STATIC_URL' and use it for widgets and all other media. 2) add 'USER_MEDIA_URL' and 'USER_MEDIA_ROOT' and use them for file storage. (using backwards compatible defaults either way). If 1), then, additionally, do we need 'STATIC_ROOT' as well? What for? It wouldn't be used anywhere in Django. I was going to go for 1), like the ticket suggests, but I now think that 2) is a much better idea. I strongly suspect that static media are far more common than user uploaded files, so doing 2) will require far fewer changes to existing apps. Also, I suspect that almost all direct use of MEDIA_URL in apps will be for static media: file fields provide a URL attribute which automatically includes MEDIA_URL, but it will be common to use MEDIA_URL for calculating URLs (e.g. in templates). Also, use of 'media' for static media is consistent with ADMIN_MEDIA_PREFIX, which is lost with option 1) If we go with 2), there is a good case for not adding USER_MEDIA_URL to the media context processor - I can't imagine know when it would be needed in the normal case. If you want to generate a URL to a file stored in a model, the only sensible way to do it is instance.somefile.url, which handles it for you. If we go for 1) we will need both STATIC_URL and MEDIA_URL in the media context processor for backwards compatibility. Are there any common uses cases I have not accounted for? Finally, once these things are sorted out, is this a small enough change that I should go ahead and commit it, or should I wait for voting on Django 1.3 features? Thanks, Luke -- "Oh, look. I appear to be lying at the bottom of a very deep, dark hole. That seems a familiar concept. What does it remind me of? Ah, I remember. Life." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: django template revision (for 1.4?)
On Thursday 27 May 2010 13:33:10 Tom Evans wrote: > I never claimed it wasn't; it was stipulated that Django's template > system wipes the floor with all others, and it is inconceivable to > think more favourably of a different system. I never said that Django wipes the floor with "all others" - I was speaking specifically of PHP. And, even then, I said that it wiped the floor with it "in so many ways", not every way - of course there are ways in which, in certain situations, PHP provides certain conveniences that you might want. I was saying that, giving these very significant advantages of Django's template system, I was very surprised to hear someone come to the overall conclusion "PHP has us beat". In many cases, I think even the perceived advantages of PHP are things we don't even want to copy, because of our deliberate design decisions (e.g. to make templates designer friendly). But if there are some real advantages of PHP over Django's templates, or some significant pain points, then of course we're interested to know. BTW, to reply to your other message, you can set variables in a limited form: http://docs.djangoproject.com/en/dev/ref/templates/builtins/#with Thanks, Luke -- "Oh, look. I appear to be lying at the bottom of a very deep, dark hole. That seems a familiar concept. What does it remind me of? Ah, I remember. Life." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: django template revision (for 1.4?)
On Thursday 27 May 2010 03:21:43 Charlie Nolan wrote: > I'm all for streamlining the template system, because I think it's > one of the less-intuitive aspects of Django at the moment (I think > PHP has us beat, and that's really sad) Care to elaborate? From my perspective Django's templates wipe the floor with PHP in so many ways (blocks/inheritance, auto-escaping, variable syntax, tag syntax, filters, all of the very useful builtin tags like cycle etc. etc.) that I can't even imagine how you could come to that conclusion. Luke -- "Oh, look. I appear to be lying at the bottom of a very deep, dark hole. That seems a familiar concept. What does it remind me of? Ah, I remember. Life." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: django template revision (for 1.4?)
On Wednesday 26 May 2010 20:24:31 Daniel wrote:
> Hi, I was just referred here with a suggestion I posted on the
> django ticket tracker. I've been studying the way we do template
> tags, and had a suggestion for how to revise them to make them a
> bit easier. I was thinking that out of the tags, I like the
> variable syntax the best -- {{ variable.attribute }}. I realize
> that if we were to use the double curly brackets on all of the
> tags, they might conflict with the namespace, but it seems like
> having "block", "if", "elsif", "else", "for", and "extends" and
> their respective 'end'ings as keywords would not be burdensome.
> It's really unpleasant typing {%%} and left arrow twice, space,
> the item name, and space for every tag and for its closing tag.
1) It sounds like your text editor is doing something less than ideal.
The fix for that is to change the way your editor behaves, or get a
new editor. We can't change the template syntax based on the mis-
behaviour of one person's editor.
2) The namespace thing is a huge deal. It doesn't only affect
builtins like 'if' etc. but every tag that is loaded from another
library. As such, this represents a massive backwards
incompatibility.
So I can pretty confidently say that this isn't going to be accepted,
sorry :-)
Regards,
Luke
--
"Oh, look. I appear to be lying at the bottom of a very deep, dark
hole. That seems a familiar concept. What does it remind me of? Ah,
I remember. Life." (Marvin the paranoid android)
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Class based generic views in 1.3?
On Thursday 13 May 2010 01:14:35 fitzgen wrote:
> * How to decorate the __call__ method without doing a pointless
> override of __call__ that just calls super so that you have
> something to @decorate on top of. Check out the meta class on line
> 175. This allows you to specify 'decorators = [login_required,
> permission_required("polls.can_vote")]' on your subclass. I showed
> this to Jacob at DjangoSki, and he seemed positive.
I personally don't like metaclasses if we can find another way. Here
are some alternatives which use Plain Old Python:
1) from Python 2.6 we could use class decorators, and we could use the
'old style' MyClass = some_class_decorator(MyClass) until we move to
Python 2.6.
2)
class Foo(View):
__call__ = some_decorator(View.__call__)
3) Add the decorators when you call 'instantiator' (which, by the way,
I would rather call 'make_view' or something more specific). You
would then have, *in views.py*:
3a) my_view = some_decorator(make_view(MyViewClass))
or possibly (though I don't think this adds much):
3b) my_view = make_view(MyViewClass,
decorators=[some_decorator])
This has some significant advantages:
- you don't need to worry about method decorators
- you can re-use MyViewClass with different decorators
- it protects us from changing the URLconf. The URLconf always
has a reference to the 'my_view' function, rather than
MyViewClass. Class-based views then remain an implementation
detail, just as they ought to be. It may well be that
a re-usable app provides some views and might switch
from class-based views to normal functions, and URLconfs
should be insulated from that change.
I don't like the idea of special-casing class based views anywhere,
whether to cope with state or anything else. I think a view should
still be 'a callable that takes a request and returns a response'. If
that means we have to add an extra line to create a view function from
a view class, so be it.
Given that it is going to be possible to use any/all of these whatever
Django provides, I think I'm quite strongly in favour of 3a), and
opposed to adding a metaclass which really doesn't add anything
significant. Metaclasses add complications for people attempting to
understand code, and should be used only when you really need them.
> * How to decorate methods, when the decorator expects the first
> argument to be request, and not self. See line 8. Ideally though,
> Django's decorators would handle this, rather than forcing the use
> of decorate_method_with on to the end users.
We already have 'django.utils.decorators.method_decorator' to cope
with this. All attempts to have decorators that automatically adapt
to functions/methods have failed. See my message here
http://groups.google.com/group/django-developers/msg/f36976f5cfbcbeb3
It has some attachments with test cases that show how our previous
attempt to do this didn't work in some situations.
One thing we could do to make it nicer for the end user is to create
our own 'method' versions of all supplied decorators i.e:
cache_page_m = method_decorator(cache_page)
for every decorator we provide, so that people don't need to do that
themselves.
However, this point may be moot given the discussion above.
Luke
--
"Mistakes: It could be that the purpose of your life is only to
serve as a warning to others." (despair.com)
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Hyperlinking template tags and filters in docs
On Friday 07 May 2010 12:23:46 Russell Keith-Magee wrote:
> The one gotcha that I found was that it assumes you have simplejson
> installed (or that you're running Python 2.6). Given that Django
> ships simplejson (and also checks for a more up to date version),
> wouldn't it makes sense to use 'from django.utils import
> simplejson as json' rather than looking for a native json?
That also assumes you have Django installed - people might not have it
on their path when they build the docs. I often use a Python2.5
virtualenv for Django installations, but have sphinx installed for
2.6, and use that to build docs. So, I think it makes sense to add
the Django import as a fallback. (BTW, in the absence of any
simplejson library, it degrades gracefully to simply not attempting to
hyperlink the template tags/filters, emitting a warning when you build
the docs).
> > Assuming no problems, are other people happy with this going in
> > before the release 1.2? It is quite cool if I say so myself :-)
>
> I agree that having hyperlinks for the template tags is cool;
> however, I'm not sold on javascript as a way of doing it. I
> haven't dug into this into any real depth, but given that there's
> Sphinx is using a 'html+django' mode to markup {% %}, I can't see
> why it shouldn't be possible to modify this markup strategy to
> include links.
The first time I looked into this method, I had no luck at all, but I
was looking from the wrong angle it seemed. Having looked further,
this is what I've found (but it is no more optimistic):
- 'html+django' is a 'Lexer', in Pygments terminology. Lexers return
a string of Tokens, which doesn't allow us to return hyperlinks
at this point.
- instead, we would have to implement our own 'Formatter' which is
like the HtmlFormatter but adds the hyperlinking that we want.
- The first major problem I see is that Sphinx does not have hooks
that allow you to choose your own formatter - it chooses
HtmlFormatter or LatexFormatter depending on your output type.
- to add your own Formatter, you have to use a setuptools entry
point. The only docs for this assume that you are installing
the software in the normal setuptools way (which I for one
never do with Django). There may be another option, I don't
fancy having to work this out myself.
- The Formatter will require a bunch of extra information that
depends on our actual libraries - that is to say, the template
tags that exist - rather than the language itself. (We do
not want to generate links to 'endfor' etc which have no docs)
Getting that information in at the right point might be very
difficult or extremely hacky. I'd be happy to proved wrong, but
the dependencies are very much the wrong way around.
The dependency problem is a general problem with documentation
and the need for hyperlinks. Sphinx handles it well by having
a 'resolving' stage in its processing which fixes up all the
cross refs. This meant I actually had a choice of implementation
strategies for how to get the list of template tags and filters
into the javascript code that creates the hyperlinks. But I'm
pretty sure we can't pass this information into the highlighting
code.
The other problem is one of relative links. The javascript code
can correctly create relative hyperlinks from any of the
documents to the template reference page through the use of
the 'pathto' function which is called in the Jinja templates
and generates a line of the javascript code. This is different
for different files depending on their level of nesting.
Again I don't think we can get that information inside the
Pygments Formatter so that it would generate different
hyperlinks depending on which source document was being
processed.
Although this is in one sense a practical problem, it is also
suggesting we are doing the "wrong thing" as far as Pygments
is concerned. It is not built with the idea that formatting
of a block of code depends on the *library* rather than
the *language grammar*, and certainly not on the disk layout
of files containing the code to be highlighted.
So, I agree that it would be nice to do this 'correctly' in rendering
the static documents, but Sphinx and Pygments are just not set up for
this AFAICS.
Luke
--
"Love is like an hourglass, with the heart filling up as the brain
empties."
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Hyperlinking template tags and filters in docs
Hi all,
I've created a patch to our docs that hyperlinks all builtin template
tags and filters in code samples that are marked as 'html+django'.
This would be pretty hard to do at the render stage, so it is
implemented using jQuery.
In order to only link genuine tags that have documentation, I had to
create my own Sphinx builder which creates a 'templatebuiltins.js',
but otherwise it is all javascript.
I have tested this with Python 2.5/Sphinx 0.5.1 and Python 2.4/Sphinx
0.6.4. I don't know which Sphinx versions we intend to support. Do I
need more extensive testing? Also if anybody with more Sphinx
experience than me spots other gotchas or problems with the patch,
please let me know.
Assuming no problems, are other people happy with this going in before
the release 1.2? It is quite cool if I say so myself :-)
Luke
--
"Love is like an hourglass, with the heart filling up as the brain
empties."
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
diff -r 4b727a43acaa docs/Makefile
--- a/docs/Makefile Thu May 06 04:25:33 2010 +
+++ b/docs/Makefile Fri May 07 01:39:12 2010 +0100
@@ -26,8 +26,8 @@
-rm -rf _build/*
html:
- mkdir -p _build/html _build/doctrees
- $(SPHINXBUILD) -b html $(ALLSPHINXOPTS) _build/html
+ mkdir -p _build/html/json _build/doctrees
+ $(SPHINXBUILD) -b djangohtml $(ALLSPHINXOPTS) _build/html
@echo
@echo "Build finished. The HTML pages are in _build/html."
diff -r 4b727a43acaa docs/_ext/djangodocs.py
--- a/docs/_ext/djangodocs.py Thu May 06 04:25:33 2010 +
+++ b/docs/_ext/djangodocs.py Fri May 07 01:39:12 2010 +0100
@@ -4,12 +4,25 @@
import docutils.nodes
import docutils.transforms
+try:
+import json
+except ImportError:
+try:
+import simplejson as json
+except ImportError:
+json = None
+import os
import sphinx
import sphinx.addnodes
try:
from sphinx import builders
except ImportError:
import sphinx.builder as builders
+try:
+import sphinx.builders.html as builders_html
+except ImportError:
+builders_html = builders
+from sphinx.util.console import bold
import sphinx.directives
import sphinx.environment
try:
@@ -56,7 +69,8 @@
app.add_directive('versionadded', parse_version_directive, 1, (1, 1, 1))
app.add_directive('versionchanged', parse_version_directive, 1, (1, 1, 1))
app.add_transform(SuppressBlockquotes)
-
+app.add_builder(DjangoStandaloneHTMLBuilder)
+
# Monkeypatch PickleHTMLBuilder so that it doesn't die in Sphinx 0.4.2
if sphinx.__version__ == '0.4.2':
monkeypatch_pickle_builder()
@@ -218,7 +232,6 @@
import cPickle as pickle
except ImportError:
import pickle
-from sphinx.util.console import bold
def handle_finish(self):
# dump the global context
@@ -248,3 +261,21 @@
builders.PickleHTMLBuilder.handle_finish = handle_finish
+
+class DjangoStandaloneHTMLBuilder(builders_html.StandaloneHTMLBuilder):
+"""
+Subclass to add some extra things we need.
+"""
+name = 'djangohtml'
+def finish(self):
+super(DjangoStandaloneHTMLBuilder, self).finish()
+if json is None:
+self.warn(bold("cannot create templatebuiltins.js due to missing simplejson dependency"))
+return
+self.info(bold("writing templatebuiltins.js..."))
+ttags = [n for (t,n) in self.env.reftargets.keys() if t == 'ttag']
+tfilters = [n for (t,n) in self.env.reftargets.keys() if t == 'tfilter']
+outfilename = os.path.join(self.outdir, "json", "templatebuiltins.js")
+f = open(outfilename, 'wb')
+f.write('var django_template_builtins = ')
+json.dump({'ttags':ttags, 'tfilters':tfilters}, f)
diff -r 4b727a43acaa docs/_static/djangodocs.css
--- a/docs/_static/djangodocs.css Thu May 06 04:25:33 2010 +
+++ b/docs/_static/djangodocs.css Fri May 07 01:39:12 2010 +0100
@@ -98,7 +98,12 @@
dt .literal, table .literal { background:none; }
#bd a.reference { text-decoration: none; }
#bd a.reference tt.literal { border-bottom: 1px #234f32 dotted; }
-
+
+/* Restore colors of pygments hyperlinked code */
+#bd .highlight .k a:link, #bd .highlight .k a:visited { color: #00; text-decoration: underline }
+#bd .highlight .nf a:link, #bd .highlight .nf a:visited { color: #99; text-decoration: underline }
+
+
/*** notes & admonitions ***/
Re: Trac workflow assistance (commiter feedback needed)
On Friday 30 April 2010 19:22:33 Idan Gazit wrote: > FWIW, I spoke with Alex the other week about turning piano-man into > a more finished product. > > So long as core guarantees that they'll at least take a look at > whatever is made, I'm +1 on rolling our own, and am willing to > champion this project. Personally it would take quite a lot to persuade me to look at it (sorry for not responding before, but I was prompted by this message). I feel similar to Russell about fragmenting the bug tracking community, but even more so - especially given that Trac is written in Python, and already has a very active community of supporters. There are Trac plugins for almost everything you can think of. I would also be extremely loathe to switch if there is *any* loss of information in the database conversion. So that includes requiring identical bug numbers (so that SVN commit messages don't become outdated) and supporting 100% of the wiki formatting in existing pages and all bug reports (or being able to convert it losslessly), and preserving the history of all changes (including flags) to bugs. It is important to be able to look back at a bug report and see what was changed by whom etc. Once you add these requirements, I suspect that you will be doing much more of a 'Trac clone' than you want to be. And we would still need to have compelling reasons to switch - bare feature parity wouldn't be anything like enough, especially given the team and community that are behind Trac. I imagine that adding several plugins to Trac would be a much better route - in particular a code review plugin, of which there are several on trac-hacks I think, and some plugins to do the auto-mail functions that Jeremy suggested. If we fail to review the existing plugins to Trac and see how easy it would be to add our own, it would be nothing but a massive case of NIH. I have written some small plugins for Trac in the past and found it fairly painless. While Trac may have it's problems, we would have to be crazy - not to mention extremely arrogant - to think that we could do better without a huge amount of work. And before anyone starts work on any of these, whichever approach, please do get some consensus about what features are actually desirable! Luke -- "It is a truth universally acknowledged, that a single man in possession of a good fortune, must be in want of a wife." (Jane Austen) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Process discussion: reboot
On Tuesday 20 April 2010 16:43:25 Alex Gaynor wrote: > In general I don't think that the fields on tickets are nearly as > liable to being inaccurate as people are making it sound. That > said marking users who are committers or triagers or what not > probably wouldn't hurt. Since our contributing rules say that you shouldn't re-open a ticket closed by a core dev, and it can be hard to find out who they are and what their Trac logins are, it is actually quite important that we have an easier way of finding out this information, especially for new contributors. It's possible this Trac plugin could help (I haven't tested it): http://trac-hacks.org/wiki/UserManagerPlugin We would need a wiki page (preferably linked from each ticket page) that included something like: = Core developers = [[UserProfilesList(role=developer)]] = Triagers = [[UserProfilesList(role=triager)]] Luke -- "I married Miss Right, I just didn't know her first name was 'Always'" Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: High Level Discussion about the Future of Django
On Monday 19 April 2010 08:50:58 Russell Keith-Magee wrote: > I was going to do a point by point teardown, but then I realized > that I already have, at DjangoCon 2009: > > http://djangocon.blip.tv/file/3043562/ > > The opening is light hearted; the hard details start about 5 > minutes in. By sheer coincidence, I think I addressed almost every > one of the tickets/API areas that you've mentioned in your post, > as well as the general question of why we reject certain ideas, > and what you need to do to get your pony into Django itself. I've never watched that before, but it's extremely good. Could we perhaps link it from the page about contributing to Django, or maybe in the FAQ under "But I’ve reminded you several times and you keep ignoring my patch!" [1]. The material in the latter actually covers some of the material in your talk and in this thread, but seeing a person saying it with concrete examples is very helpful. We can then point to official documentation when this comes up again. Luke [1] http://goo.gl/wF57 -- Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Fixing middleware/view decorator handling of content iterators
On Sunday 21 February 2010 23:07:35 Forest Bond wrote: > Simply knowing whether or > not the response is streaming and whether or not it has a known > size is sufficient for knowing whether or not you should cache it. I can't find it now, but someone definitely had a use case where they definitely wanted local caching *and* a streaming response (it was views that took a long time to generate a small amount of actual content - they outputted an occasional HTML comment to stop the connection from timing out). So I don't think the equation you make stands - we have to leave this to the developer to tell us. Implementing Russell's proposal could be as simple as: if getattr(response, 'can_modify_content', True): # go ahead and modify content This: - is backwards compatible - is easy to use - just add the attribute to your response, or even create a decorator for the view - has far better duck typing properties than using isinstance. This is really important for people who may have developed their own HttpResponse subclass. (That's not necessarily a final API, I'm just illustrating the advantages over isinstance). Django should make the defaults for this behaviour amount to 'do what I mean', but add an API which provides the explicit 'do what I say' control. Luke -- Noise proves nothing. Often a hen who has merely laid an egg cackles as if she laid an asteroid. -- Mark Twain Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: admin javacripts
On Friday 19 February 2010 14:38:01 Rajeev J Sebastian wrote: > >>> While certain parts of the admin are (or will be) using jQuery > >>> (widgets, etc.) > >> > >> This really sucks. Kindly don't do this. > > > > It already does. At least the calendar widget is done with > > jQuery. > > Bad. Comments like this and your previous one are not particularly helpful, as they don't open up discussion about *why* you think it is bad, or about what alternatives you propose, given the growing need we have to use javascript in the admin. IMO, using 'library agnostic' javascript in the admin will mean we just end up implementing our own library, which will end up being an ad hoc, informally-specified, bug-ridden, slow implementation of half of jQuery/dojo/etc, and even less likely to be able to interop with other libraries. Regards, Luke -- Clothes make the man. Naked people have little or no influence on society. -- Mark Twain Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: admin javacripts
On Friday 19 February 2010 15:44:00 Javier Guerra wrote: > just checked my install, and there's no trace of jQuery anywhere in > the admin You must be looking at an old checkout: http://code.djangoproject.com/browser/django/trunk/django/contrib/admin/options.py#L273 Luke -- Clothes make the man. Naked people have little or no influence on society. -- Mark Twain Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Serialization of single object instead of only querysets.
On Monday 15 February 2010 21:53:37 orokusaki wrote:
> The problem comes when you're writing
> your outside code (JavaScript, et all) against the API you've now
> built. You need something like user[0].first_name instead of just
> user.first_name. It's not a huge coding difference. It just seems
> broken and confusing, especially to newcomers.
def serialize_one_object_to_json(obj):
return serializers.serialize("json", [obj])[1:-1]
Regards,
Luke
--
"Our software is idiot proof. If you bought it, that's proof
you're an idiot" (Dogbert)
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
examples directory
Hi, I'm prompted by #12863 [1] to suggest that we remove the whole 'examples' directory, because: * The style of code contained in it is very atypical (it does not use templates), which makes it very poor 'example' code. * As documentation it is extremely limited, and we have great documentation elsewhere. * No-one remembers to update it. I think very few people ever look at it, and the first time I did, a few months ago, I noticed how strange it was and wanted to remove the whole directory. I'm just prompted again by this ticket to do so. What do other people think? Luke [1] http://code.djangoproject.com/ticket/12863 -- Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law. Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Backwards-incompat with CsrfViewMiddleware
On Monday 15 February 2010 09:20:28 Ivan Sagalaev wrote: > Hello everyone! > > A couple of days ago we've noticed a (potentially big) backwards > incompatibility that can bite users upgrading from 1.1 to 1.2. > > CSRF doc encourages users to enable CsrfViewMiddleware that will > check all POST requests for csrf_token and then suggests to alter > all HTML forms to include one or wrap in @csrf_exempt those views > that accept POSTs generated elsewhere. This however is no help for > views in third-party apps that users don't control. So their site > would be broken until those apps are updated. > > This has actually happened lately to users of my OpenID server > library that was missing @csrf_exempt. > > I won't try to estimate the amount of breakage that this will incur > so I'm just shedding a light on the issue. May be some Big Fat > Warning™ should be added to docs or even they should be rewritten > to suggest using individual decorators instead of a middleware. Personally I think we are *much* more secure-by-default if we have CsrfViewMiddleware enabled, and would be against changing that advice. With regards to third party apps, the problem you highlight is really a special case of the fact that every item in our release notes has to be applied to *all* code, including code you did not personally write. Since there are some breaking changes, you cannot assume that a third part app is ready for Django 1.2 unless it says so, or you've tested it yourself. Perhaps we should point that out somewhere. This case is slightly different because it is down to an interaction of a changed default setting with working code, but there will always be cases like that, and I think it is much better for developers to remember the general principle that they are responsible for whatever code they are running, whether they wrote it or not. I should also point out the possibility of doing the csrf_exempt decoration in the URLconf, as with any decorator - I don't know how feasible that is in this case. Thanks, Luke -- Hofstadter's Law: It always takes longer than you expect, even when you take into account Hofstadter's Law. Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Add Mod(%) Operator to If Tag
On Wednesday 10 February 2010 19:00:47 Jonas Obrist wrote: > I'm not sure if this was discussed before, but what about allowing > to easily register custom comparison operators? I'm thinking of a > similar API we currently have for registering template > tags/filters for if-tag operators. My gut instinct is this would be a very bad idea for readability. Defining new operators works OK in a language like Haskell (and even there it can be pretty confusing), but in a template language...? Luke -- You meet a lot of smart guys with stupid wives, but you almost never meet a smart woman with a stupid husband. (Erica Jong) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: #12804 - decorating admin views marked as invalid
On Tuesday 09 February 2010 02:40:03 Russell Keith-Magee wrote: > >> What do people think? > > > > Ugh. Assuming no one can come up with a brilliant fix so that the > > auto_adapt stuff can work when combined with arbitrary other > > decorators (I can't), I favor switching to a system that requires > > different decorators for methods vs. functions and documenting > > the backwards-incompatibility for 1.2. What you propose there > > reads pretty well to me. > > I concur. Ugh. :-) > > It's not nice but given the alternatives, I think the backwards > incompatibility is the best option. +1 to introducing method_dec > (bikeshedding - I actually prefer method_decorator), removing > @auto_adapt_to_method, and documenting the problem with > user_passes_test. OK, good, that's what I was thinking (complete with 'ugh' reaction!). Now, should we backport to 1.1.X or not? We could justify *not* backporting on the grounds that this is a fix for #12804, which does not exist in 1.1.X. Given that no-one filed a bug about this for the lifetime of 1.0 and 1.1, it's probably an obscure enough corner case in the context of 1.1.X that backporting this fix will cause more harm than good. And if someone does have a problem on 1.1.X, we can provide a work-around (which is to use method_decorator). Unless anyone shouts, I'll just commit to trunk. Luke -- "You'll be glad to know, I'm going to donate all the snot I sneeze to hospitals for mucus transfusions." (Calvin and Hobbes) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: #12804 - decorating admin views marked as invalid
is reason, the 'auto adapt' behaviour has been removed, and if
you are using these decorators on methods, you will need to manually
apply `django.utils.decorators.method_dec` to convert the decorator to
one that works with methods. You would change code from this::
class MyClass(object):
@login_required
def my_view(self, request):
pass
to this::
from django.utils.decorators import method_dec
class MyClass(object):
@method_dec(login_required)
def my_view(self, request):
or::
from django.utils.decorators import method_dec
login_required_m = method_dec(login_required)
class MyClass(object):
@login_required_m
def my_view(self, request):
>>>
This is kind of annoying, since the solution we had before seems to
work pretty well most of the time. When it doesn't, however, it is
extremely difficult to debug, and in some cases very tricky to fix.
Florian is right that if we leave it as it is, we should at least
document it. But we would then be documenting a bug, and my (mental)
attempts to come up with documentation for it just reinforce a feeling
of ickiness.
There is a halfway house position, which involves adding method_dec
while deprecating auto_adapt_to_methods, and removing use of it as
much as we can. But this would add a layer of complication, and
people will have to update their code eventually anyway.
What do people think?
Luke
--
"Yes, wearily I sit here, pain and misery my only companions. And
vast intelligence of course. And infinite sorrow. And..." (Marvin
the paranoid android)
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
from functools import wraps, update_wrapper
class MethodDecoratorAdaptor(object):
"""
Generic way of creating decorators that adapt to being
used on methods
"""
def __init__(self, decorator, func):
update_wrapper(self, func)
# NB: update the __dict__ first, *then* set our own .func and
# .decorator, in case 'func' is actually another
# MethodDecoratorAdaptor object, which has its 'func' and
# 'decorator' attributes in its own __dict__
self.decorator = decorator
self.func = func
def __call__(self, *args, **kwargs):
return self.decorator(self.func)(*args, **kwargs)
def __get__(self, instance, owner):
if instance is None:
return self
return self.decorator(self.func.__get__(instance, owner))
def auto_adapt_to_methods(decorator):
"""
Takes a decorator function, and returns a decorator-like callable that can
be used on methods as well as functions.
"""
def adapt(func):
return MethodDecoratorAdaptor(decorator, func)
return wraps(decorator)(adapt)
def striparg(func):
"""
Decorates functions that take a single string argument, and runs
strip() on that arg before calling.
"""
def wrapper(arg):
return func(arg.strip())
return wraps(func)(wrapper)
striparg_auto = auto_adapt_to_methods(striparg)
def simple_decorator(func):
"""
No-op decorator, without @auto_adapt_to_methods
"""
def wrapper(*args):
return func(*args)
return wrapper
# By definition this can't use @auto_adapt_to_methods
def sign(method):
"""
Decorates *methods* that take a single argument and return a string.
"""
def wrapper(self, arg):
return method(self, arg) + " (says " + self.__class__.__name__ + ")"
return wraps(method)(wrapper)
@striparg_auto
def insult(arg):
return arg + " sucks!"
class Stuff(object):
@striparg_auto
def deny(self, arg):
return "No you can't have " + arg
class MoreStuff(Stuff):
deny = Stuff.deny
deny2 = simple_decorator(Stuff.deny)
deny3 = sign(Stuff.deny)
deny4 = striparg_auto(Stuff.deny)
@simple_decorator
def deny5(self, arg):
return super(MoreStuff, self).deny(arg)
@sign
def deny6(self, arg):
return super(MoreStuff, self).deny(arg)
@simple_decorator
@striparg_auto
def deny7(self, arg):
return super(MoreStuff, self).deny(arg)
assert insult(" PHP ") == "PHP sucks!"
assert Stuff().deny(" a pony ") == "No you can't have a pony"
assert MoreStuff().deny(" a backwards incompatible
Re: #12804 - decorating admin views marked as invalid
Ignore my last post - my 'fix' only fixes one corner case, and not a very useful one either. I've got it fairly sorted out now, I'll post tomorrow about this. Luke On Sunday 07 February 2010 21:11:42 Luke Plant wrote: > On Sunday 07 February 2010 10:45:29 Florian Apolloner wrote: > > Hi, > > > > first of all, I agree with Luke that it's hard to fix this if > > it's possible at all [1]. The only real problem I have with this > > ticket beeing closed is the backwards compatibility issue. > > Decorating admin views worked just fine in 1.1 and r11660 broke > > it (At least I hope it worked in 1.1, but as the admin view > > wasn't decorated at all there, it should have worked). Imho we > > should at least consider documenting it, and maybe mention it in > > the release notes, as custom admin code might break by an upgrade > > from 1.1 to 1.2. > > I've thought some more about it, and I think you are right. > Descriptors hurt my head, and descriptors plus decorators and > decorator decorators (like auto_adapt_to_methods) are particularly > bad! > > Python 3.0 is actually more helpful here, as it does away with > unbound methods - they are just functions. That helps to clarify > that your pattern (as on the ticket) is not actually strange, and > ought to work - SomeClass.method is just a function, and it is the > function as originally defined, so it ought to work as an > assignment in that context. > > Anyway, could you try the attached patch? My experiments suggest > it should fix the issue, but coming up with it hurt my brain too > much for me to do any more! > > Luke > -- "Yes, wearily I sit here, pain and misery my only companions. And vast intelligence of course. And infinite sorrow. And..." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: #12801 : Allow empty non-nullable ForeignKey fields until save()
On Sunday 07 February 2010 04:07:22 Luc Saffre wrote: > Luke, I disagree with your explanations. Django behaves oddly. > > Model.save() is the place where empty non-nullable fields cause an > exception. > There is no reason for ForeignKey to behave differently. > ForeignKey fields are different in that they cause database lookups > when they are not None, and that they can cause exceptions in some > special situations, for example > - asking them to save before the foreign instance has been saved > - database integrity errors (invalid non-None pk) > > That said, it is clear that I don't know Django well enough to > decide whether it is feasible/necessary to fix this odd behaviour. It is easy to fix - a few lines in ReverseSingleRelatedObjectDescriptor - because the behaviour there is not accidental, but quite deliberate. The behaviour is only odd from one angle, and it is not very strange, because ForeignKey fields *are* a fundamentally different type of field. The corresponding 'normal' field is the '_id' field - this is a dumb value. I think you are missing the fact that this behaviour occurs *whenever* the FK val is an invalid value, not just with creating new instances. You are therefore asking either for special treatment of instances that are not saved to the database, or for it to be broken in general. Consider this code: >>> o = SomeModel.objects.get(id=1) >>> o.fk_field_id = None >>> print o.fk_field Are you are proposing that the above should print 'None' even when fk_field is non-nullable, rather than a DoesNotExist? 'o.fk_field' is a shortcut for a DB lookup of 'o.fk_field_id', and clearly the result of that lookup is *not* None/NULL. The query returns no results, and, just like every other case when a query returns no results and we expect at least one, you get a DoesNotExist exception. (In this case we skip doing the actual database query as an optimisation, but the same logic applies). Luke -- "Yes, wearily I sit here, pain and misery my only companions. And vast intelligence of course. And infinite sorrow. And..." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: #12804 - decorating admin views marked as invalid
On Sunday 07 February 2010 10:45:29 Florian Apolloner wrote: > Hi, > > first of all, I agree with Luke that it's hard to fix this if it's > possible at all [1]. The only real problem I have with this ticket > beeing closed is the backwards compatibility issue. Decorating > admin views worked just fine in 1.1 and r11660 broke it (At least > I hope it worked in 1.1, but as the admin view wasn't decorated at > all there, it should have worked). Imho we should at least > consider documenting it, and maybe mention it in the release > notes, as custom admin code might break by an upgrade from 1.1 to > 1.2. I've thought some more about it, and I think you are right. Descriptors hurt my head, and descriptors plus decorators and decorator decorators (like auto_adapt_to_methods) are particularly bad! Python 3.0 is actually more helpful here, as it does away with unbound methods - they are just functions. That helps to clarify that your pattern (as on the ticket) is not actually strange, and ought to work - SomeClass.method is just a function, and it is the function as originally defined, so it ought to work as an assignment in that context. Anyway, could you try the attached patch? My experiments suggest it should fix the issue, but coming up with it hurt my brain too much for me to do any more! Luke -- "Yes, wearily I sit here, pain and misery my only companions. And vast intelligence of course. And infinite sorrow. And..." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en. Index: django/utils/decorators.py === --- django/utils/decorators.py (revision 12380) +++ django/utils/decorators.py (working copy) @@ -35,6 +35,8 @@ def __call__(self, *args, **kwargs): return self.decorator(self.func)(*args, **kwargs) def __get__(self, instance, owner): +if instance is None: +return self return self.decorator(self.func.__get__(instance, owner)) def auto_adapt_to_methods(decorator):
Re: #12801 : Allow empty non-nullable ForeignKey fields until save()
On Saturday 06 February 2010 16:32:35 Luc Saffre wrote: > Hello, > > I am trying to understand why Luke closed my ticket #12801 > (http://code.djangoproject.com/ticket/12801). > > Luke, don't get me wrong. Thank you for having taken the time to > decide upon this and my other ticket #12708. I agreed with you for > marking #12708 as invalid, because I didn't understand the real > problem when I wrote it, so the formulations in that ticket are > rather confusing. > > But #12801 now shows so clearly an odd behaviour which should at > least be documented if it cannot be fixed. Marking #12801 as a > duplicate of an invalid ticket means to me that I should not even > *try* to write a patch. Is that really what you want to say? Yes, that is what I meant to say. I'll defend my opinion here, and let others weigh in. The issue is what should happen with the following model: >>> class Foo(Model) >>> a_date = DateTimeField() >>> bar = ForeignKey(Bar) >>> >>> f = Foo() >>> f.bar The current behaviour is that a 'DoesNotExist' exception is thrown, because f.bar_id = None, and the field is not nullable. This is different from other uninitialised fields, which return None e.g. >>> assert f.a_date is None Luc wants the behaviour to be the same i.e. no DoesNotExist exception for an uninitialised ForeignKey field. My response (in a more expanded form here than on the ticket): 1. ForeignKey fields are different from simple values, in that they cause database lookups (the only logical exception being nullable foreign keys with a PK of None), so it's reasonable for them to behave differently. 2. When that lookup occurs, whenever the PK value is not in the database, you get an exception. So if the PK value was 123456 and that did not exist, you would get an exception. So this behaviour is consistent with other values of Foo.bar_id, and changing it would break that consistency. 3. You currently get an exception if you attempt to set f.bar = None. So, 'None' is never the value of a non-nullable foreign key field. The proposed change would break that symmetry - you would not expect the 2nd line of the following code to throw an exception if the first did not: >>> assert f.bar is None >>> f.bar = None If we changed Django to accept the first line, surely we also need to change it to accept the second (which would be a bad idea, I think we'd agree). 4. There is an easy work-around if you are allergic to exceptions, which is to check the PK value (f.bar_id in the above case). Thanks, Luke -- "Yes, wearily I sit here, pain and misery my only companions. And vast intelligence of course. And infinite sorrow. And..." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Refining Django admin look proposal
On Saturday 06 February 2010 09:07:33 j...@jeffcroft.com wrote:
> To expound, it seems like you guys are thinking of a re-skinning.
> I'm not talking about a re-skinning. I'm talking about a complete
> re- thinking of the admin interface. By 1.2? Not a chance. By 1.3?
> Maybe. But I wouldn't count on it. I'm talking about bringing the
> admin to 2010, just like you all brought the models API current
> after .96. Seriously, what if the models API hadn't changed in
> five years? Do you think it would be a small project to bring it
> current? Of course not. It would be huge. This is a serious
> undertaking, and the idea of doing a "contest" feels like you
> think it's a commodity. This is a serious project for serious
> designers, not a g'damn reality TV show.
The original thread seemed to be about re-skinning at most ('refining
Django admin look'). The choice about whether it goes in is largely
down to a matter of taste and aesthetics, which is why our normal
procedures with tickets and feature voting really wouldn't work - for
an aesthetics 'bug' you can have multiple 'fixes' which are all
'correct'. To avoid pointless debate, a contest seems like a good
idea. Personally, even a re-skinning brings with it potential design
problems for existing installations, so I'd have to be convinced the
re-skinning was a big enough improvement to make up for that - I agree
with Russell that this definitely isn't something we should be doing
every few months, or even every year.
If it's a complete re-thinking of the admin interface, it's obviously
a much bigger undertaking, and brings with it huge implications in
terms of backwards compatibility. Even if just the templates were to
be re-worked, we have to consider all the people who have custom admin
templates that assume the current block layout etc. A bigger re-
thinking of the admin would involve even more changes to code.
The models API change that occurred after 0.96 also occurred before
1.0. There have been very few breaking changes to it since then (have
there been any in that department?).
Like Alex, I must have missed a lot of messages to this list if a
major re-thinking of the admin like this has been repeatedly proposed
and dismissed. And it's kind of difficult to assess what is currently
an extremely vague proposal.
Luke
--
"Whom have I in heaven but You?
And there is none upon earth that I desire besides You." Psalm 73:25
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: HTML 5
On Tuesday 02 February 2010 01:09:23 Leo Soto M. wrote: > On Mon, Feb 1, 2010 at 11:00 AM, Luke Plant > wrote: [...] > > > Changing that assumption in an elegant way would require changing > > *everything* to use a markup agnostic output tree, which you > > would then render with different 'writers'. (I'm thinking > > something like the way Pandoc and docutils work). > > If anyone wants to play with that idea, I suggest to look at the > HAML syntax: <http://haml-lang.com/tutorial.html>. The way it uses > indentation for closing tags (just like Python) and CSS syntax for > making it concise is very clever. Another possibility is the way that Genshi does it. You write an XML template, and have the choice to render XML style or HTML style. Presumably this isn't an (easy) option for us though, because we don't have an XML based templating system. For us to do the same just amounts to post-processing. Luke -- "In your presence there is fullness of joy; at your right hand are pleasures forevermore" Psalm 16:11 Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Ticket #12760 is no dublicate
On Wednesday 03 February 2010 20:21:49 kanu wrote:
> My Ticket http://code.djangoproject.com/ticket/12760 Titled :
> "related models with Foreignkey.null=True must not get deleted on
> delete of their relation" was marked as dublicate today.
>
> I don't aggree that this is a dublicate. i filed this as a bug not
> a feature request.
This 'bug' has been reported various times, and it depends on your
point of view whether it is a bug or not. There are certainly cases
where the opposite would be a bug. Consider, for instance, this
model:
class Employee(Model):
boss = ForeignKey('self', null=True)
The semantics of this model might include the idea that employees with
boss == NULL are directors. Now, if we delete employee e1 who happens
to be the boss of employee e2, and just null out e2.boss, we have
promoted e2 to being a director, which is not intended.
So this is a feature request, and on your bug, I linked to the most
recent, open bug covering it.
> I really think that it was intendet to not delete those relations.
> e.g. the comments in CollectedObjects.add say :
> # Nullable relationships can be ignored -- they are nulled out
> before # deleting, and therefore do not affect the order in which
> objects # have to be deleted.
>
> Even if it is not a bug the code appears a little confusing.
> Why is a relation updated and afterwards deleted ?
Karen is right, it allows mutually related data to be deleted without
causing an integrity error in the absence of transactions:
class A(Model):
b = ForeignKey(B)
class B(Model):
a = ForeignKey(A, null=True)
By noticing the null=True, we can null out B.a, then delete the A
object, then the B object, all without requiring a transaction.
CollectedObjects uses this logic to decide that the correct order of
deletion is A,B. Another part of the code actually does the nulling
out and deleting.
Hope that explains it, sorry if the message was rather brief, but this
has come up several times before and we can't explain it every time.
Cheers,
Luke
--
"Where a person wishes to attract, they should always be ignorant."
(Jane Austen)
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: HTML 5
lter that rewrites content according to the doctype, which is obviously a big performance hit. This is just highlighting another reason why you cannot simply switch between these two formats, and we shouldn't pretend that you can. Does anyone know of any framework that works on the same level of abstraction as Django (i.e. with developers writing raw HTML) and has a nice solution to this problem? If not, I'd suggest that we are chasing a fantasy. Even if we get it 90% of the way there, we're still going to have loads of complaints about the 10% -- which is fair enough - if you care about validity, "90% validity" counts for exactly nothing. And we will also inflict the pain of fragmentation/complaints on the authors of third-party apps. == HTML5 will save us anyway == Finally, once HTML5 becomes established, the problem goes away. All our output is HTML5 compliant (you can use both XHTML and HTML style tags in HTML5). The HTML5 spec has acknowledged that having two subtly different and incompatible formats is a bad idea, and has basically said "Let's just redefine both flavours as HTML and be done." == Conclusion == We should just ignore this whole issue for now, and simply switch to HTML5 doctypes in our provided templates at some point. That's basically what Ian Hickson suggests in his updated article. >From then on, things get much better, because backwards compatibility rules are well-defined. If people complain, wanting HTML4, tell them: - Tough. Django produces XHTML. What real world problem are you trying to solve by using HTML4? If it's just a case of some internal standard, then point out that the de-facto standard for Django apps is XHTML, and you gain a lot by fitting in with that standard. All our libraries also output XHTML (e.g. docutils) - OR - live with invalid documents. Almost every single major website out there does so, and no-one notices. (Seriously, outside of your own websites and w3.org, how many valid HTML pages can you find?) I'm saying this as the guy who wrote a Django middleware/app that validates all outgoing HTML - an app I still use it. Some spot checks on various sites of mine indicate that I'm doing better than 99.99% of the web, in that I rarely have any invalid documents. But to worry about being able to instantly switch to the doctype-du- jour -- or rather *last years'* doctype-du-jour -- *as well* as having HTML validity - that's not being a perfectionist, it's called OCD, and I'm drawing the line there. Regards, Luke [1] http://hixie.ch/advocacy/xhtml -- "Trouble: Luck can't last a lifetime, unless you die young." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: HTML 5
On Sunday 31 January 2010 06:59:00 Suno Ano wrote: > Russell> We don't currently have any plans, other than "Yes, we > need to Russell> do something about that". There were some initial > discussions Russell> during the 1.2 development phase, but they > didn't result in Russell> any concrete decisions. > > I see, I have not been aware any such discussion took place plus, > searching through the mail archives, issue tracker and wiki, there > had also been no sign of any such discussions/work in this regard. > I just wanted to know the status on HTML 5 which I now do. There was this ticket - not much of a discussion, but I think I was reflecting the general consensus at that point: http://code.djangoproject.com/ticket/12488 It does come up if you search the tracker for HTML5, as do various other tickets and pages. Luke -- "Trouble: Luck can't last a lifetime, unless you die young." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: AnonymousUser has_perm/has_module_perms function check authentication backends
On Tuesday 26 January 2010 12:12:23 Jari Pennanen wrote: > I read from "1.2 beta" thread that this might make it to the 1.2 > beta of Django, any status on that? Is someone trying to commit > the patches? Florian Apolloner pointed out that it had backwards incompatibility issues. I'm hoping to look at it tonight and see what I can do, probably along the lines of a 'supports_anonymous_users' attribute, like the 'supports_object_permissions' attribute. If anyone else can do a patch and let me know before 21:00 GMT, that would be great. Luke -- "The first ten million years were the worst. And the second ten million, they were the worst too. The third ten million, I didn't enjoy at all. After that I went into a bit of a decline." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: 1.2 beta?
On Tuesday 26 January 2010 11:30:45 Florian Apolloner wrote: > Hi, > > I am far away from beeing a commiter, but would like to get some > thoughts about the AnonymousUser permission checks written down > (I'll keep it short to not hijack this thread): By the time I > moved permission checks to the backends I didn't thought much > about anonymous users (I guess that was bad, but by that time I > never needed it). The current situation is as follows: If we pass > the AnonymousUser into the backend we will berak any code that > relied on the User object to be in the database [disclaimer: as > far as I can tell, we always said that the programmer should check > if the User is authenticated or not, but as we never passed > AnonymousUser into the backends I doubt someone checks that. > Though I guess most people used the backends to check against ldap > etc, so this would be a non issue…]. If you are okay with that, I > am +1 on whatever needs to be done to get this feature in. That's a good catch. Previously I was thinking that the change only affected the "per object" code, which was added since 1.1. The changed required to handle AnonymousUser is pretty simple, as demonstrated by the patch on the provided auth backend. However, it could easily catch people out. One possible migration strategy is to add a 'supports_anonymous_user' attribute, similar to 'supports_object_permissions'. Luke -- "The first ten million years were the worst. And the second ten million, they were the worst too. The third ten million, I didn't enjoy at all. After that I went into a bit of a decline." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Possible bug in messages framework?
On Saturday 23 January 2010 14:29:55 Tobias McNulty wrote: > That said, I have reservations about any kind > of across-the-board encoding because it makes it necessary, > when/if the cookies need to be read by JavaScript, to implement > that same decode/encode on the client side. We actually already encode many values - the SimpleCookie module does it for us: >>> c = Cookie.SimpleCookie() >>> c['test'] = 'He said "Voilà!"' >>> c.output() 'Set-Cookie: test="He said \\"Voil\\303\\240!\\""' Robust Javascript that uses cookies with any 'funny' characters already needs to be able to cope with these octal sequences, with backslashes for quotation marks, and the fact that quotation marks are added for any values with special chars (including space, comma and semi-colon). So, one method that would minimize damage would be to extend the existing octal escape mechanism to semi-colons and commas. That way, javascript only needs to implement one unquoting mechanism. I've implemented this and added the patch to #12470. This still has the disadvantage that if people are storing comma or semi-colon separated lists in a cookie, and are reading that cookie from javascript, and haven't fully implemented the reverse quoting for our existing cookie quoting, then they will have problems. Personally, I imagine that very few Django projects are storing complex values in cookies in Django - most people would just put stuff in the session, or do some AJAX these days. If people *are* storing complex values, I imagine that many will be using base64 or something (it's very easy with builtin javascript functions atob, btoa). So, currently I'm inclined towards committing the patch I just added to #12470, and adding a note in 'backwards incompatibilities'. Thoughts? Luke -- The early bird gets the worm, but the second mouse gets the cheese. --Steven Wright Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Possible bug in messages framework?
On Saturday 23 January 2010 02:44:39 Luke Plant wrote: > BTW, further research shows that we are not really RFC 2109 > compliant at all, but then again no-one is. It seems virtually > everyone (server side and client side) is using 'Netscape style' > cookies with some things adopted from RFC 2109 and RFC 2965, > including 'max-age' and the use of quoted-string, but not the all > important "Version" attribute which turns on RFC 2109 cookies. > Hardly anyone is using Set-Cookie2 from RFC 2965. So specs of any > kind are fairly meaningless here, it's a matter of what everyone > does. Actually, to add a bit more: http://www.ietf.org/mail-archive/web/http-state/current/msg00078.html http://codereview.chromium.org/17045 It's all pretty horrific, it pushes me back towards adding a layer of quoting to our cookie handling just to try to avoid it all - but a robust encoding which definitely avoids all problems. We should note that the presence of semi-colons is more likely to cause problems than commas - Internet Explorer splits on semi-colons, irrespective of quotation marks. Luke -- Sometimes I wonder if men and women really suit each other. Perhaps they should live next door and just visit now and then. (Katherine Hepburn) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Possible bug in messages framework?
On Saturday 23 January 2010 01:20:55 Sean Brant wrote:
> Whats the downside of fixing this at the core cookie handling
> level? I agree with Luke and only ran across this bug when the new
> messaging framework dropped. However if we are going to fix the
> problem, and I do think it's a problem even if its a browser bug,
> should we just fix it at the core level and handle all cookies
> down the road? Would previously stored cookies that are not url
> quoted even fail when trying to unquote? Maybe I'm wrong but this
> seems pretty backwards compatible.
It's true that the vast majority of existing cookie values will be
interpreted the same whether you URL-unquote or not. i.e.
unquote_cookie(value) == value
in many cases so this isn't a big deal. But it's not *always* true,
otherwise there is no need for unquote_cookie. And every time it's
not true is a potential bug with previously stored cookies. The most
likely scenario I can think of is if a cookie is being used to store
some query string or previous URL (like a saved search), which might
then be used literally in some way (e.g. as the parameter to
HttpResponseRedirect). By URL unquoting, when we didn't before, we
would introduce a bug.
e.g.
HttpResponseRedirect("http://foo.com/?q=fr%C3%A8re";)
is not the same as
HttpResponseRedirect("http://foo.com/?q=fr\xc3\xa8re";)
(the latter throws an exception in this case).
BTW, Turbogears' solution is actually buggy for some input:
>>> assert unquote_cookie(quote_cookie("%25")) == "%25"
Traceback (most recent call last):
File "", line 1, in
AssertionError
Coming up with your own encoding is sometimes tricker than it looks.
I don't think this is a big deal because, either way, I don't think
many people are going to be affected.
BTW, further research shows that we are not really RFC 2109 compliant
at all, but then again no-one is. It seems virtually everyone (server
side and client side) is using 'Netscape style' cookies with some
things adopted from RFC 2109 and RFC 2965, including 'max-age' and the
use of quoted-string, but not the all important "Version" attribute
which turns on RFC 2109 cookies. Hardly anyone is using Set-Cookie2
from RFC 2965. So specs of any kind are fairly meaningless here, it's
a matter of what everyone does.
Luke
--
Sometimes I wonder if men and women really suit each other. Perhaps
they should live next door and just visit now and then. (Katherine
Hepburn)
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Possible bug in messages framework?
On Friday 22 January 2010 19:20:20 Vinay Sajip wrote: > On Jan 22, 1:53 pm, SmileyChris wrote: > > If we're to accept that turbogears example, it sounds like we're > > not properly encoding the cookie in core, rather than patching > > messages. > > Yes, it appears to be a matter of luck that other browsers accept > cookie values which are invalid according to the cookie spec. This > comment on a Webkit bug has useful information: > > https://bugs.webkit.org/show_bug.cgi?id=6063#c7 Well, it depends on what you call the 'spec'. What spec says that commas in values is invalid? The 'spec' linked to on that WebKit bug is a preliminary Netscape document, which, as far as I can tell, eventually turned into RFC 2109, which surely has got to be regarded as more authoritative. RFC 2965 (which proposes Set-Cookie2) supposedly obsoletes RFC 2109, but I don't know think it is really used much. [1] As I noted on the bug [2], RFC 2109 allows values to be quoted, in which case Django is behaving correctly, and it is some browsers that are not. Our implementation of this is in fact done entirely by Python's standard library Cookie module [3]. It handles everything I can throw at it (newlines, UTF8, throws exceptions with illegal cookie names etc.). It's kind of unlikely that we've found a bug in it. Of course, it doesn't mean we shouldn't fix things to avoid this bug. But to do so would require some kind of encoding, which is almost certainly going to cause breakage of some kind with existing cookies. Turbogears' solution would be backwards compatible in most cases, but not all. (BTW, if we implemented this change, the nicer way to do is to subclass Cookie and override Cookie.value_encode() and value_decode(), rather than the way that Turbogears does it) Personally, I favour fixing our messages implementation so that it isn't an issue (which is easy, it seems, see details on #12470), and possibly just putting a note into our cookie documentation that some old WebKit based browsers have a bug that means they do not correctly handle a cookie value containing a comma followed by a space. An argument in favour of this lazy approach is that this issue, for both ourselves and Turbogears, has only ever come up in the context of using cookies for messages. Presumably that means that developers are rarely storing extended human readable text strings in cookies outside of this kind of usage, so outside of the messages app it is probably not something we need to worry about. Luke [1] http://code.google.com/webstats/2005-12/httpheaders.html [2] http://code.djangoproject.com/ticket/12470#comment:4 [3] http://docs.python.org/library/cookie.html -- Sometimes I wonder if men and women really suit each other. Perhaps they should live next door and just visit now and then. (Katherine Hepburn) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
On Tuesday 19 January 2010 21:26:17 Jacob Kaplan-Moss wrote: > So there we are. This is very much a brain dump, and I don't really > expect any concrete action to result from it. However, I found some > really interesting stuff there, and I thought I'd share. Thanks for your clarification elsewhere in this thread. One question I have (for Jacob and anyone else with experience): I would have guessed that a big issue with Django from an enterprise perspective is its use of 'singletons'. Though we don't use that terminology, we have dozens of them in the code - every other setting we have implies the existence of a singleton, since the setting can have only one value. For example, EMAIL_BACKEND means that we have one email component singleton which is used everywhere. If you wanted one set of views to use a different email backend, you are out of luck. The 'admin' app was the first major component to allow multiple instances, but there are various other apps which might need this change in theory. How much is this an issue in practice? I imagine that for most sub-systems, most people actually want singletons, but for the cases you don't, it is a big pain. Personally I think that, while there has been an unhealthy proliferation of settings which we need to trim, Django has hit the sweet spot between easy set up and configuration on the one hand, and flexibility on the other. The alternative - turning everything into components with dependency injection everywhere - could be horrific for our sweet spot. One solution to this problem would be to have multiple Django instances hosting different parts of your URL space, allowing them to have multiple settings files. This has obvious limitations, but from the perspective of a large project, perhaps it's not too bad - I imagine that large sites are already used to carving up the URL space to be served by completely different technologies in some cases, at least for hosts running LAMP or similar. Regards, Luke -- "Procrastination: Hard work often pays off after time, but laziness always pays off now." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: What The Enterprise wants from Django
On Wednesday 20 January 2010 17:15:39 sago wrote: > Startup and Settings are the big killers though, head and shoulders > above the previous issues. I've nothing much to add to your > comments other than to say that some of the Django deployments I > know of are highly heterogeneous, with various servers running > different SOA elements, some Django some not. The configuration > issues are formidable and involve all kinds of dependencies beyond > Django (or even Python). I struggle to see a way that Django could > solve those issues, but allowing more flexibility and avoiding > tying configuration to an executable module (i.e. settings.py) > would be useful. I don't understand how avoiding the settings.py mechanism will produce *more* flexibility. With an executable module storing settings, you can implement whatever mechanism you want for settings: ## settings.py ## import sys from acme import network_config network_config.load_django_settings_into_module(sys.modules[__name__]) ## end. Granted, code that does the above might need to be written, but the point of settings.py is that you *can* write it, without forcing anyone to use that system. Luke -- "Procrastination: Hard work often pays off after time, but laziness always pays off now." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: AnonymousUser has_perm/has_module_perms function check authentication backends
On Tuesday 19 January 2010 16:23:32 Harro wrote: > And I guess making it truely awesome would require permissions for > anonymoususers in the default backend too. :( > > If I have timeI'll see what I can come up with. Ticket #9444 [1] is about that, and it had a lot of opposition. It is hard or very hacky for the provided auth backend to support object-level permissions for anonymous users, because there is no obvious place to store permissions. But making it *possible* for custom auth backends to do this is a different matter, and is all we should be aiming for I think. Now for some out-loud thinking about the consequences of this patch: Once you make it possible, it is likely that the authors of re-usable apps will want to depend on this capability. That means that writing custom auth backends would now be much more common. The auth backend already covers both authorization and authentication, but if the authors of re-usable apps are encouraged to depend on it to handle authorization even for anonymous users, then it will be much more commonly required. I don't see this as necessarily a problem, it's just a shift in direction. The more I think about it, the more it seems that authorization questions really need to be decided on a per-site basis, and this mechanism is a good place to do it. (Some people object to mixing authorization and authentication, but it's a bit too late to fix that, and in practice full decoupling is tricky and overly- complex). I've thought through some other scenarios, such as having multiple types of login (on one site I use 'User' in the normal way for people with access to the admin, and a completely separate 'Member' model for completely different type of access), and I can see ways for all of these to work, although you might have to supply a custom AuthenticationMiddleware, and your own User objects which have the same interface as the supplied one. The other consequence of app authors depending on this is that apps might become more restrictive by default, and harder to "open up". Whereas before you would allow an anonymous user to, say, write a comment, or had a single setting to control it, now you will just delegate to the auth backend, which by default has no permissions for anonymous users. Again, I don't see this as particularly bad - the amount of spam these days means it's probably helpful to have things locked down by default. Regards, Luke [1] http://code.djangoproject.com/ticket/9444 -- "Pretension: The downside of being better than everyone else is that people tend to assume you're pretentious." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: AnonymousUser has_perm/has_module_perms function check authentication backends
On Tuesday 19 January 2010 14:23:06 Jannis Leidel wrote: > > I think the best argument in favor of it is using permissions > > with reusable applications. Say I have a wiki application I > > write, I don't know whether anonymous users should be able to > > edit pages, I could make it a setting, but that's ugly. Instead > > the natural thing to do is ask the auth backend and let the > > developer implement it however. > > So you would implement an authentication backend specifically for > your wiki app to be able to check if anonymous users have the > permission to edit a page? How is that less ugly than a setting? In that simple case, a setting might be easier, but it is ugly in the sense of poor separation of concerns. And it is much less flexible - what if the setting might depend on which page they are editing? Very quickly you will end up with the wiki app needing it's own permission system. The writer of the wiki app can avoid the whole question by always delegating authorisation questions to the standard mechanism. I understand your concern about the auth backend assuming 'User', not 'AnonymousUser', but we have specifically documented AnonymousUser as implementing the same interface as User, and I cannot see what harm it would cause to allow this. Luke -- "Pretension: The downside of being better than everyone else is that people tend to assume you're pretentious." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: AnonymousUser has_perm/has_module_perms function check authentication backends
On Monday 18 January 2010 21:55:58 Jannis Leidel wrote: > > Anyone got a good reason reason why this *shouldn't* go in? I'm > > +1 on committing. > > Hm, I don't see a good argument to allow anonymous users to have a > permissions, to be honest. Anonymous users are by definition not > authenticated. Giving them more meaning by being able to grant > them permissions doesn't make them anonymous anymore, right? As you say - anonymous users are by definition not *authenticated*, but that does not be that they are not *authorised*. Permissions is about authorisation, not authentication, and Harro had some good examples where you want to control authorisation for non-authenticated users in a fine-grained way. In fact, most websites assume a certain level of authorisation for non-authenticated users (ability to browse certain parts of the site etc.), so this distinction is already real, not just academic, and the patch would just make it easier to control. Luke -- "Pessimism: Every dark cloud has a silver lining, but lightning kills hundreds of people each year trying to find it." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: AnonymousUser has_perm/has_module_perms function check authentication backends
Hi Harro,
> Hmm I guess I'll just have to keep on hacking django then..
> because that 1% case is something I keep running into for every
> project in one way or another.
> And if it was designed for most apps, why was the row level
> permission bits added? It's useless without simply always being
> able to call request.user.has_perm('permission', obj)
Despite a slight overstatement in that last paragraph, your argument
seems pretty good to me. The whole point of these methods is to allow
custom backends to implement their own logic, so obviously it is
pointless to arbitrarily limit it.
The only downside is that custom backends need to be able to cope with
anonymous users being passed to the has_perm methods, but that is
already well catered for with the is_anonymous() method. It is also
better to make this change before 1.2 lands, otherwise we have a
slight backwards incompatibility if we wanted to do it in the future
(backends could break if they unexpectedly got an AnonymousUser
instead of a User).
Anyone got a good reason reason why this *shouldn't* go in? I'm +1 on
committing.
Some small tweaks to the patch will help:
- 'set()' is nicer than 'set([])'
- in topics/auth.html, it would be nice to document that the backend
should be able to cope with anonymous users being passed to
has_perm().
Luke
--
"Pessimism: Every dark cloud has a silver lining, but lightning
kills hundreds of people each year trying to find it." (despair.com)
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: DB optimization docs
I've added these docs now, or at least a good first stab at them. Suggestions for improvements are welcome, patches are more welcome, as always :-) I backported to 1.1.X, and tried to remove any anachronisms. Regards, Luke -- "Outside of a dog, a book is a man's best friend... inside of a dog, it's too dark to read." Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: DB optimization docs
On Friday 08 January 2010 14:12:56 Mat Clayton wrote: > As someone going through this pain right now, this would be very > helpful. > > Mat I don't know when I'll have more time to work on this, but I've committed the beginnings of draft of docs/topics/db/optimization.txt to my hg repos. http://bitbucket.org/spookylukey/django-trunk- lukeplant/src/tip/docs/topics/db/optimization.txt It needs to be hyperlinked to all the right bits, and filled out. But it might be of some help to you in its current state. If anyone wants to take this on as a mini-project, it would be gratefully received :-) I'm not planning on doing any more on this for at least a few days. Luke -- "My capacity for happiness you could fit into a matchbox without taking out the matches first." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
DB optimization docs
Hi all, I was prompted by this post: http://it.toolbox.com/blogs/database-soup/stuff-id-love-to-see-from- django-36278 to add some notes about some DB access optimizations (essentially the things I mentioned in my comment on that page), but then thought that even if I add them, people are unlikely to find them. It's also not possible to put admonitions everywhere in the docs to stop people doing potentially expensive things. Rather, we need a list of tips for someone looking to optimize DB queries. It could link to all the relevant documentation (select_related() etc), as well as having misc other tips. Good idea/bad idea? At the moment, the information is mainly all there, but spread out in many places - I think we need an "optimize DB access" topic. Luke -- "My capacity for happiness you could fit into a matchbox without taking out the matches first." (Marvin the paranoid android) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Call for comment: #5390 Many To Many signals
On Friday 08 January 2010 12:04:27 Russell Keith-Magee wrote:
> Any feedback on this design?
Just a few things:
1) In assessing performance impact, people need to be clear that there
is just one signal handler for the whole project, like for the
post_save signal etc. Adding a handler will affect every m2m
operation.
One possibly way to reduce the impact slightly would be to have 3
signals - m2m_add, m2m_delete, m2m_clear. A handler attached to one
would then produce no impact on the others being called. I'm not sure
it is worth it though, especially since a handler will often need to
hand all three situations, in which case the performance benefit is
cancelled out, and developers will need to attach the handler three
times.
2) What happens with 'reverse' when the m2m is to 'self'? I think your
code always generates 'reverse=False' in this case, since the target
of the relation is always the same model. This hurts my head to think
about in the abstract, but I think it needs to distinguish between
forward and reverse for non-symmetrical models.
For example, given:
class Person(Model):
fans = ManyToManyField('self', related_name='favorites',
symmetrical=False)
friends = ManyToManyField('self')
p1 = Person.create(...); p2 = Person.create(...)
The symmetrical 'friends' relation obviously should always have
'reverse=False'. But I think this call:
p1.fans.add(p2)
needs to send a different signal than
p1.favorites.add(p2)
and with your patch it wouldn't. But my brain doesn't seem to be
functioning properly today, I may have made a mistake.
Luke
--
"My capacity for happiness you could fit into a matchbox without
taking out the matches first." (Marvin the paranoid android)
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: user_passes_test decorator changes in 1.2
On Thursday 07 January 2010 00:12:08 gaz wrote: > However I thought I'd drop a line here since this isn't in the > backwards incompatible changes listed for 1.2 (I guess I'm possibly > already playing with voodoo, view_func isn't really documented so > it's my own fault I guess). Yes, that definitely falls into the category of relying on an implementation detail, rather than something that should be mentioned as a backwards incompatibility. At the level of inspecting code objects (which is essentially what your code was doing), almost any change is backwards incompatible. 'view_func' is not only not documented, it is a member of a class which is private and marked as such - _CheckLogin. The relevant changesets are: http://code.djangoproject.com/changeset/11587 http://code.djangoproject.com/changeset/11586 Between them, the '_CheckLogin' object was replaced with a more general method. That may enable you to find a solution to your problem. Personally, I'd use this as an opportunity to find a more robust way of getting that information to the template tag :-) Luke -- "Mediocrity: It takes a lot less time, and most people don't realise until it's too late." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Design and code review requested for Django string signing / signed cookies
On Wednesday 06 January 2010 17:12:29 Elias Torres wrote: > > So that would be my defence of why it's better to put the > > "purpose" namespace into the key, rather than the value, in the > > context of HMAC. I'm not an expert though. > > Can a separator solve that issue? In that instance, yes. I'm wary of other applications of HMAC producing loopholes in which the user provides the separator as part as the value being signed, and is able to generate the same string. In Tornado, they are suggesting have a separate key for signing cookies, in which case just signing "name=value" should be enough (provided the developer doesn't do something silly like make "=" part of the name of the cookie). Luke -- "Mediocrity: It takes a lot less time, and most people don't realise until it's too late." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Design and code review requested for Django string signing / signed cookies
On Wednesday 06 January 2010 04:24:15 Elias Torres wrote:
> Thanks Luke for your explanation. I think I have learned something
> here in terms of my own application security independent of
> Django's multi-app environment. Basically, you're reminding me
> that as an application, I must be careful to not sign any random
> string for a user, because it can be re-used for another purpose.
> Therefore, we include the 'purpose', salt, or namespace in the
> process. One thing I would like to ask though, is whether the salt
> needs to be part of the key or the value? If you look at
> TimestampSigner, the timestamp goes as part of the value. I think
> the same can be done with the name of the cookie. In other words
> you can always use a method like _cookie_signature as in Tornado's
> implementation [1] and always pass two parts: cookie name and
> cookie value. Technically, as long as your SECRET_KEY is
> protected, there should not be a need to creating multiple signing
> keys, especially if the data is not secret.
>
> def _cookie_signature(self, *parts):
> hash = hmac.new(SECRET_KEY, digestmod=hashlib.sha1)
> for part in parts: hash.update(part)
> return hash.hexdigest()
This is equivalent to:
hmac.new(SECRET_KEY,
digestmod=hashlib.sha1).update("".join(parts)).hexdigest()
With this, one problem is that accidental collisions between different
parts of code are easier. Consider two cookies:
Cookie 1, name = "use", value = arbitrary string set by user
Cookie 2, name = "user_id", value = server assigned once they've
logged in correctly.
By supplying "r_id123" as the value for cookie 1, I can forge a
user_id=123 cookie. If some validation is imposed on cookie 1, it
might still be possible to manipulate the system such that "r_id123"
is a valid choice, and the exploit would still work.
Actually, the implementation in Tornado does not include the cookie
name in the signature at all, making it even more vulnerable to this
kind of attack.
So that would be my defence of why it's better to put the "purpose"
namespace into the key, rather than the value, in the context of HMAC.
I'm not an expert though.
This is one of the tricky things with security - it's never just a
case of "use this API" - what you feed into the API is always
critical.
> Any thoughts on Django's auth using HMAC besides md5 and sha1
> hashing alone?
It sounds like a good idea, I'm not aware of any particular problems
with our current implementation that would make this a priority
change.
Luke
--
"Mediocrity: It takes a lot less time, and most people don't
realise until it's too late." (despair.com)
Luke Plant || http://lukeplant.me.uk/
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Possible contrib.humanize addition
On Tuesday 05 January 2010 21:24:13 harrym wrote: > I'm working a templatetag that determines whether to use 'a' or > 'an' in front of English words. My particular use case for this is > in a tumblelog app I'm developing - many different types of entry > may be added (link, html, quote, etc), and I'm linking to the 'Add > a[n] entry' pages by iterating over the different types. > Would this be considered a useful addition to contrib.humanize? Hmm, can it handle the following? an honest man a history book an historical book (debatable) My gut instinct is that it's not possible to work this out programmatically. When it comes to other languages, I imagine it's going to be even harder (if it's possible to get harder than 'impossible'), because you have things like gender and case to worry about, which certainly cannot be worked out by an algorithm. To give some examples, in French, the choice is between 'un' and 'une', depending on whether the word is masculine or feminine. In Greek, the choice is between ̔εις, ̔ενα, ̔ενος, ̔ενι, μια, μιαν, μιας, μια, ̔εν, ̔εν, ̔ενος, ̔ενι, depending on whether the word is masculine, feminine or neuter, and in nominative, accusative, genitive or dative case. Although in many cases you would probably omit the article altogether - the above words often mean "one" rather than "a". (That's NT Koine Greek, it might be different/simpler/more complicated in modern Greek). I imagine there are plenty of languages where this gets even worse, violating almost every assumption you don't even know you are making (like whether the article comes before or after or in the middle, or exists at all, etc. etc.) To summarise: if I were you, I would give up now. Luke -- "Mediocrity: It takes a lot less time, and most people don't realise until it's too late." (despair.com) Luke Plant || http://lukeplant.me.uk/ -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
