Re: Bug #11017: Oracle LIKEC query doesn't use index

[Jani Tiainen]

On Tue, 2009-12-22 at 13:00 -0700, Ian Kelly wrote:
> On Tue, Dec 22, 2009 at 12:58 AM, Jani Tiainen  wrote:
> > Escaping bug exists in Oracle 9.2.0.5 and earlier. It was fixed in
> > patchset 9.2.0.6 and any later version.
> >
> > What comes to Oracle official support options for 9.2:
> > Premier Support Ends: 31-Jul-2007
> > Extended Support Ends: 30-Jul-2010
> >
> > If any problem is encountered you're first required to upgrade to latest
> > patchset (9.2.0.8) before requesting further assistance from Oracle. And
> > Oracle has always advised to apply patchsets to db.
> >
> > I have only 9.2.0.1, but bug should appear there and I've patchsets for
> > 9.2.0.6 and up so I can see does it really happen after upgrade. I can
> > stage and run tests to see was it that or something else.
> >
> > And known workaround (that works "everywhere") is use one I pasted few
> > ticket comment before.
> >
> > It's more like design and support decision here - is there need to
> > support (obsolete) Oracle version(s)?
> 
> Hi Jani,
> 
> The only thing I need before committing the workaround is for somebody
> to verify that the bug in #5985 is reproducible in version 9.2.0.5 or
> earlier, and that the workaround indeed fixes it.  Since I don't have
> an instance with the appropriate patchset available to test it myself,
> I'd appreciate it if you would do that testing.
> 
> I for one don't see a compelling case for dropping support for Oracle
> 9 yet.  If the workaround is successful, then the only Django feature
> currently not supported for Oracle 9 is the regex filter, and I'm not
> aware of anything on the horizon that will change that.  Oracle shops
> also tend to be notoriously slow at updating.  My employer is not as
> bad as some, but even we still have some Django installations running
> on Oracle 9.
> 
> Even if we do decide to drop support for Oracle 9 (or just the early
> patchsets thereof) soon, it will take a full release cycle so that we
> can issue the appropriate deprecation warnings.  We'll still want to
> be using the workaround at least for the short term.

Finally I managed to get a vm with Oracle 9.2.0.1, so are there some
specific tests I need to run? Is there patch that you would like to use
and I could apply and run tests again to see is there any regressions?

-- 

Jani Tiainen


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Problem with history view in admin page

[Russell Keith-Magee]

On Thu, Jan 7, 2010 at 2:02 PM, Mario Briggs  wrote:
> Russ,
>
> The indexing that Oracle is supporting is 'index-on-expression'. DB2
> also supports that, but it isnt enabled for character long columns,
> since they hadnt had a request for that. I talked to the DB2 server
> folks about these use-cases and they have agreed to support this via
> index-on-expression shortly. For the immediate needs, Rahul is going
> to look at supporting this via 'hidden' columns -  Whenever there is a
> text field with an index specified, the backend will create a hidden
> and generated column (poplated with first X chars of the text field).
> The index will be created on this hidden column. When there is a
> lookup by value, the backend will actually do the lookup on the hidden
> indexed column. - you see any blockers to this ?

I'm not sure what you're asking me. "Blockers" of what?

I know almost nothing about DB2; I know only slightly more about
Oracle. I can only advise you on what Django does. In this case, the
code tells you everything you need to know: Django needs to be able to
issue the following query:

action_list = LogEntry.objects.filter(
object_id = object_id,
content_type__id__exact =
ContentType.objects.get_for_model(model).id
).select_related().order_by('action_time')

As long as you can satisfy that requirement, you've done everything
you need to do.

> BTW, index is not turned ON on LogEntry.object_id. Do you feel this
> use-case usage is low to warrant that ?

I can't say the lack of an index on that field has been a performance
problem on any site I've managed, and I don't recall seeing any
reports on the mailing list or Trac that indicate that it has caused a
problem for anyone else.

Yours,
Russ Magee %-)
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Problem with history view in admin page

[Mario Briggs]

Russ,

The indexing that Oracle is supporting is 'index-on-expression'. DB2
also supports that, but it isnt enabled for character long columns,
since they hadnt had a request for that. I talked to the DB2 server
folks about these use-cases and they have agreed to support this via
index-on-expression shortly. For the immediate needs, Rahul is going
to look at supporting this via 'hidden' columns -  Whenever there is a
text field with an index specified, the backend will create a hidden
and generated column (poplated with first X chars of the text field).
The index will be created on this hidden column. When there is a
lookup by value, the backend will actually do the lookup on the hidden
indexed column. - you see any blockers to this ?


BTW, index is not turned ON on LogEntry.object_id. Do you feel this
use-case usage is low to warrant that ?

regards
Mario
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Tai Lee]

It makes sense to me that the developer should first check that their
form is valid and second check that their model object is valid when
calling ModelForm.save(commit=False). This could be done by having the
developer check the result of model.full_validate() before calling
model.save(), or by having model objects returned by ModelForm.save
(commit=False) automatically trigger model validation when model.save
() is called, even if model.save() does not normally trigger model
validation?

Cheers.
Tai.
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

.filter() results in non-serialized data

[Subramanyam]

Hi


As per my understanding the .all() query set method results in
serialized data and .filter() does not
is there a specific reason for this (helps me in understanding the
query set methods more )

I had a problem and posted in
http://groups.google.com/group/django-users/browse_thread/thread/7f3f537f1799cf59?hl=en
and did find a work around


Thanks in advance

Regards
Subramanyam
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: user_passes_test decorator changes in 1.2

[Luke Plant]

On Thursday 07 January 2010 00:12:08 gaz wrote:

> However I thought I'd drop a line here since this isn't in the
> backwards incompatible changes listed for 1.2 (I guess I'm possibly
> already playing with voodoo, view_func isn't really documented so
>  it's my own fault I guess).

Yes, that definitely falls into the category of relying on an 
implementation detail, rather than something that should be mentioned 
as a backwards incompatibility.  At the level of inspecting code 
objects (which is essentially what your code was doing), almost any 
change is backwards incompatible.  'view_func' is not only not 
documented, it is a member of a class which is private and marked as 
such - _CheckLogin.

The relevant changesets are:

http://code.djangoproject.com/changeset/11587
http://code.djangoproject.com/changeset/11586

Between them, the '_CheckLogin' object was replaced with a more 
general method.  That may enable you to find a solution to your 
problem.

Personally, I'd use this as an opportunity to find a more robust way 
of getting that information to the template tag :-)

Luke

-- 
"Mediocrity: It takes a lot less time, and most people don't 
realise until it's too late." (despair.com)

Luke Plant || http://lukeplant.me.uk/
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Message level API awkwardness

[Tobias McNulty]

That sounds like a pretty simple addition that won't affect (m)any folks, so
I'm fine with adding such a wrapper.  I'd prefer the methods be called
'get_level' and 'set_level', however, so they're shorter and match the
naming of 'get_messages'.

Others?


On Wed, Jan 6, 2010 at 6:09 PM, Jeremy Dunck  wrote:

> I realize I'm very late giving feedback on the API, sorry and feel
> free to ignore if I'm too late.
>
> That said, from the docs, the API to set the effective messaging level
> is awkward:
>
> ==
> # Change the messages level to ensure the debug message is added.
> messages.get_messages(request).level = messages.DEBUG
> ==
>
> To *set* the messaging level, I call .get_messages?
>
> I think I understand the reason this was done-- get_messages is
> backend- and request-specific, but it is definitely surprising.
>
> Perhaps something like this would do better?
>
> messages.effective_level(request) => messages.INFO
> messages.set_effective_level(request, messages.DEBUG)
> ?
>
> Behind the scenes, it could be almost the same implementation.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-develop...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>
>
>


-- 
Tobias McNulty
Caktus Consulting Group, LLC
P.O. Box 1454
Carrboro, NC 27510
(919) 951-0052
http://www.caktusgroup.com
-- 

You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-develop...@googlegroups.com.

To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Re: Design and code review requested for Django string signing / signed cookies

[Julian]

On Jan 4, 4:47 am, Simon Willison  wrote:
> As you can see, the separator between the signature and the value
> defaults to being a ':'. I plan to move it from being an argument on
> the sign and unsign methods to being an argument to the Signer class
> constructor.

Here's my $.02 based on a cursory review.  I like the idea of putting
this logic into django.  I find myself implementing something similar
often.

This all looks ok from a security perspective, but it seems there are
a couple of things that could be simplified:

1) sep.  Why use a delimiter at all?  The fields you are encoding
(hash or hash+date) are of a determinate length.  Put those fields
first, and expect them to have the correct length when unsigning.

2) salt + 'signer'.  I don't see the security advantage to this.  If
the django secret key is compromised, you are dead.  If not, then
everything should be good.  If someone can brute-force the key from a
large sample of signatures, and I agree that is a small but real risk
(at least I thought so with MD5), then they will much more easily
reverse the salt+'signer'+secret from that.

Unless you picture the salt changing frequently.  In which case, how
does the unsigner know the salt?

I suspect there may even be a security risk to letting people use
salt.  They may use something that is known to the attacker, and it
seems like this might carry some risk in itself.

One other thought - I know you expect this to be used for texty
protocols like http cookies, but should the signer class be producing
already-encoded data?  It seems like the protocol using the signed
cookie should be in charge of encoding it.  i.e. produce an 8-bit
chunk of data.  Let the set-cookie logic decide what encoding it is
passed that binary chunk.  This would make the base signer class more
useful for applications where encoding is not necessary.  And set-
cookie should be robust enough to know when the data passed to it
needs encoding.

-=Julian=-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Using Django with Jinja2 and TEMPLATE_DEBUG=True

[Russell Keith-Magee]

On Thu, Jan 7, 2010 at 2:54 AM, Rick van Hattem  wrote:
> Hi,
>
> Maybe this question has already been asked, but I am wondering why Jinja2
> compatibility can't be fixed in a clean way. Currently the code assumes that
> if an exception has a "source" attribute that it's a Django exception and can
> be processed as such. (the code:
> http://code.djangoproject.com/browser/django/trunk/django/views/debug.py#L87 )

There is an important difference between "can't" and "hasn't". The
Django core's development priority is Django, and Django is a
'batteries included' framework. As a result, our development priority
is in making Django's template language better.

We are open to making any changes that make it easier to plug in
different template engines or ORMs (or any other external component).
We (the core) just haven't spent a lot of development time trying to
reach that particular goal.

If someone in the community wants to pursue this goal, they are
welcome to do so. Propose a patch, and as long as it doesn't
fundamentally compromise the design of Django, it stands a good chance
of getting into trunk.

> However, if all template exceptions have the same base exception (as they
> logically should) wouldn't it automatically work if you simply test for the
> exception?

No, it wouldn't (at least, not completely). Jinja wouldn't extend
Django's TemplateSyntaxError class, so using the approach you
describe, Jinja's TemplateSyntaxErrors wouldn't break the debug page,
but you wouldn't get good template error feedback either.

> There has been a previous ticket with a hacky solution:
> http://code.djangoproject.com/ticket/10216
>
> But I feel that testing for specific exceptions (as any try/except should)
> would make the code cleaner and fix this problem. Assuming that any exception
> with a source attribute can be processed as the expected Django exception is
> simply wrong.

I disagree. Checking for capabilities rather than types is the
cornerstone of duck typing. In this case, Jinja's TemplateSyntaxError
doesn't quack in quite the same way as Django's duck. I can see three
possible solutions:

 1) The capability check isn't rigorous enough. Modify the capability
check to make it more specific.
 2) Push the problem back to Jinja. As Malcolm's comment on #10216
notes, Django has a protocol for template errors and doesn't make any
claim to having Jinja compatibility. If Jinja makes claims to Django
compatibility but doesn't adhere to that protocol, then it's up to
Jinja to fix the problem.
 3) Come up with a new capability that can be used instead.

>From a cursory inspection, I'm not sure there is much we can do with
(1) - there isn't a lot of detail on Exception that can be used for a
capability check, and the only attribute that is actually needed is
'source' (albeit in a particular format that evidently Jinja doesn't
use).

(2) isn't Django's problem to fix.

(3) Seems like the most promising option. For example, most of the
call to ExceptionReporter .get_template_exception_info() could be
deferred to the exception itself. If we refactored that code into a
method on the exception itself, we could use that method as the
capability that we check for. It would be less ambiguous (not many
exceptions would have a 'get_template_exception_info' method), and it
would allow *any* exception (template or otherwise) to raise an
embedded exception that provided template details on the technical 500
page.

However, as I said originally - the existing setup works for Django at
present. If you want to improve support for Jinja... patches are
accepted :-)

Yours
Russ Magee %-)
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

user_passes_test decorator changes in 1.2

[gaz]

Hi there,

I've just downloaded the 1.2 alpha and started testing it with one of
my projects. For the most part it works well, however I've hit a bit
of a snag due to the changes to the user_passes_test decorator in
django.contrib.auth.decorators.

I have a some code which follows the paradigm of:

class Site(object):
...
def get_urls(self):
...
return urlpatterns

so that I can deploy multiple instances (I'm actually extending
django.contrib.admin.sites.AdminSite) of an app and use namespaced
urls.

Now, with 1.1 I had an index view method which was decorated with
login_required:

class Site(object):
...
@login_required
def index(self, request):
return HttpResponse('')

In a template tag I've been determining which instance of my Site I've
been using with code like this:

func, args, kwargs = resolve('/path/')
instance = func.view_func.im_self

However in 1.2, this fails because there is no 'im_self' attribute.
I've been poking around and discovered that (in the shell) I can get
the site instance with:

func, args, kwargs = resolve('/path/')
instance = func.func_closure[0].cell_contents

However when I replace my 1.1 version of the code with this recipe
above and try to access that view with the development server running
1.2 I get:

Original Traceback (most recent call last):
  File "/path/to/my/templatetag.py", line XX, in
determine_instance_for_path
instance = func.func_closure[0].cell_contents
TypeError: 'NoneType' object is unsubscriptable

I must confess I don't know what a 'cell' is, haven't heard that
python terminology before, so I will continue to look this up.

However I thought I'd drop a line here since this isn't in the
backwards incompatible changes listed for 1.2 (I guess I'm possibly
already playing with voodoo, view_func isn't really documented so it's
my own fault I guess).

On the other hand, if this is a bug, I'd be willing to work with
someone to find a solution.

Regards,
Gary
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Message level API awkwardness

[Jeremy Dunck]

I realize I'm very late giving feedback on the API, sorry and feel
free to ignore if I'm too late.

That said, from the docs, the API to set the effective messaging level
is awkward:

==
# Change the messages level to ensure the debug message is added.
messages.get_messages(request).level = messages.DEBUG
==

To *set* the messaging level, I call .get_messages?

I think I understand the reason this was done-- get_messages is
backend- and request-specific, but it is definitely surprising.

Perhaps something like this would do better?

messages.effective_level(request) => messages.INFO
messages.set_effective_level(request, messages.DEBUG)
?

Behind the scenes, it could be almost the same implementation.
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Brian Rosner]

On Jan 6, 2010, at 3:59 PM, Joseph Kocherhans wrote:

> On Wed, Jan 6, 2010 at 4:46 PM, Jeremy Dunck  wrote:
>> On Wed, Jan 6, 2010 at 3:56 PM, Joseph Kocherhans  
>> wrote:
>> ...
> On Wed, Jan 6, 2010 at 12:49 PM, Simon Willison  
> wrote:
>> ...
>> form = SecretQuestionForm( {"secret_question":"foo", "answer":"bar"} )
>> if form.is_valid():
>>p = form.save(commit=False)
>>p.user = request.user
>>p.primary_contact = somecontact
>>p.save()
>> ...
>>> I had another idea that I think might work out. What if
>>> ModelForm.validate() only validated fields on the form, like it worked
>>> before the merge, and ModelForm.save() would automatically validate
>>> the excluded fields, raising ValidationError if there was a problem?
>>> Validation for each field would only happen once, it would accommodate
>>> the commit=False idiom, and it would still ensure that the model
>>> itself is validated in common usage.
>> 
>> Note that p in the example above is the unsaved model instance, not
>> the ModelForm.  So to fix the idiom, the excluded field validation
>> would need to be done on Model.save, not ModelForm.save, right?
> 
> Ugh. Yes it would. I did mean ModelForm.save(), but as you've pointed
> out, that won't work (at least for the idiom). One of the early
> decisions was that Model.save() would never trigger validation for
> backwards compatibility purposes. Maybe something from the idea is
> salvageable, but it won't work as I presented it. I think having the
> model track which validators have been run and which haven't is a
> non-starter. That's something Honza actively avoided in the design.

Saw this after my e-mails. This does present an issue. For sake of backwards 
compatibility it would seem that the second step of validation could just be 
done by the developer? This is mostly to prevent double validation, but also 
maintain compatibility with the Django idiom.

Brian Rosner
http://oebfare.com
http://twitter.com/brosner

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Brian Rosner]

On Jan 6, 3:57 pm, Brian Rosner  wrote:

> Yeah, I think that must have been a typo in Joseph's mail. The way I read it 
> that the model knows what fields it has already validated and the call to a 
> Model.save would validate the rest.

Actually, I just realized that Model.save doesn't do validation now
does it?

Brian Rosner
http://oebfare.com
http://twitter.com/brosner
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Joseph Kocherhans]

On Wed, Jan 6, 2010 at 4:46 PM, Jeremy Dunck  wrote:
> On Wed, Jan 6, 2010 at 3:56 PM, Joseph Kocherhans  
> wrote:
> ...
 On Wed, Jan 6, 2010 at 12:49 PM, Simon Willison  
 wrote:
> ...
> form = SecretQuestionForm( {"secret_question":"foo", "answer":"bar"} )
> if form.is_valid():
>    p = form.save(commit=False)
>    p.user = request.user
>    p.primary_contact = somecontact
>    p.save()
> ...
>> I had another idea that I think might work out. What if
>> ModelForm.validate() only validated fields on the form, like it worked
>> before the merge, and ModelForm.save() would automatically validate
>> the excluded fields, raising ValidationError if there was a problem?
>> Validation for each field would only happen once, it would accommodate
>> the commit=False idiom, and it would still ensure that the model
>> itself is validated in common usage.
>
> Note that p in the example above is the unsaved model instance, not
> the ModelForm.  So to fix the idiom, the excluded field validation
> would need to be done on Model.save, not ModelForm.save, right?

Ugh. Yes it would. I did mean ModelForm.save(), but as you've pointed
out, that won't work (at least for the idiom). One of the early
decisions was that Model.save() would never trigger validation for
backwards compatibility purposes. Maybe something from the idea is
salvageable, but it won't work as I presented it. I think having the
model track which validators have been run and which haven't is a
non-starter. That's something Honza actively avoided in the design.

Joseph
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Brian Rosner]

On Jan 6, 2010, at 3:46 PM, Jeremy Dunck wrote:

> On Wed, Jan 6, 2010 at 3:56 PM, Joseph Kocherhans  
> wrote:
> ...
 On Wed, Jan 6, 2010 at 12:49 PM, Simon Willison  
 wrote:
> ...
> form = SecretQuestionForm( {"secret_question":"foo", "answer":"bar"} )
> if form.is_valid():
>p = form.save(commit=False)
>p.user = request.user
>p.primary_contact = somecontact
>p.save()
> ...
>> I had another idea that I think might work out. What if
>> ModelForm.validate() only validated fields on the form, like it worked
>> before the merge, and ModelForm.save() would automatically validate
>> the excluded fields, raising ValidationError if there was a problem?
>> Validation for each field would only happen once, it would accommodate
>> the commit=False idiom, and it would still ensure that the model
>> itself is validated in common usage.
> 
> Note that p in the example above is the unsaved model instance, not
> the ModelForm.  So to fix the idiom, the excluded field validation
> would need to be done on Model.save, not ModelForm.save, right?

Yeah, I think that must have been a typo in Joseph's mail. The way I read it 
that the model knows what fields it has already validated and the call to a 
Model.save would validate the rest.

Brian Rosner
http://oebfare.com
http://twitter.com/brosner

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Jeremy Dunck]

On Wed, Jan 6, 2010 at 3:56 PM, Joseph Kocherhans  wrote:
...
>>> On Wed, Jan 6, 2010 at 12:49 PM, Simon Willison  
>>> wrote:
...
 form = SecretQuestionForm( {"secret_question":"foo", "answer":"bar"} )
 if form.is_valid():
    p = form.save(commit=False)
    p.user = request.user
    p.primary_contact = somecontact
    p.save()
...
> I had another idea that I think might work out. What if
> ModelForm.validate() only validated fields on the form, like it worked
> before the merge, and ModelForm.save() would automatically validate
> the excluded fields, raising ValidationError if there was a problem?
> Validation for each field would only happen once, it would accommodate
> the commit=False idiom, and it would still ensure that the model
> itself is validated in common usage.

Note that p in the example above is the unsaved model instance, not
the ModelForm.  So to fix the idiom, the excluded field validation
would need to be done on Model.save, not ModelForm.save, right?
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Brian Rosner]

On Jan 6, 2010, at 2:48 PM, Łukasz Rekucki wrote:

> Maybe you could do something like this:
> 
> with form.valid_model() as model: # i'm not good at inventing names
> model.user = request.user
> model.primary_contact = somecontact
> 
> The valid_model() would be a context manager that does form validation and 
> form.save(commit=False) on enter + model validation and save() on exit. Of 
> course this will only work on Python 2.5+, so it's probably no good for 
> django 1.2. Just wanted to share an idea.

FTR, this is a pretty neat idea. The naming is a bit off, but that can be 
worked out. Unfortunately, we couldn't much with it now, but I'd like to look 
at the possibility for 1.3. Thanks for sharing.

Brian Rosner
http://oebfare.com
http://twitter.com/brosner

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Alex Gaynor]

On Wed, Jan 6, 2010 at 4:06 PM, Brian Rosner  wrote:
>
> On Jan 6, 2010, at 2:56 PM, Joseph Kocherhans wrote:
>> I had another idea that I think might work out. What if
>> ModelForm.validate() only validated fields on the form, like it worked
>> before the merge, and ModelForm.save() would automatically validate
>> the excluded fields, raising ValidationError if there was a problem?
>> Validation for each field would only happen once, it would accommodate
>> the commit=False idiom, and it would still ensure that the model
>> itself is validated in common usage.
>
> I like this. It would then provide far superior error messages in cases where 
> there was real developer error.
>
> Brian Rosner
> http://oebfare.com
> http://twitter.com/brosner
>
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-develop...@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>
>
>
>

I agree with Brian.  I also really like the context manager based API,
so for 1.3 I think it would be nice to include something like that.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your
right to say it." -- Voltaire
"The people's good is the highest law." -- Cicero
"Code can always be simpler than you think, but never as simple as you
want" -- Me
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Brian Rosner]

On Jan 6, 2010, at 2:56 PM, Joseph Kocherhans wrote:
> I had another idea that I think might work out. What if
> ModelForm.validate() only validated fields on the form, like it worked
> before the merge, and ModelForm.save() would automatically validate
> the excluded fields, raising ValidationError if there was a problem?
> Validation for each field would only happen once, it would accommodate
> the commit=False idiom, and it would still ensure that the model
> itself is validated in common usage.

I like this. It would then provide far superior error messages in cases where 
there was real developer error.

Brian Rosner
http://oebfare.com
http://twitter.com/brosner

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Joseph Kocherhans]

On Wed, Jan 6, 2010 at 3:26 PM, Waylan Limberg  wrote:
> I've only scanned the docs the other day and haven't actually used the
> new model validation stuff, so my impressions may be a little off,
> but...
>
> On Wed, Jan 6, 2010 at 2:54 PM, Joseph Kocherhans  
> wrote:
>> On Wed, Jan 6, 2010 at 12:49 PM, Simon Willison  
>> wrote:
>>> A couple of related tickets filed today about model validation:
>>>
>>> http://code.djangoproject.com/ticket/12513
>>> http://code.djangoproject.com/ticket/12521
>>>
>>> The first one describes the issue best - the new model validation code
>>> breaks the following common Django convention:
>>>
>>> form = SecretQuestionForm( {"secret_question":"foo", "answer":"bar"} )
>>> if form.is_valid():
>>>    p = form.save(commit=False)
>>>    p.user = request.user
>>>    p.primary_contact = somecontact
>>>    p.save()
>>>
>
> My initial reaction to this snippet was to wonder why the model was
> not being validated after the additional data was added/altered.
> Shouldn't you be doing:
>
>    form = SecretQuestionForm( {"secret_question":"foo", "answer":"bar"} )
>    if form.is_valid():
>        p = form.save(commit=False)
>        p.user = request.user
>        p.primary_contact = somecontact
>        if p.full_validate():        # < note this line
>            p.save()
>
> [snip]

There are a couple of problems with p.full_validate() there. First, it
would run validation a second time. Honza went to a bunch of trouble
in the design to make sure that each field would only need to be
validated once. Second, p.full_validate() would raise
ValidationErrors, not return True or False.

>> Once again, that means ModelForm won't really validate your model,
>> only your form. form.save() might still throw a database error just
>> like before the merge. We can slap a big warning in the ModelForm docs
>> though.
>
> Well, that's why I expected the extra validation check on the model
> itself. I understand the desire to have the ModelForm actually
> validate the model and work in one step, but sometimes we need the
> other way too (as you acknowledge).
>
>> One (probably unhelpful) way to make ModelForm validate your whole
>> model would be to add a Meta flag to ModelForm:
>>
>>    class UserForm(ModelForm):
>>        class Meta:
>>            # Defaults to False
>>            validate_model = True
>
> Well, what if one view uses that ModelForm one way and another view
> uses the same ModelForm the other way? What about
> ``form.is_valid(validate_model=True)``?

That's a possibility, but I think it suffers from the same problems
that the Meta option does. It just pushes the decision closer to
runtime than coding time. That's helpful in some cases, but it doesn't
solve the main part of the problem for me, which is that I don't think
model validation should be an opt-in-only thing. Maybe it needs to be
for a couple of releases though.

I had another idea that I think might work out. What if
ModelForm.validate() only validated fields on the form, like it worked
before the merge, and ModelForm.save() would automatically validate
the excluded fields, raising ValidationError if there was a problem?
Validation for each field would only happen once, it would accommodate
the commit=False idiom, and it would still ensure that the model
itself is validated in common usage.

I think this *might* also lead to a workable solution for ticket
#12507 [1], but I need to dig into the code a little more.

Joseph

[1] http://code.djangoproject.com/ticket/12507
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Łukasz Rekucki]

First time posting here, so please excuse me if my opinions aren't very
useful and my bad English.

2010/1/6 Waylan Limberg 

> I've only scanned the docs the other day and haven't actually used the
> new model validation stuff, so my impressions may be a little off,
> but...
>
> On Wed, Jan 6, 2010 at 2:54 PM, Joseph Kocherhans 
> wrote:
> > On Wed, Jan 6, 2010 at 12:49 PM, Simon Willison 
> wrote:
> >> A couple of related tickets filed today about model validation:
> >>
> >> http://code.djangoproject.com/ticket/12513
> >> http://code.djangoproject.com/ticket/12521
> >>
> >> The first one describes the issue best - the new model validation code
> >> breaks the following common Django convention:
> >>
> >> form = SecretQuestionForm( {"secret_question":"foo", "answer":"bar"} )
> >> if form.is_valid():
> >>p = form.save(commit=False)
> >>p.user = request.user
> >>p.primary_contact = somecontact
> >>p.save()
> >>
>
> My initial reaction to this snippet was to wonder why the model was
> not being validated after the additional data was added/altered.
> Shouldn't you be doing:
>
>form = SecretQuestionForm( {"secret_question":"foo", "answer":"bar"} )
>if form.is_valid():
>p = form.save(commit=False)
>p.user = request.user
>p.primary_contact = somecontact
> if p.full_validate():# < note this line
>p.save()
>
Maybe you could do something like this:

with form.valid_model() as model: # i'm not good at inventing names
model.user = request.user
model.primary_contact = somecontact

The valid_model() would be a context manager that does form validation and
form.save(commit=False) on enter + model validation and save() on exit. Of
course this will only work on Python 2.5+, so it's probably no good for
django 1.2. Just wanted to share an idea.


>
> [snip]
>
> > Once again, that means ModelForm won't really validate your model,
> > only your form. form.save() might still throw a database error just
> > like before the merge. We can slap a big warning in the ModelForm docs
> > though.
>
> Well, that's why I expected the extra validation check on the model
> itself. I understand the desire to have the ModelForm actually
> validate the model and work in one step, but sometimes we need the
> other way too (as you acknowledge).
>
> > One (probably unhelpful) way to make ModelForm validate your whole
> > model would be to add a Meta flag to ModelForm:
> >
> >class UserForm(ModelForm):
> >class Meta:
> ># Defaults to False
> >validate_model = True
>
> Well, what if one view uses that ModelForm one way and another view
> uses the same ModelForm the other way? What about
> ``form.is_valid(validate_model=True)``?
>

This seems like a good idea.


> --
> 
> \X/ /-\ `/ |_ /-\ |\|
> Waylan Limberg
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-develop...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
-- 
Łukasz Rekucki
-- 

You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-develop...@googlegroups.com.

To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Using Django with Jinja2 and TEMPLATE_DEBUG=True

[Rick van Hattem]

Hi,

Maybe this question has already been asked, but I am wondering why Jinja2 
compatibility can't be fixed in a clean way. Currently the code assumes that 
if an exception has a "source" attribute that it's a Django exception and can 
be processed as such. (the code: 
http://code.djangoproject.com/browser/django/trunk/django/views/debug.py#L87 )

However, if all template exceptions have the same base exception (as they 
logically should) wouldn't it automatically work if you simply test for the 
exception?

There has been a previous ticket with a hacky solution: 
http://code.djangoproject.com/ticket/10216

But I feel that testing for specific exceptions (as any try/except should) 
would make the code cleaner and fix this problem. Assuming that any exception 
with a source attribute can be processed as the expected Django exception is 
simply wrong.

Right now you can break the entire debugging system by simply raising an 
exception like this:
class MyException(Exception):
  source = 'Breaking the Django template debugger'

~Rick


signature.asc
Description: This is a digitally signed message part.

Re: Model validation incompatibility with existing Django idioms

[Waylan Limberg]

I've only scanned the docs the other day and haven't actually used the
new model validation stuff, so my impressions may be a little off,
but...

On Wed, Jan 6, 2010 at 2:54 PM, Joseph Kocherhans  wrote:
> On Wed, Jan 6, 2010 at 12:49 PM, Simon Willison  
> wrote:
>> A couple of related tickets filed today about model validation:
>>
>> http://code.djangoproject.com/ticket/12513
>> http://code.djangoproject.com/ticket/12521
>>
>> The first one describes the issue best - the new model validation code
>> breaks the following common Django convention:
>>
>> form = SecretQuestionForm( {"secret_question":"foo", "answer":"bar"} )
>> if form.is_valid():
>>    p = form.save(commit=False)
>>    p.user = request.user
>>    p.primary_contact = somecontact
>>    p.save()
>>

My initial reaction to this snippet was to wonder why the model was
not being validated after the additional data was added/altered.
Shouldn't you be doing:

form = SecretQuestionForm( {"secret_question":"foo", "answer":"bar"} )
if form.is_valid():
p = form.save(commit=False)
p.user = request.user
p.primary_contact = somecontact
if p.full_validate():# < note this line
p.save()

[snip]

> Once again, that means ModelForm won't really validate your model,
> only your form. form.save() might still throw a database error just
> like before the merge. We can slap a big warning in the ModelForm docs
> though.

Well, that's why I expected the extra validation check on the model
itself. I understand the desire to have the ModelForm actually
validate the model and work in one step, but sometimes we need the
other way too (as you acknowledge).

> One (probably unhelpful) way to make ModelForm validate your whole
> model would be to add a Meta flag to ModelForm:
>
>    class UserForm(ModelForm):
>        class Meta:
>            # Defaults to False
>            validate_model = True

Well, what if one view uses that ModelForm one way and another view
uses the same ModelForm the other way? What about
``form.is_valid(validate_model=True)``?

-- 

\X/ /-\ `/ |_ /-\ |\|
Waylan Limberg
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Possible contrib.humanize addition

[SmileyChris]

On Jan 5, 9:24 pm, harrym  wrote:
> I'm working a templatetag that determines whether to use 'a' or 'an'
> in front of English words. My particular use case for this is in a
> tumblelog app I'm developing - many different types of entry may be
> added (link, html, quote, etc), and I'm linking to the 'Add a[n]
>  entry' pages by iterating over the different types. Would this
> be considered a useful addition to contrib.humanize?
>
> The two main reasons against it I see are that firstly, it only works
> for English words, so would be of little use to developers using
> foreign languages, and secondly, it perhaps wouldn't be as widely used
> as the other filters in there.
>
Here's a snippet I wrote a while back you may want to check out too:
www.djangosnippets.org/snippets/1519/
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: cached template loader is new in trunk/1.2, docs does not mention that

[Karen Tracey]

On Wed, Jan 6, 2010 at 12:16 PM, Jeremy Dunck  wrote:

> On Wed, Jan 6, 2010 at 10:24 AM, Amit Upadhyay  wrote:
> > http://docs.djangoproject.com/en/dev/ref/templates/api/#loader-types, I
> > looked and could not find "cached.py"
> > in Django-1.1.1/django/template/loaders.
>
> There is a section for "django.template.loaders.cached.Loader".  Isn't
> that what you mean?
>
>
I think the problem being reported is that is is not marked up as new in
1.2/Development version.

Karen
-- 

You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-develop...@googlegroups.com.

To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Joseph Kocherhans]

On Wed, Jan 6, 2010 at 12:49 PM, Simon Willison  wrote:
> A couple of related tickets filed today about model validation:
>
> http://code.djangoproject.com/ticket/12513
> http://code.djangoproject.com/ticket/12521
>
> The first one describes the issue best - the new model validation code
> breaks the following common Django convention:
>
> form = SecretQuestionForm( {"secret_question":"foo", "answer":"bar"} )
> if form.is_valid():
>    p = form.save(commit=False)
>    p.user = request.user
>    p.primary_contact = somecontact
>    p.save()
>
> The problem is that is_valid() notices that some of the required
> fields in SecretQuestionForm (a ModelForm) have not been provided,
> even if those fields are excluded from validation using the excludes=
> or fields= properties. The exception raised is a
> UnresolvableValidationError.

I'll start off with the reasoning behind the implementation, but I
agree that the current implementation is going to bite too many people
to just call the old implementation a bug.

Form.is_valid() now triggers model validation, and the model isn't
valid. It's missing a couple of required fields. Presenting those
errors to the user filling out the form is unacceptable because there
is nothing the user can do to correct the error, and the developer
will never get a notification about a problem that can only be solved
with code.

> This definitely needs to be fixed as it presumably breaks backwards
> compatibility with lots of existing code (it breaks a common
> ModelAdmin subclass convention as well, see #12521). Can we just
> change the is_valid() logic to ignore model validation errors raised
> against fields which aren't part of the ModelForm, or is it more
> complicated than that?

It shouldn't be much more complicated than that. Model.full_validate()
takes an exclude parameter that we can use to provide a list of fields
that aren't on the form. Or we can catch those errors and just throw
them away. However, this means that part of the problem that
model-validation was meant to solve will no longer be solved.
ModelForm.is_valid() will only mean that your *form* is valid, not
your *model* (which is the pre-merge semantics anyhow).

Once again, that means ModelForm won't really validate your model,
only your form. form.save() might still throw a database error just
like before the merge. We can slap a big warning in the ModelForm docs
though.

One (probably unhelpful) way to make ModelForm validate your whole
model would be to add a Meta flag to ModelForm:

class UserForm(ModelForm):
class Meta:
# Defaults to False
validate_model = True

That would make it easy to trigger model validation, but it doesn't
really help with the convention you're talking about. I don't know. Do
people think triggering model validation in a ModelForm is something
they need to do in a practical sense? Or is validating your form
enough?

Joseph
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Model validation incompatibility with existing Django idioms

[Simon Willison]

A couple of related tickets filed today about model validation:

http://code.djangoproject.com/ticket/12513
http://code.djangoproject.com/ticket/12521

The first one describes the issue best - the new model validation code
breaks the following common Django convention:

form = SecretQuestionForm( {"secret_question":"foo", "answer":"bar"} )
if form.is_valid():
p = form.save(commit=False)
p.user = request.user
p.primary_contact = somecontact
p.save()

The problem is that is_valid() notices that some of the required
fields in SecretQuestionForm (a ModelForm) have not been provided,
even if those fields are excluded from validation using the excludes=
or fields= properties. The exception raised is a
UnresolvableValidationError.

This definitely needs to be fixed as it presumably breaks backwards
compatibility with lots of existing code (it breaks a common
ModelAdmin subclass convention as well, see #12521). Can we just
change the is_valid() logic to ignore model validation errors raised
against fields which aren't part of the ModelForm, or is it more
complicated than that?

Cheers,

Simon
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Design and code review requested for Django string signing / signed cookies

[Luke Plant]

On Wednesday 06 January 2010 17:12:29 Elias Torres wrote:

> > So that would be my defence of why it's better to put the
> > "purpose" namespace into the key, rather than the value, in the
> > context of HMAC. I'm not an expert though.
> 
> Can a separator solve that issue?

In that instance, yes.  I'm wary of other applications of HMAC 
producing loopholes in which the user provides the separator as part 
as the value being signed, and is able to generate the same string. In 
Tornado, they are suggesting have a separate key for signing cookies, 
in which case just signing "name=value" should be enough (provided the 
developer doesn't do something silly like make "=" part of the name of 
the cookie).

Luke

-- 
"Mediocrity: It takes a lot less time, and most people don't 
realise until it's too late." (despair.com)

Luke Plant || http://lukeplant.me.uk/
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: cached template loader is new in trunk/1.2, docs does not mention that

[Jeremy Dunck]

On Wed, Jan 6, 2010 at 10:24 AM, Amit Upadhyay  wrote:
> http://docs.djangoproject.com/en/dev/ref/templates/api/#loader-types, I
> looked and could not find "cached.py"
> in Django-1.1.1/django/template/loaders.

There is a section for "django.template.loaders.cached.Loader".  Isn't
that what you mean?
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Design and code review requested for Django string signing / signed cookies

[Elias Torres]

On Jan 6, 10:37 am, Luke Plant  wrote:
> On Wednesday 06 January 2010 04:24:15 Elias Torres wrote:
> > Thanks Luke for your explanation. I think I have learned something
> > here in terms of my own application security independent of
> >  Django's multi-app environment. Basically, you're reminding me
> >  that as an application, I must be careful to not sign any random
> >  string for a user, because it can be re-used for another purpose.
> >  Therefore, we include the 'purpose', salt, or namespace in the
> >  process. One thing I would like to ask though, is whether the salt
> >  needs to be part of the key or the value? If you look at
> >  TimestampSigner, the timestamp goes as part of the value. I think
> >  the same can be done with the name of the cookie. In other words
> >  you can always use a method like _cookie_signature as in Tornado's
> >  implementation [1] and always pass two parts: cookie name and
> >  cookie value. Technically, as long as your SECRET_KEY is
> >  protected, there should not be a need to creating multiple signing
> >  keys, especially if the data is not secret.
>
> > def _cookie_signature(self, *parts):
> >   hash = hmac.new(SECRET_KEY, digestmod=hashlib.sha1)
> >   for part in parts: hash.update(part)
> >     return hash.hexdigest()
>
> This is equivalent to:
>
>   hmac.new(SECRET_KEY,  
>    digestmod=hashlib.sha1).update("".join(parts)).hexdigest()
>
> With this, one problem is that accidental collisions between different
> parts of code are easier.  Consider two cookies:
>
> Cookie 1, name = "use", value = arbitrary string set by user
> Cookie 2, name = "user_id", value = server assigned once they've
> logged in correctly.
>
> By supplying "r_id123" as the value for cookie 1, I can forge a
> user_id=123 cookie.  If some validation is imposed on cookie 1, it
> might still be possible to manipulate the system such that "r_id123"
> is a valid choice, and the exploit would still work.

Agreed.

>
> Actually, the implementation in Tornado does not include the cookie
> name in the signature at all, making it even more vulnerable to this
> kind of attack.

Absolutely. I mailed the Tornado mailing list last night as well on
the matter.

>
> So that would be my defence of why it's better to put the "purpose"
> namespace into the key, rather than the value, in the context of HMAC.
> I'm not an expert though.

Can a separator solve that issue?

>
> This is one of the tricky things with security - it's never just a
> case of "use this API" - what you feed into the API is always
> critical.

Absolutely. It can become a very powerful black box.

>
> > Any thoughts on Django's auth using HMAC besides md5 and sha1
> >  hashing alone?
>
> It sounds like a good idea, I'm not aware of any particular problems
> with our current implementation that would make this a priority
> change.
>

I think you are right and the 'being able to sign other messages'
attack that HMAC protects us against is not the main issue with
passwords. However, one thing I see some folks moving towards are
slower hashing functions to better protect stolen databases against
dictionary attacks. My friend enlightened me that computing SHA1 HMACs
or hashes it's way too fast so even with a salt, you can still match
many passwords from a stolen database. bcrypt on the other hand is
much slower. But that's just an FYI, not really suggesting to use it.

Thanks for the answers. Every time the subject comes up, I learn
something else about security and passwords.

[1] http://www.mindrot.org/projects/py-bcrypt/


> Luke
>
> --
> "Mediocrity: It takes a lot less time, and most people don't
> realise until it's too late." (despair.com)
>
> Luke Plant ||http://lukeplant.me.uk/
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: cached template loader is new in trunk/1.2, docs does not mention that

[Amit Upadhyay]

Further, cached template loader does not reload template when template is
modified. I am not complaining, just feel that it should me mentioned in the
doc, as normal expectation when developing/deploying django is that one need
not restart apache when template changes, not with cached template loader.

On Wed, Jan 6, 2010 at 9:54 PM, Amit Upadhyay  wrote:

> http://docs.djangoproject.com/en/dev/ref/templates/api/#loader-types, I
> looked and could not find "cached.py"
> in Django-1.1.1/django/template/loaders.
>
> --
> Amit Upadhyay
> www.amitu.com
> +91-9820-295-512
>



-- 
Amit Upadhyay
www.amitu.com
+91-9820-295-512
-- 

You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-develop...@googlegroups.com.

To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

cached template loader is new in trunk/1.2, docs does not mention that

[Amit Upadhyay]

http://docs.djangoproject.com/en/dev/ref/templates/api/#loader-types, I
looked and could not find "cached.py"
in Django-1.1.1/django/template/loaders.

-- 
Amit Upadhyay
www.amitu.com
+91-9820-295-512
-- 

You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-develop...@googlegroups.com.

To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Re: Design and code review requested for Django string signing / signed cookies

[Luke Plant]

On Wednesday 06 January 2010 04:24:15 Elias Torres wrote:

> Thanks Luke for your explanation. I think I have learned something
> here in terms of my own application security independent of
>  Django's multi-app environment. Basically, you're reminding me
>  that as an application, I must be careful to not sign any random
>  string for a user, because it can be re-used for another purpose.
>  Therefore, we include the 'purpose', salt, or namespace in the
>  process. One thing I would like to ask though, is whether the salt
>  needs to be part of the key or the value? If you look at
>  TimestampSigner, the timestamp goes as part of the value. I think
>  the same can be done with the name of the cookie. In other words
>  you can always use a method like _cookie_signature as in Tornado's
>  implementation [1] and always pass two parts: cookie name and
>  cookie value. Technically, as long as your SECRET_KEY is
>  protected, there should not be a need to creating multiple signing
>  keys, especially if the data is not secret.
> 
> def _cookie_signature(self, *parts):
>   hash = hmac.new(SECRET_KEY, digestmod=hashlib.sha1)
>   for part in parts: hash.update(part)
> return hash.hexdigest()

This is equivalent to:

  hmac.new(SECRET_KEY,   
   digestmod=hashlib.sha1).update("".join(parts)).hexdigest()

With this, one problem is that accidental collisions between different 
parts of code are easier.  Consider two cookies:

Cookie 1, name = "use", value = arbitrary string set by user
Cookie 2, name = "user_id", value = server assigned once they've 
logged in correctly.

By supplying "r_id123" as the value for cookie 1, I can forge a 
user_id=123 cookie.  If some validation is imposed on cookie 1, it 
might still be possible to manipulate the system such that "r_id123" 
is a valid choice, and the exploit would still work.

Actually, the implementation in Tornado does not include the cookie 
name in the signature at all, making it even more vulnerable to this 
kind of attack.

So that would be my defence of why it's better to put the "purpose" 
namespace into the key, rather than the value, in the context of HMAC. 
I'm not an expert though.

This is one of the tricky things with security - it's never just a 
case of "use this API" - what you feed into the API is always 
critical.

> Any thoughts on Django's auth using HMAC besides md5 and sha1
>  hashing alone?

It sounds like a good idea, I'm not aware of any particular problems 
with our current implementation that would make this a priority 
change.

Luke

-- 
"Mediocrity: It takes a lot less time, and most people don't 
realise until it's too late." (despair.com)

Luke Plant || http://lukeplant.me.uk/
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Possible contrib.humanize addition

[Chuck Harmston]

More of an academic question, as it likely isn't a feasible solution for
Django, but might a soundex solve this problem? Best I can tell, rules for
articles, without exception, are based on the pronunciation of the following
word..

Of course, phonology can be regional, subjective, and unpredictable. "Wind"
(the flow of gases) and "wind" (circular weaving) are identical to a
template tag but have different vowel sounds. The "a" sound in "bag" is
pronounced much differently in northern Minnesota (where it's bay-g) than
they do in Baltimore.

This feels unsolvable.


On Wed, Jan 6, 2010 at 9:56 AM, Hanne Moa  wrote:

> 2010/1/6 sago :
> >> If you present some research to
> >> demonstrate how this tag could/would work for non-English languages,
> >> it would be a lot more compelling.
> >
> > That's not going to work, in any meaningful sense. That peculiarity of
> > the article is highly English-specific. The generalization would
> > surely be something like
> >
> > {% if /some-regex/.matches(word) %}{{ form1 }} {{ word }}{% else %}
> > {{ form2 }} {{ word }}{% endif %}
>
> Disclaimer: I have a masters degree in Computational Linguistics. Ths
> is a simplified account of  "last year of bachelor"-stuff:
>
> Human language cannot (mathematically proven) be modelled by a mere
> regexp, as human language is not only context-free, (needing a full
> parser) but context-sensitive (needing parsers we don't really have
> yet). Nice, yes?
>
> It cannot go in humanize but it could go in localflavor for English.
> It would be necessary with a stemmer and a replaceable wordlist
> though, as what words get "an" and what get "a" not only depends on
> country but also on specific publishing styles - and all of this has a
> tendency to change over time.
>
>
> HM
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-develop...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>
>
>
-- 

You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-develop...@googlegroups.com.

To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Re: Possible contrib.humanize addition

[Hanne Moa]

2010/1/6 sago :
>> If you present some research to
>> demonstrate how this tag could/would work for non-English languages,
>> it would be a lot more compelling.
>
> That's not going to work, in any meaningful sense. That peculiarity of
> the article is highly English-specific. The generalization would
> surely be something like
>
> {% if /some-regex/.matches(word) %}{{ form1 }} {{ word }}{% else %}
> {{ form2 }} {{ word }}{% endif %}

Disclaimer: I have a masters degree in Computational Linguistics. Ths
is a simplified account of  "last year of bachelor"-stuff:

Human language cannot (mathematically proven) be modelled by a mere
regexp, as human language is not only context-free, (needing a full
parser) but context-sensitive (needing parsers we don't really have
yet). Nice, yes?

It cannot go in humanize but it could go in localflavor for English.
It would be necessary with a stemmer and a replaceable wordlist
though, as what words get "an" and what get "a" not only depends on
country but also on specific publishing styles - and all of this has a
tendency to change over time.


HM
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Possible contrib.humanize addition

[harrym]

The code I've got so far works pretty well - I've tested it on some
medium-sized corpora and the only times the expected result was
different from the actual result was when the corpus was wrong. The
code works by first checking a few specific rules for numbers and
acromyns, then checking against a few exceptional cases (word
prefixes), then checking whether the word starts with a vowel. Most of
the rules came from some Perl code I found a while a go - just ported
them over to Python.

But I agree that this would be far too difficult ( / impossible) to
make multi-lingual so is perhaps not appropriate for inclusion in
Django.

Harry

On Jan 6, 2:17 pm, sago  wrote:
> > Hmm, can it handle the following?
>
> >  an honest man
> >  a history book
> >  an historical book (debatable)
>
> It can't, the rules for the indefinite article around 'h' are complex
> and depend on the etymology of the word used. To add complexity the
> lexicographic rules are often different to the rules for speech, and
> UK rules differ from US rules (and possibly Oz too, but I don't
> know).
>
> > If you present some research to
> > demonstrate how this tag could/would work for non-English languages,
> > it would be a lot more compelling.
>
> That's not going to work, in any meaningful sense. That peculiarity of
> the article is highly English-specific. The generalization would
> surely be something like
>
> {% if /some-regex/.matches(word) %}{{ form1 }} {{ word }}{% else %}
> {{ form2 }} {{ word }}{% endif %}
>
> where the regex is language and context dependent. There are various
> regex replacement filters/tags out in the djangosphere. Could you use
> one of them?
>
> > (That's NT Koine Greek, it might be different/simpler/more complicated
> > in modern Greek).
>
> What is it about Django and NT scholars - have you come across James
> Tauber (of Pinax fame?)
>
> Ian.
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Possible contrib.humanize addition

[James Bennett]

On Wed, Jan 6, 2010 at 8:17 AM, sago  wrote:
> What is it about Django and NT scholars - have you come across James
> Tauber (of Pinax fame?)

There are at least three Django committers who can list one or another
ancient Greek dialect among their studies. Not sure why that is, but
it does make for fun conversation over drinks.


-- 
"Bureaucrat Conrad, you are technically correct -- the best kind of correct."
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Possible contrib.humanize addition

[sago]

> Hmm, can it handle the following?
>
>  an honest man
>  a history book
>  an historical book (debatable)

It can't, the rules for the indefinite article around 'h' are complex
and depend on the etymology of the word used. To add complexity the
lexicographic rules are often different to the rules for speech, and
UK rules differ from US rules (and possibly Oz too, but I don't
know).

> If you present some research to
> demonstrate how this tag could/would work for non-English languages,
> it would be a lot more compelling.

That's not going to work, in any meaningful sense. That peculiarity of
the article is highly English-specific. The generalization would
surely be something like

{% if /some-regex/.matches(word) %}{{ form1 }} {{ word }}{% else %}
{{ form2 }} {{ word }}{% endif %}

where the regex is language and context dependent. There are various
regex replacement filters/tags out in the djangosphere. Could you use
one of them?

> (That's NT Koine Greek, it might be different/simpler/more complicated
> in modern Greek).
What is it about Django and NT scholars - have you come across James
Tauber (of Pinax fame?)

Ian.
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: WSGI support in Django

[Evert Carton]

Hi, I only drop in here every now and then. 
It may be worthwhile to mention that I never got the WSGi support fully working 
under Jython. 
I played with it under Geronimo and Tomcat. The mechanism used here is WSGI.

The issue is not with Jython but with the way servlets handle POST data. The 
aforementioned containers actually completely gobble up the POST-data (which is 
available to the servlet in an (java) inputStream, parse it into parameters, 
meaning the POST-data cannot be send forward to the WSGI adapter for parsing in 
Django. 
I've also had similiar issues with file-uploads, the issue being there blocking 
and non-blocking I/O.

Since I stumbled upon this discussion about WSGI 

DISCLAIMER:
It may have been a faulty configuration of mine, but I don't think it was, it 
may have been a problem for which there is allready a solution ... I don't 
know, and I don't think so.

Just wanted to share this ... I didn't investigate further, I did find a very 
nasty workaround that actually suited my needs better than actually fixing it.





From: Russell Keith-Magee 
To: django-developers@googlegroups.com
Sent: Tue, 5 January, 2010 0:44:57
Subject: Re: WSGI support in Django

On Mon, Jan 4, 2010 at 11:46 PM, Gustavo Narea
 wrote:
> Hi,
>
> One more update about the WSGI related improvements for Django:
>
> I have created a Mercurial branch to keep track of these changes and
> keep them synchronized with trunk:
> http://bitbucket.org/Gustavo/django-wsgi/
>
> Even though I know it's late for 1.2 at this point, please keep in mind
> that part of these enhancements were supposed to be a high priority for
> the 1.2 release [1] (GSoC-2). This branch implements more features than
> the official http-wsgi-improvements branch and is complete/working. And
> again, I am willing to help you with anything you need to get it merged,
> such as writing docs and discussing the implementation further.

There's a very important reason why the feature list is called 'High
Priority" and not "Must Have". We're all volunteers, so it's
impossible for us to know exactly how much time we will have to
dedicate to Django. Therefore, our goals need to be (and are)
expressed in terms of relative priorities, rather than in terms of
absolutes.

Malcolm was the champion behind GSOC-2; unfortunately, Malcolm has
been buried under real-life and work stuff, so he hasn't been able to
shepherd the GSOC-2 changes into trunk. This is unfortunate, but there
isn't much we can do about this.

Yours,
Russ Magee %-)

--

You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.-- 

You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-develop...@googlegroups.com.

To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Re: Possible contrib.humanize addition

[Luke Plant]

On Tuesday 05 January 2010 21:24:13 harrym wrote:
> I'm working a templatetag that determines whether to use 'a' or
>  'an' in front of English words. My particular use case for this is
>  in a tumblelog app I'm developing - many different types of entry
>  may be added (link, html, quote, etc), and I'm linking to the 'Add
>  a[n]  entry' pages by iterating over the different types.
>  Would this be considered a useful addition to contrib.humanize?

Hmm, can it handle the following?

 an honest man
 a history book
 an historical book (debatable)

My gut instinct is that it's not possible to work this out 
programmatically.  When it comes to other languages, I imagine it's 
going to be even harder (if it's possible to get harder than 
'impossible'), because you have things like gender and case to worry 
about, which certainly cannot be worked out by an algorithm.

To give some examples, in French, the choice is between 'un' and 
'une', depending on whether the word is masculine or feminine.  In 
Greek, the choice is between  ̔εις, ̔ενα, ̔ενος, ̔ενι, μια, μιαν, 
μιας, μια, ̔εν, ̔εν, ̔ενος, ̔ενι, depending on whether the word is 
masculine, feminine or neuter, and in nominative, accusative, genitive 
or dative case. Although in many cases you would probably omit the 
article altogether - the above words often mean "one" rather than "a".
(That's NT Koine Greek, it might be different/simpler/more complicated 
in modern Greek).

I imagine there are plenty of languages where this gets even worse, 
violating almost every assumption you don't even know you are making 
(like whether the article comes before or after or in the middle, or 
exists at all, etc. etc.)

To summarise: if I were you, I would give up now.

Luke

-- 
"Mediocrity: It takes a lot less time, and most people don't 
realise until it's too late." (despair.com)

Luke Plant || http://lukeplant.me.uk/
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.