Re: Call for comment: #5390 Many To Many signals

[夏恺]

Hi,

First thing first, I shall explain why this have something to do with admin.

Django's admin save m2m relationship in ModelAdmin's change_view method.
After some tracing, you would find that the m2m relationship is saved using
ModelForm's save_m2m method, which calls a method named 'save_form_data'
in django/db/models/fields/related.py. In this 'save_form_data' method, a 
setattr

is used and therefore the __set__ method in the ManyRelatedObjectsDescriptor
is used.

So, when someone save an m2m relationship in Django's admin, he/she is not
using the add/remove/clear methods directly. Instead, this operation is 
handled

by the mentioned __set__ method, which calls the add/remove/clear methods.

Before we add this m2m signal, the __set__ method works alright. It's
straightforward, simple and pythonic. But after we have introduced this m2m 
signal.

IMHO it would not work as everyone expected.

In your example, the author books relationship is changed, and a new book is
associated with this user. Let's suppose that this is done through django's 
admin.

author.books = [b1,b2,...b100]
author.books = [b1,b2,...b100, b101]


Logically, since there's only one book being added, I think I'm not alone in 
expecting
only one signal being issued in this case. but in your implementation there 
are two
signals issued. And as was mentioned in my last email, the reverse case is 
even worse,

a signal with action 'remove' would never be issued.

To summarize the possible cases in a table:

  Current __set__ method our __set__ method
Case 1:clear b1, b2 and add b1, b3   remove b2 and add b3
Case 2:clear b1 and add b1, b2add b2
Case 3:clear b1, b2 and add b1remove b2
Case 4:clear b1, b2  clear b1, b2

Case 1: [b1, b2] -> [b1, b3]
Case 2: [b1]   -> [b1, b2]
Case 3: [b1, b2] -> [b1]
Case 4: [b1, b2] -> []

I think in all cases our __set__ method would be more logical and 
reasonable.


As to performance, I think we shall make it logical before talking about 
efficiency.


If you insist that the __set__ method is not to be changed in this patch, I 
would agree.
But at least, we should add some explanatory text in the documentation 
before we

finally fix the __set__ method.


Xia Kai(夏恺)
xia...@gmail.com
http://blog.xiaket.org

--
From: "Russell Keith-Magee" 
Sent: Saturday, January 09, 2010 1:26 PM
To: 
Subject: Re: Call for comment: #5390 Many To Many signals


On Sat, Jan 9, 2010 at 9:47 AM, Xia Kai(夏恺)  wrote:

Hi all,


It isn't as simple as saying that it is "flawed" - it just doesn't
work quite as you perhaps expected. It don't think it's especially
illogical, either - in order to assign a new set, you have to clear
the old set, which is exactly what the current implementation does.

I'm also not clear what this has to do with the admin. There is
nothing about ManyRelatedObjectsDescriptor that is admin-specific.


The approach you describe sends two signals as well. One indicating
the objects that have been removed, and one that indicates the objects
that have been added.

I don't accept your argument that a "remove" action is "correct" -
it's just a different way of describing the same change.


Again, issuing a 'clear' isn't *incorrect*, it's just contrary to your
expectations.

Appropriate is a different argument. The approach you are advocating
does have one advantage - the database change implemented (and thus
the signals issued) will represent the delta of the change, rather
than a complete clear/re-add. This means that in the case of::

author.books = [b1,b2,...b100]
author.books = [b1,b2,...b100, b101]

your code will perform one insertion, whereas the existing code will
do 100 removals and 101 insertions. However, this is at the cost of
retrieving 100 items, constructing a set, and doing a set difference.
This retrieval and set difference is not a free operation - there is a
cost in both CPU and database load. Whether this is an net performance
gain is not immediately obvious, although I suspect there will be a
gain if the assignment set is large, but has a small delta. I'd need
to see some concrete performance numbers to convince me. Those
performance numbers would need to cover at least 4 cases:

   * large assignment, small delta
   * large assignment, large delta
   * small assignment, small delta
   * small assignment, large delta

The clear() problem isn't the only problem with the implementation you
propose. __set__ can accept a list of ids or a list of objects. Your
implementation requires that a list of objects has been assigned.
Ironically, if you provide a list of ids, the code still works - it
just reverts to what you refer to as "incorrect" behavior; that -
removing all the old objects, then adding new objects. However, it
does this by removing each object individually, not clearing the
o

Re: Is a partial translation acceptable?

[Russell Keith-Magee]

On Sat, Jan 9, 2010 at 8:23 AM, Jeff Balogh  wrote:
> How do you guys feel about getting an incomplete translation for some
> locales?  We might want to support some locales that Django doesn't
> have yet, but I'm trying not to overload our localizers with all the
> strings in Django (they're already sick of us).
>
> I thought that something like http://gist.github.com/272594 looked
> like a good start for them, with the technical strings and form
> validation parts.
>
> I'm inquiring so I can figure out what to do with translations when
> (if) I get them.

Obviously, we'd prefer complete translations, but a partial
translation is better than nothing - especially if you're
cherry-picking the most common strings that are seen in production. If
you have the resources to provide a bunch of initial translations for
locales we don't currently support at all, we're more than happy to
take them.

Yours,
Russ Magee %-)
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Call for comment: #5390 Many To Many signals

[Russell Keith-Magee]

On Sat, Jan 9, 2010 at 9:47 AM, Xia Kai(夏恺)  wrote:
> Hi all,
>
> IMHO this patch would still make django issue illogical signals in admin.
> frans and I have pointed out that the __set__ method in the
> ManyRelatedObjectsDescriptor is flawed[1-2], which is used by admin.

It isn't as simple as saying that it is "flawed" - it just doesn't
work quite as you perhaps expected. It don't think it's especially
illogical, either - in order to assign a new set, you have to clear
the old set, which is exactly what the current implementation does.

I'm also not clear what this has to do with the admin. There is
nothing about ManyRelatedObjectsDescriptor that is admin-specific.

> The __set__ method would save a many-to-many relationship by clearing all
> entries in a field and then reinserting them back. This would work in the
> old days when no many-to-many signal is around. However, with the addition
> of this signal, all saving through admin would issue two signals: first, a
> signal with action 'clear', and then, a signal with action 'add'. Signal
> with remove action would never be correctly issued.

The approach you describe sends two signals as well. One indicating
the objects that have been removed, and one that indicates the objects
that have been added.

I don't accept your argument that a "remove" action is "correct" -
it's just a different way of describing the same change.

> frans and I had provided some patches to work this out. However, I'm afraid
> my last solution in [2] would incorrectly issue signal with 'clear' action.
> A more appriate solution would be:
>
>       manager = self.__get__(instance)
>       previous=set(manager.all())
>       new=set(value)
>       if previous and not new:       # correction against [2] here.
>           manager.clear()
>       else:
>           added = new - previous
>           removed = previous - new
>           if removed:
>               manager.remove(*removed)
>           if added:
>               manager.add(*added)

Again, issuing a 'clear' isn't *incorrect*, it's just contrary to your
expectations.

Appropriate is a different argument. The approach you are advocating
does have one advantage - the database change implemented (and thus
the signals issued) will represent the delta of the change, rather
than a complete clear/re-add. This means that in the case of::

 author.books = [b1,b2,...b100]
 author.books = [b1,b2,...b100, b101]

your code will perform one insertion, whereas the existing code will
do 100 removals and 101 insertions. However, this is at the cost of
retrieving 100 items, constructing a set, and doing a set difference.
This retrieval and set difference is not a free operation - there is a
cost in both CPU and database load. Whether this is an net performance
gain is not immediately obvious, although I suspect there will be a
gain if the assignment set is large, but has a small delta. I'd need
to see some concrete performance numbers to convince me. Those
performance numbers would need to cover at least 4 cases:

* large assignment, small delta
* large assignment, large delta
* small assignment, small delta
* small assignment, large delta

The clear() problem isn't the only problem with the implementation you
propose. __set__ can accept a list of ids or a list of objects. Your
implementation requires that a list of objects has been assigned.
Ironically, if you provide a list of ids, the code still works - it
just reverts to what you refer to as "incorrect" behavior; that -
removing all the old objects, then adding new objects. However, it
does this by removing each object individually, not clearing the
objects as a whole.

In summary, the current behavior isn't wrong - it's just contrary to
your expectations. The change you are proposing is a feature request
that is independent of the desire to add m2m signals. The change you
propose may well be a performance improvement, especially for large
assignments with small deltas (I can think of a situation in code I
have in production where it would be a gain). The benefit for small
assignments will not be as obvious.

If you want to pursue this change:
 * Open a new ticket
 * Provide an implementation that fixes the problem I have identified
 * Perform some performance numbers to convince me that this is a net gain.

Yours,
Russ Magee %-)
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Call for comment: #5390 Many To Many signals

[夏恺]

Hi all,

IMHO this patch would still make django issue illogical signals in admin. 
frans and I have pointed out that the __set__ method in the 
ManyRelatedObjectsDescriptor is flawed[1-2], which is used by admin.


The __set__ method would save a many-to-many relationship by clearing all 
entries in a field and then reinserting them back. This would work in the 
old days when no many-to-many signal is around. However, with the addition 
of this signal, all saving through admin would issue two signals: first, a 
signal with action 'clear', and then, a signal with action 'add'. Signal 
with remove action would never be correctly issued.


frans and I had provided some patches to work this out. However, I'm afraid 
my last solution in [2] would incorrectly issue signal with 'clear' action. 
A more appriate solution would be:


   manager = self.__get__(instance)
   previous=set(manager.all())
   new=set(value)
   if previous and not new:   # correction against [2] here.
   manager.clear()
   else:
   added = new - previous
   removed = previous - new
   if removed:
   manager.remove(*removed)
   if added:
   manager.add(*added)


[1]: http://code.djangoproject.com/ticket/5390#comment:40
[2]: http://code.djangoproject.com/ticket/5390#comment:55

Xia Kai(夏恺)
xia...@gmail.com
http://blog.xiaket.org

--
From: "Russell Keith-Magee" 
Sent: Friday, January 08, 2010 8:04 PM
To: "Django Developers" 
Subject: Call for comment: #5390 Many To Many signals


Hi all,

I've just uploaded a patch for #5390, which adds signals to
many-to-many modifying operations.

This is a ticket that was a low-priority accepted item for 1.1
(ORM-18), but got bumped due to time [1]. For some reason, it wasn't
resubmitted for 1.2.

The purpose of this post is twofold:

1) To establish if there are any objections to introducing this
signal. Historically, there has been a performance objection to adding
new signals, so I want to give some numbers of the performance impact
and see if anyone objects. and

2) To get feedback on the implementation.

Barring objections or major design changes, I'd like to commit this
early next week.

1. Performance
--

The last time signals came up [2], Jacob provided some helpful
baseline data on the speed of signals:

Doing Nothing: 0.00283 (0.00028 per call)
Plain func call: 0.07054 (0.00705 per call)
Signal; 0 handlers: 0.09331 (0.00933 per call)
Signal; 1 handler : 0.79125 (0.07912 per call)
Signal; 10 handlers: 3.36051 (0.33605 per call)
Signal; 100 handlers: 27.56269 (0.000275627 per call)

* In Python, calling a function (``handle()``) takes 25 times as long
as doing nothing (``pass``).
* Dispatching a signal when there's no handlers is about 1.3 times a
function call.
* Dispatching a signal when there's one handler is about 11 times the
cost of a function call.
* Addition of additional receivers scales linearly.

The other (completely unscientific) test is the Django test suite
itself. I've run the test suite with and without the patch from #5390.
What I have observed is that the discrepancies in test runtime caused
by normal fluctuations in background system load are more significant
than the slowdown caused by adding m2m signals (that is - the vanilla
suite sometimes takes *longer* than the suite with the patch applied).

So - based on my this analysis, there is a performance impact, but it
isn't especially significant or noticeable.

2. Implementation
-

The patch I have uploaded [3] introduces a single new signal -
m2m_changed. This decision is based on design feedback from Malcolm
[4].

The signal is attached to the m2m through class (the automatically
generated m2m class that is the result of the m2m refactor). This
class is the 'sender' of the signal.

The signal handler is provided with the sender, as well as:
* The instance that contains the m2m field that is being modified
* A string tag indicating if this is an "add", "remove" or "clear" call
* A boolean indicating whether this is the 'forward' or 'reverse' 
relation.

* The class of model that is being added/removed/cleared
* The primary keys of the instances that are being added to the
relation (or None in the case of a call to clear the relation)

The docs and tests on the patch [3] give a more detailed description
and examples of the operation of the signal.

Any feedback on this design?

[1] http://code.djangoproject.com/wiki/Version1.1Features
[2] 
http://groups.google.com/group/django-developers/browse_thread/thread/5a39839e2aa8df53
[3] 
http://code.djangoproject.com/attachment/ticket/5390/t5390-r12120.1.diff
[4] 
http://groups.google.com/group/django-developers/browse_thread/thread/afe6ad7994d868ba


Yours,
Russ Magee %-) 


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
T

Is a partial translation acceptable?

[Jeff Balogh]

How do you guys feel about getting an incomplete translation for some
locales?  We might want to support some locales that Django doesn't
have yet, but I'm trying not to overload our localizers with all the
strings in Django (they're already sick of us).

I thought that something like http://gist.github.com/272594 looked
like a good start for them, with the technical strings and form
validation parts.

I'm inquiring so I can figure out what to do with translations when
(if) I get them.

Cheers,
jeff
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Porting Django to Python 3

[Russell Keith-Magee]

On Sat, Jan 9, 2010 at 2:25 AM, Dave  wrote:
> Hello everyone,
>
> My name is Dave Weber, and I'm a student at the University of Toronto,
> studying Computer Science. For one of our undergraduate courses led by
> Greg Wilson (http://www.cs.utoronto.ca/~gvwilson/), myself and a group
> of 10 other computer science students will be trying to port Django to
> Python 3.
>
> Until the end of January, we'll be studying the existing Django code
> in order to gain an understanding of how the program works. We'll be
> doing this primarily through architecture documentation and
> performance profiling. In early February we plan on beginning work on
> the port.
>
> A few of us have experience working with Django, and by the end of
> January we should have a much better understanding of it. I've been in
> touch with Jacob Kaplan-Moss, who pointed me to this group, and he
> also provided me with links about contributing to the Django project
> and Martin van Lowis' port.
>
> We don't really have any specific questions right now as we're pretty
> unfamiliar with most of the project at this point in time. However, we
> are very eager to learn as much as we can, so if you have any advice,
> warnings, or anything at all to say to us, please feel free! We'd like
> to hear from all of you as much as possible.

Hi Dave,

Sounds like an interesting project!

My best piece of advice would be to learn to love the test suite.
Django's test suite may take a long time to run, but it is quite
comprehensive, and has enabled us to complete several large internal
refactoring projects with a minimum of impact on the general user
community.

My other advice would be to get involved in the community. Don't just
treat your Python 3 port as "your CS project", independent of the rest
of the world. For example, if your porting efforts discovers a section
of code that isn't tested (or tested well), or you discover a simple
fix that will boost performance, don't be a stranger - submit a patch
and help us make Django better.

This even extends to documentation - if your porting efforts generate
architecture documentation that might be useful to the general
community, we'd love to have that contributed back to the community.

Best of luck with your project. I can't wait to see what you come up with :-)

Yours,
Russ Magee %-)
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Porting Django to Python 3

[VernonCole]

Dave:
  Wonderful!  I am presently working on a project to get adodbapi
(http://sourceforge.net/projects/adodbapi) working in django.  That
may be important to you since it is one of few db interfaces which has
a working python 3 version for Windows. Keep in touch.
--
Vernon Cole

On Jan 8, 11:25 am, Dave  wrote:
> Hello everyone,
>
> My name is Dave Weber, and I'm a student at the University of Toronto,
> studying Computer Science. For one of our undergraduate courses led by
> Greg Wilson (http://www.cs.utoronto.ca/~gvwilson/), myself and a group
> of 10 other computer science students will be trying to port Django to
> Python 3.
>
> Until the end of January, we'll be studying the existing Django code
> in order to gain an understanding of how the program works. We'll be
> doing this primarily through architecture documentation and
> performance profiling. In early February we plan on beginning work on
> the port.
>
> A few of us have experience working with Django, and by the end of
> January we should have a much better understanding of it. I've been in
> touch with Jacob Kaplan-Moss, who pointed me to this group, and he
> also provided me with links about contributing to the Django project
> and Martin van Lowis' port.
>
> We don't really have any specific questions right now as we're pretty
> unfamiliar with most of the project at this point in time. However, we
> are very eager to learn as much as we can, so if you have any advice,
> warnings, or anything at all to say to us, please feel free! We'd like
> to hear from all of you as much as possible.
>
> Best regards,
>
> Dave Weber
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Requesting help - validation on inlineformset_factory instance with invalid data in email field hanging application.

[Rebekah]

Thanks Karen and Robert, you both rock!  I put in the patch and it's
fixed :)

On Jan 8, 1:53 pm, Karen Tracey  wrote:
> On Fri, Jan 8, 2010 at 1:42 PM, Rebekah  wrote:
> > I've searched but don't seem to see anyone else experiencing this
> > issue.  I think it must be a django bug, I just want to know is this a
> > known issue, and if so, if there is a patch already in the works.
>
> > I have a view which displays a form and inline formset for user and
> > profile model data. The problem I am having is with the email field
> > validation. For simple problems, it works just fine (e.g. entering
> > "becky" will return a validation error, but entering
> > "be...@example.com" is valid).  But when I was testing, I discovered
> > that entering "be...@instansa.comm" causes the
> > application to hang, requiring a restart of the web server. There are
> > no errors, it just looks like it's stuck in a loop. This occurs
> > somewhere within the is_valid() call on the profile formset.
>
> What level of Django are you running?  This sounds much like the problem
> that triggered the release of 1.1.1 and 1.0.4:
>
> http://www.djangoproject.com/weblog/2009/oct/09/security/
>
> So if you are running anything earlier than those, the first step would be
> to update to a release that contains that fix.
>
> Karen
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Porting Django to Python 3

[Jerome Leclanche]

Best of luck in your port.

On that note, I'm hoping when the 3k port will be officially supported, it
will not be backwards compatible. The core idea of 3k itself is the lack of
backwards compatibility ...

J. Leclanche / Adys


On Fri, Jan 8, 2010 at 8:25 PM, Dave  wrote:

> Hello everyone,
>
> My name is Dave Weber, and I'm a student at the University of Toronto,
> studying Computer Science. For one of our undergraduate courses led by
> Greg Wilson (http://www.cs.utoronto.ca/~gvwilson/), myself and a group
> of 10 other computer science students will be trying to port Django to
> Python 3.
>
> Until the end of January, we'll be studying the existing Django code
> in order to gain an understanding of how the program works. We'll be
> doing this primarily through architecture documentation and
> performance profiling. In early February we plan on beginning work on
> the port.
>
> A few of us have experience working with Django, and by the end of
> January we should have a much better understanding of it. I've been in
> touch with Jacob Kaplan-Moss, who pointed me to this group, and he
> also provided me with links about contributing to the Django project
> and Martin van Lowis' port.
>
> We don't really have any specific questions right now as we're pretty
> unfamiliar with most of the project at this point in time. However, we
> are very eager to learn as much as we can, so if you have any advice,
> warnings, or anything at all to say to us, please feel free! We'd like
> to hear from all of you as much as possible.
>
> Best regards,
>
> Dave Weber
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-develop...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>
>
>
-- 

You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-develop...@googlegroups.com.

To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Re: Requesting help - validation on inlineformset_factory instance with invalid data in email field hanging application.

[Karen Tracey]

On Fri, Jan 8, 2010 at 1:42 PM, Rebekah  wrote:

> I've searched but don't seem to see anyone else experiencing this
> issue.  I think it must be a django bug, I just want to know is this a
> known issue, and if so, if there is a patch already in the works.
>
> I have a view which displays a form and inline formset for user and
> profile model data. The problem I am having is with the email field
> validation. For simple problems, it works just fine (e.g. entering
> "becky" will return a validation error, but entering
> "be...@example.com" is valid).  But when I was testing, I discovered
> that entering "be...@instansa.comm" causes the
> application to hang, requiring a restart of the web server. There are
> no errors, it just looks like it's stuck in a loop. This occurs
> somewhere within the is_valid() call on the profile formset.
>
>
What level of Django are you running?  This sounds much like the problem
that triggered the release of 1.1.1 and 1.0.4:

http://www.djangoproject.com/weblog/2009/oct/09/security/

So if you are running anything earlier than those, the first step would be
to update to a release that contains that fix.

Karen
-- 

You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-develop...@googlegroups.com.

To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Re: Requesting help - validation on inlineformset_factory instance with invalid data in email field hanging application.

[Robert Eanes]

Sounds like you are getting caught by this issue: 
http://www.djangoproject.com/weblog/2009/oct/09/security/
You should make sure you are using django 1.1.1 or 1.0.4, which have fixes for 
it.

On Jan 8, 2010, at 1:42 PM, Rebekah wrote:

> Hi,
> 
> I've searched but don't seem to see anyone else experiencing this
> issue.  I think it must be a django bug, I just want to know is this a
> known issue, and if so, if there is a patch already in the works.
> 
> I have a view which displays a form and inline formset for user and
> profile model data. The problem I am having is with the email field
> validation. For simple problems, it works just fine (e.g. entering
> "becky" will return a validation error, but entering
> "be...@example.com" is valid).  But when I was testing, I discovered
> that entering "be...@instansa.comm" causes the
> application to hang, requiring a restart of the web server. There are
> no errors, it just looks like it's stuck in a loop. This occurs
> somewhere within the is_valid() call on the profile formset.
> 
> Here's the relevant part of the view code:
> 
>user = User.objects.get(pk=request.user.pk)
>user_profile = UserProfile.objects.get_or_create(user = user)
>ProfileInlineFormset = inlineformset_factory(User,
> UserProfile,
> can_delete=False,
> 
> form=instansa_forms.ProfileForm)
>rDict = {}
>rDict['user'] = user
>rDict['title'] = 'Settings'
>rDict['request'] = request
> 
>error = None
>message = None
> 
>if request.method == "POST":
>print "this is a post"
>SettingsForm = instansa_forms.SettingsForm(request.POST,
> instance=user)
>ProfileFormset = ProfileInlineFormset(request.POST,
> request.FILES, instance=user)
> 
>print "about to validate"
>if SettingsForm.is_valid():
>print "valid settings"
>if ProfileFormset.is_valid():
>print "valid profile"
> 
> This is the relevant part of forms.py:
> 
> class ProfileForm(forms.ModelForm):
>email_to_display = forms.EmailField(required=False, max_length=75,
> widget=forms.TextInput(attrs={'size':'75'}), help_text='This email
> address will be displayed as a link on your Ask page.')
>phone_to_display = forms.CharField(required=False, help_text='This
> phone number will be displayed on your Ask page.')
>phone_admin = forms.CharField(required=False, label='Phone for
> admin', help_text='This phone number is not displayed. For use only by
> Instansa in administration of your account.')
>company_name = forms.CharField(required=False,
> widget=forms.TextInput(attrs={'size':'50'}))
>url = forms.URLField(required=False, widget=forms.TextInput(attrs=
> {'size':'50'}), label='Website URL', help_text='A link to this URL
> will be displayed on your Ask page to direct your customers back to
> your website.')
>ask_prompt = forms.CharField(required=False, widget=forms.TextInput
> (attrs={'size':'50'}), help_text='100 characters or fewer. A helpful
> prompt displayed above the question field on your Ask page to help
> users.')
>class Meta:
>model = UserProfile
>fields=
> ('email_to_display','phone_admin','phone_to_display','company_name','url','ask_prompt')
> 
> and this is how the email form is set up in models.py (though this
> error occurs when validating the standard user email as well.)
> 
>email_to_display = models.EmailField(blank=True, max_length=75)
> 
> Any insight into this problem, and a simple workaround will be greatly
> appreciated!  Forgive me if I have omitted any useful information.
> 
> Thanks,
> Becky
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-develop...@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
> 
> 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Requesting help - validation on inlineformset_factory instance with invalid data in email field hanging application.

[Rebekah]

Hi,

I've searched but don't seem to see anyone else experiencing this
issue.  I think it must be a django bug, I just want to know is this a
known issue, and if so, if there is a patch already in the works.

I have a view which displays a form and inline formset for user and
profile model data. The problem I am having is with the email field
validation. For simple problems, it works just fine (e.g. entering
"becky" will return a validation error, but entering
"be...@example.com" is valid).  But when I was testing, I discovered
that entering "be...@instansa.comm" causes the
application to hang, requiring a restart of the web server. There are
no errors, it just looks like it's stuck in a loop. This occurs
somewhere within the is_valid() call on the profile formset.

Here's the relevant part of the view code:

user = User.objects.get(pk=request.user.pk)
user_profile = UserProfile.objects.get_or_create(user = user)
ProfileInlineFormset = inlineformset_factory(User,
 UserProfile,
 can_delete=False,
 
form=instansa_forms.ProfileForm)
rDict = {}
rDict['user'] = user
rDict['title'] = 'Settings'
rDict['request'] = request

error = None
message = None

if request.method == "POST":
print "this is a post"
SettingsForm = instansa_forms.SettingsForm(request.POST,
instance=user)
ProfileFormset = ProfileInlineFormset(request.POST,
request.FILES, instance=user)

print "about to validate"
if SettingsForm.is_valid():
print "valid settings"
if ProfileFormset.is_valid():
print "valid profile"

This is the relevant part of forms.py:

class ProfileForm(forms.ModelForm):
email_to_display = forms.EmailField(required=False, max_length=75,
widget=forms.TextInput(attrs={'size':'75'}), help_text='This email
address will be displayed as a link on your Ask page.')
phone_to_display = forms.CharField(required=False, help_text='This
phone number will be displayed on your Ask page.')
phone_admin = forms.CharField(required=False, label='Phone for
admin', help_text='This phone number is not displayed. For use only by
Instansa in administration of your account.')
company_name = forms.CharField(required=False,
widget=forms.TextInput(attrs={'size':'50'}))
url = forms.URLField(required=False, widget=forms.TextInput(attrs=
{'size':'50'}), label='Website URL', help_text='A link to this URL
will be displayed on your Ask page to direct your customers back to
your website.')
ask_prompt = forms.CharField(required=False, widget=forms.TextInput
(attrs={'size':'50'}), help_text='100 characters or fewer. A helpful
prompt displayed above the question field on your Ask page to help
users.')
class Meta:
model = UserProfile
fields=
('email_to_display','phone_admin','phone_to_display','company_name','url','ask_prompt')

and this is how the email form is set up in models.py (though this
error occurs when validating the standard user email as well.)

email_to_display = models.EmailField(blank=True, max_length=75)

Any insight into this problem, and a simple workaround will be greatly
appreciated!  Forgive me if I have omitted any useful information.

Thanks,
Becky
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Porting Django to Python 3

[Dave]

Hello everyone,

My name is Dave Weber, and I'm a student at the University of Toronto,
studying Computer Science. For one of our undergraduate courses led by
Greg Wilson (http://www.cs.utoronto.ca/~gvwilson/), myself and a group
of 10 other computer science students will be trying to port Django to
Python 3.

Until the end of January, we'll be studying the existing Django code
in order to gain an understanding of how the program works. We'll be
doing this primarily through architecture documentation and
performance profiling. In early February we plan on beginning work on
the port.

A few of us have experience working with Django, and by the end of
January we should have a much better understanding of it. I've been in
touch with Jacob Kaplan-Moss, who pointed me to this group, and he
also provided me with links about contributing to the Django project
and Martin van Lowis' port.

We don't really have any specific questions right now as we're pretty
unfamiliar with most of the project at this point in time. However, we
are very eager to learn as much as we can, so if you have any advice,
warnings, or anything at all to say to us, please feel free! We'd like
to hear from all of you as much as possible.

Best regards,

Dave Weber
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Final call for feedback: Multi-db

[Brett Hoerner]

On Jan 7, 7:33 pm, Russell Keith-Magee  wrote:
> then my understanding of your proposal is that the only change is that
> read-slave won't get created under the test setup. But doesn't that
> mean that::
>
>     MyModel.objects.using('read-slave').filter(...)
>
> will fall over?

No, not in my mental image of the setup.  Take the following,

DEFAULT_ENGINE = 'postgresql_psycopg2'
DEFAULT_NAME = 'my_database'

DATABASES = {
'default': {
'ENGINE': DEFAULT_ENGINE,
'NAME': DEFAULT_NAME,
},
'read_slave': {
'ENGINE': DEFAULT_ENGINE,
'NAME': DEFAULT_NAME,
'TEST_NAME': None,
},
}

So the important thing here is that 'read_slave' *is* defined, but in
my local test settings it uses the same `DATABASE_NAME' (and host,
user, password, port) as my `default'.  The `TEST_NAME = None' change
will simply allow me to get past the error caused when `read_slave'
tries to drop and create a database that `default' has an open session
to (it just dropped and created itself, after all).

Now in code like,

MyModel.objects.using('read-slave').filter(...)

That should be a valid connection (`DATABASES['read_slave']') but it's
actually connecting to the exact same DB as `default', so a filter
should find objects created on `default', just like you'd imagine in a
real world read-slave setup.

Does that make more sense?  There's really nothing magic going on
here, it's only a matter of telling it not to drop/create the DB.  I
think maybe `TEST_NAME = None' could be confusing?  I didn't mean to
imply that the alias wasn't properly setup and functional.

Regards,
Brett
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: DB optimization docs

[Christopher Petrilli]

Luke,

I wonder if it's the right place, but one thing I have noticed most
people don't realize is that they need to run something to do pooling
between Django and the DB. I run pgpool, even when there is only one
of each since it does the connection pooling for you. This can be a
HUGE win in many cases.

Chris

On Fri, Jan 8, 2010 at 10:33 AM, Luke Plant  wrote:
> On Friday 08 January 2010 14:12:56 Mat Clayton wrote:
>> As someone going through this pain right now, this would be very
>>  helpful.
>>
>> Mat
>
> I don't know when I'll have more time to work on this, but I've
> committed the beginnings of draft of docs/topics/db/optimization.txt
> to my hg repos.
>
> http://bitbucket.org/spookylukey/django-trunk-
> lukeplant/src/tip/docs/topics/db/optimization.txt
>
> It needs to be hyperlinked to all the right bits, and filled out.  But
> it might be of some help to you in its current state.
>
> If anyone wants to take this on as a mini-project, it would be
> gratefully received :-)  I'm not planning on doing any more on this
> for at least a few days.
>
> Luke
>
> --
> "My capacity for happiness you could fit into a matchbox without
> taking out the matches first." (Marvin the paranoid android)
>
> Luke Plant || http://lukeplant.me.uk/
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Django developers" group.
> To post to this group, send email to django-develop...@googlegroups.com.
> To unsubscribe from this group, send email to 
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/django-developers?hl=en.
>
>
>
>



-- 
| Chris Petrilli
| petri...@amber.org
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: DB optimization docs

[Mat Clayton]

Thanks, much aooreciated

On Fri, Jan 8, 2010 at 3:33 PM, Luke Plant  wrote:

> On Friday 08 January 2010 14:12:56 Mat Clayton wrote:
> > As someone going through this pain right now, this would be very
> >  helpful.
> >
> > Mat
>
> I don't know when I'll have more time to work on this, but I've
> committed the beginnings of draft of docs/topics/db/optimization.txt
> to my hg repos.
>
> http://bitbucket.org/spookylukey/django-trunk-
> lukeplant/src/tip/docs/topics/db/optimization.txt
>
> It needs to be hyperlinked to all the right bits, and filled out.  But
> it might be of some help to you in its current state.
>
> If anyone wants to take this on as a mini-project, it would be
> gratefully received :-)  I'm not planning on doing any more on this
> for at least a few days.
>
> Luke
>
> --
> "My capacity for happiness you could fit into a matchbox without
> taking out the matches first." (Marvin the paranoid android)
>
> Luke Plant || http://lukeplant.me.uk/
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-develop...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>
>
>


-- 
-- 
Matthew Clayton | Founder/CEO
Wakari Limited

twitter http://www.twitter.com/matclayton

email m...@wakari.co.uk
mobile +44 7872007851

skype matclayton
-- 

You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-develop...@googlegroups.com.

To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Re: DB optimization docs

[Luke Plant]

On Friday 08 January 2010 14:12:56 Mat Clayton wrote:
> As someone going through this pain right now, this would be very
>  helpful.
> 
> Mat

I don't know when I'll have more time to work on this, but I've 
committed the beginnings of draft of docs/topics/db/optimization.txt 
to my hg repos.

http://bitbucket.org/spookylukey/django-trunk-
lukeplant/src/tip/docs/topics/db/optimization.txt

It needs to be hyperlinked to all the right bits, and filled out.  But 
it might be of some help to you in its current state.

If anyone wants to take this on as a mini-project, it would be  
gratefully received :-)  I'm not planning on doing any more on this 
for at least a few days.

Luke

-- 
"My capacity for happiness you could fit into a matchbox without 
taking out the matches first." (Marvin the paranoid android)

Luke Plant || http://lukeplant.me.uk/
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Using Django with Jinja2 and TEMPLATE_DEBUG=True

[Rick van Hattem]

On Thursday 07 January 2010 01:35:50 Russell Keith-Magee wrote:
> From a cursory inspection, I'm not sure there is much we can do with
> (1) - there isn't a lot of detail on Exception that can be used for a
> capability check, and the only attribute that is actually needed is
> 'source' (albeit in a particular format that evidently Jinja doesn't
> use).

Agreed, that is why the patch in the referenced ticket was denied.
http://code.djangoproject.com/attachment/ticket/10216/10216.diff

> (2) isn't Django's problem to fix.

If it would just be a problem with Jinja than I would agree. However, Django 
breaks on every Exception with a source attribute that doesn't follow the 
Django style completely.

> (3) Seems like the most promising option. For example, most of the
> call to ExceptionReporter .get_template_exception_info() could be
> deferred to the exception itself. If we refactored that code into a
> method on the exception itself, we could use that method as the
> capability that we check for. It would be less ambiguous (not many
> exceptions would have a 'get_template_exception_info' method), and it
> would allow *any* exception (template or otherwise) to raise an
> embedded exception that provided template details on the technical 500
> page.

Agreed, that would probably be the best solution. Although I do think that 
duck typing for catching exceptions is generally not the way to go.

Exceptions should always be caught explicitly 

> However, as I said originally - the existing setup works for Django at
> present. If you want to improve support for Jinja... patches are
> accepted :-)

As soon as I get back to the latest trunk release (I'm currently working on a 
revision that's about a year old) I will write a patch for this problem. 
Patching the old release and merging it with the current changes looks like 
it is going to be quite a tedious process.

~Rick


signature.asc
Description: This is a digitally signed message part.

Re: Model validation incompatibility with existing Django idioms

[Tobias McNulty]

On Fri, Jan 8, 2010 at 4:03 AM, James Bennett  wrote:

> On Thu, Jan 7, 2010 at 10:40 AM, Honza Král  wrote:
> > ModelForm has a save() method that saves the model. It is reasonable
> > to assume that if the form is valid, the save() method call should
> > succeed, that's why the entire model is validated.
>
> Actually, I can see a strong argument against this: if I've defined a
> form class that I think constructs valid instances, and it doesn't,
> that's a bug in my form and Django should be making this as clear as
> possible.
>
> Of course, the flip side is that the bug in my form may be something
> subtle which breaks a constraint specified at the Python level but not
> at the DB level, and so the additional automatic validation could be
> the only way to catch it.
>

For another alternative still, a constraint may exist at the database level
that Python doesn't know about (or may not be able to enforce efficiently).

I regret and apologize that I'm arriving to this thread rather late.  Is
there a document somewhere that discusses, or could someone describe, how
model validation fits between the existing form validation in Django and
constraints that are specified on the database side?

I too would opt for an implementation that makes model validation optional,
i.e., a call that developers must explicitly make, if they choose, before
saving a model.  I've always been and I continue to be wary of trying to
implement data constraints at the application level; that's something good
relational databases have been doing and improving upon for a long time and
I have little faith in my own capacity to reproduce or replace such
functionality.

Cheers,
Tobias
-- 
Tobias McNulty
Caktus Consulting Group, LLC
P.O. Box 1454
Carrboro, NC 27510
(919) 951-0052
http://www.caktusgroup.com
-- 

You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-develop...@googlegroups.com.

To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Tobias McNulty]

On Fri, Jan 8, 2010 at 8:36 AM, Honza Král  wrote:

> I just hate the name save(commit=False) when it has nothing to do with
> saving or committing, it just DOESN'T call save, it's imho misleading.
> I understand why that is and how it came to be, I just don't like it.
> I wasn't, however, proposing we get rid of this feature, just that we
> extract it to a separate method or just tell people to use
> form.instance (which is all what it does + creating save_m2m which can
> be put elsewhere).
>

There is /a lot/ of code that relies on this naming and, while it might not
be the greatest name choice in the world, it's one you get used to after
making use of it time and time again.  I'm fairly certain the 'commit'
argument is not the only instance of imperfect naming in the Django core,
and fixing all of these would leave Django users with a relatively
insurmountable quantity of deprecated code.  Frankly I'm not sure it's worth
it.

Tobias
-- 
Tobias McNulty
Caktus Consulting Group, LLC
P.O. Box 1454
Carrboro, NC 27510
(919) 951-0052
http://www.caktusgroup.com
-- 

You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-develop...@googlegroups.com.

To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

Re: DB optimization docs

[Russell Keith-Magee]

On Fri, Jan 8, 2010 at 10:04 PM, Luke Plant  wrote:
> Hi all,
>
> I was prompted by this post:
>
> http://it.toolbox.com/blogs/database-soup/stuff-id-love-to-see-from-
> django-36278
>
> to add some notes about some DB access optimizations (essentially the
> things I mentioned in my comment on that page), but then thought that
> even if I add them, people are unlikely to find them.  It's also not
> possible to put admonitions everywhere in the docs to stop people
> doing potentially expensive things.  Rather, we need a list of tips
> for someone looking to optimize DB queries.  It could link to all the
> relevant documentation (select_related() etc), as well as having misc
> other tips.
>
> Good idea/bad idea? At the moment, the information is mainly all
> there, but spread out in many places - I think we need an "optimize DB
> access" topic.

Sounds like a great idea to me. +1.

Russ %-)
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Call for comment: #5390 Many To Many signals

[Russell Keith-Magee]

On Fri, Jan 8, 2010 at 9:46 PM, Luke Plant  wrote:
> On Friday 08 January 2010 12:04:27 Russell Keith-Magee wrote:
>
>> Any feedback on this design?
>
> Just a few things:
>
> 1) In assessing performance impact, people need to be clear that there
> is just one signal handler for the whole project, like for the
> post_save signal etc.  Adding a handler will affect every m2m
> operation.

True, although this actually strengthens my argument that the
performance impact isn't significant. Jacob's performance data
indicates that there the one-time cost to attach the first signal is
roughly 3 times the O(N) cost of each subsequent handler. However, the
one-time cost only has to be paid once for the entire project, not
once per model or m2m relation.

> One possibly way to reduce the impact slightly would be to have 3
> signals - m2m_add, m2m_delete, m2m_clear.  A handler attached to one
> would then produce no impact on the others being called. I'm not sure
> it is worth it though, especially since a handler will often need to
> hand all three situations, in which case the performance benefit is
> cancelled out, and developers will need to attach the handler three
> times.

I tend to agree. Malcolm's original analysis was that "3 signals is
about 2 too many", and I can see his point - any change to an m2m will
in all likelihood need to respond to all thre signals. I'm not
convinced the performance gain alone is enough to warrant moving back
to three signals.

> 2) What happens with 'reverse' when the m2m is to 'self'? I think your
> code always generates 'reverse=False' in this case, since the target
> of the relation is always the same model.  This hurts my head to think
> about in the abstract, but I think it needs to distinguish between
> forward and reverse for non-symmetrical models.

Good catch - I didn't consider the non-symmetrical case. I'll update the patch.

Russ %-)
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: DB optimization docs

[Mat Clayton]

As someone going through this pain right now, this would be very helpful.

Mat

On Fri, Jan 8, 2010 at 2:04 PM, Luke Plant  wrote:

> Hi all,
>
> I was prompted by this post:
>
> http://it.toolbox.com/blogs/database-soup/stuff-id-love-to-see-from-
> django-36278
>
> to add some notes about some DB access optimizations (essentially the
> things I mentioned in my comment on that page), but then thought that
> even if I add them, people are unlikely to find them.  It's also not
> possible to put admonitions everywhere in the docs to stop people
> doing potentially expensive things.  Rather, we need a list of tips
> for someone looking to optimize DB queries.  It could link to all the
> relevant documentation (select_related() etc), as well as having misc
> other tips.
>
> Good idea/bad idea? At the moment, the information is mainly all
> there, but spread out in many places - I think we need an "optimize DB
> access" topic.
>
> Luke
>
> --
> "My capacity for happiness you could fit into a matchbox without
> taking out the matches first." (Marvin the paranoid android)
>
> Luke Plant || http://lukeplant.me.uk/
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-develop...@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>
>
>


-- 
-- 
Matthew Clayton | Founder/CEO
Wakari Limited

twitter http://www.twitter.com/matclayton

email m...@wakari.co.uk
mobile +44 7872007851

skype matclayton
-- 

You received this message because you are subscribed to the Google Groups "Django developers" group.

To post to this group, send email to django-develop...@googlegroups.com.

To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.

DB optimization docs

[Luke Plant]

Hi all,

I was prompted by this post: 

http://it.toolbox.com/blogs/database-soup/stuff-id-love-to-see-from-
django-36278

to add some notes about some DB access optimizations (essentially the 
things I mentioned in my comment on that page), but then thought that 
even if I add them, people are unlikely to find them.  It's also not 
possible to put admonitions everywhere in the docs to stop people 
doing potentially expensive things.  Rather, we need a list of tips 
for someone looking to optimize DB queries.  It could link to all the 
relevant documentation (select_related() etc), as well as having misc 
other tips.

Good idea/bad idea? At the moment, the information is mainly all 
there, but spread out in many places - I think we need an "optimize DB 
access" topic.

Luke

-- 
"My capacity for happiness you could fit into a matchbox without 
taking out the matches first." (Marvin the paranoid android)

Luke Plant || http://lukeplant.me.uk/
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Call for comment: #5390 Many To Many signals

[Luke Plant]

On Friday 08 January 2010 12:04:27 Russell Keith-Magee wrote:

> Any feedback on this design?

Just a few things:

1) In assessing performance impact, people need to be clear that there 
is just one signal handler for the whole project, like for the 
post_save signal etc.  Adding a handler will affect every m2m 
operation.

One possibly way to reduce the impact slightly would be to have 3 
signals - m2m_add, m2m_delete, m2m_clear.  A handler attached to one 
would then produce no impact on the others being called. I'm not sure 
it is worth it though, especially since a handler will often need to 
hand all three situations, in which case the performance benefit is 
cancelled out, and developers will need to attach the handler three 
times.

2) What happens with 'reverse' when the m2m is to 'self'? I think your 
code always generates 'reverse=False' in this case, since the target 
of the relation is always the same model.  This hurts my head to think 
about in the abstract, but I think it needs to distinguish between 
forward and reverse for non-symmetrical models. 

For example, given:

  class Person(Model):
  fans = ManyToManyField('self', related_name='favorites',
 symmetrical=False)
  friends = ManyToManyField('self')

  p1 = Person.create(...); p2 = Person.create(...)

The symmetrical 'friends' relation obviously should always have 
'reverse=False'.  But I think this call:

  p1.fans.add(p2)

needs to send a different signal than

  p1.favorites.add(p2)

and with your patch it wouldn't. But my brain doesn't seem to be 
functioning properly today, I may have made a mistake.

Luke

-- 
"My capacity for happiness you could fit into a matchbox without 
taking out the matches first." (Marvin the paranoid android)

Luke Plant || http://lukeplant.me.uk/
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[Honza Král]

On Fri, Jan 8, 2010 at 11:59 AM, koenb  wrote:
>
>
> On 8 jan, 10:03, James Bennett  wrote:
>
>> Suppose I have a ModelForm and call save(commit=False) to get the
>> instance so I can do some more work on it. I'm basically saying to
>> Django "I don't think this object is ready to be saved yet, but I need
>> the object so I can do stuff to it". If Django goes ahead and does
>> implicit model validation there and raises an exception, it's just
>> telling me what I already knew: the object's not ready to be saved.
>> But now I get to do extra work to catch the exception, and that's bad
>> and wrong.
>>
>> Calling ModelForm.save(commit=False) should simply disable model
>> validation, period. If I want to validate the instance before saving
>> it, I'll validate the instance before saving it, but commit=False is
>> as clear a way as we have of saying "I'm not going to save this yet"
>> and model validation should respect that.

I just hate the name save(commit=False) when it has nothing to do with
saving or committing, it just DOESN'T call save, it's imho misleading.
I understand why that is and how it came to be, I just don't like it.
I wasn't, however, proposing we get rid of this feature, just that we
extract it to a separate method or just tell people to use
form.instance (which is all what it does + creating save_m2m which can
be put elsewhere).

>
> The problem is that in the idiom the is_valid() call on the modelform
> tries to full_validate the instance, it is not in the save
> (commit=False) call.

Yes

> I'm with Simon here: on an incomplete modelform, let's just ignore the
> errors on excluded fields and only validate the user input at that
> point. If the developer wants to be sure the model validates after he
> adds the necessary extra information, it is his job to call validation
> before saving.

The only problem I see here is that you cannot run model.validate (so
check for unique etc.) because the user might define some custom
validation there that you have no control over and that can check
fields you don't want it to touch. We could move validate_unique
elsewhere, but that could create problem for people not wanting to
have their fields validated for uniqueness (expensive hits to the DB)





But overall I feel that "we only call model.full_validate when no
field is excluded or we are editing an existing instance" is the
desired behavior by most people.

In code that would mean that self.validate in full_clean would only be
called if not exclude and excluded form fields would be passed to
model.full_clean when adding an instance (not when editing), that's a
very simple change. We just need to document this behavior thoroughly
because it can cause confusion.



The question remains how to validate inlines in Admin. I think there
is no question that we want to call full_validate on the inlined model
eventually, the question is how to do it in a most backwards
compatible and sane way.

We need the FK to do the validation and we cannot get the FK without
the save() of the parent model. So imho it's just a question of how
much risk we are taking and at which point do we call model.save() on
the parent (after the form validates, after the inline form fields
validate).

If we take the new proposal for ModelForm behavior it could work:

MainForm.clean()

for f in inline: f.validate()

if all.valid:
  MainForm.save()

for inline in inlines:
  for form in inline;
f.instance.full_validate()
  inline.save()

the problem is when f.instance.full_validate() fails on the inline.
That's a point where the FK must exist so the parent object has to be
saved already. There is no way around this if we want to have
model-validation in admin uncrippled.
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

help needed: non-relational DB support

[Waldemar Kornewald]

Hi,
our non-relational port has come to the point where we need to
back-port the SQL layer to the query backend API (i.e., the new
query_class()). We could need some help from Django developers who
know the ORM internals really well. You can find a little introduction
to the code here:
http://bitbucket.org/wkornewald/django-nonrel-multidb/wiki/Home

It would be great if one core developer could join the project and
actually work on the code. Here's the discussion group:
http://groups.google.com/group/django-non-relational

The difficulty is that we need to convert QuerySet's intermediate
representation (QueryData) to the sql.Query representation. We're not
sure if QueryData in its current implementation has the right format
and someone who knows sql.Query better than we do could be of great
help, especially with the conversion process. Our goal is to get
non-relational backend support into Django 1.3 (which is definitely
possible, so please don't vote -1 next time if this is your only
concern).

I understand if you're currently busy with finishing 1.2, but if
you're interested in helping when will you have time?

Bye,
Waldemar Kornewald

--
http://twitter.com/wkornewald
http://bitbucket.org/wkornewald/
http://allbuttonspressed.blogspot.com/
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Call for comment: #5390 Many To Many signals

[Russell Keith-Magee]

Hi all,

I've just uploaded a patch for #5390, which adds signals to
many-to-many modifying operations.

This is a ticket that was a low-priority accepted item for 1.1
(ORM-18), but got bumped due to time [1]. For some reason, it wasn't
resubmitted for 1.2.

The purpose of this post is twofold:

1) To establish if there are any objections to introducing this
signal. Historically, there has been a performance objection to adding
new signals, so I want to give some numbers of the performance impact
and see if anyone objects. and

2) To get feedback on the implementation.

Barring objections or major design changes, I'd like to commit this
early next week.

1. Performance
--

The last time signals came up [2], Jacob provided some helpful
baseline data on the speed of signals:

Doing Nothing: 0.00283 (0.00028 per call)
Plain func call: 0.07054 (0.00705 per call)
Signal; 0 handlers: 0.09331 (0.00933 per call)
Signal; 1 handler : 0.79125 (0.07912 per call)
Signal; 10 handlers: 3.36051 (0.33605 per call)
Signal; 100 handlers: 27.56269 (0.000275627 per call)

* In Python, calling a function (``handle()``) takes 25 times as long
as doing nothing (``pass``).
* Dispatching a signal when there's no handlers is about 1.3 times a
function call.
* Dispatching a signal when there's one handler is about 11 times the
cost of a function call.
* Addition of additional receivers scales linearly.

The other (completely unscientific) test is the Django test suite
itself. I've run the test suite with and without the patch from #5390.
What I have observed is that the discrepancies in test runtime caused
by normal fluctuations in background system load are more significant
than the slowdown caused by adding m2m signals (that is - the vanilla
suite sometimes takes *longer* than the suite with the patch applied).

So - based on my this analysis, there is a performance impact, but it
isn't especially significant or noticeable.

2. Implementation
-

The patch I have uploaded [3] introduces a single new signal -
m2m_changed. This decision is based on design feedback from Malcolm
[4].

The signal is attached to the m2m through class (the automatically
generated m2m class that is the result of the m2m refactor). This
class is the 'sender' of the signal.

The signal handler is provided with the sender, as well as:
 * The instance that contains the m2m field that is being modified
 * A string tag indicating if this is an "add", "remove" or "clear" call
 * A boolean indicating whether this is the 'forward' or 'reverse' relation.
 * The class of model that is being added/removed/cleared
 * The primary keys of the instances that are being added to the
relation (or None in the case of a call to clear the relation)

The docs and tests on the patch [3] give a more detailed description
and examples of the operation of the signal.

Any feedback on this design?

[1] http://code.djangoproject.com/wiki/Version1.1Features
[2] 
http://groups.google.com/group/django-developers/browse_thread/thread/5a39839e2aa8df53
[3] http://code.djangoproject.com/attachment/ticket/5390/t5390-r12120.1.diff
[4] 
http://groups.google.com/group/django-developers/browse_thread/thread/afe6ad7994d868ba

Yours,
Russ Magee %-)
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[koenb]

On 8 jan, 10:03, James Bennett  wrote:

> Suppose I have a ModelForm and call save(commit=False) to get the
> instance so I can do some more work on it. I'm basically saying to
> Django "I don't think this object is ready to be saved yet, but I need
> the object so I can do stuff to it". If Django goes ahead and does
> implicit model validation there and raises an exception, it's just
> telling me what I already knew: the object's not ready to be saved.
> But now I get to do extra work to catch the exception, and that's bad
> and wrong.
>
> Calling ModelForm.save(commit=False) should simply disable model
> validation, period. If I want to validate the instance before saving
> it, I'll validate the instance before saving it, but commit=False is
> as clear a way as we have of saying "I'm not going to save this yet"
> and model validation should respect that.
>

The problem is that in the idiom the is_valid() call on the modelform
tries to full_validate the instance, it is not in the save
(commit=False) call.

I'm with Simon here: on an incomplete modelform, let's just ignore the
errors on excluded fields and only validate the user input at that
point. If the developer wants to be sure the model validates after he
adds the necessary extra information, it is his job to call validation
before saving.

Koen
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Model validation incompatibility with existing Django idioms

[James Bennett]

On Thu, Jan 7, 2010 at 10:40 AM, Honza Král  wrote:
> ModelForm has a save() method that saves the model. It is reasonable
> to assume that if the form is valid, the save() method call should
> succeed, that's why the entire model is validated.

Actually, I can see a strong argument against this: if I've defined a
form class that I think constructs valid instances, and it doesn't,
that's a bug in my form and Django should be making this as clear as
possible.

Of course, the flip side is that the bug in my form may be something
subtle which breaks a constraint specified at the Python level but not
at the DB level, and so the additional automatic validation could be
the only way to catch it.

> 2) Are you OK with ripping save(commit=False) functionality to some
> other method? (current functionality can stay with a deprecation
> warning for example)

No.

Suppose I have a ModelForm and call save(commit=False) to get the
instance so I can do some more work on it. I'm basically saying to
Django "I don't think this object is ready to be saved yet, but I need
the object so I can do stuff to it". If Django goes ahead and does
implicit model validation there and raises an exception, it's just
telling me what I already knew: the object's not ready to be saved.
But now I get to do extra work to catch the exception, and that's bad
and wrong.

Calling ModelForm.save(commit=False) should simply disable model
validation, period. If I want to validate the instance before saving
it, I'll validate the instance before saving it, but commit=False is
as clear a way as we have of saying "I'm not going to save this yet"
and model validation should respect that.


-- 
"Bureaucrat Conrad, you are technically correct -- the best kind of correct."
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: custom signal fails silently, depending on import path (bug ?)

[James Bennett]

On Fri, Jan 8, 2010 at 1:41 AM, ssc  wrote:
> Has anyone ever come across this before ? Could not find anything
> related in Trac, but I thought I better ask in here before I file a
> bug...

If you import a signal, using one particular path to specify the
import, you get that signal object. If you import it again later using
a different path, you get a different object. This is simply the way
Python imports work, and the solution to this "bug" is to ensure you
always use the same path (preferably the fullest possible path)
whenever importing a signal.

(also, your use of "from signals import ..." relies on implicit
relative imports within a module, which is a deprecated feature of
Python)


-- 
"Bureaucrat Conrad, you are technically correct -- the best kind of correct."
-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.