Feature request: log capture during testing
Now that Django has logging support using the Python logging module, it would be nice if the unit testing framework included the ability to capture logs, similarly to the mail.outbox functionality. I am aware of the log_capture decorator in the testfixtures package, and we are using it for some tests. Unfortunately, it doesn't correctly capture anything that's been changed by a filter, since it installs itself as a handler without preserving filters attached to existing handlers. It would be great if Django could do better! Cheers, Jody -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
[GSoC composite fields] Weekly check-in #1
Hello everyone, I hope the introductions aren't necessary since I've already introduced myself in the past [1]. Just a recap, I'll be working, under the guidance of Jacob Kaplan-Moss, on support for composite model fields which will allow users to define models with composite primary keys. The full text of my proposal is available for anyone interested to read [1][2]. You can reach me on #django-dev as 'konk', I'll be idling there most of the time (unless the server running irssi crashes) and I tend to check it quite often so feel free to ping me with any questions or comments. Same as Gregor and Jan, I created a fork on github [3] where I'll push my progress. The branch name is, surprisingly, soc2011/composite-fields. There's also the issue tracker for anyone who prefers this form of communication. In a couple days I'll hopefully post how I imagine the API with all the bloody details. Just hang on those few days until I write it down. I guess this is all I have to report for tonight, more coming soon. Michal Petrucha [1] http://groups.google.com/group/django-developers/browse_frm/thread/120d401e302f369b/82471d2407c87abf [2] http://people.ksp.sk/~johnny64/GSoC-full-proposal [3] https://github.com/konk/django signature.asc Description: Digital signature
[GSoC schema alteration] Weekly check-in #1
Hi once more, So there was 0 check-in to start the thread and now meritum. People who have read, commented and evaluated my proposal already know me, but for others I will introduce myself. My name is Jan Rzepecki, I'm going to work on django schema alteration API this year for GSoC 2011. I'm on my last year of Electronic Engineering studies on the same time in Lodz, Poland and in Nantes, France. After some experience with php, I was searching for framework, and new language for my webdevelopment freelancing. And then I found Django, few months before it had reached 1.0 version. I'm going to introduce new API to use with different applications that manage database schemas, to allow not only schema creation, but also modification. It would be good foundation for every database migration utility app, so the authors can focus on core functionality, not on particular database behaviour and SQL dialect. My proposal can be found in archive of django developers group [1]. Andrew Godwin is going to guide me through this project. I believe that his experience as South author, and my hard working, going to make this project successful and profitable for community. After the weekend I'm going to publish draft of API calls, with notion of which calls going to be deprecated for external use, so we are going to have plenty of space for healthy discussion ;) I'm also open for discussion on #django-dev channel with name `xtrqt`. After Greg Muller's example I've also created fork of django repository on github with new branch soc2011/schema-alteration [2] where I'm going to put all code. All documention of how it work I've planned to put in ReST Gist file, which in the end would form a part of official django docs. [3] So that's all for now stay tuned for API draft. PS And I have something to celebrate, I should have start with that a long time ago, but still, 32 hours ago, my first patch [quite simple one] found it's way to trunk [thx to jezdez]. Jan `xtrqt` Rzepecki | [1] http://groups.google.com/group/django-developers/msg/281ef8236c9be13a | [2] https://github.com/xtrqt/django/tree/soc2011/schema-alteration | [3] https://gist.github.com/949044 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Weekly check-in #1
Hi once more, So there was 0 check-in to start the thread and now meritum. People who have read, commented and evaluated my proposal already know me, but for others I will introduce myself. My name is Jan Rzepecki, I'm going to work on django schema alteration API this year for GSoC 2011. I'm on my last year of Electronic Engineering studies on the same time in Lodz, Poland and in Nantes, France. After some experience with php, I was searching for framework, and new language for my webdevelopment freelancing. And then I found Django, few months before it had reached 1.0 version. I'm going to introduce new API to use with different applications that manage database schemas, to allow not only schema creation, but also modification. It would be good foundation for every database migration utility app, so the authors can focus on core functionality, not on particular database behaviour and SQL dialect. My proposal can be found in archive of django developers group [1]. Andrew Godwin is going to guide me through this project. I believe that his experience as South author, and my hard working, going to make this project successful and profitable for community. After the weekend I'm going to publish draft of API calls, with notion of which calls going to be deprecated for external use, so we are going to have plenty of space for healthy discussion ;) I'm also open for discussion on #django-dev channel with name `xtrqt`. After Greg Muller's example I've also created fork of django repository on github with new branch soc2011/schema-alteration [2] where I'm going to put all code. All documention of how it work I've planned to put in ReST Gist file, which in the end would form a part of official django docs. [3] So that's all for now stay tuned for API draft. PS And I have something to celebrate, I should have start with that a long time ago, but still, 32 hours ago, my first patch [quite simple one] found it's way to trunk [thx to jezdez]. Jan `xtrqt` Rzepecki | [1] http://groups.google.com/group/django-developers/msg/281ef8236c9be13a | [2] https://github.com/xtrqt/django/tree/soc2011/schema-alteration | [3] https://gist.github.com/949044 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
[GSoC schema alteration] Weekly check-ins
Hi, So that would be my 0 check-in , purpose of this post is only to keep all following checkins under one thread. More informations starting from check-in number 1 just below. So all my further check-ins are going to be posted under check-in 0. Jan `xtrqt` Rzepecki -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: ModelForm validation - a better way?
> For reference, here's where we'd be in that case (I still prefer the
> context manager over the idea of two separate calls to something named
> "validate"):
>
> def my_view(request):
> form = MyModelForm(request.POST or None)
> try:
> with form.validate(tweak=True) as obj:
> obj.user = request.user
> except ValidationError:
> pass
> else:
> obj.save()
> return redirect(obj)
> return render(request, "foo.html", {"form": form})
>
> Or in the simple case, where no modifications to the object are needed:
>
> def my_view(request):
> form = MyModelForm(request.POST or None)
> try:
> obj = form.validate()
> except ValidationError:
> pass
> else:
> obj.save()
> return redirect(obj)
> return render(request, "foo.html", {"form": form})
>
> Carl
In my opinion, this looks good. I have some additional ideas as to
what the parameters to validate() should be, but in general I think
most of the most important issues are on the table. I like the idea
of having part of the validation take place in __enter__ and part in
__exit__; I think that maybe this should be controllable using the
arguments to validate() (i.e., using parameters like 'defer',
'skip_model', 'immediate'). I am also not entirely sure about using
ValidationError; perhaps the exceptions raised should be subclasses of
ValidationError, so that they can easily be caught all at once, but I
think it would be very useful for simple form validation and model
validation to raise different exceptions, primarily as a way to give
people some level of control over how errors are displayed. The
tricky nature of error generation inside model validation could be
helped a little if programmers were given the ability to distinguish
between error types by what subclass of ModelValidationError is
raised. I am also assuming that form._errors would be populated
inside validate(), __enter__() and __exit__(); it might be helpful to
create an api that makes it easier to add errors to form._errors.
The *problem* that I see is this:
The changes that we are discussing here will give model forms a
different validation API to standard forms. Yes, form.validate()
could also be added to the standard Form, but then what would the
context manager return from its __enter__ method? If form.validate()
is a method only of ModelForm, then validation of Forms and ModelForms
would be very different.
Now, in the general case, having different validation processes for
Form and ModelForm wouldn't be too much of a *practical* problem (in
most cases a programmer knows whether they are dealing with a Form or
a ModelForm). With regards to class-based views, however, and other
systems that depend on a consistent "abstract" Form API, an
inconsistent api between the two would be a headache, and possibly
limit the convenience and usability of such systems, I think. Also,
saying that validation of ModelForm works differently than Form in a
substantive way just feels wrong.
Regards,
Eduardo
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: "unique_together" only validated in modelform.is_valid() if ALL of the referenced fields (Ticket #13091)
Carl, Thanks for this professional reply. After rereading my post (immediately after submitting it), I realized that I was much more critical than I would normally think is fair, which is why I removed it. It's sometimes necessary, I think, to remind ourselves that most of us are volunteers here, including the core developers. I also have to say that you have made yourself available in a very generous manner of late; for that I am quite appreciative. Regards, Ed Gutierrez On Apr 28, 8:34 pm, Carl Meyer wrote: > Hi Eduardo, > > On 04/28/2011 06:36 PM, legutierr wrote: > > > This is extraordinarily discouraging. > > I can understand why. > > I've also spent a number of hours thinking about this, reviewing the > patch, considering alternatives, coming up with cases that might break, > etc. I'd like to set aside those sunk costs (which I don't think were > wasted in either case) and keep the focus on the best way to solve the > issue in Django moving forward - that's what I owe to the rest of the > core development team and to the community. > > > This is the second time that I > > have devoted tremendous energy to a patch, trying to coordinate with > > core developers, not doing any work until I get the green light from > > core developers regarding an implementation plan (trying to avoid this > > very same eventuality), only to be told, after working code + tests + > > docs have been attached to the ticket, after several iterations of > > feedback: nope, this is not the way that we want to do this policy- > > wise, there's this other approach we want to take, so never mind. > > I'm not certain what the other situation is that you're referring to, so > I can't speak to that. My observation has been that this isn't the > common experience (unfortunately, getting no attention to a bug/patch in > the first place has at times been a more common one, though that too is > getting better -- unreviewed bug count is currently zero!), but I'm > sorry you've experienced it, and I regret having contributed to it in > this case. I will certainly be more careful in the future about > expressing optimism that an approach might be workable, especially if > (as in this case) I have reservations about it from the start. > > > I can understand going through the bureaucratic rigmarole that comes > > with contributing to Django--in fact, I support it--but to go through > > all of the discussion, justification, and *time* required to get a > > simple bug fix checked in (no, this really *is* a bug--look, there are > > five other tickets filed. sure, let's analyze the problem from every > > angle. sure, I'll rewrite it so it matches exactly your > > specifications.), only to be told that someone who wasn't even > > involved in this ticket and discussion *at all* until now thinks it > > isn't worth it, makes a guy like me want to tear his hair out. You > > say that this is "in the best interests of Django", but you must know > > that Django will suffer if people like me stop wanting to contribute > > because of things like this. > > Indeed, and I hope that you don't lose interest in contributing. > > I don't think that the time spent discussing and analyzing this, even > writing and reviewing a patch, is wasted. From my perspective, it has > clearly revealed that the current approach of trying to do > partial-model-validation is broken in concept and not reliably fixable. > That's useful information, and moves us (has already moved us) towards a > better solution. > > I can't agree that this is a simple bug fix. The current behavior is > wrong in some cases. The behavior after this "fix" would still be wrong > in some cases, although fewer. A simple bug fix is one where the fix is > clear, obviously correct, and definitively solves the reported problem. > I don't think that describes this case. Model and form validation is a > complex area, and it's easy for seemingly small changes to have > unintended effects that cause more maintenance burdens down the road. > > > How often has it been the case that some really cool new feature gets > > delayed because the core developer who was working on it was unable to > > dedicate the time they thought they would be able to? Can I help move > > it along, can you work with me to write it? Why can't we check this > > one in, close two tickets (as well as satisfying three or four > > duplicates) and then move on to the more definitive fix? > > I'm committed to having these tickets closed one way or another before > Django 1.4 is released (and neither fix here would be a candidate for > backporting to a 1.3.X release anyway), so let's focus on making the > best fix we can. If the ideas we have in mind for that turn out to be > unworkable for some reason, I still think that the current patch would > be preferable to no change. > > Carl -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django
Re: ModelForm validation - a better way?
On 04/29/2011 10:02 AM, Yishai Beeri wrote:
> Of course, cleanup need not be simplistic. In fact, I think the common
> coder would never expect a CM to actually save an object on __exit__ -
> and will be surprised by the proposed behavior.
Could be - the name "finish()" was intended to give the impression that
the purpose of the CM was to wrap up everything the modelform needs to
do. That currently includes saving the object.
I'm open to the idea that we change the name of the CM to
"form.validate()" and it never saves the object; you have to call
obj.save() yourself. In a way, it feels like making users of the API do
extra work for no good reason, and opening the door to mis-use of the
API (since all changes to the object should be completed within the body
of the context manager), but perhaps this is worth it to avoid
unexpected behavior.
For reference, here's where we'd be in that case (I still prefer the
context manager over the idea of two separate calls to something named
"validate"):
def my_view(request):
form = MyModelForm(request.POST or None)
try:
with form.validate(tweak=True) as obj:
obj.user = request.user
except ValidationError:
pass
else:
obj.save()
return redirect(obj)
return render(request, "foo.html", {"form": form})
Or in the simple case, where no modifications to the object are needed:
def my_view(request):
form = MyModelForm(request.POST or None)
try:
obj = form.validate()
except ValidationError:
pass
else:
obj.save()
return redirect(obj)
return render(request, "foo.html", {"form": form})
Carl
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: ModelForm validation - a better way?
Hi Carl,
On Fri, 29 Apr 2011 17:42:32 +0300, Carl Meyer wrote:
Hi Yishai,
On 04/29/2011 08:53 AM, Yishai Beeri wrote:
First, the logic tied into the context manager does not match the
idiomatic use of context managers in Python. One expects a context
manager to "clean up" on exit, rather than *save* a new object.
I'd argue it's not totally off base. When the action you're performing
in the context manager is "tweaking this model instance before its
saved", I think it's reasonable to consider "save it if it validates,
otherwise populate form._errors" to be appropriate and expected
"cleaning up" from that action.
For
instance, what if I throw an exception inside the with block? The idiom
tells us the object will be saved - but in this suggested approach I
probably want the opposite.
The fact that context managers imply cleanup doesn't mean that cleanup
has to be defined simplistically ("save the object no matter what"). The
context manager performs appropriate cleanup. That might be saving,
Of course, cleanup need not be simplistic. In fact, I think the common
coder would never expect a CM to actually save an object on __exit__ - and
will be surprised by the proposed behavior.
Another thing not expected by
or
populating form._errors, or (if you raise an exception inside the body)
probably neither.
I think 'neither' is also not a natural answer for what should a CM do
when exiting via an exception.
Also unclear is what happens if the form
fails validation (inside the __enter__ on form.finish); an exception?
In the original proposal there would be no form validation on __enter__,
only full validation on __exit__. The proposal could be modified to do
both - that gets into the territory of your and Johannes' alternative
proposals, which are interesting.
I agree. Perhaps saying this is a "validation" context manager, rather
than a "saving" context manager, is enough to eliminate the confusion.
That, of course, means adding an explicit form.save() call after the with
block.
...
and obviously model validation cannot rely on
per-field idioms like form validation does.
I'm not sure what you mean by this last bit - model validation and
modelform validation are actually pretty close to the same thing in the
current code (modelform validation calls model validation and catches
any ValidationErrors), with the exception of possibly excluding some
fields (and doing additional form-specific validation).
I actually referred to the difference between form validation (not
ModelForm), and Model validation. In other words, part of the validation
is mostly per-field, and usually based directly on the field definition
(be it a simple form field or one created in the modelform based on the
underlying model field); this part is more tightly related to what the
user sees and does on the form, and naturally does not need the missing
knowledge about the missing fields (or later tweaks). Another part of
validation is what I called "model validation" - everything that has to do
with the model as a whole. My thrust is that it is wise to keep them
separate (or at least easily separable).
As a very rough sketch, perhaps something along the lines of:
try:
form.validate()
obj = form.save(commit=False)
obj.user = request.user
obj.validate(form=form)
obj.save()
redirect(obj)
except FormValidationError, ModelValidationError:
redisplay the form
of course, the above can also be written with a couple of if/else
clauses, but perhaps routing validation errors as exceptions may feel
more natural in Python.
I don't really like the obj.validate(form=form) line; perhaps it needs
to be form.validate_object(obj), or even a context manager such as:
with form.validation():
form.validate()
...
obj.validate() # having the form.validation CM allows the object
validation to somehow tie the errors to the form fields, if so desired
...
This ends up being pretty similar to Johannes' idea - I'll reply to his,
feel free to note any differences you think are important.
Yes, you're right. In a nutshell:
a. keep two distinct validation phases [explicit in my suggestion,
implicit in the __enter__ and __exit__ phases of Johannes' idea]
b. allow some way for the later validation (which I called for simplicity
"model validation") to still relate to the form, hang errors on form
elements etc.
c. Make the save action (at least one that commits to the DB) explicit
[here is where my suggestion differs from both your and Johannes' ones]
It feels that for (b), a context manager of some sort is the way to go.
Adding (c) to Johannes' approach might be simply to require a call to
obj.save() after the with block
Carl
Yishai
--
You received this message because you are subscribed to the Google Groups "Django
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
dj
Re: ModelForm validation - a better way?
Hi Johannes,
On 04/29/2011 09:02 AM, Johannes Dollinger wrote:
> Here's my take on the API:
>
> def my_view(request):
> form = MyModelForm(request.POST or None)
> try:
> with form.finish() as obj:
> obj.user = request.user
> return redirect(obj)
> except ValidationError:
> return render(request, "foo.html", {"form": form})
>
> The context manager would validate the form on __enter__ (only
> included fields) and raise ValidationError if it does not validate.
> On __exit__, it would raise ValidationError if model validation
> fails. This let's you defer expensive computations/queries until
> after a sanity check.
I like this. It's a little weird that we re-use ValidationError for
something rather different than its usual use (normally it represents a
specific validation error and carries data about that error - in this
case we're effectively re-raising a sort of meta-ValidationError to
represent that the entire form has failed validation. Actually I think
that's probably fine, it just took me a moment to think through).
I think it would also be more correct to use try-except-else, and put
the success-case handling in the else clause.
It looks like this view is supposed to handle both GET and POST, so I
guess your assumption is that the context manager would also raise
ValidationError in case of an unbound form? That feels odd, but it
certainly simplifies the view code. The more-explicit alternative would
be something like this:
def my_view(request):
form = MyModelForm(request.POST or None)
if request.method == "POST":
try:
with form.finish() as obj:
obj.user = request.user
except ValidationError:
pass
else:
return redirect(obj)
return render(request, "foo.html", {"form": form})
That's not too bad either.
Optionally, finish() might take a flag to defer
> all validation until __exit__, which may be required if you want to
> display all validation erros at once.
Yes, I think that would be useful.
As an added benefit, this makes
> dealing with multiple forms and formset in a single view straight
> forward, as you can simply add more `with` blocks inside the try
> block, and validation across forms can simple raise a ValidationError
> on its own.
Yes - in the cross-form-validation case, the ValidationError might want
to carry a message for the template, but I suppose the developer can
handle that themselves.
I like this direction. I guess the main objection here might be that it
requires try-except handling for every form-handling view. I don't
necessarily consider that a problem: there's something that feels right
about exception-handling, rather than if-else clauses, as the idiom for
managing form validation errors. (Clearly it has something to recommend
it, since that's already the way we handle the details of validation
internally).
Carl
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: ModelForm validation - a better way?
Hi Yishai,
On 04/29/2011 08:53 AM, Yishai Beeri wrote:
> First, the logic tied into the context manager does not match the
> idiomatic use of context managers in Python. One expects a context
> manager to "clean up" on exit, rather than *save* a new object.
I'd argue it's not totally off base. When the action you're performing
in the context manager is "tweaking this model instance before its
saved", I think it's reasonable to consider "save it if it validates,
otherwise populate form._errors" to be appropriate and expected
"cleaning up" from that action.
> For
> instance, what if I throw an exception inside the with block? The idiom
> tells us the object will be saved - but in this suggested approach I
> probably want the opposite.
The fact that context managers imply cleanup doesn't mean that cleanup
has to be defined simplistically ("save the object no matter what"). The
context manager performs appropriate cleanup. That might be saving, or
populating form._errors, or (if you raise an exception inside the body)
probably neither.
Also unclear is what happens if the form
> fails validation (inside the __enter__ on form.finish); an exception?
In the original proposal there would be no form validation on __enter__,
only full validation on __exit__. The proposal could be modified to do
both - that gets into the territory of your and Johannes' alternative
proposals, which are interesting.
> Second, and this is a general issue underlying partial validation -
> probably part of what makes this issue so hairy - the full-model
> validation, and the resulting error messages, run the risk of being
> pretty remote from what the user actually did. It feels to me that form
> validation needs to be a step that focuses on values the user entered in
> the form, and that full-model validation should come as a second step,
> possibly adding more messages and tying them to particular form
> elements.
It would have to be up to the developer tweaking the model instance to
ensure that they don't do so in a way that results in validation errors
that are confusing to the user or that the user can't fix. This is
really no different from the current situation, where if you tweak the
model before saving you're responsible to avoid IntegrityError.
That said, I do see reasons why it would be nice to have the partial
sanity check of the current style of form validation before doing the
extra work that might be involved in tweaking the model for form
validation. Both your and Johannes' proposals do that.
I think in many cases the two types of validation deserve
> separation in code; model validation might need to be more expensive
> (e.g., hit the DB),
Already, modelform validation can itself just as easily hit the DB, if
you have unique constraints involving fields included in the form.
and obviously model validation cannot rely on
> per-field idioms like form validation does.
I'm not sure what you mean by this last bit - model validation and
modelform validation are actually pretty close to the same thing in the
current code (modelform validation calls model validation and catches
any ValidationErrors), with the exception of possibly excluding some
fields (and doing additional form-specific validation).
> As a very rough sketch, perhaps something along the lines of:
>
> try:
>form.validate()
>obj = form.save(commit=False)
>obj.user = request.user
>obj.validate(form=form)
>obj.save()
>redirect(obj)
> except FormValidationError, ModelValidationError:
>redisplay the form
>
> of course, the above can also be written with a couple of if/else
> clauses, but perhaps routing validation errors as exceptions may feel
> more natural in Python.
>
> I don't really like the obj.validate(form=form) line; perhaps it needs
> to be form.validate_object(obj), or even a context manager such as:
>
> with form.validation():
>form.validate()
>...
>obj.validate() # having the form.validation CM allows the object
> validation to somehow tie the errors to the form fields, if so desired
>...
This ends up being pretty similar to Johannes' idea - I'll reply to his,
feel free to note any differences you think are important.
Carl
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Trying to find Dojo Developers
On Fri, Apr 29, 2011 at 3:57 AM, mcflurry7_11 wrote: > I was wondering if any of you could help me out. I'm sorry, but we can't. This list is really the wrong place for messages like yours. First, this is a *Django* list and second this is a list for *developers* of Django, not end user questions. I'd suggest you take this to a list for Dojo users, if one exists. Jacob -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: ModelForm validation - a better way?
Am 29.04.2011 um 04:13 schrieb Carl Meyer:
> Hi all,
>
> We have a number of tickets open (at least #12028, #13249, #13091,
> #15326, and #15860 -- #13091 is the active one) reporting problems with
> unique_together constraints in our attempts to validate arbitrary
> partial models, when validating a ModelForm with some fields excluded.
>
> Eduardo Gutierrez and I have spent some time looking at this problem
> recently, and my main feeling is that validation of arbitrarily partial
> models in the face of unique_together constraints has no reliably right
> answer, and we'd do better to move away from it altogether.
>
> Fortunately, I think we can do better. The reason we have to validate
> partial models is because of this idiom:
>
> if form.is_valid():
>obj = form.save(commit=False)
>obj.user = request.user # for instance
>obj.save()
>
> But there is no reason those tweaks to the model have to happen after
> form validation. If we encouraged an idiom where the tweaks happen
> before form validation, we could just run full model validation and
> avoid all the error-prone complexity of validating partial models.
>
> Alex and I talked over some possibilities for a context manager
> available from a new method on ModelForms, that you'd use like this
> (idea originally from Łukasz Rekucki [1], somewhat modified):
>
>def my_view(request):
>if request.method == "POST":
>form = MyModelForm(request.POST)
>with form.finish() as obj:
>obj.user = request.user
>if form.is_valid():
>return redirect(obj)
>else:
>form = MyForm()
>return render(request, "foo.html", {"form": form})
>
> form.finish() returns a context manager which returns form.instance from
> its __enter__ method, as "obj" here, allowing the user to do any
> tweaking they like within the body of the context manager. On exit, the
> context manager performs _full_ model validation. Any validation errors
> are saved to the form, as usual. If validation succeeds, the model
> instance is saved.
>
> The following check to form.is_valid(), then, is just for purposes of
> managing view logic (redirect, or redisplay form?). Actual validation
> has already happened, so it would just be checking self._errors (this
> isn't a change, that's already how .is_valid() works).
>
> For backwards compatibility, there would be no change to the existing
> behavior if you don't use the new .finish() context manager -
> form.is_valid() would trigger possibly-partial validation just as it
> does now, and do the best it can. But the admin could be converted to
> use the new idiom (with a new ModelAdmin method that would be called
> from within the context manager to allow model-tweaking before
> validation), which would solve the admin-related bugs. And the
> documentation could recommend the new idiom over the old, particularly
> for incomplete modelforms with unique_together constraints.
>
> Open questions:
>
> 1. What if you don't need to tweak the instance, but you do want to
> trigger the new full validation behavior (i.e. your form excludes some
> fields, but you initially passed an instance to the form that you're
> confident has the excluded fields already set correctly - this would be
> the case for e.g. the parent FK in admin inlines)? Perhaps form.finish()
> could take an argument that determines whether it returns the context
> manager or just returns form.instance directly?
>
> 2. Is it going to be too weird for people to adjust to the idea that
> they get their model instance out of the form before they (explicitly)
> call form.is_valid()?
>
> Other issues we've overlooked, or ways this could be improved? Use cases
> this doesn't handle?
Here's my take on the API:
def my_view(request):
form = MyModelForm(request.POST or None)
try:
with form.finish() as obj:
obj.user = request.user
return redirect(obj)
except ValidationError:
return render(request, "foo.html", {"form": form})
The context manager would validate the form on __enter__ (only included
fields) and raise ValidationError if it does not validate. On __exit__, it
would raise ValidationError if model validation fails. This let's you defer
expensive computations/queries until after a sanity check. Optionally, finish()
might take a flag to defer all validation until __exit__, which may be required
if you want to display all validation erros at once.
As an added benefit, this makes dealing with multiple forms and formset in a
single view straight forward, as you can simply add more `with` blocks inside
the try block, and validation across forms can simple raise a ValidationError
on its own.
__
Johannes
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email t
Re: ModelForm validation - a better way?
Without really suggesting a better alternative, I'd like to highlight two problems I see with this approach: First, the logic tied into the context manager does not match the idiomatic use of context managers in Python. One expects a context manager to "clean up" on exit, rather than *save* a new object. For instance, what if I throw an exception inside the with block? The idiom tells us the object will be saved - but in this suggested approach I probably want the opposite. Also unclear is what happens if the form fails validation (inside the __enter__ on form.finish); an exception? Second, and this is a general issue underlying partial validation - probably part of what makes this issue so hairy - the full-model validation, and the resulting error messages, run the risk of being pretty remote from what the user actually did. It feels to me that form validation needs to be a step that focuses on values the user entered in the form, and that full-model validation should come as a second step, possibly adding more messages and tying them to particular form elements. I think in many cases the two types of validation deserve separation in code; model validation might need to be more expensive (e.g., hit the DB), and obviously model validation cannot rely on per-field idioms like form validation does. As a very rough sketch, perhaps something along the lines of: try: form.validate() obj = form.save(commit=False) obj.user = request.user obj.validate(form=form) obj.save() redirect(obj) except FormValidationError, ModelValidationError: redisplay the form of course, the above can also be written with a couple of if/else clauses, but perhaps routing validation errors as exceptions may feel more natural in Python. I don't really like the obj.validate(form=form) line; perhaps it needs to be form.validate_object(obj), or even a context manager such as: with form.validation(): form.validate() ... obj.validate() # having the form.validation CM allows the object validation to somehow tie the errors to the form fields, if so desired ... Yishai On Fri, 29 Apr 2011 05:13:31 +0300, Carl Meyer wrote: form.is_valid()? -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: "unique_together" only validated in modelform.is_valid() if ALL of the referenced fields (Ticket #13091)
Hi Lior, (moved from another thread) On Apr 29, 12:16 am, Lior Sion wrote: > I looked at the sample you wrote on the other thread (unique together > on username and date, and having a null username with a given date) of > when the old behavior is the right one and it didn't quite convince me > - I do believe that the right implementation would fail a case of > NULLed username and repeating dates, when a unique together exists. That would clearly violate the currently-established consistent semantic for modelform validation, which is that the modelform only validates the fields that are actually included in the form. Any fields that are not included in the form, it must be assumed that they might change before saving, and their current value is unreliable (it might just be the default value for a new object, or something else entirely, it's not necessarily a value that's actually intended to be validated or saved). In essence, what you are proposing is validation of "possibly-junk" data. Because of this, any fix has to be absolutely 100% sure that it will never generate a false positive on the unique_together check because of the non-included field's value, or else we're violating the documented expectation that non-included fields' values are not part of validation. The exceptions for default and blank in the current patch was my best attempt to achieve that, but I don't even think that's sufficient - really, modelform validation can't assume anything about the data on self.instance for fields that aren't in the form, regardless of whether the field is blank or has a default. This is why I'm saying that the current expected semantics, that a modelform can provide meaningful validation while ignoring some fields of the model, is inherently broken. No matter what you do with constraints that include some included and some excluded fields (whether they are unique_together or custom clean methods, as in Mikhail's example), you're doing the wrong thing - you can't validate the constraint because you can't use some of the data that's needed to validate it, but if you drop the constraint entirely (as we do now) you require the developer to re-validate everything themselves later (and probably stuff those later model-validation errors back into the form, too, and then break out of the is_valid clause because its no longer valid), which is really ugly and painful in custom code and currently not done at all in the admin. And that's why I think the only long-term solution, that isn't just going to cause more problems, is to begin moving towards a new expectation for validating modelforms, where its expected that full model validation is run, and any tweaks to the model must be made before validation rather than after. Carl -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: "unique_together" only validated in modelform.is_valid() if ALL of the referenced fields (Ticket #13091)
Hi Mikhail,
On Apr 24, 7:46 am, Mikhail Korobov wrote:
> The issue is not only with unique_together indeed. Please correct me if I'm
> wrong, but it seems there is no way currently to use model validation with
> fields dependent on each other when one of these fields is not POSTed
> directly by user. Example:
>
> # models.py
> class Ticket(models.Model):
> created_by = models.ForeignKey(User, related_name='created_tickets')
> responsible = models.ForeignKey(User, related_name='todo')
>
> def clean():
> # we don't want to allow tickets where created_by == responsible for
> some reason
> from django.core.exceptions import ValidationError
> if self.created_by == self.responsible:
> raise ValidationError('Responsible is incorrect.')
>
> # views.py
> class TicketForm(forms.ModelForm):
> class Meta:
> model = Ticket
> exclude = ['created_by']
>
> @login_required
> def add_ticket(request):
> form = TicketForm(request.POST or None)
> if form.is_valid():
> ticket = form.save(commit=False)
> ticket.created_by = request.user
>
> # todo: handle ticket.full_clean()
>
> ticket.save()
> return redirect('ticket_list')
> return TemplateResponse(request, 'tickets/add.html', {'form': form})
>
> Model.clean method is always called on form validation (unlike field
> validators that are excluded if field is excluded from form). And we can't
> write validator for 'responsible' field or for 'created_by' field (so that
> it will be excluded from validation on form.is_valid) because this validator
> won't have an access to other model fields. So the only option here seems to
> move validation from model level to form level.
>
> Please consider this use case while rethinking model validation. The
> proposed context manager seems to be a very good idea and I hope it will fix
> this issue as well.
Sorry, I missed this somehow earlier. I agree with you that this is
another example of why attempting to do partial model validation is
simply broken. I think the context manager proposal (which I've posted
in a new thread) does fix this case as well, and I'd be happy for your
comments on it.
Carl
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: ModelForm validation - a better way?
Hi Lior, thanks for commenting. On 04/29/2011 12:16 AM, Lior Sion wrote: > I think that forcing programmers to remember quite a long process of > for validation each time is a wrong approach, especially if only done > to support backward code that behaves in "unnatural" way. I'm not sure why you think it's "quite long," other than that one of the examples above included a full view and the other one didn't. For reference, here are equivalent side-by-side examples for the case where you don't want to modify the object before saving: current: if form.is_valid(): obj = form.save() redirect(obj) proposed: obj = form.finish() if form.is_valid(): redirect(obj) And for the case where you do want to modify the object before saving: current: if form.is_valid(): obj = form.save(commit=False) obj.user = request.user obj.save() redirect(obj) proposed: with form.finish(tweak=True) as obj: obj.user = request.user if form.is_valid(): redirect(obj) (This is assuming we use a flag argument to determine whether you want the context manager. I'm not a huge fan of this, but I like it better than requiring the context manager and using "pass"). The proposal here is the same length as current code in the first case, and one line shorter in the second case. So there may be valid objections to the proposal, but I won't accept length as one of them ;-) > I looked at the sample you wrote on the other thread I'm happy to continue conversation on that, but I'll do it in the other thread for clarity. Thanks, Carl -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Trying to find Dojo Developers
Hi, I was wondering if any of you could help me out. I'm currently looking for a Dojo Developer expert to work on a 3 month contract in Belfast, Northern Ireland for a client of ours. I'm a researcher with Morgan McKinley in Dublin and we've hit a wall trying to find a dojo developer who would want to work in a contract. Do you know of any organizations in Ireland or around Europe that might provide dojo developers for contract work? Any help you could give me on this would be most appreciated. Thank you very much, mcflurry7_11 -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: ModelForm validation - a better way?
Carl,
I also ran into this issue and opened one of the tickets about it, so
I do have an interest in it.
I think that forcing programmers to remember quite a long process of
for validation each time is a wrong approach, especially if only done
to support backward code that behaves in "unnatural" way.
I looked at the sample you wrote on the other thread (unique together
on username and date, and having a null username with a given date) of
when the old behavior is the right one and it didn't quite convince me
- I do believe that the right implementation would fail a case of
NULLed username and repeating dates, when a unique together exists.
On Apr 29, 5:13 am, Carl Meyer wrote:
> Hi all,
>
> We have a number of tickets open (at least #12028, #13249, #13091,
> #15326, and #15860 -- #13091 is the active one) reporting problems with
> unique_together constraints in our attempts to validate arbitrary
> partial models, when validating a ModelForm with some fields excluded.
>
> Eduardo Gutierrez and I have spent some time looking at this problem
> recently, and my main feeling is that validation of arbitrarily partial
> models in the face of unique_together constraints has no reliably right
> answer, and we'd do better to move away from it altogether.
>
> Fortunately, I think we can do better. The reason we have to validate
> partial models is because of this idiom:
>
> if form.is_valid():
> obj = form.save(commit=False)
> obj.user = request.user # for instance
> obj.save()
>
> But there is no reason those tweaks to the model have to happen after
> form validation. If we encouraged an idiom where the tweaks happen
> before form validation, we could just run full model validation and
> avoid all the error-prone complexity of validating partial models.
>
> Alex and I talked over some possibilities for a context manager
> available from a new method on ModelForms, that you'd use like this
> (idea originally from Łukasz Rekucki [1], somewhat modified):
>
> def my_view(request):
> if request.method == "POST":
> form = MyModelForm(request.POST)
> with form.finish() as obj:
> obj.user = request.user
> if form.is_valid():
> return redirect(obj)
> else:
> form = MyForm()
> return render(request, "foo.html", {"form": form})
>
> form.finish() returns a context manager which returns form.instance from
> its __enter__ method, as "obj" here, allowing the user to do any
> tweaking they like within the body of the context manager. On exit, the
> context manager performs _full_ model validation. Any validation errors
> are saved to the form, as usual. If validation succeeds, the model
> instance is saved.
>
> The following check to form.is_valid(), then, is just for purposes of
> managing view logic (redirect, or redisplay form?). Actual validation
> has already happened, so it would just be checking self._errors (this
> isn't a change, that's already how .is_valid() works).
>
> For backwards compatibility, there would be no change to the existing
> behavior if you don't use the new .finish() context manager -
> form.is_valid() would trigger possibly-partial validation just as it
> does now, and do the best it can. But the admin could be converted to
> use the new idiom (with a new ModelAdmin method that would be called
> from within the context manager to allow model-tweaking before
> validation), which would solve the admin-related bugs. And the
> documentation could recommend the new idiom over the old, particularly
> for incomplete modelforms with unique_together constraints.
>
> Open questions:
>
> 1. What if you don't need to tweak the instance, but you do want to
> trigger the new full validation behavior (i.e. your form excludes some
> fields, but you initially passed an instance to the form that you're
> confident has the excluded fields already set correctly - this would be
> the case for e.g. the parent FK in admin inlines)? Perhaps form.finish()
> could take an argument that determines whether it returns the context
> manager or just returns form.instance directly?
>
> 2. Is it going to be too weird for people to adjust to the idea that
> they get their model instance out of the form before they (explicitly)
> call form.is_valid()?
>
> Other issues we've overlooked, or ways this could be improved? Use cases
> this doesn't handle?
>
> Thanks,
>
> Carl
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: NoSQL support
On Apr 28, 4:59 pm, Russell Keith-Magee wrote: > I haven't used the django-nonrel branch, and I'm not aware of anyone that > I know and trust that has. For what it's worth, I am currently using django-nonrel and its companions djangoappengine, djangotoolbox, and django-dbindexer [1] in a project on Google App Engine and so far I have been impressed by how stable and simple it is to use and by how closely it tracks Django trunk. I can't vouch for the implementation details as I haven't reviewed them very closely and I don't know enough about all the issues at stake, but I can vouch for its simplicity of use and for the fact that it does work in real projects. All in all, django-nonrel appears to me as a very promising solution. Cheers, Julien [1] http://www.allbuttonspressed.com/projects/djangoappengine#installation -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Suggestion for improvement to template block/extends tags
Hey, this pattern are nested decorators in templates.
Most people here are probably against having nested template tags with
the same name. I think it does only make sense when there's somehow a
distinction between a
"placeholder" and "content". Because otherwise, templates don't know
which block in the parent template need to be chosen when several have
the same name. (You'd say the most inner block, but that's not obvious
-- I think.)
I once wrote a {% decorate %} template tag for a little similar
behaviour.
https://github.com/citylive/django-template-tags/blob/master/src/django_template_tags/templatetags/decorate.py
That would become:
template.html:
{% decorate (either "a.html" or "b.html") %}
middle
{% enddecorate %}
b.html:
{% decorate "a.html" %}
left {{ decorator.content }} right
{% enddecorate %}
a.html:
first {{ decorator.content }} last
Hope that helps.
I once tried to get this into django trunk, but did not yet got
accepted.
On 29 avr, 02:11, amagee wrote:
> I sometimes run into a situation where I want a template to be able to
> extend from one of a set of possible base templates, which I achieve
> by passing a "base_template" variable in the context to the {% extends
> %} tag. Where this gets stuck, though, is if one of the possible
> bases extends one of the other possible bases.
>
> For example:
>
> base_a.html:
> first {% block content %}{% endblock %} last
>
> base_b.html
> {% extends "base_a.html" %}
> {% block content %}left {% ??? %} right{% endblock %}
>
> template.html
> {% extends (either "base.a.html" or "base_b.html") %}
> {% block content %}middle{% endblock %}
>
> I'd like to be able to code template.html so that if it extends
> base_a.html, the result is "first middle last", but if it extends
> "base_b.html", the result is "first left middle right last".
>
> I _think_ it would make sense to implement this with an improvement to
> the semantics of the {% block %} tag, starting by allowing nested tags
> with the same name.
>
> If base_b.html were:
> {% extends "base_a.html" %}
> {% block content %}left {% block content %}{% endblock %} right{%
> endblock %}
>
> Then the {% block content %} in template.html could override the
> _inner_ block in base_b.html. I think this behaviour is pretty
> logical and consistent, unless I've missed something. I've hacked
> around a bit with loader_tags.py but I'm finding it quite difficult to
> get what I want with my limited understanding of how it works.
>
> Do people think this idea makes sense? Is it worth taking the time to
> write a patch for it?
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
