Re: Proposal: Replace django.contrib.formtools.wizard

[Stephan Jäkel]

Hi,

about 4 months ago, I started this thread. I want to give some news on 
django-formwizard to keep you all up2date.


Stephan Jäkel wrote:

The last not solved problem is handling file uploads.


This problem is solved. The formwizard supports file uploads in any step. I 
solved the problem by providing or at least requiring the user to provide a 
file storage for storing temp data. The storage backend is responsible for 
removing old data/files. The formwizard only removes the temp. data when the 
wizard gets finished by the uploading user.


I'm currently working on a example storage which can clean its data itself.

I also did some work on getting the app thread-safe.

Another new feature is conditions for form-steps. Using these conditions you 
can skip/show steps depending on - for example - other steps' data. (see 
testapp2 in the test_project)


The docs got some updates too but need to be extended.. The docs are hosted 
on RTD, http://django-formwizard.readthedocs.org/


See the develop branch for the latest changes, 
http://github.com/stephrdev/django-formwizard/tree/develop


Cheers,

steph

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: CSRF Middleware/SSL/Firefox 3.6.8 bug

[Luke Plant]

On Tue, 2010-09-07 at 03:31 +0100, Luke Plant wrote:

> 1) The attacker can have the same (public) IP address as the victim if
> they are both behind the same firewall, and this will in fact be one of
> the most practical ways to launch a MitM + CSRF attack on HTTPS.

Here and in my other message I meant 'NAT router', not
'firewall' (although they might be the same thing). Sorry for any
confusion.

Luke

-- 

Luke Plant || http://lukeplant.me.uk/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: CSRF Middleware/SSL/Firefox 3.6.8 bug

[Paul McMillan]

Luke,

Your last couple replies to this thread have been very helpful. Thank
you for cogently clarifying this situation and patiently keeping at
it.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: CSRF Middleware/SSL/Firefox 3.6.8 bug

[Luke Plant]

On Mon, 2010-09-06 at 22:39 +0200, Patryk Zawadzki wrote:

> Another approach would be not to use a cookie at all. For each {%
> csrf_token %} use a slightly modified variant of the above
> encode_cookie function with:
> 
> values = {
> 'host: request.META['HTTP_HOST'],
> 'scheme': request.is_secure(),
> 'user_ip': request.META['REMOTE_ADDR'],
> 'user_agent': request.META['HTTP_USER_AGENT'],
> 'ttl': time.time() + 30*60,
> }
> 
> Then when handling a POST request, decipher the token and compare each
> META field with the ones from the request and validate ttl against
> time.time(). I believe it's not less secure than the current
> implementation and solves two problems:
>
> 1) each form served gets its own ttl, an attacker can't keep pinging
> the server to keep the token alive
> 2) each token serves for a single use and will inevitably timeout in
> 30 minutes while still allowing you to open two forms in two browser
> tabs and submit each of them separately

Your method is quite flawed:

1) Use of IP address - a bad idea for the reasons I mentioned in my
other message.

2) The use of user agent does nothing to stop an attacker, even for an
attacker who isn't a MitM:

Consider an attacker who lures you to his site e.g.:
http://evil.com/somepage.php

somepage.php can read your user agent, and make a request to
http://target.com/some-page-with-form/ with the same user agent to get a
valid CSRF token.  (If he is behind the same firewall as you, he will
automatically have the same public IP, so adding the IP address is not a
perfect cure for this, even if we could do it, which we can't).

somepage.php then returns a page which has a form which targets
http://target.com/some-page-with-form/ and includes the CSRF token. The
token is valid and unused, and the attacker can proceed.

I don't think that adding a timeout will really ever help with CSRF.
The nature of CSRF attacks means that the attacker is massively more
likely to be able to attack 'now' (which is within a few seconds, the
time for a few HTTP requests to complete) rather than 'later', so
timeouts just don't help. In addition, they add nuisance for the user -
it's quite possible for someone to leave their machine for 30 minutes
and come back to it and want to carry on what they were doing.

Luke

-- 
 A mosquito cried out in pain:
 "A chemist has poisoned my brain!"
 The cause of his sorrow
 was para-dichloro-
 diphenyltrichloroethane

Luke Plant || http://lukeplant.me.uk/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

My problem with proposed Custom FilterSpec implementation

[subs...@gmail.com]

Hello there,

I'm watching #5833 and I have a bit of a problem with the
implementation.  A good example of their proposal is here:

http://code.djangoproject.com/ticket/5833#comment:68

The API is remarkably similar to Gaynor's django-filters. My problem
with this is that your filtering essentially becomes
Model.objects.filter(x__y=x).filter(y__x=y) which creates a join each
time, whereas you may not really want a join.

To get around this limitation I propose having the FilterSpec's return
Q objects and then passing them all into filter(), however I've been
stumped as to how to do this in such a way that you allow people to
use AND or OR, depending on their preference.

I'm afraid if this issue isn't worked out the current proposal (which
is Accepted) will then become gospel and many uses cases (mine!) will
be left out.

-Steve

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #13870: Correctly handling database isolation (in PostgreSQL)

[David De La Harpe Golden]

On 07/09/10 14:05, Patryk Zawadzki wrote:

> Not deprecated in Django sense. I don't think it's going away any time
> soon. Just that it's been discouraged for a while at the
> psycopg/database level.

Well, maybe "discouraged" is fair, but it does exist and you have the
option (and if you use TransactionMiddleware with it on you can still
get isolation around views, with nothing stopping you entering
transaction management outside views either...).

Also as Gabriel H. mentioned, there's also calling connection.close() to
think about, for celery specifically perhaps you could connect a handler
for celery signal task_postrun [1] to do so for you, a bit like as is
done for the web case.

So. There are certainly other definite annoyances with django
transaction handling (what Kirit S. described as lack of composition,
see also [2]) that might merit some changes, but I'm not sure there's
immediate need for django-level changes here specifically.

[1]
http://ask.github.com/celery/reference/celery.signals.html#celery.signals.task_postrun

[2] https://docs.google.com/View?id=dgxrkmrk_2hmk4fmhg



-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #13870: Correctly handling database isolation (in PostgreSQL)

[David De La Harpe Golden]

On 07/09/10 16:16, Robert Gravsjö wrote:


>> with db.isolation(ISOLATION_SERIALIZABLE):
>>  # ...
>>
> 
> You don't want to serialize access to the database in a multi-user
> application.

Just in case: serializable transaction != serializing access - among
serializable level transactions, the property of /looking like/ they
occurred serially in some consistent order is supposed to be maintained.
Maintaining that property may well result in quite a lot of actual
serialization and transactions needing retries I suppose, depending on
sophistication of the implementation.

Anyway. Postgresql's "serializable" isn't "complete" presently, like
oracle it's actually "snapshot" isolation:
http://www.postgresql.org/docs/9.0/static/transaction-iso.html#MVCC-SERIALIZABILITY
http://wiki.postgresql.org/wiki/Serializable
http://en.wikipedia.org/wiki/Isolation_(database_systems)#SERIALIZABLE
http://en.wikipedia.org/wiki/Snapshot_isolation

Django doesn't currently have support for setting it AFAICS (and there's
currently an assert restricting to level 0/1 in _set_isolation_level()),
but maybe it would be sensible to add some support (not necessarily very
fine-grained, perhaps just a django databases OPTIONS = {'isolation':
'snapshot'}  a bit like 'autocommit' that means to use psycopg2 level
2|(0/2) instead of 1|(0/1)  [autocommit off|on] for that database).

'snapshot' just being pedantic. 'serializable' would be consistent with
postgresql/oracle usage of course.


















-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

ModelForm possible validation bug

[David Reynolds]

Hi folks,

I am running into a validation problem with ModelForms - here is a quick 
summary of what is happening

Take a simple model - here we call Model.full_clean as part of the custom save 
method to enforce the validation:

from django.db import models

class Stuff(models.Model):
"""(Stuff description)"""
name = models.CharField(max_length=100)
age = models.IntegerField()
description = models.TextField(blank=True)

def __unicode__(self):
return self.name

def save(self, *args, **kwargs):
self.full_clean()
return super(Stuff, self).save(*args, **kwargs)

Simple ModelForm for the above - nothing unusual here:

from django import forms


class StuffModelForm(forms.ModelForm):

class Meta:
model = Stuff

# Instantiate the form with some data
form = StuffModelForm({
'name': 'Fred',
'age': '56',
'description': 'Old Fred is 57',
})

# Check the form is valid
if form.is_valid():
obj = form.save(commit=False) # save but don't commit
obj.age = 27 # Change the age
obj.save() # Do a full save of the object

obj.name = 'Ted' # Change the name
obj.save() # Attempt to update the db record

# you get this error:
ValidationError: {'id': [u'Stuff with this ID already exists.']}

This happens because the ModelForm sets ModelForm.instance._adding to True and 
never cleans it up. This is then tested for in Model._perform_unique_checks  
and the uniqueness checks are executed if it is set to True which it always is 
after calling ModelForm.save()

Is this a bug? Surely it can't be that unusal to want to call 
Model.full_clean() before each save?

-- 
David Reynolds
da...@reynoldsfamily.org.uk

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #13870: Correctly handling database isolation (in PostgreSQL)

[Robert Gravsjö]

Patryk Zawadzki skrev 2010-09-07 16.47:

On Mon, Sep 6, 2010 at 7:12 PM, Patryk Zawadzki  wrote:

It would be more useful if you could explicitly
enter_isolation_block() and leave_isolation_block() as needed
(currently there is no way to commit the isolating transaction other
than doing a raw SQL query or accessing psycopg internals).


Another usecase I just came along:


I don't think this is the appropriate forum for this discussion.



Say I have to generate some unique string for the database. Something like that:


Then you either generate some form of UUID for that or let the database 
handle it in a transaction safe manner (like using SERIAL in the case of 
PostgreSQL).




 potential = base = slugify(obj.bar)
 suffix = 0
 while True:
 if not Baz.objects.filter(foo=potential).exists():
 obj.foo = potential
 obj.save()
 break
 suffix += 1
 potential = '%s-%s' % (base, suffix)

Except it's possible that another process or thread commits an
identical object right between the call to exists() and the save().
What I'd really want to do is something closer to this:

 potential = base = slugify(obj.bar)
 suffix = 0
 found = False
 while not found:
 enter_isolation_block(ISOLATION_SERIALIZABLE)
 if not Baz.objects.filter(foo=potential).exists():
 obj.foo = potential
 obj.save()
 found = True
 leave_isolation_block()
 suffix += 1
 potential = '%s-%s' % (base, suffix)

Of course with new python versions we can get the extra sugar coating:

with db.isolation(ISOLATION_SERIALIZABLE):
 # ...



You don't want to serialize access to the database in a multi-user 
application.


Regards,
roppert

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #13870: Correctly handling database isolation (in PostgreSQL)

[Patryk Zawadzki]

On Mon, Sep 6, 2010 at 7:12 PM, Patryk Zawadzki  wrote:
> It would be more useful if you could explicitly
> enter_isolation_block() and leave_isolation_block() as needed
> (currently there is no way to commit the isolating transaction other
> than doing a raw SQL query or accessing psycopg internals).

Another usecase I just came along:

Say I have to generate some unique string for the database. Something like that:

potential = base = slugify(obj.bar)
suffix = 0
while True:
if not Baz.objects.filter(foo=potential).exists():
obj.foo = potential
obj.save()
break
suffix += 1
potential = '%s-%s' % (base, suffix)

Except it's possible that another process or thread commits an
identical object right between the call to exists() and the save().
What I'd really want to do is something closer to this:

potential = base = slugify(obj.bar)
suffix = 0
found = False
while not found:
enter_isolation_block(ISOLATION_SERIALIZABLE)
if not Baz.objects.filter(foo=potential).exists():
obj.foo = potential
obj.save()
found = True
leave_isolation_block()
suffix += 1
potential = '%s-%s' % (base, suffix)

Of course with new python versions we can get the extra sugar coating:

with db.isolation(ISOLATION_SERIALIZABLE):
# ...

-- 
Patryk Zawadzki

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #13870: Correctly handling database isolation (in PostgreSQL)

[Patryk Zawadzki]

On Tue, Sep 7, 2010 at 2:57 PM, David De La Harpe Golden
 wrote:
> On 06/09/10 14:29, Patryk Zawadzki wrote:
>> The problem only exists when Django sets isolation level to 1, if you
>> use the deprecated "autocommit" setting, you will not be affected.
> Uh. Is this deprecated? At what layer? Since when?  I've been using it*
> happily for a while (modulo that one known "make sure to read once
> before writing" issue with django 1.1), was not aware of any deprecation
> of it?

Not deprecated in Django sense. I don't think it's going away any time
soon. Just that it's been discouraged for a while at the
psycopg/database level.

-- 
Patryk Zawadzki

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #13870: Correctly handling database isolation (in PostgreSQL)

[David De La Harpe Golden]

On 06/09/10 14:29, Patryk Zawadzki wrote:

> The problem only exists when Django sets isolation level to 1, if you
> use the deprecated "autocommit" setting, you will not be affected.
> 

Uh. Is this deprecated? At what layer? Since when?  I've been using it*
happily for a while (modulo that one known "make sure to read once
before writing" issue with django 1.1), was not aware of any deprecation
of it?

* i.e. django's postgresql_psycopg2  autocommit -> True setting which
amounts to setting isolation level 0 => autocommit at the psycopg2 level
(you can still enter transaction management when you want to).
http://docs.djangoproject.com/en/dev/ref/databases/#autocommit-mode

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: CSRF and Forms

[Patryk Zawadzki]

On Tue, Sep 7, 2010 at 8:38 AM, Russell Keith-Magee
 wrote:
> Firstly, I'm not wild about "secure=request.validated". This looks
> like a really simple way for people to say "secure=True" as a way of
> "fixing" CSRF support that they can't get to work. The choice of
> argument on the attribute isn't optional -- it *must* be
> request.validated. So really, it's the request that is the argument
> that needs to be passed in. The good news on this point is that a
> "request aware form" is something that has been floated in other
> discussions recently. I'll be sure to raise it at the DjangoCon
> sprints this week as a topic for discussion.

After giving it a second thought on my ride to work I think
request.is_valid() would be just as useful and would not require us
breaching the form/request layer separation.

-- 
Patryk Zawadzki

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: CSRF and Forms

[Patryk Zawadzki]

On Tue, Sep 7, 2010 at 8:38 AM, Russell Keith-Magee
 wrote:
> Firstly, I'm not wild about "secure=request.validated". This looks
> like a really simple way for people to say "secure=True" as a way of
> "fixing" CSRF support that they can't get to work. The choice of
> argument on the attribute isn't optional -- it *must* be
> request.validated. So really, it's the request that is the argument
> that needs to be passed in. The good news on this point is that a
> "request aware form" is something that has been floated in other
> discussions recently. I'll be sure to raise it at the DjangoCon
> sprints this week as a topic for discussion.

Agree. In the other CSRF thread I provided a shorter, slightly safer,
cookie-less implementation that works withut Referer headers and
limits each form's lifespan to 30 minutes. I have since implemented a
test form that works like this:

form = Form(request.POST or None, request.FILES or None, request.META)

The form is able to both issue a signed CSRF token (you can just write
{{ form.as_p }} or {{ form.csrf_token }}) and validate it (checks if
POST contains a token, checks the signature and fails form validation
if needed). It could cooperate with the middleware to redirect to an
error if no token was present at all.

Having it this way is useful as it's possible for someone to keep a
form open longer than said 30 minutes and it's more practical to
simply ask for confirmation than to drop all the entered data and
redirect to an error page.

> Secondly, IMHO there is a lot of value in the fact that Django forces
> the raising of a 403, rather than a 200 with an error message on the
> page. This isn't a form input error. It's a catastrophic problem that
> should only be observed when the user is actually under attack, or if
> cookies aren't available. Displaying a CSRF failure in the same way as
> an error for not putting 12 digits in your credit card number strikes
> me as the wrong way to visualize the error case. It implies that the
> problem can be fixed by user interaction when it can't -- at least,
> not by fixing form inputs. In order to give instructions on why
> cookies are needed and how to enable them, we need a lot more real
> estate... like a CSRF view.

See above, I see the value of the error in case of a CSRF attack. In
such cases there will be no valid token present in the payload. My
implementation can however differentiate between a missing token, an
invalid one and one that is simply too old to accept it.

-- 
Patryk Zawadzki

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #13870: Correctly handling database isolation (in PostgreSQL)

[Patryk Zawadzki]

On Tue, Sep 7, 2010 at 8:29 AM, Thomas Guettler  wrote:
> If you have a daemon that lives forever, I would do it like this (untested
> if this does not leave an idle transaction open):
> The place where the daemon hangs around if nothing can be done lives must
> not use the ORM. If there is something to be done, the daemon calls methods
> that use the commit_on_success decorators.

Unfortunately you don't always get to choose what to call with what
decorators. For example we have a DBus daemon that acts as an RPC
server. Creating a separate function for each select is not really
practical :)

-- 
Patryk Zawadzki

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: CSRF and Forms

[Russell Keith-Magee]

On Tue, Sep 7, 2010 at 4:08 AM, Patryk Zawadzki  wrote:
> Hi,
>
> Since CSRF is already being reafactored up-side down in the trunk, I
> thought it might be a good idea to propose a slight modification.

Erm... it is? That's news to me.

Django 1.2 introduced a bunch of very big changes, but those changes
are signed, sealed and delivered. I'm not aware of any major changes
that have already been made, or are planned.

> Currently CSRF either falls through to the resolved view function or
> calls settings.CSRF_FAILURE_VIEW. More than once I've found myself
> wanting something in between. In such cases I'd like the logic flow to
> be able to reach the view, just telling me that CSRF did not validate.
>
> Let's say we add a new attribute to the request, call it "validated"
> and make it default to True for GET and False for everything else. Now
> split the CSRF middleware into two separate pieces of code. One
> middleware that does the validation and sets request.validated to True
> on success. One middleware that checks for (request.method == 'POST'
> and not request.validated) and in such cases returns
> settings.CSRF_FAILURE_VIEW.
>
> "How is that useful?" I hear you ask.
>
> class SecureForm(forms.Form):
>    def __init__(self, *args, **kwargs):
>        self.secure = kwargs.pop('secure', False)
>        return super(SecureForm, self).__init__(*args, **kwargs)
>    def _clean_form(self, *args, **kwargs):
>        if not self.secure:
>            self._errors[NON_FIELD_ERRORS] = self.error_class([
>                'We could not confirm that the request originated from
> your machine. Please resubmit to continue.'
>            ])
>        else:
>            super(SecureForm, self)._clean_form(self, *args, **kwargs)
>
> def MyForm(SecureForm):
>    foo = forms.CharField()
>
> def my_view(request):
>    myform = MyForm(request.POST or None, request.FILES or None,
>            secure=request.validated)
>    if myform.is_valid():
>        # ...
>        pass
>    return direct_to_template(request, 'my.html', {'form': myform})

Hrm. I see what you're doing here.

Firstly, I'm not wild about "secure=request.validated". This looks
like a really simple way for people to say "secure=True" as a way of
"fixing" CSRF support that they can't get to work. The choice of
argument on the attribute isn't optional -- it *must* be
request.validated. So really, it's the request that is the argument
that needs to be passed in. The good news on this point is that a
"request aware form" is something that has been floated in other
discussions recently. I'll be sure to raise it at the DjangoCon
sprints this week as a topic for discussion.

Secondly, IMHO there is a lot of value in the fact that Django forces
the raising of a 403, rather than a 200 with an error message on the
page. This isn't a form input error. It's a catastrophic problem that
should only be observed when the user is actually under attack, or if
cookies aren't available. Displaying a CSRF failure in the same way as
an error for not putting 12 digits in your credit card number strikes
me as the wrong way to visualize the error case. It implies that the
problem can be fixed by user interaction when it can't -- at least,
not by fixing form inputs. In order to give instructions on why
cookies are needed and how to enable them, we need a lot more real
estate... like a CSRF view.

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: #13870: Correctly handling database isolation (in PostgreSQL)

[Thomas Guettler]

Patryk Zawadzki wrote:
> On Mon, Sep 6, 2010 at 4:47 PM, Robert Gravsjö  wrote:
>> Patryk Zawadzki skrev 2010-09-06 15.29:
>>> The isolating transaction keeps going on until you either (1) commit,
>>> (2) rollback or (3) disconnect. Django only commits/rollbacks the
>>> transactions it explicitly starts, it does not care about the
>>> implicitly started isolating transaction. That's what results in
>>> "  in transaction" and I can reproduce it with a two-line view
>>> that does a simple SELECT with no transaction middleware involved.
>> Can you please show me the code you're running to reproduce this?
> 
> Right, I misremembered the original problem. I've now found the
> testing environment.
> 
> The problem is not with regular views but with Celery tasks,
> long-running management commands such as daemons and any other place
> where you access the ORM from outside of the usual
> request→dispatcher→view→response flow

I use the TransactionMiddleware for requests, and
scripts which are started from the shell use the commit_on_success decorator.
Utility methods that are used in both ways (shell and request) don't use
transaction handling methods.

Long running tasks are an exception, here I use something like this:

for i, .. in enumerate(...):
if i%1000==0:
commit()

If you have a daemon that lives forever, I would do it like this (untested
if this does not leave an idle transaction open):
The place where the daemon hangs around if nothing can be done lives must
not use the ORM. If there is something to be done, the daemon calls methods
that use the commit_on_success decorators.

  Thomas


-- 
Thomas Guettler, http://www.thomas-guettler.de/
E-Mail: guettli (*) thomas-guettler + de

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: python 3.x

[Russell Keith-Magee]

On Sun, Sep 5, 2010 at 5:05 PM, VernonCole  wrote:
> "Once we're at a Django 2.6 minimum supported version, using 2to3 to
> maintain
> parallel implementations becomes a lot easier."
>
> As much as I admire Russ, and I do, I don't think that the above
> statement is correct.

My apologies -- in the haste of getting a response out, I was a little
lax in my choice of words. What you've described - i.e.,

* A single maintained 2.X source tree
* An auto-generated 3.X source tree,
* When 2.X support is dropped, the migration script is run one last
time to migrate the source tree to be 3.X

is pretty close to what I had in my head as the likely path. In
practice, I'm sure there will be some complications, but we won't
really know what they are until we get serious about 3.X support.

> For a short time on the pywin32 team we tried to "maintain parallel
> implementations" and found that it was a mistake we had to  undo.  The
> correct approach is to maintain a SINGLE implementation -- in Python 2
> format -- and use 2to3 as a tool when the code happens to be running
> on Python 3+.  2to3 should be run by distutils when it detects that
> setup.py is being run by Python3. It should NOT be run manually by a
> human.
>
> Then, some years in the future when the last Python 2.7 engine fades
> away, you will run 2to3 once for the last time, and THEN maintain in
> Python 3 format. You do NOT write your code with print() functions,
> etc.. Simply roll any needed refactoring into the trunk at the
> earliest opportunity, and make sure you don't break them during
> maintenance.
>
> That's my advice from my experience. The code I am supporting runs on
> any version of Python from 2.3 thru 3.1, including IronPython.

It's good to know that there are people in the community that have
done this in anger on other projects. If anyone can provide patches
for refactors that are necessary in order to simplify the 3.X
migration process, I'm happy to apply those patches -- and as I
indicated, I already have applied a couple of patches for exactly this
reason; for example changeset 13509 was to change the way we use
sorting functions to avoid a keyword argument that has been deprecated
for 3.X.

Yours,
Russ Magee %-)

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.