Re: Small problem with HttpResponseRedirect

[Paul McMillan]

As Ian said, Django does the right thing here according to my tests
too, and generates the absolute URIs required by RFC 2616. If you've
figured out some way to actually get location headers that are
noncompliant, that would be a bug, but the handling of // is correct.

-Paul

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Small problem with HttpResponseRedirect

[Ian Clelland]

On Mon, Dec 5, 2011 at 2:00 PM, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:

> Not sure if this should have a bug ticket raised or not.. wanted to get
> core devs thoughts.
>
>
> _redir = "//your/path/with/an/extra/slash/for/whatever/reason"
> HttpResponseRedirect(_redir)
> returns "Location:
> http://your/path/with/an/extra/slash/for/whatever/reason;
>
> _redir = "/your/path/with/no/extra/slash"
> HttpResponseRedirect(_redir)
> returns "Location: /your/path/with/no/extra/slash"
>
> It seems that if the URL has an extra slash on the start of it,
> redirection doesn't work properly.
>
> Should this be classed as bad user sanity checking, or
> HttpResponseRedirect() not being flexible enough?
>
> I personally feel it's not being flexible enough, but would be good to
> hear your thoughts.
>
> Cal
>

Not a core dev, but I can offer my thoughts anyway :)

First, your server should not ever be sending:

Location: /your/path/with/no/scheme

According to RFC2616, and to the comments in django/http/__init__.py, the
Location header *must* contain an absolute URL.
django.http.utils.fix_location_header is supposed to be called on every
response, by the base WSGI handler, and should fix up any URLs that don't
already start with 'http://' or 'https://'. If it doesn't include a host
part (after a //), then Django should insert the name of the currently
active host.[1]

Second, a URL starting with a // is perfectly valid, and is supposed to
represent a "scheme-relative" URL. ie., it includes the hostname, path, and
query string; everything but the actual scheme (http: or https:) They're
useful for constructing links to resources where you have to preserve
https, if the current page is secure, (to prevent browser warnings, privacy
leaks, security holes).

There was some discussion on stack overflow last week on this[2]; it mostly
relates to using // urls in markup, not in a header, though.

I think the existing code is correct; my tests show that

HttpResponseRedirect("//host/path/to/resource/")
yields
Location: http://host/path/to/resource/

and
HttpResponseRedirect("/path/to/resource/")
yields
Location: http://127.0.0.1/path/to/resource/

In both cases, the response fixes have constructed the correct absolute
url, based on what I used to construct the redirect.

I think that if you're occasionally getting a double slash in front of your
redirect urls, you should probably fix the code that is generating them, or
create a new subclass of HttpResponseRedirect that eliminates the redundant
character before adding the header.

>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>

[1] This used to be the value from the Host: request header, but I think
recent security patches have changed that.
[2]
http://stackoverflow.com/questions/8343942/what-is-the-effect-of-starting-a-url-with-and-leaving-out-http
(also referenced:
http://stackoverflow.com/questions/4978235/absolute-urls-omitting-the-protocol-scheme-in-order-to-preserve-the-one-of-the
and
four or five other related questions)

-- 
Regards,
Ian Clelland


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Ian Clelland]

On Mon, Dec 5, 2011 at 5:08 PM, Vinay Sajip  wrote:

> On Dec 6, 12:45 am, Ian Clelland  wrote:
> > After a lot of troubleshooting, and fun installing Python3.2 under
> > virtualenv (Vinay, I think I ran into the same issue as you here:
> https://github.com/pypa/virtualenv/issues/194), I almost have the complete
> > test suite running under MacPorts Python 3.2.2.
>
> That's good news about the state of the tests on MacPorts 3.2.2. I
> hope Carl reads your mail about the virtualenv issue :-)
>
> > While installing Vinay's port (cloned freshly this morning from
> Bitbucket),
> > I had to edit four files, in order to get them to run under Python 3:
> [snip]
> > Did I miss a step here, or not see an error which should have been
> > corrected earlier? Nobody else has commented about Python3 syntax errors,
> > so I'm thinking it's my process.
>
> No, I got those today, too - but not on earlier days, for many days. I
> suspect that it's something to do with merging upstream changes into
> my port, but I haven't pinned it down. I pushed my changes this
> afternoon, so you could try pulling my changes and re-testing.
>


Those definitely work better -- I can go from download to testing, with
only having to copy over the build directory. The only test failure I still
get is this one:

==
ERROR: test_existing (regressiontests.templates.loaders.EggLoaderTest)
A template can be loaded from an egg
--
Traceback (most recent call last):
  File
"/Users/ian/Code/Frameworks/py3k_django/build/tests/regressiontests/templates/loaders.py",
line 88, in test_existing
contents, template_name = egg_loader.load_template_source("y.html")
  File
"/Users/ian/Code/Frameworks/py3k_django/3k/django/template/loaders/eggs.py",
line 28, in load_template_source
raise TemplateDoesNotExist(template_name)
django.template.base.TemplateDoesNotExist: y.html

--


It appears that an error is being returned earlier, in line 25 of
template/loaders/eggs.py.

return (resource_string(app, pkg_name).decode(settings.FILE_CHARSET),
'egg:%s:%s' % (app, pkg_name))


pkg_resources.resource_string is returning a Py3k <'str'>, which has an
'encode' method, but no 'decode'. The AttributeError is getting swallowed
and re-raised as a TemplateDoesNotExist.

pkg_resouces comes from distribute, which I just installed this morning for
the virtualenv

(Pdb) import pkg_resources
(Pdb) pkg_resources.__file__
'/Users/ian/.virtualenvs/py3k_django/lib/python3.2/site-packages/distribute-0.6.21-py3.2.egg/pkg_resources.py'


If I replace line 25 with this block:

resource_name = resource_string(app, pkg_name)
if hasattr(resource_name, 'decode'):
resource_name = resource_name.decode(settings.FILE_CHARSET)
return (resource_name, 'egg:%s:%s' % (app, pkg_name))


then all (expected) tests pass.

--
Regards,
Ian Clelland


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Small problem with HttpResponseRedirect

[Yaşar Arabacı]

To be more spesific:

"Following the syntax specifications in *RFC
1808*,
urlparse recognizes a netloc only if it is properly introduced by '//'.
Otherwise the input is presumed to be a relative URL and thus to start with
a path component."

6 Aralık 2011 04:32 tarihinde Yaşar Arabacı  yazdı:

> Maybe this is about second example in here:
> http://docs.python.org/library/urlparse.html#urlparse.urlparse ?
>
>
> 2011/12/6 Luciano Pacheco 
>
>> On Tue, Dec 6, 2011 at 9:00 AM, Cal Leeming [Simplicity Media Ltd] <
>> cal.leem...@simplicitymedialtd.co.uk> wrote:
>>
>>> Not sure if this should have a bug ticket raised or not.. wanted to get
>>> core devs thoughts.
>>>
>>>
>>> _redir = "//your/path/with/an/extra/slash/for/whatever/reason"
>>> HttpResponseRedirect(_redir)
>>> returns "Location:
>>> http://your/path/with/an/extra/slash/for/whatever/reason;
>>>
>>
>> _redir var has an ambiguous string.
>>
>> Is it malformed, missing the "protocol:" like "http:" or "https:" . Or
>> "malformed" in URI's path part?
>>
>> So, it seems not obvious to handle it. Let's refuse the temptation to
>> guess. ;-)
>>
>> Regards,
>> --
>> Luciano Pacheco
>> blog.lucmult.com.br
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "Django developers" group.
>> To post to this group, send email to django-developers@googlegroups.com.
>> To unsubscribe from this group, send email to
>> django-developers+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/django-developers?hl=en.
>>
>
>
>
> --
> http://yasar.serveblog.net/
>
>


-- 
http://yasar.serveblog.net/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Small problem with HttpResponseRedirect

[Yaşar Arabacı]

Maybe this is about second example in here:
http://docs.python.org/library/urlparse.html#urlparse.urlparse ?

2011/12/6 Luciano Pacheco 

> On Tue, Dec 6, 2011 at 9:00 AM, Cal Leeming [Simplicity Media Ltd] <
> cal.leem...@simplicitymedialtd.co.uk> wrote:
>
>> Not sure if this should have a bug ticket raised or not.. wanted to get
>> core devs thoughts.
>>
>>
>> _redir = "//your/path/with/an/extra/slash/for/whatever/reason"
>> HttpResponseRedirect(_redir)
>> returns "Location:
>> http://your/path/with/an/extra/slash/for/whatever/reason;
>>
>
> _redir var has an ambiguous string.
>
> Is it malformed, missing the "protocol:" like "http:" or "https:" . Or
> "malformed" in URI's path part?
>
> So, it seems not obvious to handle it. Let's refuse the temptation to
> guess. ;-)
>
> Regards,
> --
> Luciano Pacheco
> blog.lucmult.com.br
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>



-- 
http://yasar.serveblog.net/

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Small problem with HttpResponseRedirect

[Luciano Pacheco]

On Tue, Dec 6, 2011 at 9:00 AM, Cal Leeming [Simplicity Media Ltd] <
cal.leem...@simplicitymedialtd.co.uk> wrote:

> Not sure if this should have a bug ticket raised or not.. wanted to get
> core devs thoughts.
>
>
> _redir = "//your/path/with/an/extra/slash/for/whatever/reason"
> HttpResponseRedirect(_redir)
> returns "Location:
> http://your/path/with/an/extra/slash/for/whatever/reason;
>

_redir var has an ambiguous string.

Is it malformed, missing the "protocol:" like "http:" or "https:" . Or
"malformed" in URI's path part?

So, it seems not obvious to handle it. Let's refuse the temptation to
guess. ;-)

Regards,
-- 
Luciano Pacheco
blog.lucmult.com.br

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Vinay Sajip]

On Dec 6, 12:45 am, Ian Clelland  wrote:
> After a lot of troubleshooting, and fun installing Python3.2 under
> virtualenv (Vinay, I think I ran into the same issue as you 
> here:https://github.com/pypa/virtualenv/issues/194), I almost have the 
> complete
> test suite running under MacPorts Python 3.2.2.

That's good news about the state of the tests on MacPorts 3.2.2. I
hope Carl reads your mail about the virtualenv issue :-)

> While installing Vinay's port (cloned freshly this morning from Bitbucket),
> I had to edit four files, in order to get them to run under Python 3:
[snip]
> Did I miss a step here, or not see an error which should have been
> corrected earlier? Nobody else has commented about Python3 syntax errors,
> so I'm thinking it's my process.

No, I got those today, too - but not on earlier days, for many days. I
suspect that it's something to do with merging upstream changes into
my port, but I haven't pinned it down. I pushed my changes this
afternoon, so you could try pulling my changes and re-testing.

Regards,

Vinay Sajip

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Ian Clelland]

After a lot of troubleshooting, and fun installing Python3.2 under
virtualenv (Vinay, I think I ran into the same issue as you here:
https://github.com/pypa/virtualenv/issues/194), I almost have the complete
test suite running under MacPorts Python 3.2.2.

The one unexpected error I am getting is this one:

==
ERROR: test_existing (regressiontests.templates.loaders.EggLoaderTest)
A template can be loaded from an egg
--
Traceback (most recent call last):
  File
"/Users/ian/Code/Frameworks/py3k_django/build/tests/regressiontests/templates/loaders.py",
line 88, in test_existing
contents, template_name = egg_loader.load_template_source("y.html")
  File
"/Users/ian/Code/Frameworks/py3k_django/3k/django/template/loaders/eggs.py",
line 28, in load_template_source
raise TemplateDoesNotExist(template_name)
django.template.base.TemplateDoesNotExist: y.html

--

I'm going to look further into that later tonight.

While installing Vinay's port (cloned freshly this morning from Bitbucket),
I had to edit four files, in order to get them to run under Python 3:

3k/django/contrib/gis/forms/fields.py: One multi-line split string was
being transformed incorrectly

Was:
 'transform_error' : _(u('An error occurred when transforming the geometry
')
 'to the SRID of the geometry form field.'),

Should have been:
 'transform_error' : _(u('An error occurred when transforming the geometry '
 'to the SRID of the geometry form field.')),


3k/django/contrib/gis/geoip/tests.py: Two unicode strings had their 'u'
prefix left on

3k/django/db/backends/oracle/creation.py
3k/django/utils/dictconfig.py: Both of these files were using the 'except
, ' syntax, without the 'as' keyword.

Once I fixed those, (and manually copied the tests directory into build/)
then the pre-compiler would work, and the test suite would run.

Did I miss a step here, or not see an error which should have been
corrected earlier? Nobody else has commented about Python3 syntax errors,
so I'm thinking it's my process.

-- 
Regards,
Ian Clelland


-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Small problem with HttpResponseRedirect

[Adrian Holovaty]

On Mon, Dec 5, 2011 at 4:00 PM, Cal Leeming [Simplicity Media Ltd]
 wrote:
> _redir = "//your/path/with/an/extra/slash/for/whatever/reason"
> HttpResponseRedirect(_redir)
> returns "Location: http://your/path/with/an/extra/slash/for/whatever/reason;
>
> _redir = "/your/path/with/no/extra/slash"
> HttpResponseRedirect(_redir)
> returns "Location: /your/path/with/no/extra/slash"
>
> It seems that if the URL has an extra slash on the start of it, redirection
> doesn't work properly.
>
> Should this be classed as bad user sanity checking, or
> HttpResponseRedirect() not being flexible enough?

I'd classify this one as bad user sanity checking. Doesn't seem like
the thing that happens frequently enough for us to check for it.

Adrian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Django-nonrel patches

[Adrian Holovaty]

On Sat, Dec 3, 2011 at 4:36 PM, Jonas H.  wrote:
> On Jacob's suggestion in this thread [1] back in April, I split the diff
> between Django trunk and Django-nonrel into logically separated patches.
>
> I uploaded most of them to the ticket tracker (a few things are still
> missing but these are the most important changes):

Thanks for splitting these up, Jonas! I can commit to reviewing and
committing these in the next few days. I just took care of
https://code.djangoproject.com/ticket/17335 as a first step.

Adrian

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Small problem with HttpResponseRedirect

[Cal Leeming [Simplicity Media Ltd]]

Not sure if this should have a bug ticket raised or not.. wanted to get
core devs thoughts.


_redir = "//your/path/with/an/extra/slash/for/whatever/reason"
HttpResponseRedirect(_redir)
returns "Location: http://your/path/with/an/extra/slash/for/whatever/reason;

_redir = "/your/path/with/no/extra/slash"
HttpResponseRedirect(_redir)
returns "Location: /your/path/with/no/extra/slash"

It seems that if the URL has an extra slash on the start of it, redirection
doesn't work properly.

Should this be classed as bad user sanity checking, or
HttpResponseRedirect() not being flexible enough?

I personally feel it's not being flexible enough, but would be good to hear
your thoughts.

Cal

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Vinay Sajip]

On Dec 5, 7:08 pm, Alex Gaynor  wrote:

> It depends *why* it's slow.  If it's slow as a consequence of all the bytes
> calls and extra encoding/decoding, yeah I'd say that's a blocker, because
> it'll never improve otherwise.  On the other hand if it's slow because
> psycopg happens to be slow on py3k, well, that's someone else's problem
> then and I'm ok to  merge it.

Further to my earlier post, I do now think it's my PostgreSQL
configuration. I aborted my test of unported Django running with
psycopg2 - it had run 1223 tests in 4871.121 seconds, which would mean
an average of 4 seconds per test! The full suite of 4475 or so tests
would then take almost five hours! I will try to test with psycopg2 on
another machine, but that might not be possible for a while, so I'll
have to rely on the good offices of others like Anssi for some more
timely results.

Regards,

Vinay Sajip

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Vinay Sajip]

On Dec 5, 7:08 pm, Alex Gaynor  wrote:
> It depends *why* it's slow.  If it's slow as a consequence of all the bytes
> calls and extra encoding/decoding, yeah I'd say that's a blocker, because
> it'll never improve otherwise.  On the other hand if it's slow because
> psycopg happens to be slow on py3k, well, that's someone else's problem
> then and I'm ok to  merge it.

I've just started a run on unported Django with my postgresql
configuration. No hard data yet, but it seems just as slow as the dots
get printed to the screen. I guess that might be good news for the
port, but we'll wait and see what the final time is ... just to give
an idea, I re-ran the test on the port, but without using RAMdisk for
the data (but still with fsync = off), that took around 3h 15 min ...
I've only got a GB of RAM, running in a VM, and have done no tuning
other than fsync = off, so data from other testers might be more
representative than my own.

Regards,

Vinay Sajip

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Alex Gaynor]

On Mon, Dec 5, 2011 at 2:07 PM, Carl Meyer  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 12/05/2011 11:56 AM, Jeremy Dunck wrote:
> > Carl, to be clear, if py3 pg is slower, but py2 pg is still in line w/
> > baseline, would that block merge to trunk?  It's quite possible the
> > problem lies in psycopg, but I think merging (with caveat emptor on
> > the py3/pg combination) would still be good.
>
> Yeah, I agree. If there's no regression on py2 with the branch (which
> seems to be the case given Anssi's rough data), then slow py3 is
> certainly better than no py3. Though if its really ten-fold-plus slower,
> that would make py3/pg pretty much unusable, depending whether its
> something that's particularly bad in the test suite or that would affect
> typical code similarly. This is of course all uninformed hypothetical
> speculation :-)
>
> Carl
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk7dFnQACgkQ8W4rlRKtE2cKmQCeJWaMv8xQ+OERPYGl6bZ7tv8H
> vzcAn3zz0W26OxpDCoPf+tF/oOY33gh5
> =qR3p
> -END PGP SIGNATURE-
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>
It depends *why* it's slow.  If it's slow as a consequence of all the bytes
calls and extra encoding/decoding, yeah I'd say that's a blocker, because
it'll never improve otherwise.  On the other hand if it's slow because
psycopg happens to be slow on py3k, well, that's someone else's problem
then and I'm ok to  merge it.

Alex

-- 
"I disapprove of what you say, but I will defend to the death your right to
say it." -- Evelyn Beatrice Hall (summarizing Voltaire)
"The people's good is the highest law." -- Cicero

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 12/05/2011 11:56 AM, Jeremy Dunck wrote:
> Carl, to be clear, if py3 pg is slower, but py2 pg is still in line w/
> baseline, would that block merge to trunk?  It's quite possible the
> problem lies in psycopg, but I think merging (with caveat emptor on
> the py3/pg combination) would still be good.

Yeah, I agree. If there's no regression on py2 with the branch (which
seems to be the case given Anssi's rough data), then slow py3 is
certainly better than no py3. Though if its really ten-fold-plus slower,
that would make py3/pg pretty much unusable, depending whether its
something that's particularly bad in the test suite or that would affect
typical code similarly. This is of course all uninformed hypothetical
speculation :-)

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7dFnQACgkQ8W4rlRKtE2cKmQCeJWaMv8xQ+OERPYGl6bZ7tv8H
vzcAn3zz0W26OxpDCoPf+tF/oOY33gh5
=qR3p
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Jeremy Dunck]

On Mon, Dec 5, 2011 at 7:28 AM, Carl Meyer  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 12/05/2011 08:10 AM, Vinay Sajip wrote:
>> Not too bad for a first complete pass. Over two and a half hours to
>> run, and that' s running from /dev/shm (IIUC effectively RAMdisk) and
>> with fsync = off. All else set to default values. Time to investigate
>> Jeremy's link to Frank Wiles' post ...
>
> I think first investigating any difference in performance between trunk
> and your py3k branch makes more sense than heading straight to more
> Postgres performance tweaks.

I agree.

Carl, to be clear, if py3 pg is slower, but py2 pg is still in line w/
baseline, would that block merge to trunk?  It's quite possible the
problem lies in psycopg, but I think merging (with caveat emptor on
the py3/pg combination) would still be good.

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Vinay Sajip]

On Dec 5, 3:15 pm, Carl Meyer  wrote:

> That sounds orders of magnitude slower than running the tests under
> Postgres here - and I haven't done any of the pg-config tweaking others
> have mentioned (not even turning fsync off). Which leads me to think
> that either there is some particularly unusual Postgres slowness on your
> machine, or there's a major performance regression with the 3.x port and
> Postgres (which, if it's that significant, would block merging the port).


Or maybe I was just frustrated at the slowness of the tests - they
finished in under 3 hours. Given that the sqlite tests don't feel
slow, I assumed the problem was in the postgresql configuration. So I
have removed postgresql 9.1 (in case that was a factor), installed
8.4, and will re-run the tests. As you suggested, it's a good idea to
run the tests on the trunk to confirm whether it's a postgres
configuration issue.

Another thing that's different, of course, is the psycopg2 driver for
3.x - I'm not sure if it would be expected to contribute to the
problem. I can of course run under 2.x, with the standard psycopg2
driver installed.

Anyway, I don't think we should worry too much about the performance
just yet, especially as Anssi is not apparently seeing the same kind
of slowdown. Others can confirm or refute whether this slowdown is
seen in their environments.

Regards,

Vinay Sajip

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Feature proposal: models.CALL_DELETE or effective equivalent

[Yo-Yo Ma]

Hi Carl,

Thanks for the reply. My guess is that adding a new, optional Meta
argument, (e.g. ``call_delete_on_fk_delete = True``) would be much
simpler to implement. I used ``on_delete`` to make it easy to
understand what I'm asking for, though if it is possible to achieve
this way, I'd be willing to work in that direction.

On Dec 5, 8:58 am, Carl Meyer  wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 12/05/2011 08:31 AM, Yo-Yo Ma wrote:
>
> > While I agree that signals allow for neat decoupling, they aren't as
> > DRY or quick as a simple field kwarg. Imagine you have 8 models that
> > have custom delete methods. Which is easier: A) avstracting the
> > functionality of each of the 8 delete methods into helper functions
> > and writing signals for all 8, or B) typing on_delete=CALL_DELETE? How
> > will this question be answered, objectively?
>
> Clearly the latter. But "what is easier to type" is not, of course, the
> only relevant consideration. A couple notes:
>
> 1. Implementing the proposed on_delete handler would require a major
> reworking of how on_delete handlers function. Currently, they allow for
> modifying the list of objects to be deleted, but not changing how the
> delete actually happens. To make the latter possible, while still doing
> things correctly w.r.t. deletion ordering in the presence of FKs, etc,
> would be far from a trivial patch.
>
> 2. The proposal would have abysmal performance characteristics on a
> large delete. Thus, it would have to come with big red warnings in the
> docs. I'm generally averse to adding features that require big red
> warnings in the docs.
>
> Carl
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla -http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk7c6ioACgkQ8W4rlRKtE2d2mgCgq0bOAHbMj4b9qbAeeeTi9VUx
> PFYAoNiYP5+g9r/EbNiy6qj17oKlPmfP
> =TblP
> -END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Anssi Kääriäinen]

On Dec 5, 5:28 pm, Carl Meyer  wrote:
> > Not too bad for a first complete pass. Over two and a half hours to
> > run, and that' s running from /dev/shm (IIUC effectively RAMdisk) and
> > with fsync = off. All else set to default values. Time to investigate
> > Jeremy's link to Frank Wiles' post ...
>
> I think first investigating any difference in performance between trunk
> and your py3k branch makes more sense than heading straight to more
> Postgres performance tweaks.

For me the tests do not feel any slower on Python3 than they do on
Python2. Taking a few assorted tests, other is Python2 and trunk HEAD,
other Python3 and py3k branch HEAD:

python3 ./runtests.py --settings=test_pgsql admin_views queries
transactions
Ran 362 tests in 175.297s
FAILED (errors=5, skipped=3, expected failures=1)

./runtests.py --settings=test_pgsql admin_views queries transactions
Ran 362 tests in 209.128s
OK (skipped=4, expected failures=1)

As can be seen, there is no reason to expect python3 to be slower. It
is actually faster in this little test is, but that is probably
because my machine happens to generate wildly varying benchmark
results (powersaving or something doing weird things behind the
scenes).

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Feature proposal: models.CALL_DELETE or effective equivalent

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 12/05/2011 08:31 AM, Yo-Yo Ma wrote:
> While I agree that signals allow for neat decoupling, they aren't as
> DRY or quick as a simple field kwarg. Imagine you have 8 models that
> have custom delete methods. Which is easier: A) avstracting the
> functionality of each of the 8 delete methods into helper functions
> and writing signals for all 8, or B) typing on_delete=CALL_DELETE? How
> will this question be answered, objectively?

Clearly the latter. But "what is easier to type" is not, of course, the
only relevant consideration. A couple notes:

1. Implementing the proposed on_delete handler would require a major
reworking of how on_delete handlers function. Currently, they allow for
modifying the list of objects to be deleted, but not changing how the
delete actually happens. To make the latter possible, while still doing
things correctly w.r.t. deletion ordering in the presence of FKs, etc,
would be far from a trivial patch.

2. The proposal would have abysmal performance characteristics on a
large delete. Thus, it would have to come with big red warnings in the
docs. I'm generally averse to adding features that require big red
warnings in the docs.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7c6ioACgkQ8W4rlRKtE2d2mgCgq0bOAHbMj4b9qbAeeeTi9VUx
PFYAoNiYP5+g9r/EbNiy6qj17oKlPmfP
=TblP
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Joe & Anne Tennies]

So, as a point of comparison, Postgres should take a 20-30% hit in
performance from running under KVM-QEMU or Virtualbox. It's almost 50%
under Xen. This is obviously looking worse than that.

http://www.phoronix.com/scan.php?page=article=ubuntu_1110_xenkvm=7

I know, lies and benchmarks, but it's still a point of comparison from
another person. The part I don't know how to compare is how transactions
per second translates to the test suite as I assume there isn't 3000
simultaneous transactions while running the tests.

On Mon, Dec 5, 2011 at 9:15 AM, Carl Meyer  wrote:

> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> Hi Vinay,
>
> On 12/05/2011 04:41 AM, Vinay Sajip wrote:
> [snip]
> > containing " IN ()", which the backend rejects as a syntax error]. I'm
> > running the full suite now, but I don't expect it to finish before the
> > end of the day :-(
>
> That sounds orders of magnitude slower than running the tests under
> Postgres here - and I haven't done any of the pg-config tweaking others
> have mentioned (not even turning fsync off). Which leads me to think
> that either there is some particularly unusual Postgres slowness on your
> machine, or there's a major performance regression with the 3.x port and
> Postgres (which, if it's that significant, would block merging the port).
>
> Maybe if you run the tests in 2.x under Postgres, with trunk instead of
> the 3.x-port, you could observe the comparative speed on your same machine?
>
> Carl
> -BEGIN PGP SIGNATURE-
> Version: GnuPG v1.4.10 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
>
> iEYEARECAAYFAk7c4AwACgkQ8W4rlRKtE2chPQCeNPuFvnqgAjDTlvNwPZmQArNN
> 5EYAoLRvkznDey6Y9fQ7lVwhrLGRq9u7
> =BUc4
> -END PGP SIGNATURE-
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django developers" group.
> To post to this group, send email to django-developers@googlegroups.com.
> To unsubscribe from this group, send email to
> django-developers+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-developers?hl=en.
>
>


-- 
Joe & Anne Tennies
tenn...@gmail.com

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Feature proposal: models.CALL_DELETE or effective equivalent

[Yo-Yo Ma]

On Dec 4, 10:46 pm, Justin Holmes  wrote:
> "hacks, signals, and/or patches"
>
> One of these things is not like the other

While I agree that signals allow for neat decoupling, they aren't as
DRY or quick as a simple field kwarg. Imagine you have 8 models that
have custom delete methods. Which is easier: A) avstracting the
functionality of each of the 8 delete methods into helper functions
and writing signals for all 8, or B) typing on_delete=CALL_DELETE? How
will this question be answered, objectively?

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

On 12/05/2011 08:10 AM, Vinay Sajip wrote:
> Not too bad for a first complete pass. Over two and a half hours to
> run, and that' s running from /dev/shm (IIUC effectively RAMdisk) and
> with fsync = off. All else set to default values. Time to investigate
> Jeremy's link to Frank Wiles' post ...

I think first investigating any difference in performance between trunk
and your py3k branch makes more sense than heading straight to more
Postgres performance tweaks.

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7c4wUACgkQ8W4rlRKtE2djswCfaf/NDuIyO5SRgMfmTG/JBUif
83gAn3xrbtwWrgEeBdsM9vgdPYCoSgIh
=fbQj
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Carl Meyer]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi Vinay,

On 12/05/2011 04:41 AM, Vinay Sajip wrote:
[snip]
> containing " IN ()", which the backend rejects as a syntax error]. I'm
> running the full suite now, but I don't expect it to finish before the
> end of the day :-(

That sounds orders of magnitude slower than running the tests under
Postgres here - and I haven't done any of the pg-config tweaking others
have mentioned (not even turning fsync off). Which leads me to think
that either there is some particularly unusual Postgres slowness on your
machine, or there's a major performance regression with the 3.x port and
Postgres (which, if it's that significant, would block merging the port).

Maybe if you run the tests in 2.x under Postgres, with trunk instead of
the 3.x-port, you could observe the comparative speed on your same machine?

Carl
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.10 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk7c4AwACgkQ8W4rlRKtE2chPQCeNPuFvnqgAjDTlvNwPZmQArNN
5EYAoLRvkznDey6Y9fQ7lVwhrLGRq9u7
=BUc4
-END PGP SIGNATURE-

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Vinay Sajip]

On Dec 5, 12:14 pm, Anssi Kääriäinen  wrote:

> As for test running speed: how fast is it for you? For me, using an old
> laptop, the speed is around 20-30 minutes (can't test for accurate speed
> right now). So, if it is much slower, you have something strange in your
> setup.

Well, the test has now completed:

Ran 4474 tests in 8869.701s

FAILED (failures=18, errors=12, skipped=71, expected failures=2,
unexpected successes=1)

Not too bad for a first complete pass. Over two and a half hours to
run, and that' s running from /dev/shm (IIUC effectively RAMdisk) and
with fsync = off. All else set to default values. Time to investigate
Jeremy's link to Frank Wiles' post ...

Tests are running in a (64-bit) VM with 1GB RAM on a machine with 8GB
physical RAM.

Regards,

Vinay Sajip

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Vinay Sajip]

On Dec 5, 12:14 pm, Anssi Kääriäinen  wrote:
> On 12/05/2011 01:41 PM, Vinay Sajip wrote:

>
> The test_ticket10432 should not generate any query at all. The tests in
> django/db/models/sql/where.py should ensure that in the case of empty
> __in condition the query will not be executed at all (or that part of
> the where condition is removed if it is possible for the query to return
> rows if the __in=() executes to False).
>
> The problem is that the test in where.py:
> if hasattr(value, '__iter__') and hasattr(value, 'next')
> isn't correct. It seems that in python3 the 'next' has been renamed to
> '__next__' and thus the generator will not be consumed. My python3
> knowledge is basically nonexistent, so it might be I am wrong here.

Aargh, yes. You mentioned this earlier, but I missed correcting this
when I did the other fixes. I've dealt with it now [basically, py3
defines 'next_name' which is 'next' on 2.x and '__next__' on 3.x - I
just changed it to hasattr(value, next_name)].

> As for test running speed: how fast is it for you? For me, using an old
> laptop, the speed is around 20-30 minutes (can't test for accurate speed
> right now). So, if it is much slower, you have something strange in your
> setup.

I haven't yet run it to completion, but have just now set it up to
use /dev/shm as per Florian's suggestion (as well as having fsync =
off). Will report the total time once the tests have all run.

Regards,

Vinay Sajip

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Anssi Kääriäinen]

On 12/05/2011 01:41 PM, Vinay Sajip wrote:

I've eliminated many of the errors here, but a few remain, at least
some of which are related to the psycopg2 driver under Python 3 [e.g.
test_ticket10432 leads to a SQL query apparently legitimately
containing " IN ()", which the backend rejects as a syntax error]. I'm
running the full suite now, but I don't expect it to finish before the
end of the day :-


The test_ticket10432 should not generate any query at all. The tests in 
django/db/models/sql/where.py should ensure that in the case of empty 
__in condition the query will not be executed at all (or that part of 
the where condition is removed if it is possible for the query to return 
rows if the __in=() executes to False).


The problem is that the test in where.py:
if hasattr(value, '__iter__') and hasattr(value, 'next')
isn't correct. It seems that in python3 the 'next' has been renamed to 
'__next__' and thus the generator will not be consumed. My python3 
knowledge is basically nonexistent, so it might be I am wrong here.


As for test running speed: how fast is it for you? For me, using an old 
laptop, the speed is around 20-30 minutes (can't test for accurate speed 
right now). So, if it is much slower, you have something strange in your 
setup.


 - Anssi

--
You received this message because you are subscribed to the Google Groups "Django 
developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Vinay Sajip]

On Dec 5, 11:30 am, Florian Apolloner  wrote:

>
> Use a tablespace which lies in a ram disk, you can't get any faster than
> that I/O wise ;)

I've seen posts telling how to do this, but I had hoped to avoid the
need for it ...

Regards,

Vinay Sajip

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Vinay Sajip]

On Dec 5, 12:57 am, Anssi Kääriäinen  wrote:
>
> To speed up things, here are the tests that failed for me after the
> fixes posted upthread:
>  - cache
>  - select_for_update
>  - queries
>  - admin_views
>

I've eliminated many of the errors here, but a few remain, at least
some of which are related to the psycopg2 driver under Python 3 [e.g.
test_ticket10432 leads to a SQL query apparently legitimately
containing " IN ()", which the backend rejects as a syntax error]. I'm
running the full suite now, but I don't expect it to finish before the
end of the day :-(

Regards,

Vinay

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Python 3 port - all tests now pass on 2.5.4, 2.6.2, 2.7.2 and 3.2.2 with the same codebase

[Florian Apolloner]

Hi,

On Monday, December 5, 2011 2:10:50 AM UTC+1, Vinay Sajip wrote:
>
> I've already done the fsync = off, but it didn't make that much
>
> difference :-(
>
Use a tablespace which lies in a ram disk, you can't get any faster than 
that I/O wise ;) 

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/django-developers/-/rexlWC34ro0J.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Deprecate change pk + save behavior for 1.4

[Anssi Kääriäinen]

On Dec 5, 11:43 am, Stephen Burrows 
wrote:
> I've often longed for a "read-only" concept in ModelForms. For
> example, if I want the "site" field to always have a certain value
> (i.e. the current site) but also want to enforce multiple-column
> uniqueness (i.e. slug/site). I can write it myself, but it always
> feels like I shouldn't have to. Probably since the admin already
> provides that functionality.

Me too. The good news is that it should be relatively easy to
implement this after template based form rendering is implemented. You
get the read-only widgets nearly for free. There is some added logic
in the forms itself to handle the case of "do not handle data for this
field", although that might boil down to "if field.read_only:
continue" in a couple of places.

The bad news is that template based rendering is blocked on faster
template system. A test I did some time ago showed nearly 10x slower
rendering in the worst case using template rendering compared to
current "render in Python" implementation. The funny thing was that
Jinja2 was actually _faster_ than current implementation, although
that might be because of different escaping and localization.

> Point being, that's something that, if it is implemented, should
> probably be implemented separately from the pk stuff which is
> currently being discussed.

Agreed.

> Personally, I haven't had much use for natural primary keys since I
> started using Django. (Though probably that's because it's so easy in
> django to use the AutoField pk and because so many things expect
> integer pks.) Are there a lot of people who use them?

There currently aren't that many users, and that is why the current
behaviour of save cloning the object isn't a big problem. This will
change if natural primary keys are included in Django. By definition
they are part of the data model, and in most cases subject to change.
Even if they are not subject to change, Django doesn't make it easy to
enforce that (no support for read-only form fields, and save doesn't
guard against accidental changes).

So, my opinion is:
  - It is a good idea to block accidental PK changes in .save(). I
think I have a working solution in #17332
  - It is not a good idea to do this before read-only form fields are
introduced.
  - It is a good idea to have as little state as possible for model
instances. If first item is implemented, it unfortunately requires
tracking of PK state.
  - It is a good idea to allow custom subclasses to track more state
if they wish to do so. The patch in 17332 should make this easier to
do (a little change needed, though: self._state = self._state_class()
instead of self._state = ModelState()).

In conclusion: I think the best thing to do is visit this subject
again during 1.5 development cycle.

 - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.

Re: Deprecate change pk + save behavior for 1.4

[Stephen Burrows]

I've often longed for a "read-only" concept in ModelForms. For
example, if I want the "site" field to always have a certain value
(i.e. the current site) but also want to enforce multiple-column
uniqueness (i.e. slug/site). I can write it myself, but it always
feels like I shouldn't have to. Probably since the admin already
provides that functionality.

Point being, that's something that, if it is implemented, should
probably be implemented separately from the pk stuff which is
currently being discussed.

Personally, I haven't had much use for natural primary keys since I
started using Django. (Though probably that's because it's so easy in
django to use the AutoField pk and because so many things expect
integer pks.) Are there a lot of people who use them?

--Stephen

On Dec 3, 7:58 pm, Anssi Kääriäinen  wrote:
> On Dec 3, 7:18 pm, Carl Meyer  wrote:
>
>
>
>
>
>
>
>
>
> > -BEGIN PGP SIGNED MESSAGE-
> > Hash: SHA1
>
> > On 12/03/2011 02:13 AM, Anssi K ri inen wrote:
>
> > > Admin should be fixed [#2259]. Making PK fields non-editable in
> > > ModelForms would be good, too. Is it OK to consider the current
> > > ModelForm behavior a bug, or will it need a deprecation cycle?
>
> > I think it would need a deprecation cycle. People certainly could be
> > making use of the current behavior (and there would need to be a way to
> > get the PK field back into the form, e.g. by explicitly listing it in
> > "fields"). Excluding PK from ModelForm by default isn't clearly fixing a
> > bug, it's more protecting people from unintuitive behavior.
>
> > (Note that in admin we have the option to display it read-only by
> > default; ModelForm has no such feature, we'd have to exclude it entirely
> > by default).
>
> This turns out to be a show-stopper. I think that in admin we could
> just make the field non-editable when editing an object, but editable
> when saving a new object. Save-as-new would be a problem, but probably
> some workaround could be found.
>
> But for ModelForms, what to do? You would want to have the same
> behavior as in admin: editable when adding, but non-editable when
> editing an existing object. But the problem is, we do not have the
> ability to show the value as non-editable. Could this be implemented
> by a html attribute editable="false"? Ugly, but should work for most
> cases.
>
> I do agree this needs a deprecation cycle.
>
> > > It is OK that Django doesn't support primary key updates. But if it
> > > doesn't support update of PK, then lets error out on primary key
> > > change. Current behavior of .save() is actually: "save the object to
> > > DB, except when PK has changed, then do a clone". That is a bad API
> > > for natural primary keys.
>
> > > About breaking current code: my intention is to have a flag to .save()
> > > which would allow current code to work. .save(clone=True) would get
> > > current behavior back. Setting the PK to None and doing a save will
> > > work for AutoField PKs.
>
> > So what about admin users who are currently relying on this behavior as
> > a way to clone objects with natural PKs (given that the
> > save-and-add-another button is useless with natural PKs unless you can
> > explicitly give the value for the new row)?
>
> Javascript magic? save-as-new disabled for natural PKs, instead you
> get a "copy" button which opens another window? In other words, good
> ideas welcome...
>
>  - Anssi

-- 
You received this message because you are subscribed to the Google Groups 
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com.
To unsubscribe from this group, send email to 
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/django-developers?hl=en.