Re: django-3k
> It can now convert and start the test suite, however, this doesn't > produce any results, yet. Following up to myself: it now does run the test suite to completion: Ran 1425 tests in 384.689s FAILED (failures=222, errors=539) If you are curious what the failures and errors are, just run it for yourself. Regards, Martin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
django-3k
I have now published my 3.x port on bitbucket, at http://bitbucket.org/loewis/django-3k/ It can now convert and start the test suite, however, this doesn't produce any results, yet. Feel free to use the bitbucket issue tracker to report problems or contribute changes. Regards, Martin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Porting Django to Python 3
> Some examples:
Thanks for posting them:
> >>> Template(u"{{ foo }}").render(Context({"foo":"bar"}))
> u'bar'
I get
py> Template("{{ foo }}").render(Context({b"foo":b"bar"}))
''
I think that's correct: the dictionary has no key "foo".
I'm also unsure what this has to do with UTF-8: isn't this the regular
default encoding (ASCII) that allows you to have Unicode and byte
strings compare equal?
What is the syntax for variable names (i.e. can you even have non-
ASCII characters in variable names)?
> >>> Template("{{ foo }}").render(Context({u"foo":"bar"}))
> u'bar'
py> Template(b"{{ foo }}").render(Context({"foo":b"bar"}))
'bar'
Not sure why this happens - perhaps variable names get always
converted into strings?
> >>> Template("{{ foo }}").render(Context({"foo":u"bar"}))
> u'bar'
py> Template(b"{{ foo }}").render(Context({b"foo":"bar"}))
''
Also notice that it never produces bytes in these cases. Again, I'm
unsure why this happens.
> >>> MyModel.objects.filter(my_attr="foo")
py> Choice.objects.filter(choice=b'geht so')
[]
> >>> MyModel.objects.filter(my_attr=u"foo")
py> Choice.objects.filter(choice='geht so')
[]
> >>> mymodel_instance.my_attr = "foo"
> >>> mymodel_instance.my_attr = u"foo"
>>> c=Choice()
>>> c.choice=b'fine'
>>> c.choice='fine'
>
> In addition to these things, there may be problems where dictionary
> keys and various other values have used byte strings up until now,
> with no problems, but based on assumptions that no longer hold. For
> example, declarative classes (e.g. Models) are an interesting one -
> inPython2.x, the keys of MyClass.__dict__ are byte strings, but in 3.0,
> they are unicode strings. Since non-ascii names for identifiers are
> valid inPython3.0 (thanks in part, I believe, to your good self :-),
> and also in at least some databases, this is not an academic issue.
Not sure what issue you see here. It's most likely difficult to map
such names into a relational database. Having a restriction that
requires field names to follow SQL syntax sounds reasonable to me.
As for __dict__ now containing Unicode strings - this already had a
number of consequences on the patch. My recommendation would be that
identifier-like things always get represented as (unicode) strings in
Django on 3k (or more generally, by the "standard" string type).
> Also, inPython3.0, you can have models with non-ascii names, which
> challenges some assumptions about things like the INSTALLED_APPS
> setting.
Again, I don't see a need to support these. I doubt the ability to
create them would be the primary reason why people would switch to
Python 3...
> I imagine that some of these things will 'come out in the wash', so to
> speak, and the lack of automatic conversion will help identify
> problems, but some things might come back to bite us if we don't get
> them right.
I think migration of existing applications is probably the biggest
challenge. For a new 3.x application, the guideline should be to use
the (unicode) string type throughout, and leave encodings entirely to
Django.
Regards,
Martin
--
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-develop...@googlegroups.com.
To unsubscribe from this group, send email to
django-developers+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en.
Re: Porting Django to Python 3
> > In many cases, this is true, but there are other scenarios (certain > > forms of exception handling, for example) where there is no syntax > > that's valid in both versions. That's syntax, not just libraries and > > functions. There's no way to even get a file to parse in both Python 2 > > and Python 3 in these situations. > > Martin's approach was single codebase where the 3.x version for execution is > generated by 2to3, not single source for execution across 2.x and 3.x. Thus > I'm wondering if this difference is accounted for by 2to3? Most certainly - that's the whole *point* of 2to3. Converting the cases where the syntax differs are actually the easy parts - it is very easy to find out, by static analysis, that Python 3 would reject a piece of code, and to propose a reformulation that is equivalent. That's where 2to3 shines, and thus where anybody adding 3.x support to a 2.x code base doesn't need to worry at all. When the syntax is correct in both versions, but library names differs, 2to3 still works fairly well if it guesses correctly that the name indeed refer to the renamed libraries (which has some degree of uncertainty in Python). The really difficult cases for 2to3 is where a piece of code works unmodified in both 2.x and 3.x, but does something significantly different. For example, string literals mean something different, and the result of reading from a file may be different (unicode vs. bytes in both cases). Fortunately, Django already attempts to differentiate bytes and unicode fairly well, so it was easy to fix the remaining spots which would break under 3.x. Regards, Martin -- You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-develop...@googlegroups.com. To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-developers?hl=en.
Re: Py3k port updated
> I'll give it a go, but for someone who doesn't know about running 2to3 > etc, to make things a but more obvious can you add to your page a > quick set of instructions as to what someone needs to do to try this > out. Ie., what needs to be done after a checkout, does one apply patch > and then install it with 2to3 running automatically, or is there some > other voodoo that needs to be done? >From the patch/django point of view, it is really very simple. You apply the patch, and then do path-to-python3.1 setup.py install No need to run 2to3; this is done behind the covers. The really tricky part might be that you have to use python3.1, as the 2to3 version shipped with 3.0 didn't include all of the necessary fixers (although 3.0.1 will). I would recommend to try runserver first: your application might also need porting, or might invoke parts of Django that I haven't ported. Django apparently features a large variety of error reporting scenarios, depending on where exactly in Django the error occurs. If you run into problems, feel free to contact me. Regards, Martin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Py3k port updated
> Have you by chance tried running it on top of Apache/mod_wsgi (version > from subversion which has Python 3.0 support)? No, only in runserver mode. Regards, Martin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Py3k port updated
> Did you by any chance try running the Django test suite(because that's > probably going to be the best way to spot breakages). No, I haven't (or only to the degree to find out that runtest.py is not in 3.0 syntax, as it uses incorrect except clauses). I'm fairly certain though that the test suite fails quickly, with just a few tests passing. Contributions are welcome. Regards, Martin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Py3k port updated
I have updated my Py3k port of django. As before, it is just the bare minimum to get through the tutorial; as the possibly most significant change since the previous patch, it now supports psycopg2. I have updated the Wiki page at http://wiki.python.org/moin/PortingDjangoTo3k with the new patch. The patch applies against Django's trunk; it was tested with the HEAD of Python's py3k branch (i.e. 3.1). Regards, Martin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to django-developers+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Re: Porting Django to Python 3.0 as a GSoC project
> I think you misunderstand the role of 2.6. See the seven steps under > "The recommended development model for a project that needs to support > Python 2.6 and 3.0 simultaneously..." in > http://www.python.org/dev/peps/pep-3000/#compatibility-and-transition. > Step 1 reads "Port your project to Python 2.6." I believe this recommendation is flawed. It assumes that it is acceptable that you break 2.5-and-earlier compatibility in the process of porting to 2.6. This, in turn, is assumed because it was considered impractical to have code that runs on 2.3, yet 2to3 could still fix it correctly. Again in turn, this assumption results from the actual lack of opportunity to try out any other strategy. At the PyCon sprint, I tried a different strategy for Django, and it seems to work fairly well. The authors of the PEP certainly did not mean to say that 2to3 can't possibly create correct output if applied to source that also runs on 2.5. Regards, Martin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Re: Porting Django to Python 3.0 as a GSoC project
> So this means, though, that folks running from SVN will still need to
> run setup.py every time they update, right? Not that that's a
> dealbreaker -- I think Django-on-Py3k'ers will be on the cutting edge
> anyway -- just wanna check.
Correct. distutils operates using time-stamps, so it will only overwrite
(and then re-2to3) newer files. I arrange it so that the build/ tree
contains the converted files. I never tried running it from build/, but instead
perform a full "setup.py install" to get django on sys.path.
See
http://svn.python.org/projects/python/branches/py3k/Lib/distutils/command/build_py.py
for build_py_2to3; in Django's setup.py, I use
try:
from distutils.command.build_py import build_py_2to3 as build_py
except ImportError:
from distutils.command.build_py import build_py
...
setup(...
cmdclass = {'build_py':build_py}
)
to invoke 2to3 in the build step.
Regards,
Martin
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"Django developers" group.
To post to this group, send email to django-developers@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/django-developers?hl=en
-~--~~~~--~~--~--~---
Re: Re: Porting Django to Python 3.0 as a GSoC project
> The specific issues I've run into so far: > > * Exception-catching syntax (i.e. ``except Whatever as e`` vs. > ``except Whatever, e``). 2to3 fixes these, and transparently transforms "the except clauses. > * Unicode literals (u'...'). Likewise, 2to3 removes the u"" prefix. So leave the code as-is, and have 2to3 fix it at installation time (whenever setup.py is invoked by 3.x; setup.py itself runs without changes on 3.x) > Those are the show-stoppers (for me -- maybe I'm just missing > something?) Indeed - the ability to have 2to3 fix it for you. There are more essential fixes that 2to3 supplies, in particular fixes to relative imports, plus a number of non-essential ones, such as the replacement of the unicode identifer with str. >, but I'm actually more worried about the subtle > differences between str/unicode and bytes/str... I suspect there'll be > many subtle bugs there. Indeed, this needs a lot of testing. > Ditto anything using io, really, because the > new io library isn't in any way the same as the old one (in a good way > :) Not so. open() pretty much works the same way as it did before, except that Django should open files in binary mode most of the time. > I'm also just worried about the uglyness of code needed to maintain > something like this. Like about all the ImportErrors we'll have to > catch once all the stdlib reorg happens... It's possible to factor those out if things get too unreadable. Regards, Martin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Porting Django to Python 3.0 as a GSoC project
> > You can (probably) support Python 2.x and Python 3.x out of a single > > source tree. > > From what I've read, this is true as long as the X after 2 is >= 6. > That's a problem with Django's stated intent to support Python 2.3 on > Django's release 1.0. > > Please correct me if I'm wrong on this. This is wrong; my port works nicely on 2.3 (only 2.5 tested) except for a few minor bugs in 2to3, see http://wiki.python.org/moin/PortingDjangoTo3k Regards, Martin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Re: Porting Django to Python 3.0 as a GSoC project
> I hope you won't take it the wrong way when I say I have an extremely > difficult time believing that. No hurt feelings, no. However, I would find it useful if you could add specific reservations and doubts to that. What aspects of Django (that I perhaps haven't touch yet) do you consider unmaintainable under such approach? Web server, models, and templating seem to work fine. Regards, Martin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Re: Porting Django to Python 3.0 as a GSoC project
> Except if Django has a Python 3.0 version, say, this fall, that means > how many years of supporting two parallel versions of Django and > merging features and fixes back and forth between them? Please see my recent report: 0 years, 0 months, 0 days, 0 seconds. You can (probably) support Python 2.x and Python 3.x out of a single source tree. > And if the > initial port isn't supported that long, don't we just get to port all > over again down the line? If the porting strategy is used that I'm proposing, the worst thing that may happen is that the 3.x support in Django suffers from bitrot, but it won't be wasted if people start picking it up again. More likely, people occasionally try it out again, and prevent bitrot by providing fixes. Regards, Martin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Porting Django to Python 3.0 as a GSoC project
> I'm still a bit worried about the fact that, aside from Django being a > moving target and Python 3.0 being a moving target, WSGI for Python > 3.0 is *also* a moving target; there still seems to be a fair bit that > hasn't been settled on how things ought to work. That is the reason why I think this project *should* be a SoC project. Python 3 being a moving target is a good thing for the project (although certainly a challenge for the student). It's likely that problems are detected in Python 3 in the way of porting applications such as Django, and those problems can be easily fixed before Python 3 gets released. Fixing them afterwards is much more difficult. So porting *should* start before Python 3 is released, to simplify it, not afterwards. (of course, the code should be released for general use only after Python 3 is released). Regards, Martin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Porting Django to Python 3.0
> I wasn't at PyCon, and haven't done any 3.0 porting work myself, so I > could be behind the times, but my understanding of current porting > advice (based on PEP 3000) was that it's not going to be possible to > support 2.x and 3.x from a single codebase in many cases (even with > 2to3) if Python < 2.6 needs to be supported. Right? It depends on whom you ask. People often give this advise *assuming* that it couldn't possibly work. I happen to think differently. I see no technical reason why you couldn't support all versions from, say, 2.1 up to 3.0, from a single code base, if you rely on 2to3. (there are severe technical reasons that make that difficult if you don't want to rely on 2to3, such as not being able to get the exception object in an except clause) > I also wonder, is it worth adding this stuff to Django *now* for a > transition that is likely to be quite far off, with at least two big > merges (qs-rf, nfa, possibly gis) coming up soonish? That's completely up to you; I don't want to interfere with the project management. I just wanted to make it absolutely clear that there is no *technical* reason why Django couldn't support 3.x. In addition to whether that is wise from a project management point of view, it's also a lot of work still to make it work fully, people might want to focus on other aspects, this is free software, and all that. Regards, Martin --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
Re: Porting Django to Python 3.0
> Some of the things you've identified as "Django should..." are a little > problematic since one of the current requirements (at least for 1.0) is > "Django should work out of the box with Python 2.3". Some of your > proposed changes don't even work with Python 2.5, so we need to add a > few more conditionals. I think you misunderstood. In these cases, I already changed Django to do what it should do in the patch. E.g. with the patch, it already uses io.BytesIO in 3k, but continues to use cStringIO.StringIO in earlier versions. > The only potential showstoppers I see are the need to change "except > Exception, e:" to "except Exception as e", since that isn't supported in > Python 2.5 or earlier and it's going to be hard to fake as it's a major > syntactic construction. Yes; these are the remaining two cases left. There are *many* more cases that need to be changed to use "except ... as ..."; these are all fixed by 2to3. There is a bug in 2to3 which fails to update the try-except block if there is a bare except. Once that's fixed in 2to3, these chunks can be removed from the patch; this is bug http://bugs.python.org/issue2453 > Also, relative imports aren't supported in 2.4 > or earlier whereas they seem to be required to be explicitly specified > in 2.6. In 2.6, you can still use the implicit relative imports. In 3.0, you need to make explicit relative imports. Again, this is fixed by 2to3. The remaining imports were left in because the 2to3 fixer again had a bug (http://bugs.python.org/issue2446); this should be fixed now. > The latter problem can be fixed a bit by reshuffling some of the import > paths, as we've been intending to do in some cases. We use relative > imports in a few places to avoid importing a higher-level __init__ file > because it does so much (too much) work. Relative imports are fine. David Wolever wrote a 2to3 fixer in the PyCon sprint just because I requested one to simplify porting Django. > That's a change that can be > made so everything can do "from django.foo.bar.baz..." and avoid > unnecessary "from ." ugliness. Painful and possibly quite disruptive in > places, but possible and probably worth doing. However, unnecessary to support 3.0; see above. > We already have a lot of conditional code based on sys.version_info. > We'll need to introduce some more. That explains why, e.g., we're using > md5 in some places and are testing based on behaviour (try to import > hashlib and if that fails, use the fallback). Changing to be a > sys.version_info test is not impossible eventually, although it relies > on IronPython and Jython being comparable in their version numbering and > functionality. Right. I think both Jython and IronPython try to follow the CPython feature sets by version number. In particular wrt. bytes-vs-string, it is actually simpler for them to implement 3.x instead of 2.x, as their native string type is already a Unicode type. Still, they would have to change APIs when dividing strings and bytes, so they'll likely use the major version 3 to indicate presence of the separate bytes type. > Similarly, we can't unconditionally switch to "from io import ...", > since that doesn't exist in Python 2.5, let alone earlier versions. But my patch doesn't do that! > Wrapping it in import guards will work, but the code is starting to get > a bit ugly to read. If this happens in too many places, it can be factored out into a single module, like the django.utils.py3 module that I propose. > Ditto, the reason we use upper-case names when > import from email is because that's required by 2.3. They are backwards > compatible in later versions, so we intentionally stuck with them. Which is fine, no need to change that. Again, if it gets too ugly, you can factor it out into a single place (although I think each email class is imported only once). > We need to retain one use of ClassType to generate the right > class type for our own exception subclasses -- they're old-style classes > there -- in Python 2.3 and 2.4, but that's guarded by a version_info > conditional, so hopefully 2to3.py won't be concerned about that. The one > in ModelBase.contribute_to_class can go away, though. That's the one I > haven't committed to the branch yet. Anyway, most of those issues will > become non-issues once the branch is in trunk. Ok. Out of curiosity: What was the rationale for clearing Meta.__dict__ when iterating over it? > The AdminLogNode sending strings as slices bit me in queryset-refactor, > too. I was a bit annoyed that Python allowed it because then I had to > work around it. I'm strongly tempted to fix that (i.e. fix AdminLogNode > to not be so slack) at a some point, but it's low-priority at the > moment, since it breaks indeterminate amounts of code, so I haven't had > the energy to work out the impact and/or mitigation. > > It's kind of cool, though: in Python 2.x, you can write a class that > accepts
Porting Django to Python 3.0
At the PyCon sprint, I started porting Django to Python 3.0. In the process, I had to make a number of changes to Python, so this port currently requires the Python 3.0 subversion head (soon to be released as 3.0a4). This port is just a start; it runs the tutorial with the sqlite3 backend, including the admin interface, but probably chokes on larger applications that use more features. The patch is designed to not break Django on any earlier Python versions. I put up the patch and my notes at http://wiki.python.org/moin/PortingDjangoTo3k I would like to know how I should proceed with that; options are a) split the patch into separate chunks, and contribute them separately. b) hand the entire patch over to somebody interested in completing it c) complete it, then submit it d) abandon it I would favor option a), even though this means that the django source repository would only get a part of a working 3.0 port; I might continue to work on it over the next months, although I would again hope that regular Django users take over (which I am not). --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "Django developers" group. To post to this group, send email to django-developers@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-developers?hl=en -~--~~~~--~~--~--~---
