Stefan, I cannot get it working in the current state of the repo... I
have to patch like above (hope you have a fixed font in your mail
reader).
At this point, I'm a bit confused about the Cython-internal string
table. In the Py3 context, what would be the point of having in this
table byte strings? Just for the case byte-string literals?
IMHO, it seems that in Py3 byte string are poor cousins of unicode
strings (not used for identifiers, cannot be interned and no point of
doing it), and the use cases of byte strings are going to be rather
small (passing it to C funcs specting ascii-encoded strings, internet
protocols expecting ascii). So, should Cython save (in the Py3 case)
byte strings in their internal table?
At this point, I'm not sure if meging the string tables was a good
idea. I seems that Cython should still use two tables:
* one table list identifiers. In the py2 case, their entries are
interned (unless disabled) and their types are the py2 builting str
object C-level PyString (or does py2 accept unicode identifiers?). In
the py3 case, the table entries are interned (unless disabled) and
their types are the py3 builtin str object C-level PyUnicode.
* other table list string literals, and they can be either byte
strings or unicode string, depending on the Python version and the u/b
prefix in the literal expresion...
Perhaps again I completelly confused about all this. In such a case,
just ignore my coments.
--- a/Cython/Compiler/Nodes.py Tue May 13 23:41:11 2008 +0200
+++ b/Cython/Compiler/Nodes.py Wed May 14 12:45:16 2008 -0300
@@ -4325,8 +4325,14 @@ static int __Pyx_InitStrings(__Pyx_Strin
if (t->intern)
*t->p = PyString_InternFromString(t->s);
else
+ *t->p = PyString_FromStringAndSize(t->s, t->n - 1);
+ #else
+ if (t->intern) {
+ *t->p = PyUnicode_InternFromString(t->s);
+ } else {
+ *t->p = PyUnicode_FromStringAndSize(t->s, t->n - 1);
+ }
#endif
- *t->p = PyString_FromStringAndSize(t->s, t->n - 1);
}
if (!*t->p)
return -1;
On 5/13/08, Stefan Behnel <[EMAIL PROTECTED]> wrote:
> Hi,
>
> I managed to compile lxml against Py3, although I couldn't test it so far as
> the PyFile_* functions are gone due to the new I/O system. Also, Cython's
> test
> runner didn't work right away, as it (obviously) tries to compile the tests
> using Cython, and Cython itself does not yet run under Py3. So I added a work
> around. If you run
>
> python2.5 runtests.py --no-cleanup
> python3.0 runtests.py --no-cython
>
> it will run Cython from Python 2.5 to compile and run the tests, but not
> delete the generated .c files, and then run Python3 to C-compile and run the
> tests without importing Cython at all. This allows testing the generated
> sources with different Python versions in general.
>
> However, for Py3 in particular, all tests go straight red (as expected ;) The
> problem is (again) identifiers, which are unicode strings in Py3. So we will
> have to find a way to make every one of them a unicode string, from class
> names to keyword arguments, but only on Py3. A good idea might be to actually
> remove the string interning Option and *always* intern identifiers, so that
> we
> have more control over how they enter the runtime environment. I'll look into
> that when I find the time...
>
> Regarding Cython's source itself, I tried running the 2to3 tool on Cython
> like
> this:
>
> python3.0 -m lib2to3.refactor Cython
>
> which generates a 3000 lines patch, but being only a static analysis tool, it
> breaks the ".next()" method on the scanner, thinking that what we actually
> meant was calling the new built-in "next()" function on an iterator. We might
> even work around that by actually making it an iterator, not sure if that
> will
> work.
>
> Other than I thought, the 2to3 tool does not touch string literals, so if you
> use a byte string literal in Py2 code, it will become a unicode literal after
> running 2to3. I think this is good for Cython where most byte string literals
> are really meant to be unicode literals. We might still want to take a bit of
> caution here, as there might be places where this is not the case.
>
> You can find the current Py3 branch of Cython here:
>
> http://hg.cython.org/cython-devel-py3
>
> It will (or at least should) compile things nicely on Py2 without a
> noticeable
> difference, but it will not yet compile any code correctly for Py3. I'll keep
> working on it, any help is appreciated.
>
> Have fun,
> Stefan
>
>
> _______________________________________________
> Cython-dev mailing list
> [email protected]
> http://codespeak.net/mailman/listinfo/cython-dev
>
--
Lisandro Dalcín
---------------
Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
PTLC - Güemes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594
_______________________________________________
Cython-dev mailing list
[email protected]
http://codespeak.net/mailman/listinfo/cython-dev
