Hi,
Lisandro Dalcin wrote:
> Stefan, I cannot get it working in the current state of the repo... I
> have to patch like above
[patch moved here]
> --- a/Cython/Compiler/Nodes.py Tue May 13 23:41:11 2008 +0200
> +++ b/Cython/Compiler/Nodes.py Wed May 14 12:45:16 2008 -0300
> @@ -4325,8 +4325,14 @@ static int __Pyx_InitStrings(__Pyx_Strin
> if (t->intern)
> *t->p = PyString_InternFromString(t->s);
> else
> + *t->p = PyString_FromStringAndSize(t->s, t->n - 1);
> + #else
> + if (t->intern) {
> + *t->p = PyUnicode_InternFromString(t->s);
> + } else {
> + *t->p = PyUnicode_FromStringAndSize(t->s, t->n - 1);
> + }
> #endif
> - *t->p = PyString_FromStringAndSize(t->s, t->n - 1);
> }
> if (!*t->p)
> return -1;
Can you tell me why? This actually creates strings as unicode strings that
were byte strings in the source code.
> At this point, I'm a bit confused about the Cython-internal string
> table. In the Py3 context, what would be the point of having in this
> table byte strings?
The byte strings are interned in Py2, where identifiers are byte strings, and
the unicode strings are interned in Py3, where identifiers are unicode strings.
> should Cython save (in the Py3 case)
> byte strings in their internal table?
We can't recognise the Py3 case at Cython compile time. Only the C compiler
knows what the target environment is.
> I seems that Cython should still use two tables:
>
> * one table list identifiers. [...]
> * other table list string literals [...]
That's what I was considering, too, although not quite as you describe. The
difference between the two is that the real identifiers must become either
byte strings or unicode, depending on the compile time Python version. The
normal strings must be created as they appeared in the source code, and either
unicode strings or byte strings can be interned based on the compile time
Python version. So I now added another field to the string tab that states if
the string was interned as identifier, which will then make it pop up as
either unicode or byte string depending on the C compile time Python version.
This (plux a fix for importing based on unicode module names) even gets almost
all test cases green, just a few to go. :)
One big remaining problem are the PyFile_* functions - as used in "print". I
guess we'll have to wait for 3.0b1 here to provide a fix.
Another thing is the removal of __setslice__ and __delslice__, i.e. the
sq_ass_slice slot. This means that code that uses these will no longer
compile. I added a warning, but there are two test cases that depend on it. We
might want to remove them.
I tested the generated code under Py2.3.6, Py2.4.4, Py2.5.1 and Py3.0a5 now.
Except for a couple of remaining problems in Py3, it still works in all of
them. :]
Stefan
_______________________________________________
Cython-dev mailing list
[email protected]
http://codespeak.net/mailman/listinfo/cython-dev
