On Apr 5, 2010, at 9:31 AM, Haoyu Bai wrote:
> On Sun, Apr 4, 2010 at 3:59 AM, Robert Bradshaw
> <[email protected]> wrote:
>> On Apr 3, 2010, at 5:24 AM, Stefan Behnel wrote:
>>
>>> Haoyu Bai, 03.04.2010 14:15:
>>>> On Sat, Apr 3, 2010 at 6:50 PM, Stefan Behnel<[email protected]>
>>>> wrote:
>>>>> Haoyu Bai, 03.04.2010 09:18:
>>>>>> Thinking about how to implement Cython decorators (@cclass,
>>>>>> @cfunc
>>>>>> etc.) for pure Python mode, I come up with an idea of "compile-
>>>>>> time
>>>>>> decorator". Basically, we want these decorators to have effect
>>>>>> during
>>>>>> compile time, rather than normal Python decorator that is
>>>>>> called in
>>>>>> runtime. So, to implement @cfunc, we can actually implement a
>>>>>> function
>>>>>> cfunc(node), which inputs a DefNode and transform it to a
>>>>>> CFuncDefNode, then put the function into the compile-time scope.
>>>>>
>>>>> -1, we shouldn't expose Cython compiler internals to the language
>>>>> level.
>>>>>
>>>>> It's better to use a transform that intercepts on certain
>>>>> decorators.
>>>>
>>>> By implement @cfunc as a function, I'm not mean to implement it *in
>>>> Cython*, it is still a part of the Cython compiler, as the
>>>> transforms.
>>>> And the main motivation is not to expose Cython internal to the
>>>> users,
>>>> but to have a flexible mechanism to support the various
>>>> decorators -
>>>> Cython directives, @cclass @cfunc, and even some Python builtin
>>>> decorator (@classmethod, @staticmethod and @property).
>>>>
>>>> These are decorators need to be executed in compile time.
>>>> Currently,
>>>> the code to deal with these decorators are scattered in many
>>>> places,
>>>> some in transforms, some in nodes. It would be better to have a
>>>> unified mechanism to deal with all of these decorators. That's
>>>> what I
>>>> proposed.
>>>>
>>>> Indeed, the node classes do not provide an API so we should not
>>>> expose
>>>> them to end-user. But there is nothing to prevent us to write
>>>> compile-time function to process them inside Cython (and
>>>> decorator is
>>>> just functions).
>>>
>>> In that case, there is nothing to gain from making them "compile
>>> time
>>> evaluated functions", because the implementation itself would
>>> still be
>>> inside of the compiler. So the decorator would still just be a name
>>> for
>>> something that is triggered inside of the compiler, which may be a
>>> code
>>> block, a function, a method or a whole transformation class.
>>>
>>> While I do not doubt that there is potential for a code cleanup, the
>>> main
>>> reason why the "decorators are scattered in many places" is that
>>> their
>>> implementation requires code to be executed during different
>>> compiler
>>> stages. This cannot be helped by stuffing everything into a single
>>> function.
>>
>> Also, many (most?) directives (e.g. boundscheck, cdivision,
>> infer_types) aren't best expressed as node transformations, rather
>> they simply affect the behavior of the contained code. I would
>> certainly imagine that @cfunc, @cclass, (and others) will be
>> implemented a visitor that in turn calls such transforming functions.
>>
>> - Robert
>>
>
> In my opinion, decorators should only responsible for either setting
> directives or calling a transforming method of the node to transform
> the node to another.
This is currently the case.
> Maybe we can refactor the current
> InterpretCompilerDirectives transform to add generic support for
> decorators and directives.
As mentioned earlier, one difficulty is that different transformations
need to be handled at different phases of the compilation pipeline.
Doing something like this for plugins and/or macros could be an
interesting avenue to pursue once we have full Python compatibility.
> Additionally, I think it is better to have
> something like a CythonModule.py as a counterpart of the Shadow.py,
> then it is easy to assure the content of the 'cython' module is the
> same between Cython and pure Python mode.
I'm not convinced that grouping internal Cython constructions based on
whether they have a "pure" counterpart is the best way to do things--
in my ind the directives having to do with division are best handled
in the division node, those dealing with buffers best handled in the
buffer code, etc.
> Another interesting problem is how to deal with name overriding.
Yes, this is an issue that has not been adequately solved yet. I'm not
sure it can be (see below).
> What to do if you have:
>
> import cython as C
> ...
> C = 'foo'
> ...
>
> or similar thing like:
>
> import cython
> ...
> foo = cython.cdef
> ...
from cython import cclass
if random.random() < 0.5:
cclass = lambda x: x
%cclass:
class Foo:
...
>
> and as mentioned by Robert:
>
> from cython import *
> from foo import * # where foo contains something clash to cython
>
> Note that these may also apply to some Python built-in such as
> @property, which may need some special treatment so can not just seen
> as normal Python built-in.
>
> As InterpretCompilerDirectives happens before
> AnalyseDeclarationsTransform, seems the only choice is to forbid these
> things. This could work in this way:
>
> 1. All the cython names are imported to a protected global scope
> 2. The necessary directive setting and node transforms are all done in
> the InterpretCompilerDirectives stage.
> 3. After InterpretCompilerDirectives, raise an error for any access to
> the protected global scope. ('int' and 'float' may be exceptions)
Hmm... Something like this (or at least tying whether or not something
is a cython directive into the Entry objects) may improve semantics
somewhat.
> 4. After "from cython import *", any "from foo import *" is disallowed
> (or warned?). (An alternative is to forbid "from cython import *")
I think it's much safer to simply forbid "from cython import *",
especially as its contents may change with time.
> This may be the best we can do. Of course, one can always do
> exec('cython = "foo" '), well...
At least Cython wouldn't choke on that.
- Robert
_______________________________________________
Cython-dev mailing list
[email protected]
http://codespeak.net/mailman/listinfo/cython-dev
