Martin C. Martin wrote:
> Robert Bradshaw wrote:
>
>> On May 29, 2008, at 7:10 PM, rahul garg wrote:
>>
>>> b) Cython does not construct ASTs Parse trees are used as the
>>> internal representation and Cython appears to be generating code
>>> directly from parse trees currently. Is this correct? I also see
>>> that there are proposals to add ASTs , construct control flow graph
>>> as well as for an explicit symbol table.
>>>
>>>
>> That is correct, though the resulting tree is not strictly the parse
>> tree--the parser does some analysis, many of the leaf nodes (e.g.
>> decelerator nodes) get recorded as node attributes, and there is the
>> possibility of tree transformations.
>>
>
> How does it differ from an AST then?
>
The Cython tree comes close; right after the parsing stage (at which
point it is possible to do psuedo-AST-transformations, though none is
done currently).
The difference is this:
The tree that the Cython parser generates has a structure that is
determined by how it is going to be output to C, and is not a direct
tree representation of the Cython code. For instance, "x, y, z = a, b,
c" will show up in the tree structure as (psuedo-code)
"ParallellAssignment([{x = a}, {y = b}, {z = c}])" rather than what an
AST would have, i.e. "ParallellAssignment(lhs=[{x}, {y}, {z}], rhs=[{a},
{b}, {c}])". There are small cases of variations like this all over the
place, so you generally cannot guess things about the tree structure
from how the corresponding Cython code would look.
Another example is that the Cython parser does some processing, for
instance pulls in any "cimport"ed files/modules etc directly. A purer
AST approach would just leave the "cimport" statements in place and
leave it to code further down to do something about them (allowing
transformations affecting cimports and so on).
I guess it all boils down to the fact that the Cython parser isn't
really a "parser", it's a parser + a lot of other stuff. Getting a real
AST in Cython is not such a hard task (refactor Parsing.py into two
components, putting all the non-parsing stuff in a new post-parse
transform -- this can happen gradually and part by part), but it likely
won't happen until there's a real benefit. When working on parsing Py3
code it could be a good time to look into doing some of this to
streamline Parsing.py somewhat.
Dag Sverre
_______________________________________________
Cython-dev mailing list
[email protected]
http://codespeak.net/mailman/listinfo/cython-dev
