As per Neal's prodding email, here is a thread to discuss where we want to go with the C AST to Python stuff and what I think are the core issues at the moment.
First issue is the ast-objects branch. Work is being done on it, but it still leaks some references (Neal or Martin can correct me if I am wrong). We really should choose either this branch or the current solution before really diving into coding stuff for exposing the AST so as to not waste too much time. Basically the issues are that the current solution will require using a serialization form to go from C to Python and back again. The PyObjects solution in the branch won't need this. One protects us from ending up with an unusable AST since the seralization can keep the original AST around and if the version passed back in from Python code is junk it can be tossed and the original version used. The PyObjects branch most likely won't have this since the actual AST will most likely be passed to Python code. But there is performance issues with all of this seralization compared to a simple Pyobject pointer into Pythonland. Jeremy supports the serialization option. I am personally indifferent while leaning towards the serialization. Then there is the API. First we need to decide if AST modification is allowed or not. It has been argued on my blog by someone (see http://sayspy.blogspot.com/2006/02/possibilities-of-ast.html for the entry on this whole topic which highly mirrors this email) that Guido won't okay AST transformations since it can lead to control flow changes behind the scenes. I say that is fine as long as knowing that AST transformations are occurring are sufficiently obvious. I say allow transformations. Once that is settled, I see three places for possible access to the AST. One is the command line like -m. Totally obvious to the user as long as they are not just working off of the .pyc files. Next is something like sys.ast_transformations that is a list of functions that are passed in the AST (and return a new version if modifications are allowed). This could allow chaining of AST transformations by feeding the next function with another one. Next is per-object AST access. This could get expensive since if we don't keep a copy of the AST with the code objects (which we probably shouldn't since that is wasted memory if the AST is not used a lot) we will need to read the code a second time to get the AST regenerated. I personally think we should choose an initial global access API to the AST as a starting API. I like the sys.ast_transformations idea since it is simple and gives enough access that whether read-only or read-write is allowed something like PyChecker can get the access it needs. It also allows for simple Python scripts that can install the desired functions and then compile or check the passed-in files. Obviously write accesss would be needed for optimization stuff (such as if the peepholer was rewritten in Python and used by default), but we can also expose this later if we want. In terms of 2.5, I think we really need to settle on the fate of the ast-objects branch. If we can get the very basic API for exposing the AST to Python code in 2.5 that would be great, but I don't view that as critical as choosing on the final AST implementation style since wasting work on a version that will disappear would just plain suck. It would be great to resolve this before the PyCon sprints since a good chunk of the AST-caring folk will be there for at least part of the time. -Brett _______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
