mark florisson wrote: > Hello, > > There currently is no proper debugger available to debug Cython code. > The current way we debug Cython code is: > - the print statement > - a C debugger > - pdb > > None of the aforementioned methods is very powerful and time effective > at the same time. For instance, 'print' requires recompilation and is > not so powerful, pdb doesn't provide access to anything C-level and > gdb often requires a lot of digging through generated C code. This is > why I feel that there is a need to a proper Cython debugger. > > This debugger should have, for starters, the following features: > - Cython variable/type/function name correspondence (the ability to > set breakpoints for Cython function/method names, the ability to > inspect variables by their Cython name (be it C or Python variables), > etc) > - A line number and source code correspondence with the Cython code > (ability to view the code at the C and Cython abstraction levels) > - Be able to inspect Python code, print python backtraces, etc > (there's already Misc/gdbinit in the Python source distribution and > there is the EasierPythonDebugging project) > > Such a debugger would need some kind of information, namely it would need: > - a mapping from Cython line numbers to a range of C line numbers > - a mapping from Cython names to mangled C names > - naming information with regard to scope > > My plan is to extend GDB with Python and introduce new Cython commands > that would deal with these issues. It would use the information > exported by the Cython compiler to realize this (the easiest and most > portable way would probably be to write it to a separate file in some > format (xml or json come to mind)). > Excellent! This is great news! > So I have a few questions for cython-dev. > First of all, all the scope and naming information is in the AST, so I > think the easiest way would be to traverse the AST and generate a > similar tree where nodes contain information such as: > - C name > - Cython name > - Line number > - C object or Python object > > I think this code could be called from the 'parse' closure in > Cython.Compiler.Main.create_parse() just before returning 'tree'. What > do you think? > One might need to do it a little later, because C names and so on might not have been resolved at that stage.
Because each node in the tree contains a pointer to their initial position anyway (in their "pos" attribute), I wonder if one should in fact do this at the very last, after code generation. Then we have the freedom to make up new C names during the code generation phase. Basically, this means writing a new transform and plugging it into the pipeline after code generation. Please ask for more details if you want and I hope somebody will help, although I'm a bit rushed at the moment myself. > The line number correspondence is a little harder, because I believe > that no code is currently saving Cython line number information when > generating C code. Seeing that CCodeWriter (and StringIOTree) have > these Insertion Points I think line number information would have to > be saved there, and all the code dealing with that would have to be > adjusted. Does that sound right or do you think there is an easier way > to implement this? > Actually, Cython can already emit gcc line directives through the --line-directives switch. If you can't use that directly with GDB then grepping the Cython source for "emit_linenums" should at least put you on track. Robert will know more about this, as he wrote that code. > Another question is, if I would create a patch for Cython to have > Cython emit this kind of debugger information, would it be accepted if > accompanied by tests and according to the rules etc? I would be > willing to maintain the code at least. And what is your general > opinion of this debugger? What else would you see as an essential > feature of such a Cython debugger? And do you think the actual > debugger should be an external project or should it be part of Cython > (if it's mature)? I think the former option would allow a more > flexible release schedule. > I think it is very likely that we would love to ship the necesarry modifications (once stable) as part of main Cython. If your Cython patches can become part of mainline GDB then there's no question about it at all, but even if you have to maintain a forked GDB I'm all for it (and in that case we could host the forked GDB on cython.org as well and so on). Let us know if you would like a more "official" branch on cython.org, although just keeping one on bitbucket.org is usually just as easy. Then we can pull the changes into mainline when things work well. Dag Sverre _______________________________________________ Cython-dev mailing list [email protected] http://codespeak.net/mailman/listinfo/cython-dev
