Re: [IronPython] Debugging support PythonEngine
We do actually run all our tests using both DynamicMethods and AssemblyBuilder. This has helped us catch bugs in the DynamicMethods implementation. If there is any difference, it is a bug that should be fixed. Running ipy.exe foo.py will use the AssemblyBuilder, so that code path is used heavily. In fact, more than DynamicMethods. The IL generation is actually unaware of whether the method is a DynamicMethod or a MethodBuilder. So there should not be any differences. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of J. Merrill Sent: Thursday, August 17, 2006 9:44 PM To: Discussion of IronPython Subject: Re: [IronPython] Debugging support PythonEngine Without looking at the IP source, can we be confident that the same IL is produced when a DynamicMethod is created as when AssemblyBuilder is used? Do the test cases get run with both settings of Options.GenerateDynamicMethods? One of the most frustrating debugging problems is when code fails when running normally, but works when attempting to debug, due to the code being generated differently in the two situations. I have not heard anyone else worry out loud that e.g. the code running when using the console seems to be different than what is obtained using the IronPython.Compiler, etc. How concerned should we be when we learn that the code paths are very different? At 05:49 PM 8/17/2006, Shri Borde wrote The first issue is probably the same that IL offset 0 does not have any debug information. Calling the delegate ?add? steps into a DynamicMethod which confuses VS. A workaround for now is to add this statement before creating the PythonEngine. IronPython.Compiler.Options.GenerateDynamicMethods = false; This disabled the use of DynamicMethods altogether and will instead use AssemblyBuilder. This is not generally recommended or supported since you are going to leak memory since AssemblyBuilder memory cannot be reclaimed. However, VS seems to be better able to deal with AssemblyBuilder during stepping, and it might be good enough for you. This is a case where we will would need to teach the tool (VS) to handle DynamicMethods differently than it currently does. [snip] J. Merrill / Analytical Software Corp ___ users mailing list users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Debugging support PythonEngine
Setting ClrDebuggingEnabled = true will only leak memory for the code executed using the PythonEngine.ExecuteFile API. Any other evals done by other Python modules that are loaded in the engine (using the import command) and code executed using PythonEngine.Evaluate or PythonEngine.Execute will still DynamicMethods. However, setting with IronPython.Compiler.Options.GenerateDynamicMethods = false will use AssemblyBuilder for those cases too, and so will leak more memory. We are working on providing documentation for PythonEngine. For now, the answer is simply that memory used by System.Reflection.Emit.AssemblyBuilder, MethodBuilder, etc. can not be reclaimed whereas memory used by System.Reflection.Emit.DynamicMethod can be reclaimed From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kristof Wagemans Sent: Thursday, August 17, 2006 11:43 PM To: 'Discussion of IronPython' Subject: Re: [IronPython] Debugging support PythonEngine Leaking memory with IronPython.Compiler.Options.GenerateDynamicMethods = false isnt really a big issue: ClrDebuggingEnabled = true already starts to leak memory (at least thats what the comment says). Is there any complete documentation about what actions cause memory leaks? By following this mailing list and looking at the source code comments I know of some things, but Im sure that I dont know all situations. Maybe it would be a good idea to have a setting AllowMemoryLeaks which has a default of false. Whenever you go through a code path that is going to leak memory through code generation you could throw an exception. This way, you know at least that something youre doing is wrong. If youre OK with the memory leak, then you could set AllowMemoryLeaks to true. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Shri Borde Sent: Thursday 17 August 2006 23:49 To: Discussion of IronPython Subject: Re: [IronPython] Debugging support PythonEngine The first issue is probably the same that IL offset 0 does not have any debug information. Calling the delegate add steps into a DynamicMethod which confuses VS. A workaround for now is to add this statement before creating the PythonEngine. IronPython.Compiler.Options.GenerateDynamicMethods = false; This disabled the use of DynamicMethods altogether and will instead use AssemblyBuilder. This is not generally recommended or supported since you are going to leak memory since AssemblyBuilder memory cannot be reclaimed. However, VS seems to be better able to deal with AssemblyBuilder during stepping, and it might be good enough for you. This is a case where we will would need to teach the tool (VS) to handle DynamicMethods differently than it currently does. ___ users mailing list users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Debugging support PythonEngine
Without looking at the IP source, can we be confident that the same IL is produced when a DynamicMethod is created as when AssemblyBuilder is used? Do the test cases get run with both settings of Options.GenerateDynamicMethods? One of the most frustrating debugging problems is when code fails when running normally, but works when attempting to debug, due to the code being generated differently in the two situations. I have not heard anyone else worry out loud that e.g. the code running when using the console seems to be different than what is obtained using the IronPython.Compiler, etc. How concerned should we be when we learn that the code paths are very different? At 05:49 PM 8/17/2006, Shri Borde wrote The first issue is probably the same that IL offset 0 does not have any debug information. Calling the delegate add steps into a DynamicMethod which confuses VS. A workaround for now is to add this statement before creating the PythonEngine. IronPython.Compiler.Options.GenerateDynamicMethods = false; This disabled the use of DynamicMethods altogether and will instead use AssemblyBuilder. This is not generally recommended or supported since you are going to leak memory since AssemblyBuilder memory cannot be reclaimed. However, VS seems to be better able to deal with AssemblyBuilder during stepping, and it might be good enough for you. This is a case where we will would need to teach the tool (VS) to handle DynamicMethods differently than it currently does. [snip] J. Merrill / Analytical Software Corp ___ users mailing list users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Debugging support PythonEngine
Leaking memory with IronPython.Compiler.Options.GenerateDynamicMethods = false isnt really a big issue: ClrDebuggingEnabled = true already starts to leak memory (at least thats what the comment says). Is there any complete documentation about what actions cause memory leaks? By following this mailing list and looking at the source code comments I know of some things, but Im sure that I dont know all situations. Maybe it would be a good idea to have a setting AllowMemoryLeaks which has a default of false. Whenever you go through a code path that is going to leak memory through code generation you could throw an exception. This way, you know at least that something youre doing is wrong. If youre OK with the memory leak, then you could set AllowMemoryLeaks to true. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Shri Borde Sent: Thursday 17 August 2006 23:49 To: Discussion of IronPython Subject: Re: [IronPython] Debugging support PythonEngine The first issue is probably the same that IL offset 0 does not have any debug information. Calling the delegate add steps into a DynamicMethod which confuses VS. A workaround for now is to add this statement before creating the PythonEngine. IronPython.Compiler.Options.GenerateDynamicMethods = false; This disabled the use of DynamicMethods altogether and will instead use AssemblyBuilder. This is not generally recommended or supported since you are going to leak memory since AssemblyBuilder memory cannot be reclaimed. However, VS seems to be better able to deal with AssemblyBuilder during stepping, and it might be good enough for you. This is a case where we will would need to teach the tool (VS) to handle DynamicMethods differently than it currently does. ___ users mailing list users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Debugging support PythonEngine
I have opened this bug - http://www.codeplex.com/WorkItem/View.aspx?ProjectName=IronPythonWorkItemId=2248 - to track the issue with stepping in. We are not emitting any line number information for IL offset 0. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Shri Borde Sent: Wednesday, August 16, 2006 12:29 PM To: Discussion of IronPython Subject: Re: [IronPython] Debugging support PythonEngine We have started doing some work for improving debugging as shown by the VSIP sample (http://blogs.msdn.com/aaronmar/archive/2006/02/16/533273.aspx). However, it still needs more work for it to be production-quality. By tools support, I was referring more to VSIP sample. We dont have a concrete timeline for when we will be able to improve it further. Btw, using your sample script below, in most cases, I am able to step from Python code into C# code in IronPython.dll if I hit F11 in VS. However, when IronPython.dll calls the method Add, we step into native code for a while before actually reaching the source code for Add. This looks like something we can fix by changing the debug information we generate. After doing this fix, it would be possible to mark IronPython.dll as library code (either using DebuggerNonUserCodeAttribute or through some other scheme), and be able to step from Python code into a Python method. x = 1 y = 2 def Add(a, b): return a + b z = Add(x, y) ß Stepping in here takes you to IronPython.dll source code, then native code, and then the source code for Add print z From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kristof Wagemans Sent: Saturday, August 12, 2006 1:38 AM To: 'Discussion of IronPython' Subject: Re: [IronPython] Debugging support PythonEngine Thanks for the info. Its good to hear that there will be improvement possible in this area in the future. Although, the way I interpret your explanation, its not going to be anytime soon (or even this year), because you need enhancements to the underlying platform to make this possible. Ive experimented a bit with System.Diagnostics.DebuggerNonUserCodeAttribute to see if it wasnt possible to skip stepping into IronPython code. Unfortunately, the point where I get stuck in the assembly instructions is in generated code. I dont understand this part of the code well enough to know if its possible to make the debugger skip these lines. Maybe this is why you need the changes to the platform? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Shri Borde Sent: Friday 11 August 2006 21:04 To: Discussion of IronPython Subject: Re: [IronPython] Debugging support PythonEngine Yes, we will definitely be working on improving the debugging support as it is a critical part of the development process. However, full support will need work in all parts of the tool chain including VS. Python local variables are implemented as normal MSIL variables (except in cases like closures). Hence, VS is able to display them. Currently, the best way to debug Python functions in VS while using PythonEngine would be to enable EngineOptions.ClrDebuggingEnable, open the PY file in VS and put a breakpoint where you want. Stepping in and out of Python functions will step you through methods in IronPython.dll . From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kristof Wagemans Sent: Friday, August 11, 2006 12:36 AM To: 'Discussion of IronPython' Subject: Re: [IronPython] Debugging support PythonEngine Are there plans to improve the debugging experience in the future or are you at a point where it is as good as it gets? Being able to debug makes writing and using python scripts a lot easier. Inside functions I can see not only the function parameters but also the newly defined local variables. Are these different from the global variables? I would have expected that the function locals are also stored inside a dictionary. Is it possible to step into python functions without getting into assembly instructions? This is very frustrating and degrades the debugging experience a lot. This is the more important issue for me. Im not going to be using global variables frequently, but mostly functions loaded in the PythonEngine that are called from C#. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Shri Borde Sent: Friday 11 August 2006 0:30 To: Discussion of IronPython Cc: Glenn Hackney Subject: Re: [IronPython] Debugging support PythonEngine If EngineOptions.ClrDebuggingEnabled is set, we use AssemblyBuilder, TypeBuilder, etc for PythonEngine.Executed. The code generated by AssemblyBuilder, TypeBuilder, etc supports PDB debug information tracking for the methods, and so you will be able to set breakpoints in the code. However, it does not guarantee a perfect debugging experience. PythonEngine.ExecuteFile will use a dictionary for storing global variables, and these will not be visible because VS does not know about
Re: [IronPython] Debugging support PythonEngine
Its good to hear that its fixable. But, off course, heres the next one J. With the same little script: x = 1 y = 2 def Add(a, b): x = a + b return x z = Add(x, y) print z And the following C# code: EngineOptions options = new EngineOptions(); options.ClrDebuggingEnabled = true; PythonEngine _pe = new PythonEngine(options); delegate int Add(int a, int b); _pe.ExecuteFile(@ script path ); Add add = _pe.EvaluateAsAdd(Add); int var1 = add(2, 3); int var2 = add(3, 4); The following happens: (I use a release version of Ironpython (RC2). There are no debug symbols present, so that I dont step into the engine code.) I step into ExecuteFile (F11) and get into native code. After a few lines of native code, I can go back to the source code view and Im in the Python script file. Maybe this is a similar issue as with stepping into a function? Next, I get a delegate to the Python Add function. The first time I step into the add delegate I get the stepping into a function problem and then Im inside the function. But, when I try to step into the next add delegate this doesnt work anymore: the line is marked in green in Visual Studio and I remain in the C# code. The next step into command resumes on the next line in the C# code. I cant see a reason why it would fail with the next add. Setting a breakpoint in the Python function stops the execution at that point in both cases. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Shri Borde Sent: Thursday 17 August 2006 20:25 To: Discussion of IronPython Subject: Re: [IronPython] Debugging support PythonEngine I have opened this bug - http://www.codeplex.com/WorkItem/View.aspx?ProjectName=IronPythonWorkItemId=2248 - to track the issue with stepping in. We are not emitting any line number information for IL offset 0. ___ users mailing list users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Debugging support PythonEngine
The first issue is probably the same that IL offset 0 does not have any debug information. Calling the delegate add steps into a DynamicMethod which confuses VS. A workaround for now is to add this statement before creating the PythonEngine. IronPython.Compiler.Options.GenerateDynamicMethods = false; This disabled the use of DynamicMethods altogether and will instead use AssemblyBuilder. This is not generally recommended or supported since you are going to leak memory since AssemblyBuilder memory cannot be reclaimed. However, VS seems to be better able to deal with AssemblyBuilder during stepping, and it might be good enough for you. This is a case where we will would need to teach the tool (VS) to handle DynamicMethods differently than it currently does. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kristof Wagemans Sent: Thursday, August 17, 2006 1:54 PM To: 'Discussion of IronPython' Subject: Re: [IronPython] Debugging support PythonEngine Its good to hear that its fixable. But, off course, heres the next one J. With the same little script: x = 1 y = 2 def Add(a, b): x = a + b return x z = Add(x, y) print z And the following C# code: EngineOptions options = new EngineOptions(); options.ClrDebuggingEnabled = true; PythonEngine _pe = new PythonEngine(options); delegate int Add(int a, int b); _pe.ExecuteFile(@ script path ); Add add = _pe.EvaluateAsAdd(Add); int var1 = add(2, 3); int var2 = add(3, 4); The following happens: (I use a release version of Ironpython (RC2). There are no debug symbols present, so that I dont step into the engine code.) I step into ExecuteFile (F11) and get into native code. After a few lines of native code, I can go back to the source code view and Im in the Python script file. Maybe this is a similar issue as with stepping into a function? Next, I get a delegate to the Python Add function. The first time I step into the add delegate I get the stepping into a function problem and then Im inside the function. But, when I try to step into the next add delegate this doesnt work anymore: the line is marked in green in Visual Studio and I remain in the C# code. The next step into command resumes on the next line in the C# code. I cant see a reason why it would fail with the next add. Setting a breakpoint in the Python function stops the execution at that point in both cases. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Shri Borde Sent: Thursday 17 August 2006 20:25 To: Discussion of IronPython Subject: Re: [IronPython] Debugging support PythonEngine I have opened this bug - http://www.codeplex.com/WorkItem/View.aspx?ProjectName=IronPythonWorkItemId=2248 - to track the issue with stepping in. We are not emitting any line number information for IL offset 0. ___ users mailing list users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Debugging support PythonEngine
We have started doing some work for improving debugging as shown by the VSIP sample (http://blogs.msdn.com/aaronmar/archive/2006/02/16/533273.aspx). However, it still needs more work for it to be production-quality. By tools support, I was referring more to VSIP sample. We dont have a concrete timeline for when we will be able to improve it further. Btw, using your sample script below, in most cases, I am able to step from Python code into C# code in IronPython.dll if I hit F11 in VS. However, when IronPython.dll calls the method Add, we step into native code for a while before actually reaching the source code for Add. This looks like something we can fix by changing the debug information we generate. After doing this fix, it would be possible to mark IronPython.dll as library code (either using DebuggerNonUserCodeAttribute or through some other scheme), and be able to step from Python code into a Python method. x = 1 y = 2 def Add(a, b): return a + b z = Add(x, y) Stepping in here takes you to IronPython.dll source code, then native code, and then the source code for Add print z From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kristof Wagemans Sent: Saturday, August 12, 2006 1:38 AM To: 'Discussion of IronPython' Subject: Re: [IronPython] Debugging support PythonEngine Thanks for the info. Its good to hear that there will be improvement possible in this area in the future. Although, the way I interpret your explanation, its not going to be anytime soon (or even this year), because you need enhancements to the underlying platform to make this possible. Ive experimented a bit with System.Diagnostics.DebuggerNonUserCodeAttribute to see if it wasnt possible to skip stepping into IronPython code. Unfortunately, the point where I get stuck in the assembly instructions is in generated code. I dont understand this part of the code well enough to know if its possible to make the debugger skip these lines. Maybe this is why you need the changes to the platform? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Shri Borde Sent: Friday 11 August 2006 21:04 To: Discussion of IronPython Subject: Re: [IronPython] Debugging support PythonEngine Yes, we will definitely be working on improving the debugging support as it is a critical part of the development process. However, full support will need work in all parts of the tool chain including VS. Python local variables are implemented as normal MSIL variables (except in cases like closures). Hence, VS is able to display them. Currently, the best way to debug Python functions in VS while using PythonEngine would be to enable EngineOptions.ClrDebuggingEnable, open the PY file in VS and put a breakpoint where you want. Stepping in and out of Python functions will step you through methods in IronPython.dll . From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kristof Wagemans Sent: Friday, August 11, 2006 12:36 AM To: 'Discussion of IronPython' Subject: Re: [IronPython] Debugging support PythonEngine Are there plans to improve the debugging experience in the future or are you at a point where it is as good as it gets? Being able to debug makes writing and using python scripts a lot easier. Inside functions I can see not only the function parameters but also the newly defined local variables. Are these different from the global variables? I would have expected that the function locals are also stored inside a dictionary. Is it possible to step into python functions without getting into assembly instructions? This is very frustrating and degrades the debugging experience a lot. This is the more important issue for me. Im not going to be using global variables frequently, but mostly functions loaded in the PythonEngine that are called from C#. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Shri Borde Sent: Friday 11 August 2006 0:30 To: Discussion of IronPython Cc: Glenn Hackney Subject: Re: [IronPython] Debugging support PythonEngine If EngineOptions.ClrDebuggingEnabled is set, we use AssemblyBuilder, TypeBuilder, etc for PythonEngine.Executed. The code generated by AssemblyBuilder, TypeBuilder, etc supports PDB debug information tracking for the methods, and so you will be able to set breakpoints in the code. However, it does not guarantee a perfect debugging experience. PythonEngine.ExecuteFile will use a dictionary for storing global variables, and these will not be visible because VS does not know about the dictionary. If you use PythonEngine.CreateOptimizedModule, most global variables are implemented using CLR statics, and so VS may be able to display them for you. Global variables added using an exec statement will still not be visible. If EngineOptions.ClrDebuggingEnabled is false, we will use System.Reflection.Emit.DynamicMethod. This does not support debug information tracking at all, and you will not even be able
Re: [IronPython] Debugging support PythonEngine
Thanks for the info. Its good to hear that there will be improvement possible in this area in the future. Although, the way I interpret your explanation, its not going to be anytime soon (or even this year), because you need enhancements to the underlying platform to make this possible. Ive experimented a bit with System.Diagnostics.DebuggerNonUserCodeAttribute to see if it wasnt possible to skip stepping into IronPython code. Unfortunately, the point where I get stuck in the assembly instructions is in generated code. I dont understand this part of the code well enough to know if its possible to make the debugger skip these lines. Maybe this is why you need the changes to the platform? From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Shri Borde Sent: Friday 11 August 2006 21:04 To: Discussion of IronPython Subject: Re: [IronPython] Debugging support PythonEngine Yes, we will definitely be working on improving the debugging support as it is a critical part of the development process. However, full support will need work in all parts of the tool chain including VS. Python local variables are implemented as normal MSIL variables (except in cases like closures). Hence, VS is able to display them. Currently, the best way to debug Python functions in VS while using PythonEngine would be to enable EngineOptions.ClrDebuggingEnable, open the PY file in VS and put a breakpoint where you want. Stepping in and out of Python functions will step you through methods in IronPython.dll . From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kristof Wagemans Sent: Friday, August 11, 2006 12:36 AM To: 'Discussion of IronPython' Subject: Re: [IronPython] Debugging support PythonEngine Are there plans to improve the debugging experience in the future or are you at a point where it is as good as it gets? Being able to debug makes writing and using python scripts a lot easier. Inside functions I can see not only the function parameters but also the newly defined local variables. Are these different from the global variables? I would have expected that the function locals are also stored inside a dictionary. Is it possible to step into python functions without getting into assembly instructions? This is very frustrating and degrades the debugging experience a lot. This is the more important issue for me. Im not going to be using global variables frequently, but mostly functions loaded in the PythonEngine that are called from C#. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Shri Borde Sent: Friday 11 August 2006 0:30 To: Discussion of IronPython Cc: Glenn Hackney Subject: Re: [IronPython] Debugging support PythonEngine If EngineOptions.ClrDebuggingEnabled is set, we use AssemblyBuilder, TypeBuilder, etc for PythonEngine.Executed. The code generated by AssemblyBuilder, TypeBuilder, etc supports PDB debug information tracking for the methods, and so you will be able to set breakpoints in the code. However, it does not guarantee a perfect debugging experience. PythonEngine.ExecuteFile will use a dictionary for storing global variables, and these will not be visible because VS does not know about the dictionary. If you use PythonEngine.CreateOptimizedModule, most global variables are implemented using CLR statics, and so VS may be able to display them for you. Global variables added using an exec statement will still not be visible. If EngineOptions.ClrDebuggingEnabled is false, we will use System.Reflection.Emit.DynamicMethod. This does not support debug information tracking at all, and you will not even be able to see function variables. So If EngineOptions.ClrDebuggingEnabled is will improve your debugging experience, but it wont give you a perfect experience. Do you want to help develop Dynamic languages on CLR? (http://members.microsoft.com/careers/search/details.aspx?JobID=6D4754DE-11F0-45DF-8B78-DC1B43134038) From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kristof Wagemans Sent: Thursday, August 10, 2006 2:10 PM To: Discussion of IronPython Subject: [IronPython] Debugging support PythonEngine I have been experimenting with the debugging support for the PythonEngine. When I use the following code I have several problems. PythonEngine _pe; EngineOptions options = new EngineOptions(); options.ClrDebuggingEnabled = true; _pe = new PythonEngine(options); _pe.ExecuteFile(@ script ); Test script: x = 1 y = 2 def Add(a, b): return a + b z = Add(x, y) print z I opened the script file in Visual Studio and placed a breakpoint at the beginning of the file. The application runs and breaks at the correct location. Stepping through the lines works, but I cannot see any values of the global variables. When I try to step into the function I get a notification that there is no source code available and I must show the disassembly. After I step several times
Re: [IronPython] Debugging support PythonEngine
Are there plans to improve the debugging experience in the future or are you at a point where it is as good as it gets? Being able to debug makes writing and using python scripts a lot easier. Inside functions I can see not only the function parameters but also the newly defined local variables. Are these different from the global variables? I would have expected that the function locals are also stored inside a dictionary. Is it possible to step into python functions without getting into assembly instructions? This is very frustrating and degrades the debugging experience a lot. This is the more important issue for me. Im not going to be using global variables frequently, but mostly functions loaded in the PythonEngine that are called from C#. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Shri Borde Sent: Friday 11 August 2006 0:30 To: Discussion of IronPython Cc: Glenn Hackney Subject: Re: [IronPython] Debugging support PythonEngine If EngineOptions.ClrDebuggingEnabled is set, we use AssemblyBuilder, TypeBuilder, etc for PythonEngine.Executed. The code generated by AssemblyBuilder, TypeBuilder, etc supports PDB debug information tracking for the methods, and so you will be able to set breakpoints in the code. However, it does not guarantee a perfect debugging experience. PythonEngine.ExecuteFile will use a dictionary for storing global variables, and these will not be visible because VS does not know about the dictionary. If you use PythonEngine.CreateOptimizedModule, most global variables are implemented using CLR statics, and so VS may be able to display them for you. Global variables added using an exec statement will still not be visible. If EngineOptions.ClrDebuggingEnabled is false, we will use System.Reflection.Emit.DynamicMethod. This does not support debug information tracking at all, and you will not even be able to see function variables. So If EngineOptions.ClrDebuggingEnabled is will improve your debugging experience, but it wont give you a perfect experience. Do you want to help develop Dynamic languages on CLR? (http://members.microsoft.com/careers/search/details.aspx?JobID=6D4754DE-11F0-45DF-8B78-DC1B43134038) From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kristof Wagemans Sent: Thursday, August 10, 2006 2:10 PM To: Discussion of IronPython Subject: [IronPython] Debugging support PythonEngine I have been experimenting with the debugging support for the PythonEngine. When I use the following code I have several problems. PythonEngine _pe; EngineOptions options = new EngineOptions(); options.ClrDebuggingEnabled = true; _pe = new PythonEngine(options); _pe.ExecuteFile(@ script ); Test script: x = 1 y = 2 def Add(a, b): return a + b z = Add(x, y) print z I opened the script file in Visual Studio and placed a breakpoint at the beginning of the file. The application runs and breaks at the correct location. Stepping through the lines works, but I cannot see any values of the global variables. When I try to step into the function I get a notification that there is no source code available and I must show the disassembly. After I step several times through the assembly instructions I can return to the original source code. Inside the function I can see the values of the function variables. I have tried debugging ipy.exe with the script and there I can see the global variables, but I still have the problem with stepping into a function. In ipy.exe the script file is executed in a different way. Using the same method I can also see the global variables with my PythonEngine instance. I apparently dont need to set ClrDebuggingEnabled in this case. PythonEngine _pe; _pe = new PythonEngine(); OptimizedEngineModule engineModule = _pe.CreateOptimizedModule(@ script , __main__, true); engineModule.Execute(); Are you required to use an OptimizedEngineModule to be able to debug completely? Am I forgetting some settings for debugging? Can I step directly into a function without getting into the assembly instructions? ___ users mailing list users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
Re: [IronPython] Debugging support PythonEngine
Yes, we will definitely be working on improving the debugging support as it is a critical part of the development process. However, full support will need work in all parts of the tool chain including VS. Python local variables are implemented as normal MSIL variables (except in cases like closures). Hence, VS is able to display them. Currently, the best way to debug Python functions in VS while using PythonEngine would be to enable EngineOptions.ClrDebuggingEnable, open the PY file in VS and put a breakpoint where you want. Stepping in and out of Python functions will step you through methods in IronPython.dll . From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kristof Wagemans Sent: Friday, August 11, 2006 12:36 AM To: 'Discussion of IronPython' Subject: Re: [IronPython] Debugging support PythonEngine Are there plans to improve the debugging experience in the future or are you at a point where it is as good as it gets? Being able to debug makes writing and using python scripts a lot easier. Inside functions I can see not only the function parameters but also the newly defined local variables. Are these different from the global variables? I would have expected that the function locals are also stored inside a dictionary. Is it possible to step into python functions without getting into assembly instructions? This is very frustrating and degrades the debugging experience a lot. This is the more important issue for me. Im not going to be using global variables frequently, but mostly functions loaded in the PythonEngine that are called from C#. From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Shri Borde Sent: Friday 11 August 2006 0:30 To: Discussion of IronPython Cc: Glenn Hackney Subject: Re: [IronPython] Debugging support PythonEngine If EngineOptions.ClrDebuggingEnabled is set, we use AssemblyBuilder, TypeBuilder, etc for PythonEngine.Executed. The code generated by AssemblyBuilder, TypeBuilder, etc supports PDB debug information tracking for the methods, and so you will be able to set breakpoints in the code. However, it does not guarantee a perfect debugging experience. PythonEngine.ExecuteFile will use a dictionary for storing global variables, and these will not be visible because VS does not know about the dictionary. If you use PythonEngine.CreateOptimizedModule, most global variables are implemented using CLR statics, and so VS may be able to display them for you. Global variables added using an exec statement will still not be visible. If EngineOptions.ClrDebuggingEnabled is false, we will use System.Reflection.Emit.DynamicMethod. This does not support debug information tracking at all, and you will not even be able to see function variables. So If EngineOptions.ClrDebuggingEnabled is will improve your debugging experience, but it wont give you a perfect experience. Do you want to help develop Dynamic languages on CLR? (http://members.microsoft.com/careers/search/details.aspx?JobID=6D4754DE-11F0-45DF-8B78-DC1B43134038) From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Kristof Wagemans Sent: Thursday, August 10, 2006 2:10 PM To: Discussion of IronPython Subject: [IronPython] Debugging support PythonEngine I have been experimenting with the debugging support for the PythonEngine. When I use the following code I have several problems. PythonEngine _pe; EngineOptions options = new EngineOptions(); options.ClrDebuggingEnabled = true; _pe = new PythonEngine(options); _pe.ExecuteFile(@ script ); Test script: x = 1 y = 2 def Add(a, b): return a + b z = Add(x, y) print z I opened the script file in Visual Studio and placed a breakpoint at the beginning of the file. The application runs and breaks at the correct location. Stepping through the lines works, but I cannot see any values of the global variables. When I try to step into the function I get a notification that there is no source code available and I must show the disassembly. After I step several times through the assembly instructions I can return to the original source code. Inside the function I can see the values of the function variables. I have tried debugging ipy.exe with the script and there I can see the global variables, but I still have the problem with stepping into a function. In ipy.exe the script file is executed in a different way. Using the same method I can also see the global variables with my PythonEngine instance. I apparently dont need to set ClrDebuggingEnabled in this case. PythonEngine _pe; _pe = new PythonEngine(); OptimizedEngineModule engineModule = _pe.CreateOptimizedModule(@ script , __main__, true); engineModule.Execute(); Are you required to use an OptimizedEngineModule to be able to debug completely? Am I forgetting some settings for debugging? Can I step directly into a function without getting into the assembly instructions
[IronPython] Debugging support PythonEngine
I have been experimenting with the debugging support for the PythonEngine. When I use the following code I have several problems. PythonEngine _pe; EngineOptions options = new EngineOptions(); options.ClrDebuggingEnabled = true; _pe = new PythonEngine(options); _pe.ExecuteFile(@ script ); Test script: x = 1 y = 2 def Add(a, b): return a + b z = Add(x, y) print z I opened the script file in Visual Studio and placed a breakpoint at the beginning of the file. The application runs and breaks at the correct location. Stepping through the lines works, but I cannot see any values of the global variables. When I try to step into the function I get a notification that there is no source code available and I must show the disassembly. After I step several times through the assembly instructions I can return to the original source code. Inside the function I can see the values of the function variables. I have tried debugging ipy.exe with the script and there I can see the global variables, but I still have the problem with stepping into a function. In ipy.exe the script file is executed in a different way. Using the same method I can also see the global variables with my PythonEngine instance. I apparently dont need to set ClrDebuggingEnabled in this case. PythonEngine _pe; _pe = new PythonEngine(); OptimizedEngineModule engineModule = _pe.CreateOptimizedModule(@ script , __main__, true); engineModule.Execute(); Are you required to use an OptimizedEngineModule to be able to debug completely? Am I forgetting some settings for debugging? Can I step directly into a function without getting into the assembly instructions? ___ users mailing list users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
[IronPython] Debugging support PythonEngine
I have been experimenting with the debugging support for the PythonEngine. When I use the following code I have several problems. PythonEngine _pe; EngineOptions options = new EngineOptions(); options.ClrDebuggingEnabled = true; _pe = new PythonEngine(options); _pe.ExecuteFile(@ script ); x = 1 y = 2 def Add(a, b): return a + b z = Add(x, y) print z I opened the script file in Visual Studio and placed a breakpoint at the beginning of the file. The application runs and breaks at the correct location. Stepping through the lines works, but I cannot see any values of variables. When I try to step into the function I get a notification that there is no source code available and I must show the disassembly. After I step several times through the assembly I can return to the source code. I have looked at debugging ipy.exe and there I can see the variables, but I still have the problem with stepping into a function. In ipy.exe the script file is executed in a different way. Using the same method I can also see the variables. I apparently dont need to set ClrDebuggingEnabled. PythonEngine _pe; _pe = new PythonEngine(); OptimizedEngineModule engineModule = _pe.CreateOptimizedModule(@ script , __main__, true); engineModule.Execute(); Are you required to use an OptimizedEngineModule to be able to debug? Am I forgetting some settings for debugging? Can I step directly into a function without getting into the assembly? ___ users mailing list users@lists.ironpython.com http://lists.ironpython.com/listinfo.cgi/users-ironpython.com
