[Python.NET] how to use an embedded python intepreter?

[Serge WEINSTOCK]

I'm trying to use an embedded python interpreter from C# using pythonnet (the 
python3 compatible version found at https://github.com/renshawbay/pythonnet)

My interpreter is located in 
D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime and has the "Lib" and 
"Libs" folder from the python distribution.

I've tested using the following code:

//

PythonEngine.PythonHome = 
@"D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime";

PythonEngine.ProgramName = "PythonRuntime";

PythonEngine.Initialize();

using (Py.GIL())

{

PythonEngine.RunSimpleString("print(1)");

}

//

But, it doesn't work. I get a "SystemError: PyEvalCodeEx: NULL globals". 
Everytime I try to get an object from python, the code fails.

What am I doing wrong?


___
This e-mail may contain confidential and/or privileged information. If you are 
not the intended recipient (or have received this e-mail in error) please 
notify the sender immediately and delete this e-mail. Any unauthorised copying, 
disclosure or distribution of the material in this e-mail is prohibited.

Please refer to http://www.bnpparibas.co.uk/en/email-disclaimer/ for additional 
disclosures.
_
Python.NET mailing list - PythonDotNet@python.org
https://mail.python.org/mailman/listinfo/pythondotnet

[Python.NET] Running an embedded interpreter

[Serge WEINSTOCK]

Hi,

I'm trying to use Python3.2 using the Python.Net version found at: 
https://github.com/renshawbay/pythonnet

I'm using the following simple test program:

//===
using System;
using System.IO;
using Python.Runtime;

namespace TestPythonNet
{
class Program
{
static void Main(string[] args)
{
string binDir = 
Path.GetDirectoryName(System.Reflection.Assembly.GetAssembly(typeof(Program)).Location);
string pyHome = 
@"D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime";
PythonEngine.PythonHome = 
@"D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime";
PythonEngine.ProgramName = "PythonRuntime";
Environment.SetEnvironmentVariable("PYTHONPATH",
Path.GetFullPath(Path.Combine(pyHome, "DLLs")) + ";" +
Path.GetFullPath(Path.Combine(pyHome, "Lib")) + ";" +
Path.GetFullPath(Path.Combine(pyHome, "Lib", "site-packages")) 
+ ";" +
binDir
);
Environment.SetEnvironmentVariable("PYTHONVERBOSE", "1");
PythonEngine.Initialize();
PythonEngine.ImportModule("clr");
using (Py.GIL())
{
PythonEngine.RunSimpleString(
"import clr; " +
   "a = clr.AddReference('System'); " +
"print(a.Location);" +
"from System import Environment;" +
"print(Environment.MachineName);");
}
}
}
}
//===

"D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime" is a folder where 
I've copied the DLLs and Lib folder from a python 3.2 x86 distribution.


When I run it from Visual Studio it works fine (I guess it may be related to 
the fact that I'm using python tools for Visual Studio).

But when I run it from the console, it fails with the output:

//===
Traceback (most recent call last):
  File "D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime\Lib\site.py", 
line 481, in execsitecustomize
import sitecustomize
UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: 
invalid character
Traceback (most recent call last):
  File "D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime\Lib\site.py", 
line 497, in execusercustomize
import usercustomize
UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: 
invalid character
C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll
Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: 
invalid character
//===

The "print(a.Location);" works but not the "from System import Environment". 
There are also all these errors about mbcs.

Any idea on what I'm doing wrong?

Thanks,
Serge Weinstock


___
This e-mail may contain confidential and/or privileged information. If you are 
not the intended recipient (or have received this e-mail in error) please 
notify the sender immediately and delete this e-mail. Any unauthorised copying, 
disclosure or distribution of the material in this e-mail is prohibited.

Please refer to http://www.bnpparibas.co.uk/en/email-disclaimer/ for additional 
disclosures.
_
Python.NET mailing list - PythonDotNet@python.org
https://mail.python.org/mailman/listinfo/pythondotnet

Re: [Python.NET] Running an embedded interpreter

[Serge WEINSTOCK]

Hi Tony,

I’ve tried your suggestion but it doesn’t work.

The issue seems to be more “fundamental” as the import of the .Net System 
assembly doesn’t work.

Serge

From: pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org 
[mailto:pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org]
Sent: 19 August 2014 17:16
To: pythondotnet@python.org
Subject: Re: [Python.NET] Running an embedded interpreter

Hi Serge,

'mbcs' is what python uses to mean the current configured encoding. I would 
guess that the encoding of sys.stdout is different when using visual studio 
output console than the console.

You could try a different encoding method by setting the PYTHONIOENCODING 
environment variable before starting your exe, eg:
SET PYTHONIOENCODING=utf-8:ignore

Look for PYTHONIOENCODING here https://docs.python.org/3/using/cmdline.html for 
more details.

Tony



On Tue, Aug 19, 2014 at 2:22 PM, Serge WEINSTOCK 
mailto:serge.weinst...@uk.bnpparibas.com>> 
wrote:
Hi,

I’m trying to use Python3.2 using the Python.Net version found at: 
https://github.com/renshawbay/pythonnet

I’m using the following simple test program:

//===
using System;
using System.IO;
using Python.Runtime;

namespace TestPythonNet
{
class Program
{
static void Main(string[] args)
{
string binDir = 
Path.GetDirectoryName(System.Reflection.Assembly.GetAssembly(typeof(Program)).Location);
string pyHome = 
@"D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime";
PythonEngine.PythonHome = 
@"D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime";
PythonEngine.ProgramName = "PythonRuntime";
Environment.SetEnvironmentVariable("PYTHONPATH",
Path.GetFullPath(Path.Combine(pyHome, "DLLs")) + ";" +
Path.GetFullPath(Path.Combine(pyHome, "Lib")) + ";" +
Path.GetFullPath(Path.Combine(pyHome, "Lib", "site-packages")) 
+ ";" +
binDir
);
Environment.SetEnvironmentVariable("PYTHONVERBOSE", "1");
PythonEngine.Initialize();
PythonEngine.ImportModule("clr");
using (Py.GIL())
{
PythonEngine.RunSimpleString(
"import clr; " +
   "a = clr.AddReference('System'); " +
"print(a.Location);" +
"from System import Environment;" +
"print(Environment.MachineName);");
}
}
}
}
//===

“D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime” is a folder where 
I’ve copied the DLLs and Lib folder from a python 3.2 x86 distribution.


When I run it from Visual Studio it works fine (I guess it may be related to 
the fact that I’m using python tools for Visual Studio).

But when I run it from the console, it fails with the output:

//===
Traceback (most recent call last):
  File "D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime\Lib\site.py", 
line 481, in execsitecustomize
import sitecustomize
UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: 
invalid character
Traceback (most recent call last):
  File "D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime\Lib\site.py", 
line 497, in execusercustomize
import usercustomize
UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: 
invalid character
C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System\v4.0_4.0.0.0__b77a5c561934e089\System.dll
Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'mbcs' codec can't encode characters in position 0--1: 
invalid character
//===

The “print(a.Location);" works but not the “from System import Environment”. 
There are also all these errors about mbcs.

Any idea on what I’m doing wrong?

Thanks,
Serge Weinstock


___
This e-mail may contain confidential and/or privileged information. If you are 
not the intended recipient (or have received this e-mail in error) please 
notify the sender immediately and delete this e-mail. Any unauthorised copying, 
disclosure or distribution of the material in this e-mail is prohibited.

Please refer to http://www.bnpparibas.co.uk/en/email-disclaimer/ for additional 
disclosures.

_
Python.NET mailing list - 
PythonDotNet@python.o

Re: [Python.NET] Running an embedded interpreter

[Serge WEINSTOCK]

Hi Tony,

I’ve noticed that you are the main contributor for this branch of Python for 
.Net. Thanks a lot for that contribution.

Maybe you can help me a little more with my issue. I think the main issue is 
due to the fact that I’m using Python 3.2. 

I’ve done the following tests:
*  Python 3.2 x86:
* calling .Net libraries from standard python interpreter: works fine.
* running embedded interpreter from .Net application:
  * from Visual Studio: works fine. Setting PYTHONHOME is enough. No 
need to set PYTHONPATH
  * from command line: doesn't work. Setting PYTHONPATH improves a 
little things.
*  Python 3.2 x64:
* calling .Net libraries from standard python interpreter: works fine.
* running embedded interpreter from .Net application:
  * from Visual Studio: doesn't work. Setting PYTHONPATH improves a 
little things.
  * from command line: doesn't work. Setting PYTHONPATH improves a 
little things.
*  Python 3.3 x86:
* calling .Net libraries from standard python interpreter: works fine.
* running embedded interpreter from .Net application:
  * from Visual Studio: works fine.
  * from command line: works fine.

I've also compared for the VS run or for the command line run:
 * the paths given by 'sys.modules'.  They are the same.
 * the paths of the loaded dlls as given by 'listdlls'. They are the same.

Maybe you have a clue on why running an embedded interpreter works with 3.3 but 
not 3.3

Thanks,
Serge


From: Serge WEINSTOCK 
Sent: 19 August 2014 17:45
To: 'pythondotnet@python.org'
Subject: RE: [Python.NET] Running an embedded interpreter

Hi Tony,

I’ve tried your suggestion but it doesn’t work. 

The issue seems to be more “fundamental” as the import of the .Net System 
assembly doesn’t work.

Serge

From: pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org 
[mailto:pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org] 
Sent: 19 August 2014 17:16
To: pythondotnet@python.org
Subject: Re: [Python.NET] Running an embedded interpreter

Hi Serge,

'mbcs' is what python uses to mean the current configured encoding. I would 
guess that the encoding of sys.stdout is different when using visual studio 
output console than the console.

You could try a different encoding method by setting the PYTHONIOENCODING 
environment variable before starting your exe, eg:
SET PYTHONIOENCODING=utf-8:ignore

Look for PYTHONIOENCODING here https://docs.python.org/3/using/cmdline.html for 
more details.

Tony



On Tue, Aug 19, 2014 at 2:22 PM, Serge WEINSTOCK 
 wrote:
Hi,
 
I’m trying to use Python3.2 using the Python.Net version found at: 
https://github.com/renshawbay/pythonnet
 
I’m using the following simple test program:
 
//===
using System;
using System.IO;
using Python.Runtime;
 
namespace TestPythonNet
{
    class Program
    {
    static void Main(string[] args)
    {
    string binDir = 
Path.GetDirectoryName(System.Reflection.Assembly.GetAssembly(typeof(Program)).Location);
    string pyHome = 
@"D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime";
    PythonEngine.PythonHome = 
@"D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime";
    PythonEngine.ProgramName = "PythonRuntime";
    Environment.SetEnvironmentVariable("PYTHONPATH",
    Path.GetFullPath(Path.Combine(pyHome, "DLLs")) + ";" +
    Path.GetFullPath(Path.Combine(pyHome, "Lib")) + ";" +
    Path.GetFullPath(Path.Combine(pyHome, "Lib", "site-packages")) 
+ ";" +
    binDir
    );
    Environment.SetEnvironmentVariable("PYTHONVERBOSE", "1");
    PythonEngine.Initialize();
    PythonEngine.ImportModule("clr");
    using (Py.GIL())
    {
    PythonEngine.RunSimpleString(
    "import clr; " +
   "a = clr.AddReference('System'); " +
    "print(a.Location);" +
    "from System import Environment;" +
    "print(Environment.MachineName);");
    }
    }
    }
}
//===
 
“D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime” is a folder where 
I’ve copied the DLLs and Lib folder from a python 3.2 x86 distribution.
 
 
When I run it from Visual Studio it works fine (I guess it may be related to 
the fact that I’m using python tools for Visual Studio).
 
But when I run it from the console, it fails with the output:
 
//===
Traceback (most rece

Re: [Python.NET] Running an embedded interpreter

[Serge WEINSTOCK]

Hi Tony,

I think I've found the issue:
The signatures of Py_SetPythonHome, Py_GetPythonHome, Py_SetProgramName and 
Py_GetProgramName have changed from python 2.x to python 3.x: the strings are 
now Unicode strings.

I've done the following patches (I've also added support for Py_SetPath and 
Py_GetPath as I needed to set up myself sys.path):
* runtime.cs
//==
#if PYTHON32 || PYTHON33 || PYTHON34
[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
 ExactSpelling=true, CharSet=CharSet.Ansi)]
[return: MarshalAs(UnmanagedType.LPWStr)]
internal unsafe static extern string
Py_GetProgramName();

[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
ExactSpelling=true, CharSet=CharSet.Ansi)]
internal unsafe static extern void
Py_SetProgramName([MarshalAsAttribute(UnmanagedType.LPWStr)]string 
name);

[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
ExactSpelling=true, CharSet=CharSet.Ansi)]
[return: MarshalAs(UnmanagedType.LPWStr)]
internal unsafe static extern string
Py_GetPythonHome();

[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
ExactSpelling=true, CharSet=CharSet.Ansi)]
internal unsafe static extern void
Py_SetPythonHome([MarshalAsAttribute(UnmanagedType.LPWStr)]string home);

[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
ExactSpelling=true, CharSet=CharSet.Ansi)]
[return: MarshalAs(UnmanagedType.LPWStr)]
internal unsafe static extern string
Py_GetPath();

[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
ExactSpelling=true, CharSet=CharSet.Ansi)]
internal unsafe static extern void
Py_SetPath([MarshalAsAttribute(UnmanagedType.LPWStr)]string home);
#else
[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
  ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern string
Py_GetProgramName();

[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern void
Py_SetProgramName(string name);

[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern string
Py_GetPythonHome();

[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern void
Py_SetPythonHome(string home);

[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern string
Py_GetPath();

[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern void
Py_SetPath(string home);
#endif
//==
PythonEngine.cs
//==
public static string PythonPath {
get
{
string result = Runtime.Py_GetPath();
if (result == null)
{
return "";
}
return result;
}
set
{
Runtime.Py_SetPath(value);
}
}
//==

Could you add these patches to the source repository?

Thanks,
Serge



From: pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org 
[mailto:pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org] 
Sent: 21 August 2014 14:43
To: pythondotnet@python.org
Subject: Re: [Python.NET] Running an embedded interpreter

Hi Serge,

sorry, not sure why one would work and not the other. For what it's worth, I've 
been using the 3.2 x64 version for some time now, both for calling .NET from 
python and for embedding Python into a .NET application without any problem 
like the ones you describe.

What I suggest you try is grabbing the latest code from the renshawbay repo and 
build that using setupwin.py - you might want to edit that file to build the 
debug project. Then you will be able to step through and see exactly where it's 
going wrong. You can build it from visual studio if you prefer, but you will 
have to be careful to set some of the defines correctly; look at setupwin.py to 
see what needs setting.

You can also download the pdb files and python source from python.org, which 
should allow you step into the python source code without having to build 
python yourself.

Best regards,
Tony


On Thu, Aug 21, 2014 at 10:36 AM, Serge WEINSTOCK 
 wrote:
Hi Tony,

I’ve noticed that you are the main contributor for this branch of

Re: [Python.NET] Running an embedded interpreter

[Serge WEINSTOCK]

Hi Tony,

I’ve submitted the fix. As I’m new to git and github, I’m not sure I’ve done 
the right steps. Let me know if you can’t find my patch

Thanks,
Serge

From: pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org 
[mailto:pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org]
Sent: 26 August 2014 16:54
To: pythondotnet@python.org
Subject: Re: [Python.NET] Running an embedded interpreter

Hi Serge,

ah great, good spot.

Certainly, please send a pull request to the github repo and I'll merge it.

thanks,
Tony

On Tue, Aug 26, 2014 at 1:07 AM, Serge WEINSTOCK 
mailto:serge.weinst...@uk.bnpparibas.com>> 
wrote:
Hi Tony,

I think I've found the issue:
The signatures of Py_SetPythonHome, Py_GetPythonHome, Py_SetProgramName and 
Py_GetProgramName have changed from python 2.x to python 3.x: the strings are 
now Unicode strings.

I've done the following patches (I've also added support for Py_SetPath and 
Py_GetPath as I needed to set up myself sys.path):
* runtime.cs
//==
#if PYTHON32 || PYTHON33 || PYTHON34
[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
 ExactSpelling=true, CharSet=CharSet.Ansi)]
[return: MarshalAs(UnmanagedType.LPWStr)]
internal unsafe static extern string
Py_GetProgramName();

[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
ExactSpelling=true, CharSet=CharSet.Ansi)]
internal unsafe static extern void
Py_SetProgramName([MarshalAsAttribute(UnmanagedType.LPWStr)]string 
name);

[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
ExactSpelling=true, CharSet=CharSet.Ansi)]
[return: MarshalAs(UnmanagedType.LPWStr)]
internal unsafe static extern string
Py_GetPythonHome();

[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
ExactSpelling=true, CharSet=CharSet.Ansi)]
internal unsafe static extern void
Py_SetPythonHome([MarshalAsAttribute(UnmanagedType.LPWStr)]string home);

[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
ExactSpelling=true, CharSet=CharSet.Ansi)]
[return: MarshalAs(UnmanagedType.LPWStr)]
internal unsafe static extern string
Py_GetPath();

[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
ExactSpelling=true, CharSet=CharSet.Ansi)]
internal unsafe static extern void
Py_SetPath([MarshalAsAttribute(UnmanagedType.LPWStr)]string home);
#else
[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
  ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern string
Py_GetProgramName();

[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern void
Py_SetProgramName(string name);

[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern string
Py_GetPythonHome();

[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern void
Py_SetPythonHome(string home);

[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern string
Py_GetPath();

[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern void
Py_SetPath(string home);
#endif
//==
PythonEngine.cs
//==
public static string PythonPath {
get
{
string result = Runtime.Py_GetPath();
if (result == null)
{
return "";
}
return result;
}
set
{
Runtime.Py_SetPath(value);
}
}
//==

Could you add these patches to the source repository?

Thanks,
Serge



From: 
pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org<mailto:uk.bnpparibas@python.org>
 
[mailto:pythondotnet-bounces+serge.weinstock<mailto:pythondotnet-bounces%2Bserge.weinstock>=uk.bnpparibas@python.org<mailto:uk.bnpparibas@python.org>]
Sent: 21 August 2014 14:43
To: pythondotnet@python.org<mailto:pythondotnet@python.org>
Subject: Re: [Python.NET] Running an embedded interpreter

Hi Serge,

sorry, not sure why one would work and not the other. For what it's worth, I've 
been using the 3.2 x64 version for some time now, both for calling .NET from 
python and for embedding

Re: [Python.NET] Running an embedded interpreter

[Serge WEINSTOCK]

It took me sometime to figure that I have to first fork, create a branch, apply 
the patch, commit and create a pull request but I think I have now understood 
how it works ;-)

So, I’ve created the pull request. Could you have a look at it?

Thanks,
Serge

From: pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org 
[mailto:pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org]
Sent: 27 August 2014 17:34
To: pythondotnet@python.org
Subject: Re: [Python.NET] Running an embedded interpreter

Hi Serge,

I don't see your pull request here 
https://github.com/renshawbay/pythonnet/pulls, and I can't see your fork. Could 
you send me the url of your fork and I'll take a look if I can pull the changes 
from there?

If you want to have another go at creating the pull request this might help:



On Wed, Aug 27, 2014 at 2:02 PM, Serge WEINSTOCK 
mailto:serge.weinst...@uk.bnpparibas.com>> 
wrote:
Hi Tony,

I’ve submitted the fix. As I’m new to git and github, I’m not sure I’ve done 
the right steps. Let me know if you can’t find my patch

Thanks,
Serge

From: 
pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org<mailto:uk.bnpparibas@python.org>
 
[mailto:pythondotnet-bounces+serge.weinstock<mailto:pythondotnet-bounces%2Bserge.weinstock>=uk.bnpparibas@python.org<mailto:uk.bnpparibas@python.org>]
Sent: 26 August 2014 16:54

To: pythondotnet@python.org<mailto:pythondotnet@python.org>
Subject: Re: [Python.NET] Running an embedded interpreter

Hi Serge,

ah great, good spot.

Certainly, please send a pull request to the github repo and I'll merge it.

thanks,
Tony

On Tue, Aug 26, 2014 at 1:07 AM, Serge WEINSTOCK 
mailto:serge.weinst...@uk.bnpparibas.com>> 
wrote:
Hi Tony,

I think I've found the issue:
The signatures of Py_SetPythonHome, Py_GetPythonHome, Py_SetProgramName and 
Py_GetProgramName have changed from python 2.x to python 3.x: the strings are 
now Unicode strings.

I've done the following patches (I've also added support for Py_SetPath and 
Py_GetPath as I needed to set up myself sys.path):
* runtime.cs
//==
#if PYTHON32 || PYTHON33 || PYTHON34
[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
 ExactSpelling=true, CharSet=CharSet.Ansi)]
[return: MarshalAs(UnmanagedType.LPWStr)]
internal unsafe static extern string
Py_GetProgramName();

[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
ExactSpelling=true, CharSet=CharSet.Ansi)]
internal unsafe static extern void
Py_SetProgramName([MarshalAsAttribute(UnmanagedType.LPWStr)]string 
name);

[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
ExactSpelling=true, CharSet=CharSet.Ansi)]
[return: MarshalAs(UnmanagedType.LPWStr)]
internal unsafe static extern string
Py_GetPythonHome();

[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
ExactSpelling=true, CharSet=CharSet.Ansi)]
internal unsafe static extern void
Py_SetPythonHome([MarshalAsAttribute(UnmanagedType.LPWStr)]string home);

[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
ExactSpelling=true, CharSet=CharSet.Ansi)]
[return: MarshalAs(UnmanagedType.LPWStr)]
internal unsafe static extern string
Py_GetPath();

[DllImport(Runtime.dll, CallingConvention=CallingConvention.Cdecl,
ExactSpelling=true, CharSet=CharSet.Ansi)]
internal unsafe static extern void
Py_SetPath([MarshalAsAttribute(UnmanagedType.LPWStr)]string home);
#else
[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
  ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern string
Py_GetProgramName();

[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern void
Py_SetProgramName(string name);

[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern string
Py_GetPythonHome();

[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern void
Py_SetPythonHome(string home);

[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern string
Py_GetPath();

[DllImport(Runtime.dll, CallingConvention = CallingConvention.Cdecl,
ExactSpelling = true, CharSet = CharSet.Ansi)]
internal unsafe static extern void
Py_SetPath(string home);
#endif
//===

[Python.NET] Python.Net for Python3: bug when converting a PyLong to a System.Int32

[Serge WEINSTOCK]

Hi Tony,

I've found a bug when trying to use Python integer as .Net int32. For example:
//===
using System;
using Python.Runtime;

namespace TestPythonNet
{
public class ATest
{
public long a64;
public int a32;
public void f32(int a) { Console.WriteLine("got int {0}", a);}
public void f64(long a) { Console.WriteLine("got long {0}", a); }
}


class Program
{
static void Main(string[] args)
{
PythonEngine.PythonHome = 
@"D:\src\scratch\TestPythonNet\TestPythonNet\PythonRuntime";
PythonEngine.ProgramName = "PythonRuntime";
PythonEngine.Initialize();
PythonEngine.ImportModule("clr");
using (Py.GIL())
{
try
{
PythonEngine.RunSimpleString(
@"
import clr

clr.AddReference('TestPythonNet')
from TestPythonNet import ATest

f = ATest();
f.a64 = 10 # OK
f.a32 = 10 # Fails!
f.f64(-1) # OK
f.f32(-1) # Fails!

print('all fine!')
");

}
catch (PythonException e)
{
Console.WriteLine(e);
}
}
}
}
}
//===
All the call from python using integer as .Net int will fail. This is due to 
the fact that python3 has only one type of integer: int64.

Furthermore, in the Python C API, all the conversion functions from or to int32 
have disappeared.

I think that the following patch fixes the issue:
Src/runtime/converter.cs
//===
static bool ToPrimitive(IntPtr value, Type obType, out Object result,
bool setError) {

IntPtr overflow = Exceptions.OverflowError;
TypeCode tc = Type.GetTypeCode(obType);
result = null;
   IntPtr op;
int ival;

switch(tc) {

case TypeCode.String:
string st = Runtime.GetManagedString(value);
if (st == null) {
goto type_error;
}
result = st;
return true;

case TypeCode.Int32:
// Trickery to support 64-bit platforms.
if (IntPtr.Size == 4) {
//-- FIX START
// convert it to an python int64
op = Runtime.PyNumber_Long(value);

// if the conversion fails
if (op == IntPtr.Zero) {
if (Exceptions.ExceptionMatches(overflow)) {
goto overflow;
}
  goto type_error;
}

// convert it to an int64
long lval = (long)Runtime.PyLong_AsLongLong(op);

// and try to convert it to an int32
if ((lval < int.MinValue) || (int.MaxValue < lval))
{
Runtime.Decref(op);
if (Exceptions.ExceptionMatches(overflow))
{
goto overflow;
}
goto type_error;
}
Runtime.Decref(op);
result = unchecked((int)lval);
return true;
//-- FIX END
}
else {
op = Runtime.PyNumber_Long(value)
//===

I will send a pull request to the github repo . Could you have a look at it?

Thanks,
Serge


___
This e-mail may contain confidential and/or privileged information. If you are 
not the intended recipient (or have received this e-mail in error) please 
notify the sender immediately and delete this e-mail. Any unauthorised copying, 
disclosure or distribution of the material in this e-mail is prohibited.

Please refer to http://www.bnpparibas.co.uk/en/email-disclaimer/ for additional 
disclosures.
_
Python.NET mailing list - PythonDotNet@python.org
https://mail.python.org/mailman/listinfo/pythondotnet

Re: [Python.NET] Python.Net for Python3: bug when converting a PyLong

[Serge WEINSTOCK]

Hi,

I haven’t. I’m a little busy with work now. But I will do it today or tomorrow.

I’ve found another issue when using an embedded Python interpreter. I will 
explain it in another thread

Serge

From: pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org 
[mailto:pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org]
Sent: 30 September 2014 00:44
To: pythondotnet@python.org
Subject: Re: [Python.NET] Python.Net for Python3: bug when converting a PyLong

Hi Serge,

Did you file a pull request for this one?

I'm interested in the fix too...

___
This e-mail may contain confidential and/or privileged information. If you are 
not the intended recipient (or have received this e-mail in error) please 
notify the sender immediately and delete this e-mail. Any unauthorised copying, 
disclosure or distribution of the material in this e-mail is prohibited.

Please refer to http://www.bnpparibas.co.uk/en/email-disclaimer/ for additional 
disclosures.
_
Python.NET mailing list - PythonDotNet@python.org
https://mail.python.org/mailman/listinfo/pythondotnet

[Python.NET] Python.Net for Python3: bug when using an embedded interpreter

[Serge WEINSTOCK]

Hi Tony,

You recently made a change which prevents me using the library in an embedded 
Python interpreter.

In pythonengine.cs, you "inject" decorators with the following code:

//=
public static void Initialize() {
.
IntPtr globals = Runtime.PyEval_GetGlobals();
PyDict locals = new PyDict();
try
{
IntPtr builtins = Runtime.PyEval_GetBuiltins();
Runtime.PyDict_SetItemString(locals.Handle, "__builtins__", builtins);

var assembly = Assembly.GetExecutingAssembly();
using (Stream stream = 
assembly.GetManifestResourceStream("Python.Runtime.resources.clr.py"))
using (StreamReader reader = new StreamReader(stream))
{
// add the contents of clr.py to the module
string clr_py = reader.ReadToEnd();
PyObject result = RunString(clr_py, globals, locals.Handle);
if (null == result)
throw new PythonException();
result.Dispose();
}
//=
When running from a Python script, it's fine because the Python interpreter has 
already been initialized and global variables have been defined.

The issue is that when running from an embedded interpreter. As I must first 
initialize the Python engine using PythonEngine.Initialize().

So when we reach this piece of code, there is no script running, no "Python 
context", so "PyEval_GetGlobals" returns null and "RunString" fails.

We could create an artificial global dictionary but it won't be available to 
the rest of the code.

I think it should be best to move this code into a Python module. If run from a 
script (PyEval_GetGlobals() != null), you could inject the decorators with a 
simple "from decorators_pythonnet import *". If run from an embedded 
interpreter, we could add a line for running the same code once the engine has 
been initialized.

What do you think?

Serge Weinstock


___
This e-mail may contain confidential and/or privileged information. If you are 
not the intended recipient (or have received this e-mail in error) please 
notify the sender immediately and delete this e-mail. Any unauthorised copying, 
disclosure or distribution of the material in this e-mail is prohibited.

Please refer to http://www.bnpparibas.co.uk/en/email-disclaimer/ for additional 
disclosures.
_
Python.NET mailing list - PythonDotNet@python.org
https://mail.python.org/mailman/listinfo/pythondotnet

Re: [Python.NET] Python.Net for Python3: bug when using an embedded interpreter

[Serge WEINSTOCK]

Thanks, it’s working fine.

I’ve created a pull request for handling Python3 int conversion to int32 in 32 
bits python interpreters. Could you have a look at it?

Serge

From: pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org 
[mailto:pythondotnet-bounces+serge.weinstock=uk.bnpparibas@python.org]
Sent: 30 September 2014 19:16
To: pythondotnet@python.org
Subject: Re: [Python.NET] Python.Net for Python3: bug when using an embedded 
interpreter

Should be fixed now. Please re-open if it's still a problem 
(https://github.com/renshawbay/pythonnet/issues/13).

cheers,
Tony

On Tue, Sep 30, 2014 at 4:57 PM, Tony Roberts 
mailto:t...@pyxll.com>> wrote:
Hi Serge,

I see what you're saying. It would think that bit just needs to be moved until 
after we're sure python has been initialized so it works in both cases. I don't 
see any need to move it into a separate python module; having it compiled into 
one is cleaner IMHO. If you're not subclassing .NET classes in python and so 
don't need python methods/properties reflected to .net you won't need it 
anyway, so you could just remove that bit of code from your fork for now.

I'll take a look, but if you get round to it before I do then please feel free 
to submit a pull request. I'll create an issue in github so it doesn't get 
forgotten.

thanks,
Tony


On Tue, Sep 30, 2014 at 11:59 AM, Serge WEINSTOCK 
mailto:serge.weinst...@uk.bnpparibas.com>> 
wrote:
Hi Tony,

You recently made a change which prevents me using the library in an embedded 
Python interpreter.

In pythonengine.cs, you “inject” decorators with the following code:

//=
public static void Initialize() {
.
IntPtr globals = Runtime.PyEval_GetGlobals();
PyDict locals = new PyDict();
try
{
IntPtr builtins = Runtime.PyEval_GetBuiltins();
Runtime.PyDict_SetItemString(locals.Handle, "__builtins__", builtins);

var assembly = Assembly.GetExecutingAssembly();
using (Stream stream = 
assembly.GetManifestResourceStream("Python.Runtime.resources.clr.py<http://Python.Runtime.resources.clr.py>"))
using (StreamReader reader = new StreamReader(stream))
{
// add the contents of clr.py to the module
string clr_py = reader.ReadToEnd();
PyObject result = RunString(clr_py, globals, locals.Handle);
if (null == result)
throw new PythonException();
result.Dispose();
}
//=
When running from a Python script, it’s fine because the Python interpreter has 
already been initialized and global variables have been defined.

The issue is that when running from an embedded interpreter. As I must first 
initialize the Python engine using PythonEngine.Initialize().

So when we reach this piece of code, there is no script running, no “Python 
context”, so “PyEval_GetGlobals” returns null and “RunString” fails.

We could create an artificial global dictionary but it won’t be available to 
the rest of the code.

I think it should be best to move this code into a Python module. If run from a 
script (PyEval_GetGlobals() != null), you could inject the decorators with a 
simple “from decorators_pythonnet import *”. If run from an embedded 
interpreter, we could add a line for running the same code once the engine has 
been initialized.

What do you think?

Serge Weinstock


___
This e-mail may contain confidential and/or privileged information. If you are 
not the intended recipient (or have received this e-mail in error) please 
notify the sender immediately and delete this e-mail. Any unauthorised copying, 
disclosure or distribution of the material in this e-mail is prohibited.

Please refer to http://www.bnpparibas.co.uk/en/email-disclaimer/ for additional 
disclosures.

_
Python.NET mailing list - 
PythonDotNet@python.org<mailto:PythonDotNet@python.org>
https://mail.python.org/mailman/listinfo/pythondotnet



___
This e-mail may contain confidential and/or privileged information. If you are 
not the intended recipient (or have received this e-mail in error) please 
notify the sender immediately and delete this e-mail. Any unauthorised copying, 
disclosure or distribution of the material in this e-mail is prohibited.

Please refer to http://www.bnpparibas.co.uk/en/email-disclaimer/ for additional 
disclosures.
_
Python.NET mailing list - PythonDotNet@python.org
https://mail.python.org/mailman/listinfo/pythondotnet