Re: Easiest way to access C module in Python

[Gisle Vanem]

bartc wrote:


 From inside python 2.7:
   Python 2.7.13rc1 (v2.7.13rc1:4d6fd49eeb14, Dec  3 2016, 21:49:42) [MSC 
v.1500 32 bit (Intel)] on win32
   Type "help", "copyright", "credits" or "license" for more information.
   >>> import life
   >>> dir(life)
   ['__builtins__', '__doc__', '__file__', '__name__', '__package__', 
'__test__']
   >>>

works fine.


Not really, as it's missing the attribute 'life' which is needed for the second 
part of 'life.life':


Not sure what happend there. All is working now. But as
you stated, Cython is truly a "beast".

But as I wrote to Lele privately, the main reason for the failure,
was Cygwin's Python2.7 messing up. I invoked the build from a GNU-Makefile.
Thus Cygwin's Python was picked up from /bin/bash instead of my regular
Python on PATH. A real PITA.

--
--gv
--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[bartc]

On 07/11/2017 20:08, Gisle Vanem wrote:

Lele Gaifax wrote:


On my PC, I get the following, using the "-v" option to verbosely see the
imported modules:

$ $ python -v
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
...

import life

dlopen("./life.so", 2);
import life # dynamically loaded from life.so

dir(life)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', 
'__test__', 'life']


Can you try to import the "life" module and print its "dir()" to see the
symbols it exposes?


 From inside python 2.7:
   Python 2.7.13rc1 (v2.7.13rc1:4d6fd49eeb14, Dec  3 2016, 21:49:42) 
[MSC v.1500 32 bit (Intel)] on win32

   Type "help", "copyright", "credits" or "license" for more information.
   >>> import life
   >>> dir(life)
   ['__builtins__', '__doc__', '__file__', '__name__', '__package__', 
'__test__']

   >>>

works fine.


Not really, as it's missing the attribute 'life' which is needed for the 
second part of 'life.life':


 But as I wrote, doing a:

   python -vc "import life; print(life.life())"


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[Gisle Vanem]

Lele Gaifax wrote:


On my PC, I get the following, using the "-v" option to verbosely see the
imported modules:

$ $ python -v
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
...

import life

dlopen("./life.so", 2);
import life # dynamically loaded from life.so

dir(life)

['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__test__', 
'life']

Can you try to import the "life" module and print its "dir()" to see the
symbols it exposes?


From inside python 2.7:
  Python 2.7.13rc1 (v2.7.13rc1:4d6fd49eeb14, Dec  3 2016, 21:49:42) [MSC v.1500 
32 bit (Intel)] on win32
  Type "help", "copyright", "credits" or "license" for more information.
  >>> import life
  >>> dir(life)
  ['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__test__']
  >>>

works fine. But as I wrote, doing a:
  python -vc "import life; print(life.life())"

from the cmd-line doesn't work:
  ...
  import life # dynamically loaded from life.pyd
  Traceback (most recent call last):
File "", line 1, in 
  AttributeError: 'module' object has no attribute 'life'
  ...

Inspection life.pyd shows no problems. It do export a "initlife"
function.

--
--gv
--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[Lele Gaifax]

Gisle Vanem  writes:

> python.exe -c "import life; print(life.life())"
> Traceback (most recent call last):
>   File "", line 1, in 
> AttributeError: 'module' object has no attribute 'life'
>
> Can you give a hint?

I tried with Python 2, and the same recipe works for me, on GNU/Linux:

$ python -c "import life; print life.life()"
42

I'm sorry, but I have no opportunity to try on a M$Windows system.

Anyway, is your interpreter able to load the extension module produced by the
"setup.py build_ext" step?

On my PC, I get the following, using the "-v" option to verbosely see the
imported modules:

$ $ python -v
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
...
>>> import life
dlopen("./life.so", 2);
import life # dynamically loaded from life.so
>>> dir(life)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__test__', 
'life']

Can you try to import the "life" module and print its "dir()" to see the
symbols it exposes?

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
l...@metapensiero.it  | -- Fortunato Depero, 1929.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[Gisle Vanem]

Lele Gaifax wrote:


$ python setup.py build_ext --inplace
Compiling life.pyx because it changed.
[1/1] Cythonizing life.pyx
running build_ext
building 'life' extension
creating build
creating build/temp.linux-x86_64-3.6
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes 
-g -fdebug-prefix-map=/build/python3.6-5reRaQ/python3.6-3.6.3=. 
-specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat 
-Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/tmp/ct/include 
-I/usr/include/python3.6m -c life.c -o build/temp.linux-x86_64-3.6/life.o
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes 
-g -fdebug-prefix-map=/build/python3.6-5reRaQ/python3.6-3.6.3=. 
-specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat 
-Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/tmp/ct/include 
-I/usr/include/python3.6m -c fred.c -o build/temp.linux-x86_64-3.6/fred.o
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions 
-specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro 
-specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro -g 
-fdebug-prefix-map=/build/python3.6-5reRaQ/python3.6-3.6.3=. 
-specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat 
-Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 
build/temp.linux-x86_64-3.6/life.o build/temp.linux-x86_64-3.6/fred.o -o 
/tmp/ct/life.cpython-36m-x86_64-linux-gnu.so
$ python -c "import life; print(life.life())"
42


I tried your example on Python 2.7 (Win-10 / MSVC), but failed:

python.exe setup.py build_ext --inplace
running build_ext
building 'life' extension
creating build
creating build\temp.win32-2.7
creating build\temp.win32-2.7\Release
f:\gv\VC_2017\VC\Tools\MSVC\14.11.25503\bin\HostX86\x86\cl.exe /c /nologo /Ox 
/MD /W3 /GS- /DNDEBUG
-If:\ProgramFiler\Python27\include -If:\ProgramFiler\Python27\PC /Tclife.c 
/Fobuild\temp.win32-2.7\Release\life.obj
life.c
f:\gv\VC_2017\VC\Tools\MSVC\14.11.25503\bin\HostX86\x86\cl.exe /c /nologo /Ox 
/MD /W3 /GS- /DNDEBUG
-If:\ProgramFiler\Python27\include -If:\ProgramFiler\Python27\PC /Tcfred.c 
/Fobuild\temp.win32-2.7\Release\fred.obj
fred.c
f:\gv\VC_2017\VC\Tools\MSVC\14.11.25503\bin\HostX86\x86\link.exe /DLL /nologo 
/INCREMENTAL:NO
/LIBPATH:f:\ProgramFiler\Python27\libs /LIBPATH:f:\ProgramFiler\Python27\PCbuild
/EXPORT:initlife build\temp.win32-2.7\Release\life.obj 
build\temp.win32-2.7\Release\fred.obj
/OUT:F:\ProgramFiler\Python27\test\Cython-test2\life.pyd 
/IMPLIB:build\temp.win32-2.7\Release\life.lib
/MANIFESTFILE:build\temp.win32-2.7\Release\life.pyd.manifest
   Creating library build\temp.win32-2.7\Release\life.lib and object 
build\temp.win32-2.7\Release\life.exp

python.exe -c "import life; print(life.life())"
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'module' object has no attribute 'life'


Can you give a hint?

--
--gv
--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[Rob Gaddi]

On 11/06/2017 05:59 PM, Grant Edwards wrote:

On 2017-11-06, John Pote  wrote:


I have successfully used Python to perform unit and integration tests in
the past and I'd like to do the same for some C modules I'm working with
at work. There seem to be a number of ways of doing this but being busy
at work and home I looking for the approach with the least learning curve.


When I want to test C modules (usually destined for an embedded
system) using a Python framework, I use ctypes.

https://docs.python.org/3/library/ctypes.html



I'll second ctypes.  It's pretty straightforward and I've made it do 
some very heavy lifting over the years.  Cython is definitely more work. 
 SWIG makes sense if you've got a huge number of interfaces you're trying
to map and need to automate the process, but if you've got south of 20 
I'd just do it natively with ctypes.


--
Rob Gaddi, Highland Technology -- www.highlandtechnology.com
Email address domain is currently out of order.  See above to fix.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[bartc]

On 07/11/2017 14:33, Tim Golden wrote:

On 07/11/2017 14:20, bartc wrote:


You've lost me. I had to look up pyPI and it's something to do with a 
Package Index. But I don't know how that relates to installing Cython.


Can I just step in now with my Moderator hat on and ask: please avoid a 
lengthy "educate Bart C about Python" thread. If anyone wants to take it 
up with him in private, that's up to you. But please leave it off the 
list / newsgroup.


Well, that's not going to work because my email is not valid. But I've 
already said I'm not interested, and this is anyway getting a long way 
from what the OP is trying to do. I've stated that ctypes is my 
preferred approach for that.


However I doubt I'm the only one having trouble with these peripheral 
aspects, and a mention of such problems can help others, even if it's 
for reassurance that the problems are not their fault!


(This of course doesn't apply to the expert contributors here.)

--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[Tim Golden]

On 07/11/2017 14:20, bartc wrote:

On 07/11/2017 13:30, Thomas Jollans wrote:

On 2017-11-07 12:53, bartc wrote:

Having
said that, I located pip.exe, trying typing 'pip install cffi' and it
seemed to be doing something but then failed with a bunch of errors.)


So you're missing out on all of PyPI? That's tragic. You should really
try to fix that. I'm sure people on this list will be happy to help if
you start a new thread with some details of what is happening on your
system.


You've lost me. I had to look up pyPI and it's something to do with a 
Package Index. But I don't know how that relates to installing Cython.


Can I just step in now with my Moderator hat on and ask: please avoid a 
lengthy "educate Bart C about Python" thread. If anyone wants to take it 
up with him in private, that's up to you. But please leave it off the 
list / newsgroup.


Thanks

TJG
--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[bartc]

On 07/11/2017 13:30, Thomas Jollans wrote:

On 2017-11-07 12:53, bartc wrote:

Having
said that, I located pip.exe, trying typing 'pip install cffi' and it
seemed to be doing something but then failed with a bunch of errors.)


So you're missing out on all of PyPI? That's tragic. You should really
try to fix that. I'm sure people on this list will be happy to help if
you start a new thread with some details of what is happening on your
system.


You've lost me. I had to look up pyPI and it's something to do with a 
Package Index. But I don't know how that relates to installing Cython.


The problem with pip was in Py3.4. I also have Py3.6, and that didn't 
have pip at all. Or it is something that itself needs to be installed 
first? Or is pyPI a new thing that replaces it?


I'm not a serious Python user so it's not worth the effort of going 
around in circles trying to get these things working. I remember how 
difficult it was to get Numpy going a couple of years back.


If it's already built-in, then fine, but if not, forget it. But for me 
it doesn't matter.


--
bartc



--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[Thomas Jollans]

On 2017-11-07 12:53, bartc wrote:
> Having
> said that, I located pip.exe, trying typing 'pip install cffi' and it
> seemed to be doing something but then failed with a bunch of errors.)

So you're missing out on all of PyPI? That's tragic. You should really
try to fix that. I'm sure people on this list will be happy to help if
you start a new thread with some details of what is happening on your
system.


-- 
Thomas Jollans
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[Lele Gaifax]

bartc  writes:

> OK, compiling fred.c. Is there a dependency on gcc too? This looks more like
> makefile hell.

That's pretty standard distutils functionality. I'm pretty sure that on
M$Windows it would invoke its C compiler, not gcc.

I wrote "fred.c" to get closer to the case you mentioned, but obviously that
function can very well come from a library.

> People use languages like Python to get away from this stuff.

Which people? The OP explicitly asked for a way to access a C module from
Python, and Cython surely is one option to accomplish that.

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
l...@metapensiero.it  | -- Fortunato Depero, 1929.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[bartc]

On 07/11/2017 13:11, bartc wrote:


$ python setup.py build_ext --inplace


OK, thanks. Although when I get to this bit, my system still says:

17.297
Traceback (most recent call last):
   File "setup.py", line 1, in 
     from distutils.core import setup
  


Update: if I copy the relevant files to the actual Python directory (so 
running Python as 'python' rather than '\python34\python'), the problem 
seems to be simply:


 ImportError: No module named 'Cython'

(And trying script\pip install cython generated some promising disk 
activity for about a minute but then loads of errors, although no 
definitive message about whether it installed Cython or not. But since 
it still can't import the Cython module, I guess it didn't.)


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[bartc]

On 07/11/2017 12:14, Lele Gaifax wrote:

bartc  writes:


But just staying with the "function with no arguments" for the minute (the
equivalent of Hello World for this exercise), how would it be done in
Cython? Would a working example be simple enough to show in a usenet post?


fred.c::

int fred(void) {
  return 42;
}

life.pyx::

cdef extern:
int fred()

def life():
return fred()

setup.py::

from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize

setup(
ext_modules = cythonize([Extension("life", ["life.pyx", "fred.c"])])
)

$ python setup.py build_ext --inplace


OK, thanks. Although when I get to this bit, my system still says:

17.297
Traceback (most recent call last):
  File "setup.py", line 1, in 
from distutils.core import setup
 

So obviously something is wrong with it, but I'll have to assume it 
normally works.



Compiling life.pyx because it changed.
[1/1] Cythonizing life.pyx
running build_ext
building 'life' extension
creating build
creating build/temp.linux-x86_64-3.6
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -g -fdebug-prefix-map=/build/python3.6-5reRaQ/python3.6-3.6.3=. -specs=/usr/share/dpkg/no-pie-compile.specs 
-fstack-protector-strong -Wformat -Werror=format-security -Wdate-time 
-D_FORTIFY_SOURCE=2 -fPIC -I/tmp/ct/include -I/usr/include/python3.6m -c 
life.c -o build/temp.linux-x86_64-3.6/life.o


However, it doesn't look that simple a process, and it seems to solve a 
different problem from the ctypes solution. That one took an EXISTING 
function, already compiled and linked into a binary, and does not need 
the C source nor need to compile it. Often, you will only have the 
binary shared library anyway.


-I/tmp/ct/include -I/usr/include/python3.6m -c fred.c -o 
build/temp.linux-x86_64-3.6/fred.o


OK, compiling fred.c. Is there a dependency on gcc too? This looks more 
like makefile hell. People use languages like Python to get away from 
this stuff.



x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions 
-specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro 
-specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro -g 
-fdebug-prefix-map=/build/python3.6-5reRaQ/python3.6-3.6.3=. 
-specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat 
-Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 
build/temp.linux-x86_64-3.6/life.o build/temp.linux-x86_64-3.6/fred.o -o 
/tmp/ct/life.cpython-36m-x86_64-linux-gnu.so
$ python -c "import life; print(life.life())"
42

As other said, for a single function accepting no arguments and returning a
single value Cython may be an heavy tool, but I bet you can imagine more
complex situations...


In my normal work, I'm calling C functions from interpreted code all the 
time, just not in Python (example below sig).


It's not that complicated. With this Cython solution you have the .c 
(which has to be compiled and linked using this process), you have .pyx, 
whatever that is, you have .py which is not really Python, but has to be 
processed as Cython, and then you have .py which is actual Python, which 
is the code that wanted to call that C function in the first. So simple.


I understand that Cython lets you write Python-like code with special 
annotations that allows it to be compiled to efficient native code (or 
something like that), but that's not what this task is, which is 100% 
Python calling 100% C, with both developed using their normal tools.


--
bartc

# in interpreted code, run as normal:

importdll jpeg =
clang function loadjpeg(string, ref int64, ref int64)ref byte
end

In C:

byte* loadjpeg(char* file, int64* width, int64* height) {...

--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[Lele Gaifax]

bartc  writes:

> But just staying with the "function with no arguments" for the minute (the
> equivalent of Hello World for this exercise), how would it be done in
> Cython? Would a working example be simple enough to show in a usenet post?

fred.c::

   int fred(void) {
 return 42;
   }

life.pyx::

   cdef extern:
   int fred()

   def life():
   return fred()

setup.py::

   from distutils.core import setup
   from distutils.extension import Extension
   from Cython.Build import cythonize

   setup(
   ext_modules = cythonize([Extension("life", ["life.pyx", "fred.c"])])
   )

$ python setup.py build_ext --inplace
Compiling life.pyx because it changed.
[1/1] Cythonizing life.pyx
running build_ext
building 'life' extension
creating build
creating build/temp.linux-x86_64-3.6
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes 
-g -fdebug-prefix-map=/build/python3.6-5reRaQ/python3.6-3.6.3=. 
-specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat 
-Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/tmp/ct/include 
-I/usr/include/python3.6m -c life.c -o build/temp.linux-x86_64-3.6/life.o
x86_64-linux-gnu-gcc -pthread -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes 
-g -fdebug-prefix-map=/build/python3.6-5reRaQ/python3.6-3.6.3=. 
-specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat 
-Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -fPIC -I/tmp/ct/include 
-I/usr/include/python3.6m -c fred.c -o build/temp.linux-x86_64-3.6/fred.o
x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions 
-specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro 
-specs=/usr/share/dpkg/no-pie-link.specs -Wl,-z,relro -g 
-fdebug-prefix-map=/build/python3.6-5reRaQ/python3.6-3.6.3=. 
-specs=/usr/share/dpkg/no-pie-compile.specs -fstack-protector-strong -Wformat 
-Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 
build/temp.linux-x86_64-3.6/life.o build/temp.linux-x86_64-3.6/fred.o -o 
/tmp/ct/life.cpython-36m-x86_64-linux-gnu.so
$ python -c "import life; print(life.life())"
42

As other said, for a single function accepting no arguments and returning a
single value Cython may be an heavy tool, but I bet you can imagine more
complex situations...

ciao, lele.
-- 
nickname: Lele Gaifax | Quando vivrò di quello che ho pensato ieri
real: Emanuele Gaifas | comincerò ad aver paura di chi mi copia.
l...@metapensiero.it  | -- Fortunato Depero, 1929.

-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[bartc]

On 07/11/2017 11:35, Paul Moore wrote:

On 7 November 2017 at 11:16, Chris Angelico  wrote:

Thanks for the FUD. I love it when someone, on the basis of one failed
experiment, trash-talks an excellent piece of software that would
solve the OP's problem.


It *is* true that the learning curve for Cython is steeper than that
of ctypes. But for anything more complex than simple "call a function
with no arguments", ctypes rapidly gets fairly complex and messy - and
the docs are not exactly the best. At that point, investing the time
in learning how to use Cython definitely pays off.


But just staying with the "function with no arguments" for the minute 
(the equivalent of Hello World for this exercise), how would it be done 
in Cython? Would a working example be simple enough to show in a usenet 
post?



Another option for the OP is cffi,


And perhaps another example here.

(cffi doesn't seem to be part of any of my pythons, so that would be an 
obstacle for me as installing extra stuff rarely seems to work. Having 
said that, I located pip.exe, trying typing 'pip install cffi' and it 
seemed to be doing something but then failed with a bunch of errors.)


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[Paul Moore]

On 7 November 2017 at 11:16, Chris Angelico  wrote:
> Thanks for the FUD. I love it when someone, on the basis of one failed
> experiment, trash-talks an excellent piece of software that would
> solve the OP's problem.

It *is* true that the learning curve for Cython is steeper than that
of ctypes. But for anything more complex than simple "call a function
with no arguments", ctypes rapidly gets fairly complex and messy - and
the docs are not exactly the best. At that point, investing the time
in learning how to use Cython definitely pays off.

Another option for the OP is cffi, which might offer a middle ground
(in terms of complexity vs power - it's hard to objectively assess
"complexity" without knowing the audience's background).

Paul
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[bartc]

On 07/11/2017 11:16, Chris Angelico wrote:

On Tue, Nov 7, 2017 at 10:06 PM, bartc  wrote:



My experience is different.


Thanks for the FUD. I love it when someone, on the basis of one failed
experiment, trash-talks an excellent piece of software that would
solve the OP's problem.


OK, I gave a working example that called an actual C function.

Perhaps you can give one based on Cython that does the same thing.
--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[Chris Angelico]

On Tue, Nov 7, 2017 at 10:06 PM, bartc  wrote:
> On 07/11/2017 02:23, Chris Angelico wrote:
>>
>> On Tue, Nov 7, 2017 at 12:52 PM, bartc  wrote:
>
>
>>> Cython seems very confusing to me.
>>
>>
>
>>
>>> Otherwise what /I/ would look for is ways to call C functions inside
>>> shared
>>> libraries (.dll and .so). That requires that the modules under test be
>>> wrapped as a shared library, which may be extra effort (but it will still
>>> be
>>> using C, so with familiar tools and no crossover with Python at this
>>> point).
>>>
>>> To call shared library C functions from Python I think involves the
>>> ctypes
>>> module (I've never done it). Googling 'ctypes shared library' gives some
>>> promising results.
>>
>>
>> The point of Cython is to make this easier. It's worth learning.
>
>
> My experience is different.

Thanks for the FUD. I love it when someone, on the basis of one failed
experiment, trash-talks an excellent piece of software that would
solve the OP's problem.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[bartc]

On 07/11/2017 02:23, Chris Angelico wrote:

On Tue, Nov 7, 2017 at 12:52 PM, bartc  wrote:



Cython seems very confusing to me.







Otherwise what /I/ would look for is ways to call C functions inside shared
libraries (.dll and .so). That requires that the modules under test be
wrapped as a shared library, which may be extra effort (but it will still be
using C, so with familiar tools and no crossover with Python at this point).

To call shared library C functions from Python I think involves the ctypes
module (I've never done it). Googling 'ctypes shared library' gives some
promising results.


The point of Cython is to make this easier. It's worth learning.


My experience is different.

I created a function fred() in a C module, then linked that into a .dll 
(.so on Linux should be similar).


Then I wrote this code, adapted from the first hit I found for 'python 
ctypes example':


 import ctypes

 testlib = ctypes.CDLL('c:/c/c.dll')
 testlib.fred()

And it worked perfectly (on Py2 and Py3). (I understand passing 
arguments is a bit more complicated, but at least you have a working base.)


Then I tried the first example I saw for 'cython examples' which was 
this, first create a .pyx file as suggested, then:


 from distutils.core import setup
 from Cython.Build import cythonize

 setup(ext_modules=cythonize("helloworld.pyx"))

It spend several seconds thinking about it, then I got this enlightening 
error report:


 17.297
 Traceback (most recent call last):
  File "c:\langs\a.py", line 10, in 
from distutils.core import setup
  File "c:\python36\lib\distutils\core.py", line 16, in 
from distutils.dist import Distribution
  File "c:\python36\lib\distutils\dist.py", line 10, in 
from email import message_from_file
 ImportError: cannot import name 'message_from_file'

That was Py3; on Py2, I had 30 seconds of disk activity, and nearly full 
memory that almost stalled my machine before I aborted it. (Actually I 
mispelled both 'cythonize's in the example; it didn't make any difference).


And I still have no idea how to relate this to calling a native C 
function in an external shared library.


--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[Christian Gollwitzer]

Am 07.11.17 um 02:59 schrieb Grant Edwards:

On 2017-11-06, John Pote  wrote:


I have successfully used Python to perform unit and integration tests in
the past and I'd like to do the same for some C modules I'm working with
at work. There seem to be a number of ways of doing this but being busy
at work and home I looking for the approach with the least learning curve.


When I want to test C modules (usually destined for an embedded
system) using a Python framework, I use ctypes.

https://docs.python.org/3/library/ctypes.html


Another possibility is SWIG

http://www.swig.org/

When you already have a C header file, SWIG creates a Python module out 
of it. In many cases this is sufficient to get a runnable interface, but 
sometimes you need to add additional functions (like initialisation 
functions) to your module to make it usable from Python.


Christian
--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[Chris Angelico]

On Tue, Nov 7, 2017 at 12:52 PM, bartc  wrote:
> On 07/11/2017 00:58, Chris Angelico wrote:
>>
>> On Tue, Nov 7, 2017 at 10:43 AM, John Pote 
>> wrote:
>>>
>>> Hi all,
>>> I have successfully used Python to perform unit and integration tests in
>>> the
>>> past and I'd like to do the same for some C modules I'm working with at
>>> work. There seem to be a number of ways of doing this but being busy at
>>> work
>>> and home I looking for the approach with the least learning curve.
>>>
>>> I don't want to add the C modules into the CPython build itself as I've
>>> never done this and at work it's a real pain getting admin rights to do
>>> this, and when you do it lasts just 24 hours. The C modules are likely to
>>> change frequently as bugs are found and features added.
>>
>>
>> Fortunately, you don't have to modify CPython to do this :)
>>
>>> The other option I'm considering is to use sockets and write a C wrapper
>>> round the C modules I want to test. This has the advantage for me that I
>>> know about sockets from Python & C points of view and I get complete
>>> control
>>> of the C compile process. This may be important as the C modules come
>>> from
>>> an embedded project and I don't want to change them in any way.
>>>
>>> Are there any other approachs to this problem?
>>> I'll be using Python 3.5 (work) and 3.6 (home).
>>> Feedback appriciated.
>>
>>
>> This. The best way to do this is to create a wrapper... but the best
>> way to create that wrapper is with Cython. (Not to be confused with
>> CPython.) Check out http://cython.org/ to see what it takes to carry
>> information into and out of the C module. You'd compile the Cython
>> wrapper and then import that from Python directly.
>
>
> Cython seems very confusing to me.
>
> The simplest way for Python to run C is for Python to just run the
> executable created using C. If that is possible.
>
> Otherwise what /I/ would look for is ways to call C functions inside shared
> libraries (.dll and .so). That requires that the modules under test be
> wrapped as a shared library, which may be extra effort (but it will still be
> using C, so with familiar tools and no crossover with Python at this point).
>
> To call shared library C functions from Python I think involves the ctypes
> module (I've never done it). Googling 'ctypes shared library' gives some
> promising results.

The point of Cython is to make this easier. It's worth learning.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[Grant Edwards]

On 2017-11-06, John Pote  wrote:

> I have successfully used Python to perform unit and integration tests in 
> the past and I'd like to do the same for some C modules I'm working with 
> at work. There seem to be a number of ways of doing this but being busy 
> at work and home I looking for the approach with the least learning curve.

When I want to test C modules (usually destined for an embedded
system) using a Python framework, I use ctypes.

https://docs.python.org/3/library/ctypes.html

-- 
Grant



-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[bartc]

On 07/11/2017 00:58, Chris Angelico wrote:

On Tue, Nov 7, 2017 at 10:43 AM, John Pote  wrote:

Hi all,
I have successfully used Python to perform unit and integration tests in the
past and I'd like to do the same for some C modules I'm working with at
work. There seem to be a number of ways of doing this but being busy at work
and home I looking for the approach with the least learning curve.

I don't want to add the C modules into the CPython build itself as I've
never done this and at work it's a real pain getting admin rights to do
this, and when you do it lasts just 24 hours. The C modules are likely to
change frequently as bugs are found and features added.


Fortunately, you don't have to modify CPython to do this :)


The other option I'm considering is to use sockets and write a C wrapper
round the C modules I want to test. This has the advantage for me that I
know about sockets from Python & C points of view and I get complete control
of the C compile process. This may be important as the C modules come from
an embedded project and I don't want to change them in any way.

Are there any other approachs to this problem?
I'll be using Python 3.5 (work) and 3.6 (home).
Feedback appriciated.


This. The best way to do this is to create a wrapper... but the best
way to create that wrapper is with Cython. (Not to be confused with
CPython.) Check out http://cython.org/ to see what it takes to carry
information into and out of the C module. You'd compile the Cython
wrapper and then import that from Python directly.


Cython seems very confusing to me.

The simplest way for Python to run C is for Python to just run the 
executable created using C. If that is possible.


Otherwise what /I/ would look for is ways to call C functions inside 
shared libraries (.dll and .so). That requires that the modules under 
test be wrapped as a shared library, which may be extra effort (but it 
will still be using C, so with familiar tools and no crossover with 
Python at this point).


To call shared library C functions from Python I think involves the 
ctypes module (I've never done it). Googling 'ctypes shared library' 
gives some promising results.



--
bartc
--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[Bill]

John Pote wrote:

Hi all,
I have successfully used Python to perform unit and integration tests 
in the past and I'd like to do the same for some C modules I'm working 
with at work. There seem to be a number of ways of doing this but 
being busy at work and home I looking for the approach with the least 
learning curve.


I don't want to add the C modules into the CPython build itself as 
I've never done this and at work it's a real pain getting admin rights 
to do this, and when you do it lasts just 24 hours. The C modules are 
likely to change frequently as bugs are found and features added.


The other option I'm considering is to use sockets and write a C 
wrapper round the C modules I want to test. This has the advantage for 
me that I know about sockets from Python & C points of view and I get 
complete control of the C compile process. This may be important as 
the C modules come from an embedded project and I don't want to change 
them in any way.


Are there any other approachs to this problem?
I'll be using Python 3.5 (work) and 3.6 (home).
Feedback appriciated.



Install Oracle's "Virtual Box" software on your computer.  It's free. 
Then install a version of Linux on it (or something else, if you 
prefer).  Whatever you do, if you don't like it, you can replace your 
installation in a matter of minutes.

--
https://mail.python.org/mailman/listinfo/python-list

Re: Easiest way to access C module in Python

[Chris Angelico]

On Tue, Nov 7, 2017 at 10:43 AM, John Pote  wrote:
> Hi all,
> I have successfully used Python to perform unit and integration tests in the
> past and I'd like to do the same for some C modules I'm working with at
> work. There seem to be a number of ways of doing this but being busy at work
> and home I looking for the approach with the least learning curve.
>
> I don't want to add the C modules into the CPython build itself as I've
> never done this and at work it's a real pain getting admin rights to do
> this, and when you do it lasts just 24 hours. The C modules are likely to
> change frequently as bugs are found and features added.

Fortunately, you don't have to modify CPython to do this :)

> The other option I'm considering is to use sockets and write a C wrapper
> round the C modules I want to test. This has the advantage for me that I
> know about sockets from Python & C points of view and I get complete control
> of the C compile process. This may be important as the C modules come from
> an embedded project and I don't want to change them in any way.
>
> Are there any other approachs to this problem?
> I'll be using Python 3.5 (work) and 3.6 (home).
> Feedback appriciated.

This. The best way to do this is to create a wrapper... but the best
way to create that wrapper is with Cython. (Not to be confused with
CPython.) Check out http://cython.org/ to see what it takes to carry
information into and out of the C module. You'd compile the Cython
wrapper and then import that from Python directly.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list