On Wed, 2014-12-03 at 01:07 +0000, Michael via Digitalmars-d-learn wrote: > Hi. I'm new here and this is my first post. I'm not sure this is > the right subforum for it, but wasn't sure where else to put it > either. > > I've written a library to talk to some external hardware using a > socket. It uses the std.concurrency threads to send messages > between the main D-object for the hardware and the D-object for > the sockets. I then wanted to be able to call these functions > from Python. PyD appeared to be out of date, so I've been using a
As far as I can tell PyD is still active, but in a non-funded FOSS way, i.e. work happens as and when volunteers put time and effort in. I haven't tried PyD recently but it worked fine last time I did. If can set out what you tried and what didn't work, maybe there is a PyD solution, or a fix to PyD to give a solution? > D -> C interface, and a C -> Python interface. The python code > will often run from different python threads, so I then added yet > another message-passing layer between the D->C interface and the > D->hardware interface. D's "big problem" is shared objects/dynamic link libraries. Without them you cannot interwork with Python at all. I have tried experiments on Linux creating shared libraries from D code with C linkage entry points to create classic Python extensions, and it appears to work fine. Except for having to start up the D heap and thread management, should they be needed. But that is what PyD is there for. If I took my experiments any further I would end up recreating PyD or something like it. It sounds like you are in a similar situation except that you appear to have an extra layer of C code. I am not sure a layer of C is needed between Python and D, it would be good to know more about why you seem to need it. Your use case is interesting as I have more or less given up on using D in a Python context. CPython naturally requires C linkage shared objects for non-Python code so C, C++ or D would be fine except that everyone does it in C or C++, very few people in the arena have even heard of D. PyPy brings it's own issues with non-Python code but the now have a C capability. With Cython, Pythran, and more recently Numba there is increasing less and less need for any user written non-Python code. Hardware control would be done in C and PyPy now has a C linkage mechanism. Of course for Python networking there is Twisted, Asyncio and Tornado so no Python folk are using non-Python code for handling networking. > My problem is that this code routinely causes segmentation > faults. I've spent a long time going through trying to figure out > exactly what the causes are. I think there have been some related > to D-exceptions not being handled gracefully by the C/Python > code. Some more by stdout writing from multiple threads (which > surprised me). > > I'm fairly sure I have tackled both of these issues, but it still > seems like Python threads and D threads don't mix well. When > running the same functions from D, I am able to get no errors, > but when run from Python/C it causes segfaults reliably. Without seeing your code, it is difficult to say what may or may not be the problem, but I would guess it is about D infrastructure start up. I recollect being able to reliably get segfaults this way. Are you using ctypes or CFFI on the Python side? > Sorry for the large exposition. I am currently at the point of > suspecting bugs in Phobos, but I am unskilled enough to tell for > sure, and would appreciate any help. > > The latest core dump gives a backtrace of almost entirely phobos > commands: > > #0 0x00007fe789ad3b97 in gc.gc.Gcx.fullcollect() () from > /lib/libphobos2.so.0.66 > #1 0x00007fe789ad3294 in gc.gc.Gcx.bigAlloc() () from > /lib/libphobos2.so.0.66 > #2 0x00007fe789ad0df1 in gc.gc.GC.mallocNoSync() () from > /lib/libphobos2.so.0.66 > #3 0x00007fe789ad0c15 in gc.gc.GC.malloc() () from > /lib/libphobos2.so.0.66 > #4 0x00007fe789ad6470 in gc_malloc () from > /lib/libphobos2.so.0.66 > #5 0x00007fe789ae6d36 in _d_newitemT () from > /lib/libphobos2.so.0.66 > #6 0x00007fe789e57112 in > std.array.__T8AppenderTAaZ.Appender.__T3putTAxaZ.put() () from > /usr/lib/libv5camera.so > #7 0x00007fe789e570b5 in > std.array.__T8AppenderTAaZ.Appender.__T3putTAxaZ.put() () from > /usr/lib/libv5camera.so > #8 0x00007fe789e562dc in > std.array.__T8AppenderTAaZ.Appender.__T3putTAaZ.put() () from > /usr/lib/libv5camera.so > #9 0x00007fe789e561ea in > std.array.__T8AppenderTAaZ.Appender.__T3putTxwZ.put() () from > /usr/lib/libv5camera.so > #10 0x00007fe789e5617d in > std.format.__T10formatCharTS3std5array16__T8AppenderTAaZ8AppenderZ.formatChar() > > () from /usr/lib/libv5camera.so > #11 0x00007fe789e56132 in > std.format.__T10formatCharTS3std5array16__T8AppenderTAaZ8AppenderZ.formatChar() > > () from /usr/lib/libv5camera.so > #12 0x00007fe789e61f09 in > std.concurrency.MessageBox.__T3getTS4core4time8DurationTDFNfAyaiZvZ.get() > () from /usr/lib/libv5camera.so > #13 0x00007fe789e5b4ac in > std.concurrency.MessageBox.__T3getTS4core4time8DurationTDFNaNbNiNfAyaiZvZ.get() > > () from /usr/lib/libv5camera.so > #14 0x00007fe789e57e8d in > std.typecons.__T5TupleTAyaTiTG65536kZ.Tuple.__T6__ctorTS3std8typecons24__T5TupleTAyaTiTG65536kZ5TupleZ.__ctor() > > () > from /usr/lib/libv5camera.so > #15 0x00007fe789e581f1 in > std.variant.__T8VariantNVmi32Z.VariantN.__T7handlerTS3std8typecons24__T5TupleTAyaTiTG65536kZ5TupleZ.handler() > > () > from /usr/lib/libv5camera.so > #16 0x00007fe789e57d0f in > std.typecons.__T5TupleTAyaTiTG65536kZ.Tuple.__T8opEqualsTS3std8typecons24__T5TupleTAyaTiTG65536kZ5TupleZ.opEquals() > > () > from /usr/lib/libv5camera.so > #17 0x00007fe789e57ba8 in > std.typecons.__T5TupleTAyaTiTG65536kZ.injectNamedFields() () from > /usr/lib/libv5camera.so > #18 0x00007fe789e62087 in > std.concurrency.MessageBox.__T3getTS4core4time8DurationTDFNfAyaiZvZ.get() > () from /usr/lib/libv5camera.so > #19 0x00007fe789e621a3 in > std.concurrency.MessageBox.__T3getTS4core4time8DurationTDFNfAyaiZvZ.get() > () from /usr/lib/libv5camera.so > #20 0x00007fe789e5b7f6 in > std.concurrency.MessageBox.__T3getTS4core4time8DurationTDFNaNbNiNfAyaiZvZ.get() > > () from /usr/lib/libv5camera.so > #21 0x00007fe789ac7d51 in core.thread.Thread.run() () from > /lib/libphobos2.so.0.66 > #22 0x00007fe789ac6f95 in thread_entryPoint () from > /lib/libphobos2.so.0.66 > #23 0x00007fe79cee5182 in start_thread (arg=0x7fe77aca5700) at > pthread_create.c:312 > #24 0x00007fe79cc11fbd in clone () at > ../sysdeps/unix/sysv/linux/x86_64/clone.S:111 My guess would be not properly initializing the D infrastructure from the incoming Python thread. I would suggest that you want to avoid threads crossing the boundaries and just pass data via a shared channel. Unix pipes seem to work well in this context since they provide a language independent data channel. -- Russel. ============================================================================= Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Road m: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
signature.asc
Description: This is a digitally signed message part
