Re: beginner's pyd question - exporting structs to python
On Tue, 2014-08-19 at 00:53 +, bachmeier via Digitalmars-d-learn wrote: […] I'm not sure which computational work he is referring to, but for statistical analysis, R dominates by a wide margin (although statistical analysis done in Silicon Valley, the type you read about on Hacker News, is often done using Python). My own :-) I think Python is not making inroads in the world-wide R community, but it is in the Matlab and Mathematica ones. Whether Julia acts as a disruptive technology here we will see. I write functions in D, compile as a shared library, and call from R. The more statically typed code I write, the more I like it. I'm finishing up a blog post describing my usage. This model also works for Python and D and does not require PyD (*). The issue here is that D, like C++, can provide shared objects (aka DLLs) with C linkage entry points and so Python extensions can be created. (*) Though arguably it is easier using PyD. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: beginner's pyd question - exporting structs to python
Thank to jwhear on irc who solved it for me despite claiming not
to be a pyd guru.
In case it's of benefit, here is what works:
module hellostruct;
import pyd.pyd;
import std.stdio;
import std.conv;
struct t_mystruct {
int i;
string s;
};
t_mystruct hellostruct(int[] inp) {
int i;
t_mystruct mystruct;
for (i=0;iinp.length;i++)
{
writefln(inp ~ to!string(i) ~ is ~
to!string(inp[i]));
}
writefln( * ok, bye);
mystruct.i=99;
mystruct.s=hello there;
return mystruct;
}
extern(C) void PydMain() {
def!(hellostruct)();
module_init();
wrap_struct!(
t_mystruct,
Member!s,
Member!i
)();
}
[root@fedora demark]# cat test.py
import os.path, sys
import distutils.util
# Append the directory in which the binaries were placed to
Python's sys.path,
# then import the D DLL.
libDir = os.path.join('build', 'lib.%s-%s' % (
distutils.util.get_platform(),
'.'.join(str(v) for v in sys.version_info[:2])
))
sys.path.append(os.path.abspath(libDir))
import hellostruct
inp=[1,2,3,4]
mystruct= hellostruct.hellostruct(inp)
print mystruct.i
print mystruct.s
Re: beginner's pyd question - exporting structs to python
On Mon, 2014-08-18 at 16:39 +, Laeeth Isharc via Digitalmars-d-learn wrote: Hi there. Brief introduction, and a beginner's question. I just started playing with D a couple of weeks ago. I have been programming in C on and off since the late 80s, but I do finance for a living and my programming skills grew rusty. I have a bit more time now to catch up with developments, and D looks a very intriguing robust alternative to C when python won't cut the mustard. All the cool folk doing data analysis and visualization using Python no longer bother with hand written C (*) for when pure Python won't cut the mustard. If Numba can't do the job, then Cython gets used. I have all my computational pure Python source codes running as fast as C these days thanks to Numba. (And judicious profiling.) I would say that Python folk will now only be looking to C, C++, Fortran, D, for pre-written libraries in those language. Given all the codes are written in C, C++ or Fortran with none in D… On the other hand PyD does work for me, but I only pass primitive types, not C structs. (*) Nor C++, nor D. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: beginner's pyd question - exporting structs to python
On Mon, 2014-08-18 at 17:17 +, Laeeth Isharc via Digitalmars-d-learn wrote: Thank to jwhear on irc who solved it for me despite claiming not to be a pyd guru. :-) […] distutils.util.get_platform(), […] Does os.uname() not provide sufficient information? […] print mystruct.i print mystruct.s Should use print as a function not as a statement. Use Python 3, or if you have to use Python 2 (which almost no-one does): from __future__ import print_function as the first statement. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: beginner's pyd question - exporting structs to python
All the cool folk doing data analysis and visualization using Python no longer bother with hand written C (*) for when pure Python won't cut the mustard. If Numba can't do the job, then Cython gets used. I have all my computational pure Python source codes running as fast as C these days thanks to Numba. (And judicious profiling.) I would say that Python folk will now only be looking to C, C++, Fortran, D, for pre-written libraries in those language. Given all the codes are written in C, C++ or Fortran with none in D… Thanks for the colour - I appreciate it. I have played with numba and pypy with numpy and it seems a powerful tool for some kinds of jobs. Perhaps it is my relative unfamiliarity with python, but for the time being I feel more comfortable with C type languages for other kinds of work. As a pragmatic idealist, one may as well use whatever tool seems to be pretty good generally and which one feels confident in wielding to accomplish the task at hand. Out of curiosity, what do you use D for given your views about the redundancy of C type languages for non-system programming? Should use print as a function not as a statement. Use Python 3, or if you have to use Python 2 (which almost no-one does): from __future__ import print_function as the first statement. Thank you.
Re: beginner's pyd question - exporting structs to python
On Monday, 18 August 2014 at 18:08:59 UTC, Russel Winder via Digitalmars-d-learn wrote: […] distutils.util.get_platform(), […] Does os.uname() not provide sufficient information? This was boilerplate generated by pyd.
Re: beginner's pyd question - exporting structs to python
On Mon, 2014-08-18 at 19:00 +, Laeeth Isharc via Digitalmars-d-learn wrote: On Monday, 18 August 2014 at 18:08:59 UTC, Russel Winder via Digitalmars-d-learn wrote: […] distutils.util.get_platform(), […] Does os.uname() not provide sufficient information? This was boilerplate generated by pyd. Hummm… if I get time I feel a PR in order. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: beginner's pyd question - exporting structs to python
On Mon, 2014-08-18 at 18:59 +, Laeeth Isharc via Digitalmars-d-learn wrote: […] Thanks for the colour - I appreciate it. I have played with numba and pypy with numpy and it seems a powerful tool for some kinds of jobs. Perhaps it is my relative unfamiliarity with python, but for the time being I feel more comfortable with C type languages for other kinds of work. As a pragmatic idealist, one may as well use whatever tool seems to be pretty good generally and which one feels confident in wielding to accomplish the task at hand. Whilst the hardcore Pythonistas remain Pythonistas, some of the periphery has jumped ship to Go. Sadly D did not capture these folk, it perhaps should have done. It would be easy to blame fadism, but I think the actual reasons are far less superficial. For me, NumPy has some serious problems despite being the accepted norm for computational work. The USP for Python/NumPy/SciPy/Matplotlib is that it costs an infinite amount less than Mathematica and Matlab. This is an non-trivial issue from which I am gaining a lot of business :-) Out of curiosity, what do you use D for given your views about the redundancy of C type languages for non-system programming? In a sense I could rightly be labelled a D dilettante. I had a possible startup a couple of years ago where we would have used D, but it never started. A side-effect was that I gave some support to David Simcha creating std.parallelism, but for the last couple of years my income has come from Python or Groovy/GPars with no real D activity. […] -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
Re: beginner's pyd question - exporting structs to python
On Monday, 18 August 2014 at 19:28:55 UTC, Russel Winder via Digitalmars-d-learn wrote: On Mon, 2014-08-18 at 19:00 +, Laeeth Isharc via Digitalmars-d-learn wrote: On Monday, 18 August 2014 at 18:08:59 UTC, Russel Winder via Digitalmars-d-learn wrote: […] distutils.util.get_platform(), […] Does os.uname() not provide sufficient information? This was boilerplate generated by pyd. Hummm… if I get time I feel a PR in order. Perhaps I should rephrase. I copied the example as a base, so technically it wasn't generated. But no harm in tweaking the examples.
Re: beginner's pyd question - exporting structs to python
Whilst the hardcore Pythonistas remain Pythonistas, some of the periphery has jumped ship to Go. Sadly D did not capture these folk, it perhaps should have done. It would be easy to blame fadism, but I think the actual reasons are far less superficial. So I gather that you agree that what everyone is doing may not be the best in this case (python vs D) if there are no direct network effects beyond libraries and getting help and you have the freedom to determine your own platform choices? For me, NumPy has some serious problems despite being the accepted norm for computational work. If not too offtopic, do you have a link describing, or would you briefly summarize these problems? I am intrigued. And what would you suggest in its place? Fortran? Out of curiosity, what do you use D for given your views about the redundancy of C type languages for non-system programming? In a sense I could rightly be labelled a D dilettante. I had a possible startup a couple of years ago where we would have used D, but it never started. A side-effect was that I gave some support to David Simcha creating std.parallelism, but for the last couple of years my income has come from Python or Groovy/GPars with no real D activity. Would you consider D stable enough/suitable for general financial market work with development initially by a small underresourced team? Not ultra high frequency execution - at most legging in and managing longer term positions. But I am more interested in sentiment analysis, producing technical analysis indicators that summarize market activity across many different securities, some bond arb stuff. C++ just seems so ugly, and I feel uncomfortable only having python in the toolbox. D seems so far to be quite suitable...
Re: beginner's pyd question - exporting structs to python
Dr Russel Winder 41 Buckmaster Road London SW11 1EN, UK Are there any D users groups/meetups in London? I see you are not far away (I am in Barnes). Laeeth
Re: beginner's pyd question - exporting structs to python
On Monday, 18 August 2014 at 23:12:28 UTC, Laeeth Isharc wrote: For me, NumPy has some serious problems despite being the accepted norm for computational work. If not too offtopic, do you have a link describing, or would you briefly summarize these problems? I am intrigued. And what would you suggest in its place? Fortran? I'm not sure which computational work he is referring to, but for statistical analysis, R dominates by a wide margin (although statistical analysis done in Silicon Valley, the type you read about on Hacker News, is often done using Python). I write functions in D, compile as a shared library, and call from R. The more statically typed code I write, the more I like it. I'm finishing up a blog post describing my usage.
Re: beginner's pyd question - exporting structs to python
On Mon, 2014-08-18 at 23:12 +, Laeeth Isharc via Digitalmars-d-learn wrote: Whilst the hardcore Pythonistas remain Pythonistas, some of the periphery has jumped ship to Go. Sadly D did not capture these folk, it perhaps should have done. It would be easy to blame fadism, but I think the actual reasons are far less superficial. So I gather that you agree that what everyone is doing may not be the best in this case (python vs D) if there are no direct network effects beyond libraries and getting help and you have the freedom to determine your own platform choices? Go arrived with high marketing as something new, which it is. It is a stripped down, strongly typed C with memory management, objects and extension methods, and (most importantly) goroutines as a lightweight built-in concurrency and parallelism framework based on thread pools. The language itself is very simple (apart from the consequences of semi-colon ellision/re-adding), and so captured the imagination of many. Not unreasonably. I really quite like Go. However it has some, for me, serious irritants, most especially an obsessive hatred of exceptions, enforcing return codes and error handling at the point of call – though it does support this in a neat way. One effect of the fanboi element of using go has been very rapid evolution of the eco-system, especially strong because of the package approach and the use of DVCS as a tool for accessing packages. D is not a new language so doesn't have the new shiny toy feature, but it remains an rapidly evolving language. It is definitely a better C++ but C++ is evolving fast enough that C++ folks stay with C++. So in a sense D has failed to capture the market it aimed for when it started. Also it is still playing catch up in the eco-system stakes. D does however have a very neat model of abstraction that allows for a very functional programming approach. It isn't functional programming per se, but it gets very declarative. C++ is a long way behind in this, but there is little C++ → D transfer. The problem for Python folks looking for a native code language is that D is a big language and Go is a small language. And then there is Rust… For me, NumPy has some serious problems despite being the accepted norm for computational work. If not too offtopic, do you have a link describing, or would you briefly summarize these problems? I am intrigued. And what would you suggest in its place? Fortran? I'll start a separate thread for this one. […] Would you consider D stable enough/suitable for general financial market work with development initially by a small underresourced team? Not ultra high frequency execution - at most legging in and managing longer term positions. But I am more interested in sentiment analysis, producing technical analysis indicators that summarize market activity across many different securities, some bond arb stuff. C++ just seems so ugly, and I feel uncomfortable only having python in the toolbox. D seems so far to be quite suitable... The most important thing about any project using a programming language is that the development team, testing team and deployment team all approve of the language being used. It sounds like you approve of D so go for it. The language is not totally stable, so expect incompatibilities on upgrade. I am not a fan of all this backward compatibility obsession that has gripped Fortran, C++ and Java (though I can see why it is necessary), but with D you will need to budget for upgrade-related refactoring. Personally I was happy to take that hit. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder
