Re: D is dead (was: Dicebot on leaving D: It is anarchy driven development in all its glory.)
On Tuesday, 4 September 2018 at 02:24:25 UTC, Manu wrote: On Mon, 3 Sep 2018 at 18:45, Laeeth Isharc via Digitalmars-d wrote: On Monday, 3 September 2018 at 17:15:03 UTC, Laurent Tréguier wrote: > On Monday, 3 September 2018 at 16:55:10 UTC, Jonathan M > Davis wrote: >> Most of the work that gets done is the stuff that the folks >> contributing think is the most important - frequently what >> is most important for them for what they do, and very few >> (if any) of the major contributors use or care about IDEs >> for their own use. And there's tons to do that has nothing >> to do with IDEs. There are folks who care about it enough >> to work on it, which is why projects such as VisualD exist >> at all, and AFAIK, they work reasonably well, but the only >> two ways that they're going to get more work done on them >> than is currently happening is if the folks who care about >> that sort of thing contribute or if they donate money for >> it to be worked on. Not long ago, the D Foundation >> announced that they were going to use donations to pay >> someone to work on his plugin for Visual Studio Code: >> >> https://forum.dlang.org/post/rmqvglgccmgoajmhy...@forum.dlang.org >> >> So, if you want stuff like that to get worked on, then >> donate or pitch in. >> >> The situation with D - both with IDEs and in general - has >> improved greatly over time even if it may not be where you >> want it to be. But if you're ever expecting IDE support to >> be a top priority of many of the contributors, then you're >> going to be sorely disappointed. It's the sort of thing >> that we care about because we care about D being >> successful, but it's not the sort of thing that we see any >> value in whatsoever for ourselves, and selfish as it may >> be, when we spend the time to contribute to D, we're >> generally going to work on the stuff that we see as having >> the most value for getting done what we care about. And >> there's a lot to get done which impacts pretty much every D >> user and not just those who want something that's >> IDE-related. >> >> - Jonathan M Davis > > The complaints I have is exactly why I'm myself maintaining > plugins for VSCode, Atom, and others soon. Don't worry, I > still > think D is worth putting some time and effort into and I know > actions generally get more things done than words. > I also know that tons of stuff is yet to be done in regards > to > the actual compilers and such. > > It just baffles me a bit to see the state of D in this > department, when languages like Go or Rust (hooray for yet > another comparison to Go and Rust) are a lot younger, but > already have what looks like very good tooling. > Then again they do have major industry players backing them > though... Why is Go's IDE support baffling? It was a necessity to achieve Google's commercial aims, I should think. " The key point here is our programmers are Googlers, they’re not researchers. They’re typically, fairly young, fresh out of school, probably learned Java, maybe learned C or C++, probably learned Python. They’re not capable of understanding a brilliant language but we want to use them to build good software. So, the language that we give them has to be easy for them to understand and easy to adopt." – Rob Pike I don't know the story of Rust, but if I were working on a project as large as Firefox I guess I would want an IDE too! Whereas it doesn't seem like it's so important to some of D's commercial users because they have a different context. I don't think it's overall baffling that D hasn't got the best IDE support of emerging languages. The people that contribute to it, as Jonathan says, seen to be leas interested in IDEs and no company has found it important enough to pay someone else to work on it. So far anyway but as adoption grows maybe that will change. It's been a key hurdle for as long as I've been around here. I've been saying for 10 years that no company I've ever worked at can take D seriously without industry standard IDE support. My feeling is that we have recently reached MVP status... that's a huge step, 10 years in the making ;) I think it's now at a point where more people *wouldn't* reject it on contact than those who would. But we need to go much further to make developers genuinely comfortable, and thereby go out of their way to prefer using D than C++ and pitch as such to their managers. Among all developers I've demo-ed or introduced recently, I can say for certain that developer enthusiasm is driven by their perception of the tooling in the order of 10x more than the language. That's only because you insist on working for compani
Re: D is dead (was: Dicebot on leaving D: It is anarchy driven development in all its glory.)
On Tuesday, 4 September 2018 at 05:38:49 UTC, Iain Buclaw wrote: On 4 September 2018 at 04:19, Laeeth Isharc via Digitalmars-d wrote: On Monday, 3 September 2018 at 16:07:21 UTC, RhyS wrote: A good example being the resources going into DMD, LDC, GDC... 3 Compilers for one language, when even well funded languages stick to one compiler. And now some people think its a good idea to have DMD also cross compile because "its not hard to do". No, maybe not but who will do all the testing, what resources are going to spend when things do not work for some users ( and the negative impact on their experience )... Its a long list but people do not look past this. It sounds like fun, lets think / or do it. What resources do you think go into GDC? I think Iain would love to hear about all these resources because I am not sure he has been made aware of them because they don't exist beyond him and possibly a tiny number of others helping in part at certain stages. *Looks behind self* *Looks under desk* *Looks under keyboard* There must be resources somewhere, but none appear to be within reach. :-) If Iain had a beer for every person that complained about the effort spent by team GDC without having first thanked him and his vast team then... People are sometimes quite disconnected from reality. At least I have no other explanation for people demanding others do this or do that without doing the minimum necessary to make it appealing for others to work on it. I mean my experience is that you can pay people a lot of money and ask them beforehand do you want to work on X, and it's no guarantee they actually will be willing to when it comes to it. Programmers in general can be very independent-minded people, and if somebody is looking for especially meek and compliant people then if you have come to the D forums you are in the wrong place! One can be much more persuasive with positive words than complaints. Most people are well aware of that so if they are complaining it's in my judgement because they want to complain. People with high standards will do that when they feel powerless. I'm not talking here about notorious highly intelligent trolls like Ola and sock-puppets who never seem to actually write code in D. But nobody who can keep up here is powerless. It's possible to change the world you know, and from the most unpromising start. Forget about what's realistic, and focus on what you want to achieve. Believe me, you can achieve an awful lot from the most unpromising start. People talk about how most people are not super-hackers and one shouldn't expect them to manage without polish. Well hacker is a state of mind,a way of being in the world. Ask Iain if his self-conception is as a super-hacker with l33t skillz that a mere professional programmer couldn't match and you might be surprised (I think his self-conception might be wrong, but that's Dunning Kruger in action for you). It's really much more about values and virtues then capabilities. Are you able to tolerate discomfort and the accurate initial feeling of conscious incompetence? Because that's what real learning feels like once you leave the spoon-feeding stream of education. D is a gift to the world from Walter, Andrei, and those who contributed after it was begun. Just demanding people do stuff for you without doing anything to contribute back - that's not how life works. I don't think I have ever seen this degree of a feeling of entitlement in my life! And I've been working in finance since 1993. If doesn't want to pay money towards the development of IDE integration, doesn't want to do any work themselves, then the least they could do is draw up a feature list of what's missing and find a way to help from time to time with the organisation of the work. That's the only way things ever get done anyway. Have you noticed how the documentation has gotten much better? Runnable examples too. Did that happen because people complained? No - it happened because Seb Wilzbach (and maybe others) took the initiative to make it happen and did the work themselves. A little money goes a long way in open source. So if you're a company and you're complaining and not donating money to the Foundation then what exactly do you expect? We have a few support contracts with MongoDB (a choice made before I got involved) and the legal fees alone were 20k and we pay about 30k USD a year. If a few companies contributed at that scale to the Foundation that's at least a couple of full-time developers. And if you disagree with Andrei and Walter choices about priorities you know you can just direct where the money should be spent as we are with SAoC.
porting nanomsg bindings to dlang
Hi. Everyone has heard of ZeroMQ, but the creator (or one of the main guys) has been working on a successor framework written in C. (He has an interesting paper on why using C++ was a mistake - perhaps we should get him to look at D if he has not done so already). In any case, I could not see a set of D bindings so I wrote a very rough first draft of them last night. I only picked up D a couple of months back, and it's been about twenty years since I wrote much C (I am not a developer by trade), so be kind if the results are not yet quite up to scratch. Link to the repository is here - not even worthy of alpha status: https://github.com/Laeeth/d-nanomsg/tree/master So far I have tried the first example from here (which works), and am working my way down to test the others: http://tim.dysinger.net/posts/2013-09-16-getting-started-with-nanomsg.html
porting nanomsg bindings to dlang
examples here all work: https://github.com/dysinger/nanomsg-examples/blob/master/README.org but I have not tried any larger projects.
Re: porting nanomsg bindings to dlang
Thanks. I am slowly getting to grips with all these post-stone age innovations... On Wednesday, 5 November 2014 at 20:24:41 UTC, Matt Soucy wrote: On 11/05/2014 01:12 PM, Laeeth Isharc wrote: Hi. Everyone has heard of ZeroMQ, but the creator (or one of the main guys) has been working on a successor framework written in C. (He has an interesting paper on why using C++ was a mistake - perhaps we should get him to look at D if he has not done so already). In any case, I could not see a set of D bindings so I wrote a very rough first draft of them last night. I only picked up D a couple of months back, and it's been about twenty years since I wrote much C (I am not a developer by trade), so be kind if the results are not yet quite up to scratch. Link to the repository is here - not even worthy of alpha status: https://github.com/Laeeth/d-nanomsg/tree/master So far I have tried the first example from here (which works), and am working my way down to test the others: http://tim.dysinger.net/posts/2013-09-16-getting-started-with-nanomsg.html A few small suggestions: Use a .gitignore so you're not tracking the objects and executables Convert to a dub package to make it easier for other people to incorporate into their projects Split out the test code from the library code
Project to port Bloomberg API to D
About Bloomberg -- Bloomberg provides a platform for market data, news, analytics, and messaging to financial services firms and other organisations with an interest in markets. There are some interesting developments on the horizon but, for the time being, for investment banks and hedge funds it remains the standard platform for such. Their API is public, and they provide implementations for C++, C, Python, Java, and .Net amongst others. Interesting for D? -- The financial services space might be an interesting domain for D because it's an economically important area where performance, safety and productivity all matter, and none of the existing solutions (mostly C, C++, Java, Python) are particularly satisfactory. At the same time, people in finance do not tend to be in the business of taking big technology risks with a product not yet widely seen as industry standard because the costs of messing up may be especially high (and messing up for creative reasons is much less acceptable than if you follow the herd). Perhaps in the face of new emerging realities regarding data set sizes, memory speed, and single core processing power that creates opportunity for more entrepreneurial participants to use the best tool for the job. I certainly hope so. People often want to see results quickly, so even if it's not really much work to do the port, there might be a big difference in the chance someone might consider trying (and then learning to appreciate) D. Beyond a mature numpy/pandas equivalent, the other missing piece is an analytical library for derivative pricing and risk management. QuantLib is the main open source project - it is written in C++ but has D SWIG wrappers. (There is the Java Opengamma too, but I am not so familiar with it). SWIG is not great though, and I don't think the wrappers cover all the functionality. (There was a project to port QuantLib to D but I am not sure it ever got started, and it is now defunct). It is super appealing to be able to rapidly prototype and explore ideas about largeish data sets and have the results be fast, knowing also that the code one has written can serve as the basis for something used in production (thanks to strong typing, elegant language design, etc). So I think there is a chance for D to take off within finance for some uses. Project status -- So for my own purposes - but I hope they might benefit others - I have ported the C headers and C examples to D. It's pre-alpha status - everything compiles, but I need to start up a Windows machine with Bloomberg to get it linking and get any bugs out of the port. I thought of holding back posting till they were perfect, but decided to just post what I have in case anyone else should have an interest. It may be that the C++ headers create a more appealing interface, but I do not know the current status of C++ vs D interface as the dlang.org docs are stale. For the time being the examples are just an ugly direct translation, with lots of casts and not much attention paid to beauty and elegance. But it's something, and a start. Link https://github.com/Laeeth/d-bloombergapi Laeeth.
Quandl API ported to D
Embarassingly trivial, and I don't claim it is well-written. But perhaps it will save somebody a few minutes. Quandl is The largest, most usable collection of free open data in the world. They offer a bunch of predominantly financial and economic data from different sources for free (with a pretty generous API allowance) and plan to earn a buck down the line selling commercial data and being a platform for distribution. Given the situation with Bloomberg (the banks are not terribly happy following the JPM whale/Bloomberg journalism revelations) and structural change in the market for data, this is an intriguing platform. In any case, I ported the sample C++ API to D, which was more a question of deleting redundant lines and using std.net.curl then anything else. Perhaps someone will find it useful. https://github.com/Laeeth/d-quandl Laeeth.
Re: Quandl API ported to D
On Thursday, 11 December 2014 at 10:26:47 UTC, Gary Willoughby wrote: On Thursday, 11 December 2014 at 01:25:32 UTC, Laeeth Isharc wrote: Embarassingly trivial, and I don't claim it is well-written. But perhaps it will save somebody a few minutes. Quandl is The largest, most usable collection of free open data in the world. They offer a bunch of predominantly financial and economic data from different sources for free (with a pretty generous API allowance) and plan to earn a buck down the line selling commercial data and being a platform for distribution. Given the situation with Bloomberg (the banks are not terribly happy following the JPM whale/Bloomberg journalism revelations) and structural change in the market for data, this is an intriguing platform. In any case, I ported the sample C++ API to D, which was more a question of deleting redundant lines and using std.net.curl then anything else. Perhaps someone will find it useful. https://github.com/Laeeth/d-quandl Laeeth. Interesting. Why not also add it to the dub registry? http://code.dlang.org/ You can find more information here: http://code.dlang.org/about and more about the package format here: http://code.dlang.org/package-format Thanks for the suggestion. I hope to do so when a bit more time (I am still not yet that comfortable with dub myself). Laeeth.
HDF5 bindings for D
https://github.com/Laeeth/d_hdf5 HDF5 is a very valuable tool for those working with large data sets. From HDF5group.org HDF5 is a unique technology suite that makes possible the management of extremely large and complex data collections. The HDF5 technology suite includes: * A versatile data model that can represent very complex data objects and a wide variety of metadata. * A completely portable file format with no limit on the number or size of data objects in the collection. * A software library that runs on a range of computational platforms, from laptops to massively parallel systems, and implements a high-level API with C, C++, Fortran 90, and Java interfaces. * A rich set of integrated performance features that allow for access time and storage space optimizations. * Tools and applications for managing, manipulating, viewing, and analyzing the data in the collection. * The HDF5 data model, file format, API, library, and tools are open and distributed without charge. From h5py.org: [HDF5] lets you store huge amounts of numerical data, and easily manipulate that data from NumPy. For example, you can slice into multi-terabyte datasets stored on disk, as if they were real NumPy arrays. Thousands of datasets can be stored in a single file, categorized and tagged however you want. H5py uses straightforward NumPy and Python metaphors, like dictionary and NumPy array syntax. For example, you can iterate over datasets in a file, or check out the .shape or .dtype attributes of datasets. You don't need to know anything special about HDF5 to get started. In addition to the easy-to-use high level interface, h5py rests on a object-oriented Cython wrapping of the HDF5 C API. Almost anything you can do from C in HDF5, you can do from h5py. Best of all, the files you create are in a widely-used standard binary format, which you can exchange with other people, including those who use programs like IDL and MATLAB. === As far as I know there has not really been a complete set of HDF5 bindings for D yet. Bindings should have three levels: 1. pure C API declaration 2. 'nice' D wrapper around C API (eg that knows about strings, not just char*) 3. idiomatic D interface that uses CTFE/templates I borrowed Stefan Frijter's work on (1) above to get started. I cannot keep track of things when split over too many source files, so I put everything in one file - hdf5.d. Have implemented a basic version of 2. Includes throwOnError rather than forcing checking status C style, but the exception code is not very good/complete (time + lack of experience with D exceptions). (3) will have to come later. It's more or less complete, and the examples I have translated so far mostly work. But still a work in progress. Any help/suggestions appreciated. [I am doing this for myself, so project is not as pretty as I would like in an ideal world]. https://github.com/Laeeth/d_hdf5
Re: HDF5 bindings for D
On Monday, 22 December 2014 at 05:04:10 UTC, Rikki Cattermole wrote: You seem to be missing your dub file. Would be rather hard to get it onto dub repository without it ;) Oh and keep the bindings separate from wrappers in terms of subpackages. Thanks - added now. Will work on separating out bindings when have a bit more time, but it should be easy enough.
Swiss Ephemeris / Nelder-Mead simplex
Last one for a while, I think. I wish you all a very peaceful Christmas and New Year, and let's hope 2015 brings some more positive energy to the world. Links here: https://github.com/Laeeth/d_simplex https://github.com/Laeeth/d_swisseph 1. D bindings/wrappers for the swiss ephemeris http://www.astro.com/swisseph/swephinfo_e.htm The SWISS EPHEMERIS is the high precision ephemeris developed by Astrodienst, largely based upon the DExxx ephemerides from NASA's JPL . The original release in 1997 was based on the DE405/406 ephemeris. Since release 2.00 in February 2014, it is based on the DE431 ephemeris released by JPL in September 2013. NB - Swiss Ephemeris is not free for commercial use. 2. D port of simple Nelder-Mead simplex minimisation (written by Michael F. Hutt in original C version) here. With constraints. From Wiki: https://en.wikipedia.org/wiki/Nelder-Mead_method The Nelder–Mead method or downhill simplex method or amoeba method is a commonly used nonlinear optimization technique, which is a well-defined numerical method for problems for which derivatives may not be known. However, the Nelder–Mead technique is a heuristic search method that can converge to non-stationary points[1] on problems that can be solved by alternative methods. Links here: https://github.com/Laeeth/d_simplex https://github.com/Laeeth/d_swisseph
Re: Swiss Ephemeris / Nelder-Mead simplex
On Monday, 22 December 2014 at 21:39:08 UTC, Marc Schütz wrote: On Monday, 22 December 2014 at 20:46:23 UTC, bachmeier wrote: It's been ages since I read the paper, but there is a parallel version of Nelder-Mead that is supposed to give very large performance improvements, even when used on a single processor: http://www.cs.ucsb.edu/~kyleklein/publications/neldermead.pdf It is not difficult to implement. I may look into modifying your code to implement it when I get some time. It will certainly also be advantageous to pass the functions as aliases, so that they can get inlined. Thanks, Marc. I appreciate the pointer, and if you do have time to look at the code. I confess that it can't really be called my own implementation as I simply ported it to D. There is some more clever stuff within quantlib (c++ project), but I quite liked the idea of starting with this one as it is simple, and speed is not yet vital at this stage. Laeeth.
Re: D is one if Wired's 10 Most Hardcore Tech Stories of 2014
On Saturday, 27 December 2014 at 11:15:11 UTC, Walter Bright wrote: On 12/26/2014 10:25 PM, Walter Bright wrote: http://www.wired.com/2014/12/wired-enterprise-year/ https://www.reddit.com/r/programming/comments/2qioj8/dlang_story_makes_wireds_10_most_hardcore_tech/ Congratulations, Walter. It's very satifying to see emerging mainstream recognition of something so solid and whose advantages are based in real advantages rather than marketing hype. I am personally very glad you decided on 'retirement' that you were not after all the daytime TV kind of guy, and I wanted to thank you for everything you have contributed to D and its users. I wish you a great start to the new year. Laeeth.
d_etpan: project to write bindings and wrappers for the libetpan email / IMAP library
https://github.com/Laeeth/d_etpan http://www.etpan.org/libetpan.html https://github.com/dinhviethoa/libetpan LibEtPan The purpose of this mail library is to provide a portable, efficient framework for different kinds of mail access: IMAP, SMTP, POP and NNTP. It provides an API for C language. Who uses LibEtPan? Almost all email app in Mac and iOS AppStore: see Mailcore Cisco Mobile for iPhone Claws Mail Filemaker Pro Google Apps Migration for Microsoft Exchange Playstation Vita Email Application reMail (Acquired by Google in 2010) Sparrow (Acquired by Google in 2012) Status == Pre-alpha bindings only. They compile without errors, but probably need some fixes and some key functions missing. I have not yet finished porting the demos. Later when time I will add D style wrappers.
beginner's pyd question - exporting structs to python
Hi there.
Brief introduction, and a beginner's question.
I just started playing with D a couple of weeks ago. I have been
programming in C on and off since the late 80s, but I do finance
for a living and my programming skills grew rusty. I have a bit
more time now to catch up with developments, and D looks a very
intriguing robust alternative to C when python won't cut the
mustard.
I am trying to implement the Demark technical analysis indicators
in D, and want to be able to call them from python.
So I have set up pyd okay, and the examples seem to work fine.
Running on Fedora 20. I am struggling a bit with the syntax on
wrap_struct to export a struct and function returning a struct to
python. Here is what I have:
module hello2;
import pyd.pyd;
import std.stdio;
import std.conv;
struct t_mystruct {
int i;
string s;
};
t_mystruct hello(int[] inp) {
int i;
t_mystruct mystruct;
mystruct.i=inp.length;
mystruct.s=hello there;
return mystruct;
}
extern(C) void PydMain() {
def!(hello)();
module_init();
wrap_struct!(
t_mystruct,
t_mystruct
);
}
Error:
def: hello
/root/anaconda/lib/python2.7/site-packages/celerid/infrastructure/pyd/def.d(151):
Error: static assert string parameters must be wrapped with
Docstring, Mode, etc
/root/anaconda/lib/python2.7/site-packages/celerid/infrastructure/pyd/class_wrap.d(1474):
instantiated from here: Args!(, , t_mystruct, ,
t_mystruct)
hellostruct.d(30):instantiated from here:
wrap_struct!(t_mystruct, t_mystruct)
error: command 'dmd' failed with exit status 1
[root@fedora demark]#
Apologies if it's in one of the examples - I did poke around, and
couldn't see any obvious sample.
Many thanks.
Laeeth.
Re: beginner's pyd question - exporting structs to python
Thank to jwhear on irc who solved it for me despite claiming not
to be a pyd guru.
In case it's of benefit, here is what works:
module hellostruct;
import pyd.pyd;
import std.stdio;
import std.conv;
struct t_mystruct {
int i;
string s;
};
t_mystruct hellostruct(int[] inp) {
int i;
t_mystruct mystruct;
for (i=0;iinp.length;i++)
{
writefln(inp ~ to!string(i) ~ is ~
to!string(inp[i]));
}
writefln( * ok, bye);
mystruct.i=99;
mystruct.s=hello there;
return mystruct;
}
extern(C) void PydMain() {
def!(hellostruct)();
module_init();
wrap_struct!(
t_mystruct,
Member!s,
Member!i
)();
}
[root@fedora demark]# cat test.py
import os.path, sys
import distutils.util
# Append the directory in which the binaries were placed to
Python's sys.path,
# then import the D DLL.
libDir = os.path.join('build', 'lib.%s-%s' % (
distutils.util.get_platform(),
'.'.join(str(v) for v in sys.version_info[:2])
))
sys.path.append(os.path.abspath(libDir))
import hellostruct
inp=[1,2,3,4]
mystruct= hellostruct.hellostruct(inp)
print mystruct.i
print mystruct.s
Re: beginner's pyd question - exporting structs to python
All the cool folk doing data analysis and visualization using Python no longer bother with hand written C (*) for when pure Python won't cut the mustard. If Numba can't do the job, then Cython gets used. I have all my computational pure Python source codes running as fast as C these days thanks to Numba. (And judicious profiling.) I would say that Python folk will now only be looking to C, C++, Fortran, D, for pre-written libraries in those language. Given all the codes are written in C, C++ or Fortran with none in D… Thanks for the colour - I appreciate it. I have played with numba and pypy with numpy and it seems a powerful tool for some kinds of jobs. Perhaps it is my relative unfamiliarity with python, but for the time being I feel more comfortable with C type languages for other kinds of work. As a pragmatic idealist, one may as well use whatever tool seems to be pretty good generally and which one feels confident in wielding to accomplish the task at hand. Out of curiosity, what do you use D for given your views about the redundancy of C type languages for non-system programming? Should use print as a function not as a statement. Use Python 3, or if you have to use Python 2 (which almost no-one does): from __future__ import print_function as the first statement. Thank you.
Re: beginner's pyd question - exporting structs to python
On Monday, 18 August 2014 at 18:08:59 UTC, Russel Winder via Digitalmars-d-learn wrote: […] distutils.util.get_platform(), […] Does os.uname() not provide sufficient information? This was boilerplate generated by pyd.
Re: beginner's pyd question - exporting structs to python
On Monday, 18 August 2014 at 19:28:55 UTC, Russel Winder via Digitalmars-d-learn wrote: On Mon, 2014-08-18 at 19:00 +, Laeeth Isharc via Digitalmars-d-learn wrote: On Monday, 18 August 2014 at 18:08:59 UTC, Russel Winder via Digitalmars-d-learn wrote: […] distutils.util.get_platform(), […] Does os.uname() not provide sufficient information? This was boilerplate generated by pyd. Hummm… if I get time I feel a PR in order. Perhaps I should rephrase. I copied the example as a base, so technically it wasn't generated. But no harm in tweaking the examples.
Re: beginner's pyd question - exporting structs to python
Whilst the hardcore Pythonistas remain Pythonistas, some of the periphery has jumped ship to Go. Sadly D did not capture these folk, it perhaps should have done. It would be easy to blame fadism, but I think the actual reasons are far less superficial. So I gather that you agree that what everyone is doing may not be the best in this case (python vs D) if there are no direct network effects beyond libraries and getting help and you have the freedom to determine your own platform choices? For me, NumPy has some serious problems despite being the accepted norm for computational work. If not too offtopic, do you have a link describing, or would you briefly summarize these problems? I am intrigued. And what would you suggest in its place? Fortran? Out of curiosity, what do you use D for given your views about the redundancy of C type languages for non-system programming? In a sense I could rightly be labelled a D dilettante. I had a possible startup a couple of years ago where we would have used D, but it never started. A side-effect was that I gave some support to David Simcha creating std.parallelism, but for the last couple of years my income has come from Python or Groovy/GPars with no real D activity. Would you consider D stable enough/suitable for general financial market work with development initially by a small underresourced team? Not ultra high frequency execution - at most legging in and managing longer term positions. But I am more interested in sentiment analysis, producing technical analysis indicators that summarize market activity across many different securities, some bond arb stuff. C++ just seems so ugly, and I feel uncomfortable only having python in the toolbox. D seems so far to be quite suitable...
Re: beginner's pyd question - exporting structs to python
Dr Russel Winder 41 Buckmaster Road London SW11 1EN, UK Are there any D users groups/meetups in London? I see you are not far away (I am in Barnes). Laeeth
building shared library from D code to import into cython
Hi.
I am trying to create a shared library in D linked against phobos
so that I may use this in a cython extension module for Python.
Ultimately I would like to be able to use a D class or struct
(via the C++ interface) and call it from within cython, since
cython classes cannot be instantiated without the gil (and this
prevents easy parallelisation).
I feel a bit foolish asking the question as there is a nice
example here for working with plain C using dmd as the linker,
and using dmd and gcc to create a DMD shared library statically
linked to phobos. However, I have not succeeded in creating a D
library statically linked to phobos that works with cython and
python,
http://dlang.org/dll-linux.html#dso7
I tried it first with test C code to make sure I am able to get
the C library/cython/Python interaction working.
pytest.c:
#include stdio.h
long pytest(long a)
{
return a+1;
}
int main()
{
long a =pytest(100);
printf(%ld,a);
return 0;
}
pytestpy.pyx:
cdef extern long pytest(long a)
cpdef pytestpy():
return pytest(109)
setup.py:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
setup(
ext_modules = cythonize([
Extension(pytestpy, [pytestpy.pyx],
libraries=[pytest],
)
]))
command line:
gcc -shared -o libpytest.so pytest.o
python setup.py build_ext -i
copied libpytest.so to /usr/local/lib
python
import pytestpy
pytestpy.pytestpy()
it works
now try pytest.d
import std.stdio;
extern (C) long pytest(long a)
{
return a*2;
}
void main()
{
auto a =pytest(100);
writefln(%d,a);
}
command line:
rm pytestd.o
rm libpytest.so
rm /usr/local/lib/libpytest.so
dmd -c pytest.d -fPIC
gcc -shared -o libpytest.so pytest.o -defaultlib=libphobos2.so
-L-rpath=/usr/local/lib
python
import pytestpy
Traceback (most recent call last):
File stdin, line 1, in module
ImportError: /usr/local/lib/libpytest.so: undefined symbol:
_D3std5stdio12__ModuleInfoZ
I guess it is not linking to the D runtime, but I am not sure
what I should be doing to fix.
Any thoughts appreciated.
(The next step I was going to try when this works was C++
interface vs importing as a Cython class, but I thought best to
start simple).
I am running this on 64 bit Fedora 20.
Thanks.
Laeeth.
Re: building shared library from D code to import into cython
Hi.
Thanks for the quick response.
The -defaultlib was left around from trying all kinds of
combinations of dmd and gcc. I am not used to gcc, and it will
take me some time to become properly acquainted with all the
options.
I simply could not get it to recognize libphobos no matter what
path settings I tried passing.
http://forum.dlang.org/thread/k3vfm9$1tq$1...@digitalmars.com?page=4
I don't know whether this was necessary, but I did manage to get
it to work after looking at a thread from a couple of years back.
I recompiled the D runtime and Phobos with position independent
code (PIC) and pointed everything to that version instead, and it
worked fine. (Just export PIC=True, rename the posix.mak to
Makefile, change the DMD path to /usr/sbin/dmd - in my case, and
it all worked).
Next step is to try calling D interface from Cython.
Laeeth
In case anyone else should struggle with this in future.
Makefile here:
PHOBOS_PATH=/opt/dmd2/src/phobos/generated/linux/release/64
pytestpy.so: pytest.d pytestpy.o
dmd -fPIC -c pytest.d #-shared -defaultlib=libphobos2.so
-L-rpath=.:${PHOBOS_PATH}
gcc -shared pytest.o pytestpy.o -o pytest.so -lphobos2 -lpthread
-lrt -L. -L${PHOBOS_PATH} -Wl,-rpath=.:${PHOBOS_PATH} -o
pytestpy.so -defaultlib=libphobos2.so -L-rpath=.:${PHOBOS_PATH}
pytest.o: %.d
dmd -c $
pytestpy.o: pytestpy.pyx
cython pytestpy.pyx
gcc -fPIC -c pytestpy.c -o pytestpy.o -I/usr/include/python2.7
clean:
rm -f pytestpy.c pytest.o pytestpy.o pytest.so
pytest.d
extern (C) long pytest(long a)
{
return a*2;
}
pytestpy.pyx
cdef extern long pytest(long a)
cpdef pytestpy():
return pytest(109)
print pytest(102)
setup.py
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
setup(
ext_modules = cythonize([
Extension(pytestpy, [pytestpy.pyx],
libraries=[pytest],
)
]))
On Tuesday, 7 October 2014 at 22:46:09 UTC, Freddy wrote:
On Tuesday, 7 October 2014 at 20:55:59 UTC, Laeeth Isharc wrote:
Hi.
I am trying to create a shared library in D linked against
phobos so that I may use this in a cython extension module for
Python. Ultimately I would like to be able to use a D class
or struct (via the C++ interface) and call it from within
cython, since cython classes cannot be instantiated without
the gil (and this prevents easy parallelisation).
I feel a bit foolish asking the question as there is a nice
example here for working with plain C using dmd as the linker,
and using dmd and gcc to create a DMD shared library
statically linked to phobos. However, I have not succeeded in
creating a D library statically linked to phobos that works
with cython and python,
http://dlang.org/dll-linux.html#dso7
I tried it first with test C code to make sure I am able to
get the C library/cython/Python interaction working.
pytest.c:
#include stdio.h
long pytest(long a)
{
return a+1;
}
int main()
{
long a =pytest(100);
printf(%ld,a);
return 0;
}
pytestpy.pyx:
cdef extern long pytest(long a)
cpdef pytestpy():
return pytest(109)
setup.py:
from distutils.core import setup
from distutils.extension import Extension
from Cython.Build import cythonize
setup(
ext_modules = cythonize([
Extension(pytestpy, [pytestpy.pyx],
libraries=[pytest],
)
]))
command line:
gcc -shared -o libpytest.so pytest.o
python setup.py build_ext -i
copied libpytest.so to /usr/local/lib
python
import pytestpy
pytestpy.pytestpy()
it works
now try pytest.d
import std.stdio;
extern (C) long pytest(long a)
{
return a*2;
}
void main()
{
auto a =pytest(100);
writefln(%d,a);
}
command line:
rm pytestd.o
rm libpytest.so
rm /usr/local/lib/libpytest.so
dmd -c pytest.d -fPIC
gcc -shared -o libpytest.so pytest.o -defaultlib=libphobos2.so
-L-rpath=/usr/local/lib
python
import pytestpy
Traceback (most recent call last):
File stdin, line 1, in module
ImportError: /usr/local/lib/libpytest.so: undefined symbol:
_D3std5stdio12__ModuleInfoZ
I guess it is not linking to the D runtime, but I am not sure
what I should be doing to fix.
Any thoughts appreciated.
(The next step I was going to try when this works was C++
interface vs importing as a Cython class, but I thought best
to start simple).
I am running this on 64 bit Fedora 20.
Thanks.
Laeeth.
Since when does gcc have a -defaultlib option?
---
$ man gcc | grep defaultlib
object-file-name -llibrary -nostartfiles -nodefaultlibs
This is useful when you use -nostdlib or
-nodefaultlibs but you do
-nodefaultlibs is used.
-nodefaultlibs
-nodefaultlibs is libgcc.a, a library of
Re: building shared library from D code to import into cython
Thanks for this. I am aware of pyd and will take a look at source/build process. Any thoughts on speed in 2014 of pyd vs using cython to talk to D directly via C/C++ interface? I saw this old coment here: prabhuramachandran.blogspot.co.uk/2008/09/python-vs-cython-vs-d-pyd-vs-c-swig predict that the D version's relative slowness might have something to do with Pyd's somewhat awful handling of arrays (at least in part). More low-level and verbose, but possibly faster code could be written to compensate for this if this is indeed the problem. However, this is not a very attractive solution. (Optimally, Pyd should be capable of directly pointing D arrays at numpy arrays, but this is not actually implemented.) On Friday, 10 October 2014 at 02:19:17 UTC, Ellery Newcomer wrote: On Wednesday, 8 October 2014 at 00:25:57 UTC, Laeeth Isharc wrote: Hi. Thanks for the quick response. The -defaultlib was left around from trying all kinds of combinations of dmd and gcc. I am not used to gcc, and it will take me some time to become properly acquainted with all the options. I managed to get it to compile with default dmd rpm: https://github.com/ariovistus/cythonic_d fedora 20, x86_64 Are you aware of pyd? (https://bitbucket.org/ariovistus/pyd) It knows how to build shared libraries, so I recommend you play with it, if only to watch how it does that. Biggest gotcha is starting up druntime.
Returning multiple arrays from function - struct or byref the only option?
Hi.
I have to write a bunch of functions that operate on input arrays
to return multiple output arrays.
In case helpful the inputs are price bars or economic data points
(datetime, ohlc) and the outputs are nx1 arrays (I won't say
vectors) of doubles or structs.
What is the best way to return multiple arrays in this kind of
situation. In Python I returned a tuple of numpy arrays, and the
C way would be to pass a pointer to the return destinations, and
I guess I could do the same in a D way by passing by ref not
value).
I see that I can return a struct containing dynamic arrays, and
return it (by value of course). I have read the forum discussion
a while back over how to return multiple values, and tried using
tuples. I don't see that I can return a tuple of arrays - am I
missing something?
Here is a simple case. Can I do better ?
Thanks.
import std.typecons;
import std.stdio;
struct RetStruct
{
double[] a;
double[] b;
}
RetStruct myfunction(double x)
{
RetStruct s;
double[] a,b;
a~=x+1.0;
a~=x+9.0;
b~=x+2.0;
b~=x+11.0;
s.a=a;
s.b=b;
return s;
}
void main()
{
writefln(%s,myfunction(99.0));
}
Re: Returning multiple arrays from function - struct or byref the only option?
Here is byref:
import std.typecons;
import std.stdio;
void myfunction(double x, ref double[] a, ref double[] b)
{
a~=x+1.0;
a~=x+9.0;
b~=x+2.0;
b~=x+11.0;
return;
}
void main()
{
double[] a;
double[] b;
myfunction(99.0,a,b);
writefln(%s %s,a,b);
}
Re: Returning multiple arrays from function - struct or byref the only option?
Thanks for the thoughts Meta and Ali.
Laeeth.
On Wednesday, 15 October 2014 at 17:56:06 UTC, Ali Çehreli wrote:
On 10/15/2014 09:48 AM, Laeeth Isharc wrote:
struct RetStruct
{
double[] a;
double[] b;
}
RetStruct myfunction(double x)
That's my preference. Tuples would work as well but they have
two minor issues for me:
- Unlike a struct, the members are anonymous. (Yes, tuples
members can have names as well but not when returning or
creating conveniently by 'return tuple(a, b)'.)
- Unlike Python, there is no automatic tuple expansion so one
has to refer to the members as result[0] and result[1], which
is less readable than struct members. (Yes, there is some
support for tuple expansion e.g. in foreach but it has some
issues with the automatic foreach loop counter.)
Summary: I return by struct. :)
Ali
Re: Issue with WIKI example Win32_DLLs_in_D
Funnily enough I was just playing with this last night trying to get Excel to talk to dlang DLL. I borrowed a C example elsewhere on web and used a different .def file. Something like this: LIBRARY dprop DESCRIPTION 'My DLL written in D' EXETYPE NT CODEPRELOAD DISCARDABLE DATAWRITE EXPORTS useArray = useArray usemyTest = usemyTest Where the two functions exported from D to excel were as above. Try that and let me know. If that doesn't work I will send you a link to from repository. Tuesday, 21 October 2014 at 19:59:51 UTC, Andre wrote: Hi, by copy and paste the example from http://wiki.dlang.org/Win32_DLLs_in_D exactly as described, the following errors is thrown: J:\Projects\Tests\Exampledmd test mydll.lib -g OPTLINK (R) for Win32 Release 8.00.15 test.obj(test) Error 42: Symbol Undefined _D5mydll12__ModuleInfoZ I created a batch program which executes the commands as described on the wiki page: dmd -c mydll -g dmd mydll.obj mydll.def -g -L/map C:\D\dmd2\windows\bin\implib /noi /system mydll.lib mydll.dll dmd test mydll.lib -g Could you check whats wrong with the example and maybe update the WIKI page? Thanks. Kind regards André
Dart bindings for D?
Hi. I have had a look around for these, but was not able to see them. It looks perhaps like dart_api.h is the main file to convert - I will have a crack at starting this unless anyone knows of any already in existence. Rationale for using Dart in combination with D is that I am not thrilled about learning or writing in Javascript, yet one has to do processing on the client in some language, and there seem very few viable alternatives for that. It would be nice to run D from front to back, but at least Dart has C-like syntax and is reasonably well thought out. Am I missing any existing bindings somewhere? Thanks. Laeeth.
Re: Dart bindings for D?
Hi. Thanks for all the thoughts, and sorry it has taken me a little while to reply. Adam - I liked your book very much: it really complemented the other resources out there, especially in communicating a refreshing spirit of enthusiasm and fearless exploration. ketmar - I took a look at script.d very briefly. I would love if I could use some kind of D-like and D-friendly scripting language on the client (Suliman has a point!), but my understanding is that I cannot if I want to use standard browsers without a plugin - Chrome, Firefox, etc. BTW what was the story behind dscript? It seems to have changed its name and no longer have so much connection to D: http://forum.dlang.org/thread/422be824.6030...@nospam.org It would be awesome to write front-end tools in D. However, there won't be much browser support unless you're backed by Google or Microsoft. Etienne - yes, indeed. That is exactly the problem. In theory your suggestion of compiling D to Javascript sounds intriguing, but I wonder if in practice it will be worth it on the client (I confess that you likely know much more about this than me). Even if interoperability with the ecosystem is possible, I suppose it will be clunkier to write in D compiled to JS than in Dart because there are fewer people pouring energy into the project to make it easy. I don't mind reimplementing some things on the back end in order to make it fast, beautiful, and efficient but I have no interest in re-writing anything in user interface domain. (Others may be different). As a second-best, but overall pretty appealing choice (superficially, since I haven't written anything substantial in it), Dart at least has a C-like syntax, seems to avoid most of the JS infelicities, and has some of the benefits of type-checking. It hasn't taken off yet, but my guess is it will. What kind of client are you doing? If you are writing a web page, you don't need any kind of script language API. JavaScript or dart or whatever talk with your server application through http requests or websockets, whereas script language APIs are meant for extending your application in the same process. For example, a text editor might have a script language to make custom functions for hotkeys. - Adam Ruppe Proprietary trading analytics and charting - still at an early stage so the design is not fully mapped out. I understand that if I run D on server and Dart on client then I don't need to worry about APIs. But 1) if I decide to run Dart on server and write my number-crunching analytics in D then I suppose I either need the Dart headers translated to D (API) or have the Dart web server talk to a D analytics server via a socket. And 2) I cannot say that I definitely do not want to cache things and do some work on client, and so I need to see what's possible to avoid getting trapped in purely local optimum. 3) It's a long way from being relevant, but ultimately realtime data licensing occurs at local user level for Bloomberg etc, so much better to be able to tap in to a data source the user already has. Dart makes most sense for internal web applications.- Ola Yes - exactly my situation. Dart VM is available as a standalone, which can be set up to act as a web server. But you want to integrate it into D? - Ola There are already C headers, and it is simple to wrap any C function manually so you can call it from Dart. Perhaps it would be possible to do something like PyD (I am still learning templates/CTFE so I don't know), but for now I was just thinking of converting the 3000 line odd (but mostly whitespace) header from .h to .d. Most of it's easy, but I struggled a bit with syntax for using alias in D to declare function pointer return types and parameters (I think I get it now). Why bother? D seems just perfect for the particular kind of quant financial work I want to do, but for the time being it is lacking on web server and client. Vibe.d is great, but is not a complete framework from what I have seen (which is fine, since it doesn't pretend to be one, but makes it harder if you don't want to spend your time on this). And we discussed the client above. So I need to have some way for Dart to talk to D. I don't think so, but integrating DartVM into D means you have to deal with two different garbage collectors or put a substantial amount of work into making D use the Dart collector. Thanks for the warning. I will have a low number of users, and whilst on occasion working data sets might be large, they won't stick around for very long so pre-allocating buffers should work fine (I hope). Thanks once again to everyone for all the suggestions and observations. Laeeth/ On Wednesday, 29 October 2014 at 22:12:32 UTC, Laeeth Isharc wrote: Hi. I have had a look around for these, but was not able to see them. It looks perhaps like dart_api.h is the main file to convert - I will have a crack at starting this
alias and extern(C)
Hi.
I am trying to translate the following from the Dart header:
typedef void (*Dart_MessageNotifyCallback)(Dart_Isolate
dest_isolate);
So I made a little simple test to understand callbacks in D. The
code below works fine if you remove the extern(C). But I get the
error functionpointertest.d(6): Error: basic type expected, not
extern with the code as it is.
How do I use alias with extern ?
Thanks.
Laeeth.
import std.stdio;
// test_typedef.d
//typedef void (*Dart_MessageNotifyCallback)(Dart_Isolate
dest_isolate);
alias Callback= extern(C) void function(int);
extern(C) void testfn(int i)
{
writefln(%s,i+1);
return;
}
extern(C) void testfn2(int i)
{
writefln(%s,i*i);
return;
}
void foo(Callback callback)
//void foo(void function(int) callback)
{
callback(100);
//callback(101);
}
void main()
{
foo(testfn);
foo(testfn2);
}
Re: alias and extern(C)
The code below works fine if you remove the extern(C). But I get the error functionpointertest.d(6): Error: basic type expected, not extern with the code as it is. How do I use alias with extern ? [...] alias Callback= extern(C) void function(int); Compiles as is with dmd 2.066. For 2.065 and older you have to put extern(C) in front of alias: extern(C) alias Callback= void function(int); Many thanks, anon. You are right - I was running older dmd on this machine. It works fine with your modification, as you suggest. Laeeth.
Re: Dart bindings for D?
Ah - makes sense. It is satisfyingly fast... On Thursday, 30 October 2014 at 21:33:59 UTC, ketmar via Digitalmars-d-learn wrote: On Thu, 30 Oct 2014 17:39:13 + Laeeth Isharc via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote: BTW what was the story behind dscript? It seems to have changed its name and no longer have so much connection to D: http://forum.dlang.org/thread/422be824.6030...@nospam.org we have rdmd now, which can be used in shebangs and allows to run D programs as shell scrips. it is able to track dependencies and caching compiled script, so 2nd and other invocations will be very fast.
simd and dmd compiler v 2.066 and 2.067.0-b1
Hi.
Not sure if my code is correct - I wanted to build the simplest
working example of simd use. The following compiles and works
under ldc (I have not disassessembled the result to see if it is
using simd instructions), but generates a compiler error under
dmd (2.066 and 2.067.0-b1 running under Fedora 20).
The only message is:
Internal error: backend/el.c 2874
Let me know if you think the code is okay, and if so I will see
if I can figure out how to generate a bug report (if it's not
something already known).
Thanks.
Laeeth.
import std.stdio;
import core.simd;
void main()
{
short8 vec;
foreach(i;0..8)
{
vec.ptr[i]=cast(short)i;
}
vec=3*vec;
foreach(i;0..8)
{
writefln(%s,vec.ptr[i]);
}
}
Re: simd and dmd compiler v 2.066 and 2.067.0-b1
On Monday, 3 November 2014 at 21:23:50 UTC, Marc Schütz wrote:
Reduced testcase:
import core.simd;
void main()
{
short8 vec;
vec=vec*3;
}
I've filed a bug report:
https://issues.dlang.org/show_bug.cgi?id=13674
Thanks - appreciate it. Laeeth.
rndtonl
what am I doing wrong here?
import std.math;
import std.stdio;
void main()
{
real fac;
fac=1.2;
fac=rndtonl(fac);
}
[root@fedorabox util]# dmd bug.d
bug.o: In function `_Dmain':
bug.d:(.text._Dmain+0x3b): undefined reference to `rndtonl'
collect2: error: ld returned 1 exit status
--- errorlevel 1
Re: rndtonl
Thanks, Adam. Should we perhaps make a pull to suggest updating the docs/wiki? As the point below is not what one would infer from the dlang.org library reference page. (If I say we, it's because I don't know what the protocol is, or whether my perception is right). On Tuesday, 4 November 2014 at 18:39:29 UTC, Adam D. Ruppe wrote: I think rndtonl is a C library function that isn't always present in the system. It doesn't work on my computer either and I can't find any documentation about it. It is probably not meant to be called by end users.
status of D optimizers benefiting from contracts ?
https://www.youtube.com/watch?v=e2F2pqeMLuwlist=PL4EvMyUrlAJmEfs8l6oW2BlnALiDu7kGy 31 minutes in, Walter Bright suggests that a supplementary benefit of using contrats is helping the compiler make optimisations. He uses the example of being able to do faster 32 bit arithmetic when the variables are longs but per contract within the bounds for 32 bits. I wondered to what extent D compilers are doing this already (peeking at the contract), and what plans if any there are to incorporate these in generating fast code. Thanks. Laeeth
Re: status of D optimizers benefiting from contracts ?
Thanks. Laeeth.
PyD-like wrapping for Excel/VBA and Julia?
I have a bunch of D functions I would like to make available to
Excel (and possibly Julia) without having to write wrappers for
each function individually.
For Excel, I think one needs two levels of wrapper - one is to
create a C style interface [using extern(Windows) calling
convention, and pointers to doubles or structs rather than
dynamic arrays], and the second is to write the VBA wrapper that
calls the C interface. (There may be more efficient purer ways
of doing this, but I don't wish to spend time learning Excel
internals/object models, and I know my route will work reasonably
well).
So a very simple D function:
double test(double[] inp, ref double[] oup)
{
double sum=0.0;
oup.length=inp.length;
foreach(i;0..inp.length)
{
oup[i]=inp[i]*inp[i];
sum+=oup[i];
}
return sum;
}
and my first attempt at a wrapper:
extern(Windows) double vbwrap_test(double* inp,size_t
num_inp,double* oup,size_t num_oup)
{
double[] arg_inp;
arg_inp.length=num_inp;
double[] arg_oup;
arg_oup.length=num_oup;
foreach(arg;0..num_inp)
{
arg_inp[arg]=inp[arg];
}
foreach(arg;0..num_oup)
{
arg_oup[arg]=oup[arg];
}
return test(arg_inp,arg_oup);
}
I didn't yet write the bit that copies the result from test back
to the calling double*.
Slowly learning metaprogramming/CTFE in D, and the code above was
generated from the function definition by some horrible looking D
code, ready to place into a string mixin. I need to make it more
general (to accept structs etc), and write the VBA wrapper
generation too.
But if anyone has any useful pointers or suggestions or would
like to help, do let me know. I guess this project could be of
broader application since in the financial and other sectors
people still are stuck with Excel as a front end in many cases,
for better or for worse.
I will look at LuaD and PyD and Adam's web.d for inspiration..
Julia was just something to think about further down the line. I
haven't used it much yet.
Thanks.
Laeeth.
Re: PyD-like wrapping for Excel/VBA and Julia?
On Friday, 19 December 2014 at 01:59:05 UTC, Ellery Newcomer
wrote:
On 12/18/2014 12:41 PM, Laeeth Isharc wrote:
I have a bunch of D functions I would like to make available
to Excel
(and possibly Julia) without having to write wrappers for each
function
individually.
I've thought about refactoring the reflection parts of pyd into
a reusable library for e.g. resurrecting RuD. Come to think of
it, that would probably be necessary for supporting pypy.
It'd be a heck of a lot of work, though.
For your wrapper, you can probably do something like
extern(Windows) double vbwrap_test(double* inp,size_t
num_inp,double* oup,size_t num_oup)
{
return test(inp[0 .. num_inp], arg_oup[0 .. num_oup]);
}
with .dup sprinkled in as you see fit. And you don't need to
explicitly copy the results back! Might need to take the ref
off oup in test..
Thanks for the pointers, Ellery.
What was RuD? RubyD?
There may be more efficient purer ways of doing this, but I
don't wish to spend time learning Excel internals/object
models, and I know my route will work reasonably well).
ActiveX is not internal to Excel. Being a generic component
interface, it's available over almost all microsoft technologies.
True, but how does that help me right a function I can call from
a spreadsheet? I had understood ActiveX was for scripting the
app, but not so useful if you want to write a function that can
be entered in a cell formula. So it seems to me that ActiveX
doesn't create an obvious way to avoid writing a C API wrapper
and then VBA wrapper around that.
optimization / benchmark tips a good topic for wiki ?
Replacing import core.stdc.math with import std.math in the D example increases the avg runtime from 19.64 to 23.87 seconds (~20% slower) which is consistent with OP's statement. + GDC/LDC vs DMD + nobounds, release Do you think we should start a topic on D wiki front page for benchmarking/performance tips to organize peoples' experience of what works? I took a quick look and couldn't see anything already. And it seems to be a topic that comes up quite frequently (less on forum than people doing their own benchmarks and it getting picked up on reddit etc). I am not so experienced in this area otherwise I would write a first draft myself. Laeeth
Data frames in D?
I thought about it once but quickly abandoned the idea. The primary reason was that D doesn't have REPL and is thus not suitable for interactive data exploration. The quick compile times could allow interactive data exploration I agree with other posters that a D REPL and interactive/visualization data environment would be very cool, but unfortunately doesn't exist. Batch computing is more practical, but REPLs really hook new users. I see statistical computing as a huge opportunity for D adoption. (R is just super-ugly and slow, leaving Python + its various native-code cyborg appendages as the hot new stats environment). There are tons of ways of accomplishing the same thing in D, but as far as I know there isn't a standard at this point. A statically typed dataframe is, at minimum, just a range of structs -- even more minimally, a bare *array* of structs, or alternatively just a 2-D array in a thin wrapper that provides access via column labels rather than indexes. You can manipulate these ranges with functions from std.range and std.algorithm. Missing or N/A data is a common issue, and can be represented in a variety of ways, with integers being the most annoying since there is no built-in NaN value for ints (check out the Nullable template from std.typecons). Supporting features like having *both* rows and columns are accessible via labels rather than indexes requires a little bit more wrapping. We have a NamedMatrix class at my workplace for that purpose. It's easy to overload the index operator [] for access, * for matrix multiplication, etc. CSV loads can be done with std.csv; unfortunately there's no corresponding support in that module for *writing* CSV (I've rolled my own). At my workplace we also have a MysqlConnection class that provides one-liner loading from a SQL query into minimalist, range-of-structs dataframes. Beyond that, it really depends on how you want to manipulate the dataframes. What specific things do you want to do? If you've got an idea, I could work up some sample code. So yes, there are people doing it in The Real World. Unfortunately my colleagues don't have a nice, tidy, self-contained DataFrame module to share (yet). But having one would be a great thing for D. The bigger problem though is matching the huge 3rd-party stats libraries (like CRAN for R). Since we do have an interactive shell (the pastebin), and now bindings and wrappers for hdf5 (key for large data sets) and basic seeds for a matrix library, should we start to think about what would be needed for a dataframe, and the best way to approach it, starting very simply? One doesn't need to have a comparable library to R for it to start being useful in particular use cases. Pandas and Julia would be obvious potential sources of inspiration (and it may be that one still uses them to call out to D in some cases), but rather than trying to just port pandas to D, it seems to make sense to ask how one should do it from scratch to better suit D. Laeeth.
Re: Data frames in D?
REPLs are over-hyped and have become a fashion touchstone that few dare argue against for fear of being denounced as un-hip. REPLs have their place, but in the main are nowhere near as useful as people claim. IPython Notebooks on the other hand are a balance between editor/execution environment and REPL that really has a lot going for it. Fair argument against an earlier poster but from my perspective, all I meant is that the absence of a shell is not a good reason to write off D for exploring data. Because there is a shell already that could be developed, and because one can call D from python / Julia in a notebook. Stats folks using R, love R and hate Python. Stats folk using Python, love Python and hate R. In the end it's all about what you know and can use to get the job done. To be frank (as in open rather than Jill), D hasn't got the infrastructure to compete with either R or Python and so is a non-starter in the data science arena. About the future you may or may not be right. (Whether it is commercially interesting to run workshops in D for stats people is certainly a interesting question. However given the ways that technology unfolds it may be that it is less relevant for the question I am most interested today in answering). I want to do things in D myself, and I would find a data frame helpful. I understand you don't program much in D these days, and that's a reasonable decision, but for those who want to use it to do quantish things with dataframes, perhaps we could think about how to approach the problem. And having weighed your warnings, if you have any suggestions on how best to implement this, I would be open to these also. Laeeth.
Data Frames in D - let's not wait for linear algebra; useful today in finance and Internet of Things
On Friday, 26 December 2014 at 21:31:00 UTC, aldanor wrote: On Wednesday, 25 September 2013 at 03:41:36 UTC, Jay Norwood wrote: I've been playing with the python pandas app enables interactive manipulation of tables of data in their dataframe structure, which they say is similar to the structures used in R. It appears pandas has laid claim to being a faster version of R, but is doing so basically limited to what they can exploit from moving operations back and forth from underlying cython code. Has anyone written an example app in D that manipulates dataframe type structures? Pandas has numpy as backend which does a lot of heavy lifting, so first things first -- imo D needs a fast and flexible blas/lapack-compatible multi-dimensional rectangular array library that could later serve as backend for pandas-like libraries. I don't believe I agree that we need a perfect multi-dimensional rectangular array library to serve as a backend before thinking and doing much on data frames (although it will certainly be very useful when ready). First, it seems we do have matrices, even if lacking in complete functionality for linear algebra, and the like. There is a chicken and egg aspect in the development of tools - it is rarely the case that one kind of tool necessarily totally precedes another, and often complementarities and dynamic effects between different stages. If one waits till one has everything one needs done for one, one won't get much done. Secondly, much of the kind of thing Pandas is useful for is not exactly rocket science from a quantitative perspective, but it's just the kinds of thing that is very useful if you are thinking about working with data sets of a decent size.The concepts seem to me to fit very well with std.algorithm and std.range, and can be thought of as just as way to bring out the power of the tools we alreaady have when working with data in the world as it is. See here for an example of just how simple. Remember Excel pivottables? http://pandas.pydata.org/pandas-docs/stable/groupby.html Thirdly, one of the reasons Pandas is popular is because it is written in C/Cython and very fast. It's significantly faster than Julia. One might hit roadblocks down the line when it comes to the Global Interpreter Lock and difficulty of processing larger sets quickly in Python, but at least this stage is fast and easy. So people do care about speed, but they also care about the frictions being taken away, so that they can spend their energies on addressing the problem at hand. Ie a dataframe will be helpful, in my view. Processing of log data is a growing domain - partly from internet, but also from the internet of things. See below for one company using D to process logs: http://venturebeat.com/2014/11/12/adroll-hits-gigantic-130-terabytes-of-ad-data-processed-daily-says-size-matters/ http://tech.adroll.com/blog/data/2014/11/17/d-is-for-data-science.html A poster on this forum is already using D as a library to call from R (from Reddit), which brings home the point that it isn't necessary for D to be able to do every part of the process for it to be able to take over some of the heavy work. [–]bachmeier 6 points 1 month ago I call D shared libraries from R. I've put together a library that offers similar functionality to Rcpp. I've got a presentation showing its use on Linux. Both the presentation and library code should be made available within the next couple of days. My library makes available the R API and anything in Gretl. You can allocate and manipulate R objects in D, add R assert statements in your D code, and so on. What I'm working on now is calling into GSL for optimization. These are all mature libraries - my code is just an interface. It's generally easy to call any C library from D, and modern Fortran, which provides C interoperability, is not too much harder. See here, for just one use case in the internet of things. They don't use D, but maybe they should have. And it shows an example where perhaps at least log processing could easily be handled by what we have with a few small additional data structures - even if people use outside libraries for the machine learning part. http://www.forbes.com/sites/danwoods/2014/11/04/how-splunk-caught-wall-streets-eye-by-taming-the-messy-world-of-iot-data/3/ By using Splunk software, Hrebek said that his division’s leader product is able to offer customers a real-time view of operations on a train and to use machine learning to suggest optimal strategies for driving trains along various routes. Just shaving a small percentage off of fuel costs can mean huge savings for a railroad. Why Doesn’t BI Work for the IoT? In both of the use cases just mentioned, for years, existing business intelligence technology had been applied to the problem of making sense of the data with little success. The problem is not that that it is impossible to
Re: Data Frames in D - let's not wait for linear algebra; useful today in finance and Internet of Things
Russell: I think we are agreeing. Very lightweight editor and executor of code fragments is as good, if not better, that the one line REPL. Yes - the key for me is that the absence of a shell is by no means a reason to say that D is not suited to this task. One may wish to refine what exists, but that is another question entirely. Part of the problem here is tribalism. Most data science people want to use the same tools that other data science people use, even though the issue is to differentiate themselves. Yes - we are answering two different questions. I could not care less about persuading anyone en masse in a broad sector, those who think of themselves as being 'data scientists' included. It's silly, in my view, to think of it as an established field very distinct from others, and with a fixed way of doing things. If for no other reason that things are in flux and the sector is growing quickly, which means that there is room for many different approaches, and it is premature to think the popularity of approach X or Y today means that approach 'D' can't be productive tomorrow. But as I said, I am less convinced in persuading anyone, and rather more concerned with getting a basic data frame in D up and running because I could certainly use it, and the hard work has been done already. The basics should be an evening's work for an advanced D hacker, but it will probably take me longer than that. In any case, since nobody else has come forward, I will keep working away at it. A BLAS library is certainly a precusor, as is very good data visualization tools, graphs, diagrams etc. Perhaps a prerequisite to D being seen as a contender, but I don't see how it's a prerequisite just to have a dataframe, which is really a very simple yet incredibly useful thing. Go has masses of people putting a lot of effort into Web. It's not the ideas, it's the number of people getting on board and doing things. Also about the quality of the people. (I have no view about Go, but have a very positive view on D). When things get big there is a danger they get cluttered. That's one blessing for D. To get some traction in any of these areas, finance data analysis and model building, or systems activity, it is all about people doing it, publicizing it and making things available for others to use. Yes - so do you have any thoughts on what a data frame structure should look like? I am trying to do and after that will make available. But it needs to be better than Julia in some way that makes others sit up and take notice. There has to be the ability to create some hype. Don't care ;) This concept of what is your edge is not my cup of tea because I do not see the world in those terms. Something of high quality that's highly productive will over time stand a decent chance of becoming more widely adopted, whereas trying to force it into some kind of marketing framework can prove counterproductive. Right now, the main thing I care about is solving the problem at hand, because if it solves my problem well then I am pretty sure it will be useful to others too, and be so better than if one had adopted a more consciously 'commercial/marketing' mindset. I would post the dataframe skeleton here, but it's too embarassing right now and want to read the std.variant library to see what tricks I can learn. (A data series seems kind of like a variant, but with every cell the same type). Obviously in some cases the data frame type is defined at compile time, like a struct, and that's easy. But if you are loading from a file you need to be able to have dynamic typing for the column. I don't believe I agree that we need a perfect multi-dimensional rectangular array library to serve as a backend before thinking and doing much on data frames (although it will certainly be very useful when ready). Also, if there is a ready made C or C++ library that can be made use of, do it. Well, the hard parts of arrays themselves (and it's not that fiendishly hard, I would think) seem to need to be tightly integrated with the language, so I don't see how a C/C++ library will help so much. For the linear algebra, yes... hyping it up. I recently discovered a number of hedge funds work solely on moving average based algorithmic trading. NumPy, SciPy and Pandas all have variations on this basic algorithm. Well, having worked for more or less quanty hedge funds since 98, I would think it unlikely that anyone depends only on moving averages although basic old-school trend-following certainly does work - it is just a hard sell to herding institutional investors, and does not fit very well with the concept of a 'career'. (You have to be able to see the five years of subdued returns since 2009 as just part of the cycle, which indeed may be the correct view when one sees markets as a natural phenomenon, but is not the view of asset allocators, or talented
Re: Data Frames in D - let's not wait for linear algebra; useful today in finance and Internet of Things
On Saturday, 27 December 2014 at 16:41:04 UTC, Russel Winder via Digitalmars-d-learn wrote: On Sat, 2014-12-27 at 15:33 +, Laeeth Isharc via Digitalmars-d-learn wrote: […lots of agreed uncontentious stuff :-) …] You write as if Christensen's book The Innovator's Dilemma had never been written, and nor had it been a standard textbook in business schools for some years. You may have good arguments as to why he is wrong, or why it doesn't apply to D, but you haven't set them out, as far as I am aware. In the post-production world as I know it (Nuke, etc.) The C++/Python combination has never failed to be adequate to the innovation demanded by film makers. In the image processing world the C++/Lua combination has never failed to adapt to the innovation needed by photograph tinkerers. My point was really that the customers have never found an innovative need that the extant platforms couldn't provide. I felt this was somewhat different to the Christensen argument. On the other hand, I may have missed the point… No matter how plugged in a person may be, it is impossible to be aware of everything that is going on, especially in exactly the kind of domains Christensen talks about - ones that aren't by any standard important in a spot sense to the bigger picture, but that critically provide a quiet relatively uncontested niche for the seeds of something to unfold until it is ready to break out into the broader world. So I think the point is that one shouldn't be bothered one jot by the disinclination of the people you know to want to use D, particularly since you are so plugged in to all these other worlds (and being an insider in a sense that matters today has an opportunity cost because it means one is not spending time and attention speaking to non insiders as much at that instant). New growth will come from the fringes. I think one should be very worried if the Adam Ruppe of the world would start to say D sucks - nice idea, but just not expressive enough for me, and I am switching back to Ruby and Python. Because that would indicate a loss of ground in the home niche. But somehow I don't think so...! And meantime quietly things continue to develop. What matters is not the challenges one faces, but how one deals with them. An outpouring of frustration in recent days, and the result is we are going to get better docs, better examples, and who knows what else. That's a sign of health. Will post code I have in a few days. Laeeth.
Re: Data Frames in D - let's not wait for linear algebra; useful today in finance and Internet of Things
On Monday, 29 December 2014 at 04:08:58 UTC, Vlad Levenfeld wrote: Laeeth - I am not sure exactly what your needs are but I have a fairly complete solution for generic multidimensional interfaces (template-based, bounds checked, RAII-ready, non-integer indices, the whole shebang) that I have been building. Anyway I don't want to spam the forum if I've missed the point of this discussion, but perhaps we could speak about it further over email and you could give me your opinion? I'm at vlevenf...@gmail.com Hi Vlad. Thanks v much for getting in touch. Your work sounds very interesting. I will drop you a line in coming days. Happy new year. Laeeth.
Re: Passing string literals to C
Argh - no way to edit.
What's best practice here?
D strings are not null-terminated.
===
cpling.c
char* cpling(char *s)
{
s[0]='!';
return s;
}
===
dcaller.d
extern(C) char* cpling(char* s);
void callC()
{
writefln(%s,fromStringz(cpling(hello\0)));
}
or
void callC()
{
writefln(%s,fromStringz(cpling(toStringz(hello;
}
===
am I missing a better way to do this?
Passing string literals to C
What's best practice here?
D strings are not null-terminated.
char* cpling(char *s)
{
So toString(This i
Checking C return results against NULL
Am I missing a more agreeable way to check the return value of a C function against NULL. It's fine if it's a char*, but if it returns a pointer to some kind of struct, one has to go through and convert each instance of NULL to a cast of the appropriate return type. Eg cast(funnystruct*)0 if ((tomime_fields.fld_description==cast(char*)0)) throw new Exception(show_part_info: description is NULL);
Re: Passing string literals to C
Thanks for the help. Laeeth
Re: Checking C return results against NULL
Thanks for the help. Laeeth
Re: Can the order in associative array change when keys are not midified?
On Thursday, 1 January 2015 at 18:58:04 UTC, Andrej Mitrovic via Digitalmars-d-learn wrote: On 1/1/15, Tobias Pankrath via Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote: You could implement an OrderedMap!(Key, Value) via RedBlackTree!(Tuple!(Key, Value), (a,b) = a[0] b[0]). We could add this as an alias into Phobos or perhaps as just a documentation line on the website. V good point. This is one of those small frictions that cumulatively raises the cost of adoption for newcomers who are not by nature hackers but want to get stuff done, and yet could be valuable members of the D community over time. For whatever reason, there is some resistance for many people to ask for help in a forum and it is easy to be overwhelmed and give up. Modern people don't have much tolerance for pain, even though that's an essential part of learning and being a developer. And compare python where you have OrderedDict and a search will find many examples of how to use it. I agree with comments about readability of library functions, but maybe also worth extending hints and tips for common patterns / code fragments. What is the protocol for contributing to the wiki? Can one just edit it ?
Re: simple question about using dub to import CyberShadow's aeutils
On Saturday, 3 January 2015 at 12:08:16 UTC, Vladimir Panteleev wrote: On Saturday, 3 January 2015 at 11:58:48 UTC, Laeeth Isharc wrote: import ae.utils; ae.utils is a package, perhaps you meant to import ae.utils.xml? aha. schoolboy error on my part. thank you for your help, and especially for writing such a useful library and sharing it with the world. Laeeth.
simple question about using dub to import CyberShadow's aeutils
Hi.
I would like to use the XML parser from CyberShadow's ae.utils -
I am building a tool to index RSS feeds in elasticsearch
(something like rssriver but with more complete functionality).
I am using dub to build the code.
So far I just have an empty boilerplate app.d with the line
import ae.utils;
Using the suggestion from code.dlang.org:
{
...
dependencies: {
ae: =1.0.1
}
}
It cannot find ae/utils.d If I try replacing the = with or
just ~master it still does not build. Error text below this
message. I suppose this might be because dub is designed to
import a single package, and not just part of the hierarchy.
But if I change the import to just plain import ae, I still have
problems.
What am I doing wrong? I could just copy the file over, but I
would like to try to do it properly.
Do let me know if you do not see this as the right forum to ask
about dub.
Thanks.
Laeeth.
Selected package openssl 1.0.0+1.0.0e does not match the
dependency specification in package ae (=1.1.3+1.0.1g
(optional)). Need to dub upgrade?
Target vibe-d 0.7.21 is up to date. Use --force to rebuild.
Building rsselastic ~master configuration application, build
type debug.
Compiling using dmd...
source/app.d(1): Error: module utils is in file 'ae/utils.d'
which cannot be read
import path[0] = source/
import path[1] = ../../../root/.dub/packages/ae-1.0.1
import path[2] = ../../../root/.dub/packages/openssl-1.0.0_1.0.0e
import path[3] = ../../../root/.dub/packages/vibe-d-0.7.21/source/
import path[4] = ../../../root/.dub/packages/libevent-2.0.1_2.0.16
import path[5] = /usr/include/dmd/phobos
import path[6] = /usr/include/dmd/druntime/import
FAIL
.dub/build/application-debug-linux.posix-x86_64-dmd_2067-6DD7C73322D93ABC7BBACEAF4B4E0E3D/
rsselastic executable
Error executing command run: dmd failed with exit code 1.
===
Selected package openssl 1.0.0+1.0.0e does not match the
dependency specification in package ae (=1.1.3+1.0.1g
(optional)). Need to dub upgrade?
Target vibe-d 0.7.21 is up to date. Use --force to rebuild.
Building rsselastic ~master configuration application, build
type debug.
Compiling using dmd...
source/app.d(1): Error: module ae is in file 'ae.d' which cannot
be read
import path[0] = source/
import path[1] = ../../../root/.dub/packages/ae-1.0.1
import path[2] = ../../../root/.dub/packages/openssl-1.0.0_1.0.0e
import path[3] = ../../../root/.dub/packages/vibe-d-0.7.21/source/
import path[4] = ../../../root/.dub/packages/libevent-2.0.1_2.0.16
import path[5] = /usr/include/dmd/phobos
import path[6] = /usr/include/dmd/druntime/import
FAIL
.dub/build/application-debug-linux.posix-x86_64-dmd_2067-6DD7C73322D93ABC7BBACEAF4B4E0E3D/
rsselastic executable
Error executing command run: dmd failed with exit code 1.
Other libraries - web site link, and other thoughts
At the moment it goes straight go code.dlang.org, which may be a bit overwhelming if you have just arrived at dlang.org and want to see what it can do. Is it worth changing to the library wiki write up page on libraries? And making sure link to code.dlang.org is prominent, saying code.dlang.org is a registry of D libraries hosted on github. There are currently XYZ libraries, of which ABC are under active development. At the moment, the list of libraries on the wiki is fairly complete, but not oriented towards answering the question of how do I do stuff. It would be nice to have a set of pages on this that mines the gold from the forums and makes it easy to find. Eg how do I navigate html or xml?. Use dom.d or ae, and here are some simple programs. Compare python community doc situation to ours. (We are leaner, but that doesn't mean we can't do a better job), Also say that if you cannot find the library you need, you should know that it is easy to interface to all C libraries (click for details), and that it is possible and fairly easy to interface to many C++ libraries (click here). p0nce and others are doing great work on D idioms, but what is missing is pointers to examples of larger programs. I know it is common sense if you are experienced to read the source of other projects to learn, but is it worth making this easy by having in the section on learning D some pointers to some model D code bases. The FUD crowd on slashdot always brings up the Tango / Phobos split, but this is not prominently addressed on the web site. Should this not be answered in the main FAQ? The compilation speed of dmd (and somewhat ldc) is a major advantage. It is stunning in outright terms, and almost inconceivable if you come from the world of c++. It really matters because it allows you to shift styles to iterative development. So it should be featured on the front page, and there should be quantitative benchmarks (because that's what modern people like) comparing dmd to other language compilers as well as pointing out how quickly phobos compiles. A YouTube video may make it more concrete. And there should be quotes from authoritative users of D talking about how useful they find quick compilation and iterative development. People are lazy. Or more charitably, not wealthy in terms of attention and time in the modern world. The documentation is often written from the point of view of the compiler and library writer, and this is not the point of view of the user, who wants to know what he can do, and how he can do it. In the interfacing to C tutorial, how about starting off by getting to the point. Have the following section at the beginning, and maybe even make it crisper by saying that D can utilise all existing C libraries, and all that needs to be done is to translate the header files to D. Using Existing C Libraries Since D can call C code directly, it can also call any C library functions, giving D access to the smorgasbord of existing C libraries. To do so, however, one needs to write a D interface (.di) file, which is a translation of the C .h header file for the C library into D. For popular C libraries, the first place to look for the corresponding D interface file is the Deimos Project. If it isn't there already, and you write one, please contribute it to the Deimos Project. --- In the spirit of accommodating peoples' attention poverty, why not say at the top that there are translation tools for objective C and C. Click here to download dstep binaries (linux / win etc), and here to go to the github dstep site. I confess that I struggled for ages unsuccessfully to compile dstep on my platform without realising that the binaries were available for download. Similarly for C++, the web site starts off completely on the wrong note. While D is fully capable of interfacing to C, its ability to interface to C++ is much more limited. There are three ways to do it: I hear much more limited, and think I am not going to bother with this (if I have not yet committed to the language). Whereas in actual fact, as Walter pointed out in his talk, nobody else can interface with C++ and it is a marvel to be able to do so at all, let alone in a way that is much more complete than the docs suggest. And it sounds like this phrasing is in any case outdated. Change to something like: Interfacing to C++ is a challenging endeavour for any language, but it is a challenge that must be met in order to take advantage of the large set of existing libraries and code bases in C++. It is therefore a priority for 2015 to implement an ability to link to a set of C++ that will be complete for most use cases. Walter Bright spoke recently at Microsoft on this topic, and the reader may find the following forum threads of interest. As things stand, the following features are complete:..., whilst there are limitations with respect to:
Re: London D Programmers MeetUp
Thank you, Kingsley, for a very well organized and thought-through event. I was impressed by the calibre of people that attended, and look forward to attending future meetups.
Re: London D Programmers MeetUp
One interesting anecdote: somebody in a financial services company gave an account of giving D a try as a way to prototype something quickly, intending to rewrite it later in a more conventional language. The prototype went straight into production, and they are happy with it. The C interop was important for them to have made this possible. It was interesting, because it is a serious company where people don't by any means have a hacker mentality, and it was one more point at variance with the I am a java programmer but do all my personal projects in D narrative, and also because this particular kind of company is exactly the sort of second wave early adopter one wants to have. (I hope I represented this accurately, and tried to make this as vague as possible whilst still keeping it useful since people might not want the details of what they do being public).
Re: London D Programmers MeetUp
On Thursday, 5 February 2015 at 12:06:31 UTC, Russel Winder wrote: On Wed, 2015-02-04 at 21:35 +, Iain Buclaw via Digitalmars-d-announce wrote: […] That pretty much sums up my understanding of it too. I recollect the banter including (paraphrased): if the prototype works, just put it into production, it's standard practice. Shhh... We have investors to keep happy.
Re: London D Programmers MeetUp
On Thursday, 5 February 2015 at 20:40:52 UTC, Laeeth Isharc wrote: On Thursday, 5 February 2015 at 12:06:31 UTC, Russel Winder wrote: On Wed, 2015-02-04 at 21:35 +, Iain Buclaw via Digitalmars-d-announce wrote: […] That pretty much sums up my understanding of it too. I recollect the banter including (paraphrased): if the prototype works, just put it into production, it's standard practice. Shhh... We have investors to keep happy. half-jest
Re: D Beginner Trying Manual Memory Management
I don't think you've read h5py source in enough detail :) You're right - I haven't done more than browsed it. It's based HEAVILY on duck typing. There is a question here about what to do in D. On the one hand, the flexibility of being able to open a foreign HDF5 file where you don't know beforehand the dataset type is very nice. On the other, the adaptations needed to handle this flexibly get in the way when you are dealing with your own data that has a set format and where recompilation is acceptable if it changes. Looking at the 'ease' of processing JSON, even using vibed, I think that one will need to implement both eventually, but perhaps starting with static typing. In addition, it has way MORE classes than the C++ hierarchy does. E.g., the high-level File object actually has these parents: File : Group, Group : HLObject, MutableMappingWithLock, HLObject : CommonStateObject and internally the File also keeps a reference to file id which is an instance of FileID which inherits from GroupID which inherits from ObjectID, do I need to continue? Okay - I guess there is a distinction between the interface to the outside world (where I think the h5py etc way is superior for most uses) and the implementation. Is not the reason h5py has lots of classes primarily because that is how you write good code in python, whereas in many cases this is not true in D (not that you should ban classes, but often structs + free floating functions are more suitable). PyTables, on the contrary is quite badly written (although it works quite well and there are brilliant folks on the dev team like francesc alted) and looks like a dump of C code interweaved with hackish Python code. Interesting. What do you think is low quality about the design? In h5py you can do things like file[/dataset].write(...) -- this just wouldn't work as is in a strictly typed language since the indexing operator generally returns you something of a Location type (or an interface, rather) which can be a group/datatype/dataset which is only known at runtime. Well, if you don't mind recompiling your code when the data set type changes (or you encounter a new data set) then you can do that (which is what I posted a link to earlier). It depends on your use case. It's hard to think of an application more dynamic than web sites, and yet people seem happy enough with vibed's use of compiled diet templates as the primary implementation. They would like the option of dynamic ones too, and I think this would be useful in this domain too, since one does look at foreign data on occasion. One could of course use the quick compilation of D to regenerate parts of the code when this happens. Whether or not this is acceptable depends on your use case - for some it might be okay, but obviously it is no good if you are writing a generic H5 browser/charting tool. So I think if you don't allow static dataset typing it means the flexibility of dynamic typing gets in the way for some uses (which might be most of them), but you need to add dynamic typing too. Shall we move this to a different thread and/or email, as I am afraid I have hijacked the poor original poster's request. On the refcounting question, I confess that I do not fully understand your concern, which may well reflect a lack of deep experience with D on my part. Adam Ruppe suggests that it's generally okay to rely on a struct destructor to call C cleanup code. I can appreciate this may not be true with h5 and, if you can spare the time, I would love to understand more precisely why not. Out of all of them, only the dataset supports the write method but you don't know it's going to be a dataset. See the problem? In this case I didn't quite follow. Where does this fall down ? void h5write(T)(Dataset x, T data) I have your email somewhere and will drop you a line. Or you can email me laeeth at laeeth.com. And let's create a new thread. Laeeth.
Re: idiomatic D: what to use instead of pointers in constructing a tree data structure?
On Tuesday, 13 January 2015 at 17:41:53 UTC, Tobias Pankrath wrote: On Tuesday, 13 January 2015 at 17:19:42 UTC, Laeeth Isharc wrote: The GC is allowed to move structs around, as I undestand it. Under what circumstances do I get into trouble having a pointer to them? None, a GC that moves structs around must update every pointer afterwards and as far as I know, the standard GC doesn't do that (moving things around). You may not have a pointer inside a struct that points to the struct itself. This allows the compiler to elide some copies. Got it - thanks.
Re: vibe.d Subdirectory?
On Thursday, 15 January 2015 at 14:38:16 UTC, Steven Schveighoffer wrote: On 1/14/15 11:31 AM, Laeeth Isharc wrote: To be very clear: in the simple case when you compile your vibe application from multiple source files and diet templates etc, and you will end up with an executable. This can act as a server directly, or you can make it an internal server on localhost called by php, or you can call it from nginx as a CGI type application depending on the URL. Pardon me for asking kind of a side question, but is there any online information on how to do this? I am contemplating migrating my in-house php-based application to vibe.d, but I wondered if I have to do it whole-sale (not much gratification until the whole thing is done) or piecemeal (then some parts are php, some are vibe.d, until whole thing is ported). I thought I could do the latter, but I don't know how to interact apache/php with vibe.d. The biggest question I have is how vibe.d and php can share session data. But I also have no idea which server should be in front, and how to call one another ;) It may be that you have a reason for going with apache, but in case you have not looked into it (or the situation has changed since you last looked), you might consider nginx as a front end given better stability and performance. If you make D read the PHP authentication then you need to write a new auth scheme when you have gotten rid of PHP. So it might be worth at some point considering moving this to an external service called by both D and PHP when you can do so without breaking anything. This might be worth a read: http://merbist.com/2012/04/04/building-and-implementing-a-single-sign-on-solution/
Re: Getting a safe path for a temporary file
On Saturday, 17 January 2015 at 13:47:39 UTC, Marc Schütz wrote: Is it currently possible to get the path to a safe temporary file, i.e. one that is guaranteed to be freshly created and will not override another existing file? There's `std.file.tempDir`, which doesn't create a unique file. Then there's `std.stdio.tmpfile()`, which does, but it returns a `File` object, and its `name` property is `null`. Did I miss something? IMO this is very import functionality. One use case is passing these names as command line arguments to an external program that doesn't support stdin/stdout. I agree that it would be useful. This is what I used, although there may be a better option: http://dlang.org/phobos/std_uuid.html
Re: Binutils 2.25 Released - New D demangling support
How do I get it? --- The release itself is a source package, however a safer choice is to get the release binaries through your Linux distributor. Fortunately, there have been distributions who have been shipping it as early as three weeks ago. In case it saves someone else a few minutes: for Fedora 21 (and probably others). (At your own risk if new version of binutils breaks something else). yum install fedora-repos-rawhide yum install binutils --enablerepo rawhide objdump -v
Re: Binutils 2.25 Released - New D demangling support
yum install fedora-repos-rawhide yum install binutils --enablerepo rawhide objdump -v sorry. should be: yum install fedora-release-rawhide yum install binutils --enablerepo rawhide objdump -v
Re: Getting a safe path for a temporary file
On Saturday, 17 January 2015 at 16:55:42 UTC, Marc Schütz wrote: On Saturday, 17 January 2015 at 14:37:00 UTC, Laeeth Isharc wrote: On Saturday, 17 January 2015 at 13:47:39 UTC, Marc Schütz wrote: Is it currently possible to get the path to a safe temporary file, i.e. one that is guaranteed to be freshly created and will not override another existing file? There's `std.file.tempDir`, which doesn't create a unique file. Then there's `std.stdio.tmpfile()`, which does, but it returns a `File` object, and its `name` property is `null`. Did I miss something? IMO this is very import functionality. One use case is passing these names as command line arguments to an external program that doesn't support stdin/stdout. I agree that it would be useful. This is what I used, although there may be a better option: http://dlang.org/phobos/std_uuid.html Nice idea, but it still allows for intentional collision attacks :-( The only really safe solution is one that generates (probably) unique names, then opens the file with O_EXCL|O_CREAT (or whatever other means the OS provides), and if it fails, retries with a different name. `std.stdio.tmpfile()` already does that (it uses `tmpfile(3)` under the hood), but doesn't allow access to the name. I don't follow why a collision attack is applicable in this case. Your stage 1 of generating unique names: how is this different from using a random uuid?
Re: casting SysTime to ubyte[]
I really wouldn't advise doing that. SysTime contains a long which represents the time in hnsecs since midnight, January 1st, 1 A.D., and that could be written to a file quite easily. But it also contains a reference to a TimeZone object, so what you're doing would just be writing its address to disk, which wouldn't do you any good at all, since that's specific to each run of the program, even assuming that the object exists in both runs of the program (which it would for UTC or LocalTime but not for user-constructed time zones). So, writing the stdTime (horrible name, I know) property to disk would work just fine (that's the hnsecs as a long), but you're going to have to do something smarter than that if you want to retain the time zone. And you're not going to want to try and simply cast a SysTime to a ubyte[] and do anything practical with that regardless. - Jonathan M Davis Thanks for this. I still with my C habits had the idea the time would just be a flat struct. So in this case better to write it as a string, which is what I have already done. I just wondered why the other approach didn't work, and now I understand.
Re: vibe.d Subdirectory?
Actually I want to serve some JSON packed weather data (heck I also wrote my Global Climate Model partially in D and in C) - so I guess, I can use vibe.d to build a cgi. Cool. Do you incorporate the influence of solar activity via galactic rays / cloud formation and via volcanic activity? (This is not my field, but I am looking for a model that does). However I also want to recieve some userinput, and parse it - and sometimes, php is simpler, because for a very small parsing task you dont want to worry abotu types and all. Does vibe.d support any HTML Preprocessor in the first place? If not PHP then Neco? Falcon? So that I do not always have to access my vibe.d based app, but I can simply change the page itself a bit? It won't be quicker to write the parsing code in D if you are new to the language, but it is still pretty quick when you know how. There are regexes etc too. What Ketmar says. For a robust application it might be better to put it behind a proxy like nginx anyway. In which case you can use your vibed application to serve up parts of your site and php for others. Eg user submits a form and the result is posted to a url that nginx routes to your vibed app. You could dispense entirely with the web server and do the whole thing in vibed, but you don't need to. Although its a framework, you don't have to do much to turn it into a web server if you look at the documentation on the vibed page. The forum there is also quite helpful. Laeeth
Re: D Beginner Trying Manual Memory Management
In the hierarchy example above (c++ hdf hierarchy link), by
using UFCS to implement the shared methods (which are achieved
by multiple inheritance in the c++ counterpart) did you mean
something like this?
// id.d
struct ID { int id; ... }
// location.d
struct Location { ID _id; alias _id this; ... }
// file.d
public import commonfg; // ugh
struct File { Location _location; alias _location this; ... }
// group.d
public import commonfg;
struct File { Location _location; alias _location this; ... }
// commonfg.d { ... }
enum isContainer(T) = is(T: File) || is(T : Group);
auto method1(T)(T obj, args) if (isContainer!T) { ... }
auto method2(T)(T obj, args) if (isContainer!T) { ... }
I guess two of my gripes with UFCS is (a) you really have to
// another hdf-specific thing here but a good example in
general is that some functions return you an id for an object
which is one of the location subtypes (e.g. it could be a File
or could be a Group depending on run-time conditions), so it
kind of feels natural to use polymorphism and classes for that,
but what would you do with the struct approach? The only thing
that comes to mind is Variant, but it's quite meh to use in
practice.
Void unlink(File f){}
Void unlink(Group g){}
For simple cases maybe one can keep it simple, and despite the
Byzantine interface what one is trying to do when using HDF5 is
not intrinsically so complex.
Re: D Beginner Trying Manual Memory Management
struct File { Location _location; alias _location this; ... }
// group.d
public import commonfg;
struct File { Location _location; alias _location this; ... }
// commonfg.d { ... }
enum isContainer(T) = is(T: File) || is(T : Group);
auto method1(T)(T obj, args) if (isContainer!T) { ... }
auto method2(T)(T obj, args) if (isContainer!T) { ... }
I guess two of my gripes with UFCS is (a) you really have to
// another hdf-specific thing here but a good example in
general is that some functions return you an id for an object
which is one of the location subtypes (e.g. it could be a
File or could be a Group depending on run-time conditions),
so it kind of feels natural to use polymorphism and classes
for that, but what would you do with the struct approach? The
only thing that comes to mind is Variant, but it's quite meh
to use in practice.
Void unlink(File f){}
Void unlink(Group g){}
For simple cases maybe one can keep it simple, and despite the
Byzantine interface what one is trying to do when using HDF5
is not intrinsically so complex.
So your solution is copying and pasting the code?
But now repeat that for 200 other functions and a dozen more
types that can be polymorphic in weirdest ways possible...
If you are simply have a few lines calling the API and the
validation is different enough for file and group (I haven't
written unlink yet) then why not (and move proper shared code out
into helper functions). The alternative is a long method with
lots of conditions, which may be the best in some cases but may
be harder to follow.
I do like the h5py and pytables approaches. One doesn't need to
bother too much with the implementation when using their library.
However, what I am doing is quite simple from a data perspective
- a decent amount of it, but it is not an interesting problem
from a theoretical perspective - just execution. Now if you are
higher octane as a user you may be able to see what I cannot.
But on the other hand, the Pareto principle applies, and in my
view a library should make it simple to do simple things. One
can't get there if the primary interface is a direct mapping of
the HDF5 hierarchy, and I also think that is unnecessary with D.
But I very much appreciate your work as the final result is
better for everyone that way, and you are evidently a much longer
running user of D than me. I never used C++ as it just seemed
too ugly! and I suspect the difference in backgrounds is shaping
perspectives.
What do you think the trickiest parts are with HDF5? (You
mention weird polymorphism).
Laeeth
Re: vibe.d Subdirectory?
On Wednesday, 14 January 2015 at 11:40:26 UTC, seany wrote: I am new to vibe.d and plying a bit with it. I notice, that in case of Apache, there is a root directory, often by default under /var/www or /srv/http (resp. ftp) if you are using linux, and then every time the client sends a request, apache looks in to the root directory, deduces the subdirectory from the URI, and pulls the page up from there. Then it parses the PHP / other scripting commands, and prints the HTML as is, before serving it over http. I want to know the equivalent of all these in vibe.d. The website has a documentation, but all what I find is that you will need an app.d, in a predefined directory structure. I however, do not understand, what the root directory is going to be. Is it going to be the directry where vibe.d is started? Say, I start my vibe.d under /server, then I have an app.d under /server/a/app.d and /server/b/app.d Do I access them via http://top.level.domain/a, resp /b, and app.d is like index.html / index.php which vibe.d looks for by default, or do i have to use http://top.level.domain/a/app.d To be very clear: in the simple case when you compile your vibe application from multiple source files and diet templates etc, and you will end up with an executable. This can act as a server directly, or you can make it an internal server on localhost called by php, or you can call it from nginx as a CGI type application depending on the URL. The vibed directory layout is of no relevance after compilation (unless you put your static data there).
Re: Explicit Slicing of std.container.Array
On Saturday, 24 January 2015 at 13:11:33 UTC, Nordlöw wrote:
Is there a reason why std.container.Array have to be explicitly
sliced before being processed by range algorithms such as
filter typically as
import std.container: Array;
Array!int a;
foreach (e; a[].filter!true) {}
?
Does memory allocation play a role?
I would like to see it be used in replace of D builtin arrays
without any code changes.
One thing to mention in the revised docs in the introduction to
std.algorithms.
Re: Turning Executable into Application?
On Monday, 26 January 2015 at 03:36:32 UTC, Gan wrote: With Xamarin Studio I create a D project and run it. It runs an Executable Unix file through the terminal. How can I turn that into an Application that doesn't open the Terminal? Thanks. Have you tried running your executable from the command line? I suspect Xamarin is just opening a terminal for you as a convenience.
Re: BitArray - incomplete implementation?
To avoid confusion, the below is the code that fits the error
message:
import std.bitmanip;
import std.stdio;
import std.array;
import std.range:chain;
void test()
{
int[] a=[1,2,3,4,5];
int[] b=[5,4,3,2,1];
int[] c = chain(a,b).array; // chain two arrays of int
writefln(%s,c);
}
void test2()
{
BitArray a;
a.init([1,0,1,0]);
BitArray b;
b.init([0,1,0,1]);
BitArray[] d;
d~=a;
d~=b;
BitArray[] c=chain(a,b).array; // cannot chain two bitarrays
BitArray[] c=chain([a],[b]).array; // cannot chain two bitarrays
BitArray[] e=chain(d,d).array; // cannot chain two arrays of
bitarrays
writefln(%s,c);
}
int main(string[] args)
{
test();
test2();
return 1;
}
BitArray - incomplete implementation?
Hi.
Should the following code work?
import std.bitmanip;
import std.stdio;
import std.array;
import std.range:chain;
void test()
{
int[] a=[1,2,3,4,5];
int[] b=[5,4,3,2,1];
int[] c = chain(a,b).array; // chain two arrays of int
writefln(%s,c);
}
void test2()
{
BitArray a;
a.init([1,0,1,0]);
BitArray b;
b.init([0,1,0,1]);
BitArray[] d;
d~=a;
d~=b;
BitArray[] c=chain([a],[b]).array; // cannot chain two bitarrays
BitArray[] e=chain(d,d).array; // cannot chain two arrays of
bitarrays
writefln(%s,c);
}
int main(string[] args)
{
test();
test2();
return 1;
}
bitmanip.d(23): Error: template std.range.chain cannot deduce
function from argument types !()(BitArray, BitArray), candidates
are:
/usr/include/dmd/phobos/std/range.d(2493):
std.range.chain(Ranges...)(Ranges rs) if (Ranges.length 0
allSatisfy!(isInputRange, staticMap!(Unqual, Ranges))
!is(CommonType!(staticMap!(ElementType, staticMap!(Unqual,
Ranges))) == void))
bitmanip.d(24): Error: template std.range.chain cannot deduce
function from argument types !()(BitArray[], BitArray[]),
candidates are:
/usr/include/dmd/phobos/std/range.d(2493):
std.range.chain(Ranges...)(Ranges rs) if (Ranges.length 0
allSatisfy!(isInputRange, staticMap!(Unqual, Ranges))
!is(CommonType!(staticMap!(ElementType, staticMap!(Unqual,
Ranges))) == void))
bitmanip.d(24): Error: declaration bitmanip.test2.c is already
defined
bitmanip.d(25): Error: template std.range.chain cannot deduce
function from argument types !()(BitArray[], BitArray[]),
candidates are:
/usr/include/dmd/phobos/std/range.d(2493):
std.range.chain(Ranges...)(Ranges rs) if (Ranges.length 0
allSatisfy!(isInputRange, staticMap!(Unqual, Ranges))
!is(CommonType!(staticMap!(ElementType, staticMap!(Unqual,
Ranges))) == void))
I cannot seem to concatenate - directly, or using chain - two
bitarrays, or two arrays of bitarrays.
Re: BitArray - incomplete implementation?
Yes, that error is caused by a bug of BitArray(https://issues.dlang.org/show_bug.cgi?id=13806). Having init function broke template constraints of chain(and must break dozen of other templates). pragma(msg, ElementType!(BitArray[])) // prints 'pure nothrow void(bool[] ba)' - ElementType uses init property to determine types. Thanks for this. If I recompile phobos with the pull request (having edited out the init property entirely) then the code mostly works. However, I still can't use join, joiner, or chain on two BitArrays. Eg auto x=joiner(a,b).array; I can chain two arrays of BitArrays, but not join or joiner them. If this is what ought to happen, why?
Re: foreach - premature optimization vs cultivating good habits
Thank you Adam, Bbaz and Ola for the helpful thoughts. I dumped them in a wiki page off the sandbox but needs editing and refining.
Re: foreach - premature optimization vs cultivating good habits
On Friday, 30 January 2015 at 12:55:20 UTC, Adam D. Ruppe wrote: On Friday, 30 January 2015 at 11:55:16 UTC, Laeeth Isharc wrote: As I understand it, foreach allocates when a simple C-style for using an array index would not. foreach is just syntax sugar over a for loop. If there's any allocations, it is because your code had some, it isn't inherit to the loop. The doc definition even lists the translation of foreach to for in the case of ranges explicitly: http://dlang.org/statement.html#ForeachStatement The most likely allocation would be to a user-defined opApply delegate, and you can prevent that by making it opApply(scope your_delegate) - the scope word prevents any closure allocation. Thanks, Adam. That's what I had thought (your first paragraph), but something Ola on a different thread confused me and made me think I didn't understand it, and I wanted to pin it down. The second paragraph is very helpful - appreciate it. Laeeth.
Re: simple dub question - avoiding creating a vibed project
On Monday, 5 January 2015 at 10:46:17 UTC, Rikki Cattermole wrote: On 5/01/2015 11:42 p.m., Laeeth Isharc wrote: Figured out a fix: versions: [VibeCustomMain], It is still mysterious as to why it is pulling in vibed though (I don't import it, and I didn't think ddbc did). https://github.com/mysql-d/mysql-native/blob/master/package.json aha. isn't this a poor default for dub though? ie if your parent project itself does not depend on vibed, the default should be that you take care of main yourself, and it does not try and use vibed's, no ?
simple dub question - avoiding creating a vibed project
Hi.
I am building an example for hibernated (I put a main around the
sample code extract from the website).
How do I stop dub trying to build a vibed project?
Here is my dub.json
{
name: ddbc example,
description: example for DB Connector for D language,
similar to JDBC,
authors: [Vadim Lopatin,Laeeth Isharc],
homepage: https://github.com/buggins/ddbc;,
license: Boost Software License (BSL 1.0),
dependencies: {
mysql-native: =0.0.12,
ddbc: =0.2.16,
},
targetType: executable,
libs-posix: [sqlite3, pq],
libs-windows: [sqlite3, libpq],
copyFiles-windows-x86: [ libs/win32/sqlite3.dll,
libs/win32/libpq.dll, libs/win32/intl.dll ],
sourceFiles-windows-x86 : [ libs/win32/sqlite3.lib,
libs/win32/libpq.lib ],
targetPath: .,
}
Re: simple dub question - avoiding creating a vibed project
Figured out a fix: versions: [VibeCustomMain], It is still mysterious as to why it is pulling in vibed though (I don't import it, and I didn't think ddbc did).
Re: simple dub question - avoiding creating a vibed project
I opened an issue about this last year: https://github.com/mysql-d/mysql-native/issues/44 Thanks. Laeeth.
Re: I'll be presenting at NWCPP on Jan 21 at Microsoft
On Monday, 5 January 2015 at 07:46:20 UTC, Walter Bright wrote: http://nwcpp.org/ All are invited. Now I just have to write the presentation :-( Congratulations. I hope the talk goes well. Will audio be available afterwards? At a slight tangent, has anything more recent been written on the C++ interface? I understand it is more complete than what is described on the Wiki/at dlang.org and have not been able to find a write-up of this. Thanks. Laeeth.
Re: I'll be presenting at NWCPP on Jan 21 at Microsoft
On Monday, 5 January 2015 at 21:07:04 UTC, Walter Bright wrote: On 1/5/2015 5:31 AM, Laeeth Isharc wrote: Will audio be available afterwards? NWCPP usually will post the video afterwards. At a slight tangent, has anything more recent been written on the C++ interface? I understand it is more complete than what is described on the Wiki/at dlang.org and have not been able to find a write-up of this. Sorry, nothing yet. It's a work in progress. Thanks, Walter.
D bindings and high level wrapper for systemd journal
D Language bindings and high level wrapper for the systemd journal logging service (journalctl) https://github.com/Laeeth/d_sysdlog alpha status rather than grep /var/log/messages, systemd interface makes it easy to filter by field and period of interest. metadata is stored along with regular log messages, which reduces the ability of bad actors to spoof logs Link to C API here: http://www.freedesktop.org/software/systemd/man/sd-journal.html Description systemd-journald is a system service that collects and stores logging data. It creates and maintains structured, indexed journals based on logging information that is received from a variety of sources: Kernel log messages, via kmsg Simple system log messages, via the libc syslog(3) call Structured system log messages via the native Journal API, see sd_journal_print(4) Standard output and standard error of system services Audit records, via the audit subsystem The daemon will implicitly collect numerous metadata fields for each log messages in a secure and unfakeable way. See systemd.journal-fields(7) for more information about the collected metadata. Log data collected by the journal is primarily text-based but can also include binary data where necessary. All objects stored in the journal can be up to 2^64-1 bytes in size.
Any chance of a linux dtoh?
I realize Walter has far better things to work on, but value of having a translation tool is considerable, since it opens up easy access to an enormous range of libraries. It is not much work to do the translation oneself, but in the world as it is small frictions cumulatively have large consequences. Access to libraries is a big advantage of python, and the like. Once you have the wrapper in place, I find D as productive as python, but for now it is still a bit of work first. There is also dstep, but this doesn't seem to compile right now, and I don't have time to figure out why and fix it. (I took a look, and it is not so easy to figure out). Something mambo/tango related. I have raised a bug report. Laeeth
Re: Any chance of a linux dtoh?
On Tuesday, 6 January 2015 at 14:11:19 UTC, Dicebot wrote: dstep is your only realistic chance currently. htod is completely unmaintained, even on Windows. Please report any issues found with it in relevant issue tracker. I got it the wrong way around - yes, I meant htod. I have reported the issue, and author doesn't know what is wrong.
Re: idiomatic D: what to use instead of pointers in constructing a tree data structure?
Small recommendation (apart from the reserved word issue which you fixed): it's generally considered good D style to give structs and classes names that start with capital letters, JustLikeThis. So, I suggest Node rather than node. Very minor point, and of course, your code is yours to style as you wish, but it can be helpful to meet the standard style conventions in order to make it as easy as possible for everyone else to understand. Thanks for the reminder. I use D Style when I am writing properly, but haven't yet internalized it for a quick example and old habits die hard.
Re: Wrapping a C library with its own GC + classes vs refcounted structs
Hi Aldanor. I wrote a slightly longer reply, but mislaid the file somewhere. I guess your question might relate to wrapping the HDF5 library - something that I have already done in a basic way, although I welcome your project, as no doubt we will get to a higher quality eventual solution that way. One question about accurately representing the HDF5 object hierarchy. Are you sure you wish to do this rather than present a flattened approach oriented to what makes sense to make things easy for the user in the way that is done by h5py and pytables? In terms of the actual garbage generated by this library - there are lots of small objects. The little ones are things like a file access attribute, or a schema for a dataset. But really the total size taken up by the small ones is unlikely to amount to much for scientific computing or for quant finance if you have a small number of users and are not building some kind of public web server. I think it should be satisfactory for the little objects just to wrap the C functions with a D wrapper and rely on the object destructor calling the C function to free memory. On the rare occasions when not, it will be pretty obvious to the user and he can always call destroy directly. For the big ones, maybe reference counting brings enough value to be useful - I don't know. But mostly you are either passing data to HDF5 to write, or you are receiving data from it. In the former case you pass it a pointer to the data, and I don't think it keeps it around. In the latter, you know how big the buffer needs to be, and you can just allocate something from the heap of the right size (and if using reflection, type) and use destroy on it when done. So I don't have enough experience yet with either D or HDF5 to be confident in my view, but my inclination is to think that one doesn't need to worry about reference counting. Since objects are small and there are not that many of them, relying on the destructor to be run (manually if need be) seems likely to be fine, as I understand it. I may well be wrong on this, and would like to understand the reasons if so. Laeeth.
endsWith - for a string vs an array of strings
I understand from previous discussion there is some difficulty
over immutability. I did not quite figure out what the solution
was in this case:
import std.array;
import std.string;
import std.stdio;
void main(string[] args)
{
string[] test=[1,two,three!];
auto a=arghtwo.endsWith(test);
writefln(%s,a);
}
This does not compile...
test.d(7): Error: template std.algorithm.endsWith cannot deduce
function from argument types !()(string, string[]), candidates
are:
/usr/include/dmd/phobos/std/algorithm.d(6143):
std.algorithm.endsWith(alias pred = a == b, Range,
Needles...)(Range doesThisEnd, Needles withOneOfThese) if
(isBidirectionalRange!Range Needles.length 1
is(typeof(.endsWith!pred(doesThisEnd, withOneOfThese[0])) : bool)
is(typeof(.endsWith!pred(doesThisEnd,
withOneOfThese[1..__dollar])) : uint))
/usr/include/dmd/phobos/std/algorithm.d(6210):
std.algorithm.endsWith(alias pred = a == b, R1, R2)(R1
doesThisEnd, R2 withThis) if (isBidirectionalRange!R1
isBidirectionalRange!R2
is(typeof(binaryFun!pred(doesThisEnd.back, withThis.back)) :
bool))
/usr/include/dmd/phobos/std/algorithm.d(6237):
std.algorithm.endsWith(alias pred = a == b, R, E)(R
doesThisEnd, E withThis) if (isBidirectionalRange!R
is(typeof(binaryFun!pred(doesThisEnd.back, withThis)) : bool))
Thanks.
Laeeth.
casting SysTime to ubyte[]
import std.datetime;
import std.stdio;
import std.conv;
void main(string[] arg)
{
auto a=Clock.currTime();
auto b=cast(ubyte[])a;
writefln(%s,b);
}
how do i get the time as a binary representation I can write to a
file?
Thanks.
Re: endsWith - for a string vs an array of strings
Thanks for the help to everyone. It seems a common thing to want
to check an array as one may not know the variables at compile
time. Not that it's more than a few lines to do in D. But in
terms of language adoption, small frictions can have large
consequences over time. (Modern people don't have much patience
or sticking power even if viewed rationally the ROI makes these
things trivial. That's also because in a corporate environment
management are not always patient when somebody tries something
new and not yet mainstream). Making the standard library easy
with no sharp edges might have a high payoff over time.
How would one add this note about needing to pass a tuple not an
array to the docs or wiki ? For docs, clone and submit a pull
request? How about wiki?
On Monday, 12 January 2015 at 07:49:16 UTC, Mengu wrote:
On Saturday, 10 January 2015 at 23:32:47 UTC, bearophile wrote:
Laeeth Isharc:
In D there is a feature that allows a function to accept
both an array of items and items,
yes - it is funny there is not an overloading that accepts
arrays
I meant this D feature:
void foo(T)(T[] items...) {
import std.stdio;
items.writeln;
}
void main() {
foo(red, green, blue);
foo([red, green, blue]);
auto a = [red, green, blue];
foo(a);
}
Bye,
bearophile
for the curious, expanding tuples and typetuples on ali's book
is explained at
http://ddili.org/ders/d.en/tuples.html#ix_tuples..expand and at
http://ddili.org/ders/d.en/tuples.html#ix_tuples.TypeTuple,
std.typetuple.
Re: Wrapping a C library with its own GC + classes vs refcounted structs
Laeeth. Thanks for the reply. Yes, this concerns my HDF5 wrapper project; the main concern is not that the memory consumption of course, but rather explicitly controlling lifetimes of the objects (especially objects like files -- so you are can be sure there are no zombie handles floating around). An easy way is to just use scope(exit) to either close the HDF5 object directly, or indirectly call destroy on the wrapper. If you want to make it 'idiot proof', maybe ref counting structs will get you there (at possible cost of small overhead). I personally don't tend to forget to close a file or dataset; its much easier up forget to close a data type or data space descriptor. But struct vs class depends somewhat on how you want to represent the object hierarchy in D, no ? Incidentally there are some nice things one can do using compile time code to map D structs to HDF5 types (I have implemented a simple version of this in my wrapper). A bit more work the other way around if you don't know what's in the file beforehand.
Re: How to prevent sensitive information is displayed when the extension 'exe' is modified to 'txt' on windows?
What you want is some kind of code obfuscation. The easiest thing for you is to use exe compression. It is not going to stop a dedicated attacker, but ordinary people will not be able to extract any information from it. And I guess as an alternative to the utility you linked to, you could use D's ability to run code at compile time to encrypt your sensitive literals during compilation and then decrypt them on program startup.
Re: D Beginner Trying Manual Memory Management
I see, thanks! :) I've started liking structs more and more recently as well and been pondering on how to convert a class-based code that looks like this (only the base class has any data): it's hard to tell by brief description. but having multiple inheritance immediately rings an alarm ring for me. something is very-very-very wrong if you need to have a winged whale. ;-) A real-world example: http://www.hdfgroup.org/HDF5/doc/cpplus_RM/hierarchy.html H5::File is both an H5::Location and H5::CommonFG (but not an H5::Object) H5::Group is both an H5::Object (subclass of H5::Location) and H5::CommonFG H5::Dataset is an H5::Object i see something named CommonFG here, which seems to good thing to move out of hierarchy altogether. bwah, i don't even sure that given hierarchy is good for D. C++ has no UFCS, and it's incredibly hard to check if some entity has some methods/properties in C++, so they have no other choice than to work around that limitations. it may be worthful to redesign the whole thing for D, exploiting D shiny UFCS and metaprogramming features. and, maybe, moving some things to interfaces too. I just finished reading aldanor's blog, so I know he is slightly allergic to naked functions and prefers classes ;) With Ketmar, I very much agree (predominantly as a user of HDF5 and less so as an inexperienced D programmr writing a wrapper for it). It's a pain to figure out just how to do simple things until you know the H5 library. You have to create an object for file permissions before you even get started, then similarly for the data series (datasets) within, another for the dimensions of the array, etc etc - that doesn't fit with the intrinsic nature of the domain. There is a more general question of bindings/wrappers - preserve the original structure and naming so existing code can be ported, or write a wrapper that makes it easy for the user to accomplish his objectives. It seems like for the bindings preserving the library structure is fine, but for the wrapper one might as well make things easy. Eg here https://gist.github.com/Laeeth/9637233db41a11a9d1f4 line 146. (sorry for duplication and messiness of code, which I don't claim to be perfectly written - I wanted to try something quickly and have not yet tidied up). So rather than navigate the Byzantine hierarchy, one can just do something like this (which will take a struct of PriceBar - date,open,high,low,close - and put it in your desired dataset and file, appending or overwriting as you prefer). dumpDataSpaceVector!PriceBar(file,ticker,array(priceBars[ticker]),DumpMode.truncate); which is closer to h5py in Python. (It uses reflection to figure out the contents of a non-nested struct, but won't yet cope with arrays and nested structs inside). And of course a full wrapper might be a bit more complicated, but I truly think one can do better than mapping the HDF5 hierarchy one for one. Laeeth.
Re: idiomatic D: what to use instead of pointers in constructing a tree data structure?
On Wednesday, 7 January 2015 at 14:59:58 UTC, H. S. Teoh via Digitalmars-d-learn wrote: On Wed, Jan 07, 2015 at 02:52:51PM +, Laeeth Isharc via Digitalmars-d-learn wrote: Another schoolboy question. Suppose I am constructing a tree (in this case it is an AST). In C I would have a pointer for the child to find the parent, and an array or linked list of pointers to find the children from the parent. Obviously, I could still use pointers, but that would not be idiomatic. Not true. If you're using a tree structure, you *should* use pointers. Unless you're using classes, which are by-reference, in which case you can just use the class as-is. :-) --T The GC is allowed to move structs around, as I undestand it. Under what circumstances do I get into trouble having a pointer to them?
Re: London D Programmers MeetUp
done. and gave him headsup too. is your email no spam o n e at yourfullname.com ? if not what should the first part be? or you can email me at myfirstn...@myfirstname.com On Friday, 13 February 2015 at 07:31:52 UTC, Laeeth Isharc wrote: On Wednesday, 11 February 2015 at 22:28:03 UTC, Walter Bright wrote: On 2/4/2015 1:17 PM, Laeeth Isharc wrote: One interesting anecdote: somebody in a financial services company gave an account of giving D a try as a way to prototype something quickly, intending to rewrite it later in a more conventional language. The prototype went straight into production, and they are happy with it. The C interop was important for them to have made this possible. It was interesting, because it is a serious company where people don't by any means have a hacker mentality, and it was one more point at variance with the I am a java programmer but do all my personal projects in D narrative, and also because this particular kind of company is exactly the sort of second wave early adopter one wants to have. (I hope I represented this accurately, and tried to make this as vague as possible whilst still keeping it useful since people might not want the details of what they do being public). I'd like to invite this fellow to propose a case study presentation at Dconf 2015. Can someone please forward this to him, or put him in contact with myself or Andrei? I will dig his card out unless Kingsley gets there first. (Let me know if so). Laeeth
Bindings for libmRSS: for parsing, writing and creating RSS/ATOM files or streams.
https://github.com/Laeeth/d_rss/ d_rss: D Programming Language bindings for libmRSS. http://www.autistici.org/bakunin/libmrss/doc/ mRss is a C library written by Andrea Marchesini for parsing, writing and creating RSS/ATOM files or streams. This library supports: RSS 0.91 - http://my.netscape.com/publish/formats/rss-spec-0.91.html RSS 0.92 - http://backend.userland.com/rss092 RSS 1.0 - http://web.resource.org/rss/1.0/ RSS 2.0 - http://www.rssboard.org/rss-specification ATOM 0.3 - http://www.mnot.net/drafts/draft-nottingham-atom-format-02.html ATOM 1.0 - http://tools.ietf.org/html/rfc4287
