Re: Mix struct types in the same array
On Wednesday, 18 December 2019 at 22:17:21 UTC, tirithen wrote:
On Wednesday, 18 December 2019 at 22:11:10 UTC, Sebastiaan
Koppe wrote:
If you know all types up front you can use the Sumtype library
on code.dlang.org
Thanks, it's a good starting point, the best would be if I only
needed to define that the struct would implement void
applyTo(ref User user) so that could be run in the loop to
update the User entity.
But I'll try the Sumtype library, that takes me forward, thanks
for the answer!
https://github.com/atilaneves/concepts
interface IFoo {
int foo(int i, string s) @safe;
double lefoo(string s) @safe;
}
@implements!(Foo, IFoo)
struct Foo {
int foo(int i, string s) @safe { return 0; }
double lefoo(string s) @safe { return 0; }
}
// doesn't compile
/*
@implements!(Oops, IFoo)
struct Oops {}
*/
Re: d programs conversion to c
On Wednesday, 11 December 2019 at 18:54:49 UTC, jicman wrote: Greetings! I am trying to see if there are any converters out there from d code to c. Anyone knows? Thanks. josé How many lines of code is it ? It's not that bad to do it manually with help from regex. If you're good with vim macros like Robert Schadek then that may help too. There's a project to convert C code to Rust and some day I plan to support something similar for C to D. LLVM used to have a C backend that was revived by the Julia guys. Might get somewhere with that, but if it's not too big a codebase assisted manual isn't that bad. First version of excel-d I had to do entirely manually as dpp didn't exist.
Re: Good way let low-skill people edit CSV files with predefined row names?
On Thursday, 24 October 2019 at 17:41:21 UTC, Dukc wrote: On Thursday, 24 October 2019 at 16:50:17 UTC, Dukc wrote: Hmm, I need to check whether I can do that on LibreOffice Calc. Unfortunately, no. If there's a way to do that, it's not obvious. I should be able to make an easy-to-use excel-to-csv translator using Atilas Excel utilites without too much effort. This was wrong: Atila's Excel-d enables writing plugin functions, but not reading the spreadsheets. There are other DUB utilities for that, though. I quess I will give my employer two options: Either the price variables are in an one-column CSV and I distribute the key column separately so they don't mess it up, or I take my time to do a GUI solution. Unless somebody has better ideas? Another Symmetry project allows reading Excel files and a third is wrapper and bindings around a C library to write Excel files. We use them in production daily though there may be rough edges for features we don't use. I should think you can use a Javascript library and call it from D. See trading views repo by Sebastian Koppe for an example of this. Bindings are manual currently but he will work on generating them from the Typescript bindings in time.
Re: D for sciencetific scripting / rapid protoryping
On Tuesday, 22 October 2019 at 05:58:50 UTC, Prokop Hapala wrote: I'm examining the possibility to move from Python+C/C++ to D or Python+D. I read (https://wiki.dlang.org/Programming_in_D_for_Python_Programmers) and (https://jackstouffer.com/blog/nd_slice.html), where is mentioned PyD, Mir-algorithm, all seems very promising. But I did not test it yet. [...] See autowrap. PyNih might eventually be a bit nicer than pyd but it's not yet there. We did some very early work on Julia integration and probably will finish when time. You can call R libraries from D too. If you do use pyd then ppyd might make it a bit more pleasant. Some rough edges around pyd but it's okay if you don't mind figuring things out.
Re: problems with swig generated code
On Tuesday, 3 September 2019 at 20:03:37 UTC, Martin DeMello wrote: On Sunday, 1 September 2019 at 11:19:11 UTC, DanielG wrote: Do you know whether SWIG's D generator is even being maintained? I've searched for it on the forums in the past and got the impression that it's outdated. I didn't realise that :( It was included in the current release of swig, so I figured it was maintained. It's pretty sad if it's not, because trying to access C++ libraries directly from D has some limitations (most notably not being able to create new C++ objects from D) and swig would have let things just work. I think DPP can call constructors. YMMV. We are working on a little project I started as another step in the eternal personal hackathon. Libclang isn't my cup of tea. It's almost very cool but they put in whatever the guy needed and so it's inconsistent but your alternative is code that breaks things between breakfast and teatime. Cling is used at CERN and I found libcling more pleasant. I only wrapped the cppyy fork but that actually allows you to reflect at runtime on a lot. From our DSL at work I can include a header, instantiate a templated type, create an instance of the class and call a method on it. Can, but it's not what I would call fun. I didn't yet get time to wrap the rest of the interpreter. But the idea is to make a tool for wrapping c++ via simpler extern (C++). If one isn't quite sure upfront what will work and not then it's much more pragmatic. I can't see how it won't work but we will know in a week or two. My first version is here. https://github.com/kaleidicassociates/cpp-reflect-d There is another route people don't think of. Calypso can introspect on C++ too. You might not want to use it in production but you don't need to. Either I guess you could use it to generate wrappers or you can use it to replace a chunk of cpp code. The surface area of a method must be larger than thr surface area of a chunk. Well chosen then it's easy to write glue code semi automatically. But it's nice to be able to replace a method at a time and have working code at every step. You don't need to trust Calypso in production to be able to use it for this purpose.
D for a safer Linux kernel
I noticed a Rust post so why not post. https://www.reddit.com/r/linux_programming/comments/cs0ime/d_for_a_safer_linux_kernel https://www.reddit.com/r/programming/comments/cs0iec/d_for_a_safer_linux_kernel
OT: in economic terms Moore's Law is already dead
https://www.infoq.com/presentations/moore-law-expiring/ At the same time as the arrival of Optane persistent storage in relatively chest machines changes the game a bit. If storage prices do keep falling at 40% annualised or thereabouts, it's possible one might see a little more respect being given the writing performant code than currently.
OT - Git training Lon/HK and book recommendation on taste in programming
Hi. First question - can anyone recommend git / Gitlab training providers in HK and London? Two distinct audiences - highly intelligent people that may or may not really program, and experienced developers with a finance background that could benefit from knowing how to use git properly (finance is often in the dark ages). On the former we are even getting HR, legal and compliance to start to use git for documents. So some handholding will be required. I would like a combination of classroom, small group on-premise training and somebody being in the office a few hours a week to help show people. No experience is necessarily required for the latter provided you know git well and can patiently explain things in a way less advanced people will understand. It could even be a nice part-time job for a student and we could pay well. Not that we wouldn't look at a professional either - I just mean that I am open minded. Second question. Lots of people these days start to program to solve their problems at work but they may never have been shown the basic principles of design, structuring and maintenance of their code. If I could give them one book (and a few YouTube links) what should it be ? Simple things like it's okay to write functions, start with getting the data structures right, quality is fractal (Walter making little improvements to DMD for example), value of simplicity and things that are harder to explain like the proper composition of a system. I would appreciate any suggestions on either one. Laeeth
Re: Phobos in BetterC
On Friday, 8 March 2019 at 09:24:25 UTC, Vasyl Teliman wrote: I've tried to use Mallocator in BetterC but it seems it's not available there: https://run.dlang.io/is/pp3HDq This produces a linker error. I'm wondering why Mallocator is not available in this mode (it would be intuitive to assume that it's working). Also I would like to know what parts of Phobos are available there (e.g. std.traits, std.typecons...). Thanks in advance. I would guess it's not available because it was written before betterC mode was a thing and nobody has yet updated it. If you look at spasm on code.dlang.org there is a version you can copy paste that should work in betterC mode if I remember correctly.
Re: DStep rocks [was Example of using C API from D?]
On Sunday, 2 September 2018 at 17:49:45 UTC, Russel Winder wrote:
On Sun, 2018-09-02 at 18:28 +0100, Russel Winder wrote:
[…]
It turns out that the GIR file is not usable, and so the
girtod route is not feasible. I shall try the DStep route.
Failing that it seems there is
https://github.com/WebFreak001/fontconfig-d
which is a manual transform of a snapshot of the C API, so not
an
ideal
way, but a definite backstop position. It seems someone has
trodden
the
"using Fontconfig in D" path before me.
I compiled DStep master/HEAD (v0.2.3-16-g1308991) against LLVM
6.0 and it seems to have done a rather splendid job of creating
a D binding to Fontconfig. Low-level obviously, but Fontconfig
is seriously low level anyway.
Now to work out how to make the project auto generate this D
module so as to avoid having it in the repository, and
potentially inconsistent with the platform in use.
You could also look at dpp. That's worked for most things I tried
and was written in part to avoid the problem of macros changing
behaviour at build time.
Example here:
https://run.dlang.io/?compiler=dmd=%23include%20%0Avoid%20main()%20%7B%0A%20%20%20%20printf("Hello%20dpp.");%0A%7D
https://github.com/atilaneves/dpp
Re: extern __gshared const(char)* symbol fails
On Friday, 31 August 2018 at 18:49:26 UTC, James Blachly wrote: On Friday, 31 August 2018 at 17:18:58 UTC, Neia Neutuladh wrote: On Friday, 31 August 2018 at 06:20:09 UTC, James Blachly wrote: Hi all, ... When linking to this library from D, I have declared it as: extern __gshared const(char)* seq_nt16_str; ***But this segfaults when I treat it like an array (e.g. by accessing members by index).*** I believe this should be extern extern(C)? I'm surprised that this segfaults rather than having a link error. A bare `extern` means "this symbol is defined somewhere else". `extern(C)` means "this symbol should have C linkage". I am so sorry -- I should have been more clear that this is in the context of a large header-to-D translation .d file, so the whole thing is wrapped in extern(C) via an extern(C): at the top of the file. In case you weren't aware of it, take a look at atilaneves DPP on GitHub or code.dlang.org. auto translates C headers at build time and mostly it just works. If it doesn't, file an issue and in time it will be fixed.
dtoh
Hi Walter. Can dtoh be open-sourced now that dmd is? Laeeth.
Re: Is it feasible to slowly rewrite a C++ codebase in D?
On Wednesday, 20 June 2018 at 18:47:10 UTC, Jordi Gutiérrez Hermoso wrote: I'm specifically thinking of the GNU Octave codebase: http://hg.savannah.gnu.org/hgweb/octave/file/@ It's a fairly old and complicated C++ codebase. I would like to see if I could slowly introduce some D in it, anywhere. Now, as I understand it, I would need to begin with making `main` a D function, because D needs to initialise the runtime. Is this correct? Another possibility might be in dlopen'able functions. Currently Octave uses so-called oct functions, which are nothing more than C++ object code that is dynamically loaded by the interpreter at runtime. They are compiled to the Octave C++ API, but we also have a Matlab-compatible C API that perhaps could be more amenable for D-ification. What are your ideas? If you would like to expose C function and type declarations to D, you could take a look at DPP, which allows you to just #include a C header. If you encounter a bug, please file an issue and in time we will fix it. Does not yet work for C++ except in some cases. https://github.com/atilaneves/dpp
C++ function mangling Linux GCC
Am I missing something, or should extern(C++) just work for
binding to gcc C++ on Linux. It works fine for primitives but
fails for pointer type arguments. Extern "C" works fine.
Does D know how to mangle function names based on pointer types?
I have created matching types on both sides.
Though I am using typedefs. Eg struct Foo_; typedef struct Foo_
Foo; and on D side struct Foo {}
Could that be why?
What the best way to see the types of library file function
arguments for a libfoo.a file on Linux? Sorry for the dumb
question.
Thanks.
Laeeth
Re: C++ Interop
On Saturday, 6 January 2018 at 11:17:56 UTC, Seb wrote: On Friday, 5 January 2018 at 13:02:12 UTC, qznc wrote: I'm exploring [0] C++ interop after watching Walter's presentation [1]. [...] I know about this: https://github.com/Remedy-Entertainment/binderoo https://github.com/dlang/druntime/pull/1802 Binderoo currently is Windows only. I am talking to Ethan about extending it to work on Linux too. Let me know if any other features helpful to add (no promises, but we can see).
Re: Any free stock market data API?
On Thursday, 4 January 2018 at 23:04:44 UTC, Amorphorious wrote: Most are in other languages: https://www.alphavantage.co/ https://iextrading.com/ are two free ones. I'm just hoping for a more D'ish solution. I wrote a simple api for quandl.com and somewhere I have one for yahoo. Neither being used right now so might have broken. https://github.com/Laeeth/d-quandl/blob/master/quandl.d
Re: structs inheriting from and implementing interfaces
On Friday, 29 December 2017 at 12:59:21 UTC, rjframe wrote:
On Fri, 29 Dec 2017 12:39:25 +, Nicholas Wilson wrote:
On Friday, 29 December 2017 at 12:03:59 UTC, Mike Franklin
wrote:
The problem is that interfaces are a runtime thing (e.g. you
can cast a
class to an interface)
structs implement compile time interfaces via template duck
typing
(usually enforced via an if()).
you could probably write a wrapper that introspected an
interface and
enforced that all members were implemented.
I've actually thought about doing this to get rid of a bunch of
if qualifiers in my function declarations. `static interface
{}` compiles but doesn't [currently] seem to mean anything to
the compiler, but could be a hint to the programmer that
nothing will directly implement it; it's a compile-time
interface. This would provide a more generic way of doing stuff
like `isInputRange`, etc.
Atila does something like this
https://code.dlang.org/packages/concepts
ESR on post-C landscape
He mentions D, a bit dismissively. http://esr.ibiblio.org/?p=7724=1#comment-1912717
Re: Infuriating DUB/DMD build bug.
On Saturday, 7 October 2017 at 19:34:53 UTC, WhatMeForget wrote: On Friday, 6 October 2017 at 23:02:56 UTC, Laeeth Isharc wrote: On Thursday, 5 October 2017 at 21:48:20 UTC, WhatMeWorry wrote: I've got a github project and using DUB with DMD and I keep running into this problem. I've tried deleting the entire ...\AppData\Roaming\dub\packages folder, but the problem repeats the very next build attempt. [...] See my post in learn on dmd path. The dmd path code was written in 1987 and could do with an update to process longer windows paths properly. We are working on this and I guess a chance a pull request on Monday but it depends on what else comes up. In any case it's a trivial fix. Presuming I am right about it being a path length problem. I did! But i didn't say anything because i wasn't sure if they were related. I'm pretty sure it is path related because the exact same dub.sdl files work fine on Linux and MacOS. (It's a cross platform project) Glad it is a trivial fix. Curious what it involves. Let me know if I can help out in any way. Mike Parker was kind enough to show me a manual dub local workaround for this issue. But I'll hold off now and see if your change does the fix. If it does, it will be the best timed bug fix ever :) Turned out to be more fiddly than I hoped because of unicode. Atila did the work, but it's awaiting a pre-review to be sure it's okay before submitting. In case it's helpful in the meantime, it seems to work (but use at your own risk): https://github.com/kaleidicassociates/dmd/pull/1
Re: Fast removal of character
On Wednesday, 11 October 2017 at 22:22:43 UTC, Johan Engelen
wrote:
std.string.removechars is now deprecated.
https://dlang.org/changelog/2.075.0.html#pattern-deprecate
What is now the most efficient way to remove characters from a
string, if only one type of character needs to be removed?
```
// old
auto old(string s) {
return s.removechars(",").to!int;
}
// new?
auto newnew(string s) {
return s.filter!(a => a != ',').to!int;
}
```
cheers,
Johan
There's always this:
https://github.com/dlang/undeaD/blob/master/src/undead/string.d
Re: Catching C++ Exceptions in D - Windows and Linux
On Tuesday, 12 September 2017 at 04:33:30 UTC, Nicholas Wilson wrote: On Tuesday, 12 September 2017 at 03:51:45 UTC, Laeeth Isharc wrote: Hi. I'm here in HK with Ilya, Atila, John Colvin, and Jonathan Davis. I wondered what the current state of D catching C++ exceptions was on Linux and Windows. I know that some work was done on making this possible, and my understanding is that it is, more or less - just wondered what the rough corners might be. Thanks. Laeeth. IIRC I remember Walter saying that we should only bother to be able to catch C++ exceptions derived from std::exception, as opposed to say ints or strings. I believe the line was "those who throw ints should get what they deserve". Apart from that I believe they should "just work", although I haven't tested. OT I'm sorry I could make it to HK, I'm busy with uni all this week. I could possibly do telepresence though. Nic Thanks, Nic. Sorry about the dates - was arranged last minute. I'll drop you a line shortly in any case. Laeeth.
Re: Infuriating DUB/DMD build bug.
On Thursday, 5 October 2017 at 21:48:20 UTC, WhatMeWorry wrote: I've got a github project and using DUB with DMD and I keep running into this problem. I've tried deleting the entire ...\AppData\Roaming\dub\packages folder, but the problem repeats the very next build attempt. [...] See my post in learn on dmd path. The dmd path code was written in 1987 and could do with an update to process longer windows paths properly. We are working on this and I guess a chance a pull request on Monday but it depends on what else comes up. In any case it's a trivial fix. Presuming I am right about it being a path length problem.
Re: dmd path handling is a bit dated
On Friday, 6 October 2017 at 20:11:25 UTC, Laeeth Isharc wrote: DMD path handling is a bit dated, and it's causing build I mean I imagine getcwd and tk/filespec.c might not be the only places that need updating, but I was going to start with those and see what happened.
dmd path handling is a bit dated
DMD path handling is a bit dated, and it's causing build problems for us because on Windows it's easy to end up breaking DMD's limit - particularly given how dub likes to turn everything into a relative path. Windows has so many beautiful example of the costs of legacy compat. I just wrote down 5 ways it handles paths, and then realised there's another. There's a 260 char limit (less a bit, depending) in the old Windows API, but looking at the ddmd code there's a more restrictive limit of 131 chars + null for windows when it builds paths for the base directory. I think if you use Windows 32 file namespaces (eg \?\C:\D\foo\bar, and note the difference from \?C\D\foo\bar ) and the unicode library calls then you get up to 32,767 chars. We already have much better code in Phobos (for Windows it's hard-coded at 4K) getcwd, and I think either we should call Phobos or copy/paste the Phobos function into ddmd. And I'm posting here because we can submit a pull request, but I didn't know whether to call Phobos or copy/paste as I haven't submitted more than trivial doc changes to compiler. I've written all of this up but on an internal gitlab. https://github.com/dlang/dmd/blob/2bf9a9d731e88ebd8a175bd0a990a3b651e8df82/src/ddmd/tk/filespec.c (c) 1986-1987 by NorthWest Software. It could do with an update! " So there's an obvious set of related problems. Line 119 - current working dir is 131 chars + null. and on linux it's restricted to 255+null (not sure if that limit applies anymore to linux, but who cares for now). getcwd prototype is defined here: https://github.com/dlang/dmd/blob/ebd6606840afea0034ce599815ed950fd558981c/src/ddmd/dmodule.d and this is the prototype: extern (C) char* getcwd(char* buffer, size_t maxlen); it's deprecated and replaced by the ISO function: https://docs.microsoft.com/en-gb/cpp/c-runtime-library/reference/getcwd-wgetcwd okay - so: it's wrong to use that limit on linux and OSX. Linux PATH_MAX is 4096. OpenBSD is 1024. Linux paths are unlimited, apparently (OSX can have several k chars at least). And the Windows one should at least be PATH_MAX less a bit even without using long paths. (But then if you are going to use old winapi need to check its less than PATH_MAX if you extend). https://insanecoding.blogspot.co.uk/2007/11/pathmax-simply-isnt.html on Windows and indeed other operating systems we already have the correct code to get current working directory. so we just need to update dmd to use this. https://github.com/dlang/phobos/blob/v2.076.0/std/file.d#L2681 "
Catching C++ Exceptions in D - Windows and Linux
Hi. I'm here in HK with Ilya, Atila, John Colvin, and Jonathan Davis. I wondered what the current state of D catching C++ exceptions was on Linux and Windows. I know that some work was done on making this possible, and my understanding is that it is, more or less - just wondered what the rough corners might be. Thanks. Laeeth.
Re: Dub command line knowledge sought
On Friday, 23 June 2017 at 08:26:21 UTC, Russel Winder wrote: On Fri, 2017-06-23 at 08:11 +, Nicholas Wilson via Digitalmars-d- learn wrote: On Friday, 23 June 2017 at 07:51:51 UTC, Russel Winder wrote: > I am likely just staring at and missing the data needed: > > How does one invoke dub to fetch and build, and put into a > place other that ~/.dub/… a package from Dub? dub fetch foo --version=1.0.0 mv ~/.dub/packages/foo-1.0.0 /desired/path dub add-path /desired/path dunno about how to do that in one step. This is what I feared. The real awkwardness here is that Dub stores all the compilation products nicely separated by platform, compiler, and configuration, and puts the last build in a nice named place on the assumption there will only ever be one. So generally the ~/.dub/packages/foo-1.0.0 has to be moved as above. Using unit-threaded as an example: ~/.dub/packages/unit-threaded-0.7.24/unit-threaded/.dub/build/library-debug-linux.posix-x86_64-ldc_2073-1AF38A4B25224ABFB5BB5ED68A0E4633 Is the location of the last debug build on linux using ldc2, but what is that hash code, how to make that deducible so that it isn't necessary to move, just build. Check out the Kaleidic fork maintained by John Colvin - currently in his personal repository on github. We submitted back our changes but ended up being quite a lot so not all have been accepted yet. Allows you to change location of dub repos - useful to avoid work and personal interacting without having to use containers or VMs, but it may be easy enough to change path on an adhoc basis as you wish to do. Don't recall right now, but take a look.
Re: Is D slow?
On Friday, 9 June 2017 at 19:29:35 UTC, Honey wrote: On Friday, 9 June 2017 at 18:32:06 UTC, Steven Schveighoffer wrote: Wow, so that's how D code would look like if it were C++ :) Well, I cannot (and did not try to) hide where I am coming from. ;-) The results are quite disappointing. What seems particularly strange to me is that -boundscheck=off leads to a performance decrease. That doesn't make much sense, but I'm not an ldc2 user. However, it does note in the help that -release disables bounds checks already. Sounds like a bug, then. I did replicate that issue on my box, and mucking around with the implementation didn't help. In answer to the subject, no D is not slow. However, it's quite possible that std.algorithm.bringToFront is slower than std::rotate, or SortedRange.upperBound is slower than std::upper_bound, or both. I don't think it's a design issue per se, probably more of an implementation issue. Thank you for confirming the results and your factual explanation notwithstanding my pointed question. ;-) Maybe I was expecting too much given Andrei's performance oriented talks. I realize that the conceptual groundwork is more important than a concrete implementation that can be easily improved. However, I think that real world out-of-the-box performance - particularly with respect to toy examples (since those are small enough to be literally translated) - is important for prospects to gain confidence in buying into D. At the current state, at least for such benchmarks, I think, I should not rely on standard library facilities. Unfortunately, that does not increase my confidence. Real world and toy are mutually exclusive categories, and I am not sure the empirical evidence is consistent with your perspective that it is what prospects need to see before exploring D, though that is an interesting perspective. I highly recommend Weka.io talks if you would like to see how one larger D user has found performance in practice. If you are expecting a perfectly finished glossy product then I don't think that - at least in the current year - D will be necessarily for you. Polish improves every year, but it's not the principal focus of the community currently. It's more the opposite - pay the price up front in different ways and reap the returns again and again over time.
Re: Avast virus warning?
On Monday, 5 June 2017 at 16:31:04 UTC, Anonymouse wrote: I just sent a pre-compiled .exe of my project to a friend, and his Avast anti-virus promptly quarantined it and sent it off for analysis. I tried sending him a Hello World[1] with the same results. Is this something common for d programs? Anything I can do to work around it from my end? [1]: http://www.mediafire.com/file/fc51qz141r3ns6r/helloworld.exe https://forum.avast.com/index.php?topic=203573.0 It's not that people do bad things with D. It's that dmd generates code that doesn't look like anything it has seen before.
Re: Make enum auto castable
On Sunday, 4 June 2017 at 22:52:55 UTC, Mike B Johnson wrote: I am dealing with some COM stuff and some functions use VARIANT, which can hold an enum. Instead of having to manually convert the enum(and for that matter, other things) to VARIANT, is it possible to have them automatically converted? This is because an enum is always convertible to a VARIANT but not the other way around. So, when a enum is passed to a function accepting a VARIANT, it should just work. Overloading is not an option. Not sure if this breaks your requirement but you could generate overloads or rather templated version of your variant accepting function with a mixin that introspects on functions with a certain uda in the module. See excel-d for an example.
Re: D scripting in D
On Friday, 2 June 2017 at 17:22:20 UTC, Stefan Koch wrote: On Friday, 2 June 2017 at 16:13:03 UTC, Laeeth Isharc wrote: On Friday, 2 June 2017 at 02:06:27 UTC, Mike B Johnson wrote: [...] Stefan Koch has written a good part of an interpreter for D AST, no? And I guess the lexing and parsing stage doesn't take so long, whereas not having to link saves much time. So is there any way to repurpose his work on ctfe to have an interpreter that you can call at run time, set context for and get return values back? No there is not. First it's woefully incomplete and secondly it's tightly bound to dmd, and it's semantic phases Maybe it's incomplete today, but some day it will be or could be finished. And whilst I understand the difficulty of working with the dmd codebase as it's structured today, why is it intrinsically a problem that your work is bound to dmd? T - interesting idea about ldc though that's a bit slower than dmd.
Re: D scripting in D
On Friday, 2 June 2017 at 02:06:27 UTC, Mike B Johnson wrote: I wonder if it is possible to somehow turn D in to a scripting language that can be run inside D? The point? To have the same uniform syntax for quickly developing scripts that can then be easily transferred, if desired, in to a complete binary. e.g., suppose I am working in some type of analysis software. Use a Dscript like feature to develop and test different analysis algorithms quickly(rather than using the compile and execute model)... then once everything is working, move the code to a D file and compile it directly. Since the syntax would be identical(or nearly so) it would be quite easy to copy and paste... unlike, say, using lua or some other non-D like scripting language. Stefan Koch has written a good part of an interpreter for D AST, no? And I guess the lexing and parsing stage doesn't take so long, whereas not having to link saves much time. So is there any way to repurpose his work on ctfe to have an interpreter that you can call at run time, set context for and get return values back?
Re: Code improvement for DNA reverse complement?
On Friday, 19 May 2017 at 07:29:44 UTC, biocyberman wrote: I am solving this problem http://rosalind.info/problems/revc/ as an exercise to learn D. This is my solution: https://dpaste.dzfl.pl/8aa667f962b7 Is there some D tricks I can use to make the `reverseComplement` function more concise and speedy? Any other comments for improvement of the whole solution are also much appreciated. You might try using ldc in release mode. With dmd, cost of ranges and so on might be higher than doing it in a more direct low-level way. With ldc that difference often seems to shrink to insignificance as its optimisation is more powerful. If you care about performance then use ldc or gdc. (I have less experience with gdc, but probably similar). But then it's best to compare results under ldc because the ratios of different options will be different vs dmd.
Re: Processing a gzipped csv-file by line-by-line
On Friday, 12 May 2017 at 00:18:47 UTC, H. S. Teoh wrote: On Wed, May 10, 2017 at 11:40:08PM +, Jesse Phillips via Digitalmars-d-learn wrote: [...] H.S. Teoh mentioned fastcsv but requires all the data to be in memory. Or you could use std.mmfile. But if it's decompressed data, then it would still need to be small enough to fit in memory. Well, in theory you *could* use an anonymous mapping for std.mmfile as an OS-backed virtual memory buffer to decompress into, but it's questionable whether that's really worth the effort. If you can get the zip to decompress into a range of dchar then std.csv will work with it. It is by far not the fastest, but much speed is lost since it supports input ranges and doesn't specialize on any other range type. I actually spent some time today to look into whether fastcsv can possibly be made to work with general input ranges as long as they support slicing... and immediately ran into the infamous autodecoding issue: strings are not random-access ranges because of autodecoding, so it would require either extensive code surgery to make it work, or ugly hacks to bypass autodecoding. I'm quite tempted to attempt the latter, in fact, but not now since it's getting busier at work and I don't have that much free time to spend on a major refactoring of fastcsv. Alternatively, I could possibly hack together a version of fastcsv that took a range of const(char)[] as input (rather than a single string), so that, in theory, it could handle arbitrarily large input files as long as the caller can provide a range of data blocks, e.g., File.byChunk, or in this particular case, a range of decompressed data blocks from whatever decompressor is used to extract the data. As long as you consume the individual rows without storing references to them indefinitely (don't try to make an array of the entire dataset), fastcsv's optimizations should still work, since unreferenced blocks will eventually get cleaned up by the GC when memory runs low. T I hacked your code to work with std.experimental.allocator. If I remember it was a fair bit faster for my use. Let me know if you would like me to tidy up into a pull request. Thanks for the library. Also - sent you an email. Not sure if you got it. Laeeth
Re: How to continue after the book?
On Wednesday, 29 March 2017 at 05:53:22 UTC, I Lindström wrote: Thanks all. Your answers gave me a lot more confidence in starting. What I've always found to be the hardest is to know what you can do, and that's what I use books for. "Can" in the sense of what's possible and how. These forums and the docs on the site have given me a pretty decent idea about how to proceed and do things. It's not the work itself that I dread, but more the "can I find out how to do something", but some of your answers touched on this and I will take heed. A lot to learn. You can expect me to come pester you guys if I hit a snag. Heh. plus IRC chat if you get stuck.
Re: Optimising JSON parsing leads to wierd execution timings
On Monday, 27 March 2017 at 09:05:00 UTC, Seb wrote: On Saturday, 25 March 2017 at 06:53:58 UTC, xtreak wrote: Hi, Sorry for the double post. I have asked a question at Stackoverflow regarding this : https://stackoverflow.com/questions/42992507/get-float-value-out-of-jsonvalue-in-dlang . I have a `rating` field that might have 3 which parses to JSONValue.integer or 3.4 which parses to JSONValue.floating and I need to use float. I was using exceptions that was causing the program to be slow as per the profile information. Now I have used an if clause as suggested by Adam in the answer. Though the execution time for the relevant function that used exceptions earlier reduced the program was still taking the same time. [...] If you need a fast JSON library, have a look at https://github.com/tamediadigital/asdf or https://github.com/s-ludwig/std_data_json Or search forum for fastest Json parser in the world. I use asdf though, which is fast enough and has a nice interface.
Re: How to continue after the book?
On Tuesday, 28 March 2017 at 07:27:31 UTC, I Lindström wrote: After getting the basics down, how did you continue when learning programming in general? I do have a need for which I've been trying out a few languages and D seems by far the best for me. Should I just start doing that project and learn as I go by googling and asking here, or are there some other things you did before starting your first "real" project. I learnt a lot by reading others' code in a domain that interested me intrinsically. Originally the same way Walter did - typing in programs from magazines and then changing them. Pick something that suits your skill level and is a bit beyond this - growth comes from pushing your limits and it should be a bit uncomfortable if you are doing it right.
Re: high performance client server solution in D?
On Thursday, 23 March 2017 at 13:07:54 UTC, biocyberman wrote: I am considering to use D and its library to build a high performance client-server application. The client will be a cross platform (Windows, Mac, Linux) GUI program that can synchronize analysis results with the remote central server, and analyze data locally. It will also visualize big data files (about 10GB of binary data each). The term 'high performance' means it can serve several hundreds users with desktop application speed. Further more, heavy computation tasks will be done locally on the client side. This description is still vague, I know. But that's the best I can give for now. I would choose 'dlangui' and check 'vibe.d' for a start. However, I do not need to access the central server via web browsers. Hope that you can give some thoughts about this design, what GUI library to use, and what back-end library to use. cerealed, orange, and msgpack for serialisation/deserialisation (see code.dlang.org). Depends how complicated your D structures are. I found of those three msgpack handles more (for what I am doing), but vibed json serialiseToPrettyJSON (sp?) etc covers some that msgpack chokes on (trouble with Algebraic and TaggedAlgebraic from what I remember). However for speed asdf has better performance. But I guess for binary data you may as well use msgpack. I use nanomsg over 0MQ as I was burned a couple of years back by a problem I encountered using Salt Stack (which was then based on 0MQ) where I think if the server hangs up mid-connection than it's not completely straightforward to get the clients talking to it again. 0MQ is certainly more stable. I find nanomsg easier to use, and I haven't found too many problems with it (IPC on Windows not implemented/didn't work). If you use either, you might want to protect sockets if it isn't on internal network. stunnel worked fine for me. Dlangui was fine for GUI and very happy with it on linux and Windows. Much quicker to develop for that than in browser, was our experience.
Re: How to use C code in D
On Thursday, 23 March 2017 at 18:10:20 UTC, Dillen Meijboom wrote:
Hi there,
I'm learning D for a while because it's really easy to use
C-code in D.
The problem is that I don't really get how to deal with the
data structures defined in C in D.
At one time for instance I've tried to get all environment
variables on a POSIX system. I succeeded but I think it can be
done way easier.
So my question basically is:
1. How can I learn to use C code in D? Is there any good
tutorial or other packages that deal with this a lot?
2. Is the way I'm currently doing it okay or is it wrong and do
I need to do something else?
As a reference, I'm currently using the following D code to get
all environment variables:
```
import std.array;
import std.conv: to;
import std.string: fromStringz;
extern (C) extern const char** environ;
string[string] getenv() {
string[string] env;
char* line;
for (auto data = cast(char**)environ; (line = *data) != null;
++data) {
auto keyval = to!string(fromStringz(line)).split('=');
env[keyval[0]] = keyval[1];
}
return env;
}
```
https://dlang.org/phobos/std_process.html#.environment means it's
already done for you.
C global variables are __gshared.
Reading other people's source code - starting with standard
library and maybe look at some of the bindings and wrappers on
code.dlang.org.
IRC chat to get a quick answer most of the time.
CSV - 2.5 GB/sec (sadly C++)
On Tuesday, 26 January 2016 at 22:36:31 UTC, H. S. Teoh wrote: So the moral of the story is: avoid large numbers of small allocations. If you have to do it, consider consolidating your allocations into a series of allocations of large(ish) buffers instead, and taking slices of the buffers. Thanks for sharing this, HS Teoh. I tried replacing allocations with using a Region from std.experimental.allocator (with FreeList and Quantizer on top), and then just deallocating everything in one go once I am done with the data. Seems to be a little faster, but I haven't had time to measure it. Just came across this C++ project, which seems to have astonishing performance. 7 minutes for reading a terabyte, and 2.5 to 4.5 GB/sec for reading file cold. That's pretty impressive. (Obviously they read in parallel, but I haven't yet read source to see what the other tricks might be). It would be nice to be able match that in D, though practically speaking it's probably easiest just to wrap it: http://www.wise.io/tech/paratext https://github.com/wiseio/paratext
ACM paper: CPU is the new bottleneck
From Jan 2016: https://queue.acm.org/detail.cfm?id=2874238 For the entire careers of most practicing computer scientists, a fundamental observation has consistently held true: CPUs are significantly more performant and more expensive than I/O devices. The fact that CPUs can process data at extremely high rates, while simultaneously servicing multiple I/O devices, has had a sweeping impact on the design of both hardware and software for systems of all sizes, for pretty much as long as we've been building them. This assumption, however, is in the process of being completely invalidated. === Even today, a Spark committer says that for Spark CPU is already often the bottleneck (for decompression): https://www.oreilly.com/ideas/investigating-sparks-performance
Re: From Python to Dlang
On Tuesday, 18 October 2016 at 12:03:54 UTC, Alfred Newman wrote: Hello and greetings, I'm a brand new D developer coming from Python. I decided to move to D, mainly because it's a compiled language and has a great runtime speed (and I don't feel confortable at Cython environment at all). And of course, D has a nice community and the language has nice syntax too... and is a joy to code with. However, I have some important production code in Python to migrate to D and every help will count ! So, can you pls guys suggest me any resource like "D for a Python Developer" or so ? BTW, I just ordered the "D Programming Language" book from AA. Cheers Welcome. Bear in mind that you can embed python in your D code using pyd if you want to do the port in stages. You can write D libraries for python too - that's a bit fiddlier though, and docs are a bit stale.
Re: obscure messages relating to linking and imports in 2.071.1
On Wednesday, 5 October 2016 at 12:12:24 UTC, Basile B. wrote: On Wednesday, 5 October 2016 at 11:45:49 UTC, Laeeth Isharc wrote: I noticed the problem before - previously it was my fault. I had a circulator dependency where A imported B, B did a selective import of C and C imported A selectively. That led to link problems with module constructors. [...] It looks like there's a conflict between HackerPilot's fork and the phobos version. But HackerPilot's fork is not used anymore. Have you pulled the latest versions recently (or dub upgrade if you use DUB) ? (verification: https://github.com/economicmodeling/containers/commit/528cae2880c0e5faa57d192621ad0533b0124b7b) Aha, thank you. Obvious after the fact now you point it out, and I guess that must be it. Laeeth
obscure messages relating to linking and imports in 2.071.1
I noticed the problem before - previously it was my fault. I had a circulator dependency where A imported B, B did a selective import of C and C imported A selectively. That led to link problems with module constructors. Here I noticed it in a different context. Simple two-page main code imports dateparser from code.dlang.org. dateparser uses emsi containers, which use std.experimental.allocator. If I don't import std.experimental.allocator in main module (where it isn't actually used) I get the link errors below. If I import it, it goes away. Might be fixed in 2.071.2 - will upgrade when I can. Not certain it's a bug and if it is whether it might be fixed, but if you think I should I will report. I can't share code publicly, but I can share gist privately. Laeeth. Linking... /usr/lib/libphobos2.a(gc_allocator_2c45_42b.o):(.bss+0x0): multiple definition of `_D3std12experimental9allocator12gc_allocator11GCAllocator6__initZ' ../../../../../../../../home/laeeth/.dub/packages/experimental_allocator-2.70.0-b1/libexperimental_allocator.a(gc_allocator_b2_42b.o):(.bss+0x0): first defined here /usr/lib/libphobos2.a(gc_allocator_2c45_42b.o): In function `_D3std12experimental9allocator12gc_allocator11GCAllocator8allocateMOFNemZAv': (.text._D3std12experimental9allocator12gc_allocator11GCAllocator8allocateMOFNemZAv+0x0): multiple definition of `_D3std12experimental9allocator12gc_allocator11GCAllocator8allocateMOFNemZAv' ../../../../../../../../home/laeeth/.dub/packages/experimental_allocator-2.70.0-b1/libexperimental_allocator.a(gc_allocator_b2_42b.o):../../../../../../../../home/laeeth/.dub/packages/experimental_allocator-2.70.0-b1/src/std/experimental/allocator/gc_allocator.d:(.text._D3std12experimental9allocator12gc_allocator11GCAllocator8allocateMOFNemZAv+0x0): first defined here /usr/lib/libphobos2.a(gc_allocator_2c45_42b.o): In function `_D3std12experimental9allocator12gc_allocator11GCAllocator10reallocateMOFKAvmZb': (.text._D3std12experimental9allocator12gc_allocator11GCAllocator10reallocateMOFKAvmZb+0x0): multiple definition of `_D3std12experimental9allocator12gc_allocator11GCAllocator10reallocateMOFKAvmZb' ../../../../../../../../home/laeeth/.dub/packages/experimental_allocator-2.70.0-b1/libexperimental_allocator.a(gc_allocator_b2_42b.o):../../../../../../../../home/laeeth/.dub/packages/experimental_allocator-2.70.0-b1/src/std/experimental/allocator/gc_allocator.d:(.text._D3std12experimental9allocator12gc_allocator11GCAllocator10reallocateMOFKAvmZb+0x0): first defined here /usr/lib/libphobos2.a(gc_allocator_2c45_42b.o): In function `_D3std12experimental9allocator12gc_allocator11GCAllocator22resolveInternalPointerMOFPvZAv': (.text._D3std12experimental9allocator12gc_allocator11GCAllocator22resolveInternalPointerMOFPvZAv+0x0): multiple definition of `_D3std12experimental9allocator12gc_allocator11GCAllocator22resolveInternalPointerMOFPvZAv' ../../../../../../../../home/laeeth/.dub/packages/experimental_allocator-2.70.0-b1/libexperimental_allocator.a(gc_allocator_b2_42b.o):../../../../../../../../home/laeeth/.dub/packages/experimental_allocator-2.70.0-b1/src/std/experimental/allocator/gc_allocator.d:(.text._D3std12experimental9allocator12gc_allocator11GCAllocator22resolveInternalPointerMOFPvZAv+0x0): first defined here /usr/lib/libphobos2.a(gc_allocator_2c45_42b.o): In function `_D3std12experimental9allocator12gc_allocator11GCAllocator10deallocateMOFAvZb': (.text._D3std12experimental9allocator12gc_allocator11GCAllocator10deallocateMOFAvZb+0x0): multiple definition of `_D3std12experimental9allocator12gc_allocator11GCAllocator10deallocateMOFAvZb' ../../../../../../../../home/laeeth/.dub/packages/experimental_allocator-2.70.0-b1/libexperimental_allocator.a(gc_allocator_b2_42b.o):../../../../../../../../home/laeeth/.dub/packages/experimental_allocator-2.70.0-b1/src/std/experimental/allocator/gc_allocator.d:(.text._D3std12experimental9allocator12gc_allocator11GCAllocator10deallocateMOFAvZb+0x0): first defined here /usr/lib/libphobos2.a(gc_allocator_2c45_42b.o):(.bss+0x1): multiple definition of `_D3std12experimental9allocator12gc_allocator11GCAllocator8instanceOS3std12experimental9allocator12gc_allocator11GCAllocator' ../../../../../../../../home/laeeth/.dub/packages/experimental_allocator-2.70.0-b1/libexperimental_allocator.a(gc_allocator_b2_42b.o):(.bss+0x1): first defined here /usr/lib/libphobos2.a(gc_allocator_2c45_42b.o): In function `_D3std12experimental9allocator12gc_allocator11GCAllocator7collectMOFNeZv': (.text._D3std12experimental9allocator12gc_allocator11GCAllocator7collectMOFNeZv+0x0): multiple definition of `_D3std12experimental9allocator12gc_allocator11GCAllocator7collectMOFNeZv'
Re: D to C++
On Wednesday, 31 August 2016 at 12:19:33 UTC, Cauterite wrote: On Wednesday, 31 August 2016 at 11:43:12 UTC, Nick wrote: That's quite nice, but not what I'm looking for. What Calypso does, as far as I can see, is to make it possible to compile C++ and D together. I'm looking for a compiler that takes in D code and spits out either C or C++ code. Your best option would be to use LDC with a C backend: https://www.google.com/search?q=llvm++c+backend No idea how well supported this is, I've never used LLVM myself. Julia guys resurrected the C back end for LLVM. Chap here posted about using this to compile D to bitcode to C to JS (last but using emscripten). Even without last stage, doesn't sound a great idea if you are under time pressure, as the whole process is quite fragile. It would be nice to have a robust C backend, even bearing in mind all problems with UB and difficulties one can imagine that might cause.
Re: Nogc Associative Array?
On Thursday, 25 August 2016 at 18:14:42 UTC, Illuminati wrote: Does D have a non-gc based associative array? If not, what would be a good way to go about creating one? See EMSI containers in code.dlang.org
simple idea for error messages
when it cannot match a prototype eg:
module foo;
enum Foo
{
foo
}
void bar(Foo foo)
{
}
module bar;
enum Foo
{
foo
}
void fooBar(Foo foo)
{
bar(foo);
}
rather than complain the type X does not match the type X - an
unhelpful message - could the compiler not check to see if the
type descriptions are identical, and if so print a clarifying
message about which modules they come from. it's easy the second
time, but another source of friction in learning the language in
the beginning.
Rust piece on integer safety
All the design/discussion/implementation of this scheme for handling integer overflow would be wasted if it didn’t actually find any bugs in practice. I personally have had quite a few bugs found nearly as I write them, with expressions like cmp::max(x - y, z) (they never hit the internet, so no links for them), especially when combined with testing infrastructure like quickcheck. The overflow checks have found bugs through out the ecosystem; for instance, (not exhaustive!) the standard library the compiler the built-in benchmark harness Servo image url webrender Beyond Rust, there’s a lot of evidence for the dangers of integer overflow and desire for detecting/protecting against them. It was on the CWE/SANS list of top 25 errors in 2011, languages like Swift will unconditionally check for overflow, and others like Python 3 and Haskell will avoid overflow entirely by default, via arbitrary precision integers. Furthermore, in C, several compilers have options to both make signed overflow defined as two’s complement wrapping (-fwrapv) and to catch it when it does happen (-fsanitize=signed-integer-overflow). http://huonw.github.io/blog/2016/04/myths-and-legends-about-integer-overflow-in-rust/
Re: Using D in Android App
On Saturday, 16 April 2016 at 04:04:24 UTC, Justice wrote: Is it difficult to create a D business like app and connect it to android through java for the interface? I'd rather create all the complex stuff in D and either use it natively through java(I need a UI). If it is workable, can the same be said for IOS(just recompile the D source to the IOS architecture then use it in an IOS app for the ui)? Depending on your budget, you could also look at Corona - Lua can call D easily. I haven't used on Android, but Qt QML may be worth a look (D bindings to QML). Wrt to D on android I would try porting some existing code and see how that goes. I think stuff seemed to work but because you don't yet have so much company then it may be hard to be sure you might not hit some problems till you try.
simple range question
suppose I have a forward or random access range. what's the best way to compare each element with the element 4 elements prior to that element? I could map each element to a tuple of the element and the element 4 bars previously and do it that way. any neater way ?
size_t index=-1;
should it be a compiler warning to assign a negative literal to an unsigned without a cast ?
Re: Is D a good choice for embedding python/octave/julia
On Tuesday, 15 March 2016 at 05:56:36 UTC, Ellery Newcomer wrote:
On 03/13/2016 02:36 PM, Laeeth Isharc wrote:
InterpContext context = new InterpContext();
context.py_stmts(outdent("
import numpy
a = numpy.eye(2, dtype='complex128')
"));
context.a.to_d!(Complex!double[][] )();
nitpicking, but the outdent is unnecessary, py_stmts calls it.
hm. maybe that should be documented..
I was in a hurry so just copy and pasted from the unit test... I
never do that myself.
Re: Is D a good choice for embedding python/octave/julia
On Sunday, 13 March 2016 at 18:42:59 UTC, Bastien wrote:
The sticking point is unless I commit the rest of my life to
maintaining this software, I can't write it all in D. The
algorithms change/are improved yearly; the output format from
the instrument changes once in a while and therefore these need
to be easily scripted/modified by other (non-programming)
scientists and the community that only really know python and
octave.
It's pretty easy to use D from python and python from D. The
documentation for PyD isn't so great, but once you have figured
it out, it's easy. Start with something very small (the examples
and tests in the PyD repo are the best documentation) and work up
from there.
You can even embed D in a Jupyter/iPython notebook, and write
some cells in D and some in python and have them call each other.
See PydMagic by John Colvin.
https://github.com/DlangScience/PydMagic
D can talk to Julia via C linkage already. Just extern(C) when
you declare your D function, and I guess use C style arrays.
I started porting julia.h to D, but didn't get have time to
finish. Ilya Yaroshenko, who created the ndslice library in
std.experimental, will be working on julia integration in coming
months.
By the way, you can also embed R in D and call D from R - see
work by bachmeier on bitbucket. That's very important because of
the huge numbers of R libraries. Although R is slow, there
apparently shouldn't be so much overhead in calling a C library
written for R from D.
I guess my resilience to using D for the algorithms is because
with python, I have access to numpy and matplotlib. There do
seem to be some ongoing developments though:
http://forum.dlang.org/post/mailman.4923.1434903477.7663.digitalmar...@puremagic.com
You already have access to matplotlib from D - see here (and
scroll down or search for matplotlib):
https://d.readthedocs.org/en/latest/examples.html
And you can call numpy from D, but there is some overhead, and I
wouldn't want to do it inside a tight loop.
From PyD unit tests:
InterpContext context = new InterpContext();
context.py_stmts(outdent("
import numpy
a = numpy.eye(2, dtype='complex128')
"));
context.a.to_d!(Complex!double[][] )();
Re: Things that keep D from evolving?
On Wednesday, 10 February 2016 at 20:21:22 UTC, Chris Wright wrote: On Wed, 10 Feb 2016 08:57:51 +, thedeemon wrote: Currently (at least last time I checked) GC pauses the world, then does all the marking in one thread, then all the sweeping. Right. We can do the marking in several parallel threads (this is much harder to implement but still doable), Parallel marking would not be a breaking change by any means. No user code runs during GC collections, so we can do anything. The major fly in the ointment is that creating threads normally invokes the GC, since Thread is an object, and invoking the GC during a collection isn't the best. This can be solved by preallocating several mark threads. Then you just divide the stack and roots between those threads. Moderately annoying sync issues This doesn't guarantee an even distribution of work. You can solve that problem with a queue, though that requires locking. The main wrinkle is writing a bit to shared data structures, which can be slow. On the other hand, in the mark phase, we're only ever going to write the same value to each, so it doesn't matter if GC thread A . I don't know how to tell the CPU that it doesn't have to read back the memory before writing it. and we can kick the sweeping out of stop-the-world pause and do the sweeping lazily This would be a breaking change. Right now, your destructors are guaranteed to run when no other code is running. You'd need to introduce locks in a few places. I'm not saying this is a bad thing. I think people generally wouldn't notice if we made this change. But some code would break, so we'd have to stage that change. Anyway, I'm hacking up parallel mark phase to see how it would work. I could use some GC benchmarks if anyone's got them lying around. https://github.com/D-Programming-Language/druntime/tree/master/benchmark/gcbench
Re: Things that keep D from evolving?
On Tuesday, 9 February 2016 at 13:01:29 UTC, NX wrote: On Monday, 8 February 2016 at 22:21:50 UTC, Laeeth Isharc wrote: The GC itself may still be far from perfect but its much better than it was, and there are more options now. I have found emsi containers (built on top of Andrei's allocator) pretty nice myself for my own use. Well, GC being better than it used to be doesn't change the fact it's still the worst of it's kind. I don't know if this[1] work actually got released or merged but looks like it's abandoned. Pretty sad as it seemed very promising. Anyway, I was expecting a lot more people to tell their specific problems, like "bla bla design desicion makes ARC incredibly dangerous and we can't properly interface with Objective-C without that" or like "bla bla D feature overlaps with some other stuff and requires redesign to be solved" or maybe "being unsafe (@system) by default breaks the deal"... GC is just one of the hundreds of problems with D and it was an example rather than the main point in this thread but thanks for anyone who replied. [1] http://forum.dlang.org/thread/mailman.655.1399956110.2907.digitalmar...@puremagic.com Thanks for pointing this one out. Opportunity comes dressed in work clothes,and I guess that until someone takes the initiative to integrate this with the newest version of the runtime / GC then nothing will happen. It's not true that there are no professional opportunities in D, as some people say, and I can say that for some people at least impressive contributions to the language and community have paid off personally even though it was a labour of love and not motivated by that. Good programmers don't grow on trees, and one benefit of the current size of the D community is that it's easier to make an impact and easier to stand out than in a much more crowded and mature domain where one person can only hope to achieve incremental progress. My impression is that barriers to adoption are fairly well understood by now and it's a matter of time and hard work for them to be addressed step by step. It's not only addressing negatives but also completing positive things that will help. Ndslice and porting BLAS on the numerical side and the interface with R will both increase the attractiveness of D on finance, not a small area. It's not yet mature, but knowing one can use all the R libraries is already a big win.
Re: Things that keep D from evolving?
On Monday, 8 February 2016 at 17:15:11 UTC, Wyatt wrote: On Monday, 8 February 2016 at 16:33:09 UTC, NX wrote: I see... By any chance, can we solve this issue with GC managed pointers? Maybe we could. But it's never going to happen. Even if Walter weren't fundamentally opposed to multiple pointer types in D, it wouldn't happen. You asked about things that prevent improvement, right? Here's the big one, and a major point of friction in the community: Walter and Andrei refuse to break existing code in pursuit of changes that substantially improve the language. (Never mind that code tends to break anyway.) -Wyatt I have no special knowledge but strikes this observer that they are serious about working on solutions (whether that's a better GC or alternatives or both). But some patience required as its not such a straightforward problem and its better to take time than rush and make a mistake. It wasn't all that long ago that Andrei quit and I guess he moved across country and it certainly takes time to sort out one's home office and find a new working pattern. The discussions in the mailing list are quite interesting although beyond my technical knowledge for now. The GC itself may still be far from perfect but its much better than it was, and there are more options now. I have found emsi containers (built on top of Andrei's allocator) pretty nice myself for my own use.
Re: Setting up dmd properly
On Tuesday, 12 January 2016 at 19:38:32 UTC, Jason Jeffory wrote: It seems the whole state of affairs in programming is "Lets do the most minimal work to get X to work in environment Y. To hell with everything else!". The programmers tend to do the most minimal work to code stuff that they can get away with. Since people aren't being paid to do this, and it's not enjoyable for many to make things universally easy across different environments once someone has solved their own problem, you can hardly object to the behaviour - particularly because different people are good at different things, and the guy who creates a project may not be the same guy needed to make it easy to use. Then it's more a question of treating it as a challenge to be solved. It's quite amazing how much a relatively small number of people has accomplished, and it's something of a hazard of open-source that instead of gratitude people receive far more criticism and complaint. (They say a 2:1 balance of positive:negative comments is needed for a healthy relationship). So it's an engineering or institutional challenge - how does one achieve this as a community? This isn't 1984 but coding quality has no increased much since then. A little hyperbolic? ;) We do seem to have higher quality problems today, but do you recall what code from the 80s was like? I've virtually had no problems with it. MS did good job of modernizing the toolchain... Most people that code on linux think that it should be "hard" and gui's suck, that programming is suppose to be a hazing ritual. They setup their system to work for them, and it works... anyone with problems must be ignorant and not "pro programmers". It's kinda this elitist attitude. They spend more time solving 1%'er problems than creating tools that *just* work for 99% of the people. When problems occur it is never their fault but the fault of the ignorant cave man trying to become an godly programmer. Do you think that's actually representative of the attitudes of people here? I haven't seen that. But work won't get done without a plan and without someone to actually do it and one can't force people to do things they don't want to do. A big problem is people don't know what to work on, and maybe some kind of systematic approach to identify problem needs would help. Just search "openGL dmd"(28k) and about 80% of the results are people having problems with getting openGL working with D. "openGL dmd error" has 1M results, thats nearly 30 times the results. It would be a good idea to systematize this and produce a web report so one can see in a more structured way where the biggest difficulties are. I have been talking to Adam a bit about ways we could do this using forum history. I agree with your observation that there is much friction in the way of a new user learning D and that many simply won't persevere long enough. That's nonetheless a better problem to have than having an intrinsically inferior product - one just needs to establish a process, and to have some way of organizing efforts to address these difficulties (which may include funding to a certain extent). I think it's a necessary part of the way D has developed that people have focused on the language and core library first - it's not so long that it has been stable and ready to use and over time better tooling will unfold. (Constructive criticism and ideas may help this process). D has a long way to go to make it competitive... as long as the tooling sucks and there are problems with stupid issues such as coff vs omf, installation issues, ide issues, etc... it won't get off the ground. Depends what the competition is ;) Coff vs OMF will disappear in time as people move to 64 bit. Installation questions seem to be improving. IDE support keeps getting better. For many uses, these things are a one-off price for adopting D. Whether it's feasible to pay that depends on what you are doing and the people you are working with. The D "core" seems to be mainly interested in fixing and enhancing very niche issues in D instead of working on making it a viable and usable candidate for the masses. But it is in the nature of things that disruptive technologies start off as inferior in certain respects and it's only with time that they can be a superior competitor across the board to the dominant technologies. See Clayton Christensen's work "The Innovator's Dilemma". It is what it is, and one can't wave a magic wand to force people to work for free on shiny tools to make it easy. If that's what one wants, then one can do one's small part to encourage this to happen - work on that oneself and contribute it, file bugzilla issues, fund the D foundation (once it is ready). But simply complaining won't change anything. BTW I hardly think that memory allocation, containers, documentation, and web site (recent areas of focus by leadership) are
Re: dataframe implementations
On Thursday, 19 November 2015 at 22:14:01 UTC, ZombineDev wrote: On Thursday, 19 November 2015 at 06:33:06 UTC, Jay Norwood wrote: On Wednesday, 18 November 2015 at 22:46:01 UTC, jmh530 wrote: My sense is that any data frame implementation should try to build on the work that's being done with n-dimensional slices. I've been watching that development, but I don't have a feel for where it could be applied in this case, since it appears to be focused on multi-dimensional slices of the same data type, slicing up a single range. The dataframes often consist of different data types by column. How did you see the nd slices being used? Maybe the nd slices could be applied if you considered each row to be the same structure, and slice by rows rather than operating on columns. Pandas supports a multi-dimension panel. Maybe this would be the application for nd slices by row. How about using a nd slice of Variant(s), or a more specialized type Algebraic type? [1]: http://dlang.org/phobos/std_variant Not sure it is a great idea to use a variant as the basic option when very often you will know that every cell in a particular column will be of the same type.
Re: dataframe implementations
On Monday, 2 November 2015 at 13:54:09 UTC, Jay Norwood wrote: I was reading about the Julia dataframe implementation yesterday, trying to understand their decisions and how D might implement. From my notes, 1. they are currently using a dictionary of column vectors. 2. for NA (not available) they are currently using an array of bytes, effectively as a Boolean flag, rather than a bitVector, for performance reasons. 3. they are not currently implementing hierarchical headers. 4. they are transforming non-valid symbol header strings (read from csv, for example) to valid symbols by replacing '.' with underscore and prefixing numbers with 'x', as examples. This allows use in expressions. 5. Along with 4., they currently have @with for DataVector, to allow expressions to use, for example, :symbol_name instead of dv[:symbol_name]. 6. They have operation symbols for per element operations on two vectors, for example a ./ b expresses applying the operation to the vector. 7. They currently only have row indexes, no row names or symbols. I saw someone posting that they were working on DataFrame implementation here, but haven't been able to locate any code in github, and was wondering what implementation decisions are being made here. Thanks. What do you think about the use of NaN for missing floats? In theory I could imagine wanting to distinguish between an NaN in the source file and a missing value, but in my world I never felt the need for this. For integers and bools, that is different of course.
Re: dataframe implementations
On Tuesday, 17 November 2015 at 13:56:14 UTC, Jay Norwood wrote: I looked through the dataframe code and a couple of comments... I had thought perhaps an app could read in the header info and type info from hdf5, and generate D struct definitions with column headers as symbol names. That would enable faster processing than with the associative arrays, as well as support the auto-completion that would be helpful in writing expressions. Yes - I think that one will want to have a choice between this kind of approach and using associative arrays. Because for some purposes it's not convenient to have to compile code every time you open a strange file, and on the other hand the hit with an AA sometimes will matter. The situation at the moment for me is that I have very little time to work on a correct general solution for this problem myself (yet its important for D that we do get to one). I also lack the experience with D to do it very well very quickly. I do have a couple of seasoned people from the community helping me with things, but dataframes won't be the first thing they look at, and it could be a while before we get to that. If we implement for our own needs,then I will open source it as it is commercially sensible as well as the right thing to do. But that could be a year away. Vlad Levenfeld was also looking at this a bit. The csv type info for columns could be inferred, or else stated in the reader call, as done as an option in julia. In both cases the column names would have to be valid symbol names for this to work. I believe Julia also expects this, or else does some conversion on your column names to make them valid symbols. I think the D csv processing would also need to check if the The jupyter interactive environment supports python pandas and Julia dataframe column names in the autocompletion, and so I think the D debugging environment would need to provide similar capability if it is to be considered as a fast-recompile substitute for interactive dataframe exploration. Well we don't need to get there in a single bound - already just being able to do this at all is a big improvement, and I am already using D with jupyter to do things. It seems to me that your particular examples of stock data would eventually need to handle missing data, as supported in Julia dataframes and python pandas. They both provide ways to drop or fill missing values. Did you want to support that? Yes - we should do so eventually, and there's much more that could be done. But maybe a sensible basic implementation is a start and we can refine after that. I wrote the dataframe in a couple of evenings, so I am sure it can be improved, and even rearchitected. Pull requests welcomed, and maybe we should set up a Trello to organise ideas ? Let me know if you are in.
Re: good reasons not to use D?
On Friday, 30 October 2015 at 10:35:03 UTC, Laeeth Isharc wrote: I'm writing a talk for codemesh on the use of D in finance. Sorry - I wrote this in a hurry, and I should have said on my experience of using D in finance (not the whole sector, which is absolutely enormous and very diverse), and what lessons and general things might be useful for others considering D. I can imagine lack of fixed point might be a problem. Shouldn't this be in Phobos? If BigInt is, then why not fixed point? There is this library here - had some problems compiling it last time (dmd versions), but it seems at least somewhat thought-through: https://github.com/jaypha/fixed Not sure if I replied to the suggestion that D isn't suitable for enterprise use. That's a word with a nice penumbra of connotations. If you mean you shouldn't use it for a project in a multinational where thare are 100 people working on it and without extensive experience of using D by the key people already, then sure. If you mean that D isn't suitable for serious work where there are very large amounts of money at stake, and the organisation may be a decent size then I strongly disagree. The examples of Facebook's worp, Sociomantic, EMSI, and the German guys who do the Norwegian train system ought to be sufficient to make that point...
Re: good reasons not to use D?
On Tuesday, 3 November 2015 at 23:37:36 UTC, Chris wrote: On Friday, 30 October 2015 at 10:35:03 UTC, Laeeth Isharc wrote: Interesting. Two points suggest that you should use D only for serious programming: "cases where you want to write quick one-off scripts that need to use a bunch of different libraries not yet available in D and where it doesn't make sense to wrap or port them" - quick and dirty "where you have many inexperienced programmers and they need to be productive very quickly." - quick and awkward [what does this tell us about financial programming ...] This reason is true of any other language: "where you have a lot of code in another language (especially non C, non Python) and defining an interface is not so easy;" In fact, the reasons you give (apart from the point about GUI) are true of C++, C#, Java etc. too. It's a bit generic. I was thinking of D specific reasons like lack of support for mobile platforms (not 100% yet). So your average stock broker couldn't calculate some bogus numbers on his iPad while having a latte and a Pelegrino in a posh cafe off Wallstreet. He he he. GC and lack of mobile are real reasons not to use D. Thanks, Chris, appreciate the feedback. I gave talk yesterday to a select but interested audience. Slides should be up soon - couple typos and as Andrei points out a bit too dense. (It's easy to forget when you read very quickly that not everyone does, and I should be more considerate of an audience). I really would have liked to have spent a bit more time preparing, as many things came together on other fronts so I had to do the best I could with the time available. You're right that I made it sound like D isn't a language for non-serious programming - well not quite, but more in that direction - and I should find a way to be clear about this anyway. Fact is I use it for scripts all the time. But if I ask myself, would I recommend it to someone who won't also use it for serious things, or has to depend on getting a bunch of junior guys with little experience (and with a low tolerance for discomfort) no I would not. [what does this tell us about financial programming ...] I'm an elitist (albeit a kind, open-hearted one), and I think that the research about the differences between productivity across almost any domain is right and underappreciated, and that it matters because if you don't recognize this then you won't organize your business so as to make it possible for these people to perform at that level. It's because of that and because the language appeals to me that I am here. Paper is here: https://www.evernote.com/shard/s37/sh/12b86414-ed93-472d-9a6f-db223087d869/62f13d679581134ef1d368d5b57cd2b7 Yaron Minsky said that at Jane Street they had the luxury of only hiring good programmers (and so the possible difficulty of ocaml simply wasn't a factor). I applaud the sentiment, and agree. This somewhat american conception of the rockstar programmer simply misses the point. That isn't how these people are, and they won't all be paid gazillions of dollars. One just has to have the discernment to recognize talent, earn its respect (which can only be if you make yourself worthy of it), provide working conditions that are suited to it, and figure out how to address the particular challenges of working with creative people. I don't know about financial programming in general - it's a really big sector and requirements are not the same. Even within my narrow little part of it, what I am doing (or at least a good part of it) is a bit different from anyone else I know, and the organizational approach and its flexibility is certainly so. I wouldn't characterize the difficulty level as high - what it will do is quite interesting, but in the stages I'm at for now, it's by far not the most technically difficult thing I have worked on. But I have a lot to achieve with somewhat limited resources in the beginning, which makes the challenge more interesting. This reason is true of any other language: "where you have a lot of code in another language (especially non C, non Python) and defining an interface is not so easy;" True. But it doesn't matter for Java as you aren't going to be limited by libraries and if you're thinking about using Java it's something obvious to you. Whereas if you know nothing about D (and nobody in the audience had even downloaded DMD) then it's not (and it matters much more for D). And if you want to speak credibly, it's much better to be honest - if you got it, you don't need to make it flashy. I don't see that many talks on other emerging languages by proponents talking about the rough edges, and maybe that's because it's a bad idea, but maybe it's not. If I were being paid to sell D (a job I would never take if I could not speak about the rougher aspects) then it might be different - but it was a talk from a practitioner's perspective,
Re: good reasons not to use D?
On Wednesday, 4 November 2015 at 12:08:19 UTC, Laeeth Isharc wrote: this hardly matters for Java, C++, Python etc because mostly you won't need to use a bunch of different libraries. I meant mostly you won't need to go outside that ecosystem to use a bunch of different libraries whereas with D probably you will - and whilst it is not very difficult, writing it in D as a beginner or intermediate level programmer and calling a C library will take more time and effort than a guy of the same level doing the same in python (and I expect Go). Doesn't mean you shouldn't do it if you're going to use the library intensively - it's a price you pay once, and it might be small in the bigger picture - but if you are using a gazillion different libraries, each of which will need to be wrapped, you might want to think about whether D is the right tool for that task.
Re: dataframe implementations
On Monday, 2 November 2015 at 13:54:09 UTC, Jay Norwood wrote: I was reading about the Julia dataframe implementation yesterday, trying to understand their decisions and how D might implement. From my notes, 1. they are currently using a dictionary of column vectors. 2. for NA (not available) they are currently using an array of bytes, effectively as a Boolean flag, rather than a bitVector, for performance reasons. 3. they are not currently implementing hierarchical headers. 4. they are transforming non-valid symbol header strings (read from csv, for example) to valid symbols by replacing '.' with underscore and prefixing numbers with 'x', as examples. This allows use in expressions. 5. Along with 4., they currently have @with for DataVector, to allow expressions to use, for example, :symbol_name instead of dv[:symbol_name]. 6. They have operation symbols for per element operations on two vectors, for example a ./ b expresses applying the operation to the vector. 7. They currently only have row indexes, no row names or symbols. I saw someone posting that they were working on DataFrame implementation here, but haven't been able to locate any code in github, and was wondering what implementation decisions are being made here. Thanks. Hi Jay. That may have been me. I have implemented something very basic, but you can read and write my proto dataframe to/from CSV and HDF5. The code is up here: https://github.com/Laeeth/d_dataframes You should think of it as a crude prototype that nonetheless has been useful for me, but it's done more in the old school hacker spirit of getting something working first rather than being designed properly. The reason for that is I have a lot on my plate at the moment, and technology is only one of many of these, although an important one. In time I may get someone else to work on dataframes and opensource the results, but that may be some months away. So I'd welcome any assistance, or even taking it over. I haven't really done a good job of having idiomatic access, but it's something and a start. Laeeth. I
Re: good reasons not to use D?
On Saturday, 31 October 2015 at 16:06:47 UTC, Russel Winder wrote: On Sat, 2015-10-31 at 15:41 +, tcak via Digitalmars-d-learn wrote: On Saturday, 31 October 2015 at 14:37:23 UTC, rumbu wrote: > On Friday, 30 October 2015 at 10:35:03 UTC, Laeeth Isharc > wrote: > > I'm writing a talk for codemesh on the use of D in finance. > > > > Any other thoughts? > > For finance stuff - missing a floating point decimal data > type. Things like 1.1 + 2.2 = 3.3003 I always thought that this type of arithmetic operations can be solved with BigInt, but never tried it. Since the issue is related to IEEE standard, a simulated (not supported by hardware directly) data type might be required. In that std.bigint.BigInt provides the accuracy, yes it does suffice. But it is slow. As far as I am aware only IBM Big Iron (aka mainframes, aka z-Series) has hardware decimal floating point these days. (Even though 1970s and 1980s microprocessors had the feature.) It would be nice to have fixed point numbers in Phobos, although it's not much work to implement, and there is a library solution already (which is maintained, but sometimes for a while breaks with newer versions of dmd).
Re: good reasons not to use D?
On Sunday, 1 November 2015 at 09:07:56 UTC, Ilya Yaroshenko wrote: On Friday, 30 October 2015 at 10:35:03 UTC, Laeeth Isharc wrote: Any other thoughts? Floating point operations can be extended automatically (without some kind of 'fastmath' flag) up to 80bit fp on 32 bit intel processors. This is worst solution for language that want to be used in accounting or math. Thoughts like "larger precision entails more accurate answer" are naive and wrong. --Ilya Thanks, Ilya. An important but subtle point. Funnily enough, I haven't done so much specifically heavily numerical work so far, but trust that in the medium term the dlangscience project by John Colvin and others will bear fruit. What would you suggest is a better option to address the concerns you raised?
Re: good reasons not to use D?
On Sunday, 1 November 2015 at 20:38:44 UTC, Freddy wrote: On Friday, 30 October 2015 at 10:35:03 UTC, Laeeth Isharc wrote: I'm writing a talk for codemesh on the use of D in finance. I want to start by addressing the good reasons not to use D. (We all know what the bad ones are). I don't want to get into a discussion here on them, but just wanted to make sure I cover them so I represent the state of affairs correctly. So far what comes to mind: heavy GUI stuff (so far user interface code is not what it could be); cases where you want to write quick one-off scripts that need to use a bunch of different libraries not yet available in D and where it doesn't make sense to wrap or port them; where you have a lot of code in another language (especially non C, non Python) and defining an interface is not so easy; where you have many inexperienced programmers and they need to be productive very quickly. Any other thoughts? I would advise against using D in applications where memory is essential. Idiomatic D uses a garbage collector which has a non free runtime cost. Idiomatic D doesn't need to use the GC, and may generate much less garbage anyway. Andrei's allocator is pretty easy to use, and EMSI containers on top make it even easier. And those aren't the only options.
Re: good reasons not to use D?
On Saturday, 31 October 2015 at 05:25:06 UTC, ref2401 wrote: On Friday, 30 October 2015 at 10:35:03 UTC, Laeeth Isharc wrote: I'm writing a talk for codemesh on the use of D in finance. I want to start by addressing the good reasons not to use D. (We all know what the bad ones are). I don't want to get into a discussion here on them, but just wanted to make sure I cover them so I represent the state of affairs correctly. So far what comes to mind: heavy GUI stuff (so far user interface code is not what it could be); cases where you want to write quick one-off scripts that need to use a bunch of different libraries not yet available in D and where it doesn't make sense to wrap or port them; where you have a lot of code in another language (especially non C, non Python) and defining an interface is not so easy; where you have many inexperienced programmers and they need to be productive very quickly. Any other thoughts? I'd suggest enterprise software because many necessary libraries and tools do not exist in D ecosystem. Though you've already named these reasons. Thank you very much to every one who has replied on this thread, and my apologies for not engaging - just haven't had time. But I have read them all and it will help me clarify what I say. Incidentally, there is enterprise and enterprise. What's necessary really depends on what you're trying to do. For me, so far, I don't see the lack of libraries as a major problem. How much work really is it to port some bindings? Not much in relation to the larger goal. I've done that myself for many of them I wanted so far, because I wanted to get to know the tools I was working with and you can't delegate learning. In coming months others will be helping me, which will allow me to spend a bit more time and energy on the investment aspect and the bigger picture. That's not only been my perspective, but also that of other financial adopters of languages perceived as niche - for example, Jane Street with ocaml. A benefit of the D community that also resonates with what Yaron Minsky at Jane Street says about ocaml is that there are some very good programmers here (and the less experienced ones learn from the more experienced ones, and the reason that's possible is the very thing that turns off more corporate sorts of people - it's a small community so people have more time and energy to spend). You couldn't buy for any price what people I have talked to have learnt by becoming language contributors... Laeeth.
good reasons not to use D?
I'm writing a talk for codemesh on the use of D in finance. I want to start by addressing the good reasons not to use D. (We all know what the bad ones are). I don't want to get into a discussion here on them, but just wanted to make sure I cover them so I represent the state of affairs correctly. So far what comes to mind: heavy GUI stuff (so far user interface code is not what it could be); cases where you want to write quick one-off scripts that need to use a bunch of different libraries not yet available in D and where it doesn't make sense to wrap or port them; where you have a lot of code in another language (especially non C, non Python) and defining an interface is not so easy; where you have many inexperienced programmers and they need to be productive very quickly. Any other thoughts?
Nick coghlan bdfl delegate - 27 languages to improve your Python
http://www.curiousefficiency.org/posts/2015/10/languages-to-improve-your-python.html?utm_content=buffere6909_medium=social_source=twitter.com_campaign=buffer He says nice things about D, although maybe one might say more and slightly different things. It's good to see another who doesn't believe languages are in a death match.
fwiw - study of traits of popular posts on hacker news
Of course it doesn't follow that merely aping the traits of popular posts will produce the same result as actually having the essence of whatever it is that truly makes a post popular. But this was still quite interesting, and it's nice to see people exploring the data. http://www.dkriesel.com/en/blog/2015/1025_a_data-driven_guide_to_creating_successful_hacker_news_posts
OT: The Genius Famine
Since there are some highly creative and intelligent people here, self-selected to be those who enjoy working on problems that are intrinsically interesting, I thought one or two people might enjoy reading some extracts from a forthcoming book on the topic of creative accomplishment and the endogenous personality by Professor Bruce Charlton, Professor of Medicine at Newcastle University. If it's not your bag, then no need to read it! http://geniusfamine.blogspot.co.uk/ This book is about genius: what it is, what it does, where it comes from.[1] And about geniuses: especially why there used to be so many and now there are so few; what was the effect of an era of geniuses, and what will be the consequences of our current Genius Famine. This book describes the genius as an Endogenous personality; that is, a person of high intelligence combined with a personality driven from within, an ‘inner’ –orientated personality: that is, a dominated by the Creative Triad of (1) Innate high ability, (2) Inner motivation and (3) Intuitive thinking. When high intelligence and this type of personality are confluent, a potential genius is the result. But to fulfil this potential the Endogenous personality must find and accept his own Destiny, and must undergo the trials and tribulations of a Quest before he is likely to be rewarded by an Illumination: a breakthrough. Even then, the breakthrough must be noticed, understood, accepted, implemented by society at large; and we describe how past societies were much better at recognizing and making a place for the potential genius. Because the problem is that the Endogenous personality is usually an awkward and asocial character at best; and often an actively unpleasant person and a disruptive influence. Geniuses are altruistic, in the sense that their work is primarily for the good of the group; and not for the usual social rewards such as status, money, sex, and popularity. Therefore many geniuses need to be sustained in a long-term way; and their work demands careful attention and evaluation. We argue that modern societies, by means both indirect and direct, have become hostile to genius and indifferent to the work of those relatively few remaining geniuses. However, because the work of a genius is necessary and irreplaceable, we argue for a change of attitude. Modern society needs geniuses for its own survival in the face of unfamiliar, often unprecedented, threats. Therefore, we must in future do a better job of recognizing, sustaining and accepting guidance from as many geniuses of the highest quality that can be found.
Re: D serialization temporary fixup?
On Thursday, 22 October 2015 at 16:28:30 UTC, Laeeth Isharc wrote: On Thursday, 22 October 2015 at 16:15:23 UTC, Shriramana Sharma wrote: I wanted a D equivalent to: http://doc.qt.io/qt-5/qdatastream.html https://docs.python.org/3/library/pickle.html and saw that one is under construction: http://wiki.dlang.org/Review/std.serialization But till it's finalized, I'd just like to have a quick but reliable way to store real and int data types into a binary data file and read therefrom. Is there such a solution? The size of the data is fixed, but especially since I have real values, I'd like to not write to limited fixed decimal text format. msgpack works quite well for me (see the D binding). it enforces correctness, to a certain degree.
Re: D serialization temporary fixup?
On Thursday, 22 October 2015 at 16:15:23 UTC, Shriramana Sharma wrote: I wanted a D equivalent to: http://doc.qt.io/qt-5/qdatastream.html https://docs.python.org/3/library/pickle.html and saw that one is under construction: http://wiki.dlang.org/Review/std.serialization But till it's finalized, I'd just like to have a quick but reliable way to store real and int data types into a binary data file and read therefrom. Is there such a solution? The size of the data is fixed, but especially since I have real values, I'd like to not write to limited fixed decimal text format. msgpack works quite well for me (see the D binding).
Re: OT: why do people use python when it is slow?
On Wednesday, 14 October 2015 at 18:17:29 UTC, Russel Winder wrote: On Wed, 2015-10-14 at 14:48 +, John Colvin via Digitalmars-d-learn wrote: On Wednesday, 14 October 2015 at 14:32:00 UTC, jmh530 wrote: > On Tuesday, 13 October 2015 at 23:26:14 UTC, Laeeth Isharc > wrote: > > https://www.quora.com/Why-is-Python-so-popular-despite-being-so-s > > low > > Andrei suggested posting more widely. > > I was just writing some R code yesterday after playing > around with D for a couple weeks. I accomplished more in an > afternoon of R coding than I think I had in like a month's > worth of playing around with D. The same is true for python. As someone who uses both D and Python every day, I find that - once you are proficient in both - initial productivity is higher in Python and then D starts to overtake as a project gets larger and/or has stricter requirements. I hope never to have to write anything longer than a thousand lines in Python ever again. The thing about Python is NumPy, SciPy, Pandas, Matplotlib, IPython, Jupyter, GNU Radio. The data science, bioinformatics, quant, signal provessing, etc. people do not give a sh!t which language they used, what they want is to get their results as fast as possible. Most of them do not write programs that are to last, they are effectively throw away programs. This leads them to Python (or R) and they are not really interested in learning anything else. The fact that NumPy sort of sucks in terms of performance, isn't noticed by them as they get their results "fast enough" and a lot faster than sequential Python. The fact that if they used Chapel or even D for their compute intensive code they would rapidly discover that NumPy sort of sucks never really occurs to these people as they are focussed on the results not the means of achieving them. Polyglot Python/D or Python/Chapel with Matplotlib is the way to go. But that really requires a D replacement for Pandas. Russell, thanks for your thoughts - I appreciate it. What would a Pandas replacement look like in D?
Re: LuaD: creating a flexible data filter system
On Saturday, 17 October 2015 at 13:15:17 UTC, yawniek wrote: many thanks for the valuable insights. so far i made a simple prototype with LuaD and classes, works nicely for when my niput what so far is not 100% clear is if there is a way to have a parsed msgpack or json documents being exposed in my lua code in a way so it behaves like a lua object. Ideally in a RW fashion so that changed then again can be processed by D code. http://luajit.org/ext_ffi_tutorial.html C meta methods
Re: LuaD: creating a flexible data filter system
On Friday, 16 October 2015 at 09:01:57 UTC, yawniek wrote: hi, i'm reading in a stream of data that is deserialized into individual frames. a frame is either of: a) a specific D datastructure ( struct with a few ulong,string,string[string] etc members), known at compile time b) json (preferably stdx.data.json) i now want to implement something where i can dynamically add lua filters that then get data out of these frames, create a new lua object send it back to D code where its either sent to another lua filter or at last being serialized again to json und then being processed further. ideally i would not like to copy all the data into a lua object but directly access it from lua. is there an elegant approach to do this and support both a and b cases? so far i did some benchmarks, mainly with string comparisons and it turned out that luaD is about 10x faster than mruby and python D bridges. I haven't done more than play with it so far, but see LuaJIT FFI. Obv LuaJIT is fast too. From memory I think you can access C structs directly. In addition you can write meta methods that provide nice sugar for accessing, constructors, etc.
Re: OT: why do people use python when it is slow?
On Wednesday, 14 October 2015 at 15:25:22 UTC, David DeWitt wrote: On Wednesday, 14 October 2015 at 14:48:22 UTC, John Colvin wrote: On Wednesday, 14 October 2015 at 14:32:00 UTC, jmh530 wrote: On Tuesday, 13 October 2015 at 23:26:14 UTC, Laeeth Isharc wrote: https://www.quora.com/Why-is-Python-so-popular-despite-being-so-slow Andrei suggested posting more widely. I was just writing some R code yesterday after playing around with D for a couple weeks. I accomplished more in an afternoon of R coding than I think I had in like a month's worth of playing around with D. The same is true for python. As someone who uses both D and Python every day, I find that - once you are proficient in both - initial productivity is higher in Python and then D starts to overtake as a project gets larger and/or has stricter requirements. I hope never to have to write anything longer than a thousand lines in Python ever again. That's true until you need to connect to other systems. There are countless clients built for other systems thats are used in real world applications. With web development the Python code really just becomes glue nowadays and api's. I understand D is faster until you have to build the clients for systems to connect. We have an application that uses Postgres, ElasticSearch, Kafka, Redis, etc. This is plenty fast and the productivity of Python is more than D as the clients for Elasticsearch, Postgres and various other systems are unavailable or incomplete. Sure D is faster but when you have other real world systems to connect to and time constraints on projects how can D be more productive or faster? Our python code essentially becomes the API and usage of clients to other systems which handle a majority of the hardcore processing. Once D gets established with those clients and they are battle tested then I will agree. To me productivity is more than the language itself but also building real world applications in a reasonable time-frame. D will get there but is nowhere near where Python is. Few thoughts: 1. It's easy to embed Python in your D applications. I do this for things like web scraping and when I want to write something quick to read simple XML (I just convert to JSON). 2. Of course there is a Redis client. Elasticsearch is an amazing product, but hardly requires much work to have a complete API. I made a start on this, and if I use Elasticsearch more then I'll have one done and will release it. I don't know the finer aspects of Postgres to know what is involved. 3. That raises a broader point, which is that it depends on the ultimate aim of your project and what it is about the right tradeoff between different things. It will ultimately be much more productive for me to do things in D for the reasons John alludes to. A little work to get started is neither here nor there in the major scheme of things. Adam Ruppe made the same point - it's not all that much work to put a foundation that suits you in place. You do it once (and maybe add things when something like Elasticsearch comes out), and that's it, apart from minor updates. The dollar expenditure on building these things is not enormous given the stakes involved for me. But that doesn't mean that you should get to the same answer, as it depends. 4. I am not sure that all web development is just glue, or will be going forward given what might be on the horizon, but time will tell. Laeeth.
Re: OT: why do people use python when it is slow?
On Wednesday, 14 October 2015 at 22:11:56 UTC, data pulverizer wrote: On Tuesday, 13 October 2015 at 23:26:14 UTC, Laeeth Isharc wrote: https://www.quora.com/Why-is-Python-so-popular-despite-being-so-slow Andrei suggested posting more widely. I am coming at D by way of R, C++, Python etc. so I speak as a statistician who is interested in data science applications. Welcome... Looks like we have similar interests. To sit on the deployment side, D needs to grow it's big data/noSQL infrastructure for a start, then hook into a whole ecosystem of analytic tools in an easy and straightforward manner. This will take a lot of work! Indeed. The dlangscience project managed by John Colvin is very interesting. It is not a pure stats project, but there will be many shared areas of need. He has some v interesting ideas, and being able to mix Python and D in a Jupyter notebook is rather nice (you can do this already). I believe it is easier and more effective to start on the research side. D will need: 1. A data table structure like R's data.frame or data.table. This is a dynamic data structure that represents a table that can have lots of operations applied to it. It is the data structure that separates R from most programming languages. It is what pandas tries to emulate. This includes text file and database i/o from mySQL and ODBC for a start. I fully agree, and have made a very simple start on this. See github. It's usable for my needs as they stand, although far from production ready or elegant. You can read and write to/from CSV and HDF5. I guess mysql and ODBC wouldn't be hard to add, but I don't myself need for now and won't have time to do myself. If I have space I may channel some reesources in that direction some time next year. 2. Formula class : the ability to talk about statistical models using formulas e.g. y ~ x1 + x2 + x3 etc and then use these formulas to generate model matrices for input into statistical algorithms. Sounds interesting. Take a look at Colvin's dlang science draft white paper, and see what you would add. It's a chance to shape things whilst they are still fluid. 3. Solid interface to a big data database, that allows a D data table <-> database easily Which ones do you have in mind for stats? The different choices seem to serve quite different needs. And when you say big data, how big do you typically mean ? 4. Functional programming: especially around data table and array structures. R's apply(), lapply(), tapply(), plyr and now data.table(,, by = list()) provides powerful tools for data manipulation. Any thoughts on what the design should look like? To an extent there is a balance between wanting to explore data iteratively (when you don't know where you will end up), and wanting to build a robust process for production. I have been wondering myself about using LuaJIT to strap together D building blocks for the exploration (and calling it based on a custom console built around Adam Ruppe's terminal). 5. A factor data type:for categorical variables. This is easy to implement! This ties into the creation of model matrices. 6. Nullable types makes talking about missing data more straightforward and gives you the opportunity to code them into a set value in your analysis. D is streaks ahead of Python here, but this is built into R at a basic level. So matrices with nullable types within? Is nan enough for you ? If not then could be quite expensive if back end is C. If D can get points 1, 2, 3 many people would be all over D because it is a fantastic programming language and is wicked fast. What do you like best about it ? And in your own domain, what have the biggest payoffs been in practice?
Re: OT: why do people use python when it is slow?
On Thursday, 15 October 2015 at 07:57:51 UTC, Russel Winder wrote: On Thu, 2015-10-15 at 06:48 +, data pulverizer via Digitalmars-d- learn wrote: […] A journey of a thousand miles ... Exactly. I tried to start creating a data table type object by investigating variantArray: http://forum.dlang.org/thread/hhzavwrkbrkjzfohc...@forum.dlang.org but hit the snag that D is a static programming language and may not allow the kind of behaviour you need for creating the same kind of behaviour you need in data table - like objects. I envisage such an object as being composed of arrays of vectors where each vector represents a column in a table as in R - easier for model matrix creation. Some people believe that you should work with arrays of tuple rows - which may be more big data friendly. I am not overly wedded to either approach. Anyway it seems I have hit an inherent limitation in the language. Correct me if I am wrong. The data frame needs to have dynamic behaviour bind rows and columns and return parts of itself as a data table etc and since D is a static language we cannot do this. Just because D doesn't have this now doesn't mean it cannot. C doesn't have such capability but R and Python do even though R and CPython are just C codes. Pandas data structures rely on the NumPy n-dimensional array implementation, it is not beyond the bounds of possibility that that data structure could be realized as a D module. Is R's data.table written in R or in C? In either case, it is not beyond the bounds of possibility that that data structure could be realized as a D module. The core issue is to have a seriously efficient n-dimensional array that is amenable to data parallelism and is extensible. As far as I am aware currently (I will investigate more) the NumPy array is a good native code array, but has some issues with data parallelism and Pandas has to do quite a lot of work to get the extensibility. I wonder how the R data.table works. I have this nagging feeling that like NumPy, data.table seems a lot better than it could be. From small experiments D is (and also Chapel is even more) hugely faster than Python/NumPy at things Python people think NumPy is brilliant for. Expectations of Python programmers are set by the scale of Python performance, so NumPy seems brilliant. Compared to the scale set by D and Chapel, NumPy is very disappointing. I bet the same is true of R (I have never really used R). This is therefore an opportunity for D to step in. However it is a journey of a thousand miles to get something production worthy. Python/NumPy/Pandas have had a very large number of programmer hours expended on them. Doing this poorly as a D modules is likely worse than not doing it at all. I think it's much better to start, which means solving your own problems in a way that is acceptable to you rather than letting perfection be the enemy of the good. It's always easier to do something a second time too, as you learn from successes and mistakes and you have a better idea about what you want. Of course it's better to put some thought into design early on, but that shouldn't end up in analysis paralysis. John Colvin and others are putting quite a lot of thought into dlang science, it seems to me, but he is also getting stuff done. Running D in a Jupyter notebook is something very useful. It doesn't matter that it's cosmetically imperfect at this stage, and it won't stay that way. And that's just a small step towards the bigger goal.
Re: OT: why do people use python when it is slow?
On Wednesday, 14 October 2015 at 05:42:12 UTC, Ola Fosheim Grøstad wrote: On Tuesday, 13 October 2015 at 23:26:14 UTC, Laeeth Isharc wrote: https://www.quora.com/Why-is-Python-so-popular-despite-being-so-slow Andrei suggested posting more widely. That's flaimbait: «Many really popular websites use Python. But why is that? Doesn't it affect the performance of the website?» No. Really popular websites use pre-generated content / front end caches / CDNs or wait for network traffic from distributed databases. For a long time, Ola, I am done discussing with you. But I would ask you to take more responsibility for the effect of you words. The piece you quote is from the question, and not from what I wrote. You refer to it as flame bait, and don't make this clear in what you write.
OT: why do people use python when it is slow?
https://www.quora.com/Why-is-Python-so-popular-despite-being-so-slow Andrei suggested posting more widely.
Re: AWS API Dlang, hmac sha256 function.
On Friday, 9 October 2015 at 04:04:57 UTC, holo wrote:
r.dateString = client.addRequestHeader("Authoryzation:",
"AWS4-HMAC-SHA256" ~ " " ~ "Credential=" ~ accKey ~ "/" ~
xamztime ~ "/" ~ zone ~ "/" ~ service ~ "/" ~ "aws4_request" ~
", " ~ "SignedHeaders=" ~ "content-type;host;x-amz-date" ~ ", "
~ "Signature=" ~ signature);
authorisation ??? (Or maybe authorization) ie a typo (check other
fields carefully too, just in case)
auto url = service ~ ".amazonaws.com?" ~
"Action=DescribeInstances=2013-10-15";
writeln(url);
auto content = get(url, client);
writeln(content);
}
Everything is compiling but im getting 400 (Bad Request):
[root@ultraxps aws]# ./header.d
action=DescribeRegions=2013-10-15
content-type: application/x-www-form-urlencoded; charset=utf-8
host: ec2.amazonaws.com
x-amz-date: 20151009T053800Z
AWS4-HMAC-SHA256
20151009T053800Z
20151009/us-east-1/ec2/aws4_request
888595748692147ceafafcae3941ec0d83ac42c97641e4d954d7447a00c56270
69b1e4c5212cc6b485569fdfb43f7dde94413b36c50393c55d4810ced47f167b
ec2.amazonaws.com?Action=DescribeRegions=2013-10-15
std.net.curl.CurlException@/usr/include/dlang/dmd/std/net/curl.d(824): HTTP
request returned status code 400 (Bad Request)
/tmp/.rdmd-0/rdmd-header.d-54C0A2BD6BD71C27D9AC7319D786A6F3/header(pure @safe
bool std.exception.enforce!(std.net.curl.CurlException, bool).enforce(bool,
lazy const(char)[], immutable(char)[], ulong)+0x65) [0x5004bd]
/tmp/.rdmd-0/rdmd-header.d-54C0A2BD6BD71C27D9AC7319D786A6F3/header(char[]
std.net.curl._basicHTTP!(char)._basicHTTP(const(char)[], const(void)[],
std.net.curl.HTTP)+0x1a2) [0x4fc0ba]
/tmp/.rdmd-0/rdmd-header.d-54C0A2BD6BD71C27D9AC7319D786A6F3/header(char[]
std.net.curl.get!(std.net.curl.HTTP, char).get(const(char)[],
std.net.curl.HTTP)+0x6a) [0x4fbec2]
/tmp/.rdmd-0/rdmd-header.d-54C0A2BD6BD71C27D9AC7319D786A6F3/header(_Dmain+0x840)
[0x4f8f98]
/tmp/.rdmd-0/rdmd-header.d-54C0A2BD6BD71C27D9AC7319D786A6F3/header(_D2rt6dmain211_d_run_mainUiPPaPUAAaZiZ6runAllMFZ9__lambda1MFZv+0x1f)
[0x533eab]
/tmp/.rdmd-0/rdmd-header.d-54C0A2BD6BD71C27D9AC7319D786A6F3/header(void
rt.dmain2._d_run_main(int, char**, extern (C) int
function(char[][])*).tryExec(scope void delegate())+0x2a) [0x533e06]
/tmp/.rdmd-0/rdmd-header.d-54C0A2BD6BD71C27D9AC7319D786A6F3/header(void
rt.dmain2._d_run_main(int, char**, extern (C) int
function(char[][])*).runAll()+0x2b) [0x533e67]
/tmp/.rdmd-0/rdmd-header.d-54C0A2BD6BD71C27D9AC7319D786A6F3/header(void
rt.dmain2._d_run_main(int, char**, extern (C) int
function(char[][])*).tryExec(scope void delegate())+0x2a) [0x533e06]
/tmp/.rdmd-0/rdmd-header.d-54C0A2BD6BD71C27D9AC7319D786A6F3/header(_d_run_main+0x1d2)
[0x533d86]
/tmp/.rdmd-0/rdmd-header.d-54C0A2BD6BD71C27D9AC7319D786A6F3/header(main+0x12)
[0x52cb62]
/usr/lib/libc.so.6(__libc_start_main+0xf0) [0x7f9c39f71610]
[root@ultraxps aws]#
What am i doing wrong? What is strange when when i do same with
curl its responding normally of course is not authenticated).
[root@ultraxps aws]# curl
ec2.amazonaws.com?Action=DescribeInstances=2013-10-15
[1] 20390
[root@ultraxps aws]#
MissingParameterThe request must contain the parameter
AWSAccessKeyIde1352781-c2b4-4e74-ade3-80d655efd0ac
Re: Bug? 0 is less than -10
On Thursday, 8 October 2015 at 13:32:17 UTC, Steven Schveighoffer wrote: On 10/7/15 1:27 AM, Laeeth Isharc wrote: On Wednesday, 7 October 2015 at 02:53:32 UTC, Steven Schveighoffer wrote: On 10/6/15 7:21 PM, Laeeth Isharc wrote: could we have ssize_t defined in phobos somewhere so your code ends up being portable ;) (It's trivial to do, obviously). ptrdiff_t It seems unnatural to use such a name when the variable has nothing to do with pointers - it doesn't contribute to the readability. Yes, it's trivial, but small things cumulatively matter. Adam tends to use int and when that gets mixed up with an auto size_t (eg via length) then his code doesn't compile on 64 bit. And if it happens with his code, you can imagine this isn't a problem that inexperienced users never encounter. ptrdiff_t is in the C spec, ssize_t is not. No reason to name all the types of snow here. A machine-word-sized signed integer is ptrdiff_t. -Steve Whatever - it's not that important to me. D isn't C, and has learnt from C's mistakes and infelicities. I love the beauty of C, but I'm glad I don't need to do my work in it. One reason why D is under appreciated is that these seemingly small differences make a big difference in practice, even if in theory they shouldn't. Initially I was quite offended by no loop aliasing. Until I saw how much time it saved me in avoiding silly mistakes. But really there are better things to worry about at the moment. Laeeth.
Re: Bug? 0 is less than -10
On Wednesday, 7 October 2015 at 07:38:44 UTC, Andrea Fontana wrote: On Wednesday, 7 October 2015 at 05:27:12 UTC, Laeeth Isharc wrote: On Wednesday, 7 October 2015 at 02:53:32 UTC, Steven Schveighoffer wrote: On 10/6/15 7:21 PM, Laeeth Isharc wrote: could we have ssize_t defined in phobos somewhere so your code ends up being portable ;) (It's trivial to do, obviously). ptrdiff_t -Steve It seems unnatural to use such a name when the variable has nothing to do with pointers - it doesn't contribute to the readability. Yes, it's trivial, but small things cumulatively matter. Adam tends to use int and when that gets mixed up with an auto size_t (eg via length) then his code doesn't compile on 64 bit. And if it happens with his code, you can imagine this isn't a problem that inexperienced users never encounter. IMO it seems unnatural to use size_t with a signed/negative value too. Indeed. But you end up with an inferred size_t in a foreach and using length with auto, so it makes sense to have something nicer that you can specify that won't make your code look strange. Doesn't bother me since I define ssize_t, but it might make D easier to read for others. When you see ptrdiff_t with no pointer in sight it's not very clear unless you are familiar with C idioms (and I never remember seeing this back when I learnt C - some modern thing).
Re: Bug? 0 is less than -10
On Wednesday, 7 October 2015 at 02:53:32 UTC, Steven Schveighoffer wrote: On 10/6/15 7:21 PM, Laeeth Isharc wrote: could we have ssize_t defined in phobos somewhere so your code ends up being portable ;) (It's trivial to do, obviously). ptrdiff_t -Steve It seems unnatural to use such a name when the variable has nothing to do with pointers - it doesn't contribute to the readability. Yes, it's trivial, but small things cumulatively matter. Adam tends to use int and when that gets mixed up with an auto size_t (eg via length) then his code doesn't compile on 64 bit. And if it happens with his code, you can imagine this isn't a problem that inexperienced users never encounter.
Re: Bug? 0 is less than -10
On Tuesday, 6 October 2015 at 14:55:23 UTC, Adam D. Ruppe wrote:
On Tuesday, 6 October 2015 at 14:46:56 UTC, tcak wrote:
void main(){
size_t dec = 0;
How is it generating "true" for (dec <= -10) ? Is there a
special casting or something?
size_t is unsigned, so the -10 is cast to unsigned too for the
comparison which yields some huge number.
Comparing signed to unsigned is almost always a mistake... but
one D inherited from C.
This is a reason why I prefer to use int instead of size_t
where I can but that might require casts and truncation too.
could we have ssize_t defined in phobos somewhere so your code
ends up being portable ;) (It's trivial to do, obviously).
implementing Ketmar's concept of a debugging console in D (Lua/PyD + arsd terminal-emulator)
Development environments are very personal, and perhaps it's just
my age and not always wanting to learn what the kids are using
today, but I personally find strategic writeflns more helpful
than an IDE for debugging. Still, there comes a point when that
isn't enough.
Ketmar mentioned that he creates a debugging console that he
telnets in to where he can inspect state and perhaps modify it.
(And he shared the base version, which he put into public domain).
http://forum.dlang.org/post/mailman.210.1412254777.9932.digitalmar...@puremagic.com
"most of my reasonably complex
software has well-hidden interactive console inside, so i can
connect
to it to inspect the internal state and execute some (sometimes
alot ;-) of internal commands. and i must say that integrating
such
console in C projects was tiresome. with D i can do it almost
automatically, skipping annoying "variable registration" and
wrappers
for functions."
"some naming conventions and one mixin -- and all interesting
variables and functions from the given module are automatically
registered in command console. so i can inspect and change
variables and fields, call free functions and class/struct member
functions, even write simple scripts. it's really handy. and i
connect to this console using telnet (if telnet support is
activated).
this also allows some forms of unified "automated testing" even
for GUI apps, all without building special "debug versions", i.e.
on production code."
"it supports UDA annotations for getting help ('cmd ?'),
variables and free functions. and please don't blame me, that was
the first 'serious' code i did in D, learning D as i writting the
module. ;-)"
Ketmar - I just wanted to thank you for sharing this idea. Like
so many creative insights it's obvious in hindsight and I was
kind of getting there anyway via playing with Adam Ruppe's
terminal emulator (and his Inspector app built on top of it) and
John Colvin's work on putting D in the Jupyter notebook. But it
made it much more vivid as to what it should look like.
It may not be your cup of tea, but it seems to me that running
Lua (or Python) inside a console can be a beautifully simple way
to implement this without having to write lots of code for a
custom inspector. Adam's work on terminal is quite nice - runs
on Linux and Windows (maybe OSX) and it has mouse support and you
can display images inline, which can be useful for some purposes.
Command history and easy to add shortcuts.
Since LuaD (or PyD + pretty PyD) automatically wrap D types,
there is no need to do much more to be able to inspect what's
going on (perhaps a pretty printer and some way for the script
bit to interrupt your loop and get data within relevant scope if
you haven't set a 'breakpoint' by calling the debugger from
within your code - but you would need to do that anyway).
I'm still learning Lua, and not yet very familiar with LuaD. At
the moment it segfaults on Lua exceptions - possibly because I
set the panic handler wrong, because of some incompatibility with
LuaJit (which in theory is just a drop-in for Lua), or am doing
something stupid.
But the basic idea seems very handy, and you don't need to do
anything to wrap most D types.
Gist here - just my own proof of concept to convince myself, and
it doesn't do much. (I commented out the mouse code for now, and
the display of results from Lua is just on top line, which you
may not want):
https://gist.github.com/Laeeth/086419b7c67d2d6f29db
If you type return(tests[1]["date"]) at the command line then you
will see the D struct value - no manual wrapping involved. and
easy to call a D pretty printer etc - just add the function to
the lua state (see examples).
You can see how to make better use of the terminal by looking at
Ruppe's JSON inspector (also a very useful project) here:
https://github.com/adamdruppe/inspector/blob/master/inspect.d
LuaD docs and code here:
https://jakobovrum.github.io/LuaD/index.html
http://github.com/JakobOvrum/LuaD
Terminal code here:
https://github.com/adamdruppe/terminal-emulator
But perhaps some others may find this obvious in hindsight but
very useful approach useful for debugging.
Laeeth.
Re: Threading Questions
On Monday, 5 October 2015 at 17:40:24 UTC, bitwise wrote: You may be right. I wrote a simple download manager in D using message passing. It was a little awkward at first, but in general, the spawn/send/receive API seems very intuitive. It feels awkward because the data you're working with is out of reach, but I guess it's safer that way. Any possibility of a blog post on your experience of doing so ? ;) [I should start writing some directly, but for time being, until I have my blog up and running again, I write from time to time on Quora]. A few minutes of writing now and then can have a remarkably big impact as well as clarifying your own thoughts, and the time invested is amply repaid, even viewed from a narrowly self-interested perspective. I had same experience with learning message passing. Feels like learning to eat with chopsticks in the beginning, but soon enough it feels much more civilised when it's the right tool for the job.
std.functional:partial - disambiguating templated functions
How do I persuade partial to tie itself to the appropriate overload? I have: alias bars=partial!(slurpBars!BarType,filename,startDate,endDate); where there are two overloads of slurpBars: SomeBar[] slurpBars(SomeBar)(string filename,string datasetName, typeof(SomeBar.date) startDate, typeof(SomeBar.date) endDate) SomeBar[] slurpBars(SomeBar)(hid_t filehandle,string datasetName, typeof(SomeBar.date) startDate, typeof(SomeBar.date) endDate) And I receive the following error: Error: template kprop.marketdata.retrievebars.slurpBars matches more than one template declaration: Thanks. Laeeth.
Re: std.functional:partial - disambiguating templated functions
On Sunday, 4 October 2015 at 16:37:34 UTC, John Colvin wrote: On Sunday, 4 October 2015 at 15:45:55 UTC, Laeeth Isharc wrote: How do I persuade partial to tie itself to the appropriate overload? --- As far as I can see std.functional.partial only does one argument at a time. bars=partial!(partial!(partial!(slurpBars!BarType, filename), startDate), endDate); or maybe, I'm not sure, but maybe you can do: bars=partial!(slurpBars!BarType, AliasSeq!(filename, startDate, endDate)); If you find you really need to manually mess with overloads, use http://dlang.org/traits.html#getOverloads. You may have to wrap it in AliasSeq in some situations due to grammar/parser constraints. fwiw - still doesn't work (whether I use alias or auto, trying each of your solutions). I'll look at getOverloads. Laeeth.
Re: std.functional:partial - disambiguating templated functions
On Sunday, 4 October 2015 at 16:37:34 UTC, John Colvin wrote: On Sunday, 4 October 2015 at 15:45:55 UTC, Laeeth Isharc wrote: How do I persuade partial to tie itself to the appropriate overload? I have: alias bars=partial!(slurpBars!BarType,filename,startDate,endDate); where there are two overloads of slurpBars: SomeBar[] slurpBars(SomeBar)(string filename,string datasetName, typeof(SomeBar.date) startDate, typeof(SomeBar.date) endDate) SomeBar[] slurpBars(SomeBar)(hid_t filehandle,string datasetName, typeof(SomeBar.date) startDate, typeof(SomeBar.date) endDate) And I receive the following error: Error: template kprop.marketdata.retrievebars.slurpBars matches more than one template declaration: Thanks. Laeeth. As far as I can see std.functional.partial only does one argument at a time. bars=partial!(partial!(partial!(slurpBars!BarType, filename), startDate), endDate); or maybe, I'm not sure, but maybe you can do: bars=partial!(slurpBars!BarType, AliasSeq!(filename, startDate, endDate)); If you find you really need to manually mess with overloads, use http://dlang.org/traits.html#getOverloads. You may have to wrap it in AliasSeq in some situations due to grammar/parser constraints. Thanks, John. I will give that a try. Laeeth.
Re: std.functional:partial - disambiguating templated functions
On Sunday, 4 October 2015 at 18:24:08 UTC, John Colvin wrote:
On Sunday, 4 October 2015 at 18:08:55 UTC, Laeeth Isharc wrote:
On Sunday, 4 October 2015 at 17:17:14 UTC, Laeeth Isharc wrote:
On Sunday, 4 October 2015 at 16:37:34 UTC, John Colvin wrote:
On Sunday, 4 October 2015 at 15:45:55 UTC, Laeeth Isharc
wrote:
How do I persuade partial to tie itself to the appropriate
overload?
---
As far as I can see std.functional.partial only does one
argument at a time.
bars=partial!(partial!(partial!(slurpBars!BarType,
filename), startDate), endDate);
or maybe, I'm not sure, but maybe you can do:
bars=partial!(slurpBars!BarType, AliasSeq!(filename,
startDate, endDate));
If you find you really need to manually mess with overloads,
use http://dlang.org/traits.html#getOverloads. You may have
to wrap it in AliasSeq in some situations due to
grammar/parser constraints.
fwiw - still doesn't work (whether I use alias or auto,
trying each of your solutions). I'll look at getOverloads.
How do I distinguish between two overloads that return the
same type but have different arguments? It looks like
getOverloads only deals with cases where the return type is
different, judging by the docs.
getOverloads should give you all the overloads of a function,
whether they return the same or different types. The example
in the docs just happens to have different types.
In general, return types are not considered when talking about
overloads. For example, two functions that take the same
arguments but have different return types are not overloaded,
they are in conflict.
Thanks for this. The only problem then is how to manipulate what
getOverloads returns. (No need to do this now as it's not worth
it - I just wanted to try using partial if it wasn't too much
work. easier just to make an alternate declaration with the type
in its name).
Is this not a bug in the implementation of partial?
import std.functional;
import std.stdio;
void bish(T)(string arg, string barg)
{
writefln("bishs: %s, %s",arg,barg,to!T);
}
void bish(T)(int argi, string barg)
{
writefln("bishi: %s, %s",argi,barg.to!T);
}
void main(string[] args)
{
alias b=partial!(bish!string,"hello"); // this line does not
compile
alias c=partial!(b,"therex");
b("there");
c();
}
[laeeth@engine marketdata]$ dmd partial.d
partial.d(15): Error: template partial.bish matches more than one
template declaration:
partial.d(4): bish(T)(string arg, string barg)
and
partial.d(8): bish(T)(int argi, string barg)
partial knows which overload to call - I told it!
it works fine without templated arguments. so the following
works fine:
import std.functional;
import std.stdio;
void bish(string arg, string barg)
{
writefln("bishs: %s, %s",arg,barg);
}
void bish(int argi, string barg)
{
writefln("bishi: %s, %s",argi,barg);
}
void main(string[] args)
{
alias b=partial!(bish,"hello");
alias c=partial!(b,"therex");
b("there");
c();
}
Re: std.functional:partial - disambiguating templated functions
On Sunday, 4 October 2015 at 17:17:14 UTC, Laeeth Isharc wrote: On Sunday, 4 October 2015 at 16:37:34 UTC, John Colvin wrote: On Sunday, 4 October 2015 at 15:45:55 UTC, Laeeth Isharc wrote: How do I persuade partial to tie itself to the appropriate overload? --- As far as I can see std.functional.partial only does one argument at a time. bars=partial!(partial!(partial!(slurpBars!BarType, filename), startDate), endDate); or maybe, I'm not sure, but maybe you can do: bars=partial!(slurpBars!BarType, AliasSeq!(filename, startDate, endDate)); If you find you really need to manually mess with overloads, use http://dlang.org/traits.html#getOverloads. You may have to wrap it in AliasSeq in some situations due to grammar/parser constraints. fwiw - still doesn't work (whether I use alias or auto, trying each of your solutions). I'll look at getOverloads. How do I distinguish between two overloads that return the same type but have different arguments? It looks like getOverloads only deals with cases where the return type is different, judging by the docs.
Re: std.functional:partial - disambiguating templated functions
On Sunday, 4 October 2015 at 20:34:53 UTC, John Colvin wrote:
On Sunday, 4 October 2015 at 20:26:51 UTC, John Colvin wrote:
template bish(T)
{
alias tmp = bish0!T;
alias tmp = bish1!T;
alias bish = tmp;
}
https://issues.dlang.org/show_bug.cgi?id=15156
Thanks very much, for both John.
Laeeth.
Re: OT: interesting talk by Jane Street technical guy on why they used Ocaml
On Saturday, 3 October 2015 at 15:58:38 UTC, Mengu wrote: On Saturday, 3 October 2015 at 01:41:55 UTC, Laeeth Isharc wrote: https://www.youtube.com/watch?v=hKcOkWzj0_s a little old but still relevant. talks about importance of brevity and strong types for readability (also avoiding boilerplate). two of the partners there committed to read every line of code (originally because they were terrified). very hard to code review boilerplate carefully because it is just too dull! (can't pay people enough!) [...] there's also andy smith's talk [0] at dconf 2015 on adapting D, titled "hedge fund development case study." [0] https://www.youtube.com/watch?v=0KBhb0iWsWQ Thanks! Funnily enough I rewatched the Jane Street talk because of a suggestion made by John Colvin when I was talking to Andy and him recently. It's a good talk by Andy, and I hope to build on this with him at Codemesh next month. The way languages actually get adopted is different from how people who are sitting in eg the kind of enterprise environment where they are never going to be early adopters imagine. Hence one is much better off focusing efforts on those already receptive (and who are looking for a solution to their pain) than trying to convert those who are happy with what they have or uninterested (possibly rationally so) in exploring new things. Being able to understand the codebase is underrated I think.
Re: OT: interesting talk by Jane Street technical guy on why they used Ocaml
On Sunday, 4 October 2015 at 00:45:16 UTC, Mengu wrote: i watched this talk by yaron last year when i was looking at alternatives for sml. i was taking the programming languages course on coursera by dan grossman. ocaml looked like it tooked off at the beginning of 2000s but then due to many problems it failed to be a mainstream language. interesting, thanks. I played with ocaml a little, but simply didn't have time to do more than that. I was interested in the commercial aspects of his experience, as that happens to resonate with my own experience. imho, D will never take off like go or rust because people who adopted these languages are mostly python and ruby developers. rust is only barely out of beta, and doesn't yet seem to be used in many enterprises, whereas D's status is rather different given the size of firms built on it. I know what you mean about perceptions, and that perhaps may change now you have two of the best C++ programmers working on it fulltime, and not just one ;) D has an incredibly creative and helpful community yet our community is not as enthusiastic as go's and rust's community. why do you think that is ? one reason might be different kind of use cases. one has a different emotional experience and draws different kinds of people for some kinds of projects than others, and that's surely reflected in the tone of the community. also things just have their own spirit, and that is what it is, and enthusiasm can be a positive thing, but isn't without drawbacks either. I'd say I am impressed by the sheer grit people have, and that's something important too. phobos is extremely a great library yet not very welcoming and feels overly complicated. we should reduce the amount of WTFs when reading the phobos source and docs. look at the rate of change as well as the level. the docs could be better, and we could have more blog posts. but picture is much better than when I first looked at D a couple of years back. I remember trying for ages just to get std.net.curl to work and almost giving up in despair. (I noticed recently docs were still wrong, so I fixed them). we should give ourselves credit for the distance travelled, even if there's a long road further to get to where we want. Laeeth.
Re: How will std.allocator change how we program in D?
On Friday, 2 October 2015 at 23:54:18 UTC, Taylor Hillegeist wrote: I do not come from a c++ background. but have looked at what allocators do for c++. I know in D the standard for memory management is garbage collection and if we want to manage it ourselfs we have to do things like @nogc. I was just curious how the std allocator will change how we do things. others will give better answers. I suppose containers (I think one of Andrei's next projects) will make a big difference. you can look at the following, which is used by EMSI in production (and now based on Andrei's allocator), a company whose work was recently profiled by the New York Times. https://github.com/economicmodeling/containers btw you use nogc if you want to be sure you are not allocating by mistake (or track down rogue allocations you hadn't thought about). and it gives users of your code confidence. but if you don't allocate much, you don't absolutely need to use nogc.
OT: interesting talk by Jane Street technical guy on why they used Ocaml
https://www.youtube.com/watch?v=hKcOkWzj0_s a little old but still relevant. talks about importance of brevity and strong types for readability (also avoiding boilerplate). two of the partners there committed to read every line of code (originally because they were terrified). very hard to code review boilerplate carefully because it is just too dull! (can't pay people enough!) correctness is v important if you are doing high volumes. but being able to iterate rapidly is important in other areas too. value of predictable performance in generated code. much easier to hire great programmers in ocaml. way they switched wasn't a big strategic plan. guy just turned up at a windows shop (spreadsheets with VB backends etc) in a temp job between university courses. ended up being permanent thing. started hiring people to help him with analysis. became clearer they needed a better solution - nightmare from copying/paste with spreadsheets. sent email to ocaml list and 15 responses of which 12 great and 3 he hired - great ratio. maybe do an experiment. wrote first version of system in 3 months, worked well and slowly expanded from there. part of attraction of ocaml was ability to hire. later if they need more people they can teach them. they don't hire bad programmers, and it's easy to teach good ones. F# does not perform well. needs to allocate! F# developers dismissive 'you must have bug in your program'. F# has null problem - becomes problem when calling .net libraries. windows not great for high performance (timestamps weird).
Re: Dub package with C code
On Friday, 25 September 2015 at 15:08:00 UTC, bachmeier wrote: On Friday, 25 September 2015 at 15:06:41 UTC, bachmeier wrote: First issue would be getting approval from Walter and Andrei. Second would be finding someone that knows how to do it. Should I create a new thread to open discussion on the topic? yes!
Re: Dub package with C code
On Friday, 25 September 2015 at 12:25:52 UTC, tired_eyes wrote: I meant if there is already a place where I can upload my post to. Something like blog.dlang.org OT: Once again, I'm absolutely sure tha D should have an official blog! Forums can't replace blogs, forums are for discussions, not for content presentation. +1 It also encourages people to write guest pieces, because you know the work is not wasted as it reaches a broader audience. Also, as regards existing D blogs - Planet D was a great start but is a little bit tired. Some blogs are now defunct and haven't been updated for years. It would be great to still have a link to them, but they shouldn't be all mixed up - current and stale.
