Re: A few notes on choosing between Go and D for a quick project
On Friday, 20 March 2015 at 20:34:36 UTC, Walter Bright wrote: On 3/20/2015 8:25 AM, weaselcat wrote: All of the content on rosettacode appears to be licensed under GNU FDL, I believe it would just have to be released under the GNU FDL or a similar copyleft license that fulfills the GNU FDL. http://www.gnu.org/licenses/fdl-1.2.html Sigh, looks like we can't use it. Maybe bearophile can dual-licence them - he probably wrote most of them ;-) -[=mike=-
Re: A few notes on choosing between Go and D for a quick project
Just saw this message. Thanks for trying it out. I will have to look at what Rcpp does on Windows. I've never used D on Windows either so will have to figure that out as well. On Friday, 20 March 2015 at 02:38:02 UTC, jmh530 wrote: On Wednesday, 18 March 2015 at 18:48:59 UTC, bachmeier wrote: On Wednesday, 18 March 2015 at 15:13:24 UTC, jmh530 wrote: You might be able to download the zip here: https://bitbucket.org/bachmeil/dmdinline/downloads and then install from a USB disk using install_local. I ended up trying this on my home computer because it makes life much easier. The Debian file I was trying to get for libgretl-dev was a package rather than the source, I think, and I couldn't find the source or any sort of Windows version. Anyway commands like install_librtod2(gretl=usr/lib/libgretl-1.0.so, libr=usr/lib/R/lib/libR.so) don't make much sense as .so is only for linux. I tried it anyway and got an error about the cd command not being found (wasn't the error I expected, but oh well). I seem to be able to import the dmdinline library just fine. When I run compileD(foo, txt) I get Error in file(filename, r, encoding = encoding) : cannot open the connection In addition: Warning message: In file(filename, r, encoding = encoding) : cannot open file 'C:/Users/John/Documents/.rtod2/config.R': No such file or directory I don't know where the config.R file is located. config.R will be created by install_librtod2, but it is unlikely to work on Windows, because I
Re: A few notes on choosing between Go and D for a quick project
On 2015-03-23 at 00:15, krzaq wrote:
Something like
while (n != EOF) {
n = read(fd, buf, sizeof(buf));
if (n==-1) throw(...);
if (strcmp(buf, PREFIX) == 0) {
return buf;
}
}
return NULL;
Requires no prior knowledge, and have similar effect.
I'm surprised nobody commented on no prior knowledge. How are you supposed to
guess what strcmp(buf, PREFIX) does if you don't know the function? Why is it being
compared to 0? What's this -1 magic number? What does read do? What is EOF? Unless you're
born with C/POSIX programming knowledge, this is prior knowledge. I know C well enough,
but it took me some time to understand what it does.
Indeed. I know of strcmp (because of prior knowledge) but had to have a look at man
2 read just in case to verify if read was used correctly. Of course there is also
this conveniently omitted part in throw(...). Throw what? Use switch to go over errno and
pick the suitable Exception or rather pick one generic Exception and put the output of
strerror as its msg? Why should we be bothered with such low-level tasks?
And honestly, compared to
File(/tmp/a).byChunk(4096).joiner.startsWith(s)
you can *easily* guess that you have a file - do some nonobvious magic on it -
and check if *it* starts with `s` just by reading it as plain English.
Now you've injured yourself with your own weapon. I can guess that File(path)
opens the file for reading (probably because of other language libraries) and
that byChunk(size) reads it one chunk at a time (but frankly only because it
looked similar to byLine which I've known before), but what the hell is joiner?
Does it glue ranges together so that they appear as a single contiguous one?
Oh, wait, I may have actually read about it somewhere already. But if I didn't,
I wouldn't have a clue.
What should start with s? The file, any chunk, the joiner - whatever it meant?
It is much clearer than the loop, but I'm not sure I'd guess what it does,
because of the two middle elements in the UFCS chain. This *nonobvious magic*
may have transformed the contents of the file in a way that makes startsWith(s)
do something different.
Re: A few notes on choosing between Go and D for a quick project
On Monday, 23 March 2015 at 11:31:16 UTC, FG wrote: And honestly, compared to File(/tmp/a).byChunk(4096).joiner.startsWith(s) you can *easily* guess that you have a file - do some nonobvious magic on it - and check if *it* starts with `s` just by reading it as plain English. Now you've injured yourself with your own weapon. I can guess that File(path) opens the file for reading (probably because of other language libraries) That's why I used the word guess ;) and that byChunk(size) reads it one chunk at a time (but frankly only because it looked similar to byLine which I've known before), but what the hell is joiner? Does it glue ranges together so that they appear as a single contiguous one? Oh, wait, I may have actually read about it somewhere already. But if I didn't, I wouldn't have a clue. I'd argue that joiner is intuitive enough, but I agree on byChunk. I am also baffled why this byLine/byChunk madness is necessary at all, it should be something like File(path).startsWith(s) or File(path).data.startswith(s) The same goes for stdin, something as simple as cin intvariable in C++ rises to an almost insurmountable task in D. What should start with s? The file, any chunk, the joiner - whatever it meant? It is much clearer than the loop, but I'm not sure I'd guess what it does, because of the two middle elements in the UFCS chain. This *nonobvious magic* may have transformed the contents of the file in a way that makes startsWith(s) do something different. You're right, I didn't even think of that.
Re: A few notes on choosing between Go and D for a quick project
On 3/23/15 4:59 AM, krzaq wrote: File(path).data.startswith(s) We could define File.byByte, but that's minor. I appreciate the ability to decide on the buffering strategy. -- Andrei
Re: A few notes on choosing between Go and D for a quick project
On 2015-03-23 at 12:59, krzaq wrote: I'd argue that joiner is intuitive enough, but I agree on byChunk. I am also baffled why this byLine/byChunk madness is necessary at all, it should be something like File(path).startsWith(s) or File(path).data.startswith(s) Yeah, that would be useful for example to test magic values at the beginning of files: string[] scripts; foreach (string path; dirEntries(topDir, SpanMode.depth)) if (isFile(path) File(path).startsWith(#!)) scripts ~= path; but that's the simplest case of a bigger problem, because here you just need the first few bytes, and you don't want to read the whole file, nor anything more than a sector. OTOH, there are also file formats like ZIP that put the meta information at the end of the file and scatter the rest of the data all over the place using offset information. You don't need to read everything just to grab the metadata. But, when I had a look at the sources of libzip, I went crazy seeing all the code performing tons of file seeking, reading into buffers and handling them[1]. D's std.zip took a simple approach and doesn't deal with that at all; it reads the whole file into the memory. That makes the algorithm more clearly visible, but at the same time it makes the module completely useless if you want to handle archives that are larger than the available memory, and over-the-top if all you wanted was to extract a single file from the archive or only read the directory structure. So, how do you envision something representing a file, i.e. a mix of BufferedRange and SeekableRange, that would neatly handle buffering and seeking, without you dropping to stdc IO or wanting to shoot yourself when you look at the code? [1] for your amusement: http://hg.nih.at/libzip/file/78b8e3fa72a0/lib/zip_open.c
Re: A few notes on choosing between Go and D for a quick project
On Sunday, 22 March 2015 at 03:43:33 UTC, Walter Bright wrote: I.e. there isn't low level C code that effectively uses SIMD vector registers. You have to use the auto-vectorizer, which tries to reconstruct high level operations out of C low level code, then recompile. I don't think low level hardware registers qualify as high level constructs which is the term you used. Besides, all major C compilers ship with builtin vector types and support for standard hardware vendor SIMD intrinsics. But even if you dismiss that, then even less sophisticated contemporary compiler is capable of using SIMD for carefully manually unrolled expressions. Still, even without explicit simd instructions the superscalar nature of desktop CPUs require you to break dependencies to avoid bubbles in the pipeline. So in order to optimize the filling of an array with the fibonacci sequence a plain high level library generator is insufficient. You also need to utilize the closed formula for fib(x) so that you can generate sequences in parallel, e.g. compute the sequence fib(0),fib(1)… in parallel with fib(N), fib(N+1) etc. Without having the closed formula to obtain fib(N-2) and fib(N-1) a regular optimizer will simply not be able to break the dependencies as effectively as a handwritten low level loop.
Re: A few notes on choosing between Go and D for a quick project
On Saturday, 21 March 2015 at 15:09:32 UTC, Andrei Alexandrescu wrote: On 3/20/15 9:43 PM, Sebastiaan Koppe wrote: On Saturday, 21 March 2015 at 01:31:21 UTC, Andrei Alexandrescu wrote: On 3/20/15 5:56 PM, Walter Bright wrote: On 3/20/2015 5:23 PM, Andrei Alexandrescu wrote: Yah, and uses reference counting for management. -- Andrei Ref counting won't improve splitLines, because it must keep them all. Yah, all solution based on let's keep all lines so we count them at the end are suboptimal. -- Andrei What about `.count(\n)` ? Using count is among the solutions I measured: http://stackoverflow.com/questions/28922323/improving-line-wise-i-o-operations-in-d/29153508#29153508 -- Andrei Fwiw I get similar timings to python and pypy using LDC optimized and the others are not bad. Maybe it is a Windows specific problem (I use arch). (Posted to SO).
Re: A few notes on choosing between Go and D for a quick project
Something like
while (n != EOF) {
n = read(fd, buf, sizeof(buf));
if (n==-1) throw(...);
if (strcmp(buf, PREFIX) == 0) {
return buf;
}
}
return NULL;
Requires no prior knowledge, and have similar effect.
I'm surprised nobody commented on no prior knowledge. How are
you supposed to guess what strcmp(buf, PREFIX) does if you don't
know the function? Why is it being compared to 0? What's this -1
magic number? What does read do? What is EOF? Unless you're born
with C/POSIX programming knowledge, this is prior knowledge. I
know C well enough, but it took me some time to understand what
it does.
And honestly, compared to
File(/tmp/a).byChunk(4096).joiner.startsWith(s)
you can easily guess that you have a file - do some nonobvious
magic on it - and check if it starts with `s` just by reading it
as plain English.
Re: A few notes on choosing between Go and D for a quick project
Well then write that answer. Not true, right now.
Re: A few notes on choosing between Go and D for a quick project
On Saturday, 21 March 2015 at 17:55:09 UTC, Walter Bright wrote: On 3/20/2015 3:50 AM, Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= ola.fosheim.grostad+dl...@gmail.com wrote: High level constructs may be cleaner if done right, and sometimes saves programmer time, but it will never be as fast on the standard CPU architectures we have today. The hardware favours carefully planned iterative, imperative approaches. That was true before SIMD and caching, and it is even more true now. It's less true for SIMD. To take advantage of SIMD, compilers have to reverse engineer low level loops into a higher level construct, then re-compile for SIMD. No. You have to design so that you don't get dependencies on the same vector register.
Re: A few notes on choosing between Go and D for a quick project
On 3/20/15 9:43 PM, Sebastiaan Koppe wrote: On Saturday, 21 March 2015 at 01:31:21 UTC, Andrei Alexandrescu wrote: On 3/20/15 5:56 PM, Walter Bright wrote: On 3/20/2015 5:23 PM, Andrei Alexandrescu wrote: Yah, and uses reference counting for management. -- Andrei Ref counting won't improve splitLines, because it must keep them all. Yah, all solution based on let's keep all lines so we count them at the end are suboptimal. -- Andrei What about `.count(\n)` ? Using count is among the solutions I measured: http://stackoverflow.com/questions/28922323/improving-line-wise-i-o-operations-in-d/29153508#29153508 -- Andrei
Re: A few notes on choosing between Go and D for a quick project
On 3/20/2015 3:50 AM, Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= ola.fosheim.grostad+dl...@gmail.com wrote: High level constructs may be cleaner if done right, and sometimes saves programmer time, but it will never be as fast on the standard CPU architectures we have today. The hardware favours carefully planned iterative, imperative approaches. That was true before SIMD and caching, and it is even more true now. It's less true for SIMD. To take advantage of SIMD, compilers have to reverse engineer low level loops into a higher level construct, then re-compile for SIMD.
Re: A few notes on choosing between Go and D for a quick project
On 2015-03-20 at 16:25, weaselcat wrote: On Friday, 20 March 2015 at 15:12:44 UTC, David Gileadi wrote: Someone who knows about copyright/licensing would probably need to check that it's okay if we plan to use them verbatim. If we can't then it might be worth linking to the above page from somewhere on dlang.org. All of the content on rosettacode appears to be licensed under GNU FDL, I believe it would just have to be released under the GNU FDL or a similar copyleft license that fulfills the GNU FDL. Definitely, having a link to http://rosettacode.org/wiki/Category:D would be useful. As for providing some of the examples in-place, there are two pieces of that legal puzzle: 1. The definitions of the problem to solve Most tasks on Rosetta code are common and generic: concatenate strings, sort a collection of integers using merge/quick/whatever sort, etc., so they aren't anything that had its first appearance on RC. Copy-pasting the exact phrasing of those problems from RC would be a violation, but describing them on your own certainly is not. 2. The solutions to problems Copying those code snippets in general may violate GNU FDL (if it exceeds the scope of citation), but if the complete D answer to a challenge on RC was provided by bearophile, and wasn't a modification of an existing answer, *he himself* can give you a separate licensing deal to re-use his code in whatever way you want, because when posting a solution to RC there is no mention (I assume that there isn't, because it would make no sense, but please verify) that you sign over your work to them, giving RC an *exclusive* license to use it.
Re: A few notes on choosing between Go and D for a quick project
On Friday, 20 March 2015 at 20:34:36 UTC, Walter Bright wrote: On 3/20/2015 8:25 AM, weaselcat wrote: All of the content on rosettacode appears to be licensed under GNU FDL, I believe it would just have to be released under the GNU FDL or a similar copyleft license that fulfills the GNU FDL. http://www.gnu.org/licenses/fdl-1.2.html Sigh, looks like we can't use it. Most of the content was written by bearophile, no? He licensed it to rosettacode, but is this an exclusive license - has he actually given his sole rights to use his work them? If not, you can use his work if he agrees. And I would guess if one asks rosettacode nicely they might give rights to relicense all D examples (ie especially the ones bearophile didn't write) under boost for the dlang web site. And even if not, it's still worth having them in a subdirectory under a different licence.
Re: A few notes on choosing between Go and D for a quick project
On 2015-03-21 at 12:34, Laeeth Isharc wrote: has he actually given his sole rights to use his work them? [...] ROTFL, we must be telepathically linked. :)
Re: A few notes on choosing between Go and D for a quick project
On 2015-03-19 at 09:41, Walter Bright wrote:
On 3/18/2015 4:41 PM, Walter Bright wrote:
#include stdbool.h
#include stdio.h
typedef long T;
bool find(T *array, size_t dim, T t) {
int i;
for (i = 0; i = dim; i++);
{
int v = array[i];
if (v == t)
return true;
}
}
Bugs:
1. i should be size_t
2. = should be
3. extraneous ;
4. v should be type T
5. missing return
Two more things, I believe, but they aren't really internal bugs of find:
a null array or array 2GB on 32-bit that would cause negative indexing:
assert(array != 0);
assert(dim = PTRDIFF_MAX);
Re: A few notes on choosing between Go and D for a quick project
On 3/21/2015 11:02 AM, Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= ola.fosheim.grostad+dl...@gmail.com wrote: On Saturday, 21 March 2015 at 17:55:09 UTC, Walter Bright wrote: On 3/20/2015 3:50 AM, Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= ola.fosheim.grostad+dl...@gmail.com wrote: High level constructs may be cleaner if done right, and sometimes saves programmer time, but it will never be as fast on the standard CPU architectures we have today. The hardware favours carefully planned iterative, imperative approaches. That was true before SIMD and caching, and it is even more true now. It's less true for SIMD. To take advantage of SIMD, compilers have to reverse engineer low level loops into a higher level construct, then re-compile for SIMD. No. You have to design so that you don't get dependencies on the same vector register. I know I shouldn't, but I'll bite. Show me the low level C code that effectively uses SIMD vector registers.
Re: A few notes on choosing between Go and D for a quick project
On Saturday, 21 March 2015 at 19:35:02 UTC, Walter Bright wrote: I know I shouldn't, but I'll bite. Show me the low level C code that effectively uses SIMD vector registers. You are right, you should not bite. C code is superflous, this is a general issue with efficient parallel computations. You want to avoid dependencies within a single register. E.g. Take a recurrence relation and make an efficient simd implementation for it. You might need to try to expand the terms so you have N independent formulas. If it uses floating point you will have to be careful about drift between the N formulas that are computed in parallel.
Re: A few notes on choosing between Go and D for a quick project
On 3/21/2015 2:08 PM, Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= ola.fosheim.grostad+dl...@gmail.com wrote: On Saturday, 21 March 2015 at 19:35:02 UTC, Walter Bright wrote: I know I shouldn't, but I'll bite. Show me the low level C code that effectively uses SIMD vector registers. You are right, you should not bite. C code is superflous, this is a general issue with efficient parallel computations. You want to avoid dependencies within a single register. E.g. Take a recurrence relation and make an efficient simd implementation for it. You might need to try to expand the terms so you have N independent formulas. If it uses floating point you will have to be careful about drift between the N formulas that are computed in parallel. I.e. there isn't low level C code that effectively uses SIMD vector registers. You have to use the auto-vectorizer, which tries to reconstruct high level operations out of C low level code, then recompile.
Re: A few notes on choosing between Go and D for a quick project
On Sunday, 22 March 2015 at 03:43:33 UTC, Walter Bright wrote: On 3/21/2015 2:08 PM, Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= ola.fosheim.grostad+dl...@gmail.com wrote: On Saturday, 21 March 2015 at 19:35:02 UTC, Walter Bright wrote: I know I shouldn't, but I'll bite. Show me the low level C code that effectively uses SIMD vector registers. You are right, you should not bite. C code is superflous, this is a general issue with efficient parallel computations. You want to avoid dependencies within a single register. E.g. Take a recurrence relation and make an efficient simd implementation for it. You might need to try to expand the terms so you have N independent formulas. If it uses floating point you will have to be careful about drift between the N formulas that are computed in parallel. I.e. there isn't low level C code that effectively uses SIMD vector registers. You have to use the auto-vectorizer, which tries to reconstruct high level operations out of C low level code, then recompile. at least it still compiles into efficient code for the PDP11?
Re: A few notes on choosing between Go and D for a quick project
On Wed, Mar 18, 2015 at 1:09 AM, Paulo Pinto via Digitalmars-d digitalmars-d@puremagic.com wrote: On Tuesday, 17 March 2015 at 20:50:51 UTC, Bienlein wrote: Go is only a CSP-like, it isn't CSP. cf Python-CSP and PyCSP, not to mention JCSP and GPars. I'm not really sure whether this can be put exactly that way. On a machine with 4 GB RAM you can spawn about 80.000 goroutines... What about using a JVM with green threads support or Quasar, wouldn't it be more comparable? -- Paulo It seems it is the same issue. From the Quasar user manual ( http://docs.paralleluniverse.co/quasar/): Fibers are not meant to replace threads in all circumstances. A fiber should be used when its body (the code it executes) blocks very often waiting on other fibers (e.g. waiting for messages sent by other fibers on a channel, or waiting for the value of a dataflow-variable). For long-running computations that rarely block, traditional threads are preferable. Fortunately, as we shall see, fibers and threads interoperate very well. -- Ziad
Re: A few notes on choosing between Go and D for a quick project
On 3/19/15 6:30 PM, bearophile wrote: Andrei Alexandrescu: You may want to answer there, not here. I've also posted a response. There is this, with an attach: https://issues.dlang.org/show_bug.cgi?id=11810 I destroyed: https://github.com/D-Programming-Language/phobos/pull/3089 Andrei
Re: A few notes on choosing between Go and D for a quick project
One underused resource seems to be all the examples bearophile has put on Rosetta Code: http://rosettacode.org/wiki/Category:D Indeed, I found myself frequently looking though these examples. Thank you bearophile! If he, Adam, or some other proficient D user were to do a weekly series breaking down each of those 733 examples one at a time- what idioms were used, why certain variations were more efficient- that could go a long way to introduce the language and its idioms to beginners. +1000
Re: A few notes on choosing between Go and D for a quick project
On Friday, 20 March 2015 at 05:17:11 UTC, Walter Bright wrote: On 3/19/2015 9:59 AM, Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= ola.fosheim.grostad+dl...@gmail.com wrote: On Thursday, 19 March 2015 at 00:42:51 UTC, weaselcat wrote: On Wednesday, 18 March 2015 at 12:59:17 UTC, bearophile wrote: High level constructs in D are often slower than low-level code, so in some cases you don't want to use them. I actually found that LDC does an _amazing_ job of optimizing high level constructs and converting low level code to higher level functional code resulted in minor speedups in a lot of cases. (Other performance benefits include the algorithm primitives being extensively optimized in phobos.) If the code/compiler generates suboptimal code in the first place then improvements can be somewhat random. But if you write code with good cache locality, filling the pipeline properly then there is no alternative to going low level. Btw, take a look at this: http://stackoverflow.com/questions/28922323/improving-line-wise-i-o-operations-in-d That's really bad marketing... Sigh. The Python version: --- import sys if __name__ == __main__: if (len(sys.argv) 2): sys.exit() infile = open(sys.argv[1]) linect = 0 for line in infile: linect += 1 print There are %d lines % linect -- does not allocate memory. The splitLines() version: [...] cutted Of course it does allocate memory. Python's for...in makes use of iterators and generators, already there you have some allocations going around. Not only that, there is one string being allocated for each line in the file being read, even if it isn't used. Not counting the whole minor allocations going on. -- Paulo
Re: A few notes on choosing between Go and D for a quick project
On Friday, 20 March 2015 at 00:28:24 UTC, Almighty Bob wrote: Maybe I'm a bit long in the tooth but for something like cvsReader I want an array of records, there's isnt anything else that would ever make any sense. Just use std.array.array on it. You get your array, people who see the benefit of alternative approaches get what they want.
Re: A few notes on choosing between Go and D for a quick project
On Friday, 20 March 2015 at 10:50:44 UTC, Walter Bright wrote: On 3/20/2015 12:42 AM, Paulo Pinto wrote: On Friday, 20 March 2015 at 05:17:11 UTC, Walter Bright wrote: Sigh. The Python version: --- import sys if __name__ == __main__: if (len(sys.argv) 2): sys.exit() infile = open(sys.argv[1]) linect = 0 for line in infile: linect += 1 print There are %d lines % linect -- does not allocate memory. The splitLines() version: [...] cutted Of course it does allocate memory. Python's for...in makes use of iterators and generators, already there you have some allocations going around. Not only that, there is one string being allocated for each line in the file being read, even if it isn't used. Since 'line' is never referred to again after constructed, even a simple optimizer could elide it. It would be easy to test - accumulate the lines in an array, and check the times. Which the default Python implementation doesn't have, hence my comment. Also even if it did have one, it cannot elide it as it cannot guarantee the semantics of the generators/iterators side effects will be kept. -- Paulo
Re: A few notes on choosing between Go and D for a quick project
On Thursday, 19 March 2015 at 17:40:46 UTC, weaselcat wrote: On Thursday, 19 March 2015 at 16:59:36 UTC, Ola Fosheim Grøstad If the code/compiler generates suboptimal code in the first place then improvements can be somewhat random. But if you write code with good cache locality, filling the pipeline properly then there is no alternative to going low level. Btw, take a look at this: http://stackoverflow.com/questions/28922323/improving-line-wise-i-o-operations-in-d That's really bad marketing... […] ldc -O5 -inline -release -boundscheck=off wc.d time ./wc enwiki-latest-pages-articles1.xml-p00010p1 There are 1245473 lines. ./wc enwiki-latest-pages-articles1.xml-p00010p1 0.04s user 0.08s system 98% cpu 0.125 total ahem I actually found that LDC does an _amazing_ job of optimizing high level constructs and converting low level code to I think you missed my point here. A simple loop like this should result in similar execution times for all AoT languages with decent library support and optimisation because it ought to be bandwidth limited, not CPU limited. The issue here is that when decent programmers end up asking for help on stackoverflow it exposes underlying library/compiler/language issues that needs addressing on way or another. When it comes to more complicated algorithms then it usually impossible for a high level construct to match a low level one for several reasons, a major one being that you need to tailor the algorithm to OS/caches/ALU/CPU. Pure high level code cannot easily be reduced to high quality iterative algorithms that match the hardware. High level constructs may be cleaner if done right, and sometimes saves programmer time, but it will never be as fast on the standard CPU architectures we have today. The hardware favours carefully planned iterative, imperative approaches. That was true before SIMD and caching, and it is even more true now. Try to do some signal processing and you'll quickly learn that high level code is out of the question if you want high performance.
Re: A few notes on choosing between Go and D for a quick project
On 3/20/2015 12:42 AM, Paulo Pinto wrote: On Friday, 20 March 2015 at 05:17:11 UTC, Walter Bright wrote: Sigh. The Python version: --- import sys if __name__ == __main__: if (len(sys.argv) 2): sys.exit() infile = open(sys.argv[1]) linect = 0 for line in infile: linect += 1 print There are %d lines % linect -- does not allocate memory. The splitLines() version: [...] cutted Of course it does allocate memory. Python's for...in makes use of iterators and generators, already there you have some allocations going around. Not only that, there is one string being allocated for each line in the file being read, even if it isn't used. Since 'line' is never referred to again after constructed, even a simple optimizer could elide it. It would be easy to test - accumulate the lines in an array, and check the times.
Re: A few notes on choosing between Go and D for a quick project
On 3/20/15 1:31 PM, Walter Bright wrote: On 3/20/2015 3:59 AM, Paulo Pinto wrote: On Friday, 20 March 2015 at 10:50:44 UTC, Walter Bright wrote: Since 'line' is never referred to again after constructed, even a simple optimizer could elide it. It would be easy to test - accumulate the lines in an array, and check the times. Which the default Python implementation doesn't have, hence my comment. After all these years, the default Python implementation doesn't do fairly basic optimizations? I find that a bit hard to believe. It's in all likelihood the truth. -- Andrei
Re: A few notes on choosing between Go and D for a quick project
On 3/20/15 10:26 AM, Tobias Pankrath wrote: You may want to answer there, not here. I've also posted a response. Andrei Nitpick: Your solutions that use readText validate their input and the python version probably doesn't. You could mention that (I cannot comment on SO). Yah, nitpicks should go there too. We need to have an understanding that statistically everybody is on SO and nobody here :o). Interestingly readText is faster than byChunck.joiner regardless. I'm not if it does the checking. Nitpick 2: http://www.unicode.org/versions/Unicode7.0.0/ch05.pdf (chapter 5.8) splitLines is still incomplete, missing to break on U+0085, U+000B, U+000C. Would a PR for this be accepted? Prolly. Walter? I'd say the coolest answer to this question would have been: D has not only one of the fastest, but the only correct solution to this that works with UTF8, UTF16 and UTF32 at the same time. Well then write that answer. Andrei
Re: A few notes on choosing between Go and D for a quick project
On 3/20/2015 5:23 PM, Andrei Alexandrescu wrote: Yah, and uses reference counting for management. -- Andrei Ref counting won't improve splitLines, because it must keep them all.
Re: A few notes on choosing between Go and D for a quick project
On 3/20/2015 1:40 PM, weaselcat wrote: On Friday, 20 March 2015 at 20:34:36 UTC, Walter Bright wrote: On 3/20/2015 8:25 AM, weaselcat wrote: All of the content on rosettacode appears to be licensed under GNU FDL, I believe it would just have to be released under the GNU FDL or a similar copyleft license that fulfills the GNU FDL. http://www.gnu.org/licenses/fdl-1.2.html Sigh, looks like we can't use it. all content on the dlang wiki is under the same license already? Phobos is under Boost.
Re: A few notes on choosing between Go and D for a quick project
On 3/20/2015 2:04 PM, Paulo Pinto wrote: The side effect is not keeping the string, rather generating it. for var in exp: do_something() if exp represents a iteratable or a generator, even if var is thrown away the loop needs to be preserved to keep the semantics of calling next() on the instance object that executes the for..in loop. Put other way, does DMD throw away foreach loops even if the compiler cannot prove if opApply () or popFront() generate side effects, assuming the variable isn't being used? D has pure functions, and if those pure functions return an unneeded string, the call can be discarded. BTW, it still is faster than splitLines simply because the line string is not stored. The GC can collect it and reuse the memory. Not so for splitLines, which must hold it all, meaning it'll be using memory that is not in the cache.
Re: A few notes on choosing between Go and D for a quick project
On 3/20/15 5:56 PM, Walter Bright wrote: On 3/20/2015 5:23 PM, Andrei Alexandrescu wrote: Yah, and uses reference counting for management. -- Andrei Ref counting won't improve splitLines, because it must keep them all. Yah, all solution based on let's keep all lines so we count them at the end are suboptimal. -- Andrei
Re: A few notes on choosing between Go and D for a quick project
On 3/20/15 12:42 AM, Paulo Pinto wrote: On Friday, 20 March 2015 at 05:17:11 UTC, Walter Bright wrote: On 3/19/2015 9:59 AM, Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?= ola.fosheim.grostad+dl...@gmail.com wrote: On Thursday, 19 March 2015 at 00:42:51 UTC, weaselcat wrote: On Wednesday, 18 March 2015 at 12:59:17 UTC, bearophile wrote: High level constructs in D are often slower than low-level code, so in some cases you don't want to use them. I actually found that LDC does an _amazing_ job of optimizing high level constructs and converting low level code to higher level functional code resulted in minor speedups in a lot of cases. (Other performance benefits include the algorithm primitives being extensively optimized in phobos.) If the code/compiler generates suboptimal code in the first place then improvements can be somewhat random. But if you write code with good cache locality, filling the pipeline properly then there is no alternative to going low level. Btw, take a look at this: http://stackoverflow.com/questions/28922323/improving-line-wise-i-o-operations-in-d That's really bad marketing... Sigh. The Python version: --- import sys if __name__ == __main__: if (len(sys.argv) 2): sys.exit() infile = open(sys.argv[1]) linect = 0 for line in infile: linect += 1 print There are %d lines % linect -- does not allocate memory. The splitLines() version: [...] cutted Of course it does allocate memory. Yah, and uses reference counting for management. -- Andrei
Re: A few notes on choosing between Go and D for a quick project
On 3/14/2015 2:18 AM, Russel Winder via Digitalmars-d wrote: coding of error Sounds like the title of my next book!
Re: A few notes on choosing between Go and D for a quick project
On 3/20/15 3:50 AM, Walter Bright wrote: On 3/20/2015 12:42 AM, Paulo Pinto wrote: On Friday, 20 March 2015 at 05:17:11 UTC, Walter Bright wrote: Sigh. The Python version: --- import sys if __name__ == __main__: if (len(sys.argv) 2): sys.exit() infile = open(sys.argv[1]) linect = 0 for line in infile: linect += 1 print There are %d lines % linect -- does not allocate memory. The splitLines() version: [...] cutted Of course it does allocate memory. Python's for...in makes use of iterators and generators, already there you have some allocations going around. Not only that, there is one string being allocated for each line in the file being read, even if it isn't used. Since 'line' is never referred to again after constructed, even a simple optimizer could elide it. It's not elided to the best of my knowledge. But it's reference counted so it goes from 0 to 1 and back. A simple caching allocator will have no trouble with this pattern. -- Andrei
Re: A few notes on choosing between Go and D for a quick project
On 3/19/15 4:29 PM, weaselcat wrote: voldemort types sort of feel like a hack to work around the lack of real compile-time interfaces(concepts,) No, they're largely unrelated. -- Andrei
Re: A few notes on choosing between Go and D for a quick project
On 3/20/2015 5:36 PM, Andrei Alexandrescu wrote: Nitpick 2: http://www.unicode.org/versions/Unicode7.0.0/ch05.pdf (chapter 5.8) splitLines is still incomplete, missing to break on U+0085, U+000B, U+000C. Would a PR for this be accepted? Prolly. Walter? Sure.
Re: A few notes on choosing between Go and D for a quick project
On 3/19/15 10:22 PM, Walter Bright wrote: On 3/19/2015 10:44 AM, Joakim wrote: One underused resource seems to be all the examples bearophile has put on Rosetta Code: http://rosettacode.org/wiki/Category:D If he, Adam, or some other proficient D user were to do a weekly series breaking down each of those 733 examples one at a time- what idioms were used, why certain variations were more efficient- that could go a long way to introduce the language and its idioms to beginners. It would provide enough examples for 14 years of such a weekly series, by which time D3 will be getting started! I didn't know about rosettacode. Thanks! It also might be a great resource for better examples to use in the Phobos documentation. Someone who knows about copyright/licensing would probably need to check that it's okay if we plan to use them verbatim. If we can't then it might be worth linking to the above page from somewhere on dlang.org.
Re: A few notes on choosing between Go and D for a quick project
On Friday, 20 March 2015 at 15:12:44 UTC, David Gileadi wrote: On 3/19/15 10:22 PM, Walter Bright wrote: On 3/19/2015 10:44 AM, Joakim wrote: One underused resource seems to be all the examples bearophile has put on Rosetta Code: http://rosettacode.org/wiki/Category:D If he, Adam, or some other proficient D user were to do a weekly series breaking down each of those 733 examples one at a time- what idioms were used, why certain variations were more efficient- that could go a long way to introduce the language and its idioms to beginners. It would provide enough examples for 14 years of such a weekly series, by which time D3 will be getting started! I didn't know about rosettacode. Thanks! It also might be a great resource for better examples to use in the Phobos documentation. Someone who knows about copyright/licensing would probably need to check that it's okay if we plan to use them verbatim. If we can't then it might be worth linking to the above page from somewhere on dlang.org. All of the content on rosettacode appears to be licensed under GNU FDL, I believe it would just have to be released under the GNU FDL or a similar copyleft license that fulfills the GNU FDL.
Re: A few notes on choosing between Go and D for a quick project
On Saturday, 21 March 2015 at 01:31:21 UTC, Andrei Alexandrescu wrote: On 3/20/15 5:56 PM, Walter Bright wrote: On 3/20/2015 5:23 PM, Andrei Alexandrescu wrote: Yah, and uses reference counting for management. -- Andrei Ref counting won't improve splitLines, because it must keep them all. Yah, all solution based on let's keep all lines so we count them at the end are suboptimal. -- Andrei What about `.count(\n)` ?
Re: A few notes on choosing between Go and D for a quick project
On Friday, 20 March 2015 at 20:31:49 UTC, Walter Bright wrote: On 3/20/2015 3:59 AM, Paulo Pinto wrote: On Friday, 20 March 2015 at 10:50:44 UTC, Walter Bright wrote: Since 'line' is never referred to again after constructed, even a simple optimizer could elide it. It would be easy to test - accumulate the lines in an array, and check the times. Which the default Python implementation doesn't have, hence my comment. After all these years, the default Python implementation doesn't do fairly basic optimizations? I find that a bit hard to believe. Also even if it did have one, it cannot elide it as it cannot guarantee the semantics of the generators/iterators side effects will be kept. I wonder why keeping a string would be a side effect. I'm not saying you're wrong, I don't know Python well enough to make such a judgement. It just causes me to raise one eyebrow like Spock if it does work this way. The side effect is not keeping the string, rather generating it. for var in exp: do_something() if exp represents a iteratable or a generator, even if var is thrown away the loop needs to be preserved to keep the semantics of calling next() on the instance object that executes the for..in loop. Put other way, does DMD throw away foreach loops even if the compiler cannot prove if opApply () or popFront() generate side effects, assuming the variable isn't being used? -- Paulo
Re: A few notes on choosing between Go and D for a quick project
You may want to answer there, not here. I've also posted a response. Andrei Nitpick: Your solutions that use readText validate their input and the python version probably doesn't. You could mention that (I cannot comment on SO). Interestingly readText is faster than byChunck.joiner regardless. Nitpick 2: http://www.unicode.org/versions/Unicode7.0.0/ch05.pdf (chapter 5.8) splitLines is still incomplete, missing to break on U+0085, U+000B, U+000C. Would a PR for this be accepted? I'd say the coolest answer to this question would have been: D has not only one of the fastest, but the only correct solution to this that works with UTF8, UTF16 and UTF32 at the same time.
Re: A few notes on choosing between Go and D for a quick project
On 3/20/2015 3:59 AM, Paulo Pinto wrote: On Friday, 20 March 2015 at 10:50:44 UTC, Walter Bright wrote: Since 'line' is never referred to again after constructed, even a simple optimizer could elide it. It would be easy to test - accumulate the lines in an array, and check the times. Which the default Python implementation doesn't have, hence my comment. After all these years, the default Python implementation doesn't do fairly basic optimizations? I find that a bit hard to believe. Also even if it did have one, it cannot elide it as it cannot guarantee the semantics of the generators/iterators side effects will be kept. I wonder why keeping a string would be a side effect. I'm not saying you're wrong, I don't know Python well enough to make such a judgement. It just causes me to raise one eyebrow like Spock if it does work this way.
Re: A few notes on choosing between Go and D for a quick project
On 3/20/2015 8:25 AM, weaselcat wrote: All of the content on rosettacode appears to be licensed under GNU FDL, I believe it would just have to be released under the GNU FDL or a similar copyleft license that fulfills the GNU FDL. http://www.gnu.org/licenses/fdl-1.2.html Sigh, looks like we can't use it.
Re: A few notes on choosing between Go and D for a quick project
On Friday, 20 March 2015 at 20:34:36 UTC, Walter Bright wrote: On 3/20/2015 8:25 AM, weaselcat wrote: All of the content on rosettacode appears to be licensed under GNU FDL, I believe it would just have to be released under the GNU FDL or a similar copyleft license that fulfills the GNU FDL. http://www.gnu.org/licenses/fdl-1.2.html Sigh, looks like we can't use it. all content on the dlang wiki is under the same license already?
Re: A few notes on choosing between Go and D for a quick project
On Thursday, 19 March 2015 at 00:52:12 UTC, Walter Bright wrote: On 3/18/2015 5:34 PM, cym13 wrote: Maybe there should be a part 2 to the C-to-D little tutorial, one that shows how to code at a higher level introducing gently functional structures instead of just teaching how to write C in D. To stop thinking about steps to think about transformations isn't an easy thing. Andrei and I talked about this, how we are failing to point out how to idiomatically use D to advantage. We need to do a lot better at this. related: https://p0nce.github.io/d-idioms/ http://wiki.dlang.org/Tutorials
Re: A few notes on choosing between Go and D for a quick project
On Friday, 20 March 2015 at 00:30:25 UTC, deadalnix wrote: On Thursday, 19 March 2015 at 20:43:55 UTC, Almighty Bob wrote: Its the emperor's new clothes. Type inference is useful. Deal with it. I like type inference. I dont like voldomort types, the cost/benefit is fail.
Re: A few notes on choosing between Go and D for a quick project
On Thursday, 19 March 2015 at 23:02:07 UTC, w0rp wrote: On Thursday, 19 March 2015 at 20:43:55 UTC, Almighty Bob wrote: On Thursday, 19 March 2015 at 10:07:06 UTC, John Colvin wrote: On Tuesday, 17 March 2015 at 18:29:20 UTC, Almighty Bob wrote: On Tuesday, 17 March 2015 at 11:48:15 UTC, Nick Treleaven wrote: On 17/03/2015 10:31, Almighty Bob wrote: It's far more useful for csvReader to return a type I know and can use than it is to obscure the return type for the sake of some philosophical ideal of increasing encapsulation. Part of the art of API design is to hide implementation where it's not necessarily needed. Designers might err on the side of hiding things, because how you expose something is important as it has to be maintained indefinitely. If they expose everything then the internals can never be redesigned for better performance, etc. They don't increase encapsulation. The public members of a voldomort type are still public, you still have to code to the API of the return type whether it's a regular or voldomort type. You can keep as much private or public in either case as you like. All they do take the typename out of circulation, they make life harder for the user. There's no benefit. None. But at least the library author can stroke his chin a feel smug that there's one less type in the modules' namespace. Totally missing the point. The crux of the matter is this: changing a voldemort type (assuming the public semantics are the same) is not a breaking API change, because no-one else's code ever names it. Seriously? You cant have a public API and private implementation with a regular type? That's something specific to voldomort types? Ask yourself what exactly do voldomort types enable you to hide that cant be hidden with a regular type? Just one thing. Their name. As you said no one else can ever name the type. That is no benefit to me the user. No-one has been able to even describe a benefit. Walters article on Dr Dobbs doesn't describe a benefit. It increases encapsulation you all squawk. No it doesn't. The private bits are still private, the public bits are still public. All it does is complicate the user side. Its the emperor's new clothes. It's not possible to construct a voldemort type outside of the function where it was declared, and you can't know what that type is, just what the API is. You can force type deduction so the type returned can be changed without breaking code, so long as the API is the same. That's how it increases encapsulation. voldemort types sort of feel like a hack to work around the lack of real compile-time interfaces(concepts,) the same thing IMO currently plaguing template constraints and honestly causes a lot of compiler errors to be confusing. I believe a few major patches in phobos were related to removing constraints and rewriting them as static asserts for better error handling. see: http://forum.dlang.org/thread/jronroxajqkrqitxw...@forum.dlang.org
Re: A few notes on choosing between Go and D for a quick project
On Thursday, 19 March 2015 at 23:02:07 UTC, w0rp wrote: No it doesn't. The private bits are still private, the public bits are still public. All it does is complicate the user side. Its the emperor's new clothes. It's not possible to construct a voldemort type outside of the function where it was declared, and you can't know what that type is, just what the API is. You can force type deduction so the type returned can be changed without breaking code, so long as the API is the same. That's how it increases encapsulation. Ok I can see the benefit of having a return type that can only be constructed by the function returning it. ***In some cases*** But being able to change the return type doesnt actualy increase encapsulation. you still have to return something with the same public API. Only now you have ***less*** guarantees over what your getting than you did before. Could be an array could be a linked list. Who knows? Maybe I'm a bit long in the tooth but for something like cvsReader I want an array of records, there's isnt anything else that would ever make any sense. So the idea that I now how to jump through hoops to keep hold of whatever that function returns, just in case some divvy might want change what kind of structure it returns one day, is frankly absurd.
Re: A few notes on choosing between Go and D for a quick project
On Thursday, 19 March 2015 at 20:43:55 UTC, Almighty Bob wrote: Its the emperor's new clothes. Type inference is useful. Deal with it.
Re: A few notes on choosing between Go and D for a quick project
On Friday, 13 March 2015 at 15:03:37 UTC, bearophile wrote: Andrei Alexandrescu: That's a rather random collection - strict seems to be D without the stuff bearophile dislikes. -- Andrei I am OK with that definition. Is that your best critique to those suggestions? :-) Bye, bearophile :) I'd use that.
Re: A few notes on choosing between Go and D for a quick project
On 3/18/2015 4:41 PM, Walter Bright wrote:
#include stdbool.h
#include stdio.h
typedef long T;
bool find(T *array, size_t dim, T t) {
int i;
for (i = 0; i = dim; i++);
{
int v = array[i];
if (v == t)
return true;
}
}
Bugs:
1. i should be size_t
2. = should be
3. extraneous ;
4. v should be type T
5. missing return
Re: A few notes on choosing between Go and D for a quick project
On Tuesday, 17 March 2015 at 18:29:20 UTC, Almighty Bob wrote: On Tuesday, 17 March 2015 at 11:48:15 UTC, Nick Treleaven wrote: On 17/03/2015 10:31, Almighty Bob wrote: It's far more useful for csvReader to return a type I know and can use than it is to obscure the return type for the sake of some philosophical ideal of increasing encapsulation. Part of the art of API design is to hide implementation where it's not necessarily needed. Designers might err on the side of hiding things, because how you expose something is important as it has to be maintained indefinitely. If they expose everything then the internals can never be redesigned for better performance, etc. They don't increase encapsulation. The public members of a voldomort type are still public, you still have to code to the API of the return type whether it's a regular or voldomort type. You can keep as much private or public in either case as you like. All they do take the typename out of circulation, they make life harder for the user. There's no benefit. None. But at least the library author can stroke his chin a feel smug that there's one less type in the modules' namespace. Totally missing the point. The crux of the matter is this: changing a voldemort type (assuming the public semantics are the same) is not a breaking API change, because no-one else's code ever names it. Admittedly this is sort-of true for any function that returns auto and doesn't document its return type, but that is by convention (the type is always readable in the relevant .d or .di file) whereas using a voldemort type enforces it. Note that if you really want a name to use in your code: alias IotaFloat = typeof(iota(0f, 2f, 1f));
Re: A few notes on choosing between Go and D for a quick project
On Wednesday, 18 March 2015 at 01:52:00 UTC, Laeeth Isharc wrote: On Tuesday, 17 March 2015 at 21:00:11 UTC, bachmeier wrote: On Tuesday, 17 March 2015 at 19:00:06 UTC, jmh530 wrote: In addition, further development of the ability to call D from R or Python* or Julia (or vice-versa) would also be a positive. What do you have in mind? I no longer work much with Python so my knowledge is limited, but calling D from R or Julia should be no different from calling C from those languages, as you normally compile your C code into a shared library anyway. I've done the R-D thing many times and in the process have worked with a big chunk of the R API. Things like allocating R data structures from a D function, adding assertions to your D code to allow for an easy exit when things don't work out, and calling functions in the R math library, among other things, are not difficult. (I wonder how well cython works with interfacing with D via the C++ interface, because that way you could extend python with D classes and have them be faster than going through PyD). This is a good idea. dtoh could speed up this workflow, see https://github.com/adamdruppe/dtoh/blob/master/dtoh.d and http://forum.dlang.org/post/uyuwdptkpukshxzyg...@forum.dlang.org
Re: A few notes on choosing between Go and D for a quick project
On Wednesday, 18 March 2015 at 18:48:59 UTC, bachmeier wrote: On Wednesday, 18 March 2015 at 15:13:24 UTC, jmh530 wrote: You might be able to download the zip here: https://bitbucket.org/bachmeil/dmdinline/downloads and then install from a USB disk using install_local. I ended up trying this on my home computer because it makes life much easier. The Debian file I was trying to get for libgretl-dev was a package rather than the source, I think, and I couldn't find the source or any sort of Windows version. Anyway commands like install_librtod2(gretl=usr/lib/libgretl-1.0.so, libr=usr/lib/R/lib/libR.so) don't make much sense as .so is only for linux. I tried it anyway and got an error about the cd command not being found (wasn't the error I expected, but oh well). I seem to be able to import the dmdinline library just fine. When I run compileD(foo, txt) I get Error in file(filename, r, encoding = encoding) : cannot open the connection In addition: Warning message: In file(filename, r, encoding = encoding) : cannot open file 'C:/Users/John/Documents/.rtod2/config.R': No such file or directory I don't know where the config.R file is located.
Re: A few notes on choosing between Go and D for a quick project
On 3/19/2015 9:59 AM, Ola Fosheim =?UTF-8?B?R3LDuHN0YWQi?=
ola.fosheim.grostad+dl...@gmail.com wrote:
On Thursday, 19 March 2015 at 00:42:51 UTC, weaselcat wrote:
On Wednesday, 18 March 2015 at 12:59:17 UTC, bearophile wrote:
High level constructs in D are often slower than low-level code, so in some
cases you don't want to use them.
I actually found that LDC does an _amazing_ job of optimizing high level
constructs and converting low level code to higher level functional code
resulted in minor speedups in a lot of cases.
(Other performance benefits include the algorithm primitives being extensively
optimized in phobos.)
If the code/compiler generates suboptimal code in the first place then
improvements can be somewhat random. But if you write code with good cache
locality, filling the pipeline properly then there is no alternative to going
low level.
Btw, take a look at this:
http://stackoverflow.com/questions/28922323/improving-line-wise-i-o-operations-in-d
That's really bad marketing...
Sigh. The Python version:
---
import sys
if __name__ == __main__:
if (len(sys.argv) 2):
sys.exit()
infile = open(sys.argv[1])
linect = 0
for line in infile:
linect += 1
print There are %d lines % linect
--
does not allocate memory. The splitLines() version:
--
import std.stdio;
import std.string;
import std.file;
int main(string[] args)
{
if (args.length 2) {
return 1;
}
auto c = cast(string) read(args[1]);
auto l = splitLines(c);
writeln(There are , l.length, lines.);
return 0;
}
-
allocates memory for all the lines and an array of them. No wonder it's slow!
The allocations are slow, and filling them all when they are cold-cached -
AWFUL! (It also uselessly and maddeningly auto-decodes, a misfeature of Phobos
if there ever was one.)
http://dlang.org/phobos/std_string.html#.splitLines
Re: A few notes on choosing between Go and D for a quick project
On 3/19/2015 10:44 AM, Joakim wrote: One underused resource seems to be all the examples bearophile has put on Rosetta Code: http://rosettacode.org/wiki/Category:D If he, Adam, or some other proficient D user were to do a weekly series breaking down each of those 733 examples one at a time- what idioms were used, why certain variations were more efficient- that could go a long way to introduce the language and its idioms to beginners. It would provide enough examples for 14 years of such a weekly series, by which time D3 will be getting started! I didn't know about rosettacode. Thanks! It also might be a great resource for better examples to use in the Phobos documentation.
Re: A few notes on choosing between Go and D for a quick project
Andrei Alexandrescu: You may want to answer there, not here. I've also posted a response. There is this, with an attach: https://issues.dlang.org/show_bug.cgi?id=11810 Bye, bearophile
Re: A few notes on choosing between Go and D for a quick project
On Thursday, 19 March 2015 at 20:43:55 UTC, Almighty Bob wrote: On Thursday, 19 March 2015 at 10:07:06 UTC, John Colvin wrote: On Tuesday, 17 March 2015 at 18:29:20 UTC, Almighty Bob wrote: On Tuesday, 17 March 2015 at 11:48:15 UTC, Nick Treleaven wrote: On 17/03/2015 10:31, Almighty Bob wrote: It's far more useful for csvReader to return a type I know and can use than it is to obscure the return type for the sake of some philosophical ideal of increasing encapsulation. Part of the art of API design is to hide implementation where it's not necessarily needed. Designers might err on the side of hiding things, because how you expose something is important as it has to be maintained indefinitely. If they expose everything then the internals can never be redesigned for better performance, etc. They don't increase encapsulation. The public members of a voldomort type are still public, you still have to code to the API of the return type whether it's a regular or voldomort type. You can keep as much private or public in either case as you like. All they do take the typename out of circulation, they make life harder for the user. There's no benefit. None. But at least the library author can stroke his chin a feel smug that there's one less type in the modules' namespace. Totally missing the point. The crux of the matter is this: changing a voldemort type (assuming the public semantics are the same) is not a breaking API change, because no-one else's code ever names it. Seriously? You cant have a public API and private implementation with a regular type? That's something specific to voldomort types? Ask yourself what exactly do voldomort types enable you to hide that cant be hidden with a regular type? Just one thing. Their name. As you said no one else can ever name the type. That is no benefit to me the user. No-one has been able to even describe a benefit. Walters article on Dr Dobbs doesn't describe a benefit. It increases encapsulation you all squawk. No it doesn't. The private bits are still private, the public bits are still public. All it does is complicate the user side. Its the emperor's new clothes. It's not possible to construct a voldemort type outside of the function where it was declared, and you can't know what that type is, just what the API is. You can force type deduction so the type returned can be changed without breaking code, so long as the API is the same. That's how it increases encapsulation.
Re: A few notes on choosing between Go and D for a quick project
On 3/19/15 10:40 AM, weaselcat wrote:
On Thursday, 19 March 2015 at 16:59:36 UTC, Ola Fosheim Grøstad wrote:
On Thursday, 19 March 2015 at 00:42:51 UTC, weaselcat wrote:
On Wednesday, 18 March 2015 at 12:59:17 UTC, bearophile wrote:
High level constructs in D are often slower than low-level code, so
in some cases you don't want to use them.
I actually found that LDC does an _amazing_ job of optimizing high
level constructs and converting low level code to higher level
functional code resulted in minor speedups in a lot of cases.
(Other performance benefits include the algorithm primitives being
extensively optimized in phobos.)
If the code/compiler generates suboptimal code in the first place then
improvements can be somewhat random. But if you write code with good
cache locality, filling the pipeline properly then there is no
alternative to going low level.
Btw, take a look at this:
http://stackoverflow.com/questions/28922323/improving-line-wise-i-o-operations-in-d
That's really bad marketing...
python:
time python2 wc.py enwiki-latest-pages-articles1.xml-p00010p1
There are 1245473 lines
python2 wc.py enwiki-latest-pages-articles1.xml-p00010p1
0.21s user 0.08s system 99% cpu 0.294 total
wc -l:
time wc -l enwiki-latest-pages-articles1.xml-p00010p1
1245472 enwiki-latest-pages-articles1.xml-p00010p1
wc -l enwiki-latest-pages-articles1.xml-p00010p1 0.05s user
0.02s system 96% cpu 0.072 total
iterative version:
ldc -O5 -inline -release -boundscheck=off wc.d
time ./wc
There are 1245473 lines.
./wc enwiki-latest-pages-articles1.xml-p00010p1 0.59s user
0.07s system 99% cpu 0.661 total
functional version:
writeln(There are ,
(cast(string)read(args[1])).splitter('\n').array.length, lines.);
ldc -O5 -inline -release -boundscheck=off wc.d
time ./wc enwiki-latest-pages-articles1.xml-p00010p1
There are 1245473 lines.
./wc enwiki-latest-pages-articles1.xml-p00010p1 0.04s user
0.08s system 98% cpu 0.125 total
You may want to answer there, not here. I've also posted a response.
Andrei
Re: A few notes on choosing between Go and D for a quick project
On Thursday, 19 March 2015 at 00:42:51 UTC, weaselcat wrote: On Wednesday, 18 March 2015 at 12:59:17 UTC, bearophile wrote: High level constructs in D are often slower than low-level code, so in some cases you don't want to use them. I actually found that LDC does an _amazing_ job of optimizing high level constructs and converting low level code to higher level functional code resulted in minor speedups in a lot of cases. (Other performance benefits include the algorithm primitives being extensively optimized in phobos.) If the code/compiler generates suboptimal code in the first place then improvements can be somewhat random. But if you write code with good cache locality, filling the pipeline properly then there is no alternative to going low level. Btw, take a look at this: http://stackoverflow.com/questions/28922323/improving-line-wise-i-o-operations-in-d That's really bad marketing...
Re: A few notes on choosing between Go and D for a quick project
On Thursday, 19 March 2015 at 16:59:36 UTC, Ola Fosheim Grøstad
wrote:
On Thursday, 19 March 2015 at 00:42:51 UTC, weaselcat wrote:
On Wednesday, 18 March 2015 at 12:59:17 UTC, bearophile wrote:
High level constructs in D are often slower than low-level
code, so in some cases you don't want to use them.
I actually found that LDC does an _amazing_ job of optimizing
high level constructs and converting low level code to
higher level functional code resulted in minor speedups in a
lot of cases.
(Other performance benefits include the algorithm primitives
being extensively optimized in phobos.)
If the code/compiler generates suboptimal code in the first
place then improvements can be somewhat random. But if you
write code with good cache locality, filling the pipeline
properly then there is no alternative to going low level.
Btw, take a look at this:
http://stackoverflow.com/questions/28922323/improving-line-wise-i-o-operations-in-d
That's really bad marketing...
python:
time python2 wc.py
enwiki-latest-pages-articles1.xml-p00010p1
There are 1245473 lines
python2 wc.py
enwiki-latest-pages-articles1.xml-p00010p1 0.21s
user 0.08s system 99% cpu 0.294 total
wc -l:
time wc -l enwiki-latest-pages-articles1.xml-p00010p1
1245472 enwiki-latest-pages-articles1.xml-p00010p1
wc -l enwiki-latest-pages-articles1.xml-p00010p1
0.05s user 0.02s system 96% cpu 0.072 total
iterative version:
ldc -O5 -inline -release -boundscheck=off wc.d
time ./wc
There are 1245473 lines.
./wc enwiki-latest-pages-articles1.xml-p00010p1
0.59s user 0.07s system 99% cpu 0.661 total
functional version:
writeln(There are ,
(cast(string)read(args[1])).splitter('\n').array.length,
lines.);
ldc -O5 -inline -release -boundscheck=off wc.d
time ./wc enwiki-latest-pages-articles1.xml-p00010p1
There are 1245473 lines.
./wc enwiki-latest-pages-articles1.xml-p00010p1
0.04s user 0.08s system 98% cpu 0.125 total
ahem
I actually found that LDC does an _amazing_ job of optimizing
high level constructs and converting low level code to
higher level functional code resulted in minor speedups in a
lot of cases.
Re: A few notes on choosing between Go and D for a quick project
On Thursday, 19 March 2015 at 00:52:12 UTC, Walter Bright wrote: On 3/18/2015 5:34 PM, cym13 wrote: Maybe there should be a part 2 to the C-to-D little tutorial, one that shows how to code at a higher level introducing gently functional structures instead of just teaching how to write C in D. To stop thinking about steps to think about transformations isn't an easy thing. Andrei and I talked about this, how we are failing to point out how to idiomatically use D to advantage. We need to do a lot better at this. One underused resource seems to be all the examples bearophile has put on Rosetta Code: http://rosettacode.org/wiki/Category:D If he, Adam, or some other proficient D user were to do a weekly series breaking down each of those 733 examples one at a time- what idioms were used, why certain variations were more efficient- that could go a long way to introduce the language and its idioms to beginners. It would provide enough examples for 14 years of such a weekly series, by which time D3 will be getting started!
Re: A few notes on choosing between Go and D for a quick project
On Thursday, 19 March 2015 at 10:07:06 UTC, John Colvin wrote: On Tuesday, 17 March 2015 at 18:29:20 UTC, Almighty Bob wrote: On Tuesday, 17 March 2015 at 11:48:15 UTC, Nick Treleaven wrote: On 17/03/2015 10:31, Almighty Bob wrote: It's far more useful for csvReader to return a type I know and can use than it is to obscure the return type for the sake of some philosophical ideal of increasing encapsulation. Part of the art of API design is to hide implementation where it's not necessarily needed. Designers might err on the side of hiding things, because how you expose something is important as it has to be maintained indefinitely. If they expose everything then the internals can never be redesigned for better performance, etc. They don't increase encapsulation. The public members of a voldomort type are still public, you still have to code to the API of the return type whether it's a regular or voldomort type. You can keep as much private or public in either case as you like. All they do take the typename out of circulation, they make life harder for the user. There's no benefit. None. But at least the library author can stroke his chin a feel smug that there's one less type in the modules' namespace. Totally missing the point. The crux of the matter is this: changing a voldemort type (assuming the public semantics are the same) is not a breaking API change, because no-one else's code ever names it. Seriously? You cant have a public API and private implementation with a regular type? That's something specific to voldomort types? Ask yourself what exactly do voldomort types enable you to hide that cant be hidden with a regular type? Just one thing. Their name. As you said no one else can ever name the type. That is no benefit to me the user. No-one has been able to even describe a benefit. Walters article on Dr Dobbs doesn't describe a benefit. It increases encapsulation you all squawk. No it doesn't. The private bits are still private, the public bits are still public. All it does is complicate the user side. Its the emperor's new clothes.
Re: A few notes on choosing between Go and D for a quick project
On Tuesday, 17 March 2015 at 20:50:51 UTC, Bienlein wrote: Go is only a CSP-like, it isn't CSP. cf Python-CSP and PyCSP, not to mention JCSP and GPars. I'm not really sure whether this can be put exactly that way. On a machine with 4 GB RAM you can spawn about 80.000 goroutines (aka green threads). Let's say each threads calculates a large fibonacci number. If the fibonacci calculation yields the processor frequently all 80.000 run seamingly in parallel and return their result almost at the same time. With GPars this would not work. You can only start some 2.000 Java threads on a machine with the same amount of memory. If also the fibonacci calculation in Groovy/GPars yields the processor, using GPars the first 2.000 fibonacci calculations will nevertheless crowd out the following 2.000 threads of all those 80.000 fibonacci calculations and so on. I once tried this out with both Go and Groovy/GPars. -- Bienlein What about using a JVM with green threads support or Quasar, wouldn't it be more comparable? -- Paulo
Re: A few notes on choosing between Go and D for a quick project
What about using a JVM with green threads support or Quasar, wouldn't it be more comparable? -- Paulo Long text, contents of common interest in the last section :-) Thanks for the hints, Paulo. Quasar looks interesting. The current number one actor implementation for the JVM is Akka (akka.io). It was earlier built on Hawtdispatch (https://github.com/fusesource/hawtdispatch), which is similar to Quasar but much simpler. Now they have plugged in their own scheduler. Those libraries follow the same approach that a small number of threads (small to make the overhead of context switches small) serve a large number of consumers (e.g., channels, actors, or whatever consumer abstraction). This works well as long as all tasks are short-runners. It becomes a problem once you have many long-runners. Me talking about large fibonacci numbers in my previous post was to indicate that it is all about the problem with long-runners. In Vert.x they have a separate scheduler for long-runners that can serve a certain number of tasks and rejects tasks once the max is exceeded. I will check out have they have implemented channel select in Quasar. In Go they can do this efficiently, because it is done by the built-in scheduler and not in the library. As what D is concerned it has vibe.d, which follows on principle the same approach as described above from what I understand about it. There is an old discussion about integrating vibe.d in the D distribution. This way multi-threading in D comes with batteries included as in Go. I still think this is a good idea :-).
OFFTOPIC: GPars performance vs Go [was Re: A few notes on choosing between Go and D for a quick project]
On Tue, 2015-03-17 at 20:50 +, Bienlein via Digitalmars-d wrote: Go is only a CSP-like, it isn't CSP. cf Python-CSP and PyCSP, not to mention JCSP and GPars. I'm not really sure whether this can be put exactly that way. On a machine with 4 GB RAM you can spawn about 80.000 goroutines (aka green threads). Let's say each threads calculates a large fibonacci number. If the fibonacci calculation yields the processor frequently all 80.000 run seamingly in parallel and return their result almost at the same time. With GPars this would not work. You can only start some 2.000 Java threads on a machine with the same amount of memory. If also the fibonacci calculation in Groovy/GPars yields the processor, using GPars the first 2.000 fibonacci calculations will nevertheless crowd out the following 2.000 threads of all those 80.000 fibonacci calculations and so on. I once tried this out with both Go and Groovy/GPars. This needs to go to the GPars mailing lists list so I have cross posted. A priori I do not believe the claim made here: a GPars task is submitted to a thread pool, which is exactly what the goroutines are. Thus the number of Java threads is not a bound on the number of GPars tasks. Any bounds will be provided by the Fork/Join pool. If you have code you can send me, I will run it and do the benchmarking. If not I guess I will have to write some. Until we have the results of executing code, the above must be treated as pure conjecture, which I believe will be refuted. If the claim however turns out to be true, then it highlights a bug in GPars. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: A few notes on choosing between Go and D for a quick project
On Tue, 2015-03-17 at 15:30 -0700, Andrei Alexandrescu via Digitalmars-d wrote: On 3/17/15 1:50 PM, Bienlein wrote: Go is only a CSP-like, it isn't CSP. cf Python-CSP and PyCSP, not to mention JCSP and GPars. I'm not really sure whether this can be put exactly that way. On a machine with 4 GB RAM you can spawn about 80.000 goroutines (aka green threads). Let's say each threads calculates a large fibonacci number. If the fibonacci calculation yields the processor frequently all 80.000 run seamingly in parallel and return their result almost at the same time. How many physical threads would the 80K goroutines create? -- Andrei It doesn't, you specify the number of kernel threads in the thread pool manually. The default is 1. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: A few notes on choosing between Go and D for a quick project
On Tue, 2015-03-17 at 22:49 +, deadalnix via Digitalmars-d wrote: On Tuesday, 17 March 2015 at 22:30:37 UTC, Andrei Alexandrescu wrote: On 3/17/15 1:50 PM, Bienlein wrote: Go is only a CSP-like, it isn't CSP. cf Python-CSP and PyCSP, not to mention JCSP and GPars. I'm not really sure whether this can be put exactly that way. On a machine with 4 GB RAM you can spawn about 80.000 goroutines (aka green threads). Let's say each threads calculates a large fibonacci number. If the fibonacci calculation yields the processor frequently all 80.000 run seamingly in parallel and return their result almost at the same time. How many physical threads would the 80K goroutines create? -- Andrei The Go runtime manages this. About as many as there are CPU. The Go runtime manages the threadpool and scheduling of kernel threads, but the number of them remains a number specified manually unless you want the default of 1. Benchmarking generally reveals that optimal number of kernel threads in a pool is number of CPUs ±1, it depends on the work and things like cacheline misses, hyperthreads, etc., etc. -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: A few notes on choosing between Go and D for a quick project
On Wed, 2015-03-18 at 08:58 +, Bienlein via Digitalmars-d wrote: […] Thanks for the hints, Paulo. Quasar looks interesting. The current number one actor implementation for the JVM is Akka (akka.io). It was earlier built on Hawtdispatch Possibly, but there are others that get a lot of use, GPars for example :-) […] -- Russel. = Dr Russel Winder t: +44 20 7585 2200 voip: sip:russel.win...@ekiga.net 41 Buckmaster Roadm: +44 7770 465 077 xmpp: rus...@winder.org.uk London SW11 1EN, UK w: www.russel.org.uk skype: russel_winder signature.asc Description: This is a digitally signed message part
Re: OFFTOPIC: GPars performance vs Go [was Re: A few notes on choosing between Go and D for a quick project]
A priori I do not believe the claim made here: a GPars task is
submitted
to a thread pool, which is exactly what the goroutines are.
Thus the
number of Java threads is not a bound on the number of GPars
tasks. Any
bounds will be provided by the Fork/Join pool.
Here is a GPars sample from
http://www.gpars.org/1.0.0/guide/guide/GroovyCSP.html
final class Copy implements Callable {
private final DataflowChannel inChannel
private final DataflowChannel outChannel1
private final DataflowChannel outChannel2
public def call() {
final PGroup group = Dataflow.retrieveCurrentDFPGroup()
while (true) {
def i = inChannel.val
group.task {
outChannel1 i
outChannel2 i
}.join()
}
}
If inChannel.val blocks, the thread serving the Copy instance is
blocked and lost for serving other channels. If this happens
several times the JVM has run out of threads and the application
is in limbo.
This is not the case in Go. When a goroutine is blocked by a
blocking take on an empty channel, the Go scheduler withdraws the
thread serving this goroutine and assigns it to some other
goroutine.
Re: A few notes on choosing between Go and D for a quick project
clip Bearophile, You said that Unfortunately this thinking is going out of style for good reasons. I am confused (sorry, I am at work, and didn't have time to watch the 1+ hour video you linked to - maybe some clues were there)! I often find myself feeling a bit like Elazar. Not long ago I wrote some Python code using a bunch of the functional style programming tools and I was very please with the very concise code I had generated. Then, I had to make some modifications to the code. It took me an inordinate amount of time just to figure out what the code was doing, and I had written it myself just a few days earlier! Craig Maybe your years of practice and deep familiarity with imperative code patterns - both general and individual to yourself - might have skewed the result somewhat. It seems to me that much of practical programming is about having set up quick paths in your brain for recognising and manipulating common patterns. There's a big gap between understanding something intellectually and understanding something intuitively. There is quite possibly something too that, and as I imagine with more functional experience it will come easier to me. However, I still think imperative code is generally easier to reason about because (usually) each line of code is performing a single task, whereas with functional coding the goal seems to be to cram as many operations as possible into a single line (I know that isn't the real reason, it just seems that way at times). Trying to 'unroll' everything in your head can be a challenge. Throw in a lambda function or two with the mess of braces/symbols and then you have a real puzzler.
Re: A few notes on choosing between Go and D for a quick project
On Wednesday, 18 March 2015 at 01:52:00 UTC, Laeeth Isharc wrote:
On Tuesday, 17 March 2015 at 21:00:11 UTC, bachmeier wrote:
On Tuesday, 17 March 2015 at 19:00:06 UTC, jmh530 wrote:
In addition, further development of the ability to call D
from R or Python* or Julia (or vice-versa) would also be a
positive.
What do you have in mind? I no longer work much with Python so
my knowledge is limited, but calling D from R or Julia should
be no different from calling C from those languages, as you
normally compile your C code into a shared library anyway.
I've done the R-D thing many times and in the process have
worked with a big chunk of the R API. Things like allocating R
data structures from a D function, adding assertions to your D
code to allow for an easy exit when things don't work out, and
calling functions in the R math library, among other things,
are not difficult.
PyD is pretty nice, although one wouldn't want to call it from
an inner loop. And it's easy to go via cython also. (I wonder
how well cython works with interfacing with D via the C++
interface, because that way you could extend python with D
classes and have them be faster than going through PyD).
PyD also works with numpy memoryviews I think.
For Julia, easy to call D. It would be nice to port julia.h to
D (I started, but got distracted) so D can call back Julia.
I once wrote a little test server with vibe.d and LuaD to have
Lua server pages. It compiles the pages into Lua functions and
executes them from memory. (Rikki gave me the idea). No use to
parse every page when it's called.
It's not much code at all. It's only a test and very simple, not
optimized or anything. The function in vibe.d that handled it
looks like this:
/**
* Function that handles LSP using LuaD and a home made
* parser.
*/
void luaHandler(HTTPServerRequest req, HTTPServerResponse res) {
auto url = URL(req.requestURL);
// Set url arguments (note: should decode URL)
lua[args] = req.query;
// If html/lsp page is not yet compiled, parse, compile and
store it in _S
if (!canFind(_S.keys, url.pathString[1..$])) {
auto file = std.stdio.File(public/~url.pathString[1..$],
r);
string txt;
foreach (line; file.byLine(KeepTerminator.yes)) {
txt ~= to!string(line); // to!string really necessary?
}
auto asLua = parseLua(txt);
auto func = lua.loadString(asLua);
_S[url.pathString[1..$]] = func;
}
// Call the compiled Lua function
auto ret = _S[url.pathString[1..$]].call!(LuaTable)();
string html;
foreach(size_t i, string s; ret) {
html ~= s;
}
res.writeBody(cast(ubyte[])html);
}
Re: A few notes on choosing between Go and D for a quick project
Elazar Leibovich:
I personally, would have no idea what this piece of code is
doing upon first sight. I'll have to look at the documentation
of
at least two functions to understand that, and I'll have to
think carefully about what and who would throw in case of an
error.
Something like
while (n != EOF) {
n = read(fd, buf, sizeof(buf));
if (n==-1) throw(...);
if (strcmp(buf, PREFIX) == 0) {
return buf;
}
}
return NULL;
Requires no prior knowledge, and have similar effect.
I'd rather have a loop written by hand in my production code
any day, so that when debugging it, and reading it I'll have
easier time
to understand it, even though it would cost me a few more lines
when writing the code.
Unfortunately your thinking is mostly obsolete, the programming
world (well, most of it, Go is one exception) is going in the
opposite direction, and for good reasons. An explanation:
https://channel9.msdn.com/Events/GoingNative/2013/Cpp-Seasoning
Bye,
bearophile
Re: A few notes on choosing between Go and D for a quick project
CraigDillabaugh: You said that Unfortunately this thinking is going out of style for good reasons. I am confused (sorry, I am at work, and didn't have time to watch the 1+ hour video you linked to - I said unfortunately because it's another reason for us to refactor and change our coding habits :-) maybe some clues were there)! Of course. I often find myself feeling a bit like Elazar. Not long ago I wrote some Python code using a bunch of the functional style programming tools and I was very please with the very concise code I had generated. Then, I had to make some modifications to the code. It took me an inordinate amount of time just to figure out what the code was doing, and I had written it myself just a few days earlier! There's not a single correct answer to this. Replacing long bug-prone explicit-loops code with pre-made safer algorithms is often a good idea. The effort of learning some very generic and reusable functions is usually worth it. But I've also seen plenty of Haskell code that requires you to know tens of tiny functions, often named with symbols like !!. This is worse than writing for loops. High level constructs in D are often slower than low-level code, so in some cases you don't want to use them. So as usually you have to choose wisely, because most solutions aren't perfect if you use them everywhere, there are many ways to write bad code if you don't keep your eyes and brain switched on. Bye, bearophile
Re: A few notes on choosing between Go and D for a quick project
On Friday, 13 March 2015 at 17:31:09 UTC, Andrei Alexandrescu
wrote:
For example the expression (assuming s is e.g. a string)
File(/tmp/a).byChunk(4096).joiner.startsWith(s)
opens a file, progressively reads chunks of 4KB, stitches them
together at no cost, compares against a prefix until it makes a
decision, then closes the file and returns the result. A
putative Go user wouldn't even dream of using HasPrefix
directly on a stream coming from a file; the whole endeavor
would be a function that painstakingly takes all of these steps
by hand.
I personally, would have no idea what this piece of code is doing
upon first sight. I'll have to look at the documentation of
at least two functions to understand that, and I'll have to
think carefully about what and who would throw in case of an
error.
Something like
while (n != EOF) {
n = read(fd, buf, sizeof(buf));
if (n==-1) throw(...);
if (strcmp(buf, PREFIX) == 0) {
return buf;
}
}
return NULL;
Requires no prior knowledge, and have similar effect.
I'd rather have a loop written by hand in my production code any
day,
so that when debugging it, and reading it I'll have easier time
to understand it, even though it would cost me a few more lines
when writing the code.
What if this pattern repeats a few times in the code?
I'd rather have a single function that have the explicit loop than
having this pattern in slight variations spread in the code.
Writing code is easy, maintaining it afterwards is costly.
Re: A few notes on choosing between Go and D for a quick project
On Wednesday, 18 March 2015 at 12:11:52 UTC, bearophile wrote:
Elazar Leibovich:
I personally, would have no idea what this piece of code is
doing upon first sight. I'll have to look at the documentation
of
at least two functions to understand that, and I'll have to
think carefully about what and who would throw in case of an
error.
Something like
while (n != EOF) {
n = read(fd, buf, sizeof(buf));
if (n==-1) throw(...);
if (strcmp(buf, PREFIX) == 0) {
return buf;
}
}
return NULL;
Requires no prior knowledge, and have similar effect.
I'd rather have a loop written by hand in my production code
any day, so that when debugging it, and reading it I'll have
easier time
to understand it, even though it would cost me a few more lines
when writing the code.
Unfortunately your thinking is mostly obsolete, the programming
world (well, most of it, Go is one exception) is going in the
opposite direction, and for good reasons. An explanation:
https://channel9.msdn.com/Events/GoingNative/2013/Cpp-Seasoning
Bye,
bearophile
Bearophile,
You said that Unfortunately this thinking is going out of style
for good reasons. I am confused (sorry, I am at work, and
didn't have time to watch the 1+ hour video you linked to - maybe
some clues were there)!
I often find myself feeling a bit like Elazar. Not long ago I
wrote some Python code using a bunch of the functional style
programming tools and I was very please with the very concise
code I had generated. Then, I had to make some modifications to
the code. It took me an inordinate amount of time just to figure
out what the code was doing, and I had written it myself just a
few days earlier!
Craig
Re: A few notes on choosing between Go and D for a quick project
On Wednesday, 18 March 2015 at 12:45:50 UTC, CraigDillabaugh
wrote:
On Wednesday, 18 March 2015 at 12:11:52 UTC, bearophile wrote:
Elazar Leibovich:
I personally, would have no idea what this piece of code is
doing upon first sight. I'll have to look at the
documentation of
at least two functions to understand that, and I'll have to
think carefully about what and who would throw in case of an
error.
Something like
while (n != EOF) {
n = read(fd, buf, sizeof(buf));
if (n==-1) throw(...);
if (strcmp(buf, PREFIX) == 0) {
return buf;
}
}
return NULL;
Requires no prior knowledge, and have similar effect.
I'd rather have a loop written by hand in my production code
any day, so that when debugging it, and reading it I'll have
easier time
to understand it, even though it would cost me a few more
lines
when writing the code.
Unfortunately your thinking is mostly obsolete, the
programming world (well, most of it, Go is one exception) is
going in the opposite direction, and for good reasons. An
explanation:
https://channel9.msdn.com/Events/GoingNative/2013/Cpp-Seasoning
Bye,
bearophile
Bearophile,
You said that Unfortunately this thinking is going out of
style for good reasons. I am confused (sorry, I am at work,
and didn't have time to watch the 1+ hour video you linked to -
maybe some clues were there)!
I often find myself feeling a bit like Elazar. Not long ago I
wrote some Python code using a bunch of the functional style
programming tools and I was very please with the very concise
code I had generated. Then, I had to make some modifications
to the code. It took me an inordinate amount of time just to
figure out what the code was doing, and I had written it myself
just a few days earlier!
Craig
Maybe your years of practice and deep familiarity with imperative
code patterns - both general and individual to yourself - might
have skewed the result somewhat.
It seems to me that much of practical programming is about having
set up quick paths in your brain for recognising and
manipulating common patterns. There's a big gap between
understanding something intellectually and understanding
something intuitively.
Re: A few notes on choosing between Go and D for a quick project
On Wednesday, 18 March 2015 at 12:45:50 UTC, CraigDillabaugh
wrote:
On Wednesday, 18 March 2015 at 12:11:52 UTC, bearophile wrote:
Elazar Leibovich:
I personally, would have no idea what this piece of code is
doing upon first sight. I'll have to look at the
documentation of
at least two functions to understand that, and I'll have to
think carefully about what and who would throw in case of an
error.
Something like
while (n != EOF) {
n = read(fd, buf, sizeof(buf));
if (n==-1) throw(...);
if (strcmp(buf, PREFIX) == 0) {
return buf;
}
}
return NULL;
Requires no prior knowledge, and have similar effect.
I'd rather have a loop written by hand in my production code
any day, so that when debugging it, and reading it I'll have
easier time
to understand it, even though it would cost me a few more
lines
when writing the code.
Unfortunately your thinking is mostly obsolete, the
programming world (well, most of it, Go is one exception) is
going in the opposite direction, and for good reasons. An
explanation:
https://channel9.msdn.com/Events/GoingNative/2013/Cpp-Seasoning
Bye,
bearophile
Bearophile,
You said that Unfortunately this thinking is going out of
style for good reasons. I am confused (sorry, I am at work,
and didn't have time to watch the 1+ hour video you linked to -
maybe some clues were there)!
I often find myself feeling a bit like Elazar. Not long ago I
wrote some Python code using a bunch of the functional style
programming tools and I was very please with the very concise
code I had generated. Then, I had to make some modifications
to the code. It took me an inordinate amount of time just to
figure out what the code was doing, and I had written it myself
just a few days earlier!
Craig
Simple solution: use comments :-)
Re: A few notes on choosing between Go and D for a quick project
On Tuesday, 17 March 2015 at 21:16:38 UTC, Carl Sturtivant wrote:
Characterizing the problem as Go versus D is framing it to
D's disadvantage. Broadly speaking Go is one thing D is many
things.
The whole difficulty with D encountered by a person choosing
which of (e.g.) Go and D to use lies in this difference. D as a
whole is overwhelming in this situation.
D may be spoken naturally in many different ways. This is its
strength. We need to make this strength accessible to
outsiders. If an outsider can speak D at first in a way
somewhat familiar from their previous experience, that will
provide accessibility.
We may classify outsiders as coming from C, C++, Java, Python,
Go and more. Make your own list.
Each kind of outsider needs to see an approximate reflection in
D of the machinery of their usual means of expression and its
attendant conceptual models, plus exciting possibilities to
improve and extend these. This reflection must be made workable
on all scales, from details of idioms and techniques up to the
big picture.
An outsider needs a well developed specialized way in! One that
is designed for that specific class of outsider. Given such, in
the short run the D written by one class of outsider will be
quite different in its use of the language to that written by
another, which will be quite different again to that written by
contributors to phobos.
We should accept this. They may expand their horizons within D
later. The community may find ways to help such people do this,
for example by gently suggesting better parts of D to use for
technical parts of their purposes when they post to the forums.
In the meantime they have come to D because there was a
relatively efficient way for them to do that, not a bewildering
collection of possibilities that deflected them to the
certainties of the overly prescribed ways of (e.g.) Go.
Now consider outsiders from C, C++, Java, Python, Go.
I'll leave you to consider C, C++. The articles
http://dlang.org/ctod.html
http://dlang.org/cpptod.html
http://dlang.org/pretod.html
have made a detail-oriented start with tacit assumptions about
what is going on at wider scales. These languages are closely
entangled with D's development, and so are special cases given
better treatment. These could be expanded into an entire
introduction to D for C outsiders and an entire introduction to
D for C++ outsiders.
Java is not too hard especially given the design of D's
classes. Java is almost a small sublanguage of D.
Python and Go are harder. But D is a system programming
language with [long list of powerful abstraction facilities,
many supported paradigms, wonderfulness of D]. And as such
one may hope it is possible to arrange to program in D in
Pythonesque or Goish ways. Especially if this provides a way
for D to win rather than lose out to these languages, by
providing an incremental way in.
The yield facility in Python can serve as a first example here.
This is not a part of the D language. The Fiber class in
core.thread can be made to emulate it with additional
superstructure with some effort. But this is exactly the kind
of effort an outsider from Python is not immediately equipped
to make. Go's goroutine and its attendant machinery could be
examined with a similar end in view.
What all of this suggests is that some library code may need to
be written to accomodate the way into D for a class of
outsiders. The administration of where such goes needs to make
it simple to import with a standard D installation. And if D is
too inflexible to provide something important via a library, as
is possibly the case with yield, serious consideration should
be given to (don't scream) adding it to the language.
In conclusion, D is not gaining the traction it deserves as a
potential lingua franca. I am making the case that we need to
treat it as several programming languages for the purpose of
making it rapidly usable by typical outsiders. D-python,
D-java, D-whatever --- facets of D.
This entails library additions, and possibly language
additions. This entails a lot of writing of documentation, in
effect a small book for each facet of D.
There is much to be decided about how this could be best done.
I will not pursue that here. But I do believe something
approximately like this is the only way for D to start to win
out when someone decides whether to use (e.g.) Go or D in a new
situation, or even whether to use D or stay back in Python with
all of its disadvantages. The complex cliff face of D needs
some overtly navigated paths for all kinds of explorers.
I think this is a good approach. The beauty of D is that it can
be a lot of different things to different people. Do you have any
examples yet, as in D for Python programmers? E.g. loops:
Python:
for item in list:
print The item is %s % item
D:
import std.stdio : writefln;
foreach(ref item; list) {
writefln(The item is %s, item);
}
Re: A few notes on choosing between Go and D for a quick project
On Wednesday, 18 March 2015 at 02:00:40 UTC, bachmeier wrote: Unfortunately there is little documentation (though I'm working on that). I only use Linux but I would be happy if someone that knows Windows would find that it works there. Work machine's firewall isn't letting me run install_bitbucket, which doesn't surprise me as I can't get anything like bitbucket/git or whatever to work. I can try it on my home machine when I have time. I can tell that it might be a little challenging to get libgretl1-dev installed. The gretl sourceforge's Windows page only has zips and binaries. I found the Debian source, so I might try making it with Cygwin and see if that works.
Re: A few notes on choosing between Go and D for a quick project
On Wednesday, 18 March 2015 at 13:27:54 UTC, CraigDillabaugh wrote: clip Bearophile, You said that Unfortunately this thinking is going out of style for good reasons. I am confused (sorry, I am at work, and didn't have time to watch the 1+ hour video you linked to - maybe some clues were there)! I often find myself feeling a bit like Elazar. Not long ago I wrote some Python code using a bunch of the functional style programming tools and I was very please with the very concise code I had generated. Then, I had to make some modifications to the code. It took me an inordinate amount of time just to figure out what the code was doing, and I had written it myself just a few days earlier! Craig Maybe your years of practice and deep familiarity with imperative code patterns - both general and individual to yourself - might have skewed the result somewhat. It seems to me that much of practical programming is about having set up quick paths in your brain for recognising and manipulating common patterns. There's a big gap between understanding something intellectually and understanding something intuitively. There is quite possibly something too that, and as I imagine with more functional experience it will come easier to me. However, I still think imperative code is generally easier to reason about because (usually) each line of code is performing a single task, whereas with functional coding the goal seems to be to cram as many operations as possible into a single line (I know that isn't the real reason, it just seems that way at times). Trying to 'unroll' everything in your head can be a challenge. Throw in a lambda function or two with the mess of braces/symbols and then you have a real puzzler. Careful formatting is really important for long UFCS chains using lots of lambdas etc. in D for exactly this reason. It's very easy to go ooo, that's so neat, it's all one 1 line! but it's often better to spread it out to make the semantics more obvious.
Re: A few notes on choosing between Go and D for a quick project
On 2015-03-17 08:15, Walter Bright wrote:
When Voldemort types are returned, they must be by auto. The user isn't
supposed to know what the return type is, just how to use it.
I still don't like that there's no good way to describe the API. It's
not possible to put a name (that can be used in code, or reference in
the documentation) on an API like this. Something like this would be nice:
constraint InputRange (E)
{
E front();
void popFront();
bool empty();
}
InputRange!(int) result = [1, 2, 3, 4].map(e = e * 2);
--
/Jacob Carlborg
Re: A few notes on choosing between Go and D for a quick project
I'm sorry to disrupt the conversation, but I feel like I should weigh in since I'm a user the original critic felt D should be attracting. I'm a CS undergrad doing my graduation project in D, and the only other languages I worked with seriously were Python and Java(For personal projects and coursework respectively). I actually knew about D a couple of years ago and I wanted to try it because I saw the nice features and wanted to use them for something, and only recently I got a chance to use it seriously. I was put off at first(when I learned about it) because the rhetoric was that you need to be a serious programmer to use D, it's not for hobbyists. I dove into it anyway a couple of months ago because the code snippet on the old page showcased some stuff I wanted to try using and I thought they were cool. I don't think I've ever enjoyed programming as much as I did with D. Anything that I needed to do was doable. It's only been a few months programming with it and already I feel very comfortable using it. This is despite the various bugs and weird api design choices of the standard library. It's only when I actually started seriously programming with it that I actually got a feel for it's features and how they were supposed to be used. This is where it differs from Go as far as I can tell from this thread. I learned D just by doing what I wanted to do in it, and looking up any features I needed or wanted when I wanted to use, and 99 times out of 100 I found them. This should be how D is sold. It has a higher learning curve than Go or Python, but in the end you can use it in a style you're comfortable with whether you come from C, Java, Haskell, Lisp, whatever.
Re: A few notes on choosing between Go and D for a quick project
On 2015-03-16 04:25, deadalnix wrote: Compiling an hello world is almost instantaneous as far as I experienced it. I do think it is interesting for you to share your setup so we can understand what is going on and avoid for another newbie to have the same experience. It's most likely linking that takes most of the time. I think most users will just run the compiler (including the linking) and time it. Most won't care if it's the compile time or link time that is slow. -- /Jacob Carlborg
Re: A few notes on choosing between Go and D for a quick project
On Friday, 13 March 2015 at 04:49:38 UTC, Walter Bright wrote: Google does abandon significant projects now and then, such as this one: http://google-opensource.blogspot.com/2015/03/farewell-to-google-code.html In fairness pretty much everything that was on google-code has moved, mostly to GitHub. E.g. for Go stuff see https://github.com/golang
Re: A few notes on choosing between Go and D for a quick project
On Wednesday, 18 March 2015 at 15:13:24 UTC, jmh530 wrote: On Wednesday, 18 March 2015 at 02:00:40 UTC, bachmeier wrote: Unfortunately there is little documentation (though I'm working on that). I only use Linux but I would be happy if someone that knows Windows would find that it works there. Work machine's firewall isn't letting me run install_bitbucket, which doesn't surprise me as I can't get anything like bitbucket/git or whatever to work. I can try it on my home machine when I have time. I can tell that it might be a little challenging to get libgretl1-dev installed. The gretl sourceforge's Windows page only has zips and binaries. I found the Debian source, so I might try making it with Cygwin and see if that works. I'd greatly appreciate hearing about your experiences. I was lost when I tried it on Windows, but then I don't have any experience developing on Windows (that was a big part of why I moved to Linux). Technically, the gretl dependency isn't needed, but it's got such a nice interface to the matrix algebra and regression routines. You might be able to download the zip here: https://bitbucket.org/bachmeil/dmdinline/downloads and then install from a USB disk using install_local.
Re: A few notes on choosing between Go and D for a quick project
On Wednesday, 18 March 2015 at 11:53:06 UTC, Elazar Leibovich
wrote:
On Friday, 13 March 2015 at 17:31:09 UTC, Andrei Alexandrescu
wrote:
For example the expression (assuming s is e.g. a string)
File(/tmp/a).byChunk(4096).joiner.startsWith(s)
opens a file, progressively reads chunks of 4KB, stitches them
together at no cost, compares against a prefix until it makes
a decision, then closes the file and returns the result. A
putative Go user wouldn't even dream of using HasPrefix
directly on a stream coming from a file; the whole endeavor
would be a function that painstakingly takes all of these
steps by hand.
I personally, would have no idea what this piece of code is
doing
upon first sight. I'll have to look at the documentation of
at least two functions to understand that, and I'll have to
think carefully about what and who would throw in case of an
error.
Something like
while (n != EOF) {
n = read(fd, buf, sizeof(buf));
if (n==-1) throw(...);
if (strcmp(buf, PREFIX) == 0) {
return buf;
}
}
return NULL;
Requires no prior knowledge, and have similar effect.
I'd rather have a loop written by hand in my production code
any day,
so that when debugging it, and reading it I'll have easier time
to understand it, even though it would cost me a few more lines
when writing the code.
What if this pattern repeats a few times in the code?
I'd rather have a single function that have the explicit loop
than
having this pattern in slight variations spread in the code.
Writing code is easy, maintaining it afterwards is costly.
I don't want to come off as rude, but your response sounds like
someone who has little to no experience with functional
programming(i.e, a C background given your example.)
One of the major ways D parts with the C-family is it has a
strong foothold in functional programming and blends it together
extremely well. IMO using D like a better C is hurting only
yourself - functional programming is great. : )
Re: A few notes on choosing between Go and D for a quick project
On 3/18/15 4:54 PM, David Gileadi wrote:
On 3/18/15 4:48 PM, jkpl wrote:
On Wednesday, 18 March 2015 at 23:41:41 UTC, Walter Bright wrote:
On 3/18/2015 5:45 AM, CraigDillabaugh wrote:
You said that Unfortunately this thinking is going out of style
for good
reasons. I am confused (sorry, I am at work, and didn't have time
to watch
the 1+ hour video you linked to - maybe some clues were there)!
Consider this C code:
#include stdbool.h
#include stdio.h
typedef long T;
bool find(T *array, size_t dim, T t) {
int i;
for (i = 0; i = dim; i++);
{
int v = array[i];
if (v == t)
return true;
}
}
There are several bugs in it. I've showed this to probably over 1000
programmers, and nobody found all the bugs in it (they are obvious
once pointed out). It is not easy to write bug free loop code, and
find() is a trivial function.
just for fun:
1/ '=' instead of ''
2/ array should be tested against null before entering the loop
3/ 'int v' instead of 'T v'
Got them all ?
4. The semicolon after the for loop's closing paren...
5. No return if the item isn't found, leading to undefined behavior.
There may well be more, but I'll stop spamming this topic. Sorry Walter.
Re: A few notes on choosing between Go and D for a quick project
On 3/18/2015 5:45 AM, CraigDillabaugh wrote:
You said that Unfortunately this thinking is going out of style for good
reasons. I am confused (sorry, I am at work, and didn't have time to watch
the 1+ hour video you linked to - maybe some clues were there)!
Consider this C code:
#include stdbool.h
#include stdio.h
typedef long T;
bool find(T *array, size_t dim, T t) {
int i;
for (i = 0; i = dim; i++);
{
int v = array[i];
if (v == t)
return true;
}
}
There are several bugs in it. I've showed this to probably over 1000
programmers, and nobody found all the bugs in it (they are obvious once pointed
out). It is not easy to write bug free loop code, and find() is a trivial function.
Re: A few notes on choosing between Go and D for a quick project
On 3/18/15 4:48 PM, jkpl wrote:
On Wednesday, 18 March 2015 at 23:41:41 UTC, Walter Bright wrote:
On 3/18/2015 5:45 AM, CraigDillabaugh wrote:
You said that Unfortunately this thinking is going out of style
for good
reasons. I am confused (sorry, I am at work, and didn't have time
to watch
the 1+ hour video you linked to - maybe some clues were there)!
Consider this C code:
#include stdbool.h
#include stdio.h
typedef long T;
bool find(T *array, size_t dim, T t) {
int i;
for (i = 0; i = dim; i++);
{
int v = array[i];
if (v == t)
return true;
}
}
There are several bugs in it. I've showed this to probably over 1000
programmers, and nobody found all the bugs in it (they are obvious
once pointed out). It is not easy to write bug free loop code, and
find() is a trivial function.
just for fun:
1/ '=' instead of ''
2/ array should be tested against null before entering the loop
3/ 'int v' instead of 'T v'
Got them all ?
4. The semicolon after the for loop's closing paren...
Re: A few notes on choosing between Go and D for a quick project
On 3/18/2015 4:58 PM, David Gileadi wrote: There may well be more, Yup! but I'll stop spamming this topic. Sorry Walter. Of course, what the bugs actually are is not really the point, although it is fun hunting them out. But it isn't so much fun hunting them out in a large codebase, nor is it fun spending hours tracking down a bug. The ranges+algorithms method is far easier to check by inspection and get right.
Re: A few notes on choosing between Go and D for a quick project
On 3/18/2015 4:48 PM, jkpl wrote: just for fun: 1/ '=' instead of '' 2/ array should be tested against null before entering the loop 3/ 'int v' instead of 'T v' Got them all ? Nope! I'll post late tonight.
Re: A few notes on choosing between Go and D for a quick project
On 3/18/2015 5:34 PM, cym13 wrote: Maybe there should be a part 2 to the C-to-D little tutorial, one that shows how to code at a higher level introducing gently functional structures instead of just teaching how to write C in D. To stop thinking about steps to think about transformations isn't an easy thing. Andrei and I talked about this, how we are failing to point out how to idiomatically use D to advantage. We need to do a lot better at this.
Re: A few notes on choosing between Go and D for a quick project
On Wednesday, 18 March 2015 at 23:41:41 UTC, Walter Bright wrote:
On 3/18/2015 5:45 AM, CraigDillabaugh wrote:
You said that Unfortunately this thinking is going out of
style for good
reasons. I am confused (sorry, I am at work, and didn't
have time to watch
the 1+ hour video you linked to - maybe some clues were there)!
Consider this C code:
#include stdbool.h
#include stdio.h
typedef long T;
bool find(T *array, size_t dim, T t) {
int i;
for (i = 0; i = dim; i++);
{
int v = array[i];
if (v == t)
return true;
}
}
There are several bugs in it. I've showed this to probably over
1000 programmers, and nobody found all the bugs in it (they are
obvious once pointed out). It is not easy to write bug free
loop code, and find() is a trivial function.
just for fun:
1/ '=' instead of ''
2/ array should be tested against null before entering the loop
3/ 'int v' instead of 'T v'
Got them all ?
Re: A few notes on choosing between Go and D for a quick project
On Wednesday, 18 March 2015 at 23:41:41 UTC, Walter Bright wrote:
On 3/18/2015 5:45 AM, CraigDillabaugh wrote:
You said that Unfortunately this thinking is going out of
style for good
reasons. I am confused (sorry, I am at work, and didn't
have time to watch
the 1+ hour video you linked to - maybe some clues were there)!
Consider this C code:
#include stdbool.h
#include stdio.h
typedef long T;
bool find(T *array, size_t dim, T t) {
int i;
for (i = 0; i = dim; i++);
{
int v = array[i];
if (v == t)
return true;
}
}
There are several bugs in it. I've showed this to probably over
1000 programmers, and nobody found all the bugs in it (they are
obvious once pointed out). It is not easy to write bug free
loop code, and find() is a trivial function.
i is of type int and dim of type size_t which can grow bigger so
the loop may overflow... I can't find more.
Either way, I think your point couldn't be clearer :)
Maybe there should be a part 2 to the C-to-D little tutorial,
one that shows how to code at a higher level introducing gently
functional structures instead of just teaching how to write C in
D. To stop thinking about steps to think about transformations
isn't an easy thing.
Re: A few notes on choosing between Go and D for a quick project
On Wednesday, 18 March 2015 at 12:59:17 UTC, bearophile wrote: High level constructs in D are often slower than low-level code, so in some cases you don't want to use them. I actually found that LDC does an _amazing_ job of optimizing high level constructs and converting low level code to higher level functional code resulted in minor speedups in a lot of cases. (Other performance benefits include the algorithm primitives being extensively optimized in phobos.)
