On 5/26/17 10:41 AM, John Colvin wrote:
On Wednesday, 24 May 2017 at 13:39:57 UTC, Mike Parker wrote:
Some of you may remember Jon Degenhardt's talk from one of the Silicon
Valley D meetups, where he described the performance improvements he
saw when he rewrote some of eBay's command line tools in D. He has now
put the effort into crafting a blog post on the same topic, where he
takes D version of a command-line tool written in Python and
incrementally improves its performance.
The blog:
https://dlang.org/blog/2017/05/24/faster-command-line-tools-in-d/
Reddit:
https://www.reddit.com/r/programming/comments/6d25mg/faster_command_line_tools_in_d/
I spent some time fiddling with my own manual approaches to making this
as fast, wasn't satisfied and so decided to try using Steven's iopipe
(https://github.com/schveiguy/iopipe) instead. Results were excellent.
https://gist.github.com/John-Colvin/980b11f2b7a7e23faf8dfb44bd9f1242
nice! hm....
/** something vaguely like this should be in iopipe, users shouldn't
need to write it */
auto ref runWithEncoding(alias process, FileT, Args...)(FileT file, auto
ref Args args)
stealing for iopipe, thanks :) I'll need to dedicate another slide to you...
On my machine:
python takes a little over 20s, pypy wobbles around 3.5s, v1 from the
blog takes about 3.9s, v4b took 1.45s, a version of my own that is
hideous* manages 0.78s at best, the above version with iopipe hits below
0.67s most runs.
Not bad for a process that most people would call "IO-bound" (code for
"I don't want to have to write fast code & it's all the disk's fault").
Obviously this version is a bit more code than is ideal, iopipe is
currently quite "barebones", but I don't see why with some clever
abstractions and wrappers it couldn't be the default thing that one does
even for small scripts.
The idea behind iopipe is to give you the building blocks to create
exactly the pipeline you need, without a lot of effort. Once you have
those blocks, then you make higher level functions out of it. Like you
have above :)
BTW, there is a byLineRange function that handles slicing off the
newline character inside iopipe.textpipe.
-Steve
