On Wednesday, 16 December 2015 at 15:54:09 UTC, Andrei
Alexandrescu wrote:
Right now we can nicely stream an URL as an input range. A
great extension would be to fetch several URLs at once. When
accessing r.front for that range, the user gets a pair of URL
and data chunk.
Of course the point is to make all these fetches work
simultaneously. Inside, the range would e.g. use asynchronous
curl.
Andrei
Whenever I want to do that I use vibe.d and my observe library
(based on RX observables) and the code would look a bit like this:
urls
.observe()
.zip(localPaths)
.fork(numParallel)
.concatMap!((part)
{
return downloadFile(part[0],part[1]).retry(3)
})
.subscribe(
(p){writeln("File %s downloaded to %s",p[0],p[1]);},
(e){writeln("Error ",e);},
(){writeln("Completed!");}
);
Here observe() makes an observable out of any range; fork() uses
vibe.d's Fibers to achieve concurrency; downloadFile() returns
another observable that gets concatMap'ped; and retry does what
you think it does. Also, errors get propagated without throws and
catches.
I think what I am trying to say is that a lot of stuff is already
available on code.dlang.org, just not in phobos. Which begs the
question, should it be? And if it does, shouldn't you rather
consider merging vibe.d with phobos? Or parts of it?
