Re: Trying to reproduce Node.js async waterfall pattern.. (Meta-programming)
Yes, the final callback is always called, but if an error is passed to the
callback by any of the main steps in the sequence ladder, it will
immediately jump to the final callback and not execute further steps.
OK.
What do you mean? The compiler does deduce the type of Funcs.
If you look at where I call Waterfall() in main, you'll see I had to
manually specify (Callback cb) instead of just (cb); since it didn't know
that the Funcs... were of type AsyncFunc
you can use std.typetuple.allSatisfy with a helper template:
enum isAsyncFunc(T) = is(T == AsyncFunc);
... (Funcs...)(Funcs funcs) if (allSatisfy!(isAsyncFunc, Funcs))
{ ... }
(cb) { cb(null, one);} is possible, but that means it's a function
template, not a function.
You can get this syntax by making the callbacks template arguments,
which means they must be known at compile-time. Is that OK with you or
do you need the possibility to define the callbacks at runtime?
The goal was to do as much as possible at compile time. Could you elaborate
on this a bit. I guess the answer is, yes, it's okay with me.
I mean, it's possible to get a (cb){ some code } syntax, but as that
define a function template, it has no real type: it cannot be a
runtime argument, only an alias template parameter. That means that
all callbacks must be defined in your code, none can come from user
input or runtime computation.
I don't get it: none of your callbacks have a return type per se: they
all return 'void'.
Do you want callbacks that really return something?
Yes, the callbacks at step0 should output a type in the result which is
specific to step0, and then that type should be fed in as a secondary
parameter to step1.
I didn't get that far, as I was already stuck.
OK, I get it.
thanks for your patience!
Well, we are there to explain :-)
I'll have try and code something. If I can get it to work, I'll post it there.
Trying to reproduce Node.js async waterfall pattern.. (Meta-programming)
Hi,
I just started learning D, and thought I'd throw myself in at the
deep end with some meta-programming, trying to write the
equivalent of the commonly used, async waterfall, and also,
because I'd like to use it...
If you aren't familiar with it, waterfall is a function that is
passed a sequence of functions as its arguments that are to be
executed in order... (it's a pattern for async programming).
Here is an example in Node.js...
waterfall(
function(asyncCallback){
asyncCallback(null, one);
},
function(lastResult, asyncCallback){
// lastResult equals one
asyncCallback(null, two);
},
function(lastResult, asyncCallback){
// lastResult equals two
asyncCallback(null, done);
}
,
// final callback
function (error, finalResult)
{
// result equals done
}
);
Each function is given a callback, that when called, steps the
waterfall forward on to the next function to process. If an
error is passed to the callback (instead of null), then the
waterfall stops processing and calls the final callback at the
end of the chain.
This is how I have it implemented so far, but as you can see,
there are some issues...
import std.stdio;
import std.conv;
import std.exception;
alias Callback = void delegate(Exception error, string result);
alias AsyncFunc = void function(Callback cb);
static Waterfall(TLastResult, Funcs...)(Funcs funcs, Callback
finalCallback, TLastResult lastResult)
{
static if (funcs.length)
{
auto cb = (Exception error, string result)
{
if (error is null)
Waterfall(funcs[1 .. $], finalCallback, result);
else
finalCallback(error, result);
};
funcs[0](cb);
}
else
finalCallback(null, lastResult);
}
static Waterfall(Funcs...)(Funcs funcs, Callback finalCallback)
{
static if (funcs.length)
{
auto cb = (Exception error, string result)
{
if (error is null)
Waterfall(funcs[1 .. $], finalCallback, result);
else
finalCallback(error, result);
};
funcs[0](cb);
}
else
finalCallback(null, lastResult);
}
void main()
{
Waterfall(
(Callback cb) { writeln(fn0); cb(null, one); },
(Callback cb) { writeln(fn1); cb(null, two); },
// (Callback cb) { writeln(fnError); cb(new Exception(Bad
joojoo), two); },
(Callback cb) { writeln(fn2); cb(null, done); },
(Exception error, string result)
{
if (error !is null)
writeln(Error = ~ error.to!string);
if (result !is null)
writeln(Result = ~ result.to!string);
}
);
}
1) I can't see any way to get the compiler to deduce the type of
Funcs I had an urge to somehow specialize the variadic
Funcs... but I couldn't figure out any syntax to do that.
Well, because of that, I have to repeatedly say (Callback cb)
instead of (cb).
2) I can't see a path to flow the output type of the previous
callback to the input TLastResult for the next... so it's stuck
on string :(
3) I had to use a specialization to deal with the head case;
(the first function doesn't have an input from a previous
result). Minor, but niggly.
Any input on how to proceed would be great!
thanks!
Christian
BTW, you can read more about async-waterfall here...
https://www.npmjs.org/package/async-waterfall
Re: Trying to reproduce Node.js async waterfall pattern.. (Meta-programming)
On Mon, Jun 23, 2014 at 9:39 PM, Christian Beaumont via
Digitalmars-d-learn digitalmars-d-learn@puremagic.com wrote:
Each function is given a callback, that when called, steps the waterfall
forward on to the next function to process. If an error is passed to the
callback (instead of null), then the waterfall stops processing and calls
the final callback at the end of the chain.
Just to be sure: whether or not an error is passed to a callback, the
final callback is always called?
I mean, the last callback could also be called *only when something
fails*, a bit like a default case in a switch, or an error-handling
routine.
1) I can't see any way to get the compiler to deduce the type of Funcs
What do you mean? The compiler does deduce the type of Funcs.
I had an urge to somehow specialize the variadic Funcs... but I couldn't
figure out any syntax to do that.
What do you mean by 'specialize'?
Well, because of that, I have to
repeatedly say (Callback cb) instead of (cb).
(cb) { cb(null, one);} is possible, but that means it's a function
template, not a function.
You can get this syntax by making the callbacks template arguments,
which means they must be known at compile-time. Is that OK with you or
do you need the possibility to define the callbacks at runtime?
2) I can't see a path to flow the output type of the previous callback to
the input TLastResult for the next... so it's stuck on string :(
I don't get it: none of your callbacks have a return type per se: they
all return 'void'.
Do you want callbacks that really return something?
3) I had to use a specialization to deal with the head case; (the first
function doesn't have an input from a previous result). Minor, but niggly.
You can test if func[0] takes one or two arguments:
import std.traits: isCallable, ReturnType;
static if (isCallable!(Func[0]) ReturnType!(Func[0]).length == 1)
Re: Trying to reproduce Node.js async waterfall pattern.. (Meta-programming)
Just to be sure: whether or not an error is passed to a
callback, the
final callback is always called?
I mean, the last callback could also be called *only when
something
fails*, a bit like a default case in a switch, or an
error-handling
routine.
Yes, the final callback is always called, but if an error is
passed to the callback by any of the main steps in the sequence
ladder, it will immediately jump to the final callback and not
execute further steps.
What do you mean? The compiler does deduce the type of Funcs.
If you look at where I call Waterfall() in main, you'll see I had
to manually specify (Callback cb) instead of just (cb); since it
didn't know that the Funcs... were of type AsyncFunc
What do you mean by 'specialize'?
That is to say, there is no way I can see, to say that the
variadic template parameter Funcs... are all AsyncFunc's. I
think I noticed that in Swift you can say something like
Funcs:AsyncFunc... to specialize the variadic. No swift
expert, but was just browsing around today trying to compare how
you might do it in C++ or other languages.
(cb) { cb(null, one);} is possible, but that means it's a
function
template, not a function.
You can get this syntax by making the callbacks template
arguments,
which means they must be known at compile-time. Is that OK with
you or
do you need the possibility to define the callbacks at runtime?
The goal was to do as much as possible at compile time. Could
you elaborate on this a bit. I guess the answer is, yes, it's
okay with me.
I don't get it: none of your callbacks have a return type per
se: they
all return 'void'.
Do you want callbacks that really return something?
Yes, the callbacks at step0 should output a type in the result
which is specific to step0, and then that type should be fed in
as a secondary parameter to step1.
I didn't get that far, as I was already stuck.
You can test if func[0] takes one or two arguments:
import std.traits: isCallable, ReturnType;
static if (isCallable!(Func[0]) ReturnType!(Func[0]).length
== 1)
Ah, yes, that sounds reasonable, I already thought something like
that may do the trick.
thanks for your patience!
Re: Trying to reproduce Node.js async waterfall pattern.. (Meta-programming)
Just an idea that popped into my head... Maybe I can use variant for the input/output types? I haven't looked at it yet, so I'm not sure what it does, or the performance costs. I realized that because the final callback always gets called, and the types of the intermediate steps may be different, there is actually no way at all to define a completely uniformly type safe final callback... although the point is a little moot, because the only way you end up in the final callback is either by an Exception or a Result from the finalCallback - 1 step. So you can actually never get both an Exception and a Result
