Re: how to do this meta-programming? print the address of random element's address of a variable length of arrays?
On Sunday, 13 September 2020 at 18:24:01 UTC, mw wrote:
But, I'd reflect on my experience so far on compile-time
meta-programming in D as a novice user, the big problems are:
-- in D, there are too many choices, with no clear guideline
which one is *THE* one to use for a particular purpose:
language or library mechanisms? mixin? template? AliasSeq /
aliasSeqOf? Tuple? (non-)?-eponymous version of ...?; and even
for a particular option, e.g. Tuple!(int, "MA_optInTimePeriod",
TA_MAType, "opInMAType"), there are choices to use either
token (int) or string ("MA_optInTimePeriod"). And user does not
have a strong guide on which choice is *THE* way to proceed.
Each mechanism seems to have / fit a particular purpose, but
when you start to use it, you'll probably find there are new
problems come-in later, and you want to revisit the choice you
made earlier on.
By contrast, in C: there is only *ONE* mechanism, i.e. macro,
that's it.
I think the biggest issue here is that, like you said, it's hard
to find good resources for learning how to use D's various
reflection and code-generation facilities. In the community
Discord, we've tried to collect some links in the #resources
channel, which you might find helpful:
- Philippe Sigaud's "D Template Tutorial"
http://dpldocs.info/experimental-docs/std.typecons.Tuple.html
- Bradley Chatha's "Intro to D metaprogramming"
https://bradley.chatha.dev/Blog
- Vladimir Panteleev's "Functional Image Processing in D"
https://blog.thecybershadow.net/2014/03/21/functional-image-processing-in-d/
Ideally, we'd have some official, easy-to-find place where
tutorials like this could be collected (maybe on the Wiki?), but
so far no one's taken the initiative to set that up. One of the
downsides of having a volunteer community, I suppose.
-- in D, there is no easy way to convert between token <==>
string. Given a token, does token.stringof always work to paste
with other strings to generate a new token? and given a string,
does mixin!"string" always work to be a valid token?
There's no 100% reliable way to round-trip a token to a string
and back again, because the same token can have a different
meaning depending on what scope it's used in. Adam Ruppe goes
over some alternatives to string mixins you can use to avoid this
pitfall in one of his "Tips of the Week":
http://www.arsdnet.net/this-week-in-d/2016-feb-21.html
-- in D, there is no easy way to see directly the generated
source code by the compiler at compile-time, which makes the
debug difficult during development.
By contrast, in C: one can easily see the result via: cpp -P
foo.h > foo.c
I agree that this is an issue. There are a couple compiler flags
that can help (-mixin=filename, -vcg-ast), but mostly you're
stuck with printf-style debugging via `pragma(msg)`.
Re: how to do this meta-programming? print the address of random element's address of a variable length of arrays?
On Sunday, 13 September 2020 at 10:16:46 UTC, Paul Backus wrote:
On Sunday, 13 September 2020 at 07:00:36 UTC, mw wrote:
Here it is: D wrapper for https://ta-lib.org/
https://github.com/mingwugmail/talibd
I end up using C macro to generate D functions, the single
template is this one:
https://github.com/mingwugmail/talibd/blob/master/source/talibd.h#L117
#define DECL_TA_FUNC(TA_FUNC, FUNC_INS, FUNC_OUTS,
expected_lookback) __NL__\
The most straightforward way to do this in D is with a mixin
template. Something like:
mixin template DECL_TA_FUNC(string TA_FUNC, FUNC_INS,
FUNC_OUTS, int expected_lookback)
{
bool impl(...)
{
// ...
}
// Could also wrap the whole function in a string mixin,
// but this is easier.
mixin("alias ", TA_FUNC, " = impl;");
}
Which you would then use like this:
mixin DECL_TA_FUNC!(
"TA_MA",
Tuple!(int, "MA_optInTimePeriod", TA_MAType, "opInMAType"),
Tuple!(double[], "outMA"),
MA_optInTimePeriod - 1
);
Thanks, I will do the exercise some other day.
But, I'd reflect on my experience so far on compile-time
meta-programming in D as a novice user, the big problems are:
-- in D, there are too many choices, with no clear guideline
which one is *THE* one to use for a particular purpose: language
or library mechanisms? mixin? template? AliasSeq / aliasSeqOf?
Tuple? (non-)?-eponymous version of ...?; and even for a
particular option, e.g. Tuple!(int, "MA_optInTimePeriod",
TA_MAType, "opInMAType"), there are choices to use either token
(int) or string ("MA_optInTimePeriod"). And user does not have a
strong guide on which choice is *THE* way to proceed. Each
mechanism seems to have / fit a particular purpose, but when you
start to use it, you'll probably find there are new problems
come-in later, and you want to revisit the choice you made
earlier on.
By contrast, in C: there is only *ONE* mechanism, i.e. macro,
that's it.
-- in D, there is no easy way to convert between token <==>
string. Given a token, does token.stringof always work to paste
with other strings to generate a new token? and given a string,
does mixin!"string" always work to be a valid token?
By contrast, in C: there is only *ONE* way: xxx ## yyy.
-- in D, there is no easy way to see directly the generated
source code by the compiler at compile-time, which makes the
debug difficult during development.
By contrast, in C: one can easily see the result via: cpp -P
foo.h > foo.c
As I said earlier, I'm not very experienced with C macros either,
however with some googling, I was able to work out a C macro
version to generate D code; but with all the help so far, I still
have no confidence that I can work out a solution in D to
implement this:
```
bool impl(...)
{
// ...
}
```
Re: how to do this meta-programming? print the address of random element's address of a variable length of arrays?
On Sunday, 13 September 2020 at 07:00:36 UTC, mw wrote:
Here it is: D wrapper for https://ta-lib.org/
https://github.com/mingwugmail/talibd
I end up using C macro to generate D functions, the single
template is this one:
https://github.com/mingwugmail/talibd/blob/master/source/talibd.h#L117
#define DECL_TA_FUNC(TA_FUNC, FUNC_INS, FUNC_OUTS,
expected_lookback) __NL__\
The most straightforward way to do this in D is with a mixin
template. Something like:
mixin template DECL_TA_FUNC(string TA_FUNC, FUNC_INS, FUNC_OUTS,
int expected_lookback)
{
bool impl(...)
{
// ...
}
// Could also wrap the whole function in a string mixin,
// but this is easier.
mixin("alias ", TA_FUNC, " = impl;");
}
Which you would then use like this:
mixin DECL_TA_FUNC!(
"TA_MA",
Tuple!(int, "MA_optInTimePeriod", TA_MAType, "opInMAType"),
Tuple!(double[], "outMA"),
MA_optInTimePeriod - 1
);
Re: how to do this meta-programming? print the address of random element's address of a variable length of arrays?
On Sunday, 13 September 2020 at 01:25:43 UTC, mw wrote: On Saturday, 12 September 2020 at 20:29:40 UTC, Paul Backus If you have a "real-life" application in mind for this, I'd be curious to hear what it is. I'm wrapping a C library, trying to write a single D function / template that can work with a group of C functions, hence I need that kind of signature I described. I'll post the code when I'm ready. Here it is: D wrapper for https://ta-lib.org/ https://github.com/mingwugmail/talibd I end up using C macro to generate D functions, the single template is this one: https://github.com/mingwugmail/talibd/blob/master/source/talibd.h#L117 #define DECL_TA_FUNC(TA_FUNC, FUNC_INS, FUNC_OUTS, expected_lookback) __NL__\ and the macro instantiations are on line 144, 158, 168: DECL_TA_FUNC(TA_MA, MA_INS, MA_OUTS, (MA_optInTimePeriod-1)) DECL_TA_FUNC(TA_RSI, RSI_INS, RSI_OUTS, RSI_optInTimePeriod) DECL_TA_FUNC(TA_MACD, MACD_INS, MACD_OUTS, (optInSlowPeriod+optInSignalPeriod-2)) the generated D functions are here: https://github.com/mingwugmail/talibd/blob/master/source/talibd.d#L47 you can take a look of the 3 generated functions to see the similarities, and pay attention to: -- func decl: TA_xxx( , ) -- assertions on -- func calls: TA_xxx_Lookback() -- func call: talib.TA_xxx(, arrays>, ) There are a number of C macro tricks was used, which I just did some googling to be able to get it done in C -- and with the added benefits that I can directly see the generated source file to debug in the development process. I think it's a non-trivial task to get this compile-time meta-programming done in D, if it can be done at all (I'm not sure). Anyone want to give it a try? and submit a PR :-)
Re: how to do this meta-programming? print the address of random element's address of a variable length of arrays?
On Sat, Sep 12, 2020 at 07:31:57PM +, mw via Digitalmars-d-learn wrote:
[...]
> I've tried something like this: the AliasSeq specify the logical
> divide
>
>printRandomElemAddr(AliasSeq!(extraA, extraB, extraC, extraD), a, b, c);
>
> but cannot make it work.
Yes, because AliasSeq auto-expands, so `func(AliasSeq!(A,B,C),
AliasSeq!(D,E,F))` is equivalent to `func(A,B,C,D,E,F)`.
> (I'm asking for a more general solution, e.g. what if we have 3, or N
> sets of variadic parameters?)
>
> Looks like we can only pass 1 variadic parameters, then the question
> is what's the best way to divide it?
>
> Is there any special marker (variable or type?) can be used to divide?
> and what's the staticSplit?
Maybe the nested templates hack could be used? It depends on what
exactly you're trying to do though:
template myFunc(Args1...) {
template myFunc(Args2...) {
auto myFunc(/* runtime params here */) {
// you can use Args1.length and
// Args2.length here.
...
}
}
}
myFunc!(A,B,C)!(D,E,F)(/* runtime args here */);
Alternatively, define a non-eponymous version of AliasSeq that will
allow you to pass multiple lists of compile-time parameters without
auto-expanding and coalescing them:
template WrappedSeq(T...) {
// N.B.: not eponymous
alias members = T;
}
auto myFunc(Args1, Args2)(/*runtime params*/) {
alias list1 = Args1.members; // need manual unwrap
alias list2 = Args2.members;
}
// (Yeah the caller side will look ugly. C'est la vie.)
myFunc!(WrappedSeq!(A,B,C), WrappedSeq!(D,E,F))(/*runtime args*/);
T
--
Why can't you just be a nonconformist like everyone else? -- YHL
Re: how to do this meta-programming? print the address of random element's address of a variable length of arrays?
On Saturday, 12 September 2020 at 20:29:40 UTC, Paul Backus wrote: On Saturday, 12 September 2020 at 19:31:57 UTC, mw wrote: (I'm asking for a more general solution, e.g. what if we have 3, or N sets of variadic parameters?) Looks like we can only pass 1 variadic parameters, then the question is what's the best way to divide it? Is there any special marker (variable or type?) can be used to divide? and what's the staticSplit? It's possible, but there's no widely-used technique, because it's much easier to simply pass each "parameter set" as a separate array or tuple. So your function signature would look like: auto fun(Set1, Set2, Set3)(Set1 args1, Set2 args2, Set3 args3) if (allSatisfy!(Or!(isArray, isTuple), Set1, Set2, Set3)) Or in the variadic case: auto fun(Args...)(Args args) if (allSatisfy!(Or!(isArray, isTuple), Args)) If you have a "real-life" application in mind for this, I'd be curious to hear what it is. I'm wrapping a C library, trying to write a single D function / template that can work with a group of C functions, hence I need that kind of signature I described. I'll post the code when I'm ready.
Re: how to do this meta-programming? print the address of random element's address of a variable length of arrays?
On Saturday, 12 September 2020 at 19:31:57 UTC, mw wrote: (I'm asking for a more general solution, e.g. what if we have 3, or N sets of variadic parameters?) Looks like we can only pass 1 variadic parameters, then the question is what's the best way to divide it? Is there any special marker (variable or type?) can be used to divide? and what's the staticSplit? It's possible, but there's no widely-used technique, because it's much easier to simply pass each "parameter set" as a separate array or tuple. So your function signature would look like: auto fun(Set1, Set2, Set3)(Set1 args1, Set2 args2, Set3 args3) if (allSatisfy!(Or!(isArray, isTuple), Set1, Set2, Set3)) Or in the variadic case: auto fun(Args...)(Args args) if (allSatisfy!(Or!(isArray, isTuple), Args)) If you have a "real-life" application in mind for this, I'd be curious to hear what it is.
Re: how to do this meta-programming? print the address of random element's address of a variable length of arrays?
On Saturday, 12 September 2020 at 19:06:47 UTC, Paul Backus wrote: On Saturday, 12 September 2020 at 18:16:51 UTC, mw wrote: Now, let me expand this challenge: suppose we need to add a new set of variable length extra parameters in parallel to the arrays, i.e: [...] Now the question is how to pass & handle 2 sets of variadic parameters? alias firstSet = args[0 .. $/2]; alias secondSet = args[$/2 .. $]; This solution assumes the two sets are of equal size; what if we don't have such assumption? i.e. we only know the two sets divided into two logical groups. I've tried something like this: the AliasSeq specify the logical divide printRandomElemAddr(AliasSeq!(extraA, extraB, extraC, extraD), a, b, c); but cannot make it work. (I'm asking for a more general solution, e.g. what if we have 3, or N sets of variadic parameters?) Looks like we can only pass 1 variadic parameters, then the question is what's the best way to divide it? Is there any special marker (variable or type?) can be used to divide? and what's the staticSplit?
Re: how to do this meta-programming? print the address of random element's address of a variable length of arrays?
On Saturday, 12 September 2020 at 18:16:51 UTC, mw wrote:
Now, let me expand this challenge: suppose we need to add a new
set of variable length extra parameters in parallel to the
arrays, i.e:
[...]
Now the question is how to pass & handle 2 sets of variadic
parameters?
void fun(Args...)(Args args)
if (args.length % 2 == 0)
{
alias firstSet = args[0 .. $/2];
alias secondSet = args[$/2 .. $];
// rest of function body goes here
}
You will probably also want to include some `static asserts` to
make sure each set contains arguments of the appropriate types.
Re: how to do this meta-programming? print the address of random element's address of a variable length of arrays?
On Saturday, 12 September 2020 at 14:31:59 UTC, Paul Backus wrote:
On Saturday, 12 September 2020 at 03:19:23 UTC, mw wrote:
I.e. I want to learn the generic meta-programming way to
assemble such parameter list (&(x[i], &(y[j])) at compile
time, it is possible?
It's possible if you use a helper function. Here's how:
import std.meta: allSatisfy;
import std.traits: isArray;
void printRandomElemAddr(Arrays...)(Arrays arrays)
if (allSatisfy!(isArray, Arrays))
{
auto randomElemAddr(size_t i)()
if (i < arrays.length)
{
import std.random: uniform;
return [i][uniform(0, $)];
}
import std.stdio: writeln;
import std.meta: staticMap, aliasSeqOf;
import std.range: iota;
writeln(staticMap!(randomElemAddr,
aliasSeqOf!(iota(arrays.length;
}
void main()
{
int[] a = [1];
int[] b = [2, 3];
double[] c = [4, 5, 6];
printRandomElemAddr(a);
printRandomElemAddr(a, b);
printRandomElemAddr(a, b, c);
}
Thanks, this works. staticMap and aliasSeqOf is the key.
Now, let me expand this challenge: suppose we need to add a new
set of variable length extra parameters in parallel to the
arrays, i.e:
// just use scalar type for demo
intextraA;
string extraB;
double extraC;
// need to be called as:
printRandomElemAddr(extraA, a);
printRandomElemAddr(extraA, extraB, a, b);
printRandomElemAddr(extraA, extraB, extraC, a, b, c);
basically, two sets of variadic parameters, but need to be
treated differently:
-- the 1st scalar set, just use as it is
-- the 2nd array set, need some processing (which you have done).
Now the question is how to pass & handle 2 sets of variadic
parameters?
Re: how to do this meta-programming? print the address of random element's address of a variable length of arrays?
On Saturday, 12 September 2020 at 03:19:23 UTC, mw wrote:
I.e. I want to learn the generic meta-programming way to
assemble such parameter list (&(x[i], &(y[j])) at compile time,
it is possible?
It's possible if you use a helper function. Here's how:
import std.meta: allSatisfy;
import std.traits: isArray;
void printRandomElemAddr(Arrays...)(Arrays arrays)
if (allSatisfy!(isArray, Arrays))
{
auto randomElemAddr(size_t i)()
if (i < arrays.length)
{
import std.random: uniform;
return [i][uniform(0, $)];
}
import std.stdio: writeln;
import std.meta: staticMap, aliasSeqOf;
import std.range: iota;
writeln(staticMap!(randomElemAddr,
aliasSeqOf!(iota(arrays.length;
}
void main()
{
int[] a = [1];
int[] b = [2, 3];
double[] c = [4, 5, 6];
printRandomElemAddr(a);
printRandomElemAddr(a, b);
printRandomElemAddr(a, b, c);
}
Re: how to do this meta-programming? print the address of random element's address of a variable length of arrays?
On Saturday, 12 September 2020 at 03:11:09 UTC, Ali Çehreli wrote:
On 9/11/20 6:44 PM, mw wrote:> e.g.
>
> int[] a = new int[la];
> int[] b = new int[lb];
> int[] c = new int[lc];
> int[] d = new int[ld];
>
>
> the func I want to write, e.g. for 2 arrays (instantiation)
is like this:
>
> void print_random_elem_addr(int[] x, int[] y) {
>auto i = random_int_between(0, x.length);
>auto j = random_int_between(0, y.length);
>print(&(x[i], &(y[j])); // only single print() func call
allowed!
> }
>
>
> But I want one generic function, which can be called as:
>
> print_random_elem_addr(a, b);
> print_random_elem_addr(a, b, c);
> print_random_elem_addr(a, b, c, d);
Thanks for the reply.
If they are all of same type like int[] in this case, then you
but, this is not the intention, we should suppose the array's are
heterogeneous type ...
can variable number of parameters, which means "any number of
int[] arrays" below, elements of which can be called either as
separate arguments or as a single array argument:
import std.stdio;
import std.random;
void print_random_elem_addr(int[][] arrays...) {
... to prevent passing in parameters as array of array like this.
foreach (i, array; arrays) {
const chosen = uniform(0, array.length);
writefln!"Array %s, element %s: %s"(i, chosen,
[chosen]);
actually this writefln will be called n times.
I intentionally require:
print(&(x[i], &(y[j])); // only single print() func call
allowed!
I.e. I want to learn the generic meta-programming way to assemble
such parameter list (&(x[i], &(y[j])) at compile time, it is
possible?
Re: how to do this meta-programming? print the address of random element's address of a variable length of arrays?
On 9/11/20 6:44 PM, mw wrote:> e.g.
>
> int[] a = new int[la];
> int[] b = new int[lb];
> int[] c = new int[lc];
> int[] d = new int[ld];
>
>
> the func I want to write, e.g. for 2 arrays (instantiation) is like this:
>
> void print_random_elem_addr(int[] x, int[] y) {
>auto i = random_int_between(0, x.length);
>auto j = random_int_between(0, y.length);
>print(&(x[i], &(y[j])); // only single print() func call allowed!
> }
>
>
> But I want one generic function, which can be called as:
>
> print_random_elem_addr(a, b);
> print_random_elem_addr(a, b, c);
> print_random_elem_addr(a, b, c, d);
If they are all of same type like int[] in this case, then you can
variable number of parameters, which means "any number of int[] arrays"
below, elements of which can be called either as separate arguments or
as a single array argument:
import std.stdio;
import std.random;
void print_random_elem_addr(int[][] arrays...) {
foreach (i, array; arrays) {
const chosen = uniform(0, array.length);
writefln!"Array %s, element %s: %s"(i, chosen, [chosen]);
}
}
void main() {
auto randomLengthArray() {
return new int[uniform(1, 101)];
}
auto a = randomLengthArray();
auto b = randomLengthArray();
auto c = randomLengthArray();
writeln("As independent arguments:");
print_random_elem_addr(a, b, c);
writeln("As a single argument:");
print_random_elem_addr([a, b, c]);
}
Warning: The array that is automatically generated by the first
print_random_elem_addr() call in main() is short-lived: You cannot store
a slice of it because the array that contains the arguments may be
destroyed upon leaving the function (e.g. in the "independent" case above).
Here is some more information:
http://ddili.org/ders/d.en/parameter_flexibility.html#ix_parameter_flexibility.variadic%20function
There are other ways of doing the same thing. For example, if you want
to work with different ranges, you can use tuple template parameters:
http://ddili.org/ders/d.en/templates_more.html#ix_templates_more.tuple%20template%20parameter
Ali
