Re: Templates, D way
On Friday, 8 September 2017 at 10:27:57 UTC, Corey Lubin wrote: On Tuesday, 5 September 2017 at 12:41:45 UTC, Computermatronic wrote: I find it very strange that this works, as a non-mixin template should not be able to capture the context of where it was instantiated. If you take the alias template parameters out it behaves how it should (that is an error message saying this is not accessible). Computermatronic (and crimaniak and John Colvin): The behavior is as should be expected and is part of the definition of how templates' alias parameters are supposed to work. The error that occurs when you stop using an alias parameter occurs because you're no longer forcing the capture of an enclosing scope by passing in an alias parameter that requires such a scope; not because the error should have occurred the first time as well. I just actually payed attention to what the example code does just now. Looking at the specific example, I suppose I can see what the "strangeness" that's being referred to is. I suppose that even though the context is indeed captured in whole: 1) the fact that the template's original declaration presents like an ordinary freestanding function makes it look weird (like a mixin template function might look weird referring to a context it's not in yet, except more weird because the hinting mixin keyword indicating it could be mixed in anywhere isn't there) 2) the expectation is that the sneakily stolen context will/can only be used for and via the alias parameters it was created captured for, and that the allowing of explicit `this` usage within the definition is too much of a compiler implementation leak? With regards to #2, it is true, of course. (though it makes it easier for implementations) Gotta run.
Re: Templates, D way
On Tuesday, 5 September 2017 at 12:41:45 UTC, Computermatronic wrote: I find it very strange that this works, as a non-mixin template should not be able to capture the context of where it was instantiated. If you take the alias template parameters out it behaves how it should (that is an error message saying this is not accessible). Computermatronic (and crimaniak and John Colvin): The behavior is as should be expected and is part of the definition of how templates' alias parameters are supposed to work. The error that occurs when you stop using an alias parameter occurs because you're no longer forcing the capture of an enclosing scope by passing in an alias parameter that requires such a scope; not because the error should have occurred the first time as well. Circumstances where some "context" is captured include: - template instantiations where an alias parameter requires a context in order to resolve the parameter correctly [[the instantiated function/struct/class will effectively become nested within whichever scope is determined to be needed in order to resolve the alias parameter -- whether the alias parameter is needed, used, or not]] - template instantiation of functions/structs/classes whose enclosing templates were declared within a function/struct/class [[the template instance's functions, structs, and classes will effectively become nested within the function/struct/class that the template was declared within -- whether the instantiated function/struct/class's original declaration strictly implied a need for an outer function/struct/class or not]] - and --hrmm... it's 6:20 AM; I haven't yet slept for the night; and I'm supposed to continue preparing to save myself from the strongest hurricane ever seen in the Atlantic Ocean in a couple hours, so I think that's where I'll end this partial list. Some assorted details related to some miscellaneous capture situations: - https://dlang.org/spec/template.html#nested-templates - https://dlang.org/spec/template.html#nested_template_limitation - https://dlang.org/spec/template.html#aliasparameters - https://dlang.org/spec/template.html#limitations - https://dlang.org/spec/function.html#closures
Re: Templates, D way
On Tuesday, 5 September 2017 at 22:51:45 UTC, Void-995 wrote: The whole thing is that I don't know either padding nor elements count. Those values are read from file. ... Sorry, I was not attentive enough. Yes, you can't use unions in this case.
Re: Templates, D way
On Tuesday, 5 September 2017 at 21:18:11 UTC, crimaniak wrote:
On Tuesday, 5 September 2017 at 14:55:21 UTC, Void-995 wrote:
Using unions? Count and Offset are different
depending on input data, so the address where they are is
varying depending on which file I've loaded. Or I didn't get
what you meant.
Yes, so you need separate union type for every type of input
data. But these values are known in compile time, so you can do
mixin, creating the union for each type like this:
union MyBinaryStructA
{
ubyte[...] asBytes;
struct asStruct
{
align(1):
ubyte[...] dummy1; // padding before firstList
MyBinarySubStructAForA[...] firstList;
ubyte[...] dummy2; // padding between lists
MyBinarySubStructBForA[...] secondList;
...
}
}
unittest
{
MyBinaryStructA a;
a.asBytes = someBinarySourceForA; // fill from unstructured
source
auto item = a.asStruct.firstList[1]; // access to structure
element
}
The whole thing is that I don't know either padding nor elements
count. Those values are read from file. First you need to read
whole file data and cast those bytes as pointer to file header
structure. Offset of each list is bigger then size of the parent
structure. Length of each list vary from 0 to whatever.
Re: Templates, D way
On Tuesday, 5 September 2017 at 14:55:21 UTC, Void-995 wrote:
Using unions? Count and Offset are different
depending on input data, so the address where they are is
varying depending on which file I've loaded. Or I didn't get
what you meant.
Yes, so you need separate union type for every type of input
data. But these values are known in compile time, so you can do
mixin, creating the union for each type like this:
union MyBinaryStructA
{
ubyte[...] asBytes;
struct asStruct
{
align(1):
ubyte[...] dummy1; // padding before firstList
MyBinarySubStructAForA[...] firstList;
ubyte[...] dummy2; // padding between lists
MyBinarySubStructBForA[...] secondList;
...
}
}
unittest
{
MyBinaryStructA a;
a.asBytes = someBinarySourceForA; // fill from unstructured
source
auto item = a.asStruct.firstList[1]; // access to structure
element
}
Re: Templates, D way
On Tuesday, 5 September 2017 at 14:43:19 UTC, John Colvin wrote: On Tuesday, 5 September 2017 at 12:41:45 UTC, Computermatronic wrote: On Tuesday, 5 September 2017 at 12:20:14 UTC, crimaniak wrote: [...] I find it very strange that this works, as a non-mixin template should not be able to capture the context of where it was instantiated. If you take the alias template parameters out it behaves how it should (that is an error message saying this is not accessible). https://issues.dlang.org/show_bug.cgi?id=17809 What kind of harm can this be? Can this be considered a feature?
Re: Templates, D way
On Tuesday, 5 September 2017 at 14:01:02 UTC, crimaniak wrote: On Tuesday, 5 September 2017 at 12:54:20 UTC, Void-995 wrote: Thanks, that definitely working and doesn't require mixin with strings. But while waiting for response I've tried another thing, and I doubt I would able do to that without string now: ... int %sCount; int %sOffset; Yes, you can make such custom names only with string mixin, as I know. Computermatronic gives a good example how to do it simple way. I can give two additional advises: as I understand offsets as sizes are fixed so think about using unions to access sub structures instead of these auto-generated methods; I have a long history of using tricks like this starting from CHAIN MERGE usage in GW-BASIC to add some functions and for now such things are definitely code smell for me. There is always a way to do the same, but better. Using unions? Count and Offset are different depending on input data, so the address where they are is varying depending on which file I've loaded. Or I didn't get what you meant.
Re: Templates, D way
On Tuesday, 5 September 2017 at 12:41:45 UTC, Computermatronic wrote: On Tuesday, 5 September 2017 at 12:20:14 UTC, crimaniak wrote: [...] I find it very strange that this works, as a non-mixin template should not be able to capture the context of where it was instantiated. If you take the alias template parameters out it behaves how it should (that is an error message saying this is not accessible). https://issues.dlang.org/show_bug.cgi?id=17809
Re: Templates, D way
On Tuesday, 5 September 2017 at 12:54:20 UTC, Void-995 wrote: Thanks, that definitely working and doesn't require mixin with strings. But while waiting for response I've tried another thing, and I doubt I would able do to that without string now: ... int %sCount; int %sOffset; Yes, you can make such custom names only with string mixin, as I know. Computermatronic gives a good example how to do it simple way. I can give two additional advises: as I understand offsets as sizes are fixed so think about using unions to access sub structures instead of these auto-generated methods; I have a long history of using tricks like this starting from CHAIN MERGE usage in GW-BASIC to add some functions and for now such things are definitely code smell for me. There is always a way to do the same, but better.
Re: Templates, D way
On Tuesday, 5 September 2017 at 12:41:45 UTC, Computermatronic wrote: I find it very strange that this works, ... I'm too. :) In any case, it's not a big deal. It's possible to modify this template to transfer `this` as the first parameter and modify its usage accordingly.
Re: Templates, D way
On Tuesday, 5 September 2017 at 12:54:20 UTC, Void-995 wrote:
On Tuesday, 5 September 2017 at 12:20:14 UTC, crimaniak wrote:
On Tuesday, 5 September 2017 at 11:08:57 UTC, Void-995 wrote:
@property mixin(DataList!("firstSublist",
MyBinarySubStructAForA, firstSublistMembersCount,
firstSublistMembersOffset));
I don't think string mixins are required here. It seems just
template is more simple.
T[] getBytesAs(T, alias length, alias offset)()
{
return (cast(T *)(cast(byte *)() + offset))[0 ..
length];
}
struct MyBinaryStructA
{
...
alias firstList = getBytesAs!(MyBinarySubStructAForA,
firstSublistMembersCount, firstSublistMembersOffset);
alias secondList = getBytesAs!(MyBinarySubStructBForA,
secondSublistMembersCount, secondSublistMembersOffset);
}
unittest
{
...
MyBinaryStructA *binaryData = cast(MyBinaryStructA
*)fileData.ptr;
auto a = binaryData.firstList;
}
Thanks, that definitely working and doesn't require mixin with
strings. But while waiting for response I've tried another
thing, and I doubt I would able do to that without string now:
template MyBinaryStructGenericDataList(string listName, alias
Type)
{
import std.string;
const char[] MyBinaryStructGenericDataList = format(q{
int %sCount;
int %sOffset;
@property %s[] %s() const
{
return (cast(%s *)(cast(ubyte *)() + %sOffset))[0 ..
%sCount];
}
}, listName, listName, Type.stringof, listName, Type.stringof,
listName, listName);
}
struct MyBinarySubStructAForA
{
int someIntegerFieldA;
float someFloatFieldA;
}
struct MyBinarySubStructBForA
{
int someIntegerFieldB;
float someFloatFieldB;
}
struct MyBinaryStructA
{
mixin(MyBinaryStructGenericDataList!("firstSublist",
MyBinarySubStructAForA));
mixin(MyBinaryStructGenericDataList!("secondSublist",
MyBinarySubStructBForA));
}
...
MyBinaryStructA *binaryData = cast(MyBinaryStructA
*)fileData.ptr;
...
Yes, there is definately no other way but to use string mixins
here, however if you switch to a mixin template, and put the
string mixin inside of that, you don't have to do text
replacement to use the template parameters.
Example:
mixin template GenericDataList(string name, T)
{
mixin(`
int `~name~`SublistCount;
int `~name~`SublistOffset;
@property auto `~name~`Sublist()
{
return (cast(T*)(cast(byte*) +
`~name~`SublistOffset))[0..`~name~`SublistCount];
}`);
}
struct MyBinarySubStructAForA
{
int someIntegerFieldA;
float someFloatFieldA;
}
struct MyBinarySubStructBForA
{
int someIntegerFieldB;
float someFloatFieldB;
}
struct MyBinaryStructA
{
mixin MyBinaryStructGenericDataList!("first",
MyBinarySubStructAForA);
mixin MyBinaryStructGenericDataList!("second",
MyBinarySubStructBForA);
}
Re: Templates, D way
On Tuesday, 5 September 2017 at 12:20:14 UTC, crimaniak wrote:
On Tuesday, 5 September 2017 at 11:08:57 UTC, Void-995 wrote:
@property mixin(DataList!("firstSublist",
MyBinarySubStructAForA, firstSublistMembersCount,
firstSublistMembersOffset));
I don't think string mixins are required here. It seems just
template is more simple.
T[] getBytesAs(T, alias length, alias offset)()
{
return (cast(T *)(cast(byte *)() + offset))[0 ..
length];
}
struct MyBinaryStructA
{
...
alias firstList = getBytesAs!(MyBinarySubStructAForA,
firstSublistMembersCount, firstSublistMembersOffset);
alias secondList = getBytesAs!(MyBinarySubStructBForA,
secondSublistMembersCount, secondSublistMembersOffset);
}
unittest
{
...
MyBinaryStructA *binaryData = cast(MyBinaryStructA
*)fileData.ptr;
auto a = binaryData.firstList;
}
Thanks, that definitely working and doesn't require mixin with
strings. But while waiting for response I've tried another thing,
and I doubt I would able do to that without string now:
template MyBinaryStructGenericDataList(string listName, alias
Type)
{
import std.string;
const char[] MyBinaryStructGenericDataList = format(q{
int %sCount;
int %sOffset;
@property %s[] %s() const
{
return (cast(%s *)(cast(ubyte *)() + %sOffset))[0 ..
%sCount];
}
}, listName, listName, Type.stringof, listName, Type.stringof,
listName, listName);
}
struct MyBinarySubStructAForA
{
int someIntegerFieldA;
float someFloatFieldA;
}
struct MyBinarySubStructBForA
{
int someIntegerFieldB;
float someFloatFieldB;
}
struct MyBinaryStructA
{
mixin(MyBinaryStructGenericDataList!("firstSublist",
MyBinarySubStructAForA));
mixin(MyBinaryStructGenericDataList!("secondSublist",
MyBinarySubStructBForA));
}
...
MyBinaryStructA *binaryData = cast(MyBinaryStructA *)fileData.ptr;
...
Re: Templates, D way
On Tuesday, 5 September 2017 at 12:20:14 UTC, crimaniak wrote:
On Tuesday, 5 September 2017 at 11:08:57 UTC, Void-995 wrote:
@property mixin(DataList!("firstSublist",
MyBinarySubStructAForA, firstSublistMembersCount,
firstSublistMembersOffset));
I don't think string mixins are required here. It seems just
template is more simple.
T[] getBytesAs(T, alias length, alias offset)()
{
return (cast(T *)(cast(byte *)() + offset))[0 ..
length];
}
struct MyBinaryStructA
{
...
alias firstList = getBytesAs!(MyBinarySubStructAForA,
firstSublistMembersCount, firstSublistMembersOffset);
alias secondList = getBytesAs!(MyBinarySubStructBForA,
secondSublistMembersCount, secondSublistMembersOffset);
}
unittest
{
...
MyBinaryStructA *binaryData = cast(MyBinaryStructA
*)fileData.ptr;
auto a = binaryData.firstList;
}
I find it very strange that this works, as a non-mixin template
should not be able to capture the context of where it was
instantiated. If you take the alias template parameters out it
behaves how it should (that is an error message saying this is
not accessible).
Re: Templates, D way
On Tuesday, 5 September 2017 at 11:08:57 UTC, Void-995 wrote:
@property mixin(DataList!("firstSublist",
MyBinarySubStructAForA, firstSublistMembersCount,
firstSublistMembersOffset));
I don't think string mixins are required here. It seems just
template is more simple.
T[] getBytesAs(T, alias length, alias offset)()
{
return (cast(T *)(cast(byte *)() + offset))[0 .. length];
}
struct MyBinaryStructA
{
...
alias firstList = getBytesAs!(MyBinarySubStructAForA,
firstSublistMembersCount, firstSublistMembersOffset);
alias secondList = getBytesAs!(MyBinarySubStructBForA,
secondSublistMembersCount, secondSublistMembersOffset);
}
unittest
{
...
MyBinaryStructA *binaryData = cast(MyBinaryStructA
*)fileData.ptr;
auto a = binaryData.firstList;
}
Re: Templates, D way
On Tuesday, 5 September 2017 at 11:08:57 UTC, Void-995 wrote:
Hi, everyone. I'm pretty new to D and trying my first steps in
it. Currently I'm trying to port some code from C/C++ with
pretty weird data structures and don't like idea of making
boilerplate functions for accessing sub-lists in main binary
structure (lets not talk about it's design, it's from legacy
thing I want to deal with and I can't change the format itself
or writing a lot of additional parsing code). With hour I spent
on trying I've ended with what you may see below, but I wonder
if there is more pleasant variant of re-writing that template
as I haven't found anything better in either books or online
documentation yet:
template DataList(const char[] listName, T, alias dataOffset,
alias listLength)
{
const char[] DataList = format(q{
%s[] %s()
{
return (cast(%s *)(cast(byte *)() + %s))[0 .. %s];
}
}, T.stringof, listName, T.stringof, dataOffset.stringof,
listLength.stringof);
}
struct MyBinarySubStructAForA
{
int someIntegerFieldA;
float someFloatFieldA;
}
struct MyBinarySubStructBForA
{
int someIntegerFieldB;
float someFloatFieldB;
}
struct MyBinaryStructA
{
int firstSublistMembersCount;
int firstSublistMembersOffset;
int secondSublistMembersCount;
int secondSublistMembersOffset;
@property mixin(DataList!("firstSublist",
MyBinarySubStructAForA, firstSublistMembersCount,
firstSublistMembersOffset));
@property mixin(DataList!("secondSublist",
MyBinarySubStructBForA, secondSublistMembersCount,
secondSublistMembersOffset));
}
...
MyBinaryStructA *binaryData = cast(MyBinaryStructA
*)fileData.ptr;
mixin template DataList(string listName, T, alias dataOffset,
alias listLength)
{
mixin(
q{T[] } ~ listName ~ q{() @property
{
return (cast(T*)(cast(ubyte*) + dataOffset))[0 ..
listLength];
}}
);
}
struct MyBinarySubStructAForA
{
int someIntegerFieldA;
float someFloatFieldA;
}
struct MyBinarySubStructBForA
{
int someIntegerFieldB;
float someFloatFieldB;
}
struct MyBinaryStructA
{
int firstSublistMembersCount;
int firstSublistMembersOffset;
int secondSublistMembersCount;
int secondSublistMembersOffset;
mixin DataList!("firstSublist", MyBinarySubStructAForA,
firstSublistMembersCount, firstSublistMembersOffset);
mixin DataList!("secondSublist", MyBinarySubStructBForA,
secondSublistMembersCount, secondSublistMembersOffset);
}
Templates, D way
Hi, everyone. I'm pretty new to D and trying my first steps in
it. Currently I'm trying to port some code from C/C++ with pretty
weird data structures and don't like idea of making boilerplate
functions for accessing sub-lists in main binary structure (lets
not talk about it's design, it's from legacy thing I want to deal
with and I can't change the format itself or writing a lot of
additional parsing code). With hour I spent on trying I've ended
with what you may see below, but I wonder if there is more
pleasant variant of re-writing that template as I haven't found
anything better in either books or online documentation yet:
template DataList(const char[] listName, T, alias dataOffset,
alias listLength)
{
const char[] DataList = format(q{
%s[] %s()
{
return (cast(%s *)(cast(byte *)() + %s))[0 .. %s];
}
}, T.stringof, listName, T.stringof, dataOffset.stringof,
listLength.stringof);
}
struct MyBinarySubStructAForA
{
int someIntegerFieldA;
float someFloatFieldA;
}
struct MyBinarySubStructBForA
{
int someIntegerFieldB;
float someFloatFieldB;
}
struct MyBinaryStructA
{
int firstSublistMembersCount;
int firstSublistMembersOffset;
int secondSublistMembersCount;
int secondSublistMembersOffset;
@property mixin(DataList!("firstSublist",
MyBinarySubStructAForA, firstSublistMembersCount,
firstSublistMembersOffset));
@property mixin(DataList!("secondSublist",
MyBinarySubStructBForA, secondSublistMembersCount,
secondSublistMembersOffset));
}
...
MyBinaryStructA *binaryData = cast(MyBinaryStructA *)fileData.ptr;
