Re: Fake IFTI-compatible struct constructors
On Saturday, 1 May 2021 at 23:21:33 UTC, Basile B. wrote: On Saturday, 1 May 2021 at 21:57:54 UTC, Chad Joan wrote: ... Rather, this setup doesn't work at all, because IFTI doesn't seem to work on function-templates with alias parameters. Yes your observation is correct. That should work but this is currently not implemented and referenceced as [issue 20877](https://issues.dlang.org/show_bug.cgi?id=20877) Good to know. Thank you!
Fake IFTI-compatible struct constructors
I came up with a couple techniques for making it seem like
templated structs deduce their template parameters from
constructor invocations. These are given further down in the post.
This has been a profitable exercise. However, the techniques I
came up with have some drawbacks.
Thus, I have some questions for the community:
Is there a better way?
Do any of you have other constructor-IFTI-faking techniques to
share?
Maybe one of you has encountered this already and come up
different ways to approach this. Please share if you can!
### Background:
I ended up distracted with trying to make IFTI (Implicit Function
Template Instantiation) work for a struct constructor.
I'm already aware of the normal factory function technique (ex:
have `foo(T)(T x)` construct `struct Foo(T)`) and have looked
into some of the history of this issue, like this thread from
April of 2013:
https://forum.dlang.org/thread/lkyjyhmirazaonbvf...@forum.dlang.org
I updated to DMD v2.096.1 to see if IFTI for constructors had
been implemented recently.
But it seems not, because code like this fails to compile:
```D
struct Foo(StrT)
{
StrT payload;
this(StrT text)
{
this.payload = text;
}
}
void main()
{
import std.stdio;
auto foo = Foo("x");
writeln(foo.payload);
}
```
Compilation results:
```
xfail.d(13): Error: struct `xfail.Foo` cannot deduce function
from argument types `!()(string)`, candidates are:
xfail.d(1):`Foo(StrT)`
```
What followed was a series of experiments to see if I could get
most of what I wanted by (ab)using existing D features.
### Technique 1: Mutually Exclusive Constraints
This technique was the first one I came up with.
It's disadvantage is that it only works for templated types with
parameter lists that meet specific criteria:
* At least one of the parameters must be a string or array type.
* There can't be any other versions of the template that take the
element's type at the same position.
So if I have a templated struct that parameterizes on string
types, then this works for that.
The advantage of this method (over the later method I discovered)
is that it doesn't regress the long or free-standing (non-IFTI)
form of template instantiation in any way that I've noticed. As
long as the above parameter list criteria can be met, it's
strictly an upgrade.
It looks like this:
```D
// Fake IFTI constructor for templates with at least one string
parameter.
auto Foo(Char)(Char[] text)
// The constraint basically just ensures
// that `Char` isn't a string or array.
if ( !is(Char T : T[]) )
{
import std.stdio;
Foo!(Char[]) foo;
foo.payload = text;
pragma(msg, "function Char.stringof == "~Char.stringof);
return foo;
}
struct Foo(StrT)
// This constraint ensures the opposite:
// that `StrT` is a string|array.
// It also declares `CharT` as it's
// element type, but that's quickly discarded.
if ( is(StrT CharT : CharT[]) )
{
StrT payload;
pragma(msg, "struct StrT.stringof == " ~ StrT.stringof);
/+
// These fail to compile.
// Presumably, `CharT` isn't declared /in this scope/.
CharT ch;
pragma(msg, "struct CharT.stringof == " ~ CharT.stringof);
// (It's not a big deal though. It's easy enough
// to get the element type from an array without
// the help of the template's parameter list.)
+/
}
void main()
{
import std.stdio;
// Test IFTI-based instantiation.
auto foo1 = Foo("1"); // IFTI for Foo constructor!
writeln(foo1.payload); // Prints: 1
// Test normal instantiation.
// (In other words: we try to avoid regressing this case.)
Foo!string foo2;
foo2.payload = "2";
writeln(foo2.payload); // Prints: 2
/// Accidental instantiations of the wrong template are
/// prevented by the `!is(Char T : T[])` template constraint.
// Foo!char foo3; // Error: `Foo!char` is used as a type
// foo3.payload = "3";
}
```
### Technique 2: Explicit Instantiation
I wanted a way to do this with templates that don't deal with
strings or arrays, and with templates that might accept either an
array or its element in the same parameter (or parameter
position).
So I did that, but it has a notable drawback: free-standing
instantiations like `Bar!int bar;` won't work without adding
additional template specializations for every type that might
need to be used that way.
This drawback isn't very severe for project internals, but I
think it's no good for library APIs.
I really hope someone has a way to do something like this, but
without this drawback.
Here's what this technique looks like:
```D
// This is like the usual helper function that returns
// an instance of the desired templated type.
// (AKA: a
Re: safety and auto vectorization
On Sunday, 2 August 2020 at 17:31:45 UTC, Bruce Carneal wrote:
import std;
void f0(int[] a, int[] b, int[] dst) @safe {
dst[] = a[] + b[];
}
void f1(int[] a, int[] b, int[] dst) @trusted {
const minLen = min(a.length, b.length, dst.length);
dst[0..minLen] = a[0..minLen] + b[0..minLen];
assert(dst.length == minLen);
}
I was surprised that f0 ran just fine with a.length and
b.length geq dst.length. Is that a bug or a feature?
Assuming it's a feature, are f0 and f1 morally equivalent? I
ask because f1 auto-vectorizes in ldc while f0 does not. Not
sure why. As a guess I'd say that the front end doesn't hoist
bounds checks in f0 or at least doesn't convey the info to the
back end in a comprehensible fashion. Non-guesses welcome.
I don't know what's going on auto-vectorization-wise, but to
address the behavioral issues, the next thing I would do if I
were in your shoes is something like this:
import std.stdio;
int[100] a, b, dst;
a[] = 2;
b[] = 3;
dst[] = 42;
f0(a[0..$], b[0..$], dst[0..50]); // Notice: dst is a smaller
slice.
writefln("dst[49] == %d", dst[49]); // Should be 5.
writefln("dst[50] == %d", dst[50]); // 42 or 5?
On my machine (Linux 64-bit DMD v2.093.0) it prints this:
dst[49] == 5
dst[50] == 42
Which suggests that it is doing the minimum-length calculation,
as the dst[] values outside of the lesser-sized slice were
untouched.
This was DMD, so it's going to be worth trying on your compiler
to see what you get.
Re: D on lm32-CPU: string argument on stack instead of register
On Saturday, 1 August 2020 at 08:58:03 UTC, Michael Reese wrote:
[...]
So the compiler knows how to use r1 and r2 for arguments. I
checked again in the lm32 manual
(https://www.latticesemi.com/view_document?document_id=52077),
and it says:
"As illustrated in Table 3 on page 8, the first eight function
arguments are
passed in registers. Any remaining arguments are passed on the
stack, as
illustrated in Figure 12."
So strings and structs should be passed on the stack and this
seems to be more an issue of the gcc lm32 backend than a D
issue.
Nice find!
Though if the compiler is allowed to split a single uint64_t into
two registers, I would expect it to split struct/string into two
registers as well. At least, the manual doesn't seem to
explicitly mention higher-level constructs like structs. It does
suggest a one-to-one relationship between arguments and registers
(up to a point), but GCC seems to have decided otherwise for
certain uint64_t's. (Looking at Table 3...) It even gives you two
registers for a return value: enough for a string or an array.
And if the backend/ABI weren't up for it, it would be
theoretically possible to have the frontend to lower strings
(dynamic arrays) and small structs into their components before
function calls and then also insert code on the other side to
cast them back into their original form. I'm not sure if anyone
would want to write it, though. o.O
But I just found a workaround using a wrapper function.
void write_to_host(in string msg) {
write_to_hostC(msg.ptr, msg.length);
}
I checked the assembly code on the caller side, and the call
write_to host("Hello, World!\n") is inlined. There is only one
call to write_to_hostC. This is still not nice, but I think I
can live with that for now. Now I have to figure out how make
the cast back from from pointer-length pair into a string. I'm
sure I read that somewhere before, but forgot it and was unable
to find it now on a quick google search...
That's pretty clever. I like it.
Getting from pointer-length to string might be pretty easy:
string foo = ptr[0 .. len];
D allows pointers to be indexed, like in C. But unlike C, D has
slices, and pointers can be "sliced". The result of a slice
operation, at least for primitive arrays+pointers, is always an
array (the "dynamic" ptr+length kind). Hope that helps.
And since this is D: is there maybe some CTFE magic that allows
to create these wrappers automatically? Somthing like
fix_stack!write_to_host("Hello, World!\n");
It ended up being a little more complicated than I thought it
would be. Hope I didn't ruin the fun. ;)
https://pastebin.com/y6e9mxre
Also, that part where you mentioned a 64-bit integer being passed
as a pair of registers made me start to wonder if unions could be
(ab)used to juke the ABI:
https://pastebin.com/eGfZN0SL
Good luck with your lm32/FPGA coding. That sounds like cool
stuff!
I'm doing this mainly to improve my understanding of how
embedded processors work, and how to write linker scripts for a
given environment. Although I do have actual hardware where I
can see if everything runs in the real world, I mainly use
simulations. The coolest thing in my opinion is, nowadays it
can be done using only open source tools (mainly ghdl and
verilator, the lm32 source code is open source, too). The
complete system is simulated and you can look at every logic
signal in the cpu or in the ram while the program executes.
Thanks for the insights; I've done just a little hobby electrical
stuff here-and-there, and having some frame of reference for tool
and component choice makes me feel good, even if I don't plan on
buying any lm32s or FPGAs anytime soon :) Maybe I can Google
some of that later and geek out at images of other people's
debugging sessions or something. I'm curious how they manage the
complexity that happens when circuits and massive swarms of logic
gates do their, uh, complexity thing. o.O
Re: D on lm32-CPU: string argument on stack instead of register
On Friday, 31 July 2020 at 10:22:20 UTC, Michael Reese wrote:
Hi all,
at work we put embedded lm32 soft-core CPUs in FPGAs and write
the firmware in C.
At home I enjoy writing small projects in D from time to time,
but I don't consider myself a D expert.
Now, I'm trying to run some toy examples in D on the lm32 cpu.
I'm using a recent gcc-elf-lm32. I succeeded in compiling and
running some code and it works fine.
But I noticed, when calling a function with a string argument,
the string is not stored in registers, but on the stack.
Consider a simple function (below) that writes bytes to a
peripheral (that forwards the data to the host computer via
USB). I've two versions, an ideomatic D one, and another
version where pointer and length are two distinct function
parameters.
I also show the generated assembly code. The string version is
4 instructions longer, just because of the stack manipulation.
In addition, it is also slower because it need to access the
ram, and it needs more stack space.
My question: Is there a way I can tell the D compiler to use
registers instead of stack for string arguments, or any other
trick to reduce code size while maintaining an ideomatic D
codestyle?
Best regards
Michael
// ideomatic D version
void write_to_host(in string msg) {
// a fixed address to get bytes to the host via usb
char *usb_slave = cast(char*)BaseAdr.ft232_slave;
foreach(ch; msg) {
*usb_slave = ch;
}
}
// resulting assembly code (compiled with -Os) 12 instructions
_D10firmware_d13write_to_hostFxAyaZv:
addi sp, sp, -8
addi r3, r0, 4096
sw (sp+4), r1
sw (sp+8), r2
add r1, r2, r1
.L3:
be r2,r1,.L1
lbu r4, (r2+0)
addi r2, r2, 1
sb (r3+0), r4
bi .L3
.L1:
addi sp, sp, 8
bra
// C-like version
void write_to_hostC(const char *msg, int len) {
char *ptr = cast(char*)msg;
char *usb_slave = cast(char*)BaseAdr.ft232_slave;
while (len--) {
*usb_slave = *ptr++;
}
}
// resulting assembly code (compiled with -Os) 8 instructions
_D10firmware_d14write_to_hostCFxPaiZv:
add r2, r1, r2
addi r3, r0, 4096
.L7:
be r1,r2,.L5
lbu r4, (r1+0)
addi r1, r1, 1
sb (r3+0), r4
bi .L7
.L5:
bra
Hi Michael!
Last time I checked, D doesn't have any specific type attributes
or special ways to force variables to enregister. But I could be
poorly informed. Maybe there are GDC-specific hints or something.
I hope that if anyone else knows better, they will toss in an
answer.
THAT SAID, I think there are things to try and I hope we can get
you what you want.
If you're willing to entertain more experimentation, here are my
thoughts:
---
(1) Try writing "in string" as "in const(char)[]" instead:
// ideomatic D version
void write_to_host(in const(char)[] msg) {
// a fixed address to get bytes to the host via usb
char *usb_slave = cast(char*)BaseAdr.ft232_slave;
foreach(ch; msg) {
*usb_slave = ch;
}
}
Explanation:
The "string" type is an alias for "immutable(char)[]".
In D, "immutable" is a stronger guarantee than "const". The
"const" modifier, like in C, tells the compiler that this
function shall not modify the data referenced by this
pointer/array/whatever. The "immutable" modifier is a bit
different, as it says that NO ONE will modify the data referenced
by this pointer/array/whatever, including other functions that
may or may not be concurrently executing alongside the one you're
in. So "const" constraints the callee, while "immutable"
constrains both the callee AND the caller. This makes it more
useful for some multithreaded code, because if you can accept the
potential inefficiency of needing to do more copying of data (if
you can't modify, usually you must copy instead), then you can
have more deterministic behavior and sometimes even much better
total efficiency by way of parallelization. This might not be a
guarantee you care about though, at which point you can just toss
it out completely and see if the compiler generates better code
now that it sees the same type qualifier as in the other example.
I'd actually be surprised if using "immutable" causes /less/
efficient code in this case, because it should be even /safer/ to
use the argument as-is. But it IS a difference between the two
examples, and one that might not be benefiting your cause (though
that's totally up to you).
---
(2) Try keeping the string argument, but make the function more
closely identical in semantics:
// ideomatic D version
void write_to_host(string msg) {
// a fixed address to get bytes to the host via usb
char
Re: Why does stringof not like functions with arguments?
On Thursday, 10 August 2017 at 14:51:22 UTC, Meta wrote:
On Wednesday, 9 August 2017 at 01:39:07 UTC, Jason Brady wrote:
Why does the following code error out with:
app.d(12,10): Error: function app.FunctionWithArguments (uint
i) is not callable using argument types ()
Code:
import std.stdio;
void FunctionWithoutArguments() {
}
void FunctionWithArguments(uint i) {
}
void main()
{
writeln(FunctionWithoutArguments.stringof);
writeln(FunctionWithArguments.stringof);
}
Welcome to optional parentheses hell. Please enjoy your stay.
[...]
Muahahaha it's necromancy time!
... meaning I just ran into this problem. Again. And it sucked.
And I found this thread. Again. Now it's time for me to be less
of a dummy and post my solution.
This seems to have different solutions depending on what you want
out of the function symbol. The advice already given in this
thread is great if you want to print the function's name (and
maybe a couple other things I already forgot).
But what I needed was to print the function's *signature*.
Basically, I want to
writeln(FunctionWithArguments.stringof);
and get this output:
void FunctionWithArguments(uint i)
I didn't quite get there. I got this far:
void(uint i)
But for what I'm doing right now, that's good enough.
Alright here's how it's done:
writeln(typeof(FunctionWithArguments).stringof);
So it was ultimately really easy. At least, for this one very
specific use-case. I just about kicked myself.
The previous example then becomes this:
import std.stdio;
void FunctionWithoutArguments() {
}
void FunctionWithArguments(uint i) {
}
void main()
{
writeln(typeof(FunctionWithoutArguments).stringof);
writeln(typeof(FunctionWithArguments).stringof);
}
I needed this when writing a program that checks for whether
functions visible from alias-this statements are included in the
results of __traits(getOverloads,...).
Here is the end result:
https://pastebin.com/yj3idDhp
And no. No they are not. :3
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 09:58:25 UTC, Jonathan M Davis wrote: For two types to be compared, they must be the the same type - or they must be implicitly convertible to the same type, in which case, they're converted to that type and then compared. So, as far as D's design of comparison goes, there is no need worry about comparing differing types. At most, you need to worry about what implicit type conversions exist, and D isn't big on implicit type conversions, because they tend to cause subtle bugs. So, while they definitely affect comparison, they don't affect anywhere near as much as they would in a language like C++. In general, you're not going to get very far if you're trying to make it possible to compare a user-defined type against other types in D without explicitly converting it first. - Jonathan M Davis That makes sense, but requiring types to be explicitly converted before comparisons kinda throws sand on the cake when I'm ostensibly trying to make things that interact seamlessly with existing types. "alias this" is still awesome, so it's usually fine regardless :) Thanks for the explanation.
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 13:23:15 UTC, Adam D. Ruppe wrote: On Thursday, 27 September 2018 at 05:04:09 UTC, Chad Joan wrote: As above, I think this might be a very clean and effective solution for a different class of use-cases :) I'll keep it in mind though. Yeah. And I did make one mistake: the tupleof assignment trick wouldn't work well for references, so lol it isn't much of a deep copy. You'd want to deep copy any arrays too probably. But sounds like you are already doing that, yay. You're right though, if I end up adding boilerplate anyways, I may as well have a good shallow copy to begin with.
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 14:23:48 UTC, Steven
Schveighoffer wrote:
On 9/27/18 10:20 AM, Steven Schveighoffer wrote:
typeid sometimes gives you a more derived type than TypeInfo.
Including for classes and structs.
In the past, .classinfo gave you a different thing than
typeid(obj), but now it is the same thing:
auto obj = new Object;
// classinfo and typeid are the same object
assert(obj.classinfo is typeid(obj));
// and the same type
static assert(is(typeof(obj.classinfo) ==
typeof(typeid(obj;
I wouldn't use classinfo any more, I generally use typeid.
I should add that typeid does give you the derived
TypeInfo_Class, not the concrete one.
that is:
Object obj; // = null
//typeid(obj); // segfault, can't dereference null pointer
class C {}
obj = new C;
static assert(is(typeof(obj) == Object));
writeln(typeid(obj)); // C, not Object
So it really is a drop-in replacement for classinfo. I think
classinfo is still there to avoid breaking existing code that
uses it.
-Steve
Interesting! That's yet another thing I hadn't realized had
changed. Good to know.
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 13:16:58 UTC, Adam D. Ruppe wrote: On Thursday, 27 September 2018 at 05:18:14 UTC, Chad Joan wrote: How does this work? The language reference states that typeid(Type) returns "an instance of class TypeInfo corresponding to Type". (https://dlang.org/spec/expression.html#typeid_expressions) But then the TypeInfo class doesn't seem to have a .create() method, or at least not one in the documentation: https://dlang.org/phobos/object.html#.TypeInfo I forgot about the create method, lol, that is what you want. But typeid(obj) - pass it an existing object of the type btw - when obj is a class will return TypeInfo_Class - a subclass of TypeInfo. It is *that* which has the create method. https://dlang.org/phobos/object.html#.TypeInfo_Class.create (or less horrible docs http://dpldocs.info/experimental-docs/object.TypeInfo_Class.html ) Yep, that solves the mystery! Thanks!
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 18:37:27 UTC, Chad Joan wrote: I will probably end up going with the latter suggestion and have Extended use the Root's T. That would probably make sense for what I'm doing. In my case, the T allows the caller to configure what kind of output the thing provides... IIRC (it's been a while since I touched this code o.O). Thanks for pointing that out. Wait, actually the T is the element type of the input range, like "char" if the parser is to parse UTF8 strings. I'd already figured that out too. (*ﾟｰﾟ)ゞ
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 08:56:22 UTC, Alex wrote:
On Wednesday, 26 September 2018 at 20:41:38 UTC, Chad Joan
wrote:
class Root(T)
{
T x;
}
class Extended(T) : Root!T
{
T y;
}
Sorry for a technical aside, but would this be something for
you?
https://forum.dlang.org/post/vtaxcxpufrovwfrkb...@forum.dlang.org
I mean... In either case, there is something curious in the
Extended/Root usage, as they both are bound to the same type.
And if so, you could get rid of the templatization in the
Extended class, either by templating Root on the Extended or
use the T of Root in Extended.
Perhaps it's half some form of unintended neural network
fabrication that happened as I wrote the example and half that
the original code isn't that well thought out yet. The original
code looks more like this:
template Nodes(T)
{
class Root
{
T x;
}
class Extended : Root
{
T y;
}
}
I will probably end up going with the latter suggestion and have
Extended use the Root's T. That would probably make sense for
what I'm doing. In my case, the T allows the caller to configure
what kind of output the thing provides... IIRC (it's been a while
since I touched this code o.O).
Thanks for pointing that out.
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 08:19:41 UTC, Jonathan M Davis
wrote:
On Thursday, September 27, 2018 1:41:23 AM MDT Chad Joan via
Digitalmars-d- learn wrote:
On Thursday, 27 September 2018 at 05:12:06 UTC, Jonathan M
Davis
This is also reminding me of how it's always bugged me that
there isn't a way to operator overload opEquals with a static
method (or even a free function?), given that it would allow
the class/struct implementer to guard against (or even
interact intelligently with) null values:
That's not really an issue with D. With classes, when you have
a == b, it
doesn't lower to a.opEquals(b). Rather, it lowers to
opEquals(a, b), and the
free function opEquals is defined as
bool opEquals(Object lhs, Object rhs)
{
// If aliased to the same object or both null => equal
if (lhs is rhs) return true;
// If either is null => non-equal
if (lhs is null || rhs is null) return false;
// If same exact type => one call to method opEquals
if (typeid(lhs) is typeid(rhs) ||
!__ctfe && typeid(lhs).opEquals(typeid(rhs)))
/* CTFE doesn't like typeid much. 'is' works, but
opEquals
doesn't
(issue 7147). But CTFE also guarantees that equal
TypeInfos are
always identical. So, no opEquals needed during
CTFE. */
{
return lhs.opEquals(rhs);
}
// General case => symmetric calls to method opEquals
return lhs.opEquals(rhs) && rhs.opEquals(lhs);
}
/
* Returns true if lhs and rhs are equal.
*/
bool opEquals(const Object lhs, const Object rhs)
{
// A hack for the moment.
return opEquals(cast()lhs, cast()rhs);
}
So, it already takes care of checking for null or even if the
two references point to the same object. For structs, a == b,
does lower to a.opEquals(b), but for better or worse, structs
are designed so that their init values need to be valid, or
you're going to have problems in general. Trying to work around
that is fighting a losing battle.
The spec seems to have the homogeneous cases covered: classes
with classes or structs with structs. What I'm more worried
about is stuff like when you have a class compared to a struct or
builtin type, or maybe a struct compared to a builtin type
(especially more complicated builtin types like arrays!). The
homogeneous cases are important for making a type consistent with
itself, but the other cases are important for integrating a type
with everything else in the ecosystem.
Notably, "alias this" is awesome and has more or less solved that
for me in the pedestrian cases I tend to encounter. I can write
a struct and alias this to some reference variable that will be
representative of my struct's "nullness" or other states of
existence.
But I wouldn't be surprised if there are corner-cases I haven't
encountered yet (actually I think I just remembered that this bit
me a little bit once or twice) where having a single alias-this
isn't sufficient to cover all of the possible things my
struct/class could be compared to (ex: if the type's null-state
corresponded to int.max for ints and float.nan for floats, and
you can't just use opEquals, such as when the type is a class and
could be precisely null).
Wouldn't it be helpful to have a root class type just to have
a "Top" type at runtime, even if it had no members? Ex: so
you could do things like make an array ProtoObject[] foo; that
can contain any runtime polymorphic variables.
Maybe? It's not something that I've personally found to be
particularly
useful. Once you can templatize code, the need to have a common
base class
gets pretty hard to argue for, but I don't know that it's
non-existent.
Also, for better or worse, you can already get it with void* -
and cover
more types no less (albeit less @safely). But from what I
understand of what
Andrei is intending, ProtoObject will end up being the new root
class for
all D classos, so having ProtoObject[] would work for all
extern(D) classes.
Of course, that still potentially leaves exern(C++) classes and
interfaces
(which could be extern(C++) or COM even right now and aren't
derived from
Object). So, things are already a bit weird with classes when
you start
interacting with other languages through D. is(T : Object) and
is(T == class) do _not_ mean quite the same thing even though
you'd think
that they would. And I haven't done enough with extern(C++) or
COM in D to
claim to understand all of the subtleties. If you're not
messing with them
directly or writing generic code that's going to mess with
them, it doesn't
really matter though.
-Jonathan M Davis
Gotcha. Quite a rabbit hole :)
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Thursday, 27 September 2018 at 05:12:06 UTC, Jonathan M Davis
wrote:
On Wednesday, September 26, 2018 10:20:58 PM MDT Chad Joan via
Digitalmars- d-learn wrote:
...
That's interesting! Thanks for mentioning.
If you don't mind, what are the complaints regarding Object?
Or can you link me to discussions/issues/documents that point
out the shortcomings/pitfalls?
I've probably run into a bunch of them, but I realize D has
come a long way since that original design and I wouldn't be
surprised if there's a lot more for me to learn here.
I can point you to the related DIP, though it's a WIP in
progress
https://github.com/andralex/DIPs/blob/ProtoObject/DIPs/DIP.md
There are also these enhancement requests for removing the
various member functions from Object (though they're likely to
be superceded by the DIP):
https://issues.dlang.org/show_bug.cgi?id=9769
https://issues.dlang.org/show_bug.cgi?id=9770
https://issues.dlang.org/show_bug.cgi?id=9771
https://issues.dlang.org/show_bug.cgi?id=9772
Basically, the problems tend to come in two areas:
1. Because of how inheritance works, once you have a function
on a class, you're forcing a certain set of attributes on that
function - be it type qualifiers like const or shared or scope
classes like pure or @safe. In some cases, derived classes can
be more restricted when they override the function (e.g. an
overide can be @safe when the original is @system), but that
only goes so far, and when you use the base class API, you're
stuck with whatever attributes it has. Regardless, derived
classes can't be _less_ restrictive. In fact, the only reason
that it's currently possible to use == with const class
references in D right now is because of a hack. The free
function opEquals that gets called when you use == on two class
references actually casts away const so that it can then call
the member function opEquals (which doesn't work with const).
So, if the member function opEquals mutates the object, you
actuall get undefined behavior. And because Object.opEquals
defines both the parameter and invisible this parameter as
mutable, derived classes have to do the same when they override
it; otherwise, they'd be overloading it rather than overriding
it.
You're right, I wouldn't be caught dead wearing that.
:)
But yeah, thanks for pointing that out. Now I know not to mutate
things in an opEquals, even if it makes sense from the class's
point of view, just in case. At least until this all gets sorted
out and code gets updated to not inherit from Object.
Object and its member functions really come from D1 and predate
all of the various attributes in D2 - including const. But even
if we could just add all of the attributes that we thought
should be there without worrying about breaking existing code,
there would be no right answer. For instance, while in the vast
majority of cases, opEquals really should be const, having it
be const does not work with types that lazily initialize some
members (since unlike in C++, D does not have backdoors for
const - when something is const, it really means const, and
it's undefined behavior to cast away const and mutate the
object). So, having Object.opEquals be const might work in 99%
of cases, but it wouldn't work in all. The same could be said
for other attributes such as pure or nothrow. Forcing a
particular set of attributes on these functions on everyone is
detrimental. And honestly, it really isn't necessary.
Having them on Object comes from a Java-esque design where you
don't have templates. With proper templates like D2 has, there
normally isn't a reason to operate on an Object. You templatize
the code rather than relying on a common base class. So,
there's no need to have Object.toString in order have toString
for all classes or Object.opEquals to have opEquals for all
classes. Each class can define it however it sees fit. Now,
once a particular class in a hierarchy has defined a function
like opEquals or toString, that affects any classes derived
from it, but then only the classes derived from it are
restricted by those choices, not every single class in the
entire language as has been the case with Object.
That makes sense. Also, compile-time inheritance/duck-typing
FTW, again.
This is also reminding me of how it's always bugged me that there
isn't a way to operator overload opEquals with a static method
(or even a free function?), given that it would allow the
class/struct implementer to guard against (or even interact
intelligently with) null values:
import std.stdio;
class A
{
int payload;
bool opEquals(int rhs)
{
if ( rhs == int.max )
return false;
else
return this.payload == rhs;
}
}
class B
{
int payload;
static bool opEquals(B lhs, int rhs)
{
if ( lhs is null &&
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Wednesday, 26 September 2018 at 21:25:07 UTC, Steven Schveighoffer wrote: ... Object.factory is a really old poorly supported type of reflection. I would not depend on it for anything. Roger that. Will avoid :) You are better off using your own registration system. As far as choosing the design for your problem, you can use: auto obj = typeid(obj).create(); which is going to work better, and doesn't require a linear search through all modules/classes like Object.factory. How does this work? The language reference states that typeid(Type) returns "an instance of class TypeInfo corresponding to Type". (https://dlang.org/spec/expression.html#typeid_expressions) But then the TypeInfo class doesn't seem to have a .create() method, or at least not one in the documentation: https://dlang.org/phobos/object.html#.TypeInfo It looks like what I want, so it might be very helpful. If it were me, I'd probably instead implement a clone virtual method. This way if any custom things need to occur when copying the data, it can be handled. -Steve I'm thinking I want to avoid the virtual method in this case, for reasons I wrote about in my response to Adam (https://forum.dlang.org/post/zovficijurwhuurrr...@forum.dlang.org). But I think it's probably a good suggestion in most cases; I suspect that most of the time wanting to write a "deepCopy" method is going to be in response to some problem that will respond well to just virtualizing the method. Thanks for the advice!
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Wednesday, 26 September 2018 at 21:24:07 UTC, Adam D. Ruppe
wrote:
On Wednesday, 26 September 2018 at 20:41:38 UTC, Chad Joan
wrote:
I'm implementing a deep-copy method for a tree of templated
class instances. As part of this, I need some way to copy
each node.
[...]
that isn't already handled by their deepCopy method.
I would strongly suggest just using that virtual method and
having the child classes override it, then you call it from any
of them and get the right result.
The tree nodes are potentially very diverse, but the tree
structure itself will be very homogeneous. I'm writing a parser
generator backend and the tree is expressions of various
operators (Sequence, OrderedChoice, UnorderedChoice, Repetition,
etc). I'm trying to keep it simple: everything is an expression,
and expressions can contain other expressions (though some are
always leaves in the tree). At some level, if I let things
implement their own deepCopy, then it means there are potentially
other classes and state out there to iterate that the rest of the
code doesn't know about. That could be bad, and expressions
shouldn't contain anything besides expressions!
This probably contrasts a lot with other use-cases, like
serialization. And I wouldn't be surprised if things change
later on and I end up with some kind of auxiliary virtual copy
function that does what you suggest, but is specifically for
handling special out-of-band mutable reference data that the
expressions might need to carry someday.
I suppose I've never considered just how hard/impossible it is to
have a generic way to copy things. Well, maybe a little bit at
various points, but not this bad ;) There are so many dimensions
to the problem and it seems like the context and requirements
will always be really important. So it can be made simple under
the right constraints (ex: everything is immutable!), but the
constraints are always different depending on the requirements
(ex: ... but in this hypothetical, we need mutability, so then
it's gotta happen a different way).
Object.factory kinda sux and I'd actually like to remove it
(among other people). There's no plan to actually do that, but
still, just on principle I want to turn people away.
But even as you can see, the implementation is lacking and it
isn't great design anyway - the interface with virtual methods
does better work. It also wouldn't work in ctfe anyway,
object.factory relies on runtime stuff.
Good to know!
I don't think I've even used it much, if at all. I suppose I
won't miss it if it goes ;)
If Object.factory is incapable of this, is there some other
CTFE-friendly way to copy templated class instances?
I think you can copy typeinfo().init and then call
typeinfo().defaultConstructor - this is iirc what
Object.factory does, but once you already have the typeinfo you
can use it directly and bypass the string lookup.
I'm having trouble looking this up. Could you link me to the
docs for this?
But you'd really be better off with a virtual copy method. I
say those string factory things should only be used if you are
starting with a string, like deserialization.
interface Copyable {
Copyable copy();
}
class Whatever(T) : Copyable {
Whatever!T copy() {
auto c = new Whatever!T();
c.tupleof = this.tupleof;
return c;
}
}
that kind of method. the template implements the interface so
little boilerplate and it works and can be customized etc and
fits in well. If you call it from the interface, you get an
instance of the interface (tho note since it is virtual, the
underlying type is still what you need). If you call from the
child static type, you get it right back. Yay, fits liskov and
works!
As above, I think this might be a very clean and effective
solution for a different class of use-cases :) I'll keep it in
mind though.
If I have to, I can probably make these things register
themselves in some list of delegates that can be used to
instantiate the correct class. Or something like that. But I
am hoping that there is a better way that involves less
boilerplate.
that's not a terrible idea if you need delegates keyed to
strings...
Right. At some level I just need a function that I can call like
this:
auto newThing = makeAnother(originalThing);
and perhaps makeAnother(...) can just lookup originalThing's
classname in an associative array of delegates. Or maybe I can
just hash some part of originalThing's type information. I can
put all of the ugly registration boilerplate into a mixin
template and somehow force that to always be mixed-into any
descendant classes. OK, it's probably getting too far into the
weeds now, but it seems doable and I'll reach for that if I need
to.
...
Things at least seem much more clear already. Thanks a bunch!
Re: Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
On Wednesday, 26 September 2018 at 23:32:36 UTC, Jonathan M Davis wrote: On Wednesday, September 26, 2018 3:24:07 PM MDT Adam D. Ruppe via Digitalmars-d-learn wrote: Object.factory kinda sux and I'd actually like to remove it (among other people). There's no plan to actually do that, but still, just on principle I want to turn people away. While there may not currently be plans to be remove it, as there _are_ plans to add ProtoObject as the new root class underneath Object, at some point here, it's likely that a large percentage of classes won't have anything to do with Object, so relying on Object.factory to be able to construct class Objects in general isn't likely to be a viable path in the long term - though presumably it would work for a code base that's written specifically with it in mind. Personally, I'm hoping that we eventually get to the point where Walter and Andrei are willing to outright deprecate Object itself, but I expect that ProtoObject will have to have been in use for a while before we have any chance of that happening. Either way, I think that it's clear that most code bases should go with a solution other than Object.factory if at all reasonably possible. - Jonathan M Davis That's interesting! Thanks for mentioning. If you don't mind, what are the complaints regarding Object? Or can you link me to discussions/issues/documents that point out the shortcomings/pitfalls? I've probably run into a bunch of them, but I realize D has come a long way since that original design and I wouldn't be surprised if there's a lot more for me to learn here.
Is there a way to use Object.factory with templated classes? Or some way to construct templated classes given RTTI of an instance?
Hi all,
I'm implementing a deep-copy method for a tree of templated class
instances. As part of this, I need some way to copy each node.
I want to avoid code that does things like casting objects into
byte arrays and then copying raw bytes; I want all operations to
be memory safe things that I can use at compile-time. So I
planned to make all of these have default constructors and use
Object.factory to at least create the correct instance type at
the destination. The classes can implement auxiliary copy
methods if they need to copy anything that isn't already handled
by their deepCopy method.
But I ran into a problem: Object.factory doesn't seem to be
compatible with templated classes.
Here is an example:
import std.stdio;
class Root(T)
{
T x;
}
class Extended(T) : Root!T
{
T y;
}
void main()
{
Root!int foo = new Extended!int();
auto name = foo.classinfo.name;
writefln("foo's name is '%s'", name);
// foo's name is 'main.Extended!int.Extended'
Object obj = Object.factory(name);
writefln("Is obj null? %s", obj is null);
Root!int bar = cast(Root!int)obj; // Still going to be null.
writefln("Is bar null? %s", obj is null);
//bar.x = 3; // crash!
}
I had a look at Object.factory. It seems very simple. Perhaps
too simple. I think this might be a dead end. Have I missed
something? Can it actually handle templates somehow, but I just
don't know how to calculate the correct string to hand it?
If Object.factory is incapable of this, is there some other
CTFE-friendly way to copy templated class instances?
If I have to, I can probably make these things register
themselves in some list of delegates that can be used to
instantiate the correct class. Or something like that. But I am
hoping that there is a better way that involves less boilerplate.
Thanks!
- Chad
Re: Best way to manage non-memory resources in current D, ex: database handles.
Awesome, thank you! On Thursday, 9 March 2017 at 00:47:48 UTC, Adam D. Ruppe wrote: Now, if you forget to scope(exit), it is OK, the garbage collector WILL get around to it eventually, and it is legal to work with C handles and functions from a destructor. It is only illegal to call D's garbage collector's functions or to reference memory managed by D's GC inside the destructor. C pointers are fine. It's good to have this confirmed. I'm always a bit trepidatious around destructors. Oooh, and it looks like there is more information in the language spec about @disable on struct constructors and postblits now (compared to the previous time I tried to learn about that feature). So between that and your example, I think I have a feel for how to use that. Thanks. Have a wonderful day!
How do I get names of regex captures during iteration? Populate AAs with captures?
Is there a way to get the name of a named capture when iterating
over captures from a regular expression match? I've looked at
the std.regex code and it seems like "no" to my eyes, but I
wonder if others here have... a way.
My original problem is this: I need to populate an associative
array (AA) with all named captures that successfully matched
during a regex match (and none of the captures that failed). I
was wondering what the best way to do this might be.
Thanks!
Please see comments in the below program for details and my
current progress:
void main()
{
import std.compiler;
import std.regex;
import std.range;
import std.stdio;
writefln("Compiler name:%s", std.compiler.name);
writefln("Compiler version: %s.%s", version_major,
version_minor);
writeln("");
enum pattern = `(?P\w+)\s*=\s*(?P\d+)?;`;
writefln("Regular expression: `%s`", pattern);
writeln("");
auto re = regex(pattern);
auto c = matchFirst("a = 42;", re);
reportCaptures(re, c);
c = matchFirst("a = ;", re);
reportCaptures(re, c);
}
void reportCaptures(Regex, RegexCaptures)(Regex re, RegexCaptures
captures)
{
import std.range;
import std.regex;
import std.stdio;
writefln("Captures from matched string '%s'",
captures[0]);
string[string] captureList;
// I am trying to read the captures from a regular
expression match
// into the above AA.
//
// ...
//
// This kind of works, but requires a string lookup for
each capture
// and using it in practice relies on undocumented
behavior regarding
// the return value of std.regex.Capture's
opIndex[string] method
// when the string index is a valid named capture that
was not actually
// captured during the match (ex: the named capture was
qualified with
// the ? operator or the * operator in the regex and
never appeared in
// the matched string).
foreach( captureName; re.namedCaptures )
{
auto capture = captures[captureName];
if ( capture is null )
writefln(" captures[%s] is null",
captureName);
else if ( capture.empty )
writefln(" captures[%s] is empty",
captureName);
else
{
writefln(" captures[%s] is '%s'",
captureName, capture);
captureList[captureName] = capture;
}
}
writefln("Total captures: %s", captureList);
/+
// I really want to do something like this, instead:
foreach( capture; captures )
captureList[capture.name] = capture.value;
// And, in reality, it might need to be more like this:
foreach( capture; captures )
foreach ( valueIndex, value; capture.values )
captureList[format("%s-%s",capture.name,valueIndex)] = value;
// Because, logically, named captures qualified with the
// *, +, or {} operators in regular expressions may
capture
// multiple slices.
writefln("Total captures: %s", captureList);
+/
writeln("");
}
//Output, DMD64 D Compiler v2.073.1:
//---
//
//Compiler name:Digital Mars D
//Compiler version: 2.73
//
//Regular expression: `(?P\w+)\s*=\s*(?P\d+)?;`
//
//Captures from matched string 'a = 42;'
// captures[value] is '42'
// captures[var] is 'a'
//Total captures: ["value":"42", "var":"a"]
//
//Captures from matched string 'a = ;'
// captures[value] is empty
// captures[var] is 'a'
//Total captures: ["var":"a"]
Re: How do I use CTFE to generate an immutable associative array at compile time?
On Tuesday, 21 February 2017 at 23:30:52 UTC, Chad Joan wrote:
On Tuesday, 21 February 2017 at 22:43:15 UTC, H. S. Teoh wrote:
Parsing strings at program startup is ugly and slow. What
about parsing at compile-time with CTFE into an array literal
and transforming that into an AA at startup, which should be a
lot faster?
// Warning: untested code
pure string[2][] parseTwoColumnCsv(string inputCsv)
{
import std.csv;
import std.typecons;
string[2][] result;
foreach (record;
csvReader!(Tuple!(string,string))(inputCsv)) {
result ~= [record[0], record[1]];
}
return result;
}
immutable string[string] dataLookup;
static this()
{
enum halfCookedData =
parseTwoColumnCsv(import("some_data.csv"));
foreach (p; halfCookedData) {
dataLookup[p[0]] = p[1];
}
}
T
Hi,
That makes a lot of sense and it had crossed my mind.
In my case the data sets are super small, so I'm probably going
to be lazy/productive and leave it the way it is.
Anyone else from the internet copying these examples: try to
just do it H.S. Teoh's way from the start ;)
Thanks for the suggestion.
- Chad
OK I couldn't help it:
---
pure private string[][] parseTwoColumnCsv(string inputCsv)
{
import std.csv;
import std.typecons;
string[][] result;
foreach ( record; csvReader!(Tuple!(string,string))(inputCsv) )
result ~= [record[0],record[1]];
return result;
}
pure private string[string] twoColumnArrayToAA(const string[][]
arr)
{
string[string] result;
foreach ( pair; arr )
result[pair[0]] = pair[1];
return result;
}
pure private string[string] importTwoColumnCsv(string csvFile)()
{
// Force the parse to happen at compile time.
immutable string[][] tempArray =
import(csvFile).parseTwoColumnCsv();
// Convert the parsed array into a runtime Associative Array and
return it.
return tempArray.twoColumnArrayToAA();
}
immutable string[string] dataLookup;
immutable string[string] dataLookup1;
immutable string[string] dataLookup2;
static this()
{
import std.range;
dataLookup1 = importTwoColumnCsv!"some_data1.csv";
dataLookup2 = importTwoColumnCsv!"some_data2.csv";
foreach( pair; chain(dataLookup1.byKeyValue,
dataLookup2.byKeyValue) )
dataLookup[pair.key] = pair.value;
}
void main()
{
import std.stdio;
writefln("dataLookup = %s", dataLookup);
}
---
This example also shows joining associative arrays.
Re: How do I use CTFE to generate an immutable associative array at compile time?
On Tuesday, 21 February 2017 at 22:43:15 UTC, H. S. Teoh wrote:
Parsing strings at program startup is ugly and slow. What
about parsing at compile-time with CTFE into an array literal
and transforming that into an AA at startup, which should be a
lot faster?
// Warning: untested code
pure string[2][] parseTwoColumnCsv(string inputCsv)
{
import std.csv;
import std.typecons;
string[2][] result;
foreach (record; csvReader!(Tuple!(string,string))(inputCsv))
{
result ~= [record[0], record[1]];
}
return result;
}
immutable string[string] dataLookup;
static this()
{
enum halfCookedData =
parseTwoColumnCsv(import("some_data.csv"));
foreach (p; halfCookedData) {
dataLookup[p[0]] = p[1];
}
}
T
Hi,
That makes a lot of sense and it had crossed my mind.
In my case the data sets are super small, so I'm probably going
to be lazy/productive and leave it the way it is.
Anyone else from the internet copying these examples: try to just
do it H.S. Teoh's way from the start ;)
Thanks for the suggestion.
- Chad
Re: How do I use CTFE to generate an immutable associative array at compile time?
On Tuesday, 21 February 2017 at 22:26:01 UTC, Chad Joan wrote:
On Tuesday, 21 February 2017 at 21:58:07 UTC, Stefan Koch wrote:
On Tuesday, 21 February 2017 at 21:53:23 UTC, Chad Joan wrote:
Hello all,
I'm trying to make this work:
[...]
You cannot create AA's at ctfe and carry them over to runtime
use.
You'd have to use a costum dictionary-type.
I think the vibe.d project has one you could use.
I wondered if this might be the case.
Well, I won't worry about it then. I'll have to find another
way around.
Thanks a bunch!
For the sake of the rest of the internet, here is what I'm
probably going to stick with:
---
pure string[string] parseTwoColumnCsv(string inputCsv)
{
import std.csv;
import std.typecons;
string[string] result;
foreach ( record; csvReader!(Tuple!(string,string))(inputCsv) )
result[record[0]] = record[1];
return result;
}
immutable string[string] dataLookup;
static this()
{
dataLookup = parseTwoColumnCsv(import("some_data.csv"));
}
void main()
{
import std.stdio;
writefln("dataLookup = %s", dataLookup);
}
---
In this case the AA isn't actually coded into the executable; but
at least the configuration from some_data.csv will be in the
executable as a string. The program will construct the AA at
startup. It's not as "cool", but it should get the job done.
HTH.
Re: How do I use CTFE to generate an immutable associative array at compile time?
On Tuesday, 21 February 2017 at 21:58:07 UTC, Stefan Koch wrote: On Tuesday, 21 February 2017 at 21:53:23 UTC, Chad Joan wrote: Hello all, I'm trying to make this work: [...] You cannot create AA's at ctfe and carry them over to runtime use. You'd have to use a costum dictionary-type. I think the vibe.d project has one you could use. I wondered if this might be the case. Well, I won't worry about it then. I'll have to find another way around. Thanks a bunch!
How do I use CTFE to generate an immutable associative array at compile time?
Hello all,
I'm trying to make this work:
---
pure string[string] parseTwoColumnCsv(string inputCsv)
{
import std.csv;
import std.typecons;
string[string] result;
foreach ( record; csvReader!(Tuple!(string,string))(inputCsv) )
result[record[0]] = record[1];
return result;
}
immutable string[string] dataLookup =
parseTwoColumnCsv(import("some_data.csv"));
void main()
{
import std.stdio;
writefln("dataLookup = %s", dataLookup);
}
---
But (with DMD 2.073.1) I am getting this error:
main.d(14): Error: non-constant expression [['a', 'b', 'c']:['x',
'y', 'z'], ['1', '2', '3']:['4', '5', '6']]
The case with normal (non-associative) arrays seems to work fine,
but is not what I needed:
---
pure string[][] parseTwoColumnCsv(string inputCsv)
{
import std.csv;
import std.typecons;
string[][] result;
foreach ( record; csvReader!(Tuple!(string,string))(inputCsv) )
result ~= [record[0],record[1]];
return result;
}
immutable string[][] dataLookup =
parseTwoColumnCsv(import("some_data.csv"));
void main()
{
import std.stdio;
writefln("dataLookup = %s", dataLookup);
}
---
Any idea how I can get this working?
I have tried a couple other things, like having the
parseTwoColumnCsv function return an immutable string[string] and
casting 'result' to that in the return statement, and I also
tried just casting the value returned from parseTwoColumnCsv as
it appears in the declaration of 'dataLookup'. I tried
std.exception's assumeUnique, but the function/template signature
doesn't seem to support associative arrays.
Thanks.
