Re: How to get compatible symbol names and runtime typeid names for templated classes?
On Tue, May 03, 2022 at 05:25:06PM +, cc via Digitalmars-d-learn wrote:
> On Tuesday, 3 May 2022 at 17:05:09 UTC, H. S. Teoh wrote:
> > Oops, sorry, I made a mistake. The definition of Serializable should be:
> >
> > class Serializable(Base, Derived = Object) : Base {}
>
> There we go, works with this, now I get what it's trying to do:
> ```d
> class Serializable(Base, Derived = Object) : Derived {
> ```
>
> What's the purpose of the `static struct Proxy`? The `static this()`
> seems to work without being enclosed in a structure.
Actually, come to think of it, Proxy isn't strictly necessary. You could
move the static this into Serializable and it'd work. The main purpose
here is to create a separate instance of static this() per instantiation
of Serializable, i.e., there'd be a separate instance of static this()
for each Derived class.
Since the compiler collects all static this()'s into a list of functions
that run at program startup, this allows us to initialize the
deserializers AA with functions that understand how to create an
instance of Derived at runtime. Since in the scope of the static this
we have direct access to compile-time information about Derived, it can
use compile-time introspection to inspect Derived and take the
appropriate action at runtime based on this compile-time knowledge. In
this way, we "translate" compile-time knowledge into runtime knowledge.
T
--
PNP = Plug 'N' Pray
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On Tuesday, 3 May 2022 at 17:05:09 UTC, H. S. Teoh wrote:
Oops, sorry, I made a mistake. The definition of Serializable
should be:
class Serializable(Base, Derived = Object) : Base {}
There we go, works with this, now I get what it's trying to do:
```d
class Serializable(Base, Derived = Object) : Derived {
```
What's the purpose of the `static struct Proxy`? The `static
this()` seems to work without being enclosed in a structure.
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On Tue, May 03, 2022 at 04:59:42PM +, cc via Digitalmars-d-learn wrote:
> On Tuesday, 3 May 2022 at 16:51:33 UTC, H. S. Teoh wrote:
[...]
> > > On Tuesday, 3 May 2022 at 15:08:53 UTC, H. S. Teoh wrote:
> > > > class Base : Serializable!(Base) { ... }
> > > > class Derived : Serializable!(Base, Derived) { ... }
[...]
> Hm although I am having trouble with that particular implementation:
> ```d
> class Base : Serializable!(Base) {}
> class Derived : Serializable!(Base, Derived) {}
> class Serializable(Base, Derived = Base) : Base {}
> ```
> ```
> Error: class `test.Base` circular inheritance
> Error: template instance `test.Serializable!(Base, Base)` error
> instantiating
> ```
Oops, sorry, I made a mistake. The definition of Serializable should be:
class Serializable(Base, Derived = Object) : Base {}
and the corresponding static if in the implementation should test for
Object instead of Base. Basically, it's just some way of
differentiating the base class from the derived classes, because you
need to declare the serialize methods in the base class without
`override` but in the derived classes you need `override`.
T
--
Turning your clock 15 minutes ahead won't cure lateness---you're just making
time go faster!
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On Tuesday, 3 May 2022 at 16:51:33 UTC, H. S. Teoh wrote:
On Tue, May 03, 2022 at 04:38:23PM +, cc via
Digitalmars-d-learn wrote:
On Tuesday, 3 May 2022 at 15:08:53 UTC, H. S. Teoh wrote:
>class Base : Serializable!(Base) { ... }
>class Derived : Serializable!(Base, Derived) { ... }
This is really interesting syntax, I'm surprised that works!
https://en.wikipedia.org/wiki/Curiously_recurring_template_pattern
It's a bit counterintuitive at first, but once you "get" how it
works, it's an extremely powerful technique for leveraging D's
compile-time introspection capabilities. And translating
compile-time information into runtime using static this(). ;-)
T
Hm although I am having trouble with that particular
implementation:
```d
class Base : Serializable!(Base) {}
class Derived : Serializable!(Base, Derived) {}
class Serializable(Base, Derived = Base) : Base {}
```
```
Error: class `test.Base` circular inheritance
Error: template instance `test.Serializable!(Base, Base)` error
instantiating
```
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On Tuesday, 3 May 2022 at 16:38:23 UTC, cc wrote: This is really interesting syntax, I'm surprised that works! Can read a little more on my blog about it: http://dpldocs.info/this-week-in-d/Blog.Posted_2019_06_10.html#tip-of-the-week pretty cool little pattern.
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On Tue, May 03, 2022 at 04:38:23PM +, cc via Digitalmars-d-learn wrote:
> On Tuesday, 3 May 2022 at 15:08:53 UTC, H. S. Teoh wrote:
> > class Base : Serializable!(Base) { ... }
> > class Derived : Serializable!(Base, Derived) { ... }
>
> This is really interesting syntax, I'm surprised that works!
https://en.wikipedia.org/wiki/Curiously_recurring_template_pattern
It's a bit counterintuitive at first, but once you "get" how it works,
it's an extremely powerful technique for leveraging D's compile-time
introspection capabilities. And translating compile-time information
into runtime using static this(). ;-)
T
--
Never step over a puddle, always step around it. Chances are that whatever made
it is still dripping.
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On Tuesday, 3 May 2022 at 15:08:53 UTC, H. S. Teoh wrote:
class Base : Serializable!(Base) { ... }
class Derived : Serializable!(Base, Derived) { ... }
This is really interesting syntax, I'm surprised that works!
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On 3/5/22 16:48, Adam D Ruppe wrote: Believe it or not, you don't need to touch the compiler. Open your druntime's object.d and search for `RTInfo` http://druntime.dpldocs.info/object.RTInfo.html That is instantiated for every user defined type in the program and you have the compile time info. all druntime uses it for is a tiny bit of GC info and even then only sometimes. But it could do so so so much more. Including doing custom factories and runtime reflection buildups! This looks nice, but I actually meant to allow "template this" in static contexts, as in the bug reports. I think that might indeed need compiler support? You'll make me happy if that's possible without touching the compiler!
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On Tue, May 03, 2022 at 04:38:53PM +0200, Arafel via Digitalmars-d-learn wrote:
> On 3/5/22 15:57, Adam D Ruppe wrote:
> > So doing things yourself gives you some control.
>
> Yes, it is indeed possible (I acknowledged it), but I think it's much
> more cumbersome than it should, and puts the load on the user.
>
> If templated this worked in static context (ideally everywhere else
> too), then we'd be able to implement RTTI in a 100% "pay as you go"
> way: just inherit from SerializableObject, or perhaps add a mixin to
> your own root class, and that'd be it.
>
> Actually, it would be cool to do it through an interface, although I
> don't think an interface's static constructors are invoked by the
> implementing classes... it would be cool, though.
The way I did it in my own serialization code is to use CRTP with static
ctors in templated wrapper structs.
Namely, replace:
class Base { ... }
class Derived : Base { ... }
with:
class Base : Serializable!(Base) { ... }
class Derived : Serializable!(Base, Derived) { ... }
That's the only thing user code classes need to do. The rest is done in
the Serializable proxy base class using compile-time introspection. In a
nutshell, what Serializable does is to inject serialize() and
deserialize() methods into the class hierarchy. Here's a brief sketch of
what it looks like:
class Serializable(Base, Derived = Base) : Base
{
static if (is(Base == Derived)) // this is the base of the
hierarchy
{
// Base class declarations
void serialize(...) {
... // use introspection to extract data members
}
void deserialize(...) {
... // use introspection to reconstitute data
members
}
}
else // this is a derived class in the hierarchy
{
override void serialize(...) {
... // use introspection to extract data members
}
override void deserialize(...) {
... // use introspection to reconstitute data
members
}
}
// How does the deserializer recreate an instance of
// Derived? By registering the string name of the class
// into a global hash:
static struct Proxy // N.B.: this is instantiated for each
Derived class
{
// This static this gets instantiated per
// Derived class, and uses compile-time
// knowledge about Derived to generate code for
// reconstructing an instance of Derived.
static this() {
deserializers[Derived.stringof] = {
auto obj = new Derived();
obj.deserialize(...);
return obj;
};
}
}
}
// This is module-global.
/*shared*/ static Object delegate(...)[string] deserializers;
// Global deserialize method that returns an instance of the
// class hierarchy.
Object deserialize(...) {
// Obtain class name from serialized data
string classname = ...;
// Dispatch to the correct method registered by Proxy's
// static this, that recreates the class of the required
// type.
return deserializers[classname](...);
}
The nice thing about this approach is that you have full compile-time
information about the target type `Derived`, in both the serialization
and deserialization methods. So you can use introspection to automate
away most of the boilerplate associated with serialization code. E.g.,
iterate over __traits(allMembers) to extract data fields, inspect UDAs
that allow user classes to specify how the serialization should proceed,
etc..
T
--
Doubtless it is a good thing to have an open mind, but a truly open mind should
be open at both ends, like the food-pipe, with the capacity for excretion as
well as absorption. -- Northrop Frye
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On Tuesday, 3 May 2022 at 14:38:53 UTC, Arafel wrote: Actually, it would be cool to do it through an interface, although I don't think an interface's static constructors are invoked by the implementing classes... it would be cool, though. yeah interfaces can't have constructors. I'd try it myself, but I wouldn't know where to start. Compiler internals are way beyond my comfort zone... Believe it or not, you don't need to touch the compiler. Open your druntime's object.d and search for `RTInfo` http://druntime.dpldocs.info/object.RTInfo.html That is instantiated for every user defined type in the program and you have the compile time info. all druntime uses it for is a tiny bit of GC info and even then only sometimes. But it could do so so so much more. Including doing custom factories and runtime reflection buildups!
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On 3/5/22 15:57, Adam D Ruppe wrote: So doing things yourself gives you some control. Yes, it is indeed possible (I acknowledged it), but I think it's much more cumbersome than it should, and puts the load on the user. If templated this worked in static context (ideally everywhere else too), then we'd be able to implement RTTI in a 100% "pay as you go" way: just inherit from SerializableObject, or perhaps add a mixin to your own root class, and that'd be it. Actually, it would be cool to do it through an interface, although I don't think an interface's static constructors are invoked by the implementing classes... it would be cool, though. And, in one of the bugs, you argue yourself that according to the spec, it *should* work. So please let me just whine... I mean, raise awareness ;-), in case somebody thinks it's interesting and feels brave enough to have a go at it. I'd try it myself, but I wouldn't know where to start. Compiler internals are way beyond my comfort zone...
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On Tuesday, 3 May 2022 at 13:25:14 UTC, Arafel wrote:
I'd like to do a runtime registration system myself, using a
"template this" static constructor. A simple version supporting
only default constructors would be:
Yeah, you can't template this a static constructor, but you can
just use a static constructor. It will have to be mixed into each
child class though, and the compiler won't help to remind you.
But do something along the lines of:
```d
module factory.register;
private Object function()[string] factories;
Object construct(string name) {
if(auto f = name in factories)
return (*f)();
return null;
}
mixin template Register() {
static this() {
import factory.register;
alias This = typeof(this);
// bypassing private
__traits(getMember, factory.register, "factories")
[This.mangleof] = function Object() {
// you could even delegate to a static
// function if one is present, or pass
arguments
// etc. this impossible with
Object.factory
return new This();
};
}
}
```
That code is your library. Then, to use it:
```d
import factory.register;
class MyThing {
// you have to remember to do this in each child
mixin Register;
}
void main() {
auto t = new MyThing();
// I used the mangle instead of the FQN since it
// is easier.
Object o = construct(typeof(t).mangleof);
MyThing t2 = cast(MyThing) o;
assert(t2 !is null); // assert it actually worked
}
```
Now, you can extend this a little if you're willing to add an
interface too. And if you forget to register the base class, the
interface method being not implemented will remind user they did
something wrong, and you can runtime assert to check child
classes.
Check this out:
```d
module factory.register;
private Object function()[string] factories;
Object construct(string name) {
if(auto f = name in factories)
return (*f)();
return null;
}
// adding this for the assert
bool typeIsRegistered(string name) {
return (name in factories) !is null;
}
// this interface gives runtime access to the info we need
interface Serializable {
string typeCode() const;
}
mixin template Register() {
// interface implementation
override string typeCode() const {
// casting away const for more consistent names
alias no_const = typeof(cast() this);
auto name = no_const.mangleof;
// a runtime check to help remind you if
something not registered
import factory.register;
assert(typeIsRegistered(name), "Type
"~typeof(this).stringof~" not registered!");
// also making sure the child class was registered
// by ensuring the runtime type is the same as
the static type
assert(typeid(this) == typeid(no_const), "Child
class "~typeid(this).toString()~" was not registered!");
return name;
}
static this() {
import factory.register;
alias This = typeof(this);
// bypassing private
__traits(getMember, factory.register, "factories")
[This.mangleof] = function Object() {
// you could even delegate to a static
// function if one is present, or pass
arguments
// etc. this impossible with
Object.factory
return new This();
};
}
}
```
And the usage:
```d
import factory.register;
class MyThing : Serializable {
mixin Register;
}
class Child : MyThing {
// forgot to register uh oh
// mixin Register;
}
void main() {
auto t = new MyThing();
Object o = construct(typeof(t).mangleof);
MyThing t2 = cast(MyThing) o;
assert(t2 !is null);
auto child = new Child();
// if we called this in the serialize function or even
one of those constructors' contracts
// it can verify things work by triggering the asserts
back in the library implementation
child.typeCode();
}
```
So doing things yourself gives you some control.
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On 3/5/22 14:46, Adam D Ruppe wrote:
Put a static constructor in the class which appends a factory delegate
to an array or something you can use later. Then you can use your own
thing to construct registered objects.
I'd like to do a runtime registration system myself, using a "template
this" static constructor. A simple version supporting only default
constructors would be:
```d
module test;
import std.stdio : writeln;
class MyObject {
/* static */ this(this T)() {
string type = typeid(T).name;
if (type !in generators) {
generators[type] = () => new T();
}
}
static MyObject factory(string type) {
if(type in generators) {
return generators[type]();
} else {
return null;
}
}
private:
static MyObject function()[string] generators;
}
class MyClass : MyObject {
this() {
writeln("Creating MyClass");
}
}
void main() {
auto _ = new MyClass(); // Shouldn't be needed
auto myClass = MyObject.factory("test.MyClass");
}
```
Unfortunately, this isn't currently possible:
https://issues.dlang.org/show_bug.cgi?id=10488
https://issues.dlang.org/show_bug.cgi?id=20277
(notice the big number of duplicates).
The closest feasible option is to put it in a non-static constructor,
and that's suboptimal: it forces an instantiation of the class, and it
will be run at every instantiation.
Alternatively, instruct the users to create a static constructor for
each of the classes they'd like registered (perhaps through a mixin),
but that's also quite cumbersome.
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On Tuesday, 3 May 2022 at 10:48:53 UTC, bauss wrote:
Object.factory calls TypeInfo_Class.find which just loops
through ModuleInfo and then looks if any of the entries in
localClasses has a name that matches.
Afterwards it calls the create function on the TypeInfo_Class
which of course isn't "generic" by any means.
This is where compile-time has its limits compared to runtime
type creation, because templates only live during compile-time
then it isn't really that easy to do something like this, where
it would be trivial in other languages like C#.
On Tuesday, 3 May 2022 at 12:46:56 UTC, Adam D Ruppe wrote:
On Tuesday, 3 May 2022 at 09:42:45 UTC, cc wrote:
something I can pass to `Object.factory`.
Object.factory is useless and will hopefully be removed someday.
Instead, make your own factory registration function.
Put a static constructor in the class which appends a factory
delegate to an array or something you can use later. Then you
can use your own thing to construct registered objects.
Yeah, that's unfortunate. Actually I was already doing something
similar for serialization/encoding to get the true type of an
object (making sure `Animal an = new Cat();` encodes a Cat and
not an Animal), took me a second to put two and two together and
realize I could just instantiate objects via new that way instead
of calling Object.factory.
At the moment I try to register as many relevant symbols as I can
automatically when encoding is called for a given object, such as:
```d
private mixin template RegisterModule(alias MOD) {
void RegisterModule() {
static foreach (SYM; getSymbolsByUDA!(MOD, Coder)) {
static if (is(SYM == class)) {
RegisterSerializer!SYM();
}
}
}
}
private static void[0][string] registeredModules;
private void registerModules(T)() {
enum string MODULENAME = moduleName!T;
if (MODULENAME !in registeredModules) {
registeredModules.require(MODULENAME);
mixin("import "~MODULENAME~";");
mixin("mixin RegisterModule!"~MODULENAME~";");
RegisterModule();
}
}
IPtr encode(T)(T obj) {
registerModules!T;
...
}
```
I'll have to get a little more creative for registering templated
classes then, something like this works:
```d
static void RegisterSerializer(alias SYM)(string runtimeName =
null) {
enum sym = fullyQualifiedName!SYM;
if (sym !in serialTypes) {
auto st = new SerialType!SYM;
serialTypes[sym] = st;
if (runtimeName.length && runtimeName !in serialTypes)
serialTypes[runtimeName] = st;
}
}
static void RegisterSerializer(T : Object)(T obj) {
RegisterSerializer!T(typeid(obj).name);
}
```
but I'd rather not have to instantiate an actual object just to
get its typeid().name, I suppose I can just manually construct it
from the fullyQualifiedName inserting the parenthesis and
appended portion so it matches.
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On Tuesday, 3 May 2022 at 09:42:45 UTC, cc wrote: something I can pass to `Object.factory`. Object.factory is useless and will hopefully be removed someday. Instead, make your own factory registration function. Put a static constructor in the class which appends a factory delegate to an array or something you can use later. Then you can use your own thing to construct registered objects.
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On 3/5/22 12:48, bauss wrote:
This is where compile-time has its limits compared to runtime type
creation, because templates only live during compile-time then it isn't
really that easy to do something like this, where it would be trivial in
other languages like C#.
That's something I don't really get. I totally understand that you can't
instantiate the template during runtime, but why can't already
instantiated classes be registered just like non-templated ones?
I tried the following snippet, and couldn't find C!int.C anywhere,
although it **must** be there: I can get the `TypeInfo_Class` object, so
I can clearly create new instances at runtime:
```d
import std.stdio : writeln;
class C(T) {}
class D {}
void main() {
auto c = new C!int();
auto c2 = typeid(c).create();
auto d = new D();
writeln(typeid(c).name);
writeln(typeid(c2).name);
writeln(typeid(d).name);
writeln("");
writeln;
writeln;
foreach (m; ModuleInfo) {
if (m) {
writeln(m.name);
writeln("");
foreach (c; m.localClasses) {
if (c) {
writeln(c.name);
}
}
writeln;
}
}
}
```
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On Tuesday, 3 May 2022 at 09:52:56 UTC, cc wrote:
On Tuesday, 3 May 2022 at 09:42:45 UTC, cc wrote:
Given a runtime typeid, how can I get the equivalent
fullyQualifiedName without attempting to mangle the string
myself manually? e.g. something I can pass to
`Object.factory`.
Actually, looking at this further, does Object.factory even
support templates? I'm getting null returned from any attempt
to instantiate a templated classname.
It does not.
Object.factory calls TypeInfo_Class.find which just loops through
ModuleInfo and then looks if any of the entries in localClasses
has a name that matches.
So for your example it does this check:
```
if (c.name == "test.Foo!(true)") {
return c; // c is the TypeInfo_Class that matches the given
class name
}
```
https://github.com/dlang/druntime/blob/master/src/object.d#L1661
Afterwards it calls the create function on the TypeInfo_Class
which of course isn't "generic" by any means.
This is where compile-time has its limits compared to runtime
type creation, because templates only live during compile-time
then it isn't really that easy to do something like this, where
it would be trivial in other languages like C#.
Re: How to get compatible symbol names and runtime typeid names for templated classes?
On Tuesday, 3 May 2022 at 09:42:45 UTC, cc wrote: Given a runtime typeid, how can I get the equivalent fullyQualifiedName without attempting to mangle the string myself manually? e.g. something I can pass to `Object.factory`. Actually, looking at this further, does Object.factory even support templates? I'm getting null returned from any attempt to instantiate a templated classname.
How to get compatible symbol names and runtime typeid names for templated classes?
This produces compatible strings between symbol and runtime type:
```d
class Foo {}
void main() {
alias Foo F;
writeln(fullyQualifiedName!F);
auto f = new F;
writeln(typeid(f).name);
}
```
```
test.Foo
test.Foo
```
But if the class is a template, the strings different:
```d
class Foo(bool b) {}
void main() {
alias Foo!true F;
writeln(fullyQualifiedName!F);
auto f = new F;
writeln(typeid(f).name);
}
```
```
test.Foo!(true)
test.Foo!true.Foo
```
Given a runtime typeid, how can I get the equivalent
fullyQualifiedName without attempting to mangle the string myself
manually? e.g. something I can pass to `Object.factory`.
