Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Saturday, 3 September 2016 at 16:52:50 UTC, Martin Nowak wrote:
On Tuesday, 30 August 2016 at 22:24:12 UTC, Ali Çehreli wrote:
I don't agree with the current solution:
Well let's come up with a better solution then.
Let's start by finding some proper use-cases that require
introspection on private members w/o having access to them.
Use case that I think is pretty durn important: using
Allocator.make and forwarding a call to the private constructor
on a class.
This does not work now:
---
class A
{ int b; private this(int a){b=a;} }
auto ptr = make!A(Mallocator.instance, 42); // Compile Error!
---
Allocators are second class in this respect compared to GC.
Mixins aren't a good solution to this.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Wednesday, 7 September 2016 at 19:29:05 UTC, Ali Çehreli wrote: On 09/03/2016 09:37 AM, Martin Nowak wrote: > On Wednesday, 31 August 2016 at 21:12:54 UTC, Ali Çehreli wrote: >> The recommended solution of mixing in every template instance is not a >> viable solution because that would effectively remove IFTI from D. >> What a huge loss that would be. We can't afford that. > > Exaggeration won't help us to find good solutions. Remember that private > fields never were accessible, so only some edge cases will be affected. > The need for mixin templates will remain rare. I don't see how the user of a template library can decide whether to mixin or not without knowing the current implementation if of the library. For that reason, I must mixin. For example, the following program is broken because I don't know whether writeln uses allMembers or not: // BROKEN CODE: MyStruct s; writeln(s); Do you see the problem? I can't leave that code that way. I have to figure out mixin writeln's instantiation. As you see, mixins will not be rare. Every template use must be mixed in. Well, I can only repeat what I stated several times before, private members were never accessible by name (via getMember or .name). So either writeln uses .tupleof (unnamed) or ignores private members. This is the pre 2.071 state. method| visible | accessible .name |y|n getMember |y|n .tupleof |y|y This was the 2.071.1 state (for which the allMembers fix was a valid solution). method| visible | accessible .name |n|n getMember |n|n .tupleof |y|y This is what we're now doing in 2.071.2 (see https://github.com/dlang/dmd/pull/6111), method| visible | accessible .name |n|n getMember |y|n .tupleof |y|y and what we plan to do with 2.072 or 2.073. method| visible | accessible .name |n|n getMember |y|y .tupleof |y|y In fact access checks will get removed soon.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 9/7/16 9:21 PM, Ali Çehreli wrote: On 09/03/2016 03:13 PM, Ethan Watson wrote: On Saturday, 3 September 2016 at 21:54:24 UTC, Jacob Carlborg wrote: allMembers not returning all members makes introspection entirely useless when it comes to Binderoo. Same problem with Weka's code base... getMember itself, well, I'd honestly prefer if there was a way to get to members without having to correlate with .tupleof as it will simplify Binderoo code. Again, same problem at Weka... Protection attributes are for protecting programmers from implementation changes. If I reach for allMembers or getMember, it's an explicit way of saying "I don't care about the consequences." Martin has a PR to improve on this, could you all please take a look? I can't really do much until Sunday, I'm on a WiFi connection slower than a herd of turtles, and which only works if I hold my laptop over my head. So I'll click "Send" on this then push the roof with my laptop. -- Andrei
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Wed, 07 Sep 2016 12:21:40 -0700, Ali Çehreli wrote: > Protection attributes are for protecting programmers from implementation > changes. If I reach for allMembers or getMember, it's an explicit way of > saying "I don't care about the consequences." If it's just about implementation changes, allMembers / getMember is a way of inspecting types to get what currently exists. It will continue to be correct in the face of changing fields.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Wednesday, 7 September 2016 at 15:08:17 UTC, Johan Engelen wrote: Note that the spec tells us: "The .tupleof property returns an ExpressionTuple of all the fields in the class, __excluding__ the hidden fields and __the fields in the base class__." (emphasis mine) So then you have to do __traits(parent, ) recursively...? or use std.traits.BaseClassesTuple.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 09/03/2016 09:37 AM, Martin Nowak wrote: > On Wednesday, 31 August 2016 at 21:12:54 UTC, Ali Çehreli wrote: >> The recommended solution of mixing in every template instance is not a >> viable solution because that would effectively remove IFTI from D. >> What a huge loss that would be. We can't afford that. > > Exaggeration won't help us to find good solutions. Remember that private > fields never were accessible, so only some edge cases will be affected. > The need for mixin templates will remain rare. I don't see how the user of a template library can decide whether to mixin or not without knowing the current implementation if of the library. For that reason, I must mixin. For example, the following program is broken because I don't know whether writeln uses allMembers or not: // BROKEN CODE: MyStruct s; writeln(s); Do you see the problem? I can't leave that code that way. I have to figure out mixin writeln's instantiation. As you see, mixins will not be rare. Every template use must be mixed in. Ali
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 09/03/2016 03:13 PM, Ethan Watson wrote: > On Saturday, 3 September 2016 at 21:54:24 UTC, Jacob Carlborg wrote: > allMembers not returning all members makes introspection > entirely useless when it comes to Binderoo. Same problem with Weka's code base... > getMember itself, well, I'd honestly prefer if there was a way to get to > members without having to correlate with .tupleof as it will simplify > Binderoo code. Again, same problem at Weka... Protection attributes are for protecting programmers from implementation changes. If I reach for allMembers or getMember, it's an explicit way of saying "I don't care about the consequences." Ali
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Sunday, 4 September 2016 at 12:36:43 UTC, Andrei Alexandrescu wrote: Yah, .tupleof is great to have. I think that should be enough for most introspection needs. Only the field names are missing, can those be accessed somehow? -- Andrei Note that the spec tells us: "The .tupleof property returns an ExpressionTuple of all the fields in the class, __excluding__ the hidden fields and __the fields in the base class__." (emphasis mine) So then you have to do __traits(parent, ) recursively...?
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 09/04/2016 05:37 AM, David Nadlinger wrote: > On Sunday, 4 September 2016 at 12:33:07 UTC, Andrei Alexandrescu wrote: >> Thanks for answering. Yes, we really need introspection of private >> members. One way or another we need to support that. > > Do we, though? It's easy to state a general claim like this, but I find > it hard to actually justify. > > — David Let me flip the question: What harm can there be when I pass my type to a template and that template accesses the private members? Ali
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 09/04/2016 05:51 AM, Andrei Alexandrescu wrote: * Tracing how and when a member mutates. Ali
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 09/04/2016 05:37 AM, David Nadlinger wrote: > On Sunday, 4 September 2016 at 12:33:07 UTC, Andrei Alexandrescu wrote: >> Thanks for answering. Yes, we really need introspection of private >> members. One way or another we need to support that. > > Do we, though? It's easy to state a general claim like this, but I find > it hard to actually justify. > > — David Let me try to reword what I've already said elsewhere in this thread. As the user of a library I shouldn't need to know whether a template of that library uses __traits(allMembers) or not. Unfortunately, if __traits(allMembers) depends on access rights, I am forced to mixin *every* template in my code because I don't and should not know the template's implementation. If I don't know their implementation, I have to prepare myself for the worst and mixin *every* template. That's insanity. Even if I know that they don't use __traits(allMembers) today, they may change their implementation in the future. So, I really have to mixin every template today. Further, I have to be on my toes and watch every feature of every library in case they changed e.g. a function to a function template in a new release. Oh yes, I have to mixin that new template as well! (Again, because I can't be sure whether they use __traits(allMembers) or not.) This new behavior and its mixin workaround is so insane to me that I can't even find ways to spell it out clearly. This behavior kills IFTI altogether because I don't know who uses __traits(allMembers). I can't rely on IFTI. I have to mixin every template instantiation myself. Crazy. Ali
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 9/5/16 10:38 AM, Jacob Carlborg wrote:
On 2016-09-04 14:36, Andrei Alexandrescu wrote:
Yah, .tupleof is great to have. I think that should be enough for most
introspection needs. Only the field names are missing, can those be
accessed somehow? -- Andrei
Yes:
module foo;
struct Foo
{
private int a;
}
module main;
import foo;
static assert(__traits(identifier, Foo.tupleof[0]) == "a");
Thank you. This is good news. -- Andrei
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 2016-09-04 14:51, Andrei Alexandrescu wrote: Of course the more important applications are those I can't yet imagine. The way I see it introspection must have full power and unfettered access. So definitely it must be able to pass through regular protection. * Linking up GUI controls to instance variables in the code, like Xcode/Interface Builder -- /Jacob Carlborg
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 2016-09-04 14:36, Andrei Alexandrescu wrote:
Yah, .tupleof is great to have. I think that should be enough for most
introspection needs. Only the field names are missing, can those be
accessed somehow? -- Andrei
Yes:
module foo;
struct Foo
{
private int a;
}
module main;
import foo;
static assert(__traits(identifier, Foo.tupleof[0]) == "a");
--
/Jacob Carlborg
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Monday, 5 September 2016 at 07:48:11 UTC, Johannes Pfau wrote: Am Sun, 4 Sep 2016 14:54:33 +0200 schrieb Andrei Alexandrescu : On 9/4/16 2:37 PM, David Nadlinger wrote: > [...] Static introspection is by far the most powerful feature of D. The last thing to do with it is hamstring it with nonsense like private :o). Quoting elsethread: > [...] Andrei All these examples need access to private data fields only. Most performance optimizations are only possible for private functions anyway. So is there a use case to access/call private functions? One use case would be accessing UDA on private fields, to inspect which fields need to be injected with a dependency, in aggregate type, just like spring from java does with autowire annotation. You mark a private field with autowired, and spring calls associated setter for the field. Does tupleof allow access to associated UDAs of contained fields, as well field names?
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
Am Sun, 4 Sep 2016 14:54:33 +0200 schrieb Andrei Alexandrescu : > On 9/4/16 2:37 PM, David Nadlinger wrote: > > On Sunday, 4 September 2016 at 12:33:07 UTC, Andrei Alexandrescu > > wrote: > >> Thanks for answering. Yes, we really need introspection of private > >> members. One way or another we need to support that. > > > > Do we, though? It's easy to state a general claim like this, but I > > find it hard to actually justify. > > Static introspection is by far the most powerful feature of D. The > last thing to do with it is hamstring it with nonsense like > private :o). > > Quoting elsethread: > > > There are a few cases I can think of: > > > > * Serialization and deserialization, shallow and deep. These would > > need access to the object layout, and possibly field names too (for > > cross-checking and versioning purposes). > > > > * Binary saving, loading, and fixup (a subset of > > serialization/deserialization) - that thing when you copy raw > > memory to a file and then load it raw and fix pointers up. > > > > * Memory management and garbage collection: access to field types > > allows efficient generic tracing. > > > > * Database interfacing, automated binding, object-relational > > mapping, etc. It stands to reason that field names would be needed > > and fields would be private, yet the database loader does have > > access to them. > > > > Of course the more important applications are those I can't yet > > imagine. The way I see it introspection must have full power and > > unfettered access. So definitely it must be able to pass through > > regular protection. > > > Andrei All these examples need access to private data fields only. Most performance optimizations are only possible for private functions anyway. So is there a use case to access/call private functions?
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Sunday, 4 September 2016 at 12:51:05 UTC, Andrei Alexandrescu wrote: See http://forum.dlang.org/post/ymkehalxcigswvltl...@forum.dlang.org There are a few cases I can think of: The classical ones, but again the question was for introspection without access (b/c named access through getMember was never allowed, only .tupleof). As the trade-offs for allowing access to private members don't seem that bad, we'll now try to go in the other direction, skipping visibility checks now and removing the access checks in a later release.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 9/4/16 2:37 PM, David Nadlinger wrote: On Sunday, 4 September 2016 at 12:33:07 UTC, Andrei Alexandrescu wrote: Thanks for answering. Yes, we really need introspection of private members. One way or another we need to support that. Do we, though? It's easy to state a general claim like this, but I find it hard to actually justify. Static introspection is by far the most powerful feature of D. The last thing to do with it is hamstring it with nonsense like private :o). Quoting elsethread: There are a few cases I can think of: * Serialization and deserialization, shallow and deep. These would need access to the object layout, and possibly field names too (for cross-checking and versioning purposes). * Binary saving, loading, and fixup (a subset of serialization/deserialization) - that thing when you copy raw memory to a file and then load it raw and fix pointers up. * Memory management and garbage collection: access to field types allows efficient generic tracing. * Database interfacing, automated binding, object-relational mapping, etc. It stands to reason that field names would be needed and fields would be private, yet the database loader does have access to them. Of course the more important applications are those I can't yet imagine. The way I see it introspection must have full power and unfettered access. So definitely it must be able to pass through regular protection. Andrei
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 9/4/16 5:45 AM, Martin Nowak wrote: A public setter for private members is weird How do you mean that? It's an absolute classic. Please elaborate, thanks. -- Andrei
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 9/4/16 11:09 AM, Martin Nowak wrote: On Saturday, 3 September 2016 at 16:52:50 UTC, Martin Nowak wrote: On Tuesday, 30 August 2016 at 22:24:12 UTC, Ali Çehreli wrote: Well let's come up with a better solution then. Let's start by finding some proper use-cases that require introspection on private members w/o having access to them. Still haven't heard really good use cases for the above, but the trade-offs for also allowing access to private members aren't that bad, thus it seems like the better path forward. See http://forum.dlang.org/post/ymkehalxcigswvltl...@forum.dlang.org There are a few cases I can think of: * Serialization and deserialization, shallow and deep. These would need access to the object layout, and possibly field names too (for cross-checking and versioning purposes). * Binary saving, loading, and fixup (a subset of serialization/deserialization) - that thing when you copy raw memory to a file and then load it raw and fix pointers up. * Memory management and garbage collection: access to field types allows efficient generic tracing. * Database interfacing, automated binding, object-relational mapping, etc. It stands to reason that field names would be needed and fields would be private, yet the database loader does have access to them. Of course the more important applications are those I can't yet imagine. The way I see it introspection must have full power and unfettered access. So definitely it must be able to pass through regular protection. Andrei
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Sunday, 4 September 2016 at 12:37:47 UTC, David Nadlinger wrote: Do we, though? It's easy to state a general claim like this, but I find it hard to actually justify. (And to pre-empt the collective freak-out, note that this statement is coming from somebody who has done quite a bit of pioneering serialisation/meta-programming work. — David)
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Sunday, 4 September 2016 at 12:33:07 UTC, Andrei Alexandrescu wrote: Thanks for answering. Yes, we really need introspection of private members. One way or another we need to support that. Do we, though? It's easy to state a general claim like this, but I find it hard to actually justify. — David
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 9/4/16 5:56 AM, Martin Nowak wrote: On Saturday, 3 September 2016 at 19:31:12 UTC, Jacob Carlborg wrote: That can usually be solved using .tupleof[i], but I agree. Well, .tupleof gives you a typed representation of the memory layout, to me it's something different than qualified access of fields, just a safer mean to access memory. And sure you can always memcpy data. Also introspection works on .tupleof, so it's not affected by the allMembers change and it's awkward to mix them b/c names and layout don't have a simple 1-to-1 relation. Yah, .tupleof is great to have. I think that should be enough for most introspection needs. Only the field names are missing, can those be accessed somehow? -- Andrei
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 9/4/16 5:14 AM, Martin Nowak wrote: If we really need introspection of private members than we might need to go back to the drawing board and modify the visibility concept introduced with DIP22. Thanks for answering. Yes, we really need introspection of private members. One way or another we need to support that. -- Andrei
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 2016-09-04 05:45, Martin Nowak wrote: A public setter for private members is weird, but well. The setter can do validations, manipulation and other kinds of calculations based on the given value. -- /Jacob Carlborg
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 2016-09-04 09:39, Johannes Pfau wrote: Allowing access to private fields / functions also means that those members are part of the public API/ABI of a library. So changing private members can break dependent libraries... They've always been accessible using .tupleof and taking the address of a member, before the recent lookup rule changes. -- /Jacob Carlborg
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Saturday, 3 September 2016 at 16:52:50 UTC, Martin Nowak wrote: On Tuesday, 30 August 2016 at 22:24:12 UTC, Ali Çehreli wrote: Well let's come up with a better solution then. Let's start by finding some proper use-cases that require introspection on private members w/o having access to them. Still haven't heard really good use cases for the above, but the trade-offs for also allowing access to private members aren't that bad, thus it seems like the better path forward. See http://forum.dlang.org/post/ymkehalxcigswvltl...@forum.dlang.org
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Sunday, 4 September 2016 at 08:42:43 UTC, Martin Nowak wrote: On Sunday, 4 September 2016 at 07:24:42 UTC, Basile B. wrote: The introspection creates a special structure for each property annotated with @Set, @Get, or @SetGet. This is a kind of interface for serialization/binding/object inspector (a bit like "published" in Object Pascal). But you can't access private fields outside of the current module, so you need to mixin sth. to set/get the value anyhow. But that's the point ! We are many willing to see this restriction removed for the traits because they are special., i.e they should allow to implement, using meta programming, features that less powerful languages do in intern. That's also why I propose to add a 3rd optional bool parameter to the allMembers trait: enum RespectProtection = false; enum IgnoreProtection = true; __traits(allMembers, T, RespectProtection) __traits(allMembers, T, IgnoreProtection) The 3rd optional parameter, when not specified, would be set a default value of false (equivalent to __traits(allMembers, T, RespectProtection), to prevent breakage. Because allMembers would act as a filter for getMember, getOverload...the protection check in getMember, getOverload... would be removed. This will be obviously more complex than that but that's the idea.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Sunday, 4 September 2016 at 07:24:42 UTC, Basile B. wrote: The introspection creates a special structure for each property annotated with @Set, @Get, or @SetGet. This is a kind of interface for serialization/binding/object inspector (a bit like "published" in Object Pascal). But you can't access private fields outside of the current module, so you need to mixin sth. to set/get the value anyhow. Defining de-/serialize methods in the class/struct, e.g. with a mixin template would be the cleaner and more obvious approach IMO. I see your strategy...each time someone will find an argument to make the traits omniscients you'll say, "no because it's cleaner to do like that" ? No, I'm just trying to understand the requirement for introspection without access.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 9/3/2016 2:54 PM, Jacob Carlborg wrote: On 2016-09-03 22:40, Andrei Alexandrescu wrote: That... doesn't sound good. I wonder how such important changes slip by Walter and myself unnoticed. Here's the PR that introduced the change: https://github.com/dlang/dmd/pull/6078 Looks like I'm to blame as I pulled it.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
Am Sun, 04 Sep 2016 03:29:59 + schrieb Martin Nowak : > On Sunday, 4 September 2016 at 03:21:05 UTC, Stefan Koch wrote: > > While I do understand, that there could be a potential > > performance when private members could be changed around > > because they are not visible form outside. > > > > I fail to see how we would take advantage of that without > > breaking our object-model. > > It's mostly about linkage of private methods. Allowing access to private fields / functions also means that those members are part of the public API/ABI of a library. So changing private members can break dependent libraries...
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Sunday, 4 September 2016 at 03:45:50 UTC, Martin Nowak wrote: On Saturday, 3 September 2016 at 17:23:53 UTC, Basile B. wrote: On Saturday, 3 September 2016 at 16:52:50 UTC, Martin Nowak wrote: On Tuesday, 30 August 2016 at 22:24:12 UTC, Ali Çehreli wrote: I don't agree with the current solution: Well let's come up with a better solution then. Let's start by finding some proper use-cases that require introspection on private members w/o having access to them. In my user library, the serialization is based on the @Set and @Get UDAs. Typically there would be a setter (a public method) and no getter (i.e the data is read directly either from a protected or private variable). A public setter for private members is weird, but well. No in Object Pascal that's a common to have property Foo: string read fFoo write setFoo; By the way there's an error from my part, in my lib it's @SetGet which gives access to private/protected field "directly". As a library template can't read private fields, you don't need the @Get attribute. The introspection is used to create a property descriptor. What does that mean? You're creating a property in the serialised data? The introspection creates a special structure for each property annotated with @Set, @Get, or @SetGet. This is a kind of interface for serialization/binding/object inspector (a bit like "published" in Object Pascal). https://gist.github.com/BBasile/39fb66f7a0189660182cc637ab8d698b/archive/3e0f7441cfba89b5959b78eef63b9538d137a55a.zip (you can "dub a.d") Defining de-/serialize methods in the class/struct, e.g. with a mixin template would be the cleaner and more obvious approach IMO. I see your strategy...each time someone will find an argument to make the traits omniscients you'll say, "no because it's cleaner to do like that" ?
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Saturday, 3 September 2016 at 19:31:12 UTC, Jacob Carlborg wrote: That can usually be solved using .tupleof[i], but I agree. Well, .tupleof gives you a typed representation of the memory layout, to me it's something different than qualified access of fields, just a safer mean to access memory. And sure you can always memcpy data. Also introspection works on .tupleof, so it's not affected by the allMembers change and it's awkward to mix them b/c names and layout don't have a simple 1-to-1 relation.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Saturday, 3 September 2016 at 18:56:33 UTC, Dicebot wrote: Obviously serialization libraries come to mind, especially important for binary serialization because private members still affect the layout of the struct. Let me repeat that another time, private members have never been accessible by traits. That's the main reasoning this change was based upon.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Saturday, 3 September 2016 at 17:23:53 UTC, Basile B. wrote: On Saturday, 3 September 2016 at 16:52:50 UTC, Martin Nowak wrote: On Tuesday, 30 August 2016 at 22:24:12 UTC, Ali Çehreli wrote: I don't agree with the current solution: Well let's come up with a better solution then. Let's start by finding some proper use-cases that require introspection on private members w/o having access to them. In my user library, the serialization is based on the @Set and @Get UDAs. Typically there would be a setter (a public method) and no getter (i.e the data is read directly either from a protected or private variable). A public setter for private members is weird, but well. As a library template can't read private fields, you don't need the @Get attribute. The introspection is used to create a property descriptor. What does that mean? You're creating a property in the serialised data? Defining de-/serialize methods in the class/struct, e.g. with a mixin template would be the cleaner and more obvious approach IMO.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Sunday, 4 September 2016 at 03:21:05 UTC, Stefan Koch wrote: While I do understand, that there could be a potential performance when private members could be changed around because they are not visible form outside. I fail to see how we would take advantage of that without breaking our object-model. It's mostly about linkage of private methods.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Saturday, 3 September 2016 at 17:57:08 UTC, Jonathan M Davis wrote: I do get the feeling that too much of what we do with features like this is ad-hoc without really designing things up front, and then we keep having to tweak stuff as we go along in ways that aren't always particularly nice. That's not true, there was DIP22, and it explicitly mentions how traits should work. The problem that was new and very hard to see, is that allowing traits to bypass visibility checks means we either can't replace access checks, must allow access of private members (w/ the cost tradeoffs mentioned in the announce thread), or come up with a different solution. While it's fairly straightforward to skip visibility checks during traits lookups, I couldn't come up with a useful use-case while writing tests for that implementation, and after some more thought came up with the allMembers change.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Sunday, 4 September 2016 at 03:14:18 UTC, Martin Nowak wrote: It didn't slip, but I wish Walter had at least stated his opinion on the PR before merging. My thinking is that the plebes should be able to access things via the object.member syntax by obeying the usual visibility rules. But __traits(allMembers, T) should be the reflection backdoor that gives the savvy users total access, at the obvious cost of an awkward syntax. As explained several times here and in the announce thread, private members have never been accessible, other than introspecting attributes, and making them accessible comes with a performance cost and a fairly big language change. So the real question is, why do we need introspection without access, and can we handle that few cases with mixin templates. If we really need introspection of private members than we might need to go back to the drawing board and modify the visibility concept introduced with DIP22. While I do understand, that there could be a potential performance when private members could be changed around because they are not visible form outside. I fail to see how we would take advantage of that without breaking our object-model.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Saturday, 3 September 2016 at 20:40:57 UTC, Andrei Alexandrescu wrote: On 9/3/16 7:57 PM, Jonathan M Davis via Digitalmars-d wrote: Previously __traits(allMembers, T) listed _all_ members, whereas now it wil only list the ones that are accessible. The same for __traits(derivedMembers, T). So, they'll still give you the private members if you use them inside the module in question but not elsewhere. That... doesn't sound good. I wonder how such important changes slip by Walter and myself unnoticed. It didn't slip, but I wish Walter had at least stated his opinion on the PR before merging. My thinking is that the plebes should be able to access things via the object.member syntax by obeying the usual visibility rules. But __traits(allMembers, T) should be the reflection backdoor that gives the savvy users total access, at the obvious cost of an awkward syntax. As explained several times here and in the announce thread, private members have never been accessible, other than introspecting attributes, and making them accessible comes with a performance cost and a fairly big language change. So the real question is, why do we need introspection without access, and can we handle that few cases with mixin templates. If we really need introspection of private members than we might need to go back to the drawing board and modify the visibility concept introduced with DIP22.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 9/3/16 11:54 PM, Jacob Carlborg wrote: On 2016-09-03 22:40, Andrei Alexandrescu wrote: That... doesn't sound good. I wonder how such important changes slip by Walter and myself unnoticed. Here's the PR that introduced the change: https://github.com/dlang/dmd/pull/6078 Thanks. -- Andrei
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Saturday, 3 September 2016 at 21:54:24 UTC, Jacob Carlborg wrote: Here's the PR that introduced the change: https://github.com/dlang/dmd/pull/6078 I'm certainly not going to upgrade to the next DMD if this change is retained. allMembers not returning all members makes introspection entirely useless when it comes to Binderoo. The wrong conclusions were made from that bug to begin with it seems. allMembers should return all members. getProtection should report on the protection of a symbol *regardless* of whether getMember will succeed or not (this is currently why I have my PrivacyOf workaround - to stop the compiler crashing when using a template instead of a mixin template to do introspection). getMember itself, well, I'd honestly prefer if there was a way to get to members without having to correlate with .tupleof as it will simplify Binderoo code. The .tupleof method doesn't help me when it comes to introspecting private/protected functions for C++ binding.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 2016-09-03 22:40, Andrei Alexandrescu wrote: That... doesn't sound good. I wonder how such important changes slip by Walter and myself unnoticed. Here's the PR that introduced the change: https://github.com/dlang/dmd/pull/6078 -- /Jacob Carlborg
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Saturday, 3 September 2016 at 20:54:19 UTC, Andrei Alexandrescu wrote: On 9/3/16 10:50 PM, Basile B. wrote: On Saturday, 3 September 2016 at 20:40:57 UTC, Andrei Alexandrescu wrote: Martin, any chance we can undo this change to the language? The problem is deeper. If he undoes the change then what will happen again is that the results of allMembers won't be usable by the other traits, e.g getMember, because them are still limited by the protection attributes. Where are the past discussions on this matter? -- Andrei The change in 2.071.2-beta3 to allMembers is designed to fix https://issues.dlang.org/show_bug.cgi?id=15907 which happens because of a __traits(getMember,...) on the results of a __traits(allMembers,...).
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 9/3/16 10:50 PM, Basile B. wrote: On Saturday, 3 September 2016 at 20:40:57 UTC, Andrei Alexandrescu wrote: Martin, any chance we can undo this change to the language? The problem is deeper. If he undoes the change then what will happen again is that the results of allMembers won't be usable by the other traits, e.g getMember, because them are still limited by the protection attributes. Where are the past discussions on this matter? -- Andrei
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Saturday, 3 September 2016 at 20:40:57 UTC, Andrei Alexandrescu wrote: Martin, any chance we can undo this change to the language? The problem is deeper. If he undoes the change then what will happen again is that the results of allMembers won't be usable by the other traits, e.g getMember, because them are still limited by the protection attributes.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 9/3/16 7:57 PM, Jonathan M Davis via Digitalmars-d wrote: Previously __traits(allMembers, T) listed _all_ members, whereas now it wil only list the ones that are accessible. The same for __traits(derivedMembers, T). So, they'll still give you the private members if you use them inside the module in question but not elsewhere. That... doesn't sound good. I wonder how such important changes slip by Walter and myself unnoticed. My thinking is that the plebes should be able to access things via the object.member syntax by obeying the usual visibility rules. But __traits(allMembers, T) should be the reflection backdoor that gives the savvy users total access, at the obvious cost of an awkward syntax. The fact that __traits(allMembers, T) compiles in all cases and has DIFFERENT semantics depending on whether T is in the same vs. a different module is the deadly sign of poor language design. Martin, any chance we can undo this change to the language? Andrei
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 2016-09-03 20:56, Dicebot wrote: Obviously serialization libraries come to mind, especially important for binary serialization because private members still affect the layout of the struct. That can usually be solved using .tupleof[i], but I agree. -- /Jacob Carlborg
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 2016-09-03 17:50, Martin Nowak wrote: It will, e.g. having getMember bypass protection means vibe.d would now serialize private members https://github.com/rejectedsoftware/vibe.d/blob/c1180791de61d0f8c9bfb584c2551a5b64627e32/source/vibe/internal/meta/traits.d#L146. Until now getMember was just a DotIdExp (i.e. `T.indent`/`var.ident`), so it was used to test whether sth. can be accessed. Without looking at the how the rest of the code works, it already checks if a member is public or not [1]. Or are you saying that "isPublicMember" would return a different value? [1] https://github.com/rejectedsoftware/vibe.d/blob/c1180791de61d0f8c9bfb584c2551a5b64627e32/source/vibe/internal/meta/traits.d#L159 -- /Jacob Carlborg
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 09/03/2016 07:52 PM, Martin Nowak wrote: > On Tuesday, 30 August 2016 at 22:24:12 UTC, Ali Çehreli wrote: >> I don't agree with the current solution: > > Well let's come up with a better solution then. > Let's start by finding some proper use-cases that require introspection > on private members w/o having access to them. Obviously serialization libraries come to mind, especially important for binary serialization because private members still affect the layout of the struct. signature.asc Description: OpenPGP digital signature
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Saturday, September 03, 2016 19:30:54 Andrei Alexandrescu via Digitalmars-d wrote: > On 9/3/16 6:39 PM, Andrej Mitrovic via Digitalmars-d wrote: > > On 9/3/16, Martin Nowak via Digitalmars-d wrote: > >> On Wednesday, 31 August 2016 at 13:12:30 UTC, Adam D. Ruppe wrote: > >>> Ugh, it really should just give everything and have getMember > >>> bypass it. That won't even break any code! > >> > >> It will, e.g. having getMember bypass protection means vibe.d > >> would now serialize private members > > > > Then just add a check in vibe.d itself to avoid serializing private > > members. You can still call getProtection on the symbol and skip > > serializing it. > > > > Alternatively you can use UDAs so you can mark which fields should or > > shouldn't be serializible. For example > > https://github.com/msgpack/msgpack-d/blob/6046808c2e678e27cb2e2d9314241c36 > > 1a6fd0ae/src/msgpack/attribute.d#L21 > > > > The bottom line is with restricting access to private symbols you have > > no choice on the matter, while allowing access lets you specialize > > whether to ignore the symbols or not. > > I didn't follow this closely, but clearly we need some means to access > private members for introspection purposes. I hope we don't paint > ourselves out of that corner. -- Andrei Well, that certainly seems to be exactly what we're doing with the changes that Ali is complaining about. Previously __traits(allMembers, T) listed _all_ members, whereas now it wil only list the ones that are accessible. The same for __traits(derivedMembers, T). So, they'll still give you the private members if you use them inside the module in question but not elsewhere. I would have thought that the way to handle this would be to have them list all members regardless of access level and then to have code check for the access level if it cares. I would have thought that doing stuff like looking at private symbols would be perfectly okay with no errors or deprecation warnings and that it would just be _accessing_ the symbol that would be disallowed and would require you to check for in code that used introspection. But clearly, there must be something that I misunderstand here, because that's not at all where this seems to be going. I do get the feeling that too much of what we do with features like this is ad-hoc without really designing things up front, and then we keep having to tweak stuff as we go along in ways that aren't always particularly nice. And this particular change seems to just be a reaction to problems that we ran into with changing how imports work. And maybe it's the right choice, but it sure doesn't seem like it. - Jonathan M Davis
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 9/3/16 6:39 PM, Andrej Mitrovic via Digitalmars-d wrote: On 9/3/16, Martin Nowak via Digitalmars-d wrote: On Wednesday, 31 August 2016 at 13:12:30 UTC, Adam D. Ruppe wrote: Ugh, it really should just give everything and have getMember bypass it. That won't even break any code! It will, e.g. having getMember bypass protection means vibe.d would now serialize private members Then just add a check in vibe.d itself to avoid serializing private members. You can still call getProtection on the symbol and skip serializing it. Alternatively you can use UDAs so you can mark which fields should or shouldn't be serializible. For example https://github.com/msgpack/msgpack-d/blob/6046808c2e678e27cb2e2d9314241c361a6fd0ae/src/msgpack/attribute.d#L21 The bottom line is with restricting access to private symbols you have no choice on the matter, while allowing access lets you specialize whether to ignore the symbols or not. I didn't follow this closely, but clearly we need some means to access private members for introspection purposes. I hope we don't paint ourselves out of that corner. -- Andrei
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Saturday, 3 September 2016 at 16:52:50 UTC, Martin Nowak wrote:
On Tuesday, 30 August 2016 at 22:24:12 UTC, Ali Çehreli wrote:
I don't agree with the current solution:
Well let's come up with a better solution then.
Let's start by finding some proper use-cases that require
introspection on private members w/o having access to them.
In my user library, the serialization is based on the @Set and
@Get UDAs. Typically there would be a setter (a public method)
and no getter (i.e the data is read directly either from a
protected or private variable).
The introspection is used to create a property descriptor. But
because of the protection attributes I have to mix the
introspection features (== a template) in each aggregate that
declares properties and in each descendant class that declare new
properties.
Class Foo
{
mixin features f; // so that the private stuff are visible
private:
@Get int _field;
public
@Set void field(int value){}
this()
{f.analyze!Foo;}
}
class Bar: Foo
{
mixin features f; // so that the private stuff are visible
private:
@Get int _otherfield;
public
@Set void otherfield(int value){}
this()
{f.analyze!Bar;}
}
while I could do, with the omniscient traits:
Class Foo
{
private:
@Get int _field;
public
@Set void field(int value){}
this(this T)()
{
features.analyze!T;
}
}
Class Bar: Foo
{
private:
@Get int _otherfield;
public
@Set void otherfield(int value){}
}
no need to re-mix the features for each derived class...
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Tuesday, 30 August 2016 at 22:24:12 UTC, Ali Çehreli wrote: I don't agree with the current solution: Well let's come up with a better solution then. Let's start by finding some proper use-cases that require introspection on private members w/o having access to them.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 9/3/16, Martin Nowak via Digitalmars-d wrote: > On Wednesday, 31 August 2016 at 13:12:30 UTC, Adam D. Ruppe wrote: >> Ugh, it really should just give everything and have getMember >> bypass it. That won't even break any code! > > It will, e.g. having getMember bypass protection means vibe.d > would now serialize private members Then just add a check in vibe.d itself to avoid serializing private members. You can still call getProtection on the symbol and skip serializing it. Alternatively you can use UDAs so you can mark which fields should or shouldn't be serializible. For example https://github.com/msgpack/msgpack-d/blob/6046808c2e678e27cb2e2d9314241c361a6fd0ae/src/msgpack/attribute.d#L21 The bottom line is with restricting access to private symbols you have no choice on the matter, while allowing access lets you specialize whether to ignore the symbols or not.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Wednesday, 31 August 2016 at 21:12:54 UTC, Ali Çehreli wrote:
On 08/30/2016 03:24 PM, Ali Çehreli wrote:
> v2.071.2-b3 is bringing a change for this bug:
>
> https://issues.dlang.org/show_bug.cgi?id=15907
>
> I don't agree with the current solution:
>
>
http://dlang.org/changelog/2.071.2.html#traits-members-visibility
>
> Modules should be able to use library templates without
needing to mix
> them in first.
>
> Do you think the solution in the change log usable? I don't
think so
> because silently skipping my private members is an unexpected
behavior
> that will cause bugs.
Here is a regression caused by the above change:
interface I {
void foo();
package:
void foo(int);
}
string how(alias Base, alias func)() {
return "";
}
import std.typecons;
class C : AutoImplement!(I, how) {
}
What this derives a class in std.typecons that implements the
interface, and yes such a derived class can't override/implement
package protected methods.
AutoImplement would also work better as mixin in order to
instantiate the passed in how template in the original
instantiation scope.
void main() {
auto c = new C();
c.foo();
c.foo(42);// COMPILATION ERROR:
// deneme.o: In function `_Dmain':
// deneme.d:(.text._Dmain+0x39): undefined reference to
`_D6deneme1I3fooMFiZv'
The missing interface method implementation should've give a
compile error, please file a bug.
The recommended solution of mixing in every template instance
is not a viable solution because that would effectively remove
IFTI from D. What a huge loss that would be. We can't afford
that.
Exaggeration won't help us to find good solutions. Remember that
private fields never were accessible, so only some edge cases
will be affected. The need for mixin templates will remain rare.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Saturday, 3 September 2016 at 15:48:24 UTC, ketmar wrote: just a wrapper class, which will hold the actual instantiation and a scope. most of the code should "pass thru" the wrapper (i thing that `alias this` can be used for that, but have to check it), yet `__traits` can use additional info. sure, that will require some work, but it's not that impossible, and should not ruin caching. yeah, we will waste additional 16/32 bytes per template instance this way. not that much, and it will solve the problem in most natural way. Well, if you're making a difference based on the instantiation scope inside the template, then you're are leaking it into the template, do require a new instance for every instantiation, are ruining caching, do require unique mangling and redundant codegen. We have mixin templates exactly for the purpose of instantiating them in the origin scope.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Thursday, 1 September 2016 at 19:30:41 UTC, Basile B. wrote: https://github.com/dlang/DIPs/pull/39 Co-authors welcome. Slow down a bit until we've finally decided out how to resolve the problems.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Wednesday, 31 August 2016 at 13:12:30 UTC, Adam D. Ruppe wrote: Ugh, it really should just give everything and have getMember bypass it. That won't even break any code! It will, e.g. having getMember bypass protection means vibe.d would now serialize private members https://github.com/rejectedsoftware/vibe.d/blob/c1180791de61d0f8c9bfb584c2551a5b64627e32/source/vibe/internal/meta/traits.d#L146. Until now getMember was just a DotIdExp (i.e. `T.indent`/`var.ident`), so it was used to test whether sth. can be accessed.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Saturday, 3 September 2016 at 14:48:54 UTC, Martin Nowak wrote: On Wednesday, 31 August 2016 at 05:33:50 UTC, ketmar wrote: all this mess should be resolved in compiler by assigning template *two* visibility scopes: one is template's "normal" scope (so it can see symbols from it's originating module), and second is it's "instantiation" scope. after all, this is exactly what programmer is exepecting. current solution is not a solution at all, it's a hacky workaround promoted to "official technique". This would require a new instance for every template instantiation. The instantiation scope does not leak into the template by design, it allows us to cache instantiations and greatly speed up compilation. just a wrapper class, which will hold the actual instantiation and a scope. most of the code should "pass thru" the wrapper (i thing that `alias this` can be used for that, but have to check it), yet `__traits` can use additional info. sure, that will require some work, but it's not that impossible, and should not ruin caching. yeah, we will waste additional 16/32 bytes per template instance this way. not that much, and it will solve the problem in most natural way.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Tuesday, 30 August 2016 at 22:24:12 UTC, Ali Çehreli wrote: I can't wrap my head around the fact that a library template called by my module cannot see my private members. Well, it was never possible to access them either, didn't seem to cause much confusion. Also getSymbolsByUDA is built on a hack around that access check https://github.com/dlang/phobos/blob/cb09746cb11bcbe7b730f05d29792e6252669175/std/traits.d#L6979. Note that you can pass private symbols to templates, so `hasUDA!(S.invisible, UDA)` works fine. I don't see any other examples where we'll have much issues with that change other than w/ this weird getSymbolsByUDA API, which aliases to !(S.field, S.func, S.Nested). Most templates operating on wholes types already didn't have access to private members.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Wednesday, 31 August 2016 at 05:33:50 UTC, ketmar wrote: all this mess should be resolved in compiler by assigning template *two* visibility scopes: one is template's "normal" scope (so it can see symbols from it's originating module), and second is it's "instantiation" scope. after all, this is exactly what programmer is exepecting. current solution is not a solution at all, it's a hacky workaround promoted to "official technique". This would require a new instance for every template instantiation. The instantiation scope does not leak into the template by design, it allows us to cache instantiations and greatly speed up compilation.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Wednesday, 31 August 2016 at 13:29:52 UTC, Basile B. wrote: On Wednesday, 31 August 2016 at 13:12:30 UTC, Adam D. Ruppe wrote: Ugh, it really should just give everything and have getMember bypass it. That won't even break any code! you're right. "allMembers" means "all" after all. Another reason why the idea of "allVisibleMembers" is good. Puristes will be able to use this traits without messing with "getProtection". https://github.com/dlang/DIPs/pull/39 Co-authors welcome.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 08/30/2016 03:24 PM, Ali Çehreli wrote:
> v2.071.2-b3 is bringing a change for this bug:
>
> https://issues.dlang.org/show_bug.cgi?id=15907
>
> I don't agree with the current solution:
>
> http://dlang.org/changelog/2.071.2.html#traits-members-visibility
>
> Modules should be able to use library templates without needing to mix
> them in first.
>
> Do you think the solution in the change log usable? I don't think so
> because silently skipping my private members is an unexpected behavior
> that will cause bugs.
Here is a regression caused by the above change:
interface I {
void foo();
package:
void foo(int);
}
string how(alias Base, alias func)() {
return "";
}
import std.typecons;
class C : AutoImplement!(I, how) {
}
void main() {
auto c = new C();
c.foo();
c.foo(42);// COMPILATION ERROR:
// deneme.o: In function `_Dmain':
// deneme.d:(.text._Dmain+0x39): undefined reference to
`_D6deneme1I3fooMFiZv'
// collect2: error: ld returned 1 exit status
}
WAT?
It's not reasonable for the user to somehow suspect that AutoImplement
may be using a D feature (__traits(allMembers) in this case) which
happens to not see foo(int). (Note that AutoImplement may see foo(int)
today but not tomorrow, if it's moved out of this package.)
The recommended solution of mixing in every template instance is not a
viable solution because that would effectively remove IFTI from D. What
a huge loss that would be. We can't afford that.
So, to be safe, every D code out there that happens to pass a struct to
a piece of library function would have to
1) Know that that library function happens to be a template and
2) mixin the particular instantiation of that template.
That would be the only sane thing to do. Further, what happens if a
function changes its implementation and becomes a template that happens
to call __traits(allMembers)? How can the library notify its users to
tell them to mixin an explicit instantiation?
Ali
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Tuesday, August 30, 2016 15:24:12 Ali Çehreli via Digitalmars-d wrote: > v2.071.2-b3 is bringing a change for this bug: > >https://issues.dlang.org/show_bug.cgi?id=15907 > > I don't agree with the current solution: > >http://dlang.org/changelog/2.071.2.html#traits-members-visibility > > Modules should be able to use library templates without needing to mix > them in first. Agreed. Having to mix stuff in for introspection is downright ugly. It makes far more sense to provide everything and then check the attributes as discussed elsewhere in this thread. IMHO, having allMembers give different results depending on access level is just plain broken. I confess that I see no problem being able to examine symbols that are not accessible due to the access level. They just shouldn't be usable or be involved in overload sets. Looking at their declarations and attributes and whatnot should be fine. - Jonathan M Davis
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Wednesday, 31 August 2016 at 13:12:30 UTC, Adam D. Ruppe wrote: Ugh, it really should just give everything and have getMember bypass it. That won't even break any code! you're right. "allMembers" means "all" after all. Another reason why the idea of "allVisibleMembers" is good. Puristes will be able to use this traits without messing with "getProtection".
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Wednesday, 31 August 2016 at 08:33:28 UTC, Ethan Watson wrote: That's how it used to work, but getProtection would fail if the symbol wasn't public. Which led to me using a workaround to something of this effect: Yeah, I kinda regret the design of getProtection (which is basically 100% my fault, I implemented it myself and pushed it through without thinking about private - I was only interested in public vs export for my personal use case...), but if getMember worked on private things too, getProtection would never have to return inaccessible and it would be pretty again.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
Ugh, it really should just give everything and have getMember bypass it. That won't even break any code!
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Wednesday, 31 August 2016 at 09:30:43 UTC, Ethan Watson wrote: I'm okay with this. My PrivacyLevel workaround does exactly this in fact. I keep forgetting that I'm all open sourced now and can just link directly to the full example. https://github.com/Remedy-Entertainment/binderoo/blob/master/binderoo_client/d/src/binderoo/objectprivacy.d
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Wednesday, 31 August 2016 at 09:25:52 UTC, Basile B. wrote: nice idea, but this doesn't change the fact that the traits that access the results of the "omniscient" allMember must be tweaked to access everything. I'm okay with this. My PrivacyLevel workaround does exactly this in fact. But I would like to be able to read (and write) all members of a class without needing to mixin a template and without having to resort to .tupleof. Use case here is the extensive struct use we have, if we want them to match C++ exactly then that's where mixins become potentially hairy.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Wednesday, 31 August 2016 at 08:36:37 UTC, Ethan Watson wrote: On Tuesday, 30 August 2016 at 22:24:12 UTC, Ali Çehreli wrote: I don't agree with the current solution: I'm somewhat surprised myself that "allMembers doesn't return all members" needs highlighting. Why not have a new trait "allVisibleMembers" and just fix the privacy issues? nice idea, but this doesn't change the fact that the traits that access the results of the "omniscient" allMember must be tweaked to access everything.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Tuesday, 30 August 2016 at 22:24:12 UTC, Ali Çehreli wrote: I don't agree with the current solution: I'm somewhat surprised myself that "allMembers doesn't return all members" needs highlighting. Why not have a new trait "allVisibleMembers" and just fix the privacy issues?
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Wednesday, 31 August 2016 at 08:06:05 UTC, Basile B. wrote:
allow them to see everything, then use "getProtection" if you
wanna be conform with the protection attributes.
That's how it used to work, but getProtection would fail if the
symbol wasn't public. Which led to me using a workaround to
something of this effect:
enum PrivacyLevel : string
{
Public = "public",
Private = "private",
Protected = "protected",
Export = "export",
Package = "package",
Inaccessible= "inaccessible"
};
//
template PrivacyOf( alias symbol )
{
static if( __traits( compiles, __traits( getProtection, symbol )
) )
{
enum PrivacyOf = cast(PrivacyLevel) __traits( getProtection,
symbol );
}
else
{
enum PrivacyOf = PrivacyLevel.Inaccessible;
}
}
//
Still not an ideal solution - because if I'm trying to serialise
and deserialise everything in between module reloads I still need
to do the .tupleof method to get all data members; and if I want
to define privacy levels for functions I'm automatically binding
from C++ I need to muddy those waters with UDAs etc.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Tuesday, 30 August 2016 at 22:46:58 UTC, Basile B. wrote: On Tuesday, 30 August 2016 at 22:24:12 UTC, Ali Çehreli wrote: v2.071.2-b3 is bringing a change for this bug: Yes I agree, a change of the specifications in a dot release seems a bit extreme, especially since there was a deeper problem **before** the broken imports were fixed. This problem was discovered a while back when the library traits related to UDAs were added to phobos. The little story began here: https://issues.dlang.org/show_bug.cgi?id=15335. Then the proposal to give super powers to certain traits verbs: https://issues.dlang.org/show_bug.cgi?id=15371. To be clear, the logic I see for traits "getMember", "allMember", "getOverloads", "derivedMembers" (etc, all the traits that might be today limited by the protection attribute) is: allow them to see everything, then use "getProtection" if you wanna be conform with the protection attributes. Maybe it's worth a DIP ? Casual and informal discussions have **failed**. I see now the answer that mentions ".tupleof". I see no valid logic that would allow ".tupleof" to see everything and not the traits.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 2016-08-31 02:16, Andrej Mitrovic via Digitalmars-d wrote:
PSA: If all one cares about are UDAs for fields and not functions then
.tupleof is still a viable workaround. Working example:
-
import getSymbols;
enum UDA;
struct S
{
@UDA float visible;
float dont_care;
private @UDA int invisible;
}
void main()
{
static assert(getSymbolsByUDA!(S, UDA).length == 2);
static assert(getSymbolsByUDA!(S, UDA).stringof == "tuple(visible,
invisible)");
}
-
-
module getSymbols;
import std.meta : AliasSeq;
import std.traits : staticIndexOf;
template getSymbolsByUDA ( T, alias uda )
{
alias getSymbolsByUDA = getSymbolsByUDAImpl!(T, uda);
}
template getSymbolsByUDAImpl ( T, alias uda, size_t idx = 0 )
{
static if (idx + 1 < T.tupleof.length)
{
static if (hasUDA!(T.tupleof[idx], uda))
alias Field = AliasSeq!(T.tupleof[idx]);
else
alias Field = AliasSeq!();
alias getSymbolsByUDAImpl = AliasSeq!(Field,
getSymbolsByUDAImpl!(T, uda, idx + 1));
}
else
{
static if (hasUDA!(T.tupleof[idx], uda))
alias getSymbolsByUDAImpl = AliasSeq!(T.tupleof[idx]);
else
alias getSymbolsByUDAImpl = AliasSeq!();
}
}
template hasUDA ( alias field, alias uda )
{
enum hasUDA = staticIndexOf!(uda, __traits(getAttributes, field)) != -1;
}
-
This is the reason msgpack-d still works and wasn't broken by the change.
Ah, nice workaround. Last time I tried to use __traits(getAttributes)
with a "tupleof expression" it didn't work.
--
/Jacob Carlborg
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 2016-08-31 01:01, Andrej Mitrovic via Digitalmars-d wrote: This is such a bizarre workaround to be listed in the changelog since mixing in non-mixin templates is not an official feature (am I wrong?). Yes. Originally one could not use the "mixin" keyword in front of a template. There was no difference between mixin templates and non-mixin templates, on the declaration site. Later the language was changed to allow to put "mixin" in front of a template. That restricted all mixin templates to only be used as mixin templates. It's still possible to mixin non-mixin templates, most likely to avoid breaking existing code. -- /Jacob Carlborg
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Tuesday, 30 August 2016 at 22:24:12 UTC, Ali Çehreli wrote: yeah, the whole feature smells for me. the sole need of mixin hack indicates that something is very wrong here. i never ever needed that for normal D code. and now suddenly i have to remember that some thing is a template, that it needs mixing it to work properly, etc. all this mess should be resolved in compiler by assigning template *two* visibility scopes: one is template's "normal" scope (so it can see symbols from it's originating module), and second is it's "instantiation" scope. after all, this is exactly what programmer is exepecting. current solution is not a solution at all, it's a hacky workaround promoted to "official technique".
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 8/31/16, Ali Çehreli via Digitalmars-d wrote:
> v2.071.2-b3 is bringing a change for this bug:
>
>https://issues.dlang.org/show_bug.cgi?id=15907
PSA: If all one cares about are UDAs for fields and not functions then
.tupleof is still a viable workaround. Working example:
-
import getSymbols;
enum UDA;
struct S
{
@UDA float visible;
float dont_care;
private @UDA int invisible;
}
void main()
{
static assert(getSymbolsByUDA!(S, UDA).length == 2);
static assert(getSymbolsByUDA!(S, UDA).stringof == "tuple(visible,
invisible)");
}
-
-
module getSymbols;
import std.meta : AliasSeq;
import std.traits : staticIndexOf;
template getSymbolsByUDA ( T, alias uda )
{
alias getSymbolsByUDA = getSymbolsByUDAImpl!(T, uda);
}
template getSymbolsByUDAImpl ( T, alias uda, size_t idx = 0 )
{
static if (idx + 1 < T.tupleof.length)
{
static if (hasUDA!(T.tupleof[idx], uda))
alias Field = AliasSeq!(T.tupleof[idx]);
else
alias Field = AliasSeq!();
alias getSymbolsByUDAImpl = AliasSeq!(Field,
getSymbolsByUDAImpl!(T, uda, idx + 1));
}
else
{
static if (hasUDA!(T.tupleof[idx], uda))
alias getSymbolsByUDAImpl = AliasSeq!(T.tupleof[idx]);
else
alias getSymbolsByUDAImpl = AliasSeq!();
}
}
template hasUDA ( alias field, alias uda )
{
enum hasUDA = staticIndexOf!(uda, __traits(getAttributes, field)) != -1;
}
-
This is the reason msgpack-d still works and wasn't broken by the change.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On 8/31/16, Ali Çehreli via Digitalmars-d wrote: > mixin getSymbolsByUDA!(S, UDA) symbols; This is such a bizarre workaround to be listed in the changelog since mixing in non-mixin templates is not an official feature (am I wrong?). getSymbolsByUDA is a template, not a mixin template.
Re: Usability of "allMembers and derivedMembers traits now only return visible symbols"
On Tuesday, 30 August 2016 at 22:24:12 UTC, Ali Çehreli wrote:
v2.071.2-b3 is bringing a change for this bug:
https://issues.dlang.org/show_bug.cgi?id=15907
I don't agree with the current solution:
http://dlang.org/changelog/2.071.2.html#traits-members-visibility
Modules should be able to use library templates without needing
to mix them in first.
Do you think the solution in the change log usable? I don't
think so because silently skipping my private members is an
unexpected behavior that will cause bugs.
Further, do I understand the example right? Am I supposed to
mixin the same template twice for two different types? The
following code which adds another struct does not compile:
import std.stdio;
import std.traits;
enum UDA;
struct S
{
@UDA int visible;
@UDA private int invisible;
}
// only returns symbols visible from std.traits
static assert(getSymbolsByUDA!(S, UDA).length == 1);
// mixin the template instantiation, using a name to avoid
namespace pollution
mixin getSymbolsByUDA!(S, UDA) symbols;
// as the template is instantiated in the current scope, it can
see private members
static assert(symbols.getSymbolsByUDA.length == 2);
// --- The following is added by Ali: ---
struct S2 {
@UDA int s2;
}
mixin getSymbolsByUDA!(S2, UDA) symbolsS2;// COMPILATION
ERROR:
// Error: mixin deneme.getSymbolsByUDA!(S2, UDA) TList isn't a
template
static assert(symbolsS2.getSymbolsByUDA.length == 1);
void main() {
foreach (i; 0 .. symbols.getSymbolsByUDA.length) {
// ...
}
}
I can't wrap my head around the fact that a library template
called by my module cannot see my private members.
Ali
Yes I agree, a change of the specifications in a dot release
seems a bit extreme, especially since there was a deeper problem
**before** the broken imports were fixed.
This problem was discovered a while back when the library traits
related to UDAs were added to phobos.
The little story began here:
https://issues.dlang.org/show_bug.cgi?id=15335. Then the proposal
to give super powers to certain traits verbs:
https://issues.dlang.org/show_bug.cgi?id=15371.
