Re: Beta D 2.071.2-b3
On Sunday, 4 September 2016 at 08:57:15 UTC, Martin Nowak wrote: On Saturday, 3 September 2016 at 18:06:37 UTC, Dicebot wrote: There are enough ways to leak access to private entity (alias, template argument, taking address, some compile-time introspection) to make such optimizations impossible without extensive program analysis. OK, anything else that would be impaired by allowing access to private members? I think the unsafe requirement Walter asked for would be too much for the point release. We could create an intermediate solution that ignores visibility but still keep the access checks around (basically the old behavior) until most of them will get removed in a later release. I'm on vacation and travelling at the moment, so it might take a few days until I find time for the implementation.
Re: Beta D 2.071.2-b3
On Sunday, 4 September 2016 at 08:57:15 UTC, Martin Nowak wrote: On Saturday, 3 September 2016 at 18:06:37 UTC, Dicebot wrote: There are enough ways to leak access to private entity (alias, template argument, taking address, some compile-time introspection) to make such optimizations impossible without extensive program analysis. OK Agreed that such optimizations would require quite some front-end additions (keeping track if anything aliases or escapes the symbol), so it might not be worth with the current state of LTO.
Re: Beta D 2.071.2-b3
On Saturday, 3 September 2016 at 18:06:37 UTC, Dicebot wrote: There are enough ways to leak access to private entity (alias, template argument, taking address, some compile-time introspection) to make such optimizations impossible without extensive program analysis. OK, anything else that would be impaired by allowing access to private members? I think the unsafe requirement Walter asked for would be too much for the point release. We could create an intermediate solution that ignores visibility but still keep the access checks around (basically the old behavior) until most of them will get removed in a later release.
Re: Beta D 2.071.2-b3
On Saturday, 3 September 2016 at 18:06:37 UTC, Dicebot wrote: On Saturday, 3 September 2016 at 14:40:37 UTC, Martin Nowak wrote: On Wednesday, 31 August 2016 at 09:56:17 UTC, Johan Engelen wrote: (I can only think of complicated stuff that requires pretty much whole-program analysis to prove validity, and in that case adding `private` doesn't help any more) Yes, it does help. As private prevents usage outside of a module it allows to do some optimizations that required whole program analysis otherwise, e.g. variables and functions can get internal linkage, thus reducing loads/stores and indirect calls. This is a very naive viewpoint. There are enough ways to leak access to private entity (alias, template argument, taking address, some compile-time introspection) to make such optimizations impossible without extensive program analysis. Exactly.
Re: Beta D 2.071.2-b3
On 2016-09-03 18:02, Martin Nowak wrote: Why not just use `__traits(getAttributes, var.tupleof[0])`? I've already updated my code to use the above. When I first implemented it, it was not possible to use a "tupleof expression" as argument to __traits(getAttributes). -- /Jacob Carlborg
Re: Beta D 2.071.2-b3
On Saturday, 3 September 2016 at 14:40:37 UTC, Martin Nowak wrote: Yes, it does help. As private prevents usage outside of a module it allows to do some optimizations that required whole program analysis otherwise, e.g. variables and functions can get internal linkage, thus reducing loads/stores and indirect calls. This doesn't really work that easily due to templates that depend on private data, and private data being referenced from alias parameters. Of course, all such cases can still be statically detected when compiling the module in question, but this is quite a bit less trivial than just looking at the protection level. — David
Re: Beta D 2.071.2-b3
On Wednesday, 31 August 2016 at 06:20:46 UTC, Jacob Carlborg wrote: Class/struct fields are accessible using .tupleof. I was using __traits(getAttributes) in my serialization library to get UDA's for these fields, including private ones. Which is a weird implementation, b/c there is no direct alignment between allMembers and .tupleof. Why not just use `__traits(getAttributes, var.tupleof[0])`? Just serializing private state of types seems flawed too.
Re: Beta D 2.071.2-b3
On Wednesday, 31 August 2016 at 09:56:17 UTC, Johan Engelen wrote: (I can only think of complicated stuff that requires pretty much whole-program analysis to prove validity, and in that case adding `private` doesn't help any more) Yes, it does help. As private prevents usage outside of a module it allows to do some optimizations that required whole program analysis otherwise, e.g. variables and functions can get internal linkage, thus reducing loads/stores and indirect calls.
Re: Beta D 2.071.2-b3
On Friday, 2 September 2016 at 10:29:41 UTC, David Nadlinger wrote: On Friday, 2 September 2016 at 08:57:14 UTC, Basile B. wrote: On Friday, 2 September 2016 at 08:15:53 UTC, ketmar wrote: std.traits wrappers should use __traits to build *safe* things (declaring that @trusted in the end). This has nothing to do with memory safety. Actually it does, albeit somewhat tangentially: Accessing private members from the outside must be @system since the @trusted implementation of a class might rely on nobody meddling with the private state. — David The essence of the problem is that an aggregate cannot fully introspect itself using a template located in another module. If the @system constraint is only for the private members then it's ok.
Re: Beta D 2.071.2-b3
On Fri, 02 Sep 2016 09:40:56 +, ketmar wrote: > On Friday, 2 September 2016 at 08:57:14 UTC, Basile B. wrote: >> On Friday, 2 September 2016 at 08:15:53 UTC, ketmar wrote: >>> std.traits wrappers should use __traits to build *safe* things >>> (declaring that @trusted in the end). >> >> This has nothing to do with memory safety. > > i wonder who told you that @safe code is *only* about "memory safety". Walter Bright, when @safe was being discussed, before it was implemented.
Re: Beta D 2.071.2-b3
On Friday, 2 September 2016 at 08:57:14 UTC, Basile B. wrote: On Friday, 2 September 2016 at 08:15:53 UTC, ketmar wrote: std.traits wrappers should use __traits to build *safe* things (declaring that @trusted in the end). This has nothing to do with memory safety. Actually it does, albeit somewhat tangentially: Accessing private members from the outside must be @system since the @trusted implementation of a class might rely on nobody meddling with the private state. — David
Re: Beta D 2.071.2-b3
On Fri, Sep 2, 2016 at 10:15 AM, ketmar via Digitalmars-d-announce < digitalmars-d-announce@puremagic.com> wrote: > On Friday, 2 September 2016 at 07:46:30 UTC, Rory McGuire wrote: > actually, from my PoV solution is supereasy: just remove ALL visibility > restrictions for traits. and i mean all. allMembers should return all > members, getMember should allow to access *any* existing member without > annoying messages. it is up to programmer to insert getProtection checks if > he needs it. > > traits is a low-level feature by definition, and doesn't meant to be used > directly (we have std.traits wrappers for that), so i want D devs to stop > making our, low-level coders, life harder than it is now. ;-) > > D devs just should draw the line between __traits and std.traits. first > sould be inherently unsafe, allow *everything* and impose as little > restrictions as it is possible (note: *as* *possible*, not *as* *sane* -- > all kind of insanity should be allowed, if it is possible). then, > std.traits wrappers should use __traits to build *safe* things (declaring > that @trusted in the end). > May our benevolent dictators agree with you :D (I do). If a developer is willing to research the language definition and discover __traits, you should be ready for unprotected intimacy with the hardware of your choice. And if someone just copy pastes code with __traits in it they should know that "__" in a symbol is a "WARNING here be dragons"
Re: Beta D 2.071.2-b3
On Friday, 2 September 2016 at 08:57:14 UTC, Basile B. wrote: On Friday, 2 September 2016 at 08:15:53 UTC, ketmar wrote: std.traits wrappers should use __traits to build *safe* things (declaring that @trusted in the end). This has nothing to do with memory safety. i wonder who told you that @safe code is *only* about "memory safety".
Re: Beta D 2.071.2-b3
On Friday, 2 September 2016 at 08:15:53 UTC, ketmar wrote: std.traits wrappers should use __traits to build *safe* things (declaring that @trusted in the end). This has nothing to do with memory safety. It's just that protection attributes were invented for OOP and when applied to template meta programming they suck a bit.
Re: Beta D 2.071.2-b3
On Friday, 2 September 2016 at 07:46:30 UTC, Rory McGuire wrote: actually, from my PoV solution is supereasy: just remove ALL visibility restrictions for traits. and i mean all. allMembers should return all members, getMember should allow to access *any* existing member without annoying messages. it is up to programmer to insert getProtection checks if he needs it. traits is a low-level feature by definition, and doesn't meant to be used directly (we have std.traits wrappers for that), so i want D devs to stop making our, low-level coders, life harder than it is now. ;-) D devs just should draw the line between __traits and std.traits. first sould be inherently unsafe, allow *everything* and impose as little restrictions as it is possible (note: *as* *possible*, not *as* *sane* -- all kind of insanity should be allowed, if it is possible). then, std.traits wrappers should use __traits to build *safe* things (declaring that @trusted in the end).
Re: Beta D 2.071.2-b3
On Fri, Sep 2, 2016 at 8:47 AM, ketmar via Digitalmars-d-announce <
digitalmars-d-announce@puremagic.com> wrote:
> On Friday, 2 September 2016 at 06:27:11 UTC, Rory McGuire wrote:
>
>> Perhaps @system code should just completely ignore privacy?
>>
>
> it is uncontrollable. imagine attribute inference: today, your function
> was inferred @system, and it sees everything. and tomorrow you fixed some
> other things, and now your function inferred as @safe. BOOM! without
> warning, it doesn't do what it does before (but still works!), and you
> didn't even touched it's code. this is the worst kind of breakage.
I'm meaning if the dev marks the whole module as @system. I'm not convinced
that _any_ code should ever be inferred as @system. Its not okay for people
to accidentally make @system code.
>
>
> Could have a compiler option that changes the access errors into warnings?
>>
>
> each compiler option of this kind means "we failed our design task,
> brought you the feature that is so unusable that we even made it possible
> to turn it off globally. sorry, we giving up, now it is up to you to clean
> up the mess after us."
>
You may have a point there, even if its a bit excessive. I would like this
as a pragma, but then that leads us down the road of not even bothering to
change the compiler and just use a analysis tool.
mmm, even if we did make private access illegal we can still access
whatever we want if we have the source code so...
e.g. mixin(import(moduleName!exampleSymbol).replace("private", "public"));
not sure it would always work...
Re: Beta D 2.071.2-b3
On Friday, 2 September 2016 at 06:27:11 UTC, Rory McGuire wrote: Perhaps @system code should just completely ignore privacy? it is uncontrollable. imagine attribute inference: today, your function was inferred @system, and it sees everything. and tomorrow you fixed some other things, and now your function inferred as @safe. BOOM! without warning, it doesn't do what it does before (but still works!), and you didn't even touched it's code. this is the worst kind of breakage. Could have a compiler option that changes the access errors into warnings? each compiler option of this kind means "we failed our design task, brought you the feature that is so unusable that we even made it possible to turn it off globally. sorry, we giving up, now it is up to you to clean up the mess after us."
Re: Beta D 2.071.2-b3
On 02 Sep 2016 07:40, "ketmar via Digitalmars-d-announce" < digitalmars-d-announce@puremagic.com> wrote: > > On Tuesday, 30 August 2016 at 23:54:45 UTC, Martin Nowak wrote: >> >> Allowing access to private members has a lot of implications, e.g. breaks lots of optimizations b/c you can't know who accesses sth. > > > i really HATE modern trend of turning tables. am i the only one who thinks that the machine was designed to serve the human, not vice versa? yet somehow we all trying to make our machines happy now, instead of using 'em. > > the most funny thing with it is that modern software is bloated, dog slow, resource hungry and barely usable. so all this "please the machine" movement is completely pointless! Perhaps @system code should just completely ignore privacy? Could have a compiler option that changes the access errors into warnings? Then make a separate tool that we can use to do assertions on that code. For example or company could have a policy that any @system code has 100% test coverage. Just thinking it loud. R
Re: Beta D 2.071.2-b3
On Tuesday, 30 August 2016 at 23:54:45 UTC, Martin Nowak wrote: Allowing access to private members has a lot of implications, e.g. breaks lots of optimizations b/c you can't know who accesses sth. i really HATE modern trend of turning tables. am i the only one who thinks that the machine was designed to serve the human, not vice versa? yet somehow we all trying to make our machines happy now, instead of using 'em. the most funny thing with it is that modern software is bloated, dog slow, resource hungry and barely usable. so all this "please the machine" movement is completely pointless!
Re: Beta D 2.071.2-b3
On Thursday, 1 September 2016 at 20:46:50 UTC, Steven Schveighoffer wrote: On 8/31/16 5:56 AM, Johan Engelen wrote: On Tuesday, 30 August 2016 at 23:54:45 UTC, Martin Nowak wrote: Allowing access to private members has a lot of implications, e.g. breaks lots of optimizations b/c you can't know who accesses sth. "lots of optimizations" Can you mention a few? I'm not sure optimizations are had by private variables. Some optimizations can be had because the compiler can prove a variable could not have been changed. But I don't know that private suits that need, it's more due to const or immutable. Yep. I was trying to be polite :-)
Re: Beta D 2.071.2-b3
On 8/31/16 5:56 AM, Johan Engelen wrote: On Tuesday, 30 August 2016 at 23:54:45 UTC, Martin Nowak wrote: Allowing access to private members has a lot of implications, e.g. breaks lots of optimizations b/c you can't know who accesses sth. "lots of optimizations" Can you mention a few? I'm not sure optimizations are had by private variables. Some optimizations can be had because the compiler can prove a variable could not have been changed. But I don't know that private suits that need, it's more due to const or immutable. (I can only think of complicated stuff that requires pretty much whole-program analysis to prove validity, and in that case adding `private` doesn't help any more) Where I see private helping is conceptual -- I know that if a variable is private, I can narrow to a given module all the places the variable might be read or written. If you just allow private access via anywhere, it defeats that analysis. -Steve
Re: Beta D 2.071.2-b3
On 08/31/2016 02:08 AM, Martin Nowak wrote: > On Tuesday, 30 August 2016 at 21:58:05 UTC, Basile B. wrote: >> I'm a bit sad to see that >> https://issues.dlang.org/show_bug.cgi?id=15371 was completely ignored >> to fix issue 15907. Another decision could have been to break the >> visibility for the traits allMember, getMember, derivedMember and >> getOverloads. > > Well there was reasoning to choose that solution instead of the other > (https://github.com/dlang/dmd/pull/6078) and the fact that private > members aren't accessible (set/get) is a good indication that nobody > needs this. > Adding an unsafe facility to access private members is a separate > problem, but please see the changelog for how to achieve this already by > mixing in templates. I think such change is not appropriate in a point release, not matter if it is going to happen or not. signature.asc Description: OpenPGP digital signature
Re: Beta D 2.071.2-b3
On 2016-08-31 15:51, Chris Wright wrote: And `instance_variable_get` in Ruby. Or "send", "instance_eval" and so on. In Ruby it's more of a comment, "please do call this method directly" :) -- /Jacob Carlborg
Re: Beta D 2.071.2-b3
On Wed, 31 Aug 2016 08:22:31 +0200, Jacob Carlborg wrote: > On 2016-08-31 02:04, Ali Çehreli wrote: > >> P.S. While I'm on my soapbox, I've started to think private is >> overrated anyway. A system language should allow to bypass that >> protection. private should be a recommendation only. > > I agree. Let private stop you from access symbols though the regular > ways, but let reflection bypass. Just like the JVM and .NET. And Python's double underscore name mangling, which is more to highlight that you're doing something funky. And Dart's mirrors. And `instance_variable_get` in Ruby.
Re: Beta D 2.071.2-b3
On Tuesday, 30 August 2016 at 23:54:45 UTC, Martin Nowak wrote: Allowing access to private members has a lot of implications, e.g. breaks lots of optimizations b/c you can't know who accesses sth. "lots of optimizations" Can you mention a few? (I can only think of complicated stuff that requires pretty much whole-program analysis to prove validity, and in that case adding `private` doesn't help any more) -Johan
Re: Beta D 2.071.2-b3
On 2016-08-31 08:20, Jacob Carlborg wrote: Class/struct fields are accessible using .tupleof. I was using __traits(getAttributes) in my serialization library to get UDA's for these fields, including private ones. I think this was introduced already in 2.071.0. -- /Jacob Carlborg
Re: Beta D 2.071.2-b3
On 2016-08-31 01:08, Martin Nowak wrote: Well there was reasoning to choose that solution instead of the other (https://github.com/dlang/dmd/pull/6078) and the fact that private members aren't accessible (set/get) is a good indication that nobody needs this. Adding an unsafe facility to access private members is a separate problem, but please see the changelog for how to achieve this already by mixing in templates. In addition to using .tupleof to bypass protection it's possible to return a pointer to the symbol to bypass protection as well, fixing that would require some serious flow analysis. It's also possible to inspect the symbol table to access private symbols at runtime. -- /Jacob Carlborg
Re: Beta D 2.071.2-b3
On 2016-08-31 02:04, Ali Çehreli wrote: P.S. While I'm on my soapbox, I've started to think private is overrated anyway. A system language should allow to bypass that protection. private should be a recommendation only. I agree. Let private stop you from access symbols though the regular ways, but let reflection bypass. -- /Jacob Carlborg
Re: Beta D 2.071.2-b3
On 2016-08-31 01:08, Martin Nowak wrote: Well there was reasoning to choose that solution instead of the other (https://github.com/dlang/dmd/pull/6078) and the fact that private members aren't accessible (set/get) is a good indication that nobody needs this. Class/struct fields are accessible using .tupleof. I was using __traits(getAttributes) in my serialization library to get UDA's for these fields, including private ones. -- /Jacob Carlborg
Re: Beta D 2.071.2-b3
On Wednesday, 31 August 2016 at 05:23:34 UTC, ketmar wrote: On Tuesday, 30 August 2016 at 23:54:45 UTC, Martin Nowak wrote: As in needs private members in __traits(allMembers). and in the end it just producing excessive noise in common loops that does `is(typeof(__traits(getMember, ...)))`. it would be better to either leave that alone and allow all access, or exclude inaccessible members gin `allMembers` completely. yet D choose the third way, combining the worst features from both solutions. oops. seems that i did something wrong with myself. i'm sorry, the PR you linked seems to do exactly what i described. dunno how i managed to completely misunderstand it.
Re: Beta D 2.071.2-b3
On Tuesday, 30 August 2016 at 23:54:45 UTC, Martin Nowak wrote: As in needs private members in __traits(allMembers). and in the end it just producing excessive noise in common loops that does `is(typeof(__traits(getMember, ...)))`. it would be better to either leave that alone and allow all access, or exclude inaccessible members gin `allMembers` completely. yet D choose the third way, combining the worst features from both solutions.
Re: Beta D 2.071.2-b3
On 08/30/2016 04:54 PM, Martin Nowak wrote: > On Tuesday, 30 August 2016 at 23:08:58 UTC, Martin Nowak wrote: >> On Tuesday, 30 August 2016 at 21:58:05 UTC, Basile B. wrote: >>> I'm a bit sad to see that >>> https://issues.dlang.org/show_bug.cgi?id=15371 was completely ignored >>> to fix issue 15907. Another decision could have been to break the >>> visibility for the traits allMember, getMember, derivedMember and >>> getOverloads. >> >> Well there was reasoning to choose that solution instead of the other >> (https://github.com/dlang/dmd/pull/6078) and the fact that private >> members aren't accessible (set/get) is a good indication that nobody >> needs this. > > As in needs private members in __traits(allMembers). > >> Adding an unsafe facility to access private members is a separate >> problem, but please see the changelog for how to achieve this already >> by mixing in templates. > > Allowing access to private members has a lot of implications, e.g. > breaks lots of optimizations b/c you can't know who accesses sth. It's > also yet unclear what effect this has on @safe and we'd not only have to > modify visibility but remove the access checks as well. This is clearly > too much to just fix that problem w/ certain template instantiations, > none of which should work on private members atm. > Also you'd have to change a lot of existing code, or it would now just > operate on private members. How is this supposed to work? What is the new guideline and who should bear the new guideline; the library author or the user? For this to work, the users must know that the library will call __traits(allMembers) so that they mixin and otherwise don't? That would be wrong in the other direction, no? The client needing to know about the library code? Wrong principle... What if I add a new private member to my otherwise public struct? Then I should remember to mixin all N templates that my module happens to be using right now? The only correct action that I see is that every template must be mixed-in in case I have a private member in the future or the library implementation may change in the future. I'm sorry but this is not usable. Ali P.S. While I'm on my soapbox, I've started to think private is overrated anyway. A system language should allow to bypass that protection. private should be a recommendation only.
Re: Beta D 2.071.2-b3
On Tuesday, 30 August 2016 at 23:08:58 UTC, Martin Nowak wrote: On Tuesday, 30 August 2016 at 21:58:05 UTC, Basile B. wrote: I'm a bit sad to see that https://issues.dlang.org/show_bug.cgi?id=15371 was completely ignored to fix issue 15907. Another decision could have been to break the visibility for the traits allMember, getMember, derivedMember and getOverloads. Well there was reasoning to choose that solution instead of the other (https://github.com/dlang/dmd/pull/6078) and the fact I don't understand, which was the other ? the link points to the PR that's been merged.
Re: Beta D 2.071.2-b3
On Tuesday, 30 August 2016 at 23:08:58 UTC, Martin Nowak wrote: On Tuesday, 30 August 2016 at 21:58:05 UTC, Basile B. wrote: I'm a bit sad to see that https://issues.dlang.org/show_bug.cgi?id=15371 was completely ignored to fix issue 15907. Another decision could have been to break the visibility for the traits allMember, getMember, derivedMember and getOverloads. Well there was reasoning to choose that solution instead of the other (https://github.com/dlang/dmd/pull/6078) and the fact that private members aren't accessible (set/get) is a good indication that nobody needs this. As in needs private members in __traits(allMembers). Adding an unsafe facility to access private members is a separate problem, but please see the changelog for how to achieve this already by mixing in templates. Allowing access to private members has a lot of implications, e.g. breaks lots of optimizations b/c you can't know who accesses sth. It's also yet unclear what effect this has on @safe and we'd not only have to modify visibility but remove the access checks as well. This is clearly too much to just fix that problem w/ certain template instantiations, none of which should work on private members atm. Also you'd have to change a lot of existing code, or it would now just operate on private members.
Re: Beta D 2.071.2-b3
On Tuesday, 30 August 2016 at 21:58:05 UTC, Basile B. wrote: I'm a bit sad to see that https://issues.dlang.org/show_bug.cgi?id=15371 was completely ignored to fix issue 15907. Another decision could have been to break the visibility for the traits allMember, getMember, derivedMember and getOverloads. Well there was reasoning to choose that solution instead of the other (https://github.com/dlang/dmd/pull/6078) and the fact that private members aren't accessible (set/get) is a good indication that nobody needs this. Adding an unsafe facility to access private members is a separate problem, but please see the changelog for how to achieve this already by mixing in templates.
Re: Beta D 2.071.2-b3
On 08/30/2016 02:58 PM, Basile B. wrote: On Tuesday, 30 August 2016 at 19:37:25 UTC, Martin Nowak wrote: Third beta for the 2.071.2 release. This beta fixes spurious deprecation warnings with templates using getMember (Issue 15907), please read the changelog for more details. http://dlang.org/changelog/2.071.2.html http://dlang.org/download.html#dmd_beta Please report any bugs at https://issues.dlang.org -Martin I'm a bit sad to see that https://issues.dlang.org/show_bug.cgi?id=15371 was completely ignored to fix issue 15907. Another decision could have been to break the visibility for the traits allMember, getMember, derivedMember and getOverloads. +1 I've just opened a discussion thread on the main newsgroup about the (un)usability of this fix before seeing your comment. Ali
Re: Beta D 2.071.2-b3
On Tuesday, 30 August 2016 at 19:37:25 UTC, Martin Nowak wrote: Third beta for the 2.071.2 release. This beta fixes spurious deprecation warnings with templates using getMember (Issue 15907), please read the changelog for more details. http://dlang.org/changelog/2.071.2.html http://dlang.org/download.html#dmd_beta Please report any bugs at https://issues.dlang.org -Martin I'm a bit sad to see that https://issues.dlang.org/show_bug.cgi?id=15371 was completely ignored to fix issue 15907. Another decision could have been to break the visibility for the traits allMember, getMember, derivedMember and getOverloads.
Beta D 2.071.2-b3
Third beta for the 2.071.2 release. This beta fixes spurious deprecation warnings with templates using getMember (Issue 15907), please read the changelog for more details. http://dlang.org/changelog/2.071.2.html http://dlang.org/download.html#dmd_beta Please report any bugs at https://issues.dlang.org -Martin
