Re: Friends don't let friends use inout with scope and -dip1000
On Wednesday, 22 August 2018 at 14:05:10 UTC, Steven Schveighoffer wrote: But that's not valid dip1000 code. If you call it, it should give a compiler error (r *does* escape its scope). When I complained about C++ safety to my C++ programmer colleague, he told me that the compiler just compiles what's written. On one hand I can understand what he wanted to say, but the wording is still hilarious and fits C++ very well. What I like in D is that if it's written doesn't mean that it will compile: invalid code should be rejected.
Re: Friends don't let friends use inout with scope and -dip1000
On 8/22/18 4:17 AM, Kagamin wrote: On Tuesday, 21 August 2018 at 14:04:15 UTC, Steven Schveighoffer wrote: I would guess it's no different than other inferred attributes. I would also guess that it only gets promoted to a return parameter if it's actually returned. If we can't have properly typed parameters, it feels like it has potential to prevent some patterns. But scope is not part of the type, nor is return. One of my biggest concerns about dip1000 is that the "scope-ness" or "return-ness" of a variable is hidden from the type system. It's just the compiler doing flow analysis and throwing you an error when it can't work the thing out. I'm more worried about not being able to express the flow in a way that the compiler understands, and having it complain about things that are actually safe. This prevents automatic scope promotion: template escape(T) { int[] escape1(scope int[] r) { return r; } alias escape=escape1; } But that's not valid dip1000 code. If you call it, it should give a compiler error (r *does* escape its scope). -Steve
Re: Friends don't let friends use inout with scope and -dip1000
On Tuesday, 21 August 2018 at 14:04:15 UTC, Steven Schveighoffer wrote: I would guess it's no different than other inferred attributes. I would also guess that it only gets promoted to a return parameter if it's actually returned. If we can't have properly typed parameters, it feels like it has potential to prevent some patterns. This prevents automatic scope promotion: template escape(T) { int[] escape1(scope int[] r) { return r; } alias escape=escape1; }
Re: Friends don't let friends use inout with scope and -dip1000
On Tuesday, 21 August 2018 at 13:42:31 UTC, Kagamin wrote: int[] escape(scope int[] r) { return r; //error, can't return scoped argument } ... int[] escape(T)(scope int[] r) { return r; //ok! `scope` silently promoted to `return` } You can't have strictly scoped parameter in a templated function - it's silently promoted to return parameter. Is this intended? I filed a similar bug (it uses return type inference rather than a template). Walter closed it as invalid: https://issues.dlang.org/show_bug.cgi?id=17362
Re: Friends don't let friends use inout with scope and -dip1000
On Tuesday, 21 August 2018 at 11:28:39 UTC, Nicholas Wilson wrote: On Tuesday, 21 August 2018 at 10:57:15 UTC, Atila Neves wrote: On Tuesday, 21 August 2018 at 09:50:46 UTC, Atila Neves wrote: On Monday, 20 August 2018 at 15:55:54 UTC, Kagamin wrote: On Monday, 20 August 2018 at 13:02:23 UTC, Atila Neves wrote: On Monday, 20 August 2018 at 12:56:42 UTC, Kagamin wrote: Error: address of variable s assigned to gInt with longer lifetime Looks safe to me. With dmd 2.081.2 on Arch Linux, the code above compiles with no error message. Never mind, I forgot to use -dip1000. Ok, cool, so _why_ does it work as intended now? Also, if I have to remember to annotate correctly, surely this is a massive hole in @safe dip1000? MyStruct is not a template, I presume `return` would get inferred if it was. But yeah that is annoying. At the very least then it should fail to compile if I don't add the relevant annotation, not silently accept buggy code that isn't memory safe but somehow _is_ `@safe`. That's the whole point of -dip1000, no? If I get around it by forgetting something, it's not going to work.
Re: Friends don't let friends use inout with scope and -dip1000
On 8/21/18 9:42 AM, Kagamin wrote: except for templated functions: int[] escape(scope int[] r) { return r; //error, can't return scoped argument } int[] escape(return int[] r) { return r; //ok, just as planned } int[] escape(return scope int[] r) { return r; //ok, `return scope` reduced to just `return` } int[] escape(T)(scope int[] r) { return r; //ok! `scope` silently promoted to `return` } You can't have strictly scoped parameter in a templated function - it's silently promoted to return parameter. Is this intended? I would guess it's no different than other inferred attributes. I would also guess that it only gets promoted to a return parameter if it's actually returned. As long as the *result* is scoped like the parameter. In the case of the OP in this thread, there is definitely a problem with inout and the connection to the return value. -Steve
Re: Friends don't let friends use inout with scope and -dip1000
...except for templated functions: int[] escape(scope int[] r) { return r; //error, can't return scoped argument } int[] escape(return int[] r) { return r; //ok, just as planned } int[] escape(return scope int[] r) { return r; //ok, `return scope` reduced to just `return` } int[] escape(T)(scope int[] r) { return r; //ok! `scope` silently promoted to `return` } You can't have strictly scoped parameter in a templated function - it's silently promoted to return parameter. Is this intended?
Re: Friends don't let friends use inout with scope and -dip1000
I mean if one method in structure is trusted, other methods need manual verification too.
Re: Friends don't let friends use inout with scope and -dip1000
On Tuesday, 21 August 2018 at 10:57:15 UTC, Atila Neves wrote: Also, if I have to remember to annotate correctly, surely this is a massive hole in @safe dip1000? In general, safety works per method and doesn't help much in building safe data structures, those are trusted as a whole. EMSI containers are a notable big victim of this thing.
Re: Friends don't let friends use inout with scope and -dip1000
On Tuesday, 21 August 2018 at 10:57:15 UTC, Atila Neves wrote: Never mind, I forgot to use -dip1000. Ok, cool, so _why_ does it work as intended now? Also, if I have to remember to annotate correctly, surely this is a massive hole in @safe dip1000? It thought dip1000 was impenetrable, but if I understand it (honestly that's a surprise!), `scope` has strict semantics: all in, nothing out; you don't need to think about lifetime of data passed to scope parameters, because it doesn't escape anywhere. If you want to return data extracted from argument, `return` attribute relaxes scoping rules and allows to return data and passes scoping properties from argument to return value much like `inout` does for const. Without annotation: @safe: struct MyStruct { import core.stdc.stdlib; int* ints; this(int size) @trusted { ints = cast(int*) malloc(size); } ~this() scope @trusted { free(ints); } inout(int)* ptr() inout { return ints; } } int* gInt; void f() { scope s=MyStruct(10); gInt=s.ptr; } Error: scope variable s assigned to non-scope parameter this calling MyStruct.ptr Doesn't let to call method without annotation.
Re: Friends don't let friends use inout with scope and -dip1000
On Tuesday, 21 August 2018 at 10:57:15 UTC, Atila Neves wrote: On Tuesday, 21 August 2018 at 09:50:46 UTC, Atila Neves wrote: On Monday, 20 August 2018 at 15:55:54 UTC, Kagamin wrote: On Monday, 20 August 2018 at 13:02:23 UTC, Atila Neves wrote: On Monday, 20 August 2018 at 12:56:42 UTC, Kagamin wrote: Error: address of variable s assigned to gInt with longer lifetime Looks safe to me. With dmd 2.081.2 on Arch Linux, the code above compiles with no error message. Never mind, I forgot to use -dip1000. Ok, cool, so _why_ does it work as intended now? Also, if I have to remember to annotate correctly, surely this is a massive hole in @safe dip1000? MyStruct is not a template, I presume `return` would get inferred if it was. But yeah that is annoying.
Re: Friends don't let friends use inout with scope and -dip1000
On Tuesday, 21 August 2018 at 09:50:46 UTC, Atila Neves wrote: On Monday, 20 August 2018 at 15:55:54 UTC, Kagamin wrote: On Monday, 20 August 2018 at 13:02:23 UTC, Atila Neves wrote: On Monday, 20 August 2018 at 12:56:42 UTC, Kagamin wrote: [...] I need `return` for what exactly? Your code still compiles, and my point is it shouldn't. It sure isn't memory safe. @safe: struct MyStruct { import core.stdc.stdlib; int* ints; this(int size) @trusted { ints = cast(int*) malloc(size); } ~this() @trusted { free(ints); } inout(int)* ptr() return inout { return ints; } } int* gInt; void f() { auto s=MyStruct(10); gInt=s.ptr; } Error: address of variable s assigned to gInt with longer lifetime Looks safe to me. With dmd 2.081.2 on Arch Linux, the code above compiles with no error message. Never mind, I forgot to use -dip1000. Ok, cool, so _why_ does it work as intended now? Also, if I have to remember to annotate correctly, surely this is a massive hole in @safe dip1000?
Re: Friends don't let friends use inout with scope and -dip1000
On Monday, 20 August 2018 at 15:55:54 UTC, Kagamin wrote: On Monday, 20 August 2018 at 13:02:23 UTC, Atila Neves wrote: On Monday, 20 August 2018 at 12:56:42 UTC, Kagamin wrote: [...] I need `return` for what exactly? Your code still compiles, and my point is it shouldn't. It sure isn't memory safe. @safe: struct MyStruct { import core.stdc.stdlib; int* ints; this(int size) @trusted { ints = cast(int*) malloc(size); } ~this() @trusted { free(ints); } inout(int)* ptr() return inout { return ints; } } int* gInt; void f() { auto s=MyStruct(10); gInt=s.ptr; } Error: address of variable s assigned to gInt with longer lifetime Looks safe to me. With dmd 2.081.2 on Arch Linux, the code above compiles with no error message.
Re: Friends don't let friends use inout with scope and -dip1000
On Monday, 20 August 2018 at 15:55:54 UTC, Kagamin wrote: On Monday, 20 August 2018 at 13:02:23 UTC, Atila Neves wrote: On Monday, 20 August 2018 at 12:56:42 UTC, Kagamin wrote: You need `return` attribute there, not `scope`: struct MyStruct { import core.stdc.stdlib; int* ints; this(int size) @trusted { ints = cast(int*) malloc(size); } ~this() @trusted { free(ints); } inout(int)* ptr() return inout { return ints; } } I need `return` for what exactly? Your code still compiles, and my point is it shouldn't. It sure isn't memory safe. @safe: struct MyStruct { import core.stdc.stdlib; int* ints; this(int size) @trusted { ints = cast(int*) malloc(size); } ~this() @trusted { free(ints); } inout(int)* ptr() return inout { return ints; } } int* gInt; void f() { auto s=MyStruct(10); gInt=s.ptr; } Error: address of variable s assigned to gInt with longer lifetime Looks safe to me. Is that safe as well? void f() { auto s = MyStruct(10); gInt = (() => s.ptr)(); }
Re: Friends don't let friends use inout with scope and -dip1000
On Monday, 20 August 2018 at 13:02:23 UTC, Atila Neves wrote: On Monday, 20 August 2018 at 12:56:42 UTC, Kagamin wrote: You need `return` attribute there, not `scope`: struct MyStruct { import core.stdc.stdlib; int* ints; this(int size) @trusted { ints = cast(int*) malloc(size); } ~this() @trusted { free(ints); } inout(int)* ptr() return inout { return ints; } } I need `return` for what exactly? Your code still compiles, and my point is it shouldn't. It sure isn't memory safe. @safe: struct MyStruct { import core.stdc.stdlib; int* ints; this(int size) @trusted { ints = cast(int*) malloc(size); } ~this() @trusted { free(ints); } inout(int)* ptr() return inout { return ints; } } int* gInt; void f() { auto s=MyStruct(10); gInt=s.ptr; } Error: address of variable s assigned to gInt with longer lifetime Looks safe to me.
Re: Friends don't let friends use inout with scope and -dip1000
On 8/20/18 5:43 AM, Nicholas Wilson wrote: On Monday, 20 August 2018 at 09:31:09 UTC, Atila Neves wrote: On Friday, 17 August 2018 at 13:39:29 UTC, Steven Schveighoffer wrote: // used to be scope int* ptr() { return ints; } scope inout(int)* ptr() inout { return ints; } Does scope apply to the return value or the `this` reference? I assumed the return value. I think I've read DIP1000 about a dozen times now and I still get confused. As opposed to `const` or `immutable`, `scope(T)` isn't a thing so... I don't know? A type constructor affects the type of something. So const(int) is an int that is const. const int is actually NOT a type constructor, but a storage class. It's main effect is to make the int actually const(int), but can have other effects (e.g. if it's a global, it may be put into global storage instead of thread-local). scope is not a type constructor, ever. So how do you specify the return type is scope? How do you specify a difference between the scope of the 'this' pointer, and the scope of the return value? I'm super-confused as to what dip1000 actually is doing, and how to use it. What usually happens is that qualifiers to the left of the name apply to the return type and those to the right apply `this`. Not that that _should_ make any difference since lifetime ints == lifetime this No: const int* foo() const { return null; } Error: redundant const attribute. Up until 2.080, this was a deprecation, and the result was int * What happens if you remove the return type? (i.e. scope auto) And write what instead? scope ptr() inout { return ints; } ? Yes, this is what I was thinking. -Steve
Re: Friends don't let friends use inout with scope and -dip1000
On Monday, August 20, 2018 3:43:46 AM MDT Nicholas Wilson via Digitalmars-d wrote: > On Monday, 20 August 2018 at 09:31:09 UTC, Atila Neves wrote: > > On Friday, 17 August 2018 at 13:39:29 UTC, Steven Schveighoffer > > > > wrote: > >>> // used to be scope int* ptr() { return ints; } > >>> scope inout(int)* ptr() inout { return ints; } > >> > >> Does scope apply to the return value or the `this` reference? > > > > I assumed the return value. I think I've read DIP1000 about a > > dozen times now and I still get confused. As opposed to `const` > > or `immutable`, `scope(T)` isn't a thing so... I don't know? > > What usually happens is that qualifiers to the left of the name > apply to the return type and those to the right apply `this`. Not > that that _should_ make any difference since lifetime ints == > lifetime this I don't know what happens with scope with -dip1000, but that's not how D works with any other qualifier that can affect the return type. If you don't put parens on the qualifier on the return type, it refers to the this pointer/reference. And arguably, if we're now putting scope on return types, scope(T) should be a thing, and scope should have exactly the same behavior as const, shared, etc. with regards to placement, or it's just going to cause problems. - Jonathan M Davis
Re: Friends don't let friends use inout with scope and -dip1000
AIU, `return` for `scope` is what `inout` is for `const`. I proposed to extend `inout` to mean `return`, but Walter said that they are independent.
Re: Friends don't let friends use inout with scope and -dip1000
On Monday, 20 August 2018 at 09:43:46 UTC, Nicholas Wilson wrote: On Monday, 20 August 2018 at 09:31:09 UTC, Atila Neves wrote: On Friday, 17 August 2018 at 13:39:29 UTC, Steven Schveighoffer wrote: [...] Does scope apply to the return value or the `this` reference? I assumed the return value. I think I've read DIP1000 about a dozen times now and I still get confused. As opposed to `const` or `immutable`, `scope(T)` isn't a thing so... I don't know? What usually happens is that qualifiers to the left of the name apply to the return type and those to the right apply `this`. Not that that _should_ make any difference since lifetime ints == lifetime this What happens if you remove the return type? (i.e. scope auto) And write what instead? scope ptr() inout { return ints; } ? I guess you meant `scope ptr(this This)() { return this; }`. Nothing changes from the behaviour I described.
Re: Friends don't let friends use inout with scope and -dip1000
On Monday, 20 August 2018 at 12:56:42 UTC, Kagamin wrote: You need `return` attribute there, not `scope`: struct MyStruct { import core.stdc.stdlib; int* ints; this(int size) @trusted { ints = cast(int*) malloc(size); } ~this() @trusted { free(ints); } inout(int)* ptr() return inout { return ints; } } I need `return` for what exactly? Your code still compiles, and my point is it shouldn't. It sure isn't memory safe.
Re: Friends don't let friends use inout with scope and -dip1000
You need `return` attribute there, not `scope`: struct MyStruct { import core.stdc.stdlib; int* ints; this(int size) @trusted { ints = cast(int*) malloc(size); } ~this() @trusted { free(ints); } inout(int)* ptr() return inout { return ints; } }
Re: Friends don't let friends use inout with scope and -dip1000
On Monday, 20 August 2018 at 09:31:09 UTC, Atila Neves wrote: On Friday, 17 August 2018 at 13:39:29 UTC, Steven Schveighoffer wrote: // used to be scope int* ptr() { return ints; } scope inout(int)* ptr() inout { return ints; } Does scope apply to the return value or the `this` reference? I assumed the return value. I think I've read DIP1000 about a dozen times now and I still get confused. As opposed to `const` or `immutable`, `scope(T)` isn't a thing so... I don't know? What usually happens is that qualifiers to the left of the name apply to the return type and those to the right apply `this`. Not that that _should_ make any difference since lifetime ints == lifetime this What happens if you remove the return type? (i.e. scope auto) And write what instead? scope ptr() inout { return ints; } ?
Re: Friends don't let friends use inout with scope and -dip1000
On Friday, 17 August 2018 at 13:39:29 UTC, Steven Schveighoffer wrote: On 8/17/18 3:36 AM, Atila Neves wrote: Here's a struct: - struct MyStruct { import core.stdc.stdlib; int* ints; this(int size) @trusted { ints = cast(int*) malloc(size); } ~this() @trusted { free(ints); } scope int* ptr() { return ints; } } - Let's try and be evil with -dip1000: - @safe: // struct MyStruct ... const(int) *gInt; void main() { auto s = MyStruct(10); gInt = s.ptr; } - % dmd -dip1000 scope_inout.d scope_inout.d(26): Error: scope variable this may not be returned Yay! What if instead of `auto` I write `const` instead (or immutable)? This is D we're talking about, so none of this boilerplate nonsense of writing two (or three) basically identical functions. So: - // used to be scope int* ptr() { return ints; } scope inout(int)* ptr() inout { return ints; } Does scope apply to the return value or the `this` reference? I assumed the return value. I think I've read DIP1000 about a dozen times now and I still get confused. As opposed to `const` or `immutable`, `scope(T)` isn't a thing so... I don't know? What happens if you remove the return type? (i.e. scope auto) And write what instead? - % dmd -dip1000 scope_inout.d % echo $? 0 # nope, no error here Wait, what? Turns out now it compiles. After some under-the-breath mumbling I go hit issues.dlang.org and realise that the issue already exists: https://issues.dlang.org/show_bug.cgi?id=17935 I don't see what this bug report has to do with the given case. That's because I'm an idiot and I meant this one: https://issues.dlang.org/show_bug.cgi?id=17927 This seems like a straight up bug. I agree, but I also think #17935 is a straight up bug as well... This doesn't surprise me. I'm beginning to question whether scope shouldn't have been a type constructor instead of a storage class. It's treated almost like a type constructor in most places, but the language grammar makes it difficult to be specific as to what part it applies. I'm so confused it's not even funny.
Re: Friends don't let friends use inout with scope and -dip1000
On Friday, 17 August 2018 at 13:39:29 UTC, Steven Schveighoffer wrote: On 8/17/18 3:36 AM, Atila Neves wrote: Here's a struct: - // used to be scope int* ptr() { return ints; } scope inout(int)* ptr() inout { return ints; } Does scope apply to the return value or the `this` reference? This reference. putting it like: inout(int)* ptr() inout scope { return ints; } ...does not change anything. Another thing it should AFAIK catch but doesn't: import std.stdio; @safe: struct MyStruct { int* intP; this(int val) { intP = new int(val); } int* ptr() return scope { return intP; } } int *gInt; void main() { auto s = MyStruct(10); gInt = s.ptr; writeln(*gInt); }
Re: Friends don't let friends use inout with scope and -dip1000
On 8/17/18 3:36 AM, Atila Neves wrote: Here's a struct: - struct MyStruct { import core.stdc.stdlib; int* ints; this(int size) @trusted { ints = cast(int*) malloc(size); } ~this() @trusted { free(ints); } scope int* ptr() { return ints; } } - Let's try and be evil with -dip1000: - @safe: // struct MyStruct ... const(int) *gInt; void main() { auto s = MyStruct(10); gInt = s.ptr; } - % dmd -dip1000 scope_inout.d scope_inout.d(26): Error: scope variable this may not be returned Yay! What if instead of `auto` I write `const` instead (or immutable)? This is D we're talking about, so none of this boilerplate nonsense of writing two (or three) basically identical functions. So: - // used to be scope int* ptr() { return ints; } scope inout(int)* ptr() inout { return ints; } Does scope apply to the return value or the `this` reference? What happens if you remove the return type? (i.e. scope auto) - % dmd -dip1000 scope_inout.d % echo $? 0 # nope, no error here Wait, what? Turns out now it compiles. After some under-the-breath mumbling I go hit issues.dlang.org and realise that the issue already exists: https://issues.dlang.org/show_bug.cgi?id=17935 I don't see what this bug report has to do with the given case. For reasons unfathomable to me, this is considered the _correct_ behaviour. Weirder still, writing out the boilerplate that `inout` is supposed to save us (mutable, const and immutable versions) doesn't compile, which is what one would expect. So: @safe + inout + scope + dip1000 + custom memory allocation in D gets us to the usability of C++ circa 1998. At least now we have valgrind and asan I guess. "What about template this?", I hear you ask. It kinda works. Sorta. Kinda. Behold: scope auto ptr(this T)() { return ints; } After changing the definition of `ptr` this way the code compiles fine and `ints` is escaped. Huh. However, if you change `auto s` to `scope s`, it fails to compile as intended. Very weird. This seems like a straight up bug. If you change the destructor to `scope` then it also fails to compile even if it's `auto s`. Because, _obviously_, that's totally different. I'd file an issue but given that the original one is considered not a bug for some reason, I have no idea about what I just wrote is right or not. What I do know is I found multiple ways to do nasty things to memory under the guise of @safe and -dip1000, and my understanding was that the compiler would save me from myself. In the meanwhile I'm staying away from `inout` and putting `scope` on my destructors even if I don't quite understand when destructors should be `scope`. Probably always? I have no idea. This doesn't surprise me. I'm beginning to question whether scope shouldn't have been a type constructor instead of a storage class. It's treated almost like a type constructor in most places, but the language grammar makes it difficult to be specific as to what part it applies. -Steve
Friends don't let friends use inout with scope and -dip1000
Here's a struct: - struct MyStruct { import core.stdc.stdlib; int* ints; this(int size) @trusted { ints = cast(int*) malloc(size); } ~this() @trusted { free(ints); } scope int* ptr() { return ints; } } - Let's try and be evil with -dip1000: - @safe: // struct MyStruct ... const(int) *gInt; void main() { auto s = MyStruct(10); gInt = s.ptr; } - % dmd -dip1000 scope_inout.d scope_inout.d(26): Error: scope variable this may not be returned Yay! What if instead of `auto` I write `const` instead (or immutable)? This is D we're talking about, so none of this boilerplate nonsense of writing two (or three) basically identical functions. So: - // used to be scope int* ptr() { return ints; } scope inout(int)* ptr() inout { return ints; } - % dmd -dip1000 scope_inout.d % echo $? 0 # nope, no error here Wait, what? Turns out now it compiles. After some under-the-breath mumbling I go hit issues.dlang.org and realise that the issue already exists: https://issues.dlang.org/show_bug.cgi?id=17935 For reasons unfathomable to me, this is considered the _correct_ behaviour. Weirder still, writing out the boilerplate that `inout` is supposed to save us (mutable, const and immutable versions) doesn't compile, which is what one would expect. So: @safe + inout + scope + dip1000 + custom memory allocation in D gets us to the usability of C++ circa 1998. At least now we have valgrind and asan I guess. "What about template this?", I hear you ask. It kinda works. Sorta. Kinda. Behold: scope auto ptr(this T)() { return ints; } After changing the definition of `ptr` this way the code compiles fine and `ints` is escaped. Huh. However, if you change `auto s` to `scope s`, it fails to compile as intended. Very weird. If you change the destructor to `scope` then it also fails to compile even if it's `auto s`. Because, _obviously_, that's totally different. I'd file an issue but given that the original one is considered not a bug for some reason, I have no idea about what I just wrote is right or not. What I do know is I found multiple ways to do nasty things to memory under the guise of @safe and -dip1000, and my understanding was that the compiler would save me from myself. In the meanwhile I'm staying away from `inout` and putting `scope` on my destructors even if I don't quite understand when destructors should be `scope`. Probably always? I have no idea.