subject:"Is @safe still a work\-in\-progress\?"


On 8/23/2018 6:10 AM, Atila Neves wrote:

--
struct S {
     int x;
     @safe int* foo() { return  }
}
--

% dmd -o- -dip1000 foo.d
% echo $?
0


struct S {
int x;
@safe int* foo() { return  }
}

int* bar() {
S s;
return s.foo();
}

dmd test -dip1000
test.d(3): Error: returning  escapes a reference to parameter this, 
perhaps annotate with return





Oops:

--
int* gPtr;
void main() {
     auto s = S(42);
     gPtr = s.foo;
}
--


I get:

test.d(3): Error: returning  escapes a reference to parameter this, 
perhaps annotate with return




--
int[] gSlice;
void main() {
     auto s = S([42]);
     gSlice = s.foo;
}

struct S {
     int[] x;
     @safe int[] foo() return { return x; }
}
--


T[] is treated like T* as far as scope, etc., are concerned. You should see 
identical results.


And in a struct, `this` is a `ref`, yet `scope` on a member function applies to 
`this`.


Actually, it applies to fields of `this`.

Re: Is @safe still a work-in-progress?

2018-08-28 Thread Nicholas Wilson via Digitalmars-d


On Wednesday, 29 August 2018 at 05:04:22 UTC, Walter Bright wrote:

On 8/25/2018 4:09 AM, Nicholas Wilson wrote:
On Saturday, 25 August 2018 at 02:25:41 UTC, Walter Bright 
wrote:
I'm not hostile to debate. I just don't care for "this is 
uncharted territory, so let's do nothing" which has been 
going on for probably 4 years now, coincident with "scope is 
incomplete, D sux".


I.e. lead, follow, or get out of the way :-)


Document it


Already done: https://issues.dlang.org/show_bug.cgi?id=19097


Bugzilla is not documentation. These are language changes they 
need to be in release notes and the spec.

Re: Is @safe still a work-in-progress?


On 8/28/2018 6:12 AM, Steven Schveighoffer wrote:
So this would mean a member function would have to be refactored into a 
different function with a different calling syntax. i.e:


x.foo(target);

would have to be refactored to:

target.foo(x);

or foo(target, x);


Maybe it should be anyway.


But in terms of put, strictly speaking, any call of some.pipeline.put(x) is 
wrong. It should be put(some.pipeline, x), to avoid issues with how put was 
designed.


There are some of those in Phobos, too. It's covered by the proposed addition.



I don't know how to prove anything with programming languages.
I don't mean prove like mathematical proof. I mean try to consider how this 
affects all cases instead of just the one case that will make phobos compile.


"show" is a better verb than "prove".


I don't know how to find all cases, either, except by implementing it.

Re: Is @safe still a work-in-progress?


On 8/25/2018 4:09 AM, Nicholas Wilson wrote:

On Saturday, 25 August 2018 at 02:25:41 UTC, Walter Bright wrote:
I'm not hostile to debate. I just don't care for "this is uncharted territory, 
so let's do nothing" which has been going on for probably 4 years now, 
coincident with "scope is incomplete, D sux".


I.e. lead, follow, or get out of the way :-)


Document it


Already done: https://issues.dlang.org/show_bug.cgi?id=19097

Re: Is @safe still a work-in-progress?

2018-08-28 Thread Steven Schveighoffer via Digitalmars-d


On 8/25/2018 5:42 AM, Chris M. wrote:
What about my other point then on the syntax? I think something similar to what 
I suggested would be a much more flexible solution and is worth considering.


Much more work would be needed to make that a proposal.

Re: Is @safe still a work-in-progress?


On 8/24/18 10:25 PM, Walter Bright wrote:

On 8/23/2018 6:32 AM, Steven Schveighoffer wrote:
Furthermore any member function (or UFCS function for that matter) 
REQUIRES the first parameter to be the aggregate. How do you make a 
member function that stuffs the return into a different parameter 
properly typecheck?


What I propose is that the function interface be refactored so it does 
fit into these patterns. Is that an unreasonable requirement? I don't 
know. But it doesn't seem to be, as I haven't run into it yet.


So this would mean a member function would have to be refactored into a 
different function with a different calling syntax. i.e:


x.foo(target);

would have to be refactored to:

target.foo(x);

or foo(target, x);

Aside from the adjustment in name that is necessary to make this read 
correctly, that may cause other problems (sometimes non-member functions 
aren't available if it's a template instantiation).


Phobos doesn't do this by accident. It's how constructors work (see 
above) and how pipeline programming works.


Constructors I agree are reasonable to consider `this` to be the 
return value. On that point, I would say we should definitely go ahead 
with making that rule, and I think it will lead to no confusion 
whatsoever.


pipeline programming depends on returning something other than `void`, 
so I don't see how this applies.


grep Phobos for instances of put() and see its signature. It's part of 
pipeline programming, and it's all over the place.


I stand partly corrected! Indeed you can put a void-returning function 
at the *end* of a pipeline call, I hadn't thought of that.


But in terms of put, strictly speaking, any call of some.pipeline.put(x) 
is wrong. It should be put(some.pipeline, x), to avoid issues with how 
put was designed.



It would restrict your legitimate calls.


Maybe that's a good thing. Having multiple simultaneous routes of data 
out of a function is not good practice (note that it is impossible with 
functional programming). If you absolutely must have it, the exit routes 
can be aggregated into a struct, then pass that struct as the first 
argument.


Maybe it's better to designate one sink, and have that be the result. I 
know that after inout was implemented, there were definitely cases where 
one wanted to have multiple inout routes (i.e. independent traces 
between multiple parameters for copying mutability). It may be the same 
for this, I don't know.


I want to stress that it may be a valid solution, but we should strive 
to prove the solutions are the best possible rather than just use 
duct-tape methodology.


I don't know how to prove anything with programming languages.


I don't mean prove like mathematical proof. I mean try to consider how 
this affects all cases instead of just the one case that will make 
phobos compile.


"show" is a better verb than "prove".

It should even be considered that perhaps there are better solutions 
even than the approach dip1000 has taken.


People have hypothesized that for several years, and so far none have 
been forthcoming beyond a few hand-wavy generalities.


I'm just saying if dip1000 cannot fix all the problems, that instead of 
adding weird exceptions, or the classic "you're just doing it wrong", 
maybe we should reconsider the approach.


Another case which was brought up and pretty much ignored was this one: 
https://forum.dlang.org/post/qkrdpmdqaxjadgvso...@forum.dlang.org


I also want to point out that the attitude of 'we could just fix it, 
but nobody will pull my request' is unhelpful. We want to make sure we 
have the best solution possible, don't take criticism as meaningless 
petty bickering. People are genuinely trying to make sure D is 
improved. Hostility towards reviews or debate doesn't foster that.


I'm not hostile to debate. I just don't care for "this is uncharted 
territory, so let's do nothing" which has been going on for probably 4 
years now, coincident with "scope is incomplete, D sux".


I.e. lead, follow, or get out of the way :-)


I'm opting for the latter, as the idea of band-aid PRs to get Phobos 
compiling with dip1000 just to see if dip1000 is going to work seems 
like the wrong approach to me.


The consequence of this is that getting out of the way means your PRs 
don't get pulled.


-Steve

Re: Is @safe still a work-in-progress?

2018-08-28 Thread Steven Schveighoffer via Digitalmars-d


On 8/24/18 10:28 PM, Walter Bright wrote:

On 8/23/2018 8:14 AM, Steven Schveighoffer wrote:
If I had to design a specific way to allow the common case to be easy, 
but still provide a mechanism for the uncommon cases, I would say:


1. define a compiler-recognized attribute (e.g. @__sink).
2. If @__sink is applied to any parameter, that is effectively the 
return value.
3. In the absence of a @__sink designation on non-void-returning 
functions, it applies to the return value.
4. In the absence of a @__sink designation on void returning 
functions, it applies to the first parameter.

5. Inference of @__sink happens even on non-templates.
6. If @__sink is attributed on multiple parameters, you assume all 
return parameters are assigned to all @__sink parameters for the 
purposes of verifying lifetimes are not exceeded.



'ref' is already @__sink.


No, otherwise we wouldn't need the patch you are pushing.

-Steve

Re: Is @safe still a work-in-progress?

2018-08-25 Thread Chris M. via Digitalmars-d


On Saturday, 25 August 2018 at 02:37:00 UTC, Walter Bright wrote:

On 8/23/2018 5:58 PM, Chris M. wrote:
Seems to be more of a warning of what issues we may face if 
DIP25/DIP1000 are finally implemented. It would be good to 
consider NLLs as well before D is committed. No point in 
repeating issues that have already been studied.


DIP25 waqs "finally implemented" several years ago, and works 
well. DIP1000 was implemented as well, it works, but it didn't 
cover the case of returning through a ref parameter.


There's no way to "thoroughly vet" them before implementing. It 
doesn't happen with C++, either, somebody builds an 
implementation and then people try it out.


What about my other point then on the syntax? I think something 
similar to what I suggested would be a much more flexible 
solution and is worth considering.

Re: Is @safe still a work-in-progress?

2018-08-25 Thread Nicholas Wilson via Digitalmars-d


On Saturday, 25 August 2018 at 02:25:41 UTC, Walter Bright wrote:
I'm not hostile to debate. I just don't care for "this is 
uncharted territory, so let's do nothing" which has been going 
on for probably 4 years now, coincident with "scope is 
incomplete, D sux".


I.e. lead, follow, or get out of the way :-)


Document it and we will help you get it reviewed and merged. 
Until you do that we will stay in your way. Heck, once we 
understand it we may even lead, but we can't do that if we have 
no clue WTF its supposed to do or why the changes are being made.


At the very least PRs need detailed changelog entries and 
preferably spec updates and revisions to the DIP1000 proposal 
document. Heck even a wiki page would be useful. I'm trying to 
make the spec page on memory safe programming the authoritative 
source see https://github.com/dlang/dlang.org/pull/2453

Re: Is @safe still a work-in-progress?

2018-08-24 Thread Walter Bright via Digitalmars-d


On 8/23/2018 5:58 PM, Chris M. wrote:
Seems to be more of a warning of what issues we may face if DIP25/DIP1000 are 
finally implemented. It would be good to consider NLLs as well before D is 
committed. No point in repeating issues that have already been studied.


DIP25 waqs "finally implemented" several years ago, and works well. DIP1000 was 
implemented as well, it works, but it didn't cover the case of returning through 
a ref parameter.


There's no way to "thoroughly vet" them before implementing. It doesn't happen 
with C++, either, somebody builds an implementation and then people try it out.

Re: Is @safe still a work-in-progress?

2018-08-24 Thread Walter Bright via Digitalmars-d


On 8/23/2018 6:32 AM, Steven Schveighoffer wrote:
Furthermore any member function (or UFCS function for that matter) REQUIRES the 
first parameter to be the aggregate. How do you make a member function that 
stuffs the return into a different parameter properly typecheck?


What I propose is that the function interface be refactored so it does fit into 
these patterns. Is that an unreasonable requirement? I don't know. But it 
doesn't seem to be, as I haven't run into it yet.



Phobos doesn't do this by accident. It's how constructors work (see above) and 
how pipeline programming works.


Constructors I agree are reasonable to consider `this` to be the return value. 
On that point, I would say we should definitely go ahead with making that rule, 
and I think it will lead to no confusion whatsoever.


pipeline programming depends on returning something other than `void`, so I 
don't see how this applies.


grep Phobos for instances of put() and see its signature. It's part of pipeline 
programming, and it's all over the place.



You would have to consider the shortest liftetime and assume everything goes 
there.


That's right.



It would restrict your legitimate calls.


Maybe that's a good thing. Having multiple simultaneous routes of data out of a 
function is not good practice (note that it is impossible with functional 
programming). If you absolutely must have it, the exit routes can be aggregated 
into a struct, then pass that struct as the first argument.



I want to stress that it may be a valid solution, but we should strive to prove 
the solutions are the best possible rather than just use duct-tape methodology.


I don't know how to prove anything with programming languages.


It should even be considered that perhaps there are better solutions even than 
the approach dip1000 has taken.


People have hypothesized that for several years, and so far none have been 
forthcoming beyond a few hand-wavy generalities.



I also want to point out that the attitude of 'we could just fix it, but nobody 
will pull my request' is unhelpful. We want to make sure we have the best 
solution possible, don't take criticism as meaningless petty bickering. People 
are genuinely trying to make sure D is improved. Hostility towards reviews or 
debate doesn't foster that.


I'm not hostile to debate. I just don't care for "this is uncharted territory, 
so let's do nothing" which has been going on for probably 4 years now, 
coincident with "scope is incomplete, D sux".


I.e. lead, follow, or get out of the way :-)

Re: Is @safe still a work-in-progress?

2018-08-24 Thread Walter Bright via Digitalmars-d


On 8/23/2018 8:14 AM, Steven Schveighoffer wrote:
If I had to design a specific way to allow the common case to be easy, but still 
provide a mechanism for the uncommon cases, I would say:


1. define a compiler-recognized attribute (e.g. @__sink).
2. If @__sink is applied to any parameter, that is effectively the return value.
3. In the absence of a @__sink designation on non-void-returning functions, it 
applies to the return value.
4. In the absence of a @__sink designation on void returning functions, it 
applies to the first parameter.

5. Inference of @__sink happens even on non-templates.
6. If @__sink is attributed on multiple parameters, you assume all return 
parameters are assigned to all @__sink parameters for the purposes of verifying 
lifetimes are not exceeded.



'ref' is already @__sink.

Re: Is @safe still a work-in-progress?

On Friday, 24 August 2018 at 00:13:48 UTC, Mike Franklin wrote:

On Thursday, 23 August 2018 at 23:36:07 UTC, Chris M. wrote:

Heck, now that I'm looking at it, DIP25 seems like a more
restricted form of Rust's lifetimes. Let me know if I'm just
completely wrong about this, but

I think DIP 25 is analogous to Problem #3 for Rust's
Non-Lexical Lifetimes:

http://smallcultfollowing.com/babysteps/blog/2016/04/27/non-lexical-lifetimes-introduction/#problem-case-3-conditional-control-flow-across-functions

http://smallcultfollowing.com/babysteps/blog/2016/05/09/non-lexical-lifetimes-adding-the-outlives-relation/#problem-case-3-revisited

Seems to be more of a warning of what issues we may face if
DIP25/DIP1000 are finally implemented. It would be good to
consider NLLs as well before D is committed. No point in
repeating issues that have already been studied.

would basically be like (pseudosyntax)

@safe ref'a int identity(ref'a int x) {
return x; // fine
}

Maybe the more sane thing would be a syntax that visually ties
them together as above. Obviously we're looking at possibly
breaking changes, but how widespread would they be?

void betty(ref'a scope int* r, scope'a int* p); // syntax is
not so nice since I just arbitrarily stuck them on different
keywords, but that's besides the point here

I wish I had been more involved in D when DIP 25 and DIP 1000
were being proposed, as I don't think the designs were
thoroughly vetted. It's taken me at least a year to even begin
getting a grasp on it.

I think DIP 25 and DIP 1000 should have been combined and
thought of holistically as simply "annotated lifetimes in D"
rather than separate things. I think then it becomes easier to
visualize what the problem is and see, potentially many,
alternatives.

Given the investments that have already been made in DIP 25 and
DIP 1000, it's going to take an extremely motivated individual
to fight an uphill battle to change direction now, I'm afraid.
If working on D was my full-time job, I'd do it, but who in
this community has such resources.

I think the main problem is that these were created before the
DIP process was fully fleshed out, so who knows how much vetting
they got. At least they are still compiler switches and probably
not widely used, so it's not like we're already too far gone. I
do also agree they should be worked on in conjunction, the hard
part is finding someone to take ownership and put in the extra
effort.

Mike

Re: Is @safe still a work-in-progress?

2018-08-23 Thread Mike Franklin via Digitalmars-d


On Thursday, 23 August 2018 at 23:36:07 UTC, Chris M. wrote:

Heck, now that I'm looking at it, DIP25 seems like a more 
restricted form of Rust's lifetimes. Let me know if I'm just 
completely wrong about this, but


I think DIP 25 is analogous to Problem #3 for Rust's Non-Lexical 
Lifetimes:


http://smallcultfollowing.com/babysteps/blog/2016/04/27/non-lexical-lifetimes-introduction/#problem-case-3-conditional-control-flow-across-functions

http://smallcultfollowing.com/babysteps/blog/2016/05/09/non-lexical-lifetimes-adding-the-outlives-relation/#problem-case-3-revisited


would basically be like (pseudosyntax)

@safe ref'a int identity(ref'a int x) {
return x; // fine
}

Maybe the more sane thing would be a syntax that visually ties 
them together as above. Obviously we're looking at possibly 
breaking changes, but how widespread would they be?


void betty(ref'a scope int* r, scope'a int* p); // syntax is 
not so nice since I just arbitrarily stuck them on different 
keywords, but that's besides the point here


I wish I had been more involved in D when DIP 25 and DIP 1000 
were being proposed, as I don't think the designs were thoroughly 
vetted.  It's taken me at least a year to even begin getting a 
grasp on it.


I think DIP 25 and DIP 1000 should have been combined and thought 
of holistically as simply "annotated lifetimes in D" rather than 
separate things.  I think then it becomes easier to visualize 
what the problem is and see, potentially many, alternatives.


Given the investments that have already been made in DIP 25 and 
DIP 1000, it's going to take an extremely motivated individual to 
fight an uphill battle to change direction now, I'm afraid.  If 
working on D was my full-time job, I'd do it, but who in this 
community has such resources.


Mike

Re: Is @safe still a work-in-progress?


On Thursday, 23 August 2018 at 23:58:00 UTC, jmh530 wrote:

On Thursday, 23 August 2018 at 23:36:07 UTC, Chris M. wrote:


Heck, now that I'm looking at it, DIP25 seems like a more 
restricted form of Rust's lifetimes. Let me know if I'm just 
completely wrong about this, but

[snip]


Check out DIP1000

https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md


I've read through it already, I'm just throwing out an idea for 
what I think is a more flexible solution to Walter's current 
issue which seems to be stemming more from DIP25 rather than 
DIP1000. More effort but I think it'd be worth it in the long run.

Re: Is @safe still a work-in-progress?

2018-08-23 Thread jmh530 via Digitalmars-d


On Thursday, 23 August 2018 at 23:36:07 UTC, Chris M. wrote:


Heck, now that I'm looking at it, DIP25 seems like a more 
restricted form of Rust's lifetimes. Let me know if I'm just 
completely wrong about this, but

[snip]


Check out DIP1000

https://github.com/dlang/DIPs/blob/master/DIPs/DIP1000.md

Re: Is @safe still a work-in-progress?


On Thursday, 23 August 2018 at 15:48:00 UTC, Chris M. wrote:
On Thursday, 23 August 2018 at 15:14:07 UTC, Steven 
Schveighoffer wrote:

On 8/23/18 9:32 AM, Steven Schveighoffer wrote:

[...]


Actually, thinking about this, the shortest lifetime is 
dictated by how it is called, so there is no valid way to 
determine which one makes sense when compiling the function.


In order for this to work, you'd have to attribute it somehow. 
I can see that is likely going to be way more cumbersome than 
it's worth.


If I had to design a specific way to allow the common case to 
be easy, but still provide a mechanism for the uncommon cases, 
I would say:


1. define a compiler-recognized attribute (e.g. @__sink).
2. If @__sink is applied to any parameter, that is effectively 
the return value.
3. In the absence of a @__sink designation on 
non-void-returning functions, it applies to the return value.
4. In the absence of a @__sink designation on void returning 
functions, it applies to the first parameter.

5. Inference of @__sink happens even on non-templates.
6. If @__sink is attributed on multiple parameters, you assume 
all return parameters are assigned to all @__sink parameters 
for the purposes of verifying lifetimes are not exceeded.


Ugly to specify, but might actually be pretty non-intrusive to 
use.


-Steve


This is more a general reply to the thread.

If I think I'm getting a good grasp on the issue here, it seems 
like something Rust already solved with lifetime annotations. 
Could they or something similar be leveraged for D as well?


https://doc.rust-lang.org/1.9.0/book/lifetimes.html

Current solution just seems too specific and very restrictive.


Heck, now that I'm looking at it, DIP25 seems like a more 
restricted form of Rust's lifetimes. Let me know if I'm just 
completely wrong about this, but


@safe ref int identity(return ref int x) {
return x; // fine
}

would basically be like (pseudosyntax)

@safe ref'a int identity(ref'a int x) {
return x; // fine
}

Maybe the more sane thing would be a syntax that visually ties 
them together as above. Obviously we're looking at possibly 
breaking changes, but how widespread would they be?


void betty(ref'a scope int* r, scope'a int* p); // syntax is not 
so nice since I just arbitrarily stuck them on different 
keywords, but that's besides the point here

Re: Is @safe still a work-in-progress?

2018-08-23 Thread Steven Schveighoffer via Digitalmars-d

On Thursday, 23 August 2018 at 15:14:07 UTC, Steven Schveighoffer 
wrote:

On 8/23/18 9:32 AM, Steven Schveighoffer wrote:

[...]


Actually, thinking about this, the shortest lifetime is 
dictated by how it is called, so there is no valid way to 
determine which one makes sense when compiling the function.


In order for this to work, you'd have to attribute it somehow. 
I can see that is likely going to be way more cumbersome than 
it's worth.


If I had to design a specific way to allow the common case to 
be easy, but still provide a mechanism for the uncommon cases, 
I would say:


1. define a compiler-recognized attribute (e.g. @__sink).
2. If @__sink is applied to any parameter, that is effectively 
the return value.
3. In the absence of a @__sink designation on 
non-void-returning functions, it applies to the return value.
4. In the absence of a @__sink designation on void returning 
functions, it applies to the first parameter.

5. Inference of @__sink happens even on non-templates.
6. If @__sink is attributed on multiple parameters, you assume 
all return parameters are assigned to all @__sink parameters 
for the purposes of verifying lifetimes are not exceeded.


Ugly to specify, but might actually be pretty non-intrusive to 
use.


-Steve


This is more a general reply to the thread.

If I think I'm getting a good grasp on the issue here, it seems 
like something Rust already solved with lifetime annotations. 
Could they or something similar be leveraged for D as well?


https://doc.rust-lang.org/1.9.0/book/lifetimes.html

Current solution just seems too specific and very restrictive.

Re: Is @safe still a work-in-progress?


On 8/23/18 9:32 AM, Steven Schveighoffer wrote:

On 8/23/18 4:58 AM, Walter Bright wrote:

On 8/22/2018 6:50 AM, Steven Schveighoffer wrote:
As for things being made "more flexible in the future" this basically 
translates to code breakage. For example, if you are depending on 
only the first parameter being considered the "return" value, and all 
of a sudden it changes to encompass all your parameters, your 
existing code may fail to compile, even if it's correctly safe and 
properly annotated.


It's a good point. But I don't see an obvious use case for considering 
all the ref parameters as being returns.


You would have to consider the shortest liftetime and assume everything 
goes there. It would restrict your legitimate calls. Only mitigating 
factor may be if you take the ones you aren't going to modify as const 
or inout.


Actually, thinking about this, the shortest lifetime is dictated by how 
it is called, so there is no valid way to determine which one makes 
sense when compiling the function.


In order for this to work, you'd have to attribute it somehow. I can see 
that is likely going to be way more cumbersome than it's worth.


If I had to design a specific way to allow the common case to be easy, 
but still provide a mechanism for the uncommon cases, I would say:


1. define a compiler-recognized attribute (e.g. @__sink).
2. If @__sink is applied to any parameter, that is effectively the 
return value.
3. In the absence of a @__sink designation on non-void-returning 
functions, it applies to the return value.
4. In the absence of a @__sink designation on void returning functions, 
it applies to the first parameter.

5. Inference of @__sink happens even on non-templates.
6. If @__sink is attributed on multiple parameters, you assume all 
return parameters are assigned to all @__sink parameters for the 
purposes of verifying lifetimes are not exceeded.


Ugly to specify, but might actually be pretty non-intrusive to use.

-Steve

Re: Is @safe still a work-in-progress?

2018-08-23 Thread Steven Schveighoffer via Digitalmars-d


On 8/23/18 4:58 AM, Walter Bright wrote:

On 8/22/2018 6:50 AM, Steven Schveighoffer wrote:

What about:

size_t put(sink, parameters...)

Does this qualify as the sink being the "return" type? Obviously the 
real return can't contain any references, so it trivially can be ruled 
out as the destination of any escaping parameters.


Your reasoning is correct, but currently it only applies with 'void' 
return types.



Or how about a member function that takes a ref parameter? Is `this` 
the "return" or is the ref parameter the "return"?


`this` is the ref parameter. In particular, consider a constructor:

   struct S {
  int* p;
  this(return scope int* p) { this.p = p; }
   }

   int i;
   S s = S();

This code appears in Phobos, and it is very reasonable to expect it to 
check as safe.


What I mean to say is, we have a semantic today -- the return value is 
hooked to any `return` parameters, end of story. This is clear, concise, 
and easy to understand.


You are saying that in some cases, the return value is actually 
deposited in the `this` parameter. In cases where the actual return type 
is void, OK, I see that we can tack on that rule without issues.


Furthermore any member function (or UFCS function for that matter) 
REQUIRES the first parameter to be the aggregate. How do you make a 
member function that stuffs the return into a different parameter 
properly typecheck? What rule do we tack on then? It's going to be 
confusing to anyone who writes their API thinking about how it's call 
syntax reads, not how the compiler wants to do flow analysis.


Not to mention, the keyword is `return`, not `returnorfirstparam`. It's 
still going to be confusing no matter how you look at it.


My problem with the idea is that it is going to seem flaky -- we are 
using convention to dictate what is actually the return parameter, vs. 
what semantically happens inside the function. It's going to confuse 
anyone trying to do it a different way. I've experienced this in the 
past with things like toHash, where if you didn't define it with the 
exact signature, it wouldn't actually be used.


I realize obviously, that `put` is already specified. But as I said in 
the bug report, we should think twice about defining rules based 
solely on how Phobos does things, and calling that the solution.


Phobos doesn't do this by accident. It's how constructors work (see 
above) and how pipeline programming works.


Constructors I agree are reasonable to consider `this` to be the return 
value. On that point, I would say we should definitely go ahead with 
making that rule, and I think it will lead to no confusion whatsoever.


pipeline programming depends on returning something other than `void`, 
so I don't see how this applies.


It's more when you are setting members via properties where this comes 
into play. We need it -- we need this ability to tell the compiler "this 
parameter connects to this other parameter". I just don't know if the 
proposed rules are a) good enough for the general case, and b) don't 
cause more confusion than they are worth.


As for things being made "more flexible in the future" this basically 
translates to code breakage. For example, if you are depending on only 
the first parameter being considered the "return" value, and all of a 
sudden it changes to encompass all your parameters, your existing code 
may fail to compile, even if it's correctly safe and properly annotated.


It's a good point. But I don't see an obvious use case for considering 
all the ref parameters as being returns.


You would have to consider the shortest liftetime and assume everything 
goes there. It would restrict your legitimate calls. Only mitigating 
factor may be if you take the ones you aren't going to modify as const 
or inout.


> I want to ensure Atila is successful with this. But that means 
Phobos has to

compile with dip1000. So I need to make it work.



I think it's a very worthy goal to make Phobos work, and a great proof 
of concept for dip1000's veracity.


However, one-off rules just to make it work with existing code go 
against that goal IMO. Rules that stand on their own I think will fare 
better than ones that are loopholes to allow existing code to compile.


I couldn't come up with a better idea than this, and this one works.



I want to stress that it may be a valid solution, but we should strive 
to prove the solutions are the best possible rather than just use 
duct-tape methodology.


It should even be considered that perhaps there are better solutions 
even than the approach dip1000 has taken.


I also want to point out that the attitude of 'we could just fix it, but 
nobody will pull my request' is unhelpful. We want to make sure we have 
the best solution possible, don't take criticism as meaningless petty 
bickering. People are genuinely trying to make sure D is improved. 
Hostility towards reviews or debate doesn't foster that.


-Steve

Re: Is @safe still a work-in-progress?

2018-08-23 Thread Atila Neves via Digitalmars-d


On Thursday, 23 August 2018 at 08:48:15 UTC, Walter Bright wrote:

On 8/22/2018 3:52 AM, Atila Neves wrote:
On Wednesday, 22 August 2018 at 09:05:13 UTC, Walter Bright 
wrote:

On 8/21/2018 8:58 PM, Nicholas Wilson wrote:
On Tuesday, 21 August 2018 at 14:31:02 UTC, Atila Neves 
wrote:
The problem is that the code we write doesn't deal directly 
with pointers - see the recent confusion in this forum over 
where `scope` on the left applies to the `this` pointer or 
the one returned by the member function.


Kagamin just told me I needed to use `return` instead of 
`scope` to get things to work and I'm still not sure why.


The way I think about it is if you have a function that 
takes a pointer, any pointer, and either returns it or a 
pointer derived from it (dereferencing or indexing) that 
argument must be marked `return`. In your case it was a 
pointer derived from `this` so `return` must be applied to 
`this`.



Another way to think about it is this:

   S s;
   return 

We all know that is an error. The idea is to have a way to 
express that for:


    S s;
    return s.foo();

and:

    S s;
    return foo();

so that the compiler knows that the return value of foo() is 
attached to the lifetime of s. Pretty much everything flows 
from that.


Would the guideline below be correct?

"Add scope to every non-template member function that isn't 
meant to escape this and add return to every non-template 
member function that returns all or part of `this` by pointer 
or ref if you want the compiler to check that nothing gets 
escaped in @safe code."


Being a template doesn't make any difference, except that it 
will helpfully infer these things.


The reason I wrote "non-template" is precisely because attributes 
get inferred for templates and therefore it would be at best 
redundant to annotate.


What about the guideline being correct or not?



Also, since 'this' is passed by 'ref' to struct member 
functions, it cannot escape anyway with dip1000:


  struct S {
int x;
@safe ref int foo() { return x; }
  }

  dmd test -dip1000
  test.d(4): Error: returning this.x escapes a reference to 
parameter this, perhaps annotate with return


Returning by ref is good, since it's a lot harder to escape it. 
Since variables can't be declared ref, I can't pass it to a ref 
global. However:


--
struct S {
int x;
@safe int* foo() { return  }
}
--

% dmd -o- -dip1000 foo.d
% echo $?
0

Oops:

--
int* gPtr;
void main() {
auto s = S(42);
gPtr = s.foo;
}
--

"Don't use pointers then!". Ok:

--
int[] gSlice;
void main() {
auto s = S([42]);
gSlice = s.foo;
}

struct S {
int[] x;
@safe int[] foo() return { return x; }
}
--

% dmd -o- -dip1000 bar.d
% echo $?
0

Oops. I can't say `int[] x;` since it doesn't apply to fields.


`scope` is for pointers, `ref` does not need further annotation.


I always forget this, it's confusing. It doesn't help that slices 
have pointers, so I guess `scope` is for them too?


And in a struct, `this` is a `ref`, yet `scope` on a member 
function applies to `this`.


Atila

Re: Is @safe still a work-in-progress?

2018-08-23 Thread ag0aep6g via Digitalmars-d


On Thursday, 23 August 2018 at 08:48:15 UTC, Walter Bright wrote:
Being a template doesn't make any difference, except that it 
will helpfully infer these things.


Including the `return` attribute.

Also, since 'this' is passed by 'ref' to struct member 
functions, it cannot escape anyway with dip1000:


  struct S {
int x;
@safe ref int foo() { return x; }
  }

  dmd test -dip1000
  test.d(4): Error: returning this.x escapes a reference to 
parameter this, perhaps annotate with return


If we add the `return` attribute, we can escape a pointer to 
`this`. And since attributes are often inferred, it might come as 
a surprise to the programmer.


However, as far as I understand DIP 1000, it should make sure 
that the returned pointer won't outlive the struct instance. And 
I guess that's why it's ok to infer the `return` attribute?


That depends on DMD not over-estimating the lifetimes of things, 
though. Unfortunately, it does just that. When in doubt, DMD goes 
with infinite (e.g., `malloc`). I think that's a problem.


But I've already demonstrated my ignorance regarding DIP 1000 in 
a long, not very fruitful discussion with Atila on Bugzilla [1], 
so maybe I just don't get it.



[1] https://issues.dlang.org/show_bug.cgi?id=19183

Re: Is @safe still a work-in-progress?

2018-08-23 Thread Walter Bright via Digitalmars-d


On 8/22/2018 6:50 AM, Steven Schveighoffer wrote:

What about:

size_t put(sink, parameters...)

Does this qualify as the sink being the "return" type? Obviously the real return 
can't contain any references, so it trivially can be ruled out as the 
destination of any escaping parameters.


Your reasoning is correct, but currently it only applies with 'void' return 
types.


Or how about a member function that takes a ref parameter? Is `this` the 
"return" or is the ref parameter the "return"?


`this` is the ref parameter. In particular, consider a constructor:

  struct S {
 int* p;
 this(return scope int* p) { this.p = p; }
  }

  int i;
  S s = S();

This code appears in Phobos, and it is very reasonable to expect it to check as 
safe.



My problem with the idea is that it is going to seem flaky -- we are using 
convention to dictate what is actually the return parameter, vs. what 
semantically happens inside the function. It's going to confuse anyone trying to 
do it a different way. I've experienced this in the past with things like 
toHash, where if you didn't define it with the exact signature, it wouldn't 
actually be used.


I realize obviously, that `put` is already specified. But as I said in the bug 
report, we should think twice about defining rules based solely on how Phobos 
does things, and calling that the solution.


Phobos doesn't do this by accident. It's how constructors work (see above) and 
how pipeline programming works.



As for things being made "more flexible in the future" this basically translates 
to code breakage. For example, if you are depending on only the first parameter 
being considered the "return" value, and all of a sudden it changes to encompass 
all your parameters, your existing code may fail to compile, even if it's 
correctly safe and properly annotated.


It's a good point. But I don't see an obvious use case for considering all the 
ref parameters as being returns.



> I want to ensure Atila is successful with this. But that means Phobos has to

compile with dip1000. So I need to make it work.



I think it's a very worthy goal to make Phobos work, and a great proof of 
concept for dip1000's veracity.


However, one-off rules just to make it work with existing code go against that 
goal IMO. Rules that stand on their own I think will fare better than ones that 
are loopholes to allow existing code to compile.


I couldn't come up with a better idea than this, and this one works.

Re: Is @safe still a work-in-progress?

2018-08-23 Thread Walter Bright via Digitalmars-d


On 8/22/2018 3:52 AM, Atila Neves wrote:

On Wednesday, 22 August 2018 at 09:05:13 UTC, Walter Bright wrote:

On 8/21/2018 8:58 PM, Nicholas Wilson wrote:

On Tuesday, 21 August 2018 at 14:31:02 UTC, Atila Neves wrote:
The problem is that the code we write doesn't deal directly with pointers - 
see the recent confusion in this forum over where `scope` on the left 
applies to the `this` pointer or the one returned by the member function.


Kagamin just told me I needed to use `return` instead of `scope` to get 
things to work and I'm still not sure why.


The way I think about it is if you have a function that takes a pointer, any 
pointer, and either returns it or a pointer derived from it (dereferencing or 
indexing) that argument must be marked `return`. In your case it was a 
pointer derived from `this` so `return` must be applied to `this`.



Another way to think about it is this:

   S s;
   return 

We all know that is an error. The idea is to have a way to express that for:

    S s;
    return s.foo();

and:

    S s;
    return foo();

so that the compiler knows that the return value of foo() is attached to the 
lifetime of s. Pretty much everything flows from that.


Would the guideline below be correct?

"Add scope to every non-template member function that isn't meant to escape this 
and add return to every non-template member function that returns all or part of 
`this` by pointer or ref if you want the compiler to check that nothing gets 
escaped in @safe code."


Being a template doesn't make any difference, except that it will helpfully 
infer these things.


Also, since 'this' is passed by 'ref' to struct member functions, it cannot 
escape anyway with dip1000:


  struct S {
int x;
@safe ref int foo() { return x; }
  }

  dmd test -dip1000
  test.d(4): Error: returning this.x escapes a reference to parameter this, 
perhaps annotate with return


`scope` is for pointers, `ref` does not need further annotation.

Re: Is @safe still a work-in-progress?

2018-08-22 Thread Steven Schveighoffer via Digitalmars-d


On 8/22/18 5:23 AM, Walter Bright wrote:

On 8/21/2018 6:07 PM, Mike Franklin wrote:
The proposed idea wants to make the first parameter, if it's `ref`, 
special.


This is because Phobos is written with functions of the form:

     void put(sink, parameters...)

which corresponds to:

     sink.put(parameters...)

The two forms are fairly interchangeable, made more so by the Uniform 
Function Call Syntax.





 > Why not the first `ref` parameter regardless of whether it's the 
absolute first in the list.  Why not the last `ref` parameter?  Why not 
all `ref` parameters?


Good question. If this fairly restricted form solves the problems, then 
there is no need for the more flexible form. Things can always be made 
more flexible in the future, but tightening things can be pretty 
disruptive. Hence, unless there is an obvious and fairly strong case 
case for the flexibility, then it should be avoided for now.


What about:

size_t put(sink, parameters...)

Does this qualify as the sink being the "return" type? Obviously the 
real return can't contain any references, so it trivially can be ruled 
out as the destination of any escaping parameters.


Or how about a member function that takes a ref parameter? Is `this` the 
"return" or is the ref parameter the "return"?


My problem with the idea is that it is going to seem flaky -- we are 
using convention to dictate what is actually the return parameter, vs. 
what semantically happens inside the function. It's going to confuse 
anyone trying to do it a different way. I've experienced this in the 
past with things like toHash, where if you didn't define it with the 
exact signature, it wouldn't actually be used.


I realize obviously, that `put` is already specified. But as I said in 
the bug report, we should think twice about defining rules based solely 
on how Phobos does things, and calling that the solution.


As for things being made "more flexible in the future" this basically 
translates to code breakage. For example, if you are depending on only 
the first parameter being considered the "return" value, and all of a 
sudden it changes to encompass all your parameters, your existing code 
may fail to compile, even if it's correctly safe and properly annotated.




I want to ensure Atila is successful with this. But that means Phobos 
has to compile with dip1000. So I need to make it work.




I think it's a very worthy goal to make Phobos work, and a great proof 
of concept for dip1000's veracity.


However, one-off rules just to make it work with existing code go 
against that goal IMO. Rules that stand on their own I think will fare 
better than ones that are loopholes to allow existing code to compile.


-Steve

Re: Is @safe still a work-in-progress?


On Wednesday, 22 August 2018 at 11:02:00 UTC, Seb wrote:

No, it's behind a flag, so you can't really say that we're 
shipping it as "production ready release".


The changes to Phobos are not behind a flag.  We're making 
changes to Phobos in the release branch to accommodate a 
draft/experimental/choose-your-adjective feature.


Mike

Re: Is @safe still a work-in-progress?

2018-08-22 Thread Seb via Digitalmars-d


On Wednesday, 22 August 2018 at 02:18:15 UTC, Mike Franklin wrote:
On Wednesday, 22 August 2018 at 01:07:28 UTC, Mike Franklin 
wrote:



But what bothers me the most...


Something else that rubs me the wrong way is that DIP 1000 is 
currently in a status of `DRAFT`:  
https://github.com/dlang/DIPs/blob/master/DIPs/README.md


What the heck is going on here?  We're adding features to the 
compiler and modifying Phobos in production releases based on a 
`DRAFT` proposal?


No, it's behind a flag, so you can't really say that we're 
shipping it as "production ready release".
In fact I think we should have a hell of a lot more of such 
experimental flags.
This would allow us to be able to merge things quickly, and gain 
real-world feedback and testing on complicated matters instead of 
PRs stalling to death in the queue.


For reference, Rust has currently 148 opt-in experimental 
language features:


https://doc.rust-lang.org/unstable-book/index.html

Re: Is @safe still a work-in-progress?

2018-08-22 Thread Atila Neves via Digitalmars-d


On Wednesday, 22 August 2018 at 09:05:13 UTC, Walter Bright wrote:

On 8/21/2018 8:58 PM, Nicholas Wilson wrote:

On Tuesday, 21 August 2018 at 14:31:02 UTC, Atila Neves wrote:
The problem is that the code we write doesn't deal directly 
with pointers - see the recent confusion in this forum over 
where `scope` on the left applies to the `this` pointer or 
the one returned by the member function.


Kagamin just told me I needed to use `return` instead of 
`scope` to get things to work and I'm still not sure why.


The way I think about it is if you have a function that takes 
a pointer, any pointer, and either returns it or a pointer 
derived from it (dereferencing or indexing) that argument must 
be marked `return`. In your case it was a pointer derived from 
`this` so `return` must be applied to `this`.



Another way to think about it is this:

   S s;
   return 

We all know that is an error. The idea is to have a way to 
express that for:


S s;
return s.foo();

and:

S s;
return foo();

so that the compiler knows that the return value of foo() is 
attached to the lifetime of s. Pretty much everything flows 
from that.


Would the guideline below be correct?

"Add scope to every non-template member function that isn't meant 
to escape this and add return to every non-template member 
function that returns all or part of `this` by pointer or ref if 
you want the compiler to check that nothing gets escaped in @safe 
code."

Re: Is @safe still a work-in-progress?


On Wednesday, 22 August 2018 at 09:23:26 UTC, Walter Bright wrote:

dip1000 has been around for two years, and its predecessor 
dip25 several years now. Plenty of time for anyone to comment 
and/or propose something better.


Part of the problem is that the implementation keeps changing 
without keeping the documentation in sync.  For example you're 
implementing all of these inference rules without documenting 
them:


https://github.com/dlang/dmd/pull/8346
https://github.com/dlang/dmd/pull/8408

I asked you about, instead of inferring the attributes, allowing 
users to add such logic themselves:


void foo(T)(T x)
if (__traits(isPointer, T)) { T = scope T; }
{ }

But, then the PR got rubber-stamped, and now here we are.

Mike

Re: Is @safe still a work-in-progress?


On Wednesday, 22 August 2018 at 09:23:26 UTC, Walter Bright wrote:

I want to ensure Atila is successful with this. But that means 
Phobos has to compile with dip1000. So I need to make it work.


There's a good chance you'll get your PR merged when it's 
documented and vetted, then you'll be able to make it work for 
Atila.  The only think holding it up is you.


I'm not asking for much.  Sufficiently document the idea, open it 
up for comment and Q & A, transfer the documentation to the spec 
to accompany the DMD implementation PR.


Assuming there's no major flaws in the design, it should get 
merged.


Mike

P.S. Actually, I've been trying to document it myself, since you 
don't seem willing to, but it's going to take me a lot longer to 
figure out what's in your head than it would take you.

Re: Is @safe still a work-in-progress?


On Wednesday, 22 August 2018 at 09:23:26 UTC, Walter Bright wrote:

The proposed idea wants to make the first parameter, if it's 
`ref`, special.


This is because Phobos is written with functions of the form:

void put(sink, parameters...)

which corresponds to:

sink.put(parameters...)

The two forms are fairly interchangeable, made more so by the 
Uniform Function Call Syntax.


Makes perfect sense. This is the kind of stuff I'd like you to 
put in a formal document and present to us as an RFC along with 
your PR.  Then transfer that information to the spec to accompany 
the DMD PR, after you've received feedback.


Mike

Re: Is @safe still a work-in-progress?

2018-08-22 Thread Walter Bright via Digitalmars-d


On 8/21/2018 6:07 PM, Mike Franklin wrote:

The proposed idea wants to make the first parameter, if it's `ref`, special.


This is because Phobos is written with functions of the form:

void put(sink, parameters...)

which corresponds to:

sink.put(parameters...)

The two forms are fairly interchangeable, made more so by the Uniform Function 
Call Syntax.


> Why not the first `ref` parameter regardless of whether it's the absolute 
first in the list.  Why not the last `ref` parameter?  Why not all `ref` parameters?


Good question. If this fairly restricted form solves the problems, then there is 
no need for the more flexible form. Things can always be made more flexible in 
the future, but tightening things can be pretty disruptive. Hence, unless there 
is an obvious and fairly strong case case for the flexibility, then it should be 
avoided for now.



But what bothers me the most is I think it's missing the bigger picture:  D 
needs a way to annotate lifetimes.  Maybe `scope` and `return` with weird 
conditions based on the order of parameters and their attributes are the way to 
go.  Maybe there's another way that hasn't yet been considered.


Put together a thorough description of the proposal, justify it, ask the larger 
community for comment, vet it, and document it. At least that's what it's going 
to take to get me to take action on the PR.


dip1000 has been around for two years, and its predecessor dip25 several years 
now. Plenty of time for anyone to comment and/or propose something better.


---

I want to ensure Atila is successful with this. But that means Phobos has to 
compile with dip1000. So I need to make it work.

Re: Is @safe still a work-in-progress?

2018-08-22 Thread Walter Bright via Digitalmars-d


On 8/21/2018 8:58 PM, Nicholas Wilson wrote:

On Tuesday, 21 August 2018 at 14:31:02 UTC, Atila Neves wrote:
The problem is that the code we write doesn't deal directly with pointers - 
see the recent confusion in this forum over where `scope` on the left applies 
to the `this` pointer or the one returned by the member function.


Kagamin just told me I needed to use `return` instead of `scope` to get things 
to work and I'm still not sure why.


The way I think about it is if you have a function that takes a pointer, any 
pointer, and either returns it or a pointer derived from it (dereferencing or 
indexing) that argument must be marked `return`. In your case it was a pointer 
derived from `this` so `return` must be applied to `this`.



Another way to think about it is this:

   S s;
   return 

We all know that is an error. The idea is to have a way to express that for:

S s;
return s.foo();

and:

S s;
return foo();

so that the compiler knows that the return value of foo() is attached to the 
lifetime of s. Pretty much everything flows from that.

Re: Is @safe still a work-in-progress?

2018-08-22 Thread Walter Bright via Digitalmars-d


On 8/21/2018 2:17 PM, Atila Neves wrote:
Well, no. The syntax isn't the same for member functions. The examples from the 
actual DIP don't compile. There it says:


---
scope can be applied to function return values (even though it is not a type 
qualifier). It must be applied to the left of the declaration, in the same way 
ref is:


scope int* foo(); // applies to return value


Except:

---
struct MyStruct { scope int* foo() scope; }

foo.d(1): Error: redundant attribute scope
---
Meaning the first `scope` actually applies to `this`. Writing this out as a 
non-member function won't help me declare member functions!


I still don't know how to return a ref/pointer that's scoped. And I thought I'd 
written code that did that. Maybe I did. I'm very confused.


Here's how you make it work:

---
@safe:

struct MyStruct {
ref int foo() return;
int* bar() return;
}

ref int sun() {
MyStruct s;
return s.foo(); // returning s.foo() escapes a reference to local variable s
}

int* moon() {
MyStruct s;
return s.bar(); // returning s.bar() escapes a reference to local variable s
}
---
In effect, the 'return' on 'foo()' says:

The return value of foo() contains the address of 'this', and if the return 
escapes the scope of what 'this' is a ref to, then it's an error.

Re: Is @safe still a work-in-progress?

2018-08-22 Thread Atila Neves via Digitalmars-d

On Wednesday, 22 August 2018 at 03:58:42 UTC, Nicholas Wilson 
wrote:

On Tuesday, 21 August 2018 at 14:31:02 UTC, Atila Neves wrote:
The problem is that the code we write doesn't deal directly 
with pointers - see the recent confusion in this forum over 
where `scope` on the left applies to the `this` pointer or the 
one returned by the member function.


Kagamin just told me I needed to use `return` instead of 
`scope` to get things to work and I'm still not sure why.


The way I think about it is if you have a function that takes a 
pointer, any pointer, and either returns it or a pointer 
derived from it (dereferencing or indexing) that argument must 
be marked `return`. In your case it was a pointer derived from 
`this` so `return` must be applied to `this`.


I guess my problem is that DIP1000 talks about returning scope 
values and they don't seem to actually exist in the 
implementation.

Re: Is @safe still a work-in-progress?

2018-08-22 Thread Patrick Schluter via Digitalmars-d


On Wednesday, 22 August 2018 at 04:49:15 UTC, Mike Franklin wrote:
On Wednesday, 22 August 2018 at 04:23:52 UTC, Jonathan M Davis 
wrote:


The reality of the matter is that the DIP system is a formal 
way to propose language changes in order to convince Walter 
and Andrei that those changes should be implemented, whereas 
if Walter or Andrei writes the DIP, they're already convinced. 
This isn't a democracy. Walter is the BDFL, and it's his call. 
So, I really don't think that it's hypocritical


Walter and Andrei need to have their ideas vetted by the 
community, not in an effort to convince anyone, but for quality 
assurance, to ensure they're not overlooking something.


It is hypocritical an arrogant to believe that only our ideas 
have flaws and require scrutiny.


The formal DIP process was put in place after DIP1000. I would 
even daresay that the process was put in place because of the 
issue with DIP1000 (the rigorously checked DIP's are all >1000 
for that reason).

Re: Is @safe still a work-in-progress?

2018-08-22 Thread walker via Digitalmars-d


On Wednesday, 22 August 2018 at 02:18:15 UTC, Mike Franklin wrote:
Something else that rubs me the wrong way is that DIP 1000 is 
currently in a status of `DRAFT`:  
https://github.com/dlang/DIPs/blob/master/DIPs/README.md


What the heck is going on here?  We're adding features to the 
compiler and modifying Phobos in production releases based on a 
`DRAFT` proposal?


Furthermore, I find it hypocritical that some of us are put 
through a disproportionately burdensome DIP process requiring 
thorough documentation, multiple peer reviews, excessive 
delays, and judgment that defaults to "no" for some of the most 
minute changes to the language, but a game-changing feature 
like DIP 1000 can just be amended on a whim.


+1,
For valuable contributors like you, it's far from friendly.
Friendly atmosphere increases creativity.
Although, for an industrial-level language, it should be careful 
to add many features (A core engine that can simulate a lot of 
easy to use sugars is important, for those potential hackers. )

Re: Is @safe still a work-in-progress?


On Wednesday, 22 August 2018 at 05:39:05 UTC, Mike Franklin wrote:

I understand that Walter's DIPs have been put through the 
process just like the others, but with regard to the specific 
issue in this thread 
(https://issues.dlang.org/show_bug.cgi?id=19097), the 
accompanying PR (https://github.com/dlang/dmd/pull/8504), and 
similarly undocumented PR 
(https://github.com/dlang/dmd/pull/8408), they are amending DIP 
1000 and DIP 25 "under the table".  There is no accompanying PR 
to the specification, no formal rational, no RFC from the 
community, etc...


A 3rd example (https://github.com/dlang/dmd/pull/8346) to throw a 
little more salt on the wound.  The DIP25/1000 rabbit hole is 
deepening behind the curtain.


Mike

Re: Is @safe still a work-in-progress?


On Wednesday, 22 August 2018 at 05:04:25 UTC, Mike Parker wrote:

Whatever the status of DIP 1000, I would point out that that 
one of Walter's DIPs is in Community Review right now after 
sitting in the PR queue in Draft Review for a while. Once this 
review stage is done, it will go back into the queue to await 
Final Review like any other DIP. The only difference between 
this and other DIPs is that there will be no Formal Assessment 
for it. And this is the second DIP from Walter that has gone 
through the process since I've come on board, with DIP 1008 
being put on pause at his request.


I understand that Walter's DIPs have been put through the process 
just like the others, but with regard to the specific issue in 
this thread (https://issues.dlang.org/show_bug.cgi?id=19097), the 
accompanying PR (https://github.com/dlang/dmd/pull/8504), and 
similarly undocumented PR 
(https://github.com/dlang/dmd/pull/8408), they are amending DIP 
1000 and DIP 25 "under the table".  There is no accompanying PR 
to the specification, no formal rational, no RFC from the 
community, etc...


Yet I and others have to go through the DIP process for much less 
significant changes to the language, and rightly so:


https://github.com/dlang/dmd/pull/7310
https://github.com/dlang/dmd/pull/7079

All I want to see from Walter is:
1) a sufficiently documented proposal for his idea
2) an RFC from the community
3) a PR to the spec documenting the final design

I don't think that's too much to ask.

Mike

Re: Is @safe still a work-in-progress?


On Wednesday, 22 August 2018 at 04:49:15 UTC, Mike Franklin wrote:

It is hypocritical an arrogant to believe that only our ideas 
have flaws and require scrutiny.


Sorry, that was poorly stated and conveyed the wrong intent.  It 
should read:


It is hypocritical an arrogant to believe that only our ideas 
should require thorough documentation and scrutiny.


Mike

Re: Is @safe still a work-in-progress?

2018-08-21 Thread Mike Parker via Digitalmars-d


On Wednesday, 22 August 2018 at 02:18:15 UTC, Mike Franklin wrote:

Furthermore, I find it hypocritical that some of us are put 
through a disproportionately burdensome DIP process requiring 
thorough documentation, multiple peer reviews, excessive 
delays, and judgement that defaults to "no" for some of the 
most minute changes to the language, but a game-changing 
feature like DIP 1000 can just be amended on a whim.




DIP 1000 is in a bit of limbo at the moment. When I took over the 
process from Mihails, he told me it was stalled and that the 
proposal did not match the implementation. So I haven't touched 
it, which is why it's still marked as Draft. At some point, 
Walter will revise it to match the implementation and then we'll 
discuss how to handle it.


Whatever the status of DIP 1000, I would point out that that one 
of Walter's DIPs is in Community Review right now after sitting 
in the PR queue in Draft Review for a while. Once this review 
stage is done, it will go back into the queue to await Final 
Review like any other DIP. The only difference between this and 
other DIPs is that there will be no Formal Assessment for it. And 
this is the second DIP from Walter that has gone through the 
process since I've come on board, with DIP 1008 being put on 
pause at his request.

Re: Is @safe still a work-in-progress?

On Wednesday, 22 August 2018 at 04:23:52 UTC, Jonathan M Davis 
wrote:


The reality of the matter is that the DIP system is a formal 
way to propose language changes in order to convince Walter and 
Andrei that those changes should be implemented, whereas if 
Walter or Andrei writes the DIP, they're already convinced. 
This isn't a democracy. Walter is the BDFL, and it's his call. 
So, I really don't think that it's hypocritical


Walter and Andrei need to have their ideas vetted by the 
community, not in an effort to convince anyone, but for quality 
assurance, to ensure they're not overlooking something.


It is hypocritical an arrogant to believe that only our ideas 
have flaws and require scrutiny.


Mike

Re: Is @safe still a work-in-progress?

2018-08-21 Thread Jonathan M Davis via Digitalmars-d

On Tuesday, August 21, 2018 8:18:15 PM MDT Mike Franklin via Digitalmars-d 
wrote:
> On Wednesday, 22 August 2018 at 01:07:28 UTC, Mike Franklin wrote:
> > But what bothers me the most...
>
> Something else that rubs me the wrong way is that DIP 1000 is
> currently in a status of `DRAFT`:
> https://github.com/dlang/DIPs/blob/master/DIPs/README.md
>
> What the heck is going on here?  We're adding features to the
> compiler and modifying Phobos in production releases based on a
> `DRAFT` proposal?
>
> Furthermore, I find it hypocritical that some of us are put
> through a disproportionately burdensome DIP process requiring
> thorough documentation, multiple peer reviews, excessive delays,
> and judgement that defaults to "no" for some of the most minute
> changes to the language, but a game-changing feature like DIP
> 1000 can just be amended on a whim.

The reality of the matter is that the DIP system is a formal way to propose
language changes in order to convince Walter and Andrei that those changes
should be implemented, whereas if Walter or Andrei writes the DIP, they're
already convinced. This isn't a democracy. Walter is the BDFL, and it's his
call. So, I really don't think that it's hypocritical, but I also do think
that DIP 1000 probably should have gone through more peer review. From what
I can tell, outside of the simple cases, pretty much everyone has a really
hard time understanding it. The situation will likely improve once Phobos
properly supports it, and you can more reasonably use it, but the whole
thing defnitely seems to be overly complicated given what it's supposed to
be doing and what benefits you get from it. Personally, I think that it
seems pretty reasonable as long as user-defined types don't get involved,
but once they do, it's a mess.

- Jonathan M Davis

Re: Is @safe still a work-in-progress?

2018-08-21 Thread Nicholas Wilson via Digitalmars-d


On Tuesday, 21 August 2018 at 14:31:02 UTC, Atila Neves wrote:
The problem is that the code we write doesn't deal directly 
with pointers - see the recent confusion in this forum over 
where `scope` on the left applies to the `this` pointer or the 
one returned by the member function.


Kagamin just told me I needed to use `return` instead of 
`scope` to get things to work and I'm still not sure why.


The way I think about it is if you have a function that takes a 
pointer, any pointer, and either returns it or a pointer derived 
from it (dereferencing or indexing) that argument must be marked 
`return`. In your case it was a pointer derived from `this` so 
`return` must be applied to `this`.

Re: Is @safe still a work-in-progress?


On Wednesday, 22 August 2018 at 01:07:28 UTC, Mike Franklin wrote:


But what bothers me the most...


Something else that rubs me the wrong way is that DIP 1000 is 
currently in a status of `DRAFT`:  
https://github.com/dlang/DIPs/blob/master/DIPs/README.md


What the heck is going on here?  We're adding features to the 
compiler and modifying Phobos in production releases based on a 
`DRAFT` proposal?


Furthermore, I find it hypocritical that some of us are put 
through a disproportionately burdensome DIP process requiring 
thorough documentation, multiple peer reviews, excessive delays, 
and judgement that defaults to "no" for some of the most minute 
changes to the language, but a game-changing feature like DIP 
1000 can just be amended on a whim.


Mike

Re: Is @safe still a work-in-progress?

2018-08-20 Thread Steven Schveighoffer via Digitalmars-d


On Tuesday, 21 August 2018 at 21:17:25 UTC, Atila Neves wrote:

I don't have merge rights. I took a look anyway and it mostly 
looks ok, but I'm not familiar enough with that part of the 
codebase.


It's not the implementation that's preventing it from being 
merged.  It's the idea itself, weak rationale, lack of peer 
review, lack of consideration for alternatives, and lack of 
documentation supporting it.  It reeks of "designed on a whim to 
quickly patch some oversight in the design DIP1000 while trying 
to get Phobos to compile with -dip1000".


With the proposed idea, order of parameters matters.  We'll need 
to establish a convention that return parameters must be declared 
first, which is opposite of 
https://dlang.org/phobos/std_algorithm_mutation.html#copy. Is 
that a good idea?  Maybe it is. We haven't vetted the design yet, 
so I'm not sure.  Why haven't we vetted the design?  Because 
there currently isn't one; there's just an informal 
back-of-the-napkin memo uploaded to a bugzilla entry.


The proposed idea wants to make the first parameter, if it's 
`ref`, special. Why not the first `ref` parameter regardless of 
whether it's the absolute first in the list.  Why not the last 
`ref` parameter?  Why not all `ref` parameters?


But what bothers me the most is I think it's missing the bigger 
picture:  D needs a way to annotate lifetimes.  Maybe `scope` and 
`return` with weird conditions based on the order of parameters 
and their attributes are the way to go.  Maybe there's another 
way that hasn't yet been considered.


Put together a thorough description of the proposal, justify it, 
ask the larger community for comment, vet it, and document it.  
At least that's what it's going to take to get me to take action 
on the PR.


Or maybe someone else is willing to just rubber stamp it in the 
interest of expediency.


Mike

Re: Is @safe still a work-in-progress?

2018-08-21 Thread Atila Neves via Digitalmars-d


On Tuesday, 21 August 2018 at 19:36:39 UTC, Walter Bright wrote:

On 8/21/2018 7:31 AM, Atila Neves wrote:
The problem is that the code we write doesn't deal directly 
with pointers - see the recent confusion in this forum over 
where `scope` on the left applies to the `this` pointer or the 
one returned by the member function.


That's what I was saying :-)

The way to deal with this is make a copy of the code, then 
rewrite it so it does the exact same thing, but with pointers 
and refs only. No member functions, no delegates, no dynamic 
arrays. Then it is MUCH MUCH easier to see where the 
annotations go.



Well, no. The syntax isn't the same for member functions. The 
examples from the actual DIP don't compile. There it says:


---
scope can be applied to function return values (even though it is 
not a type qualifier). It must be applied to the left of the 
declaration, in the same way ref is:


scope int* foo(); // applies to return value


Except:

---
struct MyStruct { scope int* foo() scope; }

foo.d(1): Error: redundant attribute scope
---

Meaning the first `scope` actually applies to `this`. Writing 
this out as a non-member function won't help me declare member 
functions!


I still don't know how to return a ref/pointer that's scoped. And 
I thought I'd written code that did that. Maybe I did. I'm very 
confused.



BTW, the annotations do not break things. The worst that will 
happen is the compiler will complain in @safe code that they 
are incorrect, and you'll need to fix it or make it @trusted. 
The compiler is also pretty good about inferring the correct 
annotations, at least for templates and lambdas, which helps 
enormously.


In my opinion, the worst that can happen is I successfully 
compile my @safe code with -dip1000 and the resulting binary 
isn't memory-safe, which is what's been happening to me.


However, dip1000 not working with Phobos is a huge impediment 
to success, and so pulling 8504 is critical.


I don't have merge rights. I took a look anyway and it mostly 
looks ok, but I'm not familiar enough with that part of the 
codebase.

Re: Is @safe still a work-in-progress?

2018-08-21 Thread Walter Bright via Digitalmars-d


On 8/21/2018 7:31 AM, Atila Neves wrote:
The problem is that the code we write doesn't deal directly with pointers - see 
the recent confusion in this forum over where `scope` on the left applies to the 
`this` pointer or the one returned by the member function.


That's what I was saying :-)

The way to deal with this is make a copy of the code, then rewrite it so it does 
the exact same thing, but with pointers and refs only. No member functions, no 
delegates, no dynamic arrays. Then it is MUCH MUCH easier to see where the 
annotations go.



Kagamin just told me I needed to use `return` instead of `scope` to get things 
to work and I'm still not sure why.


Also: destructors? Always `scope`? Sometimes? I just add `scope` when the 
compiler complains at this point.


I think it's interesting that when I played with Rust I didn't have problems 
fighting the borrow checker at all. DIP1000 is supposed to have the same safety 
without the Rust complications but currently Rust is much easier to understand 
for me.


It doesn't help that the current implementation of -dip1000 doesn't match the 
document it's supposedly based on.


All good points. But I cannot make any progress when nobody is willing to pull 
my PRs that improve the situation.


https://github.com/dlang/dmd/pull/8504

BTW, the annotations do not break things. The worst that will happen is the 
compiler will complain in @safe code that they are incorrect, and you'll need to 
fix it or make it @trusted. The compiler is also pretty good about inferring the 
correct annotations, at least for templates and lambdas, which helps enormously.


However, dip1000 not working with Phobos is a huge impediment to success, and so 
pulling 8504 is critical.

Re: Is @safe still a work-in-progress?

2018-08-21 Thread Atila Neves via Digitalmars-d


On Tuesday, 21 August 2018 at 00:09:03 UTC, Walter Bright wrote:

On 8/20/2018 6:46 AM, Steven Schveighoffer wrote:
I would, but I have no idea how dip1000 is supposed to work. I 
think only you understand it. Even looking at the PR that you 
have been citing over and over, I can't make heads or tails of 
what it does or what it allows.


The way to understand dip1000 is to forget about `this`, 
arrays, structs, classes, delegates, etc. Just think about 
pointers - functions that take pointers as arguments, and 
return pointers.


That simplifies things enormously.

Once that makes sense, then deconstruct the higher level 
constructs into pointers, and then you'll see how they work.


This is why my presentation does it all in terms of pointers:

http://dconf.org/2017/talks/bright.html

If you're having trouble understanding a particular example, 
rewrite it in terms of pointers. If it still is inscrutable, 
then ask about the pointer version here.


(When someone gives me some complex thing and says "I don't 
understand scope here", I always first rewrite it in terms of 
pointers. It's the same thing I do with bug reports that use 
templates - I manually instantiate the templates first.)


The problem is that the code we write doesn't deal directly with 
pointers - see the recent confusion in this forum over where 
`scope` on the left applies to the `this` pointer or the one 
returned by the member function.


Kagamin just told me I needed to use `return` instead of `scope` 
to get things to work and I'm still not sure why.


Also: destructors? Always `scope`? Sometimes? I just add `scope` 
when the compiler complains at this point.


I think it's interesting that when I played with Rust I didn't 
have problems fighting the borrow checker at all. DIP1000 is 
supposed to have the same safety without the Rust complications 
but currently Rust is much easier to understand for me.


It doesn't help that the current implementation of -dip1000 
doesn't match the document it's supposedly based on.

Re: Is @safe still a work-in-progress?

2018-08-20 Thread Walter Bright via Digitalmars-d


On 8/20/2018 6:46 AM, Steven Schveighoffer wrote:
I would, but I have no idea how dip1000 is supposed to work. I think only you 
understand it. Even looking at the PR that you have been citing over and over, I 
can't make heads or tails of what it does or what it allows.


The way to understand dip1000 is to forget about `this`, arrays, structs, 
classes, delegates, etc. Just think about pointers - functions that take 
pointers as arguments, and return pointers.


That simplifies things enormously.

Once that makes sense, then deconstruct the higher level constructs into 
pointers, and then you'll see how they work.


This is why my presentation does it all in terms of pointers:

http://dconf.org/2017/talks/bright.html

If you're having trouble understanding a particular example, rewrite it in terms 
of pointers. If it still is inscrutable, then ask about the pointer version here.


(When someone gives me some complex thing and says "I don't understand scope 
here", I always first rewrite it in terms of pointers. It's the same thing I do 
with bug reports that use templates - I manually instantiate the templates first.)

Re: Is @safe still a work-in-progress?

2018-08-20 Thread Walter Bright via Digitalmars-d


On 8/20/2018 6:08 AM, Atila Neves wrote:

As mentioned in my other thread, I went to file one and it was declared not a 
bug:

https://issues.dlang.org/show_bug.cgi?id=17927


I'll look at it again.

Re: Is @safe still a work-in-progress?

2018-08-20 Thread Jonathan M Davis via Digitalmars-d

On Monday, August 20, 2018 7:46:48 AM MDT Steven Schveighoffer via 
Digitalmars-d wrote:
> I would, but I have no idea how dip1000 is supposed to work. I think
> only you understand it. Even looking at the PR that you have been citing
> over and over, I can't make heads or tails of what it does or what it
> allows.

The basic idea of DIP 1000 seems simple enough, but once you get into the
details, it gets hard to understand fast.

And from what work I've done to make std.datetime work with it better, I'm
very much inclined to think that it's just too intrusive for too little
gain. I mean, it's great that you can know that a pointer isn't going to
escape, but once you have to start marking up member functions all over the
place just so that the type can work with scope? It just seems like it's too
viral. I honestly do not understand why much of anything in
std.datetime.systime needs scope, but I was forced to mark up SysTime all
over the place in order for scope variables of type SysTime to work just
because it contains a class reference - one which cannot posssibly ever be
on the stack, since you can't pass a scope argument to a member function to
have it then stored in a member variable. So, why it has any effect on why
scope is required on member functions, I have no idea.

User-defined types can't contain anything that's scope (since that would
mean that they escaped), and yet, they potentially have to be marked up with
scope all over the place in case someone qualifies a variable of that type
with scope somewhere. And of course, because of how stuff like ranges work,
you're quickly in trouble if they're marked with scope, since then they
can't be wrapped by other ranges, making them useless for pretty much
anything other than foreach. scope seems great for toy examples or stuff
that's extern(C), but once user-defined types get involved, it's a mess.

I've got partially completed changes to std.datetime.interval towards making
it work with scope better that I need to get back to, but to be honest, the
more I've tried to do anything with -dip1000, the less I've like it, and
it's really hard to be motivated to go to the effort of making anything
compatible with it.

- Jonathan M Davis

Re: Is @safe still a work-in-progress?


On 8/17/18 11:04 PM, Walter Bright wrote:

On 8/17/2018 11:17 AM, bachmeier wrote:
This is a good example of D needing to evolve or fizzle out. I don't 
see evidence that the community has yet figured out how to evolve the 
language. If it had, these problems would not be around for so many 
years.


We deprecate features of D all the time. (Remember the D1 => D2 
wrenching change?)


Hm... if you are going for "all the time", the example of D1 to D2 
transition is pretty dated.


I'd say more like the addition of UDAs was a big evolution. Or maybe UFCS.

The reason @safe cannot be default at the moment it because -dip1000 
needs work, and nobody is willing to pitch in and review/pull my PRs on it.


I would, but I have no idea how dip1000 is supposed to work. I think 
only you understand it. Even looking at the PR that you have been citing 
over and over, I can't make heads or tails of what it does or what it 
allows.


-Steve

Re: Is @safe still a work-in-progress?

2018-08-20 Thread Atila Neves via Digitalmars-d


On Saturday, 18 August 2018 at 02:17:01 UTC, Walter Bright wrote:

On 8/17/2018 4:33 AM, Atila Neves wrote:

I've been using -dip1000 a lot lately.


Great news!


:)



I hit two bugs yesterday.


In D? or in your code?


In dmd.


Bugs don't count if they're not in bugzilla!


As mentioned in my other thread, I went to file one and it was 
declared not a bug:


https://issues.dlang.org/show_bug.cgi?id=17927

I don't know how that's possible given that supposedly @safe code 
is doing pretty bad things. That disheartened me enough to not 
continue although I did file this one:


https://issues.dlang.org/show_bug.cgi?id=19173

Re: Is @safe still a work-in-progress?


On 8/17/2018 5:30 AM, Atila Neves wrote:

It's not easy though. You have to either be building your own phobos


Not really. You only have to be able to import phobos files. You don't have to 
compile something that will link.


I.e. just copy the phobos source tree, add @safe to it, and point the compiler 
at that one.



or sudo 
editing files in /usr/include. You can, but it's a pain. Then there's finding 
out exactly where in a chain of 10 template functions that it became @system...


I recently fixed the compiler to say which template overload was used. It's a 
huge time saver.


But you're currently doomed on this anyway, as an awful lot of Phobos winds up 
calling put(), which cannot be inferred as @safe without:


https://github.com/dlang/dmd/pull/8504

Re: Is @safe still a work-in-progress?


On 8/17/2018 11:17 AM, bachmeier wrote:
This is a good example of D needing to evolve or fizzle out. I don't see 
evidence that the community has yet figured out how to evolve the language. If 
it had, these problems would not be around for so many years.


We deprecate features of D all the time. (Remember the D1 => D2 wrenching 
change?)

The reason @safe cannot be default at the moment it because -dip1000 needs work, 
and nobody is willing to pitch in and review/pull my PRs on it.

Re: Is @safe still a work-in-progress?


On 8/17/2018 4:33 AM, Atila Neves wrote:

I've been using -dip1000 a lot lately.


Great news!


I hit two bugs yesterday.


In D? or in your code?


When it works, it's great, _except_:

. @safe isn't default
. -dip1000 isn't default
. Good luck figuring out why your template functions aren't @safe and coaxing 
the compiler to tell you why it's inferring the attributes it is.


I empathize with that, having run into the same problem. A couple months ago I 
improved the resulting error messages, which make it a lot easier to track down 
why it is inferring what it does.


Bonus points 
if it's a Phobos function so you can't slap `@safe` on its definition.


Sometimes I think it is better to just slap them with @safe and to hell with 
unsafe parameters. :-)



Did I mention the bugs?


Bugs don't count if they're not in bugzilla!

Re: Is @safe still a work-in-progress?


On 8/17/2018 3:01 AM, Jonathan M Davis wrote:

we still don't have a real plan as to how we're going to make
-dip1000 the default.
We do have a plan. Get Phobos to compile with -dip1000. As to why that hasn't 
happened yet, see:


https://github.com/dlang/dmd/pull/8504

Re: Is @safe still a work-in-progress?