Re: Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[Jonathan M Davis via Digitalmars-d]

On Thursday, October 20, 2016 23:18:14 Nicholas Wilson via Digitalmars-d 
wrote:
> On Thursday, 20 October 2016 at 01:04:35 UTC, Jonathan M Davis
>
> wrote:
> > The transitivity of const shoot stuff in the foot pretty
> > thoroughly in a number of cases. A prime example would be
> > ranges, because they have to be mutated to be iterated over. If
> > the function actually took a range directly, you wouldn't
> > bother with const ref, but it could be an object that contains
> > a range, and because you can't normally get a tail-const range
> > from a const range (aside from dynamic arrays), it can become
> > quite difficult to actually iterate over the range. e.g.
> >
> > auto foo(ref const(Bar) bar)
> > {
> >
> > auto range = bar.getSomeRange();
> > ...
> >
> > }
>
> Is it legal to `.save` a const range, and then use it (provided
> it does not mutate any object reachable from bar)?

Not really. isForwardRange requires that save return _exactly_ the same type
as the range. So, it really can't work as a const function (and
const(MyRangeType) will never pass isInputRange or isFowardRange, because
popFront won't compile). You _can_ declare an overload of save that's const
and returns something different, but then it's not actually part of the
range API, and you can't rely on it working. At that point, you might as
well make up your own function, since what you're doing is non-standand
anyway.

Regardless of all that though, it's not always even possible for a range to
have a function that returns a tail-const version of the range because of
how the internals of the range work. So, even if we had a standard way to
deal with that, it couldn't work in all cases, and really, it would have to
be dealt with separately from traits like isForwardRange. Arguably, to match
arrays, we should be defining an opSlice for ranges which returns a
tail-const version of the range (and then we could have a trait like
hasTailConstSlicing), but that's not currently part of the range API, and
it's actually pretty hard to define even for simple ranges (e.g. you have to
use static if carefully to avoid recursive template instantiations).

So, the best way to handle this is very much an open question. For now, the
reality of the matter is that there is no standard way for const ranges to
work.

- Jonathan M Davis

Re: Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[ag0aep6g via Digitalmars-d]

On 10/21/2016 01:18 AM, Nicholas Wilson wrote:

Is it legal to `.save` a const range, and then use it (provided it does
not mutate any object reachable from bar)?


Sure, if it doesn't involve a cast, i.e. if save is const.

Re: Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[Nicholas Wilson via Digitalmars-d]

On Thursday, 20 October 2016 at 01:04:35 UTC, Jonathan M Davis 
wrote:
The transitivity of const shoot stuff in the foot pretty 
thoroughly in a number of cases. A prime example would be 
ranges, because they have to be mutated to be iterated over. If 
the function actually took a range directly, you wouldn't 
bother with const ref, but it could be an object that contains 
a range, and because you can't normally get a tail-const range 
from a const range (aside from dynamic arrays), it can become 
quite difficult to actually iterate over the range. e.g.


auto foo(ref const(Bar) bar)
{
auto range = bar.getSomeRange();
...
}



Is it legal to `.save` a const range, and then use it (provided 
it does not mutate any object reachable from bar)?


Because getSomeRange would have to be const (or inout) in order 
to be called, the return type is going to have be const if 
there are any indirections in it. And if it's const, and it's a 
range, it's useless.


If this were C++, const wouldn't be transitive, so it would be 
trivial to get an iterator or range which wasn't const and did 
not violate const or attempt to work around it in any way and 
thus could be iterated. But in D, const is far too restrictive 
for that to work.


This is the kind of problem that quickly makes you start 
wondering whether it's even worth trying to use const in D at 
all.


- Jonathan M Davis

Re: Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[Andrei Alexandrescu via Digitalmars-d]

On 10/19/16 9:04 PM, Walter Bright wrote:

The only way to move this forward is to write a DIP.


I'd be willing to shepherd a DIP if a couple of people want to get 
serious about it. -- Andrei

Re: Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[Manu via Digitalmars-d]

On 20 October 2016 at 11:04, Walter Bright via Digitalmars-d <
digitalmars-d@puremagic.com> wrote:

> On 10/19/2016 5:26 PM, Manu via Digitalmars-d wrote:
>
>> Right, I was arguing this for years. Using 'scope' to make the concept
>> @safe.
>> It seemed that it in the past the key reason for rejecting it was because
>> it was
>> unsafe to pass an rvalue-temp to a function where it's unknown if the
>> function
>> can cause it to outlive the function call... scope was the obvious
>> resolution to
>> that (along with a lot of other issues related to safety).
>>
>
> C++ has had two goes at rvalue references. Any serious proposal for that
> for D needs to include an analysis of what went right/wrong with the C++
> one, and how the D one gets it right.
>

We're not talking about rvalue references...? I'm not sure where this
conversation got confused.

The only way to move this forward is to write a DIP. Having the various
> bits of information spread out over various posts for months (years?) is
> never going to work.
>

Like, 6 or 7 years ;) .. But I think we're talking about different things
at this point.

Re: Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[Jonathan M Davis via Digitalmars-d]

On Thursday, October 20, 2016 10:23:35 Manu via Digitalmars-d wrote:
> On 20 October 2016 at 01:38, Jonathan M Davis via Digitalmars-d <
>
> digitalmars-d@puremagic.com> wrote:
> > On Wednesday, October 19, 2016 07:55:19 Andrei Alexandrescu via
> > Digitalmars-d
> >
> > wrote:
> > > This was C++'s big un' that led to many complications. If the overload
> > > weren't ambiguous, a large part of rvalue references would have been
> > > unneeded. (Universal references would still have been necessary for
> > > perfect forwarding, but that's not the bulk.)
> > >
> > > In order to avoid such issues, we steered clear off binding rvalues to
> > > ref parameters in the D language. As I mentioned to Ethan, I do agree a
> > > careful definition may be able to avoid the fallout that happened in
> > > C++. It would be a fair amount of work.
> >
> > The other big problem is that D's const is so much more restrictive than
> > C++'s that even if const ref accepted rvalues, a large portion of the
> > time,
> > it would be too restrictive to be useful.
>
> I've never seen a piece of code in C++ that receives const& that isn't
> strictly read-only.
> I can't imagine from experience how D's const would change the usefulness
> of the pattern.

The transitivity of const shoot stuff in the foot pretty thoroughly in a
number of cases. A prime example would be ranges, because they have to be
mutated to be iterated over. If the function actually took a range directly,
you wouldn't bother with const ref, but it could be an object that contains
a range, and because you can't normally get a tail-const range from a const
range (aside from dynamic arrays), it can become quite difficult to actually
iterate over the range. e.g.

auto foo(ref const(Bar) bar)
{
auto range = bar.getSomeRange();
...
}

Because getSomeRange would have to be const (or inout) in order to be
called, the return type is going to have be const if there are any
indirections in it. And if it's const, and it's a range, it's useless.

If this were C++, const wouldn't be transitive, so it would be trivial to
get an iterator or range which wasn't const and did not violate const or
attempt to work around it in any way and thus could be iterated. But in D,
const is far too restrictive for that to work.

This is the kind of problem that quickly makes you start wondering whether
it's even worth trying to use const in D at all.

- Jonathan M Davis

Re: Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[Walter Bright via Digitalmars-d]

On 10/19/2016 5:26 PM, Manu via Digitalmars-d wrote:

Right, I was arguing this for years. Using 'scope' to make the concept @safe.
It seemed that it in the past the key reason for rejecting it was because it was
unsafe to pass an rvalue-temp to a function where it's unknown if the function
can cause it to outlive the function call... scope was the obvious resolution to
that (along with a lot of other issues related to safety).


C++ has had two goes at rvalue references. Any serious proposal for that for D 
needs to include an analysis of what went right/wrong with the C++ one, and how 
the D one gets it right.


The only way to move this forward is to write a DIP. Having the various bits of 
information spread out over various posts for months (years?) is never going to 
work.

Re: Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[Manu via Digitalmars-d]

On 20 October 2016 at 04:18, Namespace via Digitalmars-d <
digitalmars-d@puremagic.com> wrote:

> On Wednesday, 19 October 2016 at 18:15:25 UTC, Jonathan M Davis wrote:
>
>> Which then causes the problem that it becomes much less clear whether ref
>> is supposed to be modifying its argument or is just trying to avoid copying
>> it - though good documentation can mitigate that problem.
>>
>> - Jonathan M Davis
>>
>
> As long as it would be marked with scope (or some similar attribute) so
> that it cannot escape, it would be fine, wouldn't it?
>

Right, I was arguing this for years. Using 'scope' to make the concept
@safe.
It seemed that it in the past the key reason for rejecting it was because
it was unsafe to pass an rvalue-temp to a function where it's unknown if
the function can cause it to outlive the function call... scope was the
obvious resolution to that (along with a lot of other issues related to
safety).

Re: Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[Manu via Digitalmars-d]

On 20 October 2016 at 01:38, Jonathan M Davis via Digitalmars-d <
digitalmars-d@puremagic.com> wrote:

> On Wednesday, October 19, 2016 07:55:19 Andrei Alexandrescu via
> Digitalmars-d
> wrote:
> > This was C++'s big un' that led to many complications. If the overload
> > weren't ambiguous, a large part of rvalue references would have been
> > unneeded. (Universal references would still have been necessary for
> > perfect forwarding, but that's not the bulk.)
> >
> > In order to avoid such issues, we steered clear off binding rvalues to
> > ref parameters in the D language. As I mentioned to Ethan, I do agree a
> > careful definition may be able to avoid the fallout that happened in
> > C++. It would be a fair amount of work.
>
> The other big problem is that D's const is so much more restrictive than
> C++'s that even if const ref accepted rvalues, a large portion of the time,
> it would be too restrictive to be useful.
>

I've never seen a piece of code in C++ that receives const& that isn't
strictly read-only.
I can't imagine from experience how D's const would change the usefulness
of the pattern.

Re: Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[Namespace via Digitalmars-d]

On Wednesday, 19 October 2016 at 19:19:35 UTC, Jonathan M Davis 
wrote:
That's an orthogonal issue. My point is that normally, a 
parameter is a ref parameter, because the function is going to 
use that value and potentially mutate it in the process, and 
you want the original variable that was passed in to be mutated 
rather than for the function to be operating on a copy. 
However, once you can pass rvalues to ref parameters, there 
will likely be a sharp increase in the number of ref parameters 
whose entire purpose in being ref is to avoid a copy rather 
than because the original variable is supposed to be mutated. 
That increases the risk of accidentally mutating function 
arguments as well as making it far less obvious when a function 
is supposed to be mutating its argument. C++ solved that 
problem by making it so that only const ref parameters could 
take rvalues, whereas we would be totally open to it if 
non-const ref parameters accepted rvalues.


Whether the argument escapes the function doesn't matter for 
any of that. There may be good reasons why you don't want it 
to, in which case, if scope is implemented to prevent ref 
parameters from escaping, scope will give you that. But just 
because you want to mutate the ref argument doesn't necessarily 
mean that you care about it escaping. You _do_ care if the 
purpose is simply to avoid a copy, because you don't want the 
rvalue to escape, since that would be an @safety issue, and so 
it would make sense to require scope in that case, but at best, 
that means that the lack of scope indicates that the ref 
argument is supposed to be mutated as opposed to simply avoid a 
copy, whereas scope ref says nothing about whether the ref 
argument is supposed to be mutated or simply avoid a copy - 
just that whatever the argument is, it should no escape.


Ok, I understand what you mean, but as long as the argument 
cannot escape I have a different opinion.


So, arguably, it makes more sense to have a new attribute that 
makes it specifically so that a ref accepts rvalues (e.g. 
@rvalue ref) rather than making ref in general accept rvalues 
(the new attribute could even imply scope, since it would be 
required), but that would mean adding yet another attribute, 
and we arguably have too many of those already.


- Jonathan M Davis


Yes, we have way to many. So it would make more sense if we add 
helper/wrappers into phobos (at least for the time being) and 
refer to them.
My byRef "hack" or even my last experiment below could lower the 
dissatisfaction.



struct Vector2f
{
float x, y;
}

void one(ref const Vector2f v)
{
writeln(v.x, '|', v.y);
}

void two(ref const Vector2f source, ref const Vector2f target)
{
writefln("From (%.2f|%.2f) to (%.2f|%.2f)", source.x, 
source.y, target.x, target.y);

}

void invoke(alias f, V...)(V vs)
{
f(vs);
}

void invoke(F, V...)(F f, V vs)
{
f(vs);
}

invoke!one(Vector2f(10, 20));
invoke!two(Vector2f(1, 2), Vector2f(3, 4));

invoke(, Vector2f(10, 20));
invoke(, Vector2f(1, 2), Vector2f(3, 4));

Re: Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[Jonathan M Davis via Digitalmars-d]

On Wednesday, October 19, 2016 18:18:43 Namespace via Digitalmars-d wrote:
> On Wednesday, 19 October 2016 at 18:15:25 UTC, Jonathan M Davis
>
> wrote:
> > Which then causes the problem that it becomes much less clear
> > whether ref is supposed to be modifying its argument or is just
> > trying to avoid copying it - though good documentation can
> > mitigate that problem.
> >
> > - Jonathan M Davis
>
> As long as it would be marked with scope (or some similar
> attribute) so that it cannot escape, it would be fine, wouldn't
> it?

That's an orthogonal issue. My point is that normally, a parameter is a ref
parameter, because the function is going to use that value and potentially
mutate it in the process, and you want the original variable that was passed
in to be mutated rather than for the function to be operating on a copy.
However, once you can pass rvalues to ref parameters, there will likely be a
sharp increase in the number of ref parameters whose entire purpose in being
ref is to avoid a copy rather than because the original variable is supposed
to be mutated. That increases the risk of accidentally mutating function
arguments as well as making it far less obvious when a function is supposed
to be mutating its argument. C++ solved that problem by making it so that
only const ref parameters could take rvalues, whereas we would be totally
open to it if non-const ref parameters accepted rvalues.

Whether the argument escapes the function doesn't matter for any of that.
There may be good reasons why you don't want it to, in which case, if scope
is implemented to prevent ref parameters from escaping, scope will give you
that. But just because you want to mutate the ref argument doesn't
necessarily mean that you care about it escaping. You _do_ care if the
purpose is simply to avoid a copy, because you don't want the rvalue to
escape, since that would be an @safety issue, and so it would make sense to
require scope in that case, but at best, that means that the lack of scope
indicates that the ref argument is supposed to be mutated as opposed to
simply avoid a copy, whereas scope ref says nothing about whether the ref
argument is supposed to be mutated or simply avoid a copy - just that
whatever the argument is, it should no escape.

So, arguably, it makes more sense to have a new attribute that makes it
specifically so that a ref accepts rvalues (e.g. @rvalue ref) rather than
making ref in general accept rvalues (the new attribute could even imply
scope, since it would be required), but that would mean adding yet another
attribute, and we arguably have too many of those already.

- Jonathan M Davis

Re: Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[Namespace via Digitalmars-d]

On Wednesday, 19 October 2016 at 18:15:25 UTC, Jonathan M Davis 
wrote:
Which then causes the problem that it becomes much less clear 
whether ref is supposed to be modifying its argument or is just 
trying to avoid copying it - though good documentation can 
mitigate that problem.


- Jonathan M Davis


As long as it would be marked with scope (or some similar 
attribute) so that it cannot escape, it would be fine, wouldn't 
it?

Re: Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[Jonathan M Davis via Digitalmars-d]

On Wednesday, October 19, 2016 12:48:54 Andrei Alexandrescu via Digitalmars-d 
wrote:
> On 10/19/2016 11:38 AM, Jonathan M Davis via Digitalmars-d wrote:
> > On Wednesday, October 19, 2016 07:55:19 Andrei Alexandrescu via
> > Digitalmars-d>
> > wrote:
> >> This was C++'s big un' that led to many complications. If the overload
> >> weren't ambiguous, a large part of rvalue references would have been
> >> unneeded. (Universal references would still have been necessary for
> >> perfect forwarding, but that's not the bulk.)
> >>
> >> In order to avoid such issues, we steered clear off binding rvalues to
> >> ref parameters in the D language. As I mentioned to Ethan, I do agree a
> >> careful definition may be able to avoid the fallout that happened in
> >> C++. It would be a fair amount of work.
> >
> > The other big problem is that D's const is so much more restrictive than
> > C++'s that even if const ref accepted rvalues, a large portion of the
> > time,
> > it would be too restrictive to be useful.
>
> That's why if we allow binding rvalues to references, we'd allow it
> regardless of const. -- Andrei

Which then causes the problem that it becomes much less clear whether ref is
supposed to be modifying its argument or is just trying to avoid copying it
- though good documentation can mitigate that problem.

- Jonathan M Davis

Re: Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[Andrei Alexandrescu via Digitalmars-d]

On 10/19/2016 11:38 AM, Jonathan M Davis via Digitalmars-d wrote:

On Wednesday, October 19, 2016 07:55:19 Andrei Alexandrescu via Digitalmars-d
wrote:

This was C++'s big un' that led to many complications. If the overload
weren't ambiguous, a large part of rvalue references would have been
unneeded. (Universal references would still have been necessary for
perfect forwarding, but that's not the bulk.)

In order to avoid such issues, we steered clear off binding rvalues to
ref parameters in the D language. As I mentioned to Ethan, I do agree a
careful definition may be able to avoid the fallout that happened in
C++. It would be a fair amount of work.


The other big problem is that D's const is so much more restrictive than
C++'s that even if const ref accepted rvalues, a large portion of the time,
it would be too restrictive to be useful.


That's why if we allow binding rvalues to references, we'd allow it 
regardless of const. -- Andrei

Re: Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[Jonathan M Davis via Digitalmars-d]

On Wednesday, October 19, 2016 07:55:19 Andrei Alexandrescu via Digitalmars-d 
wrote:
> This was C++'s big un' that led to many complications. If the overload
> weren't ambiguous, a large part of rvalue references would have been
> unneeded. (Universal references would still have been necessary for
> perfect forwarding, but that's not the bulk.)
>
> In order to avoid such issues, we steered clear off binding rvalues to
> ref parameters in the D language. As I mentioned to Ethan, I do agree a
> careful definition may be able to avoid the fallout that happened in
> C++. It would be a fair amount of work.

The other big problem is that D's const is so much more restrictive than
C++'s that even if const ref accepted rvalues, a large portion of the time,
it would be too restrictive to be useful.

- Jonathan M Davis

Binding rvalues to ref [WAS: I close BIP27. I won't be pursuing BIPs anymore]

[Andrei Alexandrescu via Digitalmars-d]

On 10/19/16 1:47 AM, Manu via Digitalmars-d wrote:

On 19 October 2016 at 06:22, Andrei Alexandrescu via Digitalmars-d
> wrote:

On 10/18/2016 04:15 PM, Atila Neves wrote:


I think I get it; I'm just not sure given the comments that pop
up in
the forum. Isn't one of the main reasons distinguishing between
these two?

void fun(ref const Foo);
void fun(Foo);

If they can't be distinguished, you don't get move semantics
"for free".


That's right, thanks Atila. -- Andreu


This is obvious though, lval calls the first, rval calls the second.
Surely most programmers would intuitively expect that behaviour? Is
there some issue there?


You're missing context. The conversation went like this:

Jonathan: `Andrei said rvalue references were a mistake in C++'

Andrei: `No, I said binding rvalues to const& was a mistake of C++, 
which subsequently led to the necessity of rvalue references.'


Atila: `Indeed, if you cannot distinguish between void fun(ref const 
Foo) and void fun(Foo) then you're going to have a bad time.'


(NOTA BENE: at this point it became unclear whether we discuss C++ or 
hypothetical D code. Atila used D syntax in a discussion about C++'s issue.)


Me: `Affirmative.'

You: `But wait, it's intuitive: lvalues bind to the first, rvalues bind 
to the last.'


Well, I'd agree it's intuitive but C++ won't allow it. These overloads 
are ambiguous in C++ code:


==
struct S {};
void foo(S);
void foo(const S&); // no error so far (which doesn't help either)

int main() {
S a;
foo(a); // ERROR! Ambiguous call!
const S b;
foo(b); // ERROR! Ambiguous call!
foo(S()); // ERROR! Ambiguous call!
extern const S goo();
foo(goo()); // ERROR! Ambiguous call!
}
==

This was C++'s big un' that led to many complications. If the overload 
weren't ambiguous, a large part of rvalue references would have been 
unneeded. (Universal references would still have been necessary for 
perfect forwarding, but that's not the bulk.)


In order to avoid such issues, we steered clear off binding rvalues to 
ref parameters in the D language. As I mentioned to Ethan, I do agree a 
careful definition may be able to avoid the fallout that happened in 
C++. It would be a fair amount of work.



Andrei