Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 1 April 2018 at 11:55, Timon Gehr via Digitalmars-d
 wrote:
> On 01.04.2018 19:20, Andrei Alexandrescu wrote:
>>
>> On 3/28/18 7:50 AM, Timon Gehr wrote:
>>>
>>> "The proposal could be amended to accept mutable ref's depending on the
>>> value-judgement balancing these 2 use cases. Sticking with const requires no
>>> such value judgement to be made at this time, and it's much easier to relax
>>> the spec in the future with emergence of evidence to do so."
>>>
>>> Just get it right the first time. "const" is a serious API restriction,
>>> and it shouldn't be forced on anyone, even intermittently until they figure
>>> out that it is too restrictive (as well as viral).
>>
>>
>> A great way to move things forward here, Timon, is to write a pull request
>> against the DIP with motivating text and examples.
>
>
> I agree, but unfortunately I have many other things on my plate right now.
> Add to this that there are six or seven other DIPs that I really ought to
> finish/write/implement/rebase. I'll try to get back to this soon. Here, I
> was trying to make sure that popular misconceptions do not gain more
> traction.

I'm convinced. There are lots of reasons it should not only apply to const.
I only originally went that way because it was more conservative,
easier to relax, and because the stated reason why you can't _already_
do this thing is allegedly because people freak out at the idea that a
function might return data into a temporary. 'const' prevents that
from being a possibility, but unlike C++, there are significant useful
cases for not restricting to const.
The most exciting for me is pipeline programming (ie, UFCS chains),
which are a major winning feature of D. They'll be more convenient
among other things.

Re: rvalues -> ref (yup... again!)

[Timon Gehr via Digitalmars-d]

On 01.04.2018 19:20, Andrei Alexandrescu wrote:

On 3/28/18 7:50 AM, Timon Gehr wrote:
"The proposal could be amended to accept mutable ref's depending on 
the value-judgement balancing these 2 use cases. Sticking with const 
requires no such value judgement to be made at this time, and it's 
much easier to relax the spec in the future with emergence of evidence 
to do so."


Just get it right the first time. "const" is a serious API 
restriction, and it shouldn't be forced on anyone, even intermittently 
until they figure out that it is too restrictive (as well as viral).


A great way to move things forward here, Timon, is to write a pull 
request against the DIP with motivating text and examples.


I agree, but unfortunately I have many other things on my plate right 
now. Add to this that there are six or seven other DIPs that I really 
ought to finish/write/implement/rebase. I'll try to get back to this 
soon. Here, I was trying to make sure that popular misconceptions do not 
gain more traction.

Re: rvalues -> ref (yup... again!)

[Andrei Alexandrescu via Digitalmars-d]

On 3/28/18 7:50 AM, Timon Gehr wrote:
"The proposal could be amended to accept mutable ref's depending on the 
value-judgement balancing these 2 use cases. Sticking with const 
requires no such value judgement to be made at this time, and it's much 
easier to relax the spec in the future with emergence of evidence to do 
so."


Just get it right the first time. "const" is a serious API restriction, 
and it shouldn't be forced on anyone, even intermittently until they 
figure out that it is too restrictive (as well as viral).


A great way to move things forward here, Timon, is to write a pull 
request against the DIP with motivating text and examples.

Re: rvalues -> ref (yup... again!)

[Andrei Alexandrescu via Digitalmars-d]

On 3/31/18 9:28 AM, Timon Gehr wrote:

On 30.03.2018 11:06, Atila Neves wrote:


Right, and I was assuming (perhaps incorrectly) that this existing 
code was C++, hence me being on board with binding rvalues to const 
ref there.


That must not happen. D const and C++ const don't even mean the same 
thing, and now suddenly you will see people use extern(C++) just to get 
the binding of rvalues to ref. Add to this the confusing overload 
behavior if const signifies anything other than read-only.


This is a very important aspect that must not be overlooked.

Allow me to reiterate: the one way to get this going is convert the 
forum dialog into a team and a DIP.



Andrei

Re: rvalues -> ref (yup... again!)

[Timon Gehr via Digitalmars-d]

On 30.03.2018 11:06, Atila Neves wrote:


Right, and I was assuming (perhaps incorrectly) that this existing code 
was C++, hence me being on board with binding rvalues to const ref there.


That must not happen. D const and C++ const don't even mean the same 
thing, and now suddenly you will see people use extern(C++) just to get 
the binding of rvalues to ref. Add to this the confusing overload 
behavior if const signifies anything other than read-only.

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 30 March 2018 at 18:52, Walter Bright via Digitalmars-d
 wrote:
> On 3/30/2018 6:03 PM, Manu wrote:
>>
>> Sadly, I don't think I'll be able to make it to DConf this year...
>
>
> :-(
>
>> which is probably a reason for rejoice of literally everybody
>> attending! :P
>
>
> You'll be missed.

Thank your stars... I wouldn't be able to resist heckling everyone on
this the whole time! :P

Re: rvalues -> ref (yup... again!)

[Jonathan M Davis via Digitalmars-d]

On Friday, March 30, 2018 18:52:38 Walter Bright via Digitalmars-d wrote:
> On 3/30/2018 6:03 PM, Manu wrote:
> > Sadly, I don't think I'll be able to make it to DConf this year...
> :
> :-(
> :
> > which is probably a reason for rejoice of literally everybody
> > attending! :P
>
> You'll be missed.

My usual response to that is that it's better to be missed than to be hit.
;)

But yes, it will be a shame if it's unable to come.

- Jonathan M Davis

Re: rvalues -> ref (yup... again!)

[Jonathan M Davis via Digitalmars-d]

On Friday, March 30, 2018 18:03:49 Manu via Digitalmars-d wrote:
> On 30 March 2018 at 16:09, Jonathan M Davis via Digitalmars-d
> > And honestly, I don't really want to encourage the use of const. It's
> > fine if folks use it where it really works, and it's potentially even
> > quite valuable there, but it seems like too often folks try to add
> > const in places where it's just going to cause problems. It's
> > particularly bad when folks try to add const to generic code - e.g. we
> > recently had to revert a commit to std.random which worked with dynamic
> > arrays but not other ranges because of const. And there have been other
> > cases where folks have wanted to try to make stuff in Phobos
> > "const-correct", which would cause problems. So, I'm not a big fan of
> > the idea of doing anything that would make folks want to use const
> > more.
>
> It's interesting... I recognise the general backlash against const. I
> personally just make myself a `struct Mutable(T) { ... }` which I use
> when I want to violate const ;)

Just don't do anything where you cast away const and then mutate. That
violates the type system and results in undefined behavior. And depending on
what the compiler chooses to do based on const, you could get some rather
subtle and nasty bugs.

> But if it turns out that const is useless, then we really need to
> reconsider the basic design of const >_<
> Like, what's the point? You're advocating active discouragement of
> const... why is there a feature which it's accepted is a loaded-gun?
> Handing it to people creates a high probability they'll shoot
> themselves in the feet.

const makes some sense when you want to write code that accepts both mutable
and immutable arguments, and there are times where it works perfectly fine
to use const. I see no problem with using const in such situations. The
problem is that there are a _lot_ cases where const simply can't be used in
D, because you need to mutate _something_ involved, even if it's not
something that's part of the "logical" state of the object - e.g. in D, you
can't put a mutex in a type and then have it protect anything in a const
member function, because locking the mutex would require mutating it, and D
provides no backdoors like C++'s mutable. What's const is actually const.

Also, const tends to interact very badly with generic code. Whatever API
you're duck typing has to include const as part of it, or you can't use
const - e.g. ranges can't be const, because they have to be mutated to be
iterated, and because the range API does not require that properties like
front or empty be const, no generic code can assume that even those work
with const (and for many ranges, they _can_'t be const - especially when one
range wraps another). In general, it can be quite difficult to even do
something like have a tail-const range over a container. Sometimes, const
can be made to work in more complex situations, but often it can't, and when
it can't, it's often a royal pain.

For non-generic code that can clearly treat an object as fully const without
needing any backdoors, const is just fine, and it may even help prevent
bugs. But for a _lot_ of code - especially idiomatic D code that does a lot
with templates and ranges - const simply doesn't work.

So, const can be used on some level, but anyone who tries to be
"const-correct" in D like many of us usually try to do in C++ is in for a
world of frustration.

If you haven't, I'd suggest that you read this article I recently wrote on
the topic:

http://jmdavisprog.com/articles/why-const-sucks.html

And here's the newsgroup thread discussing it:

https://forum.dlang.org/thread/mailman.581.1520247475.3374.digitalmars-d-annou...@puremagic.com

> > Now, if you can convince Walter and Andrei to allow const ref to accept
> > rvalues, then fine. I think that that's definitely worse than an
> > attribute specifically for that, given how limiting const is, but it
> > wouldn't screw up normal ref in the process, which is what I'm most
> > worried about here. So, I don't think that going with const would be
> > the best solution to the problem, but it's far better than making ref
> > in general accept rvalues.
>
> Useful examples using 'return ref' have been presented, which I find
> quite compelling too. That's an interesting case of non-const ref.
> Safe to say I'm not convincing anyone of anything in any way other
> than DIP form.

Yes, at this point, convincing Walter or Andrei will require writing a DIP.
A well-written proposal that clearly doesn't have the downsides that they're
so against may have a chance with them, but without something that formally
and clearly provides the arguments, I don't think that they're even going to
pay much attention at this point. The topic has been debated to death
previously, and without a DIP, it really doesn't matter what Walter and
Andrei think, since at this point, they wouldn't introduce a language change
like that without a DIP. So, they'll just do 

Re: rvalues -> ref (yup... again!)

[Walter Bright via Digitalmars-d]

On 3/30/2018 6:03 PM, Manu wrote:

Sadly, I don't think I'll be able to make it to DConf this year...


:-(


which is probably a reason for rejoice of literally everybody
attending! :P


You'll be missed.


It would be nice to workshop it in person though.

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 30 March 2018 at 16:09, Jonathan M Davis via Digitalmars-d
 wrote:
> On Friday, March 30, 2018 14:47:06 Manu via Digitalmars-d wrote:
>> On 30 March 2018 at 12:24, Jonathan M Davis via Digitalmars-d
>> > I want to be able to look at a function signature, see ref without other
>> > qualifiers, and be reasonably certain that the function is supposed to
>> > be
>> > mutating that argument, whereas if ref by itself accepted rvalues, then
>> > we lose that. If an attribute were used to make it allow rvalues, then
>> > we wouldn't.
>>
>> I don't know any reason why someone wouldn't attribute the argument
>> 'const' if it doesn't intend to write to it. Likewise 'return ref' if
>> it's going to be modified and returned.
>> I'm not sure your concern is actually a thing...?
>
> const would do it, but given how restrictive const is in D, I don't see how
> it would be very reasonable to restrict passing rvalues to const.

I'm actually coming around to the idea of not restricting it to const...
But what I'm trying to say is, the number of cases where your
principle will feel violated should be as infrequent as the cases
where const is insufficient for 'reasons' ;)
As far as I know, in usages that I've ever encountered, I can't
imagine a case where you would feel violated :)


> And honestly, I don't really want to encourage the use of const. It's fine if
> folks use it where it really works, and it's potentially even quite valuable
> there, but it seems like too often folks try to add const in places where
> it's just going to cause problems. It's particularly bad when folks try to
> add const to generic code - e.g. we recently had to revert a commit to
> std.random which worked with dynamic arrays but not other ranges because of
> const. And there have been other cases where folks have wanted to try to
> make stuff in Phobos "const-correct", which would cause problems. So, I'm
> not a big fan of the idea of doing anything that would make folks want to
> use const more.

It's interesting... I recognise the general backlash against const. I
personally just make myself a `struct Mutable(T) { ... }` which I use
when I want to violate const ;)
But if it turns out that const is useless, then we really need to
reconsider the basic design of const >_<
Like, what's the point? You're advocating active discouragement of
const... why is there a feature which it's accepted is a loaded-gun?
Handing it to people creates a high probability they'll shoot
themselves in the feet.


> Now, if you can convince Walter and Andrei to allow const ref to accept
> rvalues, then fine. I think that that's definitely worse than an attribute
> specifically for that, given how limiting const is, but it wouldn't screw up
> normal ref in the process, which is what I'm most worried about here. So, I
> don't think that going with const would be the best solution to the problem,
> but it's far better than making ref in general accept rvalues.

Useful examples using 'return ref' have been presented, which I find
quite compelling too. That's an interesting case of non-const ref.
Safe to say I'm not convincing anyone of anything in any way other
than DIP form.


But I'd like to understand your concern better. You say it's about
scanning an API and understanding some details from it based on seeing
'ref' written there... how does a function accepting an rvalue
interact with your visibility of the API?
Like, your criticism is with respect to understanding the API at a
glance... I don't understand how this proposal interferes with that in
any way?

Sadly, I don't think I'll be able to make it to DConf this year...
which is probably a reason for rejoice of literally everybody
attending! :P
It would be nice to workshop it in person though.

Re: rvalues -> ref (yup... again!)

[Jonathan M Davis via Digitalmars-d]

On Friday, March 30, 2018 14:47:06 Manu via Digitalmars-d wrote:
> On 30 March 2018 at 12:24, Jonathan M Davis via Digitalmars-d
> > I want to be able to look at a function signature, see ref without other
> > qualifiers, and be reasonably certain that the function is supposed to
> > be
> > mutating that argument, whereas if ref by itself accepted rvalues, then
> > we lose that. If an attribute were used to make it allow rvalues, then
> > we wouldn't.
>
> I don't know any reason why someone wouldn't attribute the argument
> 'const' if it doesn't intend to write to it. Likewise 'return ref' if
> it's going to be modified and returned.
> I'm not sure your concern is actually a thing...?

const would do it, but given how restrictive const is in D, I don't see how
it would be very reasonable to restrict passing rvalues to const. And
honestly, I don't really want to encourage the use of const. It's fine if
folks use it where it really works, and it's potentially even quite valuable
there, but it seems like too often folks try to add const in places where
it's just going to cause problems. It's particularly bad when folks try to
add const to generic code - e.g. we recently had to revert a commit to
std.random which worked with dynamic arrays but not other ranges because of
const. And there have been other cases where folks have wanted to try to
make stuff in Phobos "const-correct", which would cause problems. So, I'm
not a big fan of the idea of doing anything that would make folks want to
use const more.

Now, if you can convince Walter and Andrei to allow const ref to accept
rvalues, then fine. I think that that's definitely worse than an attribute
specifically for that, given how limiting const is, but it wouldn't screw up
normal ref in the process, which is what I'm most worried about here. So, I
don't think that going with const would be the best solution to the problem,
but it's far better than making ref in general accept rvalues.

- Jonathan M Davis

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 30 March 2018 at 12:24, Jonathan M Davis via Digitalmars-d
 wrote:
> On Friday, March 30, 2018 11:01:23 Manu via Digitalmars-d wrote:
>> On 29 March 2018 at 21:08, Jonathan M Davis via Digitalmars-d
>>
>>  wrote:
>> > On Thursday, March 29, 2018 23:28:54 Nick Sabalausky  via Digitalmars-d
>> >
>> > wrote:
>> >> On 03/23/2018 09:06 PM, Jonathan M Davis wrote:
>> >> > My biggest concern in all of this is that I don't want to see ref
>> >> > start
>> >> > accepting rvalues as has been occasionally discussed. It needs to be
>> >> > clear when a function is accept an argument by ref because it's going
>> >> > to mutate the object and when it's accepting by ref because it wants
>> >> > to
>> >> > avoid a copy.
>> >>
>> >> That ship sailed ages ago: It's already unclear. If we want to fix
>> >> that,
>> >> we can fix it, but blocking rvalue ref does nothing for that cause.
>> >
>> > Really? And how often does ref get used just to avoid copying? You can
>> > almost always look at a function, see that it accepts ref, and know that
>> > it's supposed to mutate the argument. Functions that want to avoid
>> > copying lvalues usually use auto ref, not ref, whereas if ref accepted
>> > rvalues, a number of folks would start using it all over the place to
>> > avoid copying. Right now, folks rarely use ref that way, because it
>> > becomes too annoying to call the function with an rvalue. So, while it
>> > might not be the case 100% of the time right now that ref is used with
>> > the purpose of mutating the argument, it almost always is. As such, you
>> > can pretty reliably look at a function signature and expect that if one
>> > of its parameters is ref, it's going to be mutating that argument. The
>> > function that accepts an argument by ref with no intention of mutating
>> > it is very much the exception, and I really don't want to see that
>> > change.
>>
>> Interesting. Just understand that you're trading that feeling for a
>> suite of edge cases though. Accepting asymmetric calling rules is a
>> pretty big cost to pay for that 'nice thought'.
>> That idea also dismisses the existence of the set of cases where ref
>> is genuinely useful/correct. Your sentiment effectively puts those use
>> cases into the position of 2nd-class citizens.
>> I understand your sentiment, but I think the cost is not balanced, or
>> even particularly fair with respect to users in those niche groups :/
>
> I'm not arguing against having a way to indicate that a parameter accepts
> both rvalues and lvalues and that the rvalues get copied to an invisible
> variable so that they can be passed as an lvalue. I'm arguing against simply
> making ref have that behavior. Assuming that the details of how that worked
> internally didn't involve problematic stuff like the rvalue reference stuff
> that Andrei and Walter are so against, having an attribute such as @rvalue
> to attach to a ref parameter to allow it to accept rvalues would be fine
> with me. I just don't want ref by itself to lose its current semantics,
> because that would dilute its meaning and increase its amiguity.

https://imgflip.com/i/27gjv9


> I want to be able to look at a function signature, see ref without other
> qualifiers, and be reasonably certain that the function is supposed to be
> mutating that argument, whereas if ref by itself accepted rvalues, then we
> lose that. If an attribute were used to make it allow rvalues, then we
> wouldn't.

I don't know any reason why someone wouldn't attribute the argument
'const' if it doesn't intend to write to it. Likewise 'return ref' if
it's going to be modified and returned.
I'm not sure your concern is actually a thing...?

Re: rvalues -> ref (yup... again!)

[Jonathan M Davis via Digitalmars-d]

On Friday, March 30, 2018 11:01:23 Manu via Digitalmars-d wrote:
> On 29 March 2018 at 21:08, Jonathan M Davis via Digitalmars-d
>
>  wrote:
> > On Thursday, March 29, 2018 23:28:54 Nick Sabalausky  via Digitalmars-d
> >
> > wrote:
> >> On 03/23/2018 09:06 PM, Jonathan M Davis wrote:
> >> > My biggest concern in all of this is that I don't want to see ref
> >> > start
> >> > accepting rvalues as has been occasionally discussed. It needs to be
> >> > clear when a function is accept an argument by ref because it's going
> >> > to mutate the object and when it's accepting by ref because it wants
> >> > to
> >> > avoid a copy.
> >>
> >> That ship sailed ages ago: It's already unclear. If we want to fix
> >> that,
> >> we can fix it, but blocking rvalue ref does nothing for that cause.
> >
> > Really? And how often does ref get used just to avoid copying? You can
> > almost always look at a function, see that it accepts ref, and know that
> > it's supposed to mutate the argument. Functions that want to avoid
> > copying lvalues usually use auto ref, not ref, whereas if ref accepted
> > rvalues, a number of folks would start using it all over the place to
> > avoid copying. Right now, folks rarely use ref that way, because it
> > becomes too annoying to call the function with an rvalue. So, while it
> > might not be the case 100% of the time right now that ref is used with
> > the purpose of mutating the argument, it almost always is. As such, you
> > can pretty reliably look at a function signature and expect that if one
> > of its parameters is ref, it's going to be mutating that argument. The
> > function that accepts an argument by ref with no intention of mutating
> > it is very much the exception, and I really don't want to see that
> > change.
>
> Interesting. Just understand that you're trading that feeling for a
> suite of edge cases though. Accepting asymmetric calling rules is a
> pretty big cost to pay for that 'nice thought'.
> That idea also dismisses the existence of the set of cases where ref
> is genuinely useful/correct. Your sentiment effectively puts those use
> cases into the position of 2nd-class citizens.
> I understand your sentiment, but I think the cost is not balanced, or
> even particularly fair with respect to users in those niche groups :/

I'm not arguing against having a way to indicate that a parameter accepts
both rvalues and lvalues and that the rvalues get copied to an invisible
variable so that they can be passed as an lvalue. I'm arguing against simply
making ref have that behavior. Assuming that the details of how that worked
internally didn't involve problematic stuff like the rvalue reference stuff
that Andrei and Walter are so against, having an attribute such as @rvalue
to attach to a ref parameter to allow it to accept rvalues would be fine
with me. I just don't want ref by itself to lose its current semantics,
because that would dilute its meaning and increase its amiguity.

I want to be able to look at a function signature, see ref without other
qualifiers, and be reasonably certain that the function is supposed to be
mutating that argument, whereas if ref by itself accepted rvalues, then we
lose that. If an attribute were used to make it allow rvalues, then we
wouldn't.

- Jonathan M Davis

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 30 March 2018 at 02:47, Kagamin via Digitalmars-d
 wrote:
>
> The only criticism against rvalue references I saw was when the
> reference outlives the temporary.

That's the only *technical* criticism I've ever heard too.
Fortunately, D now has mechanisms for preventing that; D's @safety
story with respect to reference lifetime seems to be solid now.
'return ref', and enforcing ref pointers don't escape the callee
cleared to the road to allow accepting temporaries safety.

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 30 March 2018 at 02:06, Atila Neves via Digitalmars-d
 wrote:
> On Tuesday, 27 March 2018 at 18:14:18 UTC, Manu wrote:
>>
>> On 27 March 2018 at 00:14, Atila Neves via Digitalmars-d
>>  wrote:
>>>
>>> On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote:


 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
  wrote:
>
>
> On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
>>
>>
>
>>> That's _if_ T is big and _if_ it even gets copied,
>>
>>
>> You've just described the exact anatomy of a ref function!
>> You wouldn't write a function to receive T by ref UNLESS T was both
>> big, and the function probably won't inline (therefore definitely
>> copy), and that condition will be triggered by any of the list of
>> reasons I've said a bunch of times (extern, dll, lib, virtual, etc).
>> People don't just love writing ref (well, some people might), but they
>> use it deliberately, typically in user-facing boundary API's for these
>> exact reasons.
>
>
> I know. I was arguing that those cases are uncommon and the API pain is
> therefore not too big of an issue. I'm pretty sure that you feel it more
> because of what you write in D.

Totally.
But I think it's uncommon at least in-part because of this
inconvenience. It's at least a little bit circular. Even you're saying
above "yeah, just do by-val, it's *probably* insignificant"... you
only say that because you've allowed this inconvenience to change your
behaviour. And I think that's to be expected.
Other reasons that it's not common:
extern(C++) is not common.
OOP (virtuals) in D are fairly uncommon. We don't OOP much in D.
Closed-source (binary lib) distribution is uncommon... possibly
unheard of? (yet!)
DLL's are still problematic onvarious fronts, and I don't think they
enjoy anywhere near as much use as they deserve.

Most of these aren't desirable... we'd like to see more commercial
(closed source?) users, DLL's should be more popular than they are...
and more extern(C++) means more people using a key development
investment in D.
So yeah, I agree it's relatively uncommon if you don't interact with
one of the niches where it tends to be common! :P


>> Right. I'm talking about deliberate use of ref... Or *existing*
>> (deliberate) use of ref, as is the case in almost all my my cases. The
>> code already exists.
>
>
> Right, and I was assuming (perhaps incorrectly) that this existing code was
> C++, hence me being on board with binding rvalues to const ref there.

Right. But I'm not a fan is just rearranging the edge cases by
limiting it to a different set of cases and not applying it generally.
This also interacts with meta constructions, and while there are ANY
cases where calling doesn't just work as usual and it needs special
case handling, meta will still need static-if's to handle those cases.
We're just changing the criteria for the 'if'.
By allowing fully symmetric function calling rules, only then does the
noisy case-handling logic disappear from meta.


>> Only if you ARE moving, and not copying. D must deep copy too if you
>> actually copy.
>> Your example assumes C++ doesn't have a move constructor. D has
>> implicit move semantics, so you can only make an equivalent comparison
>> where C++ also defines the move constructor so the move case doesn't
>> pollute the ref comparison.
>
> I wasn't assuming the lack of a move constructor. What I was saying is that
> passing by value in C++ will usually mean a copy, whereas in D it usually
> means a move.

Right. But we're not talking about move's, we're talking about
NOT-move's (ie, preventing copies by passing by ref) ;)
Despite the appearance that we might be talking about rvalues, we're
actually talking about passing lvalues by ref... that's WHY you write
a function to accept its args by ref; to prevent deep copies when
passing lvalues.
The issue is, it's super-common to call functions with rvalues, and
the edge cases created by using ref are annoying and asymmetric, hence
such functions should receive rvalues too.


 In C++'s case, it's not that references were deficient at being
 references that C++ needed rval-references, it's that references were
 deficient at being move-able.
>>>
>>> There were deficient at being moveable because temporaries can bind to
>>> const T&.
>>
>> ... what? That's just not true at all.
>> If temporaries couldn't bind to C++ ref, then you *definitely*
>> wouldn't be able to move it, because you can guarantee that someone
>> else owns the reference.
>
> Precisely. That's how D works.

Sorry, I'm lost now. I don't understand your initial point. You
created a relationship between ref's accepting rvalues, and the reason
that C++ introduced rvalue-references.
No such relationship exists... and I was trying to show that your
reasoning was inverted.


>> rvalue references were introduced in C++ to capture the calls with
>> rvalues into 

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 29 March 2018 at 21:08, Jonathan M Davis via Digitalmars-d
 wrote:
> On Thursday, March 29, 2018 23:28:54 Nick Sabalausky  via Digitalmars-d
> wrote:
>> On 03/23/2018 09:06 PM, Jonathan M Davis wrote:
>> > My biggest concern in all of this is that I don't want to see ref start
>> > accepting rvalues as has been occasionally discussed. It needs to be
>> > clear when a function is accept an argument by ref because it's going
>> > to mutate the object and when it's accepting by ref because it wants to
>> > avoid a copy.
>> That ship sailed ages ago: It's already unclear. If we want to fix that,
>> we can fix it, but blocking rvalue ref does nothing for that cause.
>
> Really? And how often does ref get used just to avoid copying? You can
> almost always look at a function, see that it accepts ref, and know that
> it's supposed to mutate the argument. Functions that want to avoid copying
> lvalues usually use auto ref, not ref, whereas if ref accepted rvalues, a
> number of folks would start using it all over the place to avoid copying.
> Right now, folks rarely use ref that way, because it becomes too annoying to
> call the function with an rvalue. So, while it might not be the case 100% of
> the time right now that ref is used with the purpose of mutating the
> argument, it almost always is. As such, you can pretty reliably look at a
> function signature and expect that if one of its parameters is ref, it's
> going to be mutating that argument. The function that accepts an argument by
> ref with no intention of mutating it is very much the exception, and I
> really don't want to see that change.

Interesting. Just understand that you're trading that feeling for a
suite of edge cases though. Accepting asymmetric calling rules is a
pretty big cost to pay for that 'nice thought'.
That idea also dismisses the existence of the set of cases where ref
is genuinely useful/correct. Your sentiment effectively puts those use
cases into the position of 2nd-class citizens.
I understand your sentiment, but I think the cost is not balanced, or
even particularly fair with respect to users in those niche groups :/

Re: rvalues -> ref (yup... again!)

[Kagamin via Digitalmars-d]

On Wednesday, 28 March 2018 at 16:15:53 UTC, Manu wrote:
I discussed that in that document. I'm happy to remove const, 
but it requires a value judgement on the meaning of non-const 
in this case. It becomes controversial without const, but I'm 
personally happy to remove it if you can make The argument in 
favour. Can you give me some ideas where it would be useful?


doesn't make sense because the output would be immediately 
discarded; such a function call given an rvalue as argument 
likely represents an accidental mistake on the users part, and 
we can catch that invalid code.


Most obvious:

void Close(ref HANDLE h)
{
  CloseHandle(h);
  h=INVALID_HANDLE;
}

If you want to take input argument, then of course mark it as 
input, but if not then not. Why the callee would need to care if 
the argument is rvalue or not? The only criticism against rvalue 
references I saw was when the reference outlives the temporary.

Re: rvalues -> ref (yup... again!)

[Atila Neves via Digitalmars-d]

On Tuesday, 27 March 2018 at 18:14:18 UTC, Manu wrote:
On 27 March 2018 at 00:14, Atila Neves via Digitalmars-d 
 wrote:

On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote:


On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d 
 wrote:


On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:





That's _if_ T is big and _if_ it even gets copied,


You've just described the exact anatomy of a ref function!
You wouldn't write a function to receive T by ref UNLESS T was 
both
big, and the function probably won't inline (therefore 
definitely
copy), and that condition will be triggered by any of the list 
of
reasons I've said a bunch of times (extern, dll, lib, virtual, 
etc).
People don't just love writing ref (well, some people might), 
but they
use it deliberately, typically in user-facing boundary API's 
for these

exact reasons.


I know. I was arguing that those cases are uncommon and the API 
pain is therefore not too big of an issue. I'm pretty sure that 
you feel it more because of what you write in D.



Right. I'm talking about deliberate use of ref... Or *existing*
(deliberate) use of ref, as is the case in almost all my my 
cases. The

code already exists.


Right, and I was assuming (perhaps incorrectly) that this 
existing code was C++, hence me being on board with binding 
rvalues to const ref there.


Only if you ARE moving, and not copying. D must deep copy too 
if you

actually copy.
Your example assumes C++ doesn't have a move constructor. D has
implicit move semantics, so you can only make an equivalent 
comparison
where C++ also defines the move constructor so the move case 
doesn't

pollute the ref comparison.


I wasn't assuming the lack of a move constructor. What I was 
saying is that passing by value in C++ will usually mean a copy, 
whereas in D it usually means a move.


Also, irrespective of whether move semantics are performed 
(eliding
potential deep copying, as in your example), the binary memcpy 
still
has to be performed when handling values by-val, unless RVO 
(we're not
talking about return values), or inlining is able to eliminate 
it.


Good point about memcpy.

In C++'s case, it's not that references were deficient at 
being references that C++ needed rval-references, it's that 
references were deficient at being move-able.



There were deficient at being moveable because temporaries can 
bind to const T&.


... what? That's just not true at all.
If temporaries couldn't bind to C++ ref, then you *definitely*
wouldn't be able to move it, because you can guarantee that 
someone

else owns the reference.


Precisely. That's how D works.


rvalue references were introduced in C++ to capture the calls 
with
rvalues into a separate function call, exactly the same way as 
the
by-value overload will catch the rvalues in D (and perform an 
implicit

move).


Yes.

It was impossible for C++ to implement D's implicit move 
semantics

when receiving by-value for reasons that have nothing to do with
references; C++ allows interior pointers which breaks implicit 
moving,
C++ can overload default constructor which also breaks implicit 
moving

(no implicit way to reset the state of the prior owner).


I hadn't thought about the implications of interior pointers. 
Very good point.



References have no interaction with move semantics.


Even given interior pointers, I disagree.

But again, we're not talking about move semantics here, we're 
just talking about references ;)


See comment above ;)


I'd love to know what that would look like.


That's exactly what I've been saying. For like, 9 years..
It looks like this:
https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
 (contribution appreciated)


I was unaware of this (or I forgot). After reading it I'm not 
sure of what corner cases might arise, but if I'm getting it 
right I think it could work.



And as far as I
can tell, it basically only affects me, because I do so much 
work

against established C++ code! >_<


That's entirely possible. I can use my fingers to count the 
number of times I've written `extern(C++)`.


Atila

Re: rvalues -> ref (yup... again!)

[Jonathan M Davis via Digitalmars-d]

On Thursday, March 29, 2018 23:28:54 Nick Sabalausky  via Digitalmars-d 
wrote:
> On 03/23/2018 09:06 PM, Jonathan M Davis wrote:
> > My biggest concern in all of this is that I don't want to see ref start
> > accepting rvalues as has been occasionally discussed. It needs to be
> > clear when a function is accept an argument by ref because it's going
> > to mutate the object and when it's accepting by ref because it wants to
> > avoid a copy.
> That ship sailed ages ago: It's already unclear. If we want to fix that,
> we can fix it, but blocking rvalue ref does nothing for that cause.

Really? And how often does ref get used just to avoid copying? You can
almost always look at a function, see that it accepts ref, and know that
it's supposed to mutate the argument. Functions that want to avoid copying
lvalues usually use auto ref, not ref, whereas if ref accepted rvalues, a
number of folks would start using it all over the place to avoid copying.
Right now, folks rarely use ref that way, because it becomes too annoying to
call the function with an rvalue. So, while it might not be the case 100% of
the time right now that ref is used with the purpose of mutating the
argument, it almost always is. As such, you can pretty reliably look at a
function signature and expect that if one of its parameters is ref, it's
going to be mutating that argument. The function that accepts an argument by
ref with no intention of mutating it is very much the exception, and I
really don't want to see that change.

- Jonathan M Davis

Re: rvalues -> ref (yup... again!)

[Nick Sabalausky (Abscissa) via Digitalmars-d]

On 03/24/2018 12:03 AM, Manu wrote:

Why aim for "it often works", when we want "it always works"? Forcing const
upon people who want to pass rvalues by reference is just not good enough.
It is bad language design.


I think you need to re-read the whole thread, and understand what
we're even talking about.
Nobody wants to pass rvalues by mutable-ref... 


I do. The ban serves no useful purpose (at least not any purpose that D 
doesn't *already* fail at - like knowing at the callsite whether a param 
is intended to be mutated). And it would permit the "avoid a copy" 
benefits in a completely orthogonal way - *without* relying on the param 
fitting D's transitive const requirements.

Re: rvalues -> ref (yup... again!)

[Nick Sabalausky (Abscissa) via Digitalmars-d]

On 03/23/2018 09:06 PM, Jonathan M Davis wrote:


My biggest concern in all of this is that I don't want to see ref start
accepting rvalues as has been occasionally discussed. It needs to be clear
when a function is accept an argument by ref because it's going to mutate
the object and when it's accepting by ref because it wants to avoid a copy.



That ship sailed ages ago: It's already unclear. If we want to fix that, 
we can fix it, but blocking rvalue ref does nothing for that cause.

Re: rvalues -> ref (yup... again!)

[Rubn via Digitalmars-d]

On Thursday, 29 March 2018 at 20:22:47 UTC, Dgame wrote:

On Thursday, 29 March 2018 at 20:05:48 UTC, Rubn wrote:

On Thursday, 29 March 2018 at 19:11:30 UTC, Dgame wrote:
Just to be sure it does not got los: You know that you can 
avoid the temp/copy if you add one method to your struct, yes?



import std.stdio;

struct Big {
string name;
float[1000] values;

this(string name) {
this.name = name;
}

@disable
this(this);

ref auto byRef() inout {
return this;
}
}

void foo(ref const Big b) {
writeln(b.name);
}

void main() {
Big b = Big("#1");
foo(b);
foo(Big("#2").byRef);
}


That works like a charm and avoids any need for rvalue 
references. Just add it as mixin template in Phobos or 
something like that.


Doesn't work with built-in types like float.


Why would you want to use a float as a rvalue reference?


In templates to avoid template bloat with auto ref.

Just adds bloat for operators like opBinary if you want that 
to be ref.


foo((a.byRef + b.byRef * c.byRef).byRef)

// vs

foo(a + b * c);

It's kind of funny all this talk about allowing temporaries to 
bind to refs being messy, yet you can already bind a temporary 
to a ref in a messy way using that.


Yeah, it is a bit messy. It is not perfect, but is does avoid 
any temp var!

Let's look at a Vector2f example with the following opBinary:


auto opBinary(string op)(ref const Vector2f v) {
return Vector2f(mixin("this.x" ~ op ~ "v.x"), 
mixin("this.y" ~ op ~ "v.y"));

}

void foo(ref const Vector2f v) {
writeln(v.x, ':', v.y);
}

foo((Vector2f(2, 4).byRef + Vector2f(4, 6).byRef).byRef);


Since opBinary needs to be a template, you can combine my 
solution with auto ref:



auto opBinary(string op)(auto ref const Vector2f v) {
return Vector2f(mixin("this.x" ~ op ~ "v.x"), 
mixin("this.y" ~ op ~ "v.y"));

}

void foo(ref const Vector2f v) {
writeln(v.x, ':', v.y);
}

foo((Vector2f(2, 4) + Vector2f(4, 6)).byRef);


That is cleaner. :)


Just adding more template bloat and it is still messy, it's not 
like we are trying to find a work around to a problem. A solution 
already exists, the cleaner syntax you'd get with rvalue 
references (along with better compatibility with C++ and other 
benefits) can't be beat.

Re: rvalues -> ref (yup... again!)

[Dgame via Digitalmars-d]

On Thursday, 29 March 2018 at 20:05:48 UTC, Rubn wrote:

On Thursday, 29 March 2018 at 19:11:30 UTC, Dgame wrote:
Just to be sure it does not got los: You know that you can 
avoid the temp/copy if you add one method to your struct, yes?



import std.stdio;

struct Big {
string name;
float[1000] values;

this(string name) {
this.name = name;
}

@disable
this(this);

ref auto byRef() inout {
return this;
}
}

void foo(ref const Big b) {
writeln(b.name);
}

void main() {
Big b = Big("#1");
foo(b);
foo(Big("#2").byRef);
}


That works like a charm and avoids any need for rvalue 
references. Just add it as mixin template in Phobos or 
something like that.


Doesn't work with built-in types like float.


Why would you want to use a float as a rvalue reference?

Just adds bloat for operators like opBinary if you want that to 
be ref.


foo((a.byRef + b.byRef * c.byRef).byRef)

// vs

foo(a + b * c);

It's kind of funny all this talk about allowing temporaries to 
bind to refs being messy, yet you can already bind a temporary 
to a ref in a messy way using that.


Yeah, it is a bit messy. It is not perfect, but is does avoid any 
temp var!

Let's look at a Vector2f example with the following opBinary:


auto opBinary(string op)(ref const Vector2f v) {
return Vector2f(mixin("this.x" ~ op ~ "v.x"), mixin("this.y" 
~ op ~ "v.y"));

}

void foo(ref const Vector2f v) {
writeln(v.x, ':', v.y);
}

foo((Vector2f(2, 4).byRef + Vector2f(4, 6).byRef).byRef);


Since opBinary needs to be a template, you can combine my 
solution with auto ref:



auto opBinary(string op)(auto ref const Vector2f v) {
return Vector2f(mixin("this.x" ~ op ~ "v.x"), mixin("this.y" 
~ op ~ "v.y"));

}

void foo(ref const Vector2f v) {
writeln(v.x, ':', v.y);
}

foo((Vector2f(2, 4) + Vector2f(4, 6)).byRef);


That is cleaner. :)

Re: rvalues -> ref (yup... again!)

[Rubn via Digitalmars-d]

On Thursday, 29 March 2018 at 19:11:30 UTC, Dgame wrote:
Just to be sure it does not got los: You know that you can 
avoid the temp/copy if you add one method to your struct, yes?



import std.stdio;

struct Big {
string name;
float[1000] values;

this(string name) {
this.name = name;
}

@disable
this(this);

ref auto byRef() inout {
return this;
}
}

void foo(ref const Big b) {
writeln(b.name);
}

void main() {
Big b = Big("#1");
foo(b);
foo(Big("#2").byRef);
}


That works like a charm and avoids any need for rvalue 
references. Just add it as mixin template in Phobos or 
something like that.


Doesn't work with built-in types like float.

Just adds bloat for operators like opBinary if you want that to 
be ref.


foo((a.byRef + b.byRef * c.byRef).byRef)

// vs

foo(a + b * c);

It's kind of funny all this talk about allowing temporaries to 
bind to refs being messy, yet you can already bind a temporary to 
a ref in a messy way using that.

Re: rvalues -> ref (yup... again!)

[Dgame via Digitalmars-d]

Just to be sure it does not got los: You know that you can avoid 
the temp/copy if you add one method to your struct, yes?



import std.stdio;

struct Big {
string name;
float[1000] values;

this(string name) {
this.name = name;
}

@disable
this(this);

ref auto byRef() inout {
return this;
}
}

void foo(ref const Big b) {
writeln(b.name);
}

void main() {
Big b = Big("#1");
foo(b);
foo(Big("#2").byRef);
}


That works like a charm and avoids any need for rvalue 
references. Just add it as mixin template in Phobos or something 
like that.

Re: rvalues -> ref (yup... again!)

[Nick Treleaven via Digitalmars-d]

On Saturday, 24 March 2018 at 17:34:09 UTC, Manu wrote:
You mutate a temporary that times out at the end of the 
statement...

your statement is never assigned to anything, and has no effect.


That is solved by having the ref function return its argument (so 
it can be chained):


struct S;
ref S modify(return ref S s);

S(data).modify.writeln;

This rvalue pattern would be disallowed if argument `s` was 
const. Why not make `modify` just return a copy then, maybe the 
optimizer could remove the copy? So that you can also use it with 
lvalues:


S s;
modify(s);

Re: rvalues -> ref (yup... again!)

[Timon Gehr via Digitalmars-d]

On 28.03.2018 20:20, Manu wrote:

On 28 March 2018 at 05:22, Timon Gehr via Digitalmars-d
 wrote:

On 27.03.2018 20:14, Manu wrote:


That's exactly what I've been saying. For like, 9 years..
It looks like this:

https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
   (contribution appreciated)



"Temporary destruction
Destruction of any temporaries occurs naturally at the end of the scope, as
usual."

This is actually unusual.
 ...

...
So, what's wrong?
...


In my example, the temporary is destroyed after/at the end of the 
function call, not at the end of the scope.


Re-reading the DIP, I think you meant the right thing, but the wording 
is a bit confusing. Maybe just clarify that "the scope" is the one your 
rewrite introduces implicitly, or explicitly state that the lifetime 
ends at the end of the function call.





"Overload resolution
...



Note that lvalues prefer the ref overload because the ref overload is more
specialized. The new rule is the only instance where a less specialized
overload is preferred.


I've never heard any discussion involving the term 'specialised', or
seen any definition where overloading prefers a "more specialised'
version... is that a thing?
Given:
   void f(int);
   void f(const(int));
   f(10);

That calls the 'int' one, but it could call either one...


The overload resolution rules in D have four different matching levels:

- exact match
- match with type qualifier conversion
- match with general implicit conversion
- no match

The matching level for one overload is the minimal matching level for 
any argument. In your example f(int) matches exactly, but f(const(int)) 
matches with type qualifier conversion, therefore f(int) is chosen as it 
is the unique function that matches best.


Only if two overloads match with the same best level is specialization 
used. An overload A is more specialized than another overload B if we 
can call B with all arguments with which we can call A. As it is 
possible to call a by-value function with an lvalue or an rvalue, but 
ref cannot be called with an rvalue, ref is more specialized.



that's definitely not choosing a 'more specialised' match.
...


Implicit conversions are ignored when checking for specialization so, 
yes, here both functions are equally specialized. However, f(int*) is 
more specialized than f(const(int)*):


---
import std.stdio;

void f(int* a){
writeln("A");
}
void f(const(int)* b){
writeln("B");
}

void main(){
f(new immutable(int)); // guess what this prints. :)
}
---

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 28 March 2018 at 05:38, Timon Gehr via Digitalmars-d
 wrote:
> On 27.03.2018 22:25, Rubn wrote:
>>
>>
>> D already has move semantics, an easy solution to this is to just use
>> another keyword. It doesn't have to bind to const ref to get what is
>> desired:
>>
>> // what was suggested in the original DIP, since scope is being used for
>> something else now
>> void foo(@temp ref value)
>> {
>> }
>>
>> Now you don't have this problem. You only get this behavior when you
>> basically say you don't care whether it is a temporary or not.
>
> Another benefit of this solution is that the overload resolution rules are
> obvious. foo(@temp ref T value) is less specialized than both foo(T value)
> and foo(ref T value).
>
> @Manu: Consider this.

This defeats the entire point to me. I want symmetrical calling code
in all cases... the current edges are a massive pain in the arse.
In the event of yet-another-attribute, then we just shift the set of
edge cases onto that attribute instead, and it makes no practical
difference in the end.

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 28 March 2018 at 05:22, Timon Gehr via Digitalmars-d
 wrote:
> On 27.03.2018 20:14, Manu wrote:
>>
>> That's exactly what I've been saying. For like, 9 years..
>> It looks like this:
>>
>> https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
>>   (contribution appreciated)
>
>
> "Temporary destruction
> Destruction of any temporaries occurs naturally at the end of the scope, as
> usual."
>
> This is actually unusual.
>
> ---
> import std.stdio;
> struct S{
> ~this(){
> writeln("destroyed!");
> }
> }
>
> void foo(S s){}
>
> void main(){
> foo(S());
> writeln("end the scope");
> }
> ---
>
> prints:
>
> destroyed!
> end the scope

Right, exactly...
So, what's wrong?


> "Overload resolution
> ...
> This follows existing language rules. No change is proposed here."
>
> Yes, you _are_ proposing a change. Right now rvalues "prefer" the by-value
> overload because the ref overload _does not match at all_. Now you make it
> match both, so you are adding additional disambiguation rules. You need to
> be more explicit about those.

Oh right... yeah okay, I'll tweak the language.


> Note that lvalues prefer the ref overload because the ref overload is more
> specialized. The new rule is the only instance where a less specialized
> overload is preferred.

I've never heard any discussion involving the term 'specialised', or
seen any definition where overloading prefers a "more specialised'
version... is that a thing?
Given:
  void f(int);
  void f(const(int));
  f(10);

That calls the 'int' one, but it could call either one... that's
definitely not choosing a 'more specialised' match.


> You also need to specify the interactions with matching levels
> (https://dlang.org/spec/function.html#function-overloading):
>
> E.g., your DIP is compatible with the following behavior:
>
> ---
> import std.stdio;
>
> struct S{}
> void fun(S){ writeln("A"); }
> void fun(ref const(S)){ writeln("B"); }
>
> void main(){
> fun(S()); // calls A
> S s;
> fun(s); // calls A
>
> const(S) t;
> fun(t); // calls B
> fun(const(S)()); // calls B
> }
> ---
>
> The first example will cause friction when people try to add an explicit
> rvalue overload alongside a previous catch-it-all overload, the second
> example shows a breaking language change.
>
> You cannot "fix" the first example without introducing breaking language
> changes. The above code compiles and runs in current D.
>
> This just smells bad. Remove the "const" requirement.

This is very compelling reason to remove the const.

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 28 Mar. 2018 4:35 am, "Timon Gehr via Digitalmars-d" <
digitalmars-d@puremagic.com> wrote:

On 27.03.2018 20:14, Manu wrote:

> That's exactly what I've been saying. For like, 9 years..
> It looks like this:
> https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx
> -rval_to_ref.md
>   (contribution appreciated)
>
> As far as I can tell, it's completely benign, it just eliminates the
> annoying edge cases when interacting with functions that take
> arguments by ref. There's no spill-over affect anywhere that I'm aware
> of, and if you can find a single wart, I definitely want to know about
> it.
>

???


I've asked so many times for a technical destruction, nobody will
> present any opposition that is anything other than a rejection *in
> principle*. This is a holy war, not a technical one.
>

That's extremely unfair. It is just a bad idea to overload D const for this
purpose. Remove the "const" requirement and I'm on board.


I discussed that in that document. I'm happy to remove const, but it
requires a value judgement on the meaning of non-const in this case. It
becomes controversial without const, but I'm personally happy to remove it
if you can make The argument in favour. Can you give me some ideas where it
would be useful?

Re: rvalues -> ref (yup... again!)

[Timon Gehr via Digitalmars-d]

On 27.03.2018 22:25, Rubn wrote:


D already has move semantics, an easy solution to this is to just use 
another keyword. It doesn't have to bind to const ref to get what is 
desired:


// what was suggested in the original DIP, since scope is being used for 
something else now

void foo(@temp ref value)
{
}

Now you don't have this problem. You only get this behavior when you 
basically say you don't care whether it is a temporary or not.
Another benefit of this solution is that the overload resolution rules 
are obvious. foo(@temp ref T value) is less specialized than both foo(T 
value) and foo(ref T value).


@Manu: Consider this.

Re: rvalues -> ref (yup... again!)

[Timon Gehr via Digitalmars-d]

On 27.03.2018 20:14, Manu wrote:

That's exactly what I've been saying. For like, 9 years..
It looks like this:
https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
  (contribution appreciated)


"Temporary destruction
Destruction of any temporaries occurs naturally at the end of the scope, 
as usual."


This is actually unusual.

---
import std.stdio;
struct S{
~this(){
writeln("destroyed!");
}
}

void foo(S s){}

void main(){
foo(S());
writeln("end the scope");
}
---

prints:

destroyed!
end the scope



"Overload resolution
...
This follows existing language rules. No change is proposed here."

Yes, you _are_ proposing a change. Right now rvalues "prefer" the 
by-value overload because the ref overload _does not match at all_. Now 
you make it match both, so you are adding additional disambiguation 
rules. You need to be more explicit about those.


Note that lvalues prefer the ref overload because the ref overload is 
more specialized. The new rule is the only instance where a less 
specialized overload is preferred.



You also need to specify the interactions with matching levels 
(https://dlang.org/spec/function.html#function-overloading):


E.g., your DIP is compatible with the following behavior:

---
import std.stdio;

struct S{}
void fun(S){ writeln("A"); }
void fun(ref const(S)){ writeln("B"); }

void main(){
fun(S()); // calls A
S s;
fun(s); // calls A

const(S) t;
fun(t); // calls B
fun(const(S)()); // calls B
}
---


The first example will cause friction when people try to add an explicit 
rvalue overload alongside a previous catch-it-all overload, the second 
example shows a breaking language change.


You cannot "fix" the first example without introducing breaking language 
changes. The above code compiles and runs in current D.


This just smells bad. Remove the "const" requirement.

Re: rvalues -> ref (yup... again!)

[Timon Gehr via Digitalmars-d]

On 28.03.2018 13:34, Timon Gehr wrote:

On 27.03.2018 20:14, Manu wrote:

That's exactly what I've been saying. For like, 9 years..
It looks like this:
https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md 


  (contribution appreciated)

As far as I can tell, it's completely benign, it just eliminates the
annoying edge cases when interacting with functions that take
arguments by ref. There's no spill-over affect anywhere that I'm aware
of, and if you can find a single wart, I definitely want to know about
it.


???
 >> I've asked so many times for a technical destruction, nobody will

present any opposition that is anything other than a rejection *in
principle*. This is a holy war, not a technical one.


That's extremely unfair. It is just a bad idea to overload D const for 
this purpose. Remove the "const" requirement and I'm on board.


"The proposal could be amended to accept mutable ref's depending on the 
value-judgement balancing these 2 use cases. Sticking with const 
requires no such value judgement to be made at this time, and it's much 
easier to relax the spec in the future with emergence of evidence to do so."


Just get it right the first time. "const" is a serious API restriction, 
and it shouldn't be forced on anyone, even intermittently until they 
figure out that it is too restrictive (as well as viral).

Re: rvalues -> ref (yup... again!)

[Timon Gehr via Digitalmars-d]

On 27.03.2018 20:14, Manu wrote:

That's exactly what I've been saying. For like, 9 years..
It looks like this:
https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
  (contribution appreciated)

As far as I can tell, it's completely benign, it just eliminates the
annoying edge cases when interacting with functions that take
arguments by ref. There's no spill-over affect anywhere that I'm aware
of, and if you can find a single wart, I definitely want to know about
it.


???


I've asked so many times for a technical destruction, nobody will
present any opposition that is anything other than a rejection *in
principle*. This is a holy war, not a technical one.


That's extremely unfair. It is just a bad idea to overload D const for 
this purpose. Remove the "const" requirement and I'm on board.

Re: rvalues -> ref (yup... again!)

[Kagamin via Digitalmars-d]

On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
By contrast, people will NOT forgive the fact that they have to 
change:


func(f(x), f(y), f(z));

to:

T temp = f(x);
T temp2 = f(y);
T temp3 = f(z);
func(temp, temp2, temp3);

That's just hideous and in-defensible.

A better story would be:

func(f(x), f(y), f(z));
=>
func(x.f, y.f, z.f);


Another workaround:

auto r(T)(T a)
{
struct R { T val; }
return R(a);
}
void f(in ref int p);
int main()
{
f(1.r.val);
return 0;
}

Re: rvalues -> ref (yup... again!)

[Rubn via Digitalmars-d]

On Wednesday, 28 March 2018 at 00:56:29 UTC, kinke wrote:

On Tuesday, 27 March 2018 at 23:59:09 UTC, Rubn wrote:
Just adding a few writeln it isn't able to remove the function 
entirely anymore and can't optimize it out.


Well writeln() here involves number -> string formatting, GC, 
I/O, template bloat... There are indeed superfluous memcpy's in 
your foo() there (although the forward and bar calls are still 
inlined), which after a quick glance seem to be LLVM optimizer 
shortcomings, the IR emitted by LDC looks fine.
For an abitrary external function, it's all fine as it should 
be, boiling down to a single memcpy in foo() and a direct 
memset in main(): https://run.dlang.io/is/O1aeLK


Well somethings wrong if writeln causes optimization to not 
occur, if that is the case then it'd be best to just use printf() 
instead. Anyways using small examples to show optimization is 
usually not what's going to happen in actual code. Functions are 
rarely that simple, and if adding a single writeln() to a call is 
enough to eliminate that optimization, I can only imagine what 
other little things do as well.

Re: rvalues -> ref (yup... again!)

[kinke via Digitalmars-d]

On Tuesday, 27 March 2018 at 23:59:09 UTC, Rubn wrote:
Just adding a few writeln it isn't able to remove the function 
entirely anymore and can't optimize it out.


Well writeln() here involves number -> string formatting, GC, 
I/O, template bloat... There are indeed superfluous memcpy's in 
your foo() there (although the forward and bar calls are still 
inlined), which after a quick glance seem to be LLVM optimizer 
shortcomings, the IR emitted by LDC looks fine.
For an abitrary external function, it's all fine as it should be, 
boiling down to a single memcpy in foo() and a direct memset in 
main(): https://run.dlang.io/is/O1aeLK

Re: rvalues -> ref (yup... again!)

[kinke via Digitalmars-d]

On Tuesday, 27 March 2018 at 23:35:44 UTC, kinke wrote:

On Tuesday, 27 March 2018 at 21:52:25 UTC, Rubn wrote:
It happens with LDC too, not sure how it would be able to know 
to do any kind of optimization like that unless it was able to 
inline every single function called into one function and be 
able to do optimize it from there. I don't imagine that'll be 
likely though.


It does it in your code sample with `-O`, there's no call to 
bar and the foo() by-value arg is memcpy'd to the global.


For reference: https://run.dlang.io/is/2vDEXP
Note that main() boils down to a `memset(, 10, 1024); return 
0;`:


_Dmain:
.cfi_startproc
pushq   %rax
.Lcfi0:
.cfi_def_cfa_offset 16
data16
leaqonlineapp.Foo onlineapp.gfoo@TLSGD(%rip), %rdi
data16
data16
rex64
callq   __tls_get_addr@PLT
movl$10, %esi
movl$1024, %edx
movq%rax, %rdi
callq   memset@PLT
xorl%eax, %eax
popq%rcx
retq

Re: rvalues -> ref (yup... again!)

[Rubn via Digitalmars-d]

On Tuesday, 27 March 2018 at 23:35:44 UTC, kinke wrote:

On Tuesday, 27 March 2018 at 21:52:25 UTC, Rubn wrote:
It happens with LDC too, not sure how it would be able to know 
to do any kind of optimization like that unless it was able to 
inline every single function called into one function and be 
able to do optimize it from there. I don't imagine that'll be 
likely though.


It does it in your code sample with `-O`, there's no call to 
bar and the foo() by-value arg is memcpy'd to the global.


If you compile everything with LTO, your code and all 3rd-party 
libs as well as druntime/Phobos, LLVM is able to optimize the 
whole program as if it were inside a single gigantic 'object' 
file in LLVM bitcode IR, and is thus indeed theoretically able 
to inline *all* functions.


A bit off topic now but anyways:

Well that example I posted didn't do anything, so it would 
optimize it out quite easily. The entire function was excluded 
essentially. Just adding a few writeln it isn't able to remove 
the function entirely anymore and can't optimize it out. Idk if 
you want to try some different options but flto didn't do 
anything for it.


https://godbolt.org/g/bLdpnm

import std.stdio : writeln;

struct Foo {
ubyte[1024] data;

this(int a)
{
data[0] = cast(ubyte)a;
}
}

void foo(T)(auto ref T t) {
import std.functional: forward;
writeln(gfoo.data[0]);
bar(forward!t);
writeln(gfoo.data[0]);
}

__gshared Foo gfoo;

void bar(T)(auto ref T t) {
import std.algorithm.mutation : move;
writeln(gfoo.data[0]);
move(t, gfoo);
}

void main() {
foo(Foo(10));
}

Re: rvalues -> ref (yup... again!)

[kinke via Digitalmars-d]

On Tuesday, 27 March 2018 at 21:52:25 UTC, Rubn wrote:
It happens with LDC too, not sure how it would be able to know 
to do any kind of optimization like that unless it was able to 
inline every single function called into one function and be 
able to do optimize it from there. I don't imagine that'll be 
likely though.


It does it in your code sample with `-O`, there's no call to bar 
and the foo() by-value arg is memcpy'd to the global.


If you compile everything with LTO, your code and all 3rd-party 
libs as well as druntime/Phobos, LLVM is able to optimize the 
whole program as if it were inside a single gigantic 'object' 
file in LLVM bitcode IR, and is thus indeed theoretically able to 
inline *all* functions.

Re: rvalues -> ref (yup... again!)

[H. S. Teoh via Digitalmars-d]

On Tue, Mar 27, 2018 at 09:52:25PM +, Rubn via Digitalmars-d wrote:
> On Tuesday, 27 March 2018 at 20:38:35 UTC, H. S. Teoh wrote:
> > On Tue, Mar 27, 2018 at 08:25:36PM +, Rubn via Digitalmars-d wrote:
> > [...]
> > > _D7example__T3fooTSQr3FooZQnFNbNiNfQrZv:
> > >   push rbp
> > >   mov rbp, rsp
> > >   sub rsp, 3104
> > >   lea rax, [rbp + 16]
> > >   lea rdi, [rbp - 2048]
> > >   lea rcx, [rbp - 1024]
> > >   mov edx, 1024
> > >   mov rsi, rcx
> > >   mov qword ptr [rbp - 2056], rdi
> > >   mov rdi, rsi
> > >   mov rsi, rax
> > >   mov qword ptr [rbp - 2064], rcx
> > >   call memcpy@PLT<- hidden copy
> > [...]
> > 
> > Is this generated by dmd, or gdc/ldc?
> > 
> > Generally, when it comes to performance issues, I don't even bother
> > looking at dmd-generated code anymore.  If the extra copying is
> > still happening with gdc -O2 / ldc -O, then you have a point.
> > Otherwise, it doesn't really say very much.
> > 
> > 
> > T
> 
> It happens with LDC too, not sure how it would be able to know to do
> any kind of optimization like that unless it was able to inline every
> single function called into one function and be able to do optimize it
> from there.  I don't imagine that'll be likely though.

You'll be surprised.  Don't underestimate the power of modern
optimizers.  I've seen LDC do inlining that's so aggressive, that it
essentially evaluated an entire series of function calls at compile-time
(likely on the IR) and generated a single instruction to load the answer
into the return register at runtime. :-D  Of course, it still generated
the individual functions, but those are never actually called at
runtime.

(On one occasion, this produced odd-looking "benchmark" results where the
ldc executable computed the answer in exactly 0ms, whereas everyone else
took a lot longer than that. :-D  (Well, it was probably a few nanosecs
while the CPU decoded and ran the instruction, but I don't think any
benchmark could measure that!))

For your code example, you might want to look at the code generated for
callers of the function, since when compiling individual functions in
isolation, LDC is obligated to follow the ABI, which could include
redundant copying. But if inlining was possible, it could generate very
different code.


T

-- 
Dogs have owners ... cats have staff. -- Krista Casada

Re: rvalues -> ref (yup... again!)

[Rubn via Digitalmars-d]

On Tuesday, 27 March 2018 at 20:38:35 UTC, H. S. Teoh wrote:
On Tue, Mar 27, 2018 at 08:25:36PM +, Rubn via 
Digitalmars-d wrote: [...]

_D7example__T3fooTSQr3FooZQnFNbNiNfQrZv:
  push rbp
  mov rbp, rsp
  sub rsp, 3104
  lea rax, [rbp + 16]
  lea rdi, [rbp - 2048]
  lea rcx, [rbp - 1024]
  mov edx, 1024
  mov rsi, rcx
  mov qword ptr [rbp - 2056], rdi
  mov rdi, rsi
  mov rsi, rax
  mov qword ptr [rbp - 2064], rcx
  call memcpy@PLT<- hidden copy

[...]

Is this generated by dmd, or gdc/ldc?

Generally, when it comes to performance issues, I don't even 
bother looking at dmd-generated code anymore.  If the extra 
copying is still happening with gdc -O2 / ldc -O, then you have 
a point. Otherwise, it doesn't really say very much.



T


It happens with LDC too, not sure how it would be able to know to 
do any kind of optimization like that unless it was able to 
inline every single function called into one function and be able 
to do optimize it from there. I don't imagine that'll be likely 
though.

Re: rvalues -> ref (yup... again!)

[H. S. Teoh via Digitalmars-d]

On Tue, Mar 27, 2018 at 08:25:36PM +, Rubn via Digitalmars-d wrote:
[...]
> _D7example__T3fooTSQr3FooZQnFNbNiNfQrZv:
>   push rbp
>   mov rbp, rsp
>   sub rsp, 3104
>   lea rax, [rbp + 16]
>   lea rdi, [rbp - 2048]
>   lea rcx, [rbp - 1024]
>   mov edx, 1024
>   mov rsi, rcx
>   mov qword ptr [rbp - 2056], rdi
>   mov rdi, rsi
>   mov rsi, rax
>   mov qword ptr [rbp - 2064], rcx
>   call memcpy@PLT<- hidden copy
[...]

Is this generated by dmd, or gdc/ldc?

Generally, when it comes to performance issues, I don't even bother
looking at dmd-generated code anymore.  If the extra copying is still
happening with gdc -O2 / ldc -O, then you have a point. Otherwise, it
doesn't really say very much.


T

-- 
People tell me that I'm skeptical, but I don't believe them.

Re: rvalues -> ref (yup... again!)

[Rubn via Digitalmars-d]

On Tuesday, 27 March 2018 at 15:50:37 UTC, Atila Neves wrote:

It's fine for references to
just be references in D. We're not struggling to make 
references
move-able in D, that's not a thing, we already have move 
semantics.

Any extension of this conversation about references into C++
rvalue-references (T&&) and or move-semantics are red-herrings.
There's no such problem in D that needs to be resolved, and the
existing solution is excellent.


If I'm reading you correctly (which I might not), you seem to 
be saying that there's a way forward in which:


1) D's move semantics aren't affected
2) No rvalue references are introduced
3) Temporaries can bind to ref const(T)

I'd love to know what that would look like.

Atila



Well currently if you only have this implemented:

void foo(const ref Type);

Type temp = Type(10);
foo(temp);

Where the hell are you going to do your move semantics? You can't 
do it anyways currently, it's completely meaningless cause you 
can't.


void foo(Type);
void foo(const ref Type);

foo(Type(10));

Now we have move semantics with an additional definition.


With the proposed change, nothing about that would change. A 
temporary is only passed to a const ref as a "last resort". If it 
can do a move instead, it will do the move. The only change that 
is desired is to make the first sample code above have nicer 
syntax. That's it, like in the first example you don't care about 
it being a temporary or not.

Re: rvalues -> ref (yup... again!)

[Peter Campbell via Digitalmars-d]

On Tuesday, 27 March 2018 at 18:14:18 UTC, Manu wrote:

That's exactly what I've been saying. For like, 9 years..
It looks like this:
https://github.com/TurkeyMan/DIPs/blob/ref_args/DIPs/DIP1xxx-rval_to_ref.md
 (contribution appreciated)


I've followed this thread since it was made as this has been one 
of the very few disappointments of the language for me. I only 
tend to write game code and use D for hobbyist projects whilst 
using C++ full-time as a junior at a small games company. Even if 
I take the attitude that I should use D as it is intended, 
instead of trying to write C++ by using D, it always felt 
unnecessarily obstructive to require me to make a temporary 
variable to avoid copying something simple like a vector or a 
matrix. It feels very restrictive when trying to express 
mathematical calculations in a concise manner.


Thanks for writing that DIP, you have covered everything I would 
love to see in great detail with good examples! I honestly 
couldn't think of anything more that could be added.

Re: rvalues -> ref (yup... again!)

[Rubn via Digitalmars-d]

On Tuesday, 27 March 2018 at 07:33:12 UTC, Atila Neves wrote:

On Tuesday, 27 March 2018 at 00:30:24 UTC, Rubn wrote:

On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:

C++ T&& (Rvalue reference) -> D T


Not really, in C++ it is an actual reference and you get to 
choose which function actually does the move. In D it just 
does the copy when passed to the function.


It doesn't copy.


It copies the memory, so it does 2 memcpy's in the sense where as 
C++ only calls its move constructor once.



So you can't do this in D.

void bar(T&& t)
{
// actually move contents of T
}

void foo(T&& t)
{
bar(std::forward(t)); // Can't do this in D without 
making another actual copy cause it isn't a reference

}



You can most definitely do this in D:

void bar(T)(auto ref T t) {
// T is a ref for lvalues, by value for rvalues
}

void foo(T)(auto ref T t) {
import std.functional: forward;
bar(forward!t);
}


More to the point:

import std.stdio;

struct Foo {
ubyte[] data;

this(int n) {
writeln("ctor n = ", n);
data.length = n;
}

this(this) {
writeln("postBlit n = ", data.length);
data = data.dup;
}
}

void foo(T)(auto ref T t) {
import std.functional: forward;
bar(forward!t);
}

void bar(T)(auto ref T t) {
writeln("bar: ", t.data[0]);
}

void main() {
bar(Foo(10));
auto f = Foo(5);
f.data[0] = 42;
bar(f);
}


The output is:

ctor n = 10
bar: 0
ctor n = 5
bar: 42

Notice the lack of "postBlit" in the output. No copies were 
made. In D, by value *does not* mean copy. And given that, 
contrary to C++, the compiler doesn't write the postBlit 
constructor for you, you'd only ever get copies if you 
implemented it yourself!


Well for starters your code is wrong. You are calling bar() 
instead of foo().


D has hidden implementation details, from your perspective it 
looks like it isn't doing any copying from the high-level 
viewpoint, but from the low-level viewpoint your object has been 
copied (moved memory) multiple times.


struct Foo {
ubyte[1024] data;
}

void foo(T)(auto ref T t) {
import std.functional: forward;
bar(forward!t);
}

Foo gfoo;

void bar(T)(auto ref T t) {
import std.algorithm.mutation : move;
move(t, gfoo);
}

void main() {
foo(Foo(10));
}

_D7example__T3fooTSQr3FooZQnFNbNiNfQrZv:
  push rbp
  mov rbp, rsp
  sub rsp, 3104
  lea rax, [rbp + 16]
  lea rdi, [rbp - 2048]
  lea rcx, [rbp - 1024]
  mov edx, 1024
  mov rsi, rcx
  mov qword ptr [rbp - 2056], rdi
  mov rdi, rsi
  mov rsi, rax
  mov qword ptr [rbp - 2064], rcx
  call memcpy@PLT<- hidden copy


Then the other copy is in move() to gfoo. That hidden copy will 
happen for every additional function call you try to pass Foo 
through.



What's a concrete example that you would be required to know 
whether a const& is a temporary or not.


To know whether or not you can move instead of copy. If it's a 
temporary, you can move. If it's not, you have to copy. Since 
temporaries bind to const T& in C++, you might have a 
temporary, or you might have an lvalue. Since you don't know, 
you have to copy. To support move semantics, C++ got T&&, which 
lvalues can't bind to. So if you have a T&&, you know it's 
about to go away and a move is possible.


In D, if it's ref then it can't be a temporary. If it's a value 
then it can, and it gets moved.


D already has move semantics, an easy solution to this is to just 
use another keyword. It doesn't have to bind to const ref to get 
what is desired:


// what was suggested in the original DIP, since scope is being 
used for something else now

void foo(@temp ref value)
{
}

Now you don't have this problem. You only get this behavior when 
you basically say you don't care whether it is a temporary or not.


So what's your problem with it now ?

I've come across a few pains of such. It make be easier to use 
but it comes at a performance hit. In part binaries become 
huge because of how "init" is implemented.


struct StaticArray(T, size_t capacity)
{
size_t length;
T[capacity] values;
}

Copying the above structure copies unnecessary data for any 
move/copy operation. Eg when length = 0, it'll still copy 
everything. This includes initialization.


This is that rare type for which moving is the same as copying. 
In that case (assuming it gets copied, see my reply to Manu), 
pass by ref. You won't be able to pass in temporaries, but I 
think that's a small price to pay for not having rvalue 
references.


In this case specifically, I don't know why you wouldn't just 
slice it when passing to functions.


Atila


This wasn't an example for rvalue references. This was an example 
illustrating the negative results of the current "pain-free" 
system. I have a 100 mb binary, 90 mb of that come from a single 
structure. I mean sure you don't really have to worry about 
implementing move constructors and such but it is far from being 
pain free, 

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 27 March 2018 at 00:14, Atila Neves via Digitalmars-d
 wrote:
> On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote:
>>
>> On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
>>  wrote:
>>>
>>> On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:


 Forked from the x^^y thread...
 
>>>
>>>
>>>
>>> C++ T&& (forwarding reference) -> D auto ref T
>>> C++ T&& (Rvalue reference) -> D T
>>> C++ const T& -> D T
>>
>>
>> Yeah, no... T may be big. Copying a large thing sucks. Memory copying
>> is the slowest thing computers can do.
>
>
> That's _if_ T is big and _if_ it even gets copied,

You've just described the exact anatomy of a ref function!
You wouldn't write a function to receive T by ref UNLESS T was both
big, and the function probably won't inline (therefore definitely
copy), and that condition will be triggered by any of the list of
reasons I've said a bunch of times (extern, dll, lib, virtual, etc).
People don't just love writing ref (well, some people might), but they
use it deliberately, typically in user-facing boundary API's for these
exact reasons.


> the combination of which
> I think happens very rarely. When that happens I think that the temporary
> isn't a big deal. This code:
>
> struct Foo { int[1024] data; }
> int byValue(Foo f) { return f.data[42]; }
>
> Generates this assembly (ldc2 -O3, clang does the same for C++):
>
>  <_D3foo7byValueFSQo3FooZi>:
>0:   8b 84 24 b0 00 00 00moveax,DWORD PTR [rsp+0xb0]
>7:   c3  ret
>
> And I wrote a type for which a move and a copy are the same on purpose: in
> "real life" it's more likely that the memory will be dynamically allocated,
> probably held in a slice, and moved instead.
>
> Are there cases in which there will be an expensive copy? Yes, then pass by
> ref/pointer. But measure first, and prefer to pass by value by default.

Right. I'm talking about deliberate use of ref... Or *existing*
(deliberate) use of ref, as is the case in almost all my my cases. The
code already exists.
As you assess, use of ref in D is fairly rare. I've been looking for
cases where other people are inconvenienced by this... hard to find.
I suspect there are reasons for this. One of them is that this
inconvenience suppresses it; ie, you will choose not to use a ref even
when you might prefer to. Others include the fact that extern(C++)
isn't super popular, DLL's aren't popular, closed-source distributed
code is non popular, OOP is not popular, etc.


> It's different in C++ - stick a std::vector or std::string in your struct
> and passing by value (usually) copies the dynamically allocated memory. In D
> it moves.

Only if you ARE moving, and not copying. D must deep copy too if you
actually copy.
Your example assumes C++ doesn't have a move constructor. D has
implicit move semantics, so you can only make an equivalent comparison
where C++ also defines the move constructor so the move case doesn't
pollute the ref comparison.
Also, irrespective of whether move semantics are performed (eliding
potential deep copying, as in your example), the binary memcpy still
has to be performed when handling values by-val, unless RVO (we're not
talking about return values), or inlining is able to eliminate it.
Also, we're not talking about move semantics!


>> As an API author, exactly as in C++, you will make a judgement on a
>> case-by-case basis on this matter. It may be by-value, it may be by
>> const-ref. It depends on a bunch of things, and they are points for
>> consideration by the API author, not the user.
>
>
> You can still do that in D. There well may be a reason to pass by const ref.
> I'm just saying that there aren't that many, and in that in those rare cases
> a temporary is fine. Especially if the alternative are rvalue references.

We're not talking about rvalue references, we're talking about normal
references >_<


>> He's trying to say that C++ introduced rvalue references because normal
>> references weren't able to allow for move semantics to exist. It's a
>> red-herring. D already has move semantics, they work well, and they're not
>> on trial here.
>>
>> In C++'s case, it's not that references were deficient at being
>> references that C++ needed rval-references, it's that references were
>> deficient at being move-able.
>
>
> There were deficient at being moveable because temporaries can bind to const
> T&.

... what? That's just not true at all.
If temporaries couldn't bind to C++ ref, then you *definitely*
wouldn't be able to move it, because you can guarantee that someone
else owns the reference. You can't move references, under any
circumstances, in either language... and that's actually the whole
point of references.

rvalue references were introduced in C++ to capture the calls with
rvalues into a separate function call, exactly the same way as the
by-value overload will catch the rvalues in D (and perform an implicit

Re: rvalues -> ref (yup... again!)

[Atila Neves via Digitalmars-d]

On Tuesday, 27 March 2018 at 02:41:12 UTC, Manu wrote:

He's trying to say that C++ introduced rvalue references 
because normal references weren't able to allow for move 
semantics to exist. It's a red-herring. D already has move 
semantics, they work well, and they're not on trial here.


In C++'s case, it's not that references were deficient at being
references that C++ needed rval-references, it's that 
references were

deficient at being move-able.


There were deficient at being moveable because temporaries can 
bind to const T&.



That is not a problem that exists in D.


Because temporaries can't bind to ref const(T).


It's fine for references to
just be references in D. We're not struggling to make references
move-able in D, that's not a thing, we already have move 
semantics.

Any extension of this conversation about references into C++
rvalue-references (T&&) and or move-semantics are red-herrings.
There's no such problem in D that needs to be resolved, and the
existing solution is excellent.


If I'm reading you correctly (which I might not), you seem to be 
saying that there's a way forward in which:


1) D's move semantics aren't affected
2) No rvalue references are introduced
3) Temporaries can bind to ref const(T)

I'd love to know what that would look like.

Atila

Re: rvalues -> ref (yup... again!)

[Atila Neves via Digitalmars-d]

On Tuesday, 27 March 2018 at 00:30:24 UTC, Rubn wrote:

On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:

C++ T&& (Rvalue reference) -> D T


Not really, in C++ it is an actual reference and you get to 
choose which function actually does the move. In D it just does 
the copy when passed to the function.


It doesn't copy.


So you can't do this in D.

void bar(T&& t)
{
// actually move contents of T
}

void foo(T&& t)
{
bar(std::forward(t)); // Can't do this in D without 
making another actual copy cause it isn't a reference

}



You can most definitely do this in D:

void bar(T)(auto ref T t) {
// T is a ref for lvalues, by value for rvalues
}

void foo(T)(auto ref T t) {
import std.functional: forward;
bar(forward!t);
}


More to the point:

import std.stdio;

struct Foo {
ubyte[] data;

this(int n) {
writeln("ctor n = ", n);
data.length = n;
}

this(this) {
writeln("postBlit n = ", data.length);
data = data.dup;
}
}

void foo(T)(auto ref T t) {
import std.functional: forward;
bar(forward!t);
}

void bar(T)(auto ref T t) {
writeln("bar: ", t.data[0]);
}

void main() {
bar(Foo(10));
auto f = Foo(5);
f.data[0] = 42;
bar(f);
}


The output is:

ctor n = 10
bar: 0
ctor n = 5
bar: 42

Notice the lack of "postBlit" in the output. No copies were made. 
In D, by value *does not* mean copy. And given that, contrary to 
C++, the compiler doesn't write the postBlit constructor for you, 
you'd only ever get copies if you implemented it yourself!


What's a concrete example that you would be required to know 
whether a const& is a temporary or not.


To know whether or not you can move instead of copy. If it's a 
temporary, you can move. If it's not, you have to copy. Since 
temporaries bind to const T& in C++, you might have a temporary, 
or you might have an lvalue. Since you don't know, you have to 
copy. To support move semantics, C++ got T&&, which lvalues can't 
bind to. So if you have a T&&, you know it's about to go away and 
a move is possible.


In D, if it's ref then it can't be a temporary. If it's a value 
then it can, and it gets moved.


I've come across a few pains of such. It make be easier to use 
but it comes at a performance hit. In part binaries become huge 
because of how "init" is implemented.


struct StaticArray(T, size_t capacity)
{
size_t length;
T[capacity] values;
}

Copying the above structure copies unnecessary data for any 
move/copy operation. Eg when length = 0, it'll still copy 
everything. This includes initialization.


This is that rare type for which moving is the same as copying. 
In that case (assuming it gets copied, see my reply to Manu), 
pass by ref. You won't be able to pass in temporaries, but I 
think that's a small price to pay for not having rvalue 
references.


In this case specifically, I don't know why you wouldn't just 
slice it when passing to functions.


Atila

Re: rvalues -> ref (yup... again!)

[Atila Neves via Digitalmars-d]

On Monday, 26 March 2018 at 19:24:13 UTC, Manu wrote:
On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d 
 wrote:

On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:


Forked from the x^^y thread...




C++ T&& (forwarding reference) -> D auto ref T
C++ T&& (Rvalue reference) -> D T
C++ const T& -> D T


Yeah, no... T may be big. Copying a large thing sucks. Memory 
copying

is the slowest thing computers can do.


That's _if_ T is big and _if_ it even gets copied, the 
combination of which I think happens very rarely. When that 
happens I think that the temporary isn't a big deal. This code:


struct Foo { int[1024] data; }
int byValue(Foo f) { return f.data[42]; }

Generates this assembly (ldc2 -O3, clang does the same for C++):

 <_D3foo7byValueFSQo3FooZi>:
   0:   8b 84 24 b0 00 00 00moveax,DWORD PTR [rsp+0xb0]
   7:   c3  ret

And I wrote a type for which a move and a copy are the same on 
purpose: in "real life" it's more likely that the memory will be 
dynamically allocated, probably held in a slice, and moved 
instead.


Are there cases in which there will be an expensive copy? Yes, 
then pass by ref/pointer. But measure first, and prefer to pass 
by value by default.


It's different in C++ - stick a std::vector or std::string in 
your struct and passing by value (usually) copies the dynamically 
allocated memory. In D it moves.


As an API author, exactly as in C++, you will make a judgement 
on a
case-by-case basis on this matter. It may be by-value, it may 
be by
const-ref. It depends on a bunch of things, and they are points 
for

consideration by the API author, not the user.


You can still do that in D. There well may be a reason to pass by 
const ref. I'm just saying that there aren't that many, and in 
that in those rare cases a temporary is fine. Especially if the 
alternative are rvalue references.


Atila

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 26 March 2018 at 17:30, Rubn via Digitalmars-d
 wrote:
> On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:
>>
>> C++ T&& (Rvalue reference) -> D T
>
>
> Not really, in C++ it is an actual reference and you get to choose which
> function actually does the move. In D it just does the copy when passed to
> the function. So you can't do this in D.
>
> void bar(T&& t)
> {
> // actually move contents of T
> }
>
> void foo(T&& t)
> {
> bar(std::forward(t)); // Can't do this in D without making another
> actual copy cause it isn't a reference
> }

You can still get there with D, if you assume the forwarding function
is simple enough to inline (and the code is public).
I was cautiously concerned about this for a long time, but I've never
made a noise about it. I've digested and resolved that's okay though
:)
There are very few cases where you will fail to achieve equivalent
efficiency, and I think D's amazingly simplified move semantics more
than compensate for any conceivable loss.


>> If replacing const T& with T chafes, I understand. I used to feel that way
>> too. It's _possible_ that would incur a penalty in copying/moving, but IME
>> the cost is either 0, negligible, or negative (!).
>>
>> As mentioned above, if calling C++ code there's no choice about using T
>> instead of const T&, so for pragmatic reasons that should be allowed. But
>> only there, because...
>>
>>> Can you please explain these 'weirdities'?
>>> What are said "major unintended consequences"?
>>
>>
>> Rvalue references. They exist because of being able to bind temporaries to
>> const T& in C++, which means there's no way of knowing if your const T& was
>> originally a temporary or not. To disambiguate C++11 introduced the type
>> system horror that are rvalue references (which also do double-duty in
>> enabling perfect forwarding!).
>
>
> What's a concrete example that you would be required to know whether a
> const& is a temporary or not. I don't really see it otherwise. The solution
> everyone is saying to use as an alternative would be the literal case of how
> it would be implemented in the language.

He's trying to say that C++ introduced rvalue references because
normal references weren't able to allow for move semantics to exist.
It's a red-herring. D already has move semantics, they work well, and
they're not on trial here.

In C++'s case, it's not that references were deficient at being
references that C++ needed rval-references, it's that references were
deficient at being move-able.
That is not a problem that exists in D. It's fine for references to
just be references in D. We're not struggling to make references
move-able in D, that's not a thing, we already have move semantics.
Any extension of this conversation about references into C++
rvalue-references (T&&) and or move-semantics are red-herrings.
There's no such problem in D that needs to be resolved, and the
existing solution is excellent.

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 26 March 2018 at 19:25, Manu  wrote:
> On 26 March 2018 at 16:21, Rubn via Digitalmars-d
>  wrote:
>> On Monday, 26 March 2018 at 22:48:38 UTC, Walter Bright wrote:
>>>
>>> On 3/26/2018 12:24 PM, Manu wrote:

 On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
>
> C++ const T& -> D T


 Yeah, no... T may be big. Copying a large thing sucks. Memory copying
 is the slowest thing computers can do.
 As an API author, exactly as in C++, you will make a judgement on a
 case-by-case basis on this matter. It may be by-value, it may be by
 const-ref. It depends on a bunch of things, and they are points for
 consideration by the API author, not the user.
>>>
>>>
>>> Copying does suck, I agree. Consider the following:
>>>
>>> void foo(T t) { foo(t); } <= add this overload
>>> void foo(ref T t) { ... }
>>> T aaa();
>>>
>>> foo(aaa());
>>>
>>> With inlining, I suspect we can get the compiler to not make any extra
>>> copies. It's not that different from NRVO. And as a marvy bonus, no weird
>>> semantic problems (as Atila mentioned).
>>
>>
>> How do you add this overload for the following?
>>
>>
>> void foo(ref T t) { ... }
>>
>> void function(ref int) func = 
>> int aaa();
>>
>> func(aaa()); // err
>
> Exactly.

We're just kicking the can. And the only reason to do so is
ideological, as far as I can tell.
I want to hear an argument against... or any issue that's introduced
by allowing the implicit temporary?

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 26 March 2018 at 16:21, Rubn via Digitalmars-d
 wrote:
> On Monday, 26 March 2018 at 22:48:38 UTC, Walter Bright wrote:
>>
>> On 3/26/2018 12:24 PM, Manu wrote:
>>>
>>> On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d

 C++ const T& -> D T
>>>
>>>
>>> Yeah, no... T may be big. Copying a large thing sucks. Memory copying
>>> is the slowest thing computers can do.
>>> As an API author, exactly as in C++, you will make a judgement on a
>>> case-by-case basis on this matter. It may be by-value, it may be by
>>> const-ref. It depends on a bunch of things, and they are points for
>>> consideration by the API author, not the user.
>>
>>
>> Copying does suck, I agree. Consider the following:
>>
>> void foo(T t) { foo(t); } <= add this overload
>> void foo(ref T t) { ... }
>> T aaa();
>>
>> foo(aaa());
>>
>> With inlining, I suspect we can get the compiler to not make any extra
>> copies. It's not that different from NRVO. And as a marvy bonus, no weird
>> semantic problems (as Atila mentioned).
>
>
> How do you add this overload for the following?
>
>
> void foo(ref T t) { ... }
>
> void function(ref int) func = 
> int aaa();
>
> func(aaa()); // err

Exactly.

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 26 March 2018 at 15:48, Walter Bright via Digitalmars-d
 wrote:
> On 3/26/2018 12:24 PM, Manu wrote:
>>
>> On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
>>>
>>> C++ const T& -> D T
>>
>>
>> Yeah, no... T may be big. Copying a large thing sucks. Memory copying
>> is the slowest thing computers can do.
>> As an API author, exactly as in C++, you will make a judgement on a
>> case-by-case basis on this matter. It may be by-value, it may be by
>> const-ref. It depends on a bunch of things, and they are points for
>> consideration by the API author, not the user.
>
>
> Copying does suck, I agree. Consider the following:
>
> void foo(T t) { foo(t); } <= add this overload
> void foo(ref T t) { ... }
> T aaa();
>
> foo(aaa());
>
> With inlining, I suspect we can get the compiler to not make any extra
> copies. It's not that different from NRVO. And as a marvy bonus, no weird
> semantic problems (as Atila mentioned).

It's a terrible experience to add 2^n overloads, just to abuse the
parameter list (as in your example) to create the temporary in an
implicit manner. The compiler can easily create the exact same
temporary without requiring a bunch of overloads to be declared.
What's the advantage of requiring that effort from the user, rather
than just doing it at the call site?
There's side effects to that hack too. Now there are overloads; so
taking function pointers becomes awkward (might be important in some
cases). If the interface has a binary boundary (static lib/dll) then
what does that look like with respect to inlining? Where do the
overloads go if the function is virtual?
I already know the answers to these questions, but the point is,
there's a whole lot more baggage introduced into the scene that just
doesn't need to be there.

So, while I agree that's an existing workaround, it kinda misses the
point. This thread isn't about inlining, it's about NOT inlining. You
don't use ref args unless you have a reason to, and that reason is
likely to have friction with that particular work-around.

Re: rvalues -> ref (yup... again!)

[Rubn via Digitalmars-d]

On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:

C++ T&& (Rvalue reference) -> D T


Not really, in C++ it is an actual reference and you get to 
choose which function actually does the move. In D it just does 
the copy when passed to the function. So you can't do this in D.


void bar(T&& t)
{
// actually move contents of T
}

void foo(T&& t)
{
bar(std::forward(t)); // Can't do this in D without making 
another actual copy cause it isn't a reference

}

If replacing const T& with T chafes, I understand. I used to 
feel that way too. It's _possible_ that would incur a penalty 
in copying/moving, but IME the cost is either 0, negligible, or 
negative (!).


As mentioned above, if calling C++ code there's no choice about 
using T instead of const T&, so for pragmatic reasons that 
should be allowed. But only there, because...



Can you please explain these 'weirdities'?
What are said "major unintended consequences"?


Rvalue references. They exist because of being able to bind 
temporaries to const T& in C++, which means there's no way of 
knowing if your const T& was originally a temporary or not. To 
disambiguate C++11 introduced the type system horror that are 
rvalue references (which also do double-duty in enabling 
perfect forwarding!).


What's a concrete example that you would be required to know 
whether a const& is a temporary or not. I don't really see it 
otherwise. The solution everyone is saying to use as an 
alternative would be the literal case of how it would be 
implemented in the language.


D doesn't have or need rvalue references _because_ of not 
allowing temporaries to bind to ref const(T). You get move 
semantics in D without the pain. That's a win in my book.


Atila


I've come across a few pains of such. It make be easier to use 
but it comes at a performance hit. In part binaries become huge 
because of how "init" is implemented.


struct StaticArray(T, size_t capacity)
{
size_t length;
T[capacity] values;
}

Copying the above structure copies unnecessary data for any 
move/copy operation. Eg when length = 0, it'll still copy 
everything. This includes initialization.

Re: rvalues -> ref (yup... again!)

[Rubn via Digitalmars-d]

On Monday, 26 March 2018 at 22:48:38 UTC, Walter Bright wrote:

On 3/26/2018 12:24 PM, Manu wrote:

On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d

C++ const T& -> D T


Yeah, no... T may be big. Copying a large thing sucks. Memory 
copying

is the slowest thing computers can do.
As an API author, exactly as in C++, you will make a judgement 
on a
case-by-case basis on this matter. It may be by-value, it may 
be by
const-ref. It depends on a bunch of things, and they are 
points for

consideration by the API author, not the user.


Copying does suck, I agree. Consider the following:

void foo(T t) { foo(t); } <= add this overload
void foo(ref T t) { ... }
T aaa();

foo(aaa());

With inlining, I suspect we can get the compiler to not make 
any extra copies. It's not that different from NRVO. And as a 
marvy bonus, no weird semantic problems (as Atila mentioned).


How do you add this overload for the following?


void foo(ref T t) { ... }

void function(ref int) func = 
int aaa();

func(aaa()); // err

Re: rvalues -> ref (yup... again!)

[Walter Bright via Digitalmars-d]

On 3/26/2018 12:24 PM, Manu wrote:

On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d

C++ const T& -> D T


Yeah, no... T may be big. Copying a large thing sucks. Memory copying
is the slowest thing computers can do.
As an API author, exactly as in C++, you will make a judgement on a
case-by-case basis on this matter. It may be by-value, it may be by
const-ref. It depends on a bunch of things, and they are points for
consideration by the API author, not the user.


Copying does suck, I agree. Consider the following:

void foo(T t) { foo(t); } <= add this overload
void foo(ref T t) { ... }
T aaa();

foo(aaa());

With inlining, I suspect we can get the compiler to not make any extra copies. 
It's not that different from NRVO. And as a marvy bonus, no weird semantic 
problems (as Atila mentioned).

Re: rvalues -> ref (yup... again!)

[jmh530 via Digitalmars-d]

On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:


Can you please explain these 'weirdities'?
What are said "major unintended consequences"?
Explain how the situation if implemented would be any different 
than

the workaround?

This seems even simpler than the pow thing to me.
Rewrite:
func(f());
as:
{ auto __t0 = f(); func(__t0); }


How is that worse than the code you have to write:
T temp = f();
T zero = 0;
func(temp, zero);



I feel like this example wasn't really concrete enough for me. I 
wrote a version below that I think made it a little clearer for 
myself.


-

import std.stdio : writeln;

struct Foo
{
int data;
}

int foo(Foo x)
{
writeln("here");
return x.data;
}

int foo(ref Foo x)
{
writeln("there");
return x.data;
}


void main()
{
auto x = Foo(5);
auto y = foo(x);
writeln(y);

auto z = foo(Foo(5));
writeln(z);
}

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 26 March 2018 at 11:13, John Colvin via Digitalmars-d
 wrote:
> On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:
>>
>> On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
>>>
>>> Forked from the x^^y thread...
>>> 
>>
>>
>> There are too many replies on this thread, addressing all the comments
>> would take forever and pollute the thread itself. So forgive me if I say
>> something that was covered already by someone else.
>>
>> AFAIK being able to bind rvalues to `ref const(T)`, only makes sense when
>> calling C++ functions that take `const T&` (especially since that is
>> common). I have not yet heard any other use for them. I'd be in favour of
>> allowing it _only_ for `extern(C++)` functions. Otherwise use `auto ref` or
>> have overloads for pass-by-value and pass-by-ref.
>> I too, once a recent immigrant from the lands of C++, used to keep writing
>> `ref const(T)`. I just pass by value now.
>>
>> C++ T&& (forwarding reference) -> D auto ref T
>> C++ T&& (Rvalue reference) -> D T
>> C++ const T& -> D T
>> C++ T& -> D ref T
>>
>> If replacing const T& with T chafes, I understand. I used to feel that way
>> too. It's _possible_ that would incur a penalty in copying/moving, but IME
>> the cost is either 0, negligible, or negative (!).
>
>
> I'm tearing my remaining stubs of hair out trying to understand why memory
> copies (not talking about copy constructors) are needed when passing an
> rvalue to a non-ref function:
> https://stackoverflow.com/questions/49474685/passing-rvalue-to-non-ref-parameter-why-cant-the-compiler-elide-the-copy

Passing rvalues to non-ref functions may elide a memory copy. Moves
can be very efficient.
But we're talking about ref functions right? Not not-ref functions...?

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 26 March 2018 at 07:40, Atila Neves via Digitalmars-d
 wrote:
> On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
>>
>> Forked from the x^^y thread...
>> 
>
>
> There are too many replies on this thread, addressing all the comments would
> take forever and pollute the thread itself. So forgive me if I say something
> that was covered already by someone else.
>
> AFAIK being able to bind rvalues to `ref const(T)`, only makes sense when
> calling C++ functions that take `const T&` (especially since that is
> common). I have not yet heard any other use for them. I'd be in favour of
> allowing it _only_ for `extern(C++)` functions. Otherwise use `auto ref` or
> have overloads for pass-by-value and pass-by-ref.
> I too, once a recent immigrant from the lands of C++, used to keep writing
> `ref const(T)`. I just pass by value now.
>
> C++ T&& (forwarding reference) -> D auto ref T
> C++ T&& (Rvalue reference) -> D T
> C++ const T& -> D T

Yeah, no... T may be big. Copying a large thing sucks. Memory copying
is the slowest thing computers can do.
As an API author, exactly as in C++, you will make a judgement on a
case-by-case basis on this matter. It may be by-value, it may be by
const-ref. It depends on a bunch of things, and they are points for
consideration by the API author, not the user.

> C++ T& -> D ref T

I agree the other 3 cases are correct.

Re: rvalues -> ref (yup... again!)

[John Colvin via Digitalmars-d]

On Monday, 26 March 2018 at 14:40:03 UTC, Atila Neves wrote:

On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

Forked from the x^^y thread...



There are too many replies on this thread, addressing all the 
comments would take forever and pollute the thread itself. So 
forgive me if I say something that was covered already by 
someone else.


AFAIK being able to bind rvalues to `ref const(T)`, only makes 
sense when calling C++ functions that take `const T&` 
(especially since that is common). I have not yet heard any 
other use for them. I'd be in favour of allowing it _only_ for 
`extern(C++)` functions. Otherwise use `auto ref` or have 
overloads for pass-by-value and pass-by-ref.
I too, once a recent immigrant from the lands of C++, used to 
keep writing `ref const(T)`. I just pass by value now.


C++ T&& (forwarding reference) -> D auto ref T
C++ T&& (Rvalue reference) -> D T
C++ const T& -> D T
C++ T& -> D ref T

If replacing const T& with T chafes, I understand. I used to 
feel that way too. It's _possible_ that would incur a penalty 
in copying/moving, but IME the cost is either 0, negligible, or 
negative (!).


I'm tearing my remaining stubs of hair out trying to understand 
why memory copies (not talking about copy constructors) are 
needed when passing an rvalue to a non-ref function: 
https://stackoverflow.com/questions/49474685/passing-rvalue-to-non-ref-parameter-why-cant-the-compiler-elide-the-copy

Re: rvalues -> ref (yup... again!)

[Atila Neves via Digitalmars-d]

On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

Forked from the x^^y thread...



There are too many replies on this thread, addressing all the 
comments would take forever and pollute the thread itself. So 
forgive me if I say something that was covered already by someone 
else.


AFAIK being able to bind rvalues to `ref const(T)`, only makes 
sense when calling C++ functions that take `const T&` (especially 
since that is common). I have not yet heard any other use for 
them. I'd be in favour of allowing it _only_ for `extern(C++)` 
functions. Otherwise use `auto ref` or have overloads for 
pass-by-value and pass-by-ref.
I too, once a recent immigrant from the lands of C++, used to 
keep writing `ref const(T)`. I just pass by value now.


C++ T&& (forwarding reference) -> D auto ref T
C++ T&& (Rvalue reference) -> D T
C++ const T& -> D T
C++ T& -> D ref T

If replacing const T& with T chafes, I understand. I used to feel 
that way too. It's _possible_ that would incur a penalty in 
copying/moving, but IME the cost is either 0, negligible, or 
negative (!).


As mentioned above, if calling C++ code there's no choice about 
using T instead of const T&, so for pragmatic reasons that should 
be allowed. But only there, because...



Can you please explain these 'weirdities'?
What are said "major unintended consequences"?


Rvalue references. They exist because of being able to bind 
temporaries to const T& in C++, which means there's no way of 
knowing if your const T& was originally a temporary or not. To 
disambiguate C++11 introduced the type system horror that are 
rvalue references (which also do double-duty in enabling perfect 
forwarding!).


D doesn't have or need rvalue references _because_ of not 
allowing temporaries to bind to ref const(T). You get move 
semantics in D without the pain. That's a win in my book.


Atila

Re: rvalues -> ref (yup... again!)

[Rubn via Digitalmars-d]

On Sunday, 25 March 2018 at 01:43:43 UTC, Jonathan M Davis wrote:

On Sunday, March 25, 2018 00:34:38 Rubn via Digitalmars-d wrote:

On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote:
> Auto ref allows the unnecessary copy to be avoided for 
> lvalues and creates a temporary (as part of passing the 
> value) for rvalues. It has downsides (virtual functions and 
> extern(C++), but it does directly address the problem you're 
> talking about, unless I have totally misunderstood you.


You are trading syntax bloat for binary bloat. Having 4 
parameters with auto ref can generate 16 different variants 
for the exact same function. If you are doing any sort of 
mathematical calculations you could cause a potential cache 
miss for calling the same exact function because one of your 
parameters didn't end up being the same type of value. You are 
causing all this bloat and potential slowdowns all to avoid 
having to do this:


float value0 = a + b;
float value1 = c + d;
float value2 = e + f;
someFunc(value0, value1, value2);

That's an inherent flaw in design. You obviously agree that 
there is a problem, but how can you justify "auto ref" being a 
"good" solution to the problem? It's a horrible one, it causes 
excessive bloat and potential slowdowns just to get a cleaner 
readable syntax. That shouldn't be a solution that is deemed 
acceptable.


How good or bad it is depends on what you're doing and how many 
auto ref parameters there are in your code. If you're using it 
occasionally, it's not a problem at all, whereas if you're 
using it all over the place, you do get a lot of template bloat.


Regardless, John's point was that auto ref solves the problem 
of being able to call a function with both lvalues and rvalues 
without copying lvalues, and he didn't understand why anyone 
was trying to argue that it doesn't do that. And I agree with 
him on that point. It does not help with virtual functions or 
extern(C++), and it creates a lot of template bloat if it's 
used heavily, so there are downsides to it, but it _does_ solve 
the basic problem of being able to call a function with both 
lvalues and rvalues without copying the lvalues. It just 
doesn't solve it in a way that everyone considers acceptable.


That's a horrible way to look at it. You can still technically 
drive a car with square wheels, it may not be a very comfortable 
or efficient ride, but it still __does__ solve the basic problem 
of moving the car. Because it works we shouldn't look at any 
other solution.


auto ref also helps with forwarding refness, so it's useful for 
more than just avoiding copying lvalues, but the entire reason 
that auto ref was originally added to the language was to solve 
this exact problem. And maybe we need a different solution for 
some use cases (like virtual functions or cases where the 
template bloat is deemed unacceptable), but auto ref is in the 
language to solve this problem. So, much as at may make sense 
to argue that it's not a good solution to this problem, it 
really doesn't make sense to argue that it has nothing to do 
with it.


I never said it has nothing to do with it. It's just a really 
horrible solution.


If you are going to add an exception just for two minor cases, at 
that point you might as well start looking at adding rvalue 
references, cause that's what it is going to lead to. It'd 
provided better integration with C++ and provide an 
__acceptable__ solution to the problem (not one that solves the 
problem in a roundabout square-wheel-like way) that doesn't 
create runtime bloat and slowdown just to have clean readable 
syntax.

Re: rvalues -> ref (yup... again!)

[Nick Sabalausky (Abscissa) via Digitalmars-d]

On 03/24/2018 03:03 AM, Jonathan M Davis wrote:

On Saturday, March 24, 2018 01:37:10 Nick Sabalausky  via Digitalmars-d
wrote:

Why require the callee's author to add boilerplate? Just do it for all
ref params that are given an rvalue.


Because if the point of the function accepting its argument by ref is
because it's going to mutate the argument, then it makes no sense for it to
accept rvalues, 


1. That *isn't* always the core point of a function which takes a 
non-const ref argument.


2. It's the caller who decides whether or not the ref-result is needed, 
not the callee.


3. The frequent recurring complaints about no rvalue references are a 
testament that this is too common a use-case for the current "manually 
insert temporaries" workaround to be satisfactory.


4. If the whole point of disallowing rvalue references is to prevent 
accidents due to callers not realizing a param is being passed by ref 
(as it sounds like you're suggesting), then it's nothing but a broken 
half-solution, because it fails to provide that safety (and thus fails 
its own charter) when that same caller, once again not realizing a param 
is ref, passes an *lvalue* without expecting it to change. *If* the 
problem we want to solve here really is making sure a caller knows when 
a param is ref, we've failed, and the only way to actually accomplish it 
is the C# approach: Require callers to mark their ref args with "ref" 
and raise an error when they don't.

Re: rvalues -> ref (yup... again!)

[Jonathan M Davis via Digitalmars-d]

On Sunday, March 25, 2018 00:34:38 Rubn via Digitalmars-d wrote:
> On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote:
> > Auto ref allows the unnecessary copy to be avoided for lvalues
> > and creates a temporary (as part of passing the value) for
> > rvalues. It has downsides (virtual functions and extern(C++),
> > but it does directly address the problem you're talking about,
> > unless I have totally misunderstood you.
>
> You are trading syntax bloat for binary bloat. Having 4
> parameters with auto ref can generate 16 different variants for
> the exact same function. If you are doing any sort of
> mathematical calculations you could cause a potential cache miss
> for calling the same exact function because one of your
> parameters didn't end up being the same type of value. You are
> causing all this bloat and potential slowdowns all to avoid
> having to do this:
>
> float value0 = a + b;
> float value1 = c + d;
> float value2 = e + f;
> someFunc(value0, value1, value2);
>
> That's an inherent flaw in design. You obviously agree that there
> is a problem, but how can you justify "auto ref" being a "good"
> solution to the problem? It's a horrible one, it causes excessive
> bloat and potential slowdowns just to get a cleaner readable
> syntax. That shouldn't be a solution that is deemed acceptable.

How good or bad it is depends on what you're doing and how many auto ref
parameters there are in your code. If you're using it occasionally, it's not
a problem at all, whereas if you're using it all over the place, you do get
a lot of template bloat.

Regardless, John's point was that auto ref solves the problem of being able
to call a function with both lvalues and rvalues without copying lvalues,
and he didn't understand why anyone was trying to argue that it doesn't do
that. And I agree with him on that point. It does not help with virtual
functions or extern(C++), and it creates a lot of template bloat if it's
used heavily, so there are downsides to it, but it _does_ solve the basic
problem of being able to call a function with both lvalues and rvalues
without copying the lvalues. It just doesn't solve it in a way that everyone
considers acceptable.

auto ref also helps with forwarding refness, so it's useful for more than
just avoiding copying lvalues, but the entire reason that auto ref was
originally added to the language was to solve this exact problem. And maybe
we need a different solution for some use cases (like virtual functions or
cases where the template bloat is deemed unacceptable), but auto ref is in
the language to solve this problem. So, much as at may make sense to argue
that it's not a good solution to this problem, it really doesn't make sense
to argue that it has nothing to do with it.

- Jonathan M Davis

Re: rvalues -> ref (yup... again!)

[Rubn via Digitalmars-d]

On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote:
Here is a small proof of concept I made to demonstrate how easy 
it seems to be to use `auto ref` to call a C++ virtual const& 
function without incurring any more copies than would happen 
with the same calls from C++. I'm sure it could be improved a 
lot, but does the basic concept match what you would need?


// D source file:

/// mix this in to your extern(C++) class with a list of the 
functions where you

/// want to be able to pass rvalues to ref parameters.
auto rValueRefCalls(Funcs ...)()
{
string ret;
foreach (foo; Funcs)
ret ~= `extern(D) void ` ~ __traits(identifier, foo) ~ 
`(Args ...)(auto ref Args args)
if (__traits(compiles, (` ~ __traits(identifier, foo) 
~ `)(args)))

{
()(args);
}`;

return ret;
}

extern(C++)
{
class A
{
void foo(const ref int v);
mixin(rValueRefCalls!foo);
}

A makeA();
}

void main()
{
int x = 3;
auto a = makeA();
a.foo(x);
a.foo(3);
}


// C++ source file:

#include

class A
{
public:
virtual void foo(const int& v)
{
printf("%d\n", v);
}
};

A *makeA()
{
return new A;
}


That isn't going to scale to the number of potential functions 
that are going to need this. It's going to cause slow compile 
speeds and create a bloated binary.

Re: rvalues -> ref (yup... again!)

[Rubn via Digitalmars-d]

On Saturday, 24 March 2018 at 23:03:36 UTC, John Colvin wrote:
Auto ref allows the unnecessary copy to be avoided for lvalues 
and creates a temporary (as part of passing the value) for 
rvalues. It has downsides (virtual functions and extern(C++), 
but it does directly address the problem you're talking about, 
unless I have totally misunderstood you.


You are trading syntax bloat for binary bloat. Having 4 
parameters with auto ref can generate 16 different variants for 
the exact same function. If you are doing any sort of 
mathematical calculations you could cause a potential cache miss 
for calling the same exact function because one of your 
parameters didn't end up being the same type of value. You are 
causing all this bloat and potential slowdowns all to avoid 
having to do this:


float value0 = a + b;
float value1 = c + d;
float value2 = e + f;
someFunc(value0, value1, value2);

That's an inherent flaw in design. You obviously agree that there 
is a problem, but how can you justify "auto ref" being a "good" 
solution to the problem? It's a horrible one, it causes excessive 
bloat and potential slowdowns just to get a cleaner readable 
syntax. That shouldn't be a solution that is deemed acceptable.

Re: rvalues -> ref (yup... again!)

[John Colvin via Digitalmars-d]

On Saturday, 24 March 2018 at 17:30:35 UTC, Manu wrote:
On 24 March 2018 at 04:57, John Colvin via Digitalmars-d 
 wrote:

On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:


Forked from the x^^y thread...

On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d 
 wrote:


On 3/23/2018 11:09 AM, Manu wrote:



[...]



Rvalue references are not trivial and can have major 
unintended consequences. They're a rather ugly feature in 
C++, with weirdities. I doubt D will ever have them.



Can you please explain these 'weirdities'?
What are said "major unintended consequences"?
Explain how the situation if implemented would be any 
different than

the workaround?

This seems even simpler than the pow thing to me.
Rewrite:
func(f());
as:
{ auto __t0 = f(); func(__t0); }



I understand what you want, but I'm struggling to understand 
why it's such a huge deal.


Because it makes this kind of D code that interacts with C++
objectively worse than C++, and there's no reason for it.
You can't say to someone who just frustrated-ly doubled their 
line

count by manually introducing a bunch of temporaries in a tiny
function that appears to do something so simple as 'call a 
function',
that "oh yeah, isn't it cool that you can't just call your 
functions

anymore! isn't D cool! we should switch to D right?"
It's embarrassing. I've been put in the position where I have 
to try
and 'explain' this feature quite some number of times... they 
usually
just give me 'the look'™; ya know, quietly wondering if I'm 
still
sane, and all I end up with is someone who's about 95% less 
convinced

that D is cool than they were 5 seconds beforehand.
What pisses me off is that's such a pointless thing to happen, 
because

this issue is so trivial!

In my experience, people are evaluating how D will materially 
impact
the exact same code they're already writing in C++. This is one 
of
those ways that they will be materially impacted, and it's 
almost
enough all on its own to cause people to dismiss the entire 
thing on

the spot.
Pretty much the best case at this phase is that the D code is 
exactly

the same as C++. If we can trim off a few parens here and there
(ufcs?), maybe remove some '::' operators (modules that don't 
suck),

that's a huge win.


The reason you want to pass by reference is for performance, 
to avoid copying the data at the call boundary.


So there are 2 cases: an lvalue needs to be passed, or an 
rvalue needs to be passed.


1. The address of the lvalue is passed.

2. The rvalue is copied to a local, then the address of that 
local is passed.


So in the rvalue case, you're not getting the performance 
benefit of passing by reference, because you have to copy to a 
local anyway.


What I would do in D currently to get the same performance and 
API:


void foo(float[32] v) { foo(v); }
void foo(ref float[32] v) { ... }

or

void foo()(auto ref float[32] v) { ... }


Can't be extern(C++), can't be virtual either (both is likely).
I said before; you're talking about Scott Meyers 'universal
references' as a language concept, and I'm just talking about 
calling

a function.


but I dont' get how or why. It's exactly D's solution to the 
problem.


It doesn't solve the problem... it doesn't even address the 
problem. You're talking about a totally different thing >_<




Auto ref allows the unnecessary copy to be avoided for lvalues 
and creates a temporary (as part of passing the value) for 
rvalues. It has downsides (virtual functions and extern(C++), but 
it does directly address the problem you're talking about, unless 
I have totally misunderstood you.


Here is a small proof of concept I made to demonstrate how easy 
it seems to be to use `auto ref` to call a C++ virtual const& 
function without incurring any more copies than would happen with 
the same calls from C++. I'm sure it could be improved a lot, but 
does the basic concept match what you would need?


// D source file:

/// mix this in to your extern(C++) class with a list of the 
functions where you

/// want to be able to pass rvalues to ref parameters.
auto rValueRefCalls(Funcs ...)()
{
string ret;
foreach (foo; Funcs)
ret ~= `extern(D) void ` ~ __traits(identifier, foo) ~ 
`(Args ...)(auto ref Args args)
if (__traits(compiles, (` ~ __traits(identifier, foo) ~ 
`)(args)))

{
()(args);
}`;

return ret;
}

extern(C++)
{
class A
{
void foo(const ref int v);
mixin(rValueRefCalls!foo);
}

A makeA();
}

void main()
{
int x = 3;
auto a = makeA();
a.foo(x);
a.foo(3);
}


// C++ source file:

#include

class A
{
public:
virtual void foo(const int& v)
{
printf("%d\n", v);
}
};

A *makeA()
{
return new A;
}

Re: rvalues -> ref (yup... again!)

[Rubn via Digitalmars-d]

On Saturday, 24 March 2018 at 11:57:25 UTC, John Colvin wrote:
I understand what you want, but I'm struggling to understand 
why it's such a huge deal.


The reason you want to pass by reference is for performance, to 
avoid copying the data at the call boundary.


It's pretty simple:

float foo() { ... }
ref float bar() { ... }

void someFunc(ref float);

someFunc(bar()); // ok

float temp = foo();
someFunc(temp); // Have to create a temporary anyways for the 
function to work
someFunc(foo()); // Compile error, need to use the hideous code 
above



So there really isn't any performance penalty cause if you want 
to call that function you are going to have to create a temporary 
variable anyways, but now you can't just call the function in one 
line. It requires multiple lines and a temporary variable name.


This becomes especially horrible for math libraries:

void someFunc(ref Vector3) { ... }

someFunc(a + b); // Can't do this

Vector3 temp = a + b;
someFunc(temp); // Have to do this

So there's no performance penalty for what he is requesting, but 
allows for a cleaner syntax to do the exact same thing.


So there are 2 cases: an lvalue needs to be passed, or an 
rvalue needs to be passed.


1. The address of the lvalue is passed.

2. The rvalue is copied to a local, then the address of that 
local is passed.


So in the rvalue case, you're not getting the performance 
benefit of passing by reference, because you have to copy to a 
local anyway.


What I would do in D currently to get the same performance and 
API:


void foo(float[32] v) { foo(v); }
void foo(ref float[32] v) { ... }

or

void foo()(auto ref float[32] v) { ... }

What is so totally unacceptable about those solutions? I 
personally like the second because it scales better to multiple 
parameters. I know you have said it's not relevant and annoying 
that people bring up auto ref, but I dont' get how or why. It's 
exactly D's solution to the problem.


It doesn't scale better, that's part of the problem:

void foo()(auto ref MyType1, auto ref MyType2, auto ref MyType3, 
auto ref MyType4) { ... }


The above has the possibility of generating 16 different 
functions that basically all do the exact same thing. It creates 
excessive bloat, and now what if you want to take the address of 
the function? You can't cause you have to choose between one of 
the 16 variants.


This is template bloat at it's finest, except it isn't even doing 
anything useful. I can only imagine telling someone they have to 
code gen 16 identical functions just to be able to call a 
function without having to create useless temporary variables in 
the scope a function is being called in.

Re: rvalues -> ref (yup... again!)

[Dgame via Digitalmars-d]

Here is what I've used if I had to: 
https://p0nce.github.io/d-idioms/#Rvalue-references:-Understanding-auto-ref-and-then-not-using-it

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 24 March 2018 at 07:56, kinke via Digitalmars-d
 wrote:
> On Saturday, 24 March 2018 at 13:49:13 UTC, Timon Gehr wrote:
>>
>> What I'm saying is that I don't really buy Jonathan's argument. Basically,
>> you should just pass the correct arguments to functions, as you always need
>> to do. If you cannot use the result of some mutation that you need to use,
>> you will probably notice.
>
>
> I agree, but restricting it to const ref would be enough for almost all of
> my use cases. The MS C++ compiler just emits a warning when binding an
> rvalue to a mutable ref ('nonstandard extension used'), I'd find that
> absolutely viable for D too.
>
>> There are only three sensible ways to fix the problem:
>>
>> 1. Just allow rvalue arguments to bind to ref parameters. (My preferred
>> solution, though it will make the overloading rules slightly more
>> complicated.)
>
>
> I always thought the main concern was potential escaping refs to the rvalue,
> which would be solvable by allowing rvalues to be bound to `scope ref`
> params only. That'd be my preferred choice as well.

I touched on this.
It sounds reasonable at first glance. But the reasoning goes:
1. for safety reasons, we won't allow implicit stack temporaries to
pass to functions that may escape them.
2. support passing temporaries only to 'scope ref'
3. realise that the implicit temp created by an rvalue is identical to
the manually authored temp (or any other stack locals in the caller),
you find that the only reasonable option to satisfy the alleged safety
concern, is that all stack variables may never be passed to any
function that's not 'scope ref'.

Ie, you can't do this for safety reasons, because the exact same
reasoning would exclude all stack variables ever from being passed the
same way. The proposition is self-defeating.
I was attracted to this idea for a short while, until I realised it
was ridiculous. ;)


>> 3. Continue to require code bloat (auto ref) or manual boilerplate
>> (overloads). (I'm not very fond of this option, but it does not require a
>> language change.)
>
>
> While `auto ref` seems to have worked out surprisingly well for code written
> in D, it doesn't solve the problem when interfacing with (many) external C++
> functions taking structs (represented in D by structs as well) by (mostly
> const) ref. You're forced to declare lvalues for all of these args,
> uglifying the code substantially.

It's also not great for libraries. auto ref functions are template
functions... what if I want to export that function? I can't.
There are practical API and ABI reasons that you don't want every
function to be a template function.
Library authors carefully control which functions are 'real' functions
and which are templates, for a wide variety of reasons. Binary libs
are a thing. DLL's are a thing.

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 24 March 2018 at 06:49, Timon Gehr via Digitalmars-d
 wrote:
> On 24.03.2018 05:03, Manu wrote:
>>
>> I have no idea what this paragraph means... can you elaborate further
>> what you're talking about?
>
> This works:
>
> struct S{
> int x;
> void inc(){
> this.x++; // note: 'this' is passed by ref
> }
> }
>
> void main(){
> S().inc();
> }
>
> but this does not:
>
> struct S{
> int x;
> }
> void inc(ref S self){
> self.x++; // note: 'self' is passed by ref
> }
>
> void main(){
> S().inc();
> }
>
> I.e. there is a special case where your rewrite is already applied. Note how
> "inc" cannot even be made const.
>
> What I'm saying is that I don't really buy Jonathan's argument. Basically,
> you should just pass the correct arguments to functions, as you always need
> to do. If you cannot use the result of some mutation that you need to use,
> you will probably notice.

Your example demonstrates the exact reason why rvalue->ref is const&
in C++, and illegal in D though.
You mutate a temporary that times out at the end of the statement...
your statement is never assigned to anything, and has no effect.
If your statement is assigned to something, then you already have an
lvalue to pass to such a function that receives mutable ref.

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 24 March 2018 at 04:57, John Colvin via Digitalmars-d
 wrote:
> On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:
>>
>> Forked from the x^^y thread...
>>
>> On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d
>>  wrote:
>>>
>>> On 3/23/2018 11:09 AM, Manu wrote:


 [...]
>>>
>>>
>>> Rvalue references are not trivial and can have major unintended
>>> consequences. They're a rather ugly feature in C++, with weirdities. I doubt
>>> D will ever have them.
>>
>>
>> Can you please explain these 'weirdities'?
>> What are said "major unintended consequences"?
>> Explain how the situation if implemented would be any different than
>> the workaround?
>>
>> This seems even simpler than the pow thing to me.
>> Rewrite:
>> func(f());
>> as:
>> { auto __t0 = f(); func(__t0); }
>>
>
> I understand what you want, but I'm struggling to understand why it's such a
> huge deal.

Because it makes this kind of D code that interacts with C++
objectively worse than C++, and there's no reason for it.
You can't say to someone who just frustrated-ly doubled their line
count by manually introducing a bunch of temporaries in a tiny
function that appears to do something so simple as 'call a function',
that "oh yeah, isn't it cool that you can't just call your functions
anymore! isn't D cool! we should switch to D right?"
It's embarrassing. I've been put in the position where I have to try
and 'explain' this feature quite some number of times... they usually
just give me 'the look'™; ya know, quietly wondering if I'm still
sane, and all I end up with is someone who's about 95% less convinced
that D is cool than they were 5 seconds beforehand.
What pisses me off is that's such a pointless thing to happen, because
this issue is so trivial!

In my experience, people are evaluating how D will materially impact
the exact same code they're already writing in C++. This is one of
those ways that they will be materially impacted, and it's almost
enough all on its own to cause people to dismiss the entire thing on
the spot.
Pretty much the best case at this phase is that the D code is exactly
the same as C++. If we can trim off a few parens here and there
(ufcs?), maybe remove some '::' operators (modules that don't suck),
that's a huge win.


> The reason you want to pass by reference is for performance, to avoid
> copying the data at the call boundary.
>
> So there are 2 cases: an lvalue needs to be passed, or an rvalue needs to be
> passed.
>
> 1. The address of the lvalue is passed.
>
> 2. The rvalue is copied to a local, then the address of that local is
> passed.
>
> So in the rvalue case, you're not getting the performance benefit of passing
> by reference, because you have to copy to a local anyway.
>
> What I would do in D currently to get the same performance and API:
>
> void foo(float[32] v) { foo(v); }
> void foo(ref float[32] v) { ... }
>
> or
>
> void foo()(auto ref float[32] v) { ... }

Can't be extern(C++), can't be virtual either (both is likely).
I said before; you're talking about Scott Meyers 'universal
references' as a language concept, and I'm just talking about calling
a function.


> but I dont' get how or why. It's exactly D's solution to the problem.

It doesn't solve the problem... it doesn't even address the problem.
You're talking about a totally different thing >_<


> There's a little more work to be done when thinking about extern(C++) and/or
> virtual functions, but most code for most people isn't made of virtual
> extern(C++) functions that take large value types can't accept the cost of
> copying a few lvalues.

Correct, as I said before; people are getting bent out of shape over a
thing that will likely never affect them!
This will likely have very little impact on normal D code because D
const, auto ref, D move semantics, etc. It will have large impact on
interaction with C++ code, and the impression D is able to make on
that set of users. They're an important target market.

Re: rvalues -> ref (yup... again!)

[Johannes Pfau via Digitalmars-d]

Am Sat, 24 Mar 2018 17:10:53 + schrieb Johannes Pfau:

> Am Sat, 24 Mar 2018 01:04:00 -0600 schrieb Jonathan M Davis:
> 
>> As it stands, because a function can't accept rvalues by ref, it's
>> usually reasonable to assume that a function accepts its argument by
>> ref because it's mutating that argument rather than simply because it's
>> trying to avoid a copy. If ref suddenly starts accepting rvalues, then
>> we lose that.
> 
> Any reason you can't simply use `ref` to imply 'modifies value' and
> `const ref` as 'passed by ref for performance reasons'?

Sorry, I see Manu already asked the same question.

-- 
Johannes

Re: rvalues -> ref (yup... again!)

[Johannes Pfau via Digitalmars-d]

Am Sat, 24 Mar 2018 01:04:00 -0600 schrieb Jonathan M Davis:

> As it stands, because a function can't accept rvalues by ref, it's
> usually reasonable to assume that a function accepts its argument by ref
> because it's mutating that argument rather than simply because it's
> trying to avoid a copy. If ref suddenly starts accepting rvalues, then
> we lose that.

Any reason you can't simply use `ref` to imply 'modifies value' and 
`const ref` as 'passed by ref for performance reasons'?

-- 
Johannes

Re: rvalues -> ref (yup... again!)

[kinke via Digitalmars-d]

On Saturday, 24 March 2018 at 15:36:14 UTC, Timon Gehr wrote:

On 24.03.2018 15:56, kinke wrote:
I agree, but restricting it to const ref would be enough for 
almost all of my use cases. The MS C++ compiler just emits a 
warning when binding an rvalue to a mutable ref ('nonstandard 
extension used'), I'd find that absolutely viable for D too.

...


A warning is not viable. (There's no good way to fix it.)


As long as specific warnings cannot be suppressed via pragmas, 
one would need to predeclare the lvalue to get rid of it; fine 
IMHO for the, as I expect, very rare use cases.


There is no difference between escaping refs to an rvalue and 
escaping refs to a short-lived lvalue, as the callee has no 
idea where the address is coming from anyway. According to 
Walter, ref parameters are not supposed to be escaped, and 
@safe will enforce it.


Alright, the less keywords overhead, the better. :)

You can add additional overloads on the D side. (This can even 
be automated using a string mixin.)


Right I can, but I don't want to add 7 overloads for a C++ 
function taking 3 params by const ref. Even if autogenerated by 
some tool or fancy mixins, the code's legibility would suffer a 
lot. D's syntax is IMO one of its strongest selling points, and 
that shouldn't degrade when it comes to C(++) interop.

Re: rvalues -> ref (yup... again!)

[Timon Gehr via Digitalmars-d]

On 24.03.2018 15:56, kinke wrote:

On Saturday, 24 March 2018 at 13:49:13 UTC, Timon Gehr wrote:
What I'm saying is that I don't really buy Jonathan's argument. 
Basically, you should just pass the correct arguments to functions, as 
you always need to do. If you cannot use the result of some mutation 
that you need to use, you will probably notice.


I agree, but restricting it to const ref would be enough for almost all 
of my use cases. The MS C++ compiler just emits a warning when binding 
an rvalue to a mutable ref ('nonstandard extension used'), I'd find that 
absolutely viable for D too.

...


A warning is not viable. (There's no good way to fix it.)


There are only three sensible ways to fix the problem:

1. Just allow rvalue arguments to bind to ref parameters. (My 
preferred solution, though it will make the overloading rules slightly 
more complicated.)


I always thought the main concern was potential escaping refs to the 
rvalue, which would be solvable by allowing rvalues to be bound to 
`scope ref` params only. That'd be my preferred choice as well.

...


There is no difference between escaping refs to an rvalue and escaping 
refs to a short-lived lvalue, as the callee has no idea where the 
address is coming from anyway. According to Walter, ref parameters are 
not supposed to be escaped, and @safe will enforce it.


Also, AFAIU, "scope" in "scope ref T" already applies to "T", not "ref".

2. Add some _new_ annotation for ref parameters that signifies that 
you want the same treatment for them that the implicit 'this' 
reference gets. (A close second.)


*Shudder*.
...


Well, it beats "const".

3. Continue to require code bloat (auto ref) or manual boilerplate 
(overloads). (I'm not very fond of this option, but it does not 
require a language change.)


While `auto ref` seems to have worked out surprisingly well for code 
written in D, it doesn't solve the problem when interfacing with (many) 
external C++ functions taking structs (represented in D by structs as 
well) by (mostly const) ref. You're forced to declare lvalues for all of 
these args, uglifying the code substantially.


You can add additional overloads on the D side. (This can even be 
automated using a string mixin.)

Re: rvalues -> ref (yup... again!)

[kinke via Digitalmars-d]

On Saturday, 24 March 2018 at 13:49:13 UTC, Timon Gehr wrote:
What I'm saying is that I don't really buy Jonathan's argument. 
Basically, you should just pass the correct arguments to 
functions, as you always need to do. If you cannot use the 
result of some mutation that you need to use, you will probably 
notice.


I agree, but restricting it to const ref would be enough for 
almost all of my use cases. The MS C++ compiler just emits a 
warning when binding an rvalue to a mutable ref ('nonstandard 
extension used'), I'd find that absolutely viable for D too.



There are only three sensible ways to fix the problem:

1. Just allow rvalue arguments to bind to ref parameters. (My 
preferred solution, though it will make the overloading rules 
slightly more complicated.)


I always thought the main concern was potential escaping refs to 
the rvalue, which would be solvable by allowing rvalues to be 
bound to `scope ref` params only. That'd be my preferred choice 
as well.


2. Add some _new_ annotation for ref parameters that signifies 
that you want the same treatment for them that the implicit 
'this' reference gets. (A close second.)


*Shudder*.

3. Continue to require code bloat (auto ref) or manual 
boilerplate (overloads). (I'm not very fond of this option, but 
it does not require a language change.)


While `auto ref` seems to have worked out surprisingly well for 
code written in D, it doesn't solve the problem when interfacing 
with (many) external C++ functions taking structs (represented in 
D by structs as well) by (mostly const) ref. You're forced to 
declare lvalues for all of these args, uglifying the code 
substantially.

Re: rvalues -> ref (yup... again!)

[Timon Gehr via Digitalmars-d]

On 24.03.2018 05:03, Manu wrote:

On 23 March 2018 at 20:17, Timon Gehr via Digitalmars-d
 wrote:

On 24.03.2018 02:16, Manu wrote:


This is an interesting point, but I don't think it changes the balance
in any way. Thinking of the majority of my anecdotal cases, I don't
think it would be a problem.
Something complex enough for const to be a problem likely doesn't
conform to this pattern.



Why aim for "it often works", when we want "it always works"? Forcing const
upon people who want to pass rvalues by reference is just not good enough.
It is bad language design.


I think you need to re-read the whole thread, and understand what
we're even talking about. ...


That will not be necessary. I wouldn't even have had to read it the 
first time. Those discussions always go the same way:


M: I wish we could pass rvalue arguments to ref parameters.
J: That would be terrible, as people would then pass rvalues as ref by 
accident and not see the mutation that the author of the function 
intended them to see.

M: Only do it for const ref parameters then.
T: No, this has nothing to do with const.

(M can be replaced by a variety of other letters; this is a somewhat 
common feature request.)



Nobody wants to pass rvalues by mutable-ref... that's completely
pointless, since it's an rvalue that will timeout immediately anyway.
Passing by const-ref is perfectly acceptable.
...


Your temporary might have mutable indirections. Maybe you don't want to 
be forced to annotate your methods as const, limiting your future options.



I suspect Jonathan's talking about classic D situations with const
like, I might pass by const-ref, but then I can't call a getter that
caches the result.
That's a classic problem with D's const, and that's not on debate
here. I don't think that has any impact on this proposal; people
understand what const means in D, and that's no different here than
anywhere else.
...


Your proposal _changes_ the meaning of const. I.e., "const does all it 
did previously and now it also allows rvalues to be passed to ref 
functions". This is bad, as one has little to do with the other, yet now 
you couple them. Programmers who want to pass rvalues as ref do not 
necessarily want to use D const on their objects.





Also I think the point about documenting mutation intent is moot, as rvalues
can be receivers for method calls, which will _already_ pass an rvalue by
reference no matter whether it intends to mutate it or not. We can require
some special annotation for this behavior, but I'd be perfectly fine without
it.


I have no idea what this paragraph means... can you elaborate further
what you're talking about?



This works:

struct S{
int x;
void inc(){
this.x++; // note: 'this' is passed by ref
}
}

void main(){
S().inc();
}

but this does not:

struct S{
int x;
}
void inc(ref S self){
self.x++; // note: 'self' is passed by ref
}

void main(){
S().inc();
}

I.e. there is a special case where your rewrite is already applied. Note 
how "inc" cannot even be made const.


What I'm saying is that I don't really buy Jonathan's argument. 
Basically, you should just pass the correct arguments to functions, as 
you always need to do. If you cannot use the result of some mutation 
that you need to use, you will probably notice.



There are only three sensible ways to fix the problem:

1. Just allow rvalue arguments to bind to ref parameters. (My preferred 
solution, though it will make the overloading rules slightly more 
complicated.)


2. Add some _new_ annotation for ref parameters that signifies that you 
want the same treatment for them that the implicit 'this' reference 
gets. (A close second.)


3. Continue to require code bloat (auto ref) or manual boilerplate 
(overloads). (I'm not very fond of this option, but it does not require 
a language change.)

Re: rvalues -> ref (yup... again!)

[MattCoder via Digitalmars-d]

On Saturday, 24 March 2018 at 11:57:25 UTC, John Colvin wrote:
I understand what you want, but I'm struggling to understand 
why it's such a huge deal.

...
What I would do in D currently to get the same performance and 
API:


void foo(float[32] v) { foo(v); }
void foo(ref float[32] v) { ... }

or

void foo()(auto ref float[32] v) { ... }

What is so totally unacceptable about those solutions?


I hope OP answers that, because that was what I tried to refer in 
my post.


As your example, it works like in C and I'd prefer to use it in 
my code, instead of passing a rvalue as reference.


Matt.

Re: rvalues -> ref (yup... again!)

[John Colvin via Digitalmars-d]

On Friday, 23 March 2018 at 22:01:44 UTC, Manu wrote:

Forked from the x^^y thread...

On 23 March 2018 at 12:16, Walter Bright via Digitalmars-d 
 wrote:

On 3/23/2018 11:09 AM, Manu wrote:


[...]


Rvalue references are not trivial and can have major 
unintended consequences. They're a rather ugly feature in C++, 
with weirdities. I doubt D will ever have them.


Can you please explain these 'weirdities'?
What are said "major unintended consequences"?
Explain how the situation if implemented would be any different 
than

the workaround?

This seems even simpler than the pow thing to me.
Rewrite:
func(f());
as:
{ auto __t0 = f(); func(__t0); }



I understand what you want, but I'm struggling to understand why 
it's such a huge deal.


The reason you want to pass by reference is for performance, to 
avoid copying the data at the call boundary.


So there are 2 cases: an lvalue needs to be passed, or an rvalue 
needs to be passed.


1. The address of the lvalue is passed.

2. The rvalue is copied to a local, then the address of that 
local is passed.


So in the rvalue case, you're not getting the performance benefit 
of passing by reference, because you have to copy to a local 
anyway.


What I would do in D currently to get the same performance and 
API:


void foo(float[32] v) { foo(v); }
void foo(ref float[32] v) { ... }

or

void foo()(auto ref float[32] v) { ... }

What is so totally unacceptable about those solutions? I 
personally like the second because it scales better to multiple 
parameters. I know you have said it's not relevant and annoying 
that people bring up auto ref, but I dont' get how or why. It's 
exactly D's solution to the problem.


There's a little more work to be done when thinking about 
extern(C++) and/or virtual functions, but most code for most 
people isn't made of virtual extern(C++) functions that take 
large value types can't accept the cost of copying a few lvalues.

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 24 March 2018 at 00:04, Jonathan M Davis via Digitalmars-d
 wrote:
> On Saturday, March 24, 2018 01:37:10 Nick Sabalausky  via Digitalmars-d
> wrote:
>> On 03/23/2018 07:46 PM, Jonathan M Davis wrote:
>> > On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d
> wrote:
>> >> It never made any sense to me that there could be any problem
>> >> with the compiler automatically creating a temporary hidden
>> >> lvalue so a ref could be taken. If there IS any problem, I can
>> >> only imagine it would be symptomatic of a different, larger
>> >> problem.
>> >
>> > It can be a serious API problem if you can't look at ref and know that
>> > the intention is that the original argument's value will be used and
>> > then mutated (whereas with out, it will be mutated, but the original
>> > value won't be used).
>>
>> ???. That's equally true when an lvalue is passed in.
>>
>> > However, that could be solved by having a different attribute indicate
>> > that the idea is that the compiler will accept rvalues and create
>> > temporaries for them if they're passed instead of an lvalue. Then, the
>> > situation is clear.
>> Why require the callee's author to add boilerplate? Just do it for all
>> ref params that are given an rvalue.
>
> Because if the point of the function accepting its argument by ref is
> because it's going to mutate the argument, then it makes no sense for it to
> accept rvalues, whereas if the point of the function accepting its argument
> by ref is to just avoid a copy, then it makes sense to accept both. It
> should be clear from the API which is intended.

Write const; it's as clear as day!

Re: rvalues -> ref (yup... again!)

[Jonathan M Davis via Digitalmars-d]

On Saturday, March 24, 2018 01:37:10 Nick Sabalausky  via Digitalmars-d 
wrote:
> On 03/23/2018 07:46 PM, Jonathan M Davis wrote:
> > On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d 
wrote:
> >> It never made any sense to me that there could be any problem
> >> with the compiler automatically creating a temporary hidden
> >> lvalue so a ref could be taken. If there IS any problem, I can
> >> only imagine it would be symptomatic of a different, larger
> >> problem.
> >
> > It can be a serious API problem if you can't look at ref and know that
> > the intention is that the original argument's value will be used and
> > then mutated (whereas with out, it will be mutated, but the original
> > value won't be used).
>
> ???. That's equally true when an lvalue is passed in.
>
> > However, that could be solved by having a different attribute indicate
> > that the idea is that the compiler will accept rvalues and create
> > temporaries for them if they're passed instead of an lvalue. Then, the
> > situation is clear.
> Why require the callee's author to add boilerplate? Just do it for all
> ref params that are given an rvalue.

Because if the point of the function accepting its argument by ref is
because it's going to mutate the argument, then it makes no sense for it to
accept rvalues, whereas if the point of the function accepting its argument
by ref is to just avoid a copy, then it makes sense to accept both. It
should be clear from the API which is intended. As it stands, because a
function can't accept rvalues by ref, it's usually reasonable to assume that
a function accepts its argument by ref because it's mutating that argument
rather than simply because it's trying to avoid a copy. If ref suddenly
starts accepting rvalues, then we lose that.

With C++, you know the difference because, only const& accepts rvalues, and
whether it's const or some other attribute, I'd very much like that any ref
that accepts rvalues in D be marked with something to indicate that it does
rather than making ref do double-duty and make it unclear what it's there
for.

- Jonathan M Davis

Re: rvalues -> ref (yup... again!)

[Nick Sabalausky (Abscissa) via Digitalmars-d]

On 03/23/2018 07:46 PM, Jonathan M Davis wrote:

On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d wrote:

It never made any sense to me that there could be any problem
with the compiler automatically creating a temporary hidden
lvalue so a ref could be taken. If there IS any problem, I can
only imagine it would be symptomatic of a different, larger
problem.


It can be a serious API problem if you can't look at ref and know that the
intention is that the original argument's value will be used and then
mutated (whereas with out, it will be mutated, but the original value won't
be used).


???. That's equally true when an lvalue is passed in.


However, that could be solved by having a different attribute indicate that
the idea is that the compiler will accept rvalues and create temporaries for
them if they're passed instead of an lvalue. Then, the situation is clear.


Why require the callee's author to add boilerplate? Just do it for all 
ref params that are given an rvalue.



As for rvalue references in general, I can never remember what exactly the
problem is. IIRC, part of it relates to the fact that it makes it impossible
for the compiler to know whether it's dealing with an actual lvalue or not,


As long as the compiler takes the advocated "automatic hidden temporary" 
approach, then it *is* an actual lvalue. We're talking nothing more than 
sugar here. Sugar for what we're already forced to do manually. There 
*cannot* be a technical problem here that isn't *already* a problem with 
the author manually creating a temp var.

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 23 March 2018 at 20:19, Timon Gehr via Digitalmars-d
 wrote:
> On 24.03.2018 04:14, Manu wrote:
>>
>> On 23 March 2018 at 20:03, Timon Gehr via Digitalmars-d
>>  wrote:
>>>
>>> On 24.03.2018 01:35, Manu wrote:


 Okay, let's read 'const ref' every time I say 'ref'. I thought that
 would be fairly safe to assume. Sorry!
>>>
>>>
>>>
>>> Absolutely not. It makes absolutely no sense to restrict rvalue
>>> references
>>> to const objects. (Recall that const is transitive and actually prevents
>>> mutation. This is not C++.)
>>
>>
>> We're not talking about rvalue-references... were talking about
>> not-rvalue-references to temporaries.
>>
>
> Let me rephrase:
>
> Absolutely not. It makes absolutely no sense to force const for
> not-rvalue-references to temporaries. (Recall that const is transitive and
> actually prevents mutation. This is not C++.)

Yes, I know... why would you mutate an argument who's life doesn't
extend beyond the call? Any mutation is pointless.

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 23 March 2018 at 20:17, Timon Gehr via Digitalmars-d
 wrote:
> On 24.03.2018 02:16, Manu wrote:
>>
>> This is an interesting point, but I don't think it changes the balance
>> in any way. Thinking of the majority of my anecdotal cases, I don't
>> think it would be a problem.
>> Something complex enough for const to be a problem likely doesn't
>> conform to this pattern.
>
>
> Why aim for "it often works", when we want "it always works"? Forcing const
> upon people who want to pass rvalues by reference is just not good enough.
> It is bad language design.

I think you need to re-read the whole thread, and understand what
we're even talking about.
Nobody wants to pass rvalues by mutable-ref... that's completely
pointless, since it's an rvalue that will timeout immediately anyway.
Passing by const-ref is perfectly acceptable.

I suspect Jonathan's talking about classic D situations with const
like, I might pass by const-ref, but then I can't call a getter that
caches the result.
That's a classic problem with D's const, and that's not on debate
here. I don't think that has any impact on this proposal; people
understand what const means in D, and that's no different here than
anywhere else.


> Also I think the point about documenting mutation intent is moot, as rvalues
> can be receivers for method calls, which will _already_ pass an rvalue by
> reference no matter whether it intends to mutate it or not. We can require
> some special annotation for this behavior, but I'd be perfectly fine without
> it.

I have no idea what this paragraph means... can you elaborate further
what you're talking about?

Re: rvalues -> ref (yup... again!)

[Timon Gehr via Digitalmars-d]

On 24.03.2018 04:14, Manu wrote:

On 23 March 2018 at 20:03, Timon Gehr via Digitalmars-d
 wrote:

On 24.03.2018 01:35, Manu wrote:


Okay, let's read 'const ref' every time I say 'ref'. I thought that
would be fairly safe to assume. Sorry!



Absolutely not. It makes absolutely no sense to restrict rvalue references
to const objects. (Recall that const is transitive and actually prevents
mutation. This is not C++.)


We're not talking about rvalue-references... were talking about
not-rvalue-references to temporaries.



Let me rephrase:

Absolutely not. It makes absolutely no sense to force const for 
not-rvalue-references to temporaries. (Recall that const is transitive 
and actually prevents mutation. This is not C++.)

Re: rvalues -> ref (yup... again!)

[Timon Gehr via Digitalmars-d]

On 24.03.2018 02:16, Manu wrote:

This is an interesting point, but I don't think it changes the balance
in any way. Thinking of the majority of my anecdotal cases, I don't
think it would be a problem.
Something complex enough for const to be a problem likely doesn't
conform to this pattern.


Why aim for "it often works", when we want "it always works"? Forcing 
const upon people who want to pass rvalues by reference is just not good 
enough. It is bad language design.


Also I think the point about documenting mutation intent is moot, as 
rvalues can be receivers for method calls, which will _already_ pass an 
rvalue by reference no matter whether it intends to mutate it or not. We 
can require some special annotation for this behavior, but I'd be 
perfectly fine without it.

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 23 March 2018 at 20:03, Timon Gehr via Digitalmars-d
 wrote:
> On 24.03.2018 01:35, Manu wrote:
>>
>> Okay, let's read 'const ref' every time I say 'ref'. I thought that
>> would be fairly safe to assume. Sorry!
>
>
> Absolutely not. It makes absolutely no sense to restrict rvalue references
> to const objects. (Recall that const is transitive and actually prevents
> mutation. This is not C++.)

We're not talking about rvalue-references... were talking about
not-rvalue-references to temporaries.

Re: rvalues -> ref (yup... again!)

[Timon Gehr via Digitalmars-d]

On 24.03.2018 01:35, Manu wrote:

Okay, let's read 'const ref' every time I say 'ref'. I thought that
would be fairly safe to assume. Sorry!


Absolutely not. It makes absolutely no sense to restrict rvalue 
references to const objects. (Recall that const is transitive and 
actually prevents mutation. This is not C++.)

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 23 March 2018 at 18:06, Jonathan M Davis via Digitalmars-d
 wrote:
> On Friday, March 23, 2018 17:20:09 Manu via Digitalmars-d wrote:
>> On 23 March 2018 at 16:58, Jonathan M Davis via Digitalmars-d
>>
>>  wrote:
>> > On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote:
>> >> Well, to be honest I still can't understand why would you want to
>> >> pass a RValue as reference.
>> >
>> > Well, it's frequently the case that you don't want to copy an object if
>> > you don't have to - especially in the gaming world, where every ounce
>> > of performance matters. If a function accepts an argument by ref, then
>> > no copy is made, but then you have to pass an lvalue, making it really
>> > annoying to call the function when what you have is an rvalue. On the
>> > other hand, if the function doesn't accept its argument by ref, then
>> > you can pass an rvalue (and it will be moved, making that efficient),
>> > but then lvalues get copied when that's often not what you want. C++'s
>> > solution to this was rvalue references.
>>
>> Ummm... rvalue-references are something completely different.
>> rval-ref's are C++'s solution to move semantics.
>> C++ just simply accepts rvalues passed to const& args. It makes a temp
>> and passes the ref, as you expect.
>
> It was my understanding that that _was_ an rvalue reference, and I remember
> that Andrei was against const ref accepting rvalues in D due to issues with
> rvalue references.

C++ const& to 'rvalues' are just lvalue refs to temporaries, exactly
as I propose here.
rval-ref's are something completely different (all about move
semantics), and have nothing to do with this conversation.

There is a potentially interesting parallel conversation which
discusses how to interact with extern(C++) functions that receive
rvalue ref's, but that actually is a complex conversation, and no
simple answers exist.


> In any case, I have a terrible time remembering
> Andrei's exact arguments, but he feels very strongly about them, so anyone
> looking to convince him is going to have a hard time of it.

Fortunately, I'm not trying to make any sort of argument for rvalue-ref's in D.


> My biggest concern in all of this is that I don't want to see ref start
> accepting rvalues as has been occasionally discussed. It needs to be clear
> when a function is accept an argument by ref because it's going to mutate
> the object and when it's accepting by ref because it wants to avoid a copy.

It's not going to mutate the argument, because it's const.


> The addition of const solves that problem for C++, but given how restrictive
> const is in D, I doubt that much of anyone would ultimately be very happy
> with using const ref in their code very often.

I expect I'll be 100% satisfied. And if not, it'll be something very
close to 100%.
This pattern is quite unlikely to proliferate within D code natively
(because auto ref, D move semantics, classes-as-ref-types, and such),
but it's essential for interacting with C++.


> auto ref has everything to do with a function accepting both rvalues and
> lvalues without making a copy. auto ref is the solution that was introduced
> into D to solve that very problem.

Yeah, somehow that emerged from this conversation years ago. I
aggressively expressed at the time that I never accepted it as a
solution, because it's not.
It's got nothing to do with this issue, and I said at the time that
I'll be very annoyed if it starts getting raised in this context ;)


> It's just that it has limitations that
> make it inappropriate in a number of cases, so you don't consider it a
> solution to your problem.

It's orthogonal to this conversation. It's the ability for templates
to automate the ref-ness of args for calling efficiency.
It's something like scott myers 'universal references'; ie,
`template void func(T&& arg)`. It's really got nothing to
do with this conversation.


>> > So, C++ gives you control over which you do, and it's not
>> > necessarily straightforward as to which you should use (though plenty of
>> > older C++ programmers likely just use const& all over the place out of
>> > habit).
>>
>> D gives you the same set of options; except that passing args by ref
>> is a PITA in D, and ruins your code.
>
> Which is why I said that D doesn't give you the same set of options.

It does though; D just forces you to write the temps that should be
implicit by hand. There's no reason for this. It just makes code ugly,
people angry, and there is no advantage.

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 23 March 2018 at 17:58, Jonathan M Davis via Digitalmars-d
 wrote:
> On Friday, March 23, 2018 17:35:11 Manu via Digitalmars-d wrote:
>> > but that by itself isn't
>> > enough if you want it to be clear whether a function is supposed to be
>> > mutating the argument
>>
>> Functions that receive const args make it pretty clear that they don't
>> intend to mutate the arg.
>
> Yes, but with how restrictive const is in D, I have a very hard time
> believing that it's going to work well to start using const ref much even if
> it accepted rvalues.

This is an interesting point, but I don't think it changes the balance
in any way. Thinking of the majority of my anecdotal cases, I don't
think it would be a problem.
Something complex enough for const to be a problem likely doesn't
conform to this pattern.

Further, extern(C++) functions that receive const& args are 'const
ref' regardless of D's const semantics. So for extern(C++), which I
suspect is a high percentage of cases where this issue is significant
(because D doesn't naturally follow C++'s pattern so much anyway),
that point doesn't matter.

Re: rvalues -> ref (yup... again!)

[Jonathan M Davis via Digitalmars-d]

On Friday, March 23, 2018 17:20:09 Manu via Digitalmars-d wrote:
> On 23 March 2018 at 16:58, Jonathan M Davis via Digitalmars-d
>
>  wrote:
> > On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote:
> >> Well, to be honest I still can't understand why would you want to
> >> pass a RValue as reference.
> >
> > Well, it's frequently the case that you don't want to copy an object if
> > you don't have to - especially in the gaming world, where every ounce
> > of performance matters. If a function accepts an argument by ref, then
> > no copy is made, but then you have to pass an lvalue, making it really
> > annoying to call the function when what you have is an rvalue. On the
> > other hand, if the function doesn't accept its argument by ref, then
> > you can pass an rvalue (and it will be moved, making that efficient),
> > but then lvalues get copied when that's often not what you want. C++'s
> > solution to this was rvalue references.
>
> Ummm... rvalue-references are something completely different.
> rval-ref's are C++'s solution to move semantics.
> C++ just simply accepts rvalues passed to const& args. It makes a temp
> and passes the ref, as you expect.

It was my understanding that that _was_ an rvalue reference, and I remember
that Andrei was against const ref accepting rvalues in D due to issues with
rvalue references. It's quite possible that I misremember though, and I
certainly am not an expert on all of the issues with rvalues and references
in C++, and my C++ knowledge is getting increasingly rusty, since I don't
use C++ much anymore. In any case, I have a terrible time remembering
Andrei's exact arguments, but he feels very strongly about them, so anyone
looking to convince him is going to have a hard time of it.

My biggest concern in all of this is that I don't want to see ref start
accepting rvalues as has been occasionally discussed. It needs to be clear
when a function is accept an argument by ref because it's going to mutate
the object and when it's accepting by ref because it wants to avoid a copy.
The addition of const solves that problem for C++, but given how restrictive
const is in D, I doubt that much of anyone would ultimately be very happy
with using const ref in their code very often.

> > In many cases, it works great, whereas in others,
> > it doesn't work at all (e.g. virtual functions), and it can result in
> > template bloat.
>
> And templates, that's another case where it fails.
> auto-ref is something else unrelated to this topic, and it's useful in
> a different set of cases for a different set of uses/reasons. It's got
> nothing to do with this.

auto ref has everything to do with a function accepting both rvalues and
lvalues without making a copy. auto ref is the solution that was introduced
into D to solve that very problem. It's just that it has limitations that
make it inappropriate in a number of cases, so you don't consider it a
solution to your problem.

> > So, C++ gives you control over which you do, and it's not
> > necessarily straightforward as to which you should use (though plenty of
> > older C++ programmers likely just use const& all over the place out of
> > habit).
>
> D gives you the same set of options; except that passing args by ref
> is a PITA in D, and ruins your code.

Which is why I said that D doesn't give you the same set of options.

- Jonathan M Davis

Re: rvalues -> ref (yup... again!)

[Jonathan M Davis via Digitalmars-d]

On Friday, March 23, 2018 17:35:11 Manu via Digitalmars-d wrote:
> > but that by itself isn't
> > enough if you want it to be clear whether a function is supposed to be
> > mutating the argument
>
> Functions that receive const args make it pretty clear that they don't
> intend to mutate the arg.

Yes, but with how restrictive const is in D, I have a very hard time
believing that it's going to work well to start using const ref much even if
it accepted rvalues.

- Jonathan M Davis

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 23 March 2018 at 17:48, MattCoder via Digitalmars-d
 wrote:
>
> Question:
>
> In C++ the signature of the function which will receive the references like
> in this case, need to be "const ref" parameters, right? - If yes, then since
> it's const ref parameter, will not change the value passed, even if it's
> lvalue, right?

Right. It's so obvious isn't it ;)

Re: rvalues -> ref (yup... again!)

[MattCoder via Digitalmars-d]

On Friday, 23 March 2018 at 23:58:05 UTC, Jonathan M Davis wrote:
On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d 
wrote:
Well, to be honest I still can't understand why would you want 
to pass a RValue as reference.


Well, it's frequently the case that you don't want to copy an 
object if you don't have to...

- Jonathan M Davis


Well the concept it's OK. (Differences between passing by value 
vs reference, copy etc.). Except for the const thing in C++, 
because I don't know this language, and by the way thanks for 
explaining that.


Question:

In C++ the signature of the function which will receive the 
references like in this case, need to be "const ref" parameters, 
right? - If yes, then since it's const ref parameter, will not 
change the value passed, even if it's lvalue, right?


Matt.

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 23 March 2018 at 16:46, Jonathan M Davis via Digitalmars-d
 wrote:
> On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d wrote:
>> It never made any sense to me that there could be any problem
>> with the compiler automatically creating a temporary hidden
>> lvalue so a ref could be taken. If there IS any problem, I can
>> only imagine it would be symptomatic of a different, larger
>> problem.
>
> It can be a serious API problem if you can't look at ref and know that the
> intention is that the original argument's value will be used and then
> mutated (whereas with out, it will be mutated, but the original value won't
> be used).

Okay, let's read 'const ref' every time I say 'ref'. I thought that
would be fairly safe to assume. Sorry!


> However, that could be solved by having a different attribute indicate that
> the idea is that the compiler will accept rvalues and create temporaries for
> them if they're passed instead of an lvalue. Then, the situation is clear.

No, no attributes.. Just accept any argument to const-ref!
The function's not gonna change it.


> As for rvalue references in general, I can never remember what exactly the
> problem is. IIRC, part of it relates to the fact that it makes it impossible
> for the compiler to know whether it's dealing with an actual lvalue or not,
> which has serious @safety implications, and in the case of C++, complicates
> things considerably. Andrei has plenty of nasty things to say about how
> rvalue references in C++ were a huge mistake. It's just that I can never
> remember the arguments very well.
>
> The use of scope with DIP 1000 may reduce the problem such that scope ref
> could be made to work @safely (I don't know)

True, but if you follow that line of reasoning, then this case must
equally be banned:

T temp = f();
func(temp);

The @safety fear is that the pointer to the stack arg may escape, and
that's identical whether the argument is an explicit temp, or an
implicit one.


> but that by itself isn't
> enough if you want it to be clear whether a function is supposed to be
> mutating the argument

Functions that receive const args make it pretty clear that they don't
intend to mutate the arg.


> I get the impression that this issue is one where it seems like a small one
> on the surface but that when you get into the guts of what it actually means
> to implement it, it gets nasty.

It really doesn't... the compiler just make the same temp that I type
with my hands. That's literally all that it needs to do. I'd bet money
it's a one-paragraph change.


> Clearly, it's a PR issue in Manu's world, but it may also be a performance 
> problem. I
> don't know. Either way, it's clear that Manu and others like him think that
> the fact that D doesn't have rvalue references is a serious deficiency.

That's an understatement, but yes :P
We want to call C++ code. The D code shouldn't be objectively worse
than the C++ code we're trying to escape, otherwise we're undermining
our gravity towards D.
We can't have a situation that goes "I wrote this 3 line function in
D, but now it's 6 lines because I had to break args to temps, it's
objectively worse than the equivalent C++ function... why am I writing
it in D again?".

Re: rvalues -> ref (yup... again!)

[Manu via Digitalmars-d]

On 23 March 2018 at 16:58, Jonathan M Davis via Digitalmars-d
 wrote:
> On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote:
>> Well, to be honest I still can't understand why would you want to
>> pass a RValue as reference.
>
> Well, it's frequently the case that you don't want to copy an object if you
> don't have to - especially in the gaming world, where every ounce of
> performance matters. If a function accepts an argument by ref, then no copy
> is made, but then you have to pass an lvalue, making it really annoying to
> call the function when what you have is an rvalue. On the other hand, if the
> function doesn't accept its argument by ref, then you can pass an rvalue
> (and it will be moved, making that efficient), but then lvalues get copied
> when that's often not what you want. C++'s solution to this was rvalue
> references.

Ummm... rvalue-references are something completely different.
rval-ref's are C++'s solution to move semantics.
C++ just simply accepts rvalues passed to const& args. It makes a temp
and passes the ref, as you expect.


> That way, as long as the parameter is const, it can accept both
> rvalues and lvalues, and it won't copy unless it has to. The closest
> analogue that D has to this is auto ref, which requires templates, which may
> or may not be acceptable.

auto-ref == template function, which by definition is NOT an
extern(C++) function. auto-ref may also resolve to NOT a ref, which
means a move... data structures that are large (ie, a vector or
matrix) still have to copy a bunch of memory, even if the copy is
algorithmically 'cheap'.


> In many cases, it works great, whereas in others,
> it doesn't work at all (e.g. virtual functions), and it can result in
> template bloat.

And templates, that's another case where it fails.
auto-ref is something else unrelated to this topic, and it's useful in
a different set of cases for a different set of uses/reasons. It's got
nothing to do with this.


> The whole situation is complicated by the fact that sometimes it's actually
> faster to pass by value and copy the argument, even if it's an lvalue.

The api author wouldn't have made the arg a ref in that case.


> Passing by const& can often result in unnecessary copies if anywhere in the
> chain passes the object by value, whereas if it's passed by value, the same
> object can often be moved multiple times, avoiding any copies**.

**avoiding __calls to the copy constructor__. The memory is likely to
still be moved around a bunch of times.
The case you're thinking of is when values are *returned* by value, in
that case, copy elision is possible, and the result can be constructed
in place. That's a completely unrelated problem... you don't do
return-by-ref ;)


> It's my
> understanding that prior to C++11, it was considered best practice to pass
> by const& as much as possible to avoid copies but that after C++11 (which
> added move constructors), it's often considered better to pass by value,
> because then in many cases, the compiler can move the object instead of
> copying it**.

** Assuming the object is tiny, but has an expensive copy constructor.
In the case where an object is large (and has a primitive, or no copy
constructor) it doesn't change the situation; you still wanna pass a
big thing by ref in all cases; ie, vector/matrix.


> So, C++ gives you control over which you do, and it's not
> necessarily straightforward as to which you should use (though plenty of
> older C++ programmers likely just use const& all over the place out of
> habit).

D gives you the same set of options; except that passing args by ref
is a PITA in D, and ruins your code.

Re: rvalues -> ref (yup... again!)

[Jonathan M Davis via Digitalmars-d]

On Friday, March 23, 2018 23:35:29 MattCoder via Digitalmars-d wrote:
> Well, to be honest I still can't understand why would you want to
> pass a RValue as reference.

Well, it's frequently the case that you don't want to copy an object if you
don't have to - especially in the gaming world, where every ounce of
performance matters. If a function accepts an argument by ref, then no copy
is made, but then you have to pass an lvalue, making it really annoying to
call the function when what you have is an rvalue. On the other hand, if the
function doesn't accept its argument by ref, then you can pass an rvalue
(and it will be moved, making that efficient), but then lvalues get copied
when that's often not what you want. C++'s solution to this was rvalue
references. That way, as long as the parameter is const, it can accept both
rvalues and lvalues, and it won't copy unless it has to. The closest
analogue that D has to this is auto ref, which requires templates, which may
or may not be acceptable. In many cases, it works great, whereas in others,
it doesn't work at all (e.g. virtual functions), and it can result in
template bloat.

The whole situation is complicated by the fact that sometimes it's actually
faster to pass by value and copy the argument, even if it's an lvalue.
Passing by const& can often result in unnecessary copies if anywhere in the
chain passes the object by value, whereas if it's passed by value, the same
object can often be moved multiple times, avoiding any copies. It's my
understanding that prior to C++11, it was considered best practice to pass
by const& as much as possible to avoid copies but that after C++11 (which
added move constructors), it's often considered better to pass by value,
because then in many cases, the compiler can move the object instead of
copying it. So, C++ gives you control over which you do, and it's not
necessarily straightforward as to which you should use (though plenty of
older C++ programmers likely just use const& all over the place out of
habit).

D supports moving out of the box without move constructors, so it does a
good job of handling the cases where a move is most appropriate, but it
doesn't have rvalue references, so it doesn't have the same flexibility as
C++ when you want to pass something by reference. So, anyone looking to pass
stuff by reference all over the place (like Manu and his coworkers) is going
to find the lack of rvalue references in D to be really annoying.

- Jonathan M Davis

Re: rvalues -> ref (yup... again!)

[Jonathan M Davis via Digitalmars-d]

On Friday, March 23, 2018 22:44:35 Nick Sabalausky via Digitalmars-d wrote:
> It never made any sense to me that there could be any problem
> with the compiler automatically creating a temporary hidden
> lvalue so a ref could be taken. If there IS any problem, I can
> only imagine it would be symptomatic of a different, larger
> problem.

It can be a serious API problem if you can't look at ref and know that the
intention is that the original argument's value will be used and then
mutated (whereas with out, it will be mutated, but the original value won't
be used). So, having ref in general accept rvalues would be a huge problem.
However, that could be solved by having a different attribute indicate that
the idea is that the compiler will accept rvalues and create temporaries for
them if they're passed instead of an lvalue. Then, the situation is clear.

As for rvalue references in general, I can never remember what exactly the
problem is. IIRC, part of it relates to the fact that it makes it impossible
for the compiler to know whether it's dealing with an actual lvalue or not,
which has serious @safety implications, and in the case of C++, complicates
things considerably. Andrei has plenty of nasty things to say about how
rvalue references in C++ were a huge mistake. It's just that I can never
remember the arguments very well.

The use of scope with DIP 1000 may reduce the problem such that scope ref
could be made to work @safely (I don't know), but that by itself isn't
enough if you want it to be clear whether a function is supposed to be
mutating the argument or if it's just trying to avoid copying it. C++ solves
that problem by requiring const with rvalue references, but given how D's
const works, that really wouldn't make sense. So, we'd have to do something
else.

I get the impression that this issue is one where it seems like a small one
on the surface but that when you get into the guts of what it actually means
to implement it, it gets nasty.

I think that most of us have just take the approach of passing everything by
value unless the type is clearly too expensive to copy for that to make
sense, in which case, it's then passed by ref or auto ref, or we make it a
reference type. I don't know how much the problem with the lack of rvalue
references in D is a PR issue, because C++ programmers get annoyed about it
and deem D to be subpar as a result, and how much it's an actual performance
problem. I don't work in Manu's world, so I don't what really makes sense
there. For me, passing by value is usually a non-issue, and it's easy enough
to work around the problem in the rare cases where it is a problem. Clearly,
it's a PR issue in Manu's world, but it may also be a performance problem. I
don't know. Either way, it's clear that Manu and others like him think that
the fact that D doesn't have rvalue references is a serious deficiency.

- Jonathan M Davis