[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=8845

Walter Bright  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 CC||bugzi...@digitalmars.com
 Resolution|--- |FIXED

--- Comment #22 from Walter Bright  ---
Resolved by https://github.com/dlang/dmd/pull/9817

--

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=8845

Carsten Blüggel  changed:

   What|Removed |Added

 CC||chi...@posteo.net

--

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=8845

--- Comment #21 from Nick Treleaven  ---
I came up with a lvalueOf!rvalue template which hopefully is memory-safe. Not
tested with Walter's DIP1000 scope branch (yet).

enum typeof(v[0])[v.length] staticArray(v...) = [v];

ref const(T) lvalueOf(alias rvalue, T=typeof(rvalue))
(return ref T rv = *staticArray!rvalue.ptr) @trusted
{
return rv;
}

auto foo(ref const int i) {assert(i == -1);}

@safe @nogc unittest
{
static assert(!__traits(compiles, foo(0-1)));
foo(lvalueOf!(0-1));
}

--

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=8845

Nick Treleaven  changed:

   What|Removed |Added

 CC||ntrel-...@mybtinternet.com
   See Also||https://issues.dlang.org/sh
   ||ow_bug.cgi?id=9238

--- Comment #20 from Nick Treleaven  ---
(In reply to Manu from comment #0)
> I've heard arguments about the safety of the operation, but the 'workaround'
> is just to create a temporary, which has identical security properties.
> ...
> Perhaps this behaviour could be restricted to ref const, or ref in, if we're
> getting worried about the safety of the operation? That would perhaps even
> improve on how behaves now.

I think rvalues could be passed to const ref parameters so long as that
parameter is not return ref. The reason is in case we decide to support local
refs initialized from a return ref function.

--

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845



--- Comment #3 from Manu turkey...@gmail.com 2013-11-27 02:25:05 PST ---
(In reply to comment #2)
 Is there a better version of this with all the auto-ref stuff?

What do you mean?
The solution that we discussed extensively at dconf never made an appearance.
This is still precisely where it's always been as far as I know.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845



--- Comment #4 from yebblies yebbl...@gmail.com 2013-11-27 21:33:35 EST ---
(In reply to comment #3)
 (In reply to comment #2)
  Is there a better version of this with all the auto-ref stuff?
 
 What do you mean?
 The solution that we discussed extensively at dconf never made an appearance.
 This is still precisely where it's always been as far as I know.

I was hoping there was a less vague report somewhere with all the dconf stuff
in it.  On a related note, do you know if there's a final-by-default report out
there somewhere?

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845



--- Comment #5 from Manu turkey...@gmail.com 2013-11-27 02:39:31 PST ---
(In reply to comment #4)
 (In reply to comment #3)
  (In reply to comment #2)
   Is there a better version of this with all the auto-ref stuff?
  
  What do you mean?
  The solution that we discussed extensively at dconf never made an 
  appearance.
  This is still precisely where it's always been as far as I know.
 
 I was hoping there was a less vague report somewhere with all the dconf stuff
 in it.  On a related note, do you know if there's a final-by-default report 
 out
 there somewhere?

I don't think there is. That would have been far too sensible :)

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845



--- Comment #6 from yebblies yebbl...@gmail.com 2013-11-27 22:14:24 EST ---
(In reply to comment #5)
 I don't think there is. That would have been far too sensible :)

Well, if you can remember where the discussion and agreement was, it might be a
good idea to make one and link it.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845



--- Comment #8 from Manu turkey...@gmail.com 2013-11-27 04:02:46 PST ---
(In reply to comment #6)
 (In reply to comment #5)
  I don't think there is. That would have been far too sensible :)
 
 Well, if you can remember where the discussion and agreement was, it might be 
 a
 good idea to make one and link it.

IIRC, it was actually you that proposed the winning deprecation path.
It was comparable to the introduction of mandatory override. I think you
suggested one version introduces a deprecation warning where override is stated
but virtual is not present on the base, then the next version enforces it.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845



--- Comment #9 from bearophile_h...@eml.cc 2013-11-27 04:14:34 PST ---
(In reply to comment #7)

 AFAIK Walter agreed for final-by-default:
 
 http://forum.dlang.org/thread/yzsqwejxqlnzryhrk...@forum.dlang.org?page=28#post-koqkhc:244nn:241:40digitalmars.com

Is that design decision written in some page of the D wiki?
Do you need to add a virtual keyword to D?

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845



--- Comment #10 from Andrej Mitrovic andrej.mitrov...@gmail.com 2013-11-27 
04:15:56 PST ---
(In reply to comment #9)
 (In reply to comment #7)
 
  AFAIK Walter agreed for final-by-default:
  
  http://forum.dlang.org/thread/yzsqwejxqlnzryhrk...@forum.dlang.org?page=28#post-koqkhc:244nn:241:40digitalmars.com
 
 Is that design decision written in some page of the D wiki?
 Do you need to add a virtual keyword to D?

I don't know the details. I guess someone should make a DIP and mark it as
approved.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845



--- Comment #11 from Manu turkey...@gmail.com 2013-11-27 04:26:49 PST ---
(In reply to comment #9)
 (In reply to comment #7)
 
  AFAIK Walter agreed for final-by-default:
  
  http://forum.dlang.org/thread/yzsqwejxqlnzryhrk...@forum.dlang.org?page=28#post-koqkhc:244nn:241:40digitalmars.com
 
 Is that design decision written in some page of the D wiki?
 Do you need to add a virtual keyword to D?

Not sure. Don't think so. I think most of the conversation is in that thread
(and prior ones like it), and verbally at dconf... :/

Yes, a virtual keyword would be required. override would cause a warning at
first if used on a function that wasn't either override or virtual, and then it
would become an error. final would be used to seal a hierarchy, as is useful in
java, C++ and 'sealed' in C#.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845



--- Comment #12 from yebblies yebbl...@gmail.com 2013-11-27 23:47:39 EST ---
(In reply to comment #8)
 (In reply to comment #6)
  (In reply to comment #5)
   I don't think there is. That would have been far too sensible :)
  
  Well, if you can remember where the discussion and agreement was, it might 
  be a
  good idea to make one and link it.
 
 IIRC, it was actually you that proposed the winning deprecation path.
 It was comparable to the introduction of mandatory override. I think you
 suggested one version introduces a deprecation warning where override is 
 stated
 but virtual is not present on the base, then the next version enforces it.

I eventually realized that override implies virtual, and therefore only the
introducing methods need to be modified.  Up until that point I was pretty
skeptical.

I've created an enhancement for it: issue 11616
Feel free to CC yourselves if you're interested.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845



--- Comment #13 from Manu turkey...@gmail.com 2013-11-27 04:53:07 PST ---
(In reply to comment #12)
 (In reply to comment #8)
  (In reply to comment #6)
   (In reply to comment #5)
I don't think there is. That would have been far too sensible :)
   
   Well, if you can remember where the discussion and agreement was, it 
   might be a
   good idea to make one and link it.
  
  IIRC, it was actually you that proposed the winning deprecation path.
  It was comparable to the introduction of mandatory override. I think you
  suggested one version introduces a deprecation warning where override is 
  stated
  but virtual is not present on the base, then the next version enforces it.
 
 I eventually realized that override implies virtual, and therefore only the
 introducing methods need to be modified.  Up until that point I was pretty
 skeptical.

Yes, it's a much less damaging change than the introduction of 'override',
since it only affects the base, rather than all branches/leaves as override
did. And now that we have override, introduction of virtual is perfectly safe,
we can properly produce accurate warnings/errors where it's missing.

 I've created an enhancement for it: issue 11616
 Feel free to CC yourselves if you're interested.

I'm just creating a DIP for it... should I bother?

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845



--- Comment #14 from yebblies yebbl...@gmail.com 2013-11-27 23:56:31 EST ---
(In reply to comment #13)
 
 I'm just creating a DIP for it... should I bother?

If you want to.  I don't think it is necessary though.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845



--- Comment #15 from Manu turkey...@gmail.com 2013-11-27 05:04:31 PST ---
(In reply to comment #14)
 (In reply to comment #13)
  
  I'm just creating a DIP for it... should I bother?
 
 If you want to.  I don't think it is necessary though.

Well if there's no need, I won't.
I'm just trying to remember if there were any fine-details that we covered that
are worth noting on there.

The only detail I remember debate over, was the handling of 'final' for sealing
hierarchies; ie, do you need 'override final', or is 'final' enough when
specified on a method whose base is virtual? I think 'final' is enough, but it
was an argument of clarity.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845


Maxim Fomin ma...@maxim-fomin.ru changed:

   What|Removed |Added

 CC||ma...@maxim-fomin.ru


--- Comment #16 from Maxim Fomin ma...@maxim-fomin.ru 2013-11-27 05:09:21 PST 
---
Addressing Manu problem in first post: you cannot do it directly, but can
workaround by 

void foo(T)(ref const T i) {}

ref lvalueOf(T)(T expression)
{
   return [expression][0];
}

struct S(T)
{
   T t;
   ref get() { return t; }
   //alias get this;
}

void main()
{
   //foo(0-0);
   foo((0-0).lvalueOf);
   foo(S!int(0-0).get);
}

Second one is ugly but does not consume heap memory as first one, but it also
requires that foo() does not save reference somewhere (operation is safe
because S!int(0-0) is allocated within main scope, so passing pointer is fine,
until it is saved).

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845



--- Comment #17 from Manu turkey...@gmail.com 2013-11-27 07:18:38 PST ---
(In reply to comment #16)
 Addressing Manu problem in first post: you cannot do it directly, but can
 workaround by 
 
 void foo(T)(ref const T i) {}
 
 ref lvalueOf(T)(T expression)
 {
return [expression][0];
 }
 
 struct S(T)
 {
T t;
ref get() { return t; }
//alias get this;
 }
 
 void main()
 {
//foo(0-0);
foo((0-0).lvalueOf);
foo(S!int(0-0).get);
 }
 
 Second one is ugly but does not consume heap memory as first one, but it also
 requires that foo() does not save reference somewhere (operation is safe
 because S!int(0-0) is allocated within main scope, so passing pointer is fine,
 until it is saved).

One allocates, the other is just a subversion of the type system. It's
basically the same (and less clear?) than just doing:
  int temp = 0-0;
  foo(temp);

Either way, it's equally unsafe, and the discussion regarding rvalue-ref
parameters was mainly around safety concerns.

I was in the camp arguing for a 'scope ref' concept, which would rely on escape
analysis to enforce that a 'scope ref' doesn't escape, but we were never going
to win that argument.

The plan that Walter was most enthusiastic about was, if the function is passed
an rvalue, generate an implicit temp in the caller's stack and pass it through.
If the function being called returns a ref, it would add an implicit function
exit condition which would validate that the ref being returned is not within
the function's stack (controlled by -noboundscheck for optimisation).
This way, it would be safe to return a ref that was passed in to a function,
but not a function local. This would cascade outwards.

Frankly, I just want a solution. I like the 'scope ref' concept, but it depends
on escape analysis working well, and I'm not that precious about
implementation, I just want something that works. It's a major inhibitor to
maths intensive code, like image/geometry/physics/audio processing.
Particularly where fairly 'primitive' objects like vectors, matrices,
quaternions, colours, etc are concerned.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845



--- Comment #18 from Maxim Fomin ma...@maxim-fomin.ru 2013-11-27 07:45:10 PST 
---
(In reply to comment #17)

 One allocates, the other is just a subversion of the type system. 

Actually each of them allocates. You either allocate on stack or in heap.
Advantage of heap is that you don't care about escaping references and
advantage of stack is efficiency. Second example is obviously not a subversion
of the type system. 

 It's
 basically the same (and less clear?) than just doing:
   int temp = 0-0;
   foo(temp);
 

No, it is not the same. By the way, it was you who complained about writing
code without dummy variables. 

 Either way, it's equally unsafe, and the discussion regarding rvalue-ref
 parameters was mainly around safety concerns.
 

No, first is safe, second is conditionally safe if reference does not escape.
By the way, you wrote that creating temporary and passing pointer is acceptable
(either you do this or language automatically does).

 If the function being called returns a ref, it would add an implicit function
 exit condition which would validate that the ref being returned is not within
 the function's stack (controlled by -noboundscheck for optimisation).
 This way, it would be safe to return a ref that was passed in to a function,
 but not a function local. This would cascade outwards.
 

The idea has flaw, it does not take into account that there are ways for stack
pointer to escape other than through return. 

 Frankly, I just want a solution. I like the 'scope ref' concept, but it 
 depends
 on escape analysis working well, and I'm not that precious about
 implementation, I just want something that works. It's a major inhibitor to
 maths intensive code, like image/geometry/physics/audio processing.
 Particularly where fairly 'primitive' objects like vectors, matrices,
 quaternions, colours, etc are concerned.

Frankly, there is excess of people wanting a solution and shortage of people
executing them.

I think you need to decide what exactly do you want: allocating temporary by
language (which is unlikely to happen), using workaround with the purpose not
to have dummy variables (seems to be no), writing DIP for the issue (which
having been written and discussed goes to archive), or somebody to fix scope
ref (which is unlikely to happen soon).

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845


Zach the Mystic reachz...@gmail.com changed:

   What|Removed |Added

 CC||reachz...@gmail.com


--- Comment #19 from Zach the Mystic reachz...@gmail.com 2013-11-27 11:12:32 
PST ---
(In reply to comment #17)
 I was in the camp arguing for a 'scope ref' concept, which would rely on 
 escape
 analysis to enforce that a 'scope ref' doesn't escape, but we were never going
 to win that argument.
 
 The plan that Walter was most enthusiastic about was, if the function is 
 passed
 an rvalue, generate an implicit temp in the caller's stack and pass it 
 through.
 If the function being called returns a ref, it would add an implicit function
 exit condition which would validate that the ref being returned is not within
 the function's stack (controlled by -noboundscheck for optimisation).
 This way, it would be safe to return a ref that was passed in to a function,
 but not a function local. This would cascade outwards.

Related ref safety Issue 9537.

I don't think you need to win the 'scope ref' issue because the -noboundscheck
version will work fine. Then temporaries will be safe, so long as the called
functions don't save their addresses off of the stack somewhere.

To ensure safety, I think the compiler needs to keep track of some bits on its
parameters and expressions. Some ideas:

Each reference variable has a 'mayBeLocal' bit and a 'mayBeRefParam' bit. Each
function has a 'mayReturnRefParam' bit. All known references to locals have
'mayBeLocal' set, and all ref parameters have 'mayBeRefParam' set. Any attempt
to return a reference to a 'mayBeLocal' causes the appropriate error. Returning
a 'mayBeRefParam' reference causes the function to be marked
'mayReturnRefParam', and the result will be the safest of what was passed in.
Any time the compiler can't prove that a reference parameter will not be
returned, it marks the function as 'mayReturnRefParam' and requires the runtime
stack bounds check. Now ref returns are safe.

The above scheme is blunt. It will unnecessarily restrict a few cases where
more than one ref parameter is passed, but only one is returned:

ref T fun(ref T a, ref T b) {
  return a;
}

ref T gun(ref T a) {
  T b;
  return fun(a, b); // fails unnecessarily, because b is local
}

If there were a lot of such cases in practice, some addition to this simple
scheme may be necessary.

Other kinds of unsafe escapes would have to be handled in other ways (e.g. a
bit 'mayGetSavedOffStack' on parameters). But the main case of ref returns is
handled at least. Then rvalue refs would be safe.

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

https://d.puremagic.com/issues/show_bug.cgi?id=8845


yebblies yebbl...@gmail.com changed:

   What|Removed |Added

 CC||yebbl...@gmail.com


--- Comment #2 from yebblies yebbl...@gmail.com 2013-11-24 17:59:58 EST ---
Is there a better version of this with all the auto-ref stuff?

-- 
Configure issuemail: https://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---

[Issue 8845] Can't pass immediate or rvalue args to ref function parameters

[d-bugmail]

http://d.puremagic.com/issues/show_bug.cgi?id=8845


bearophile_h...@eml.cc changed:

   What|Removed |Added

 CC||bearophile_h...@eml.cc


--- Comment #1 from bearophile_h...@eml.cc 2012-10-17 16:50:18 PDT ---
(In reply to comment #0)

 This is particularly common when working with linear algebra. Vectors,
 matrices, quaternions are surely the most likely to produce this pattern.

Yes, it's a common pattern (Example: I have seen it often in a little
ray-tracer).

-- 
Configure issuemail: http://d.puremagic.com/issues/userprefs.cgi?tab=email
--- You are receiving this mail because: ---