Re: DIP 1014--Hooking D's struct move semantics--Has Been Accepted

[12345swordy via Digitalmars-d-announce]

On Thursday, 8 November 2018 at 03:04:06 UTC, Jonathan M Davis 
wrote:
On Wednesday, November 7, 2018 6:54:54 PM MST Mike Parker via 
Digitalmars-d- announce wrote:
I'm happy to announce that Walter and Andrei have rendered 
their verdict on DIP 1014. They were in agreement on two 
points: they don't like it, but they know we need it. Given 
that there are no other alternative proposals and that they 
could see no alternative themselves, they decided to accept 
this DIP without modification.


I think that that probably sums the situation up pretty nicely. 
The fact that we need something like this is just plain ugly, 
and if it starts getting used frequently, there's definitely a 
problem, but there are use cases, where it's going to be 
invaluable.


- Jonathan M Davis


This pull request is undoubtedly a major factor of Walter and 
Andrei approval of DIP 1014

https://github.com/dlang/druntime/pull/2310

Alex

Re: DIP 1014--Hooking D's struct move semantics--Has Been Accepted

[Per Nordlöw via Digitalmars-d-announce]

On Thursday, 8 November 2018 at 02:36:21 UTC, test wrote:

Thanks to all for the hard work.

Look forward DIP1016 to be approved.


Agreed!

Re: DIP 1014--Hooking D's struct move semantics--Has Been Accepted

[Jonathan M Davis via Digitalmars-d-announce]

On Wednesday, November 7, 2018 6:54:54 PM MST Mike Parker via Digitalmars-d-
announce wrote:
> I'm happy to announce that Walter and Andrei have rendered their
> verdict on DIP 1014. They were in agreement on two points: they
> don't like it, but they know we need it. Given that there are no
> other alternative proposals and that they could see no
> alternative themselves, they decided to accept this DIP without
> modification.

I think that that probably sums the situation up pretty nicely. The fact
that we need something like this is just plain ugly, and if it starts
getting used frequently, there's definitely a problem, but there are use
cases, where it's going to be invaluable.

- Jonathan M Davis

Re: DIP 1014--Hooking D's struct move semantics--Has Been Accepted

[test via Digitalmars-d-announce]

On Thursday, 8 November 2018 at 01:54:54 UTC, Mike Parker wrote:
I'm happy to announce that Walter and Andrei have rendered 
their verdict on DIP 1014. They were in agreement on two 
points: they don't like it, but they know we need it. Given 
that there are no other alternative proposals and that they 
could see no alternative themselves, they decided to accept 
this DIP without modification.


Thanks to Shachar Shemesh for pushing the DIP through to the 
end of the process and for his patience throughout. And 
congratulations on obtaining a successful result.




Thanks to all for the hard work.

Look forward DIP1016 to be approved.

Re: DIP 1014

[Stanislav Blinov via Digitalmars-d]

On Thursday, 4 October 2018 at 12:08:38 UTC, Shachar Shemesh 
wrote:


Two distinct things. Kinke was talking about how to pass a 
struct through the ABI. You are talking about special-casing a 
specific name.


Not just name, but argument passing as well.

Not to mention, your special case is to transform it to 
something you can *already* specify in the language. Why?


Because that syntax pertains specifically to construction, which 
is what a compiler move is; is not currently used by the language 
(the fact that the compiler doesn't error on it is an oversight); 
enforces calling convention.


Which is, however, not a reason to formalize it and make it a 
requirement for an isolated specific case, such as this one, 
utilizing a syntax that is currently not used by the language.


There is positively nothing in DIP 1014 that is "syntax not 
used by the language". Quite the contrary.


Which is what I said in the very next sentence, so I'm not sure 
what your point is here. It's like we're having a discussion but 
we aren't at the same time.


As opposed to trying to fit existing language semantics to 
something that the language didn't seem to want to allow in 
the first place.


Formalize it as a suggestion, and we can discuss the "as 
opposed to".


Alright, let's get back to it after the weekend then.

Like I said, I think there's a lot you're glossing over here 
(such as backwards compatibility).


Backwards compatibility? With what, exactly? Non-existing 
explicit moves?

Re: DIP 1014

[Shachar Shemesh via Digitalmars-d]

On 04/10/18 13:43, Stanislav Blinov wrote:

* move the data as part of the call hook rather than before
* Use a different name and signature on the hook function


Yes, exactly.





It would have to be special if you don't want to leave room for the 
compiler implementors.


That's not how standards work. If you don't want compiler implementors 
to have a choice in the matter, you put MUST in the specs. Doing 
anything else is, by and large, considered harmful.


The calling convention for particular types (i.e. 
those that do have a move hook defined) would have to be enforced in 
some way. See the neighbor thread wrt move semantics by kinke.


Two distinct things. Kinke was talking about how to pass a struct 
through the ABI. You are talking about special-casing a specific name.


Not to mention, your special case is to transform it to something you 
can *already* specify in the language. Why?


Which is, however, not a reason to formalize it and make it a 
requirement for an isolated specific case, such as this one, utilizing a 
syntax that is currently not used by the language.


There is positively nothing in DIP 1014 that is "syntax not used by the 
language". Quite the contrary.


As opposed to trying 
to fit existing language semantics to something that the language didn't 
seem to want to allow in the first place.


Formalize it as a suggestion, and we can discuss the "as opposed to". 
Like I said, I think there's a lot you're glossing over here (such as 
backwards compatibility).


Shachar

Re: DIP 1014

[Stanislav Blinov via Digitalmars-d]

On Thursday, 4 October 2018 at 08:32:44 UTC, Shachar Shemesh 
wrote:

On 04/10/18 11:16, Paolo Invernizzi wrote:
While I want to thank you both, about the quality of this 
thread, what kind of "consequences that go beyond what I think 
you understand" are you thinking of? Can you give an example?


Assuming I understand Stanislav's proposal correctly (an


Yes it seems you do.

assumption I'm reluctant to make, hence my request for 
something more formal), it boils down to two points:


* move the data as part of the call hook rather than before
* Use a different name and signature on the hook function


Yes, exactly.

The first one we can argue for or against. My original proposal 
was phrased the way it was precisely because that's the way 
copying works in D (copy first, patch the data later). About a 
week after I submitted it, Andrei came forward with requesting 
to move to copy constructors.


The second, to me, is a non-starter. The only way you'd get a 
function who's signature is:

void someName(Type rhs);
But which actually maintains rhs's address from before the call 
is if the compiler treats "someName" as a special case, and 
emits code which would normally be emitted for the function:

void someName(ref Type rhs);


It would have to be special if you don't want to leave room for 
the compiler implementors. The calling convention for particular 
types (i.e. those that do have a move hook defined) would have to 
be enforced in some way. See the neighbor thread wrt move 
semantics by kinke.


That's why it was important for me to clear up whether there is 
*ever* a case in the current language where that happens 
(answer: only by accident, which is the same as saying "no").


Which is, however, not a reason to formalize it and make it a 
requirement for an isolated specific case, such as this one, 
utilizing a syntax that is currently not used by the language. As 
opposed to trying to fit existing language semantics to something 
that the language didn't seem to want to allow in the first place.


So to get that to work, you'd need to insert a special case 
into the ABI of the language: if the function's name is 
someName, treat it differently.


Yes, that's what I'm talking about.

At this point, you might as well call a spade a spade, and just 
give the function that signature explicitly. 
s/someName/opPostMove/, and you get DIP 1014 (as far as point 
#2 is concerned).

Re: DIP 1014

[Shachar Shemesh via Digitalmars-d]

On 04/10/18 11:16, Paolo Invernizzi wrote:
While I want to thank you both, about the quality of this thread, what 
kind of "consequences that go beyond what I think you understand" are 
you thinking of? Can you give an example?


Assuming I understand Stanislav's proposal correctly (an assumption I'm 
reluctant to make, hence my request for something more formal), it boils 
down to two points:


* move the data as part of the call hook rather than before
* Use a different name and signature on the hook function

The first one we can argue for or against. My original proposal was 
phrased the way it was precisely because that's the way copying works in 
D (copy first, patch the data later). About a week after I submitted it, 
Andrei came forward with requesting to move to copy constructors.


The second, to me, is a non-starter. The only way you'd get a function 
who's signature is:

void someName(Type rhs);

But which actually maintains rhs's address from before the call is if 
the compiler treats "someName" as a special case, and emits code which 
would normally be emitted for the function:


void someName(ref Type rhs);

That's why it was important for me to clear up whether there is *ever* a 
case in the current language where that happens (answer: only by 
accident, which is the same as saying "no").


So to get that to work, you'd need to insert a special case into the ABI 
of the language: if the function's name is someName, treat it differently.


At this point, you might as well call a spade a spade, and just give the 
function that signature explicitly. s/someName/opPostMove/, and you get 
DIP 1014 (as far as point #2 is concerned).


Shachar

Re: DIP 1014

[Paolo Invernizzi via Digitalmars-d]

On Thursday, 4 October 2018 at 08:10:31 UTC, Shachar Shemesh 
wrote:

On 04/10/18 11:05, Stanislav Blinov wrote:
On Thursday, 4 October 2018 at 03:06:35 UTC, Shachar Shemesh 
wrote:



[...]


For the love of Pete, that program was an example of how a 
move hook should work, *not* a demonstration of achieving the 
DIP behavior without changing the language. I know the example 
is brittle and have said as much.


The example isn't brittle. It is simply not an example.

If you want to leave it out, however, then I think you should 
submit an orderly proposal. The changes you seem to be 
suggesting have consequences that go beyond what I think you 
understand, and there can be no serious discussion of it while 
it is not clear from your posts which part of what you say is 
the relevant one.


Shachar


While I want to thank you both, about the quality of this thread, 
what kind of "consequences that go beyond what I think you 
understand" are you thinking of? Can you give an example?


Thanks,
Paolo

Re: DIP 1014

[Shachar Shemesh via Digitalmars-d]

On 04/10/18 11:05, Stanislav Blinov wrote:

On Thursday, 4 October 2018 at 03:06:35 UTC, Shachar Shemesh wrote:

If you do *anything* to that program, and that includes even changing 
its compilation flags (try enabling inlining), it will stop working.


You should have known that when you found out it doesn't work on ldc: 
ldc and dmd use the same front-end. If you think something works 
fundamentally different between the two, you are probably wrong.


For the love of Pete, that program was an example of how a move hook 
should work, *not* a demonstration of achieving the DIP behavior without 
changing the language. I know the example is brittle and have said as much.


The example isn't brittle. It is simply not an example.

If you want to leave it out, however, then I think you should submit an 
orderly proposal. The changes you seem to be suggesting have 
consequences that go beyond what I think you understand, and there can 
be no serious discussion of it while it is not clear from your posts 
which part of what you say is the relevant one.


Shachar

Re: DIP 1014

[Stanislav Blinov via Digitalmars-d]

On Thursday, 4 October 2018 at 03:06:35 UTC, Shachar Shemesh 
wrote:


If you do *anything* to that program, and that includes even 
changing its compilation flags (try enabling inlining), it will 
stop working.


You should have known that when you found out it doesn't work 
on ldc: ldc and dmd use the same front-end. If you think 
something works fundamentally different between the two, you 
are probably wrong.


For the love of Pete, that program was an example of how a move 
hook should work, *not* a demonstration of achieving the DIP 
behavior without changing the language. I know the example is 
brittle and have said as much.

Re: DIP 1014

[Manu via Digitalmars-d]

On Wed, Oct 3, 2018 at 11:00 PM Timothee Cour via Digitalmars-d
 wrote:
>
> @Manu, @Jonathan M Davis
>
> > GNU's std::string implementation stores an interior pointer! >_<
>
> it's not just GNU's std::string ; it can crop up in other places, see
> https://github.com/Syniurge/Calypso/issues/70 in opencv (cv:: MatStep)

Sure. Certainly, it shows up in C++ fairly often... but I'm working on
the STL containers, and I didn't think there were any implementations
that did that (because inefficient use of space), but turns out the
GNU implementation bent me over, at least as far as I've encountered
yet.
It's kinda got me stuck.

Re: DIP 1014

[Timothee Cour via Digitalmars-d]

@Manu, @Jonathan M Davis

> GNU's std::string implementation stores an interior pointer! >_<

it's not just GNU's std::string ; it can crop up in other places, see
https://github.com/Syniurge/Calypso/issues/70 in opencv (cv:: MatStep)


On Wed, Oct 3, 2018 at 8:10 PM Shachar Shemesh via Digitalmars-d
 wrote:
>
> On 03/10/18 23:25, Stanislav Blinov wrote:
> > It *is* true when the type doesn't have a destructor. Extending that to
> > a move hook, it will also be true because destruction will be elided.
> > I know what you're talking about, that happens for types that have
> > destructors.
>
> No, destructors have nothing to do with it, as well they shouldn't. The
> whole point of D moving structs around is that no destruction is needed.
> It took me a while to figure out why your program does appear to work.
> At first I thought it was because of inlining, but that was wrong.
>
> The reason your test case works (sometimes, if you don't breath on it
> too heavily) is because the object is actually moved twice. Once when
> returning from the function into the variable, and another when copied
> into opAssign's argument. This results in it returning to its original
> address.
>
> If you do *anything* to that program, and that includes even changing
> its compilation flags (try enabling inlining), it will stop working.
>
> You should have known that when you found out it doesn't work on ldc:
> ldc and dmd use the same front-end. If you think something works
> fundamentally different between the two, you are probably wrong.
>
> To verify my guess is right, I tried the following change: add to
> createCounter and createCounterNoNRV in your original program (no
> destructors) the following two lines:
>int a;
>write(a);
>
> You have added another local variable to the functions, but otherwise
> changed absolutely nothing. You will notice your program now has an offset.
>
> Shachar

Re: DIP 1014

[Shachar Shemesh via Digitalmars-d]

On 03/10/18 23:25, Stanislav Blinov wrote:
It *is* true when the type doesn't have a destructor. Extending that to 
a move hook, it will also be true because destruction will be elided.
I know what you're talking about, that happens for types that have 
destructors.


No, destructors have nothing to do with it, as well they shouldn't. The 
whole point of D moving structs around is that no destruction is needed. 
It took me a while to figure out why your program does appear to work. 
At first I thought it was because of inlining, but that was wrong.


The reason your test case works (sometimes, if you don't breath on it 
too heavily) is because the object is actually moved twice. Once when 
returning from the function into the variable, and another when copied 
into opAssign's argument. This results in it returning to its original 
address.


If you do *anything* to that program, and that includes even changing 
its compilation flags (try enabling inlining), it will stop working.


You should have known that when you found out it doesn't work on ldc: 
ldc and dmd use the same front-end. If you think something works 
fundamentally different between the two, you are probably wrong.


To verify my guess is right, I tried the following change: add to 
createCounter and createCounterNoNRV in your original program (no 
destructors) the following two lines:

  int a;
  write(a);

You have added another local variable to the functions, but otherwise 
changed absolutely nothing. You will notice your program now has an offset.


Shachar

Re: DIP 1014

[Stanislav Blinov via Digitalmars-d]

On Wednesday, 3 October 2018 at 18:58:37 UTC, Shachar Shemesh 
wrote:



On 03/10/18 20:43, Stanislav Blinov wrote:
On Wednesday, 3 October 2018 at 15:33:00 UTC, Shachar Shemesh 
wrote:
I.e. - I am asserting if a move was not caught. The program 
fails to run on either ldc or dmd. To me, this makes perfect 
sense as for the way D is built. In essence, opAssign isn't 
guaranteed to run. Feel free to build a struct where that 
assert passes to convince me.


That's a slightly different issue here.


Well, I view this issue as a deal breaker. If you need to move 
the object *in order* to pass it to your move hook, then 
anything that requires knowing the address of the old instance 
will, by definition, not work.


I feel like we're still not on the same page here. 
this(typeof(this)) doesn't work like a move hook in D right now. 
I'm *suggesting* making that be a move ctor, instead of 
opPostMove from your DIP (see below).


Look at the output. The operator is being run, it can't *not* 
run,


Sure it can. Just look at the example I posted on the other 
thread 
(https://forum.dlang.org/post/pp2v16$1014$1...@digitalmars.com). 
The hook you mention is downright @disabled there.


In fact, had that not been the case, this DIP would never have 
happened.


Yup, we're definitely not on the same page :) That's not what I'm 
talking about at all.



Here is the flaw in your logic:

    void opAssign(Tracker rhs)

rhs is passed by value. This means that already at the point 
opAssign is called, rhs *already* has a different address 
than the one it was passed in with.


Currently that is only true if you define a destructor.


No, that's not true. Try printing the instance's address in the 
constructor and again in your operator.


It *is* true when the type doesn't have a destructor. Extending 
that to a move hook, it will also be true because destruction 
will be elided.
I know what you're talking about, that happens for types that 
have destructors.


In the presence of a move hook, 'rhs' would first have to pass 
through that hook, which will not take destructors into 
account at all.


I'm sorry, I'm not following. What is the difference between 
what you're proposing and opPostMove as defined?


1. with this(typeof(this)) the type of argument would never 
change. With opPostMove, it may. Remember that 'is(Tracker == 
const(Tracker))' is false.

2. you won't have to always move all the bits unconditionally.
3. symmetry with this(this)


This is the run I got:
$ ./movetest2
Address of temporary is 'b382e390', counter points to 'b382e390'
... which is '0' bytes from the address of temporary.


...I'm not sure what I should have seen, or what I should have 
concluded from it. This is your original program, unmodified.


This illustrates the intended behavior of a move hook if it 
existed in the language. The 'rhs' that was passed to the call 
was constructed at that same address ( == 
). I.e. this is how I'm suggesting a 
this(typeof(this)) *could* work.


this(typeof(this)), of course, would need to be special in the 
ABI, but again, that's one special function instead of two.


No. My proposal requires one amendment to argument passing in 
the ABI, but no special cases at all. Changes to the ABI are 
not the same as changes to the run time library.


How so? Or, more to the point, what's argument passing OR runtime 
have to do with this?


Let's take a step back for a moment and look at what should 
actually be happening for this hook to work (which you briefly 
mention in the DIP):


1. The compiler constructs the value. In your case, it 
constructs two:


It does not. It copies the bits from one to the other.


Poor choice of words on my part. It *creates* two. Whereas with 
this(typeof(this)) no implicit copying of bits is required, a-la 
a C++ move constructor. The programmer is free to choose the bits 
they need, or do a blit-and-patch if so desired. Except that 
unlike a C++ move constructor, no state bookkeeping would be 
necessary (same is true with your DIP as is).


the original and the new one. In my case, it constructs the 
original and then passes it over to the move ctor (one blit 
potentially avoided).

2. It calls the hook (move ctor).


I'm not sure I follow you on that one. What did you mean?


In your case, that would be when it calls __move_post_blt.


3. In your case, it calls the opPostMove.


In all cases, you need to call the hook, whatever it is, for 
those structs that have it, and do some default handling for 
those that don't.


Correct, which would just be whatever the compilers already do.

4. In any case, it *doesn't* destruct the original. Ever. The 
alternative would be to force the programmer to put the 
original back into valid state, and suddenly we're back to C++ 
with all it's pleasantries.


That last part is quite different from the current model, in 
which the compiler always destructs function arguments. That's 
why my example fails when a destructor is present.


Like 

Re: DIP 1014

[Stanislav Blinov via Digitalmars-d]

On Wednesday, 3 October 2018 at 18:38:50 UTC, Manu wrote:
On Wed, Oct 3, 2018 at 7:00 AM Stanislav Blinov via 
Digitalmars-d  wrote:



Any function in D that has a signature of the form

ReturnType foo(Type x);

in C++ would have an equivalent signature of

ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);


What are you talking about? Equivalent C++ is:

ReturnType foo(Type x);


C++ has rvalue references, move semantics are explicit. D doesn't 
have any of that. Perhaps I wasn't quite clear in that above 
statement though.


Given some type Bar, compare these two calls in C++ in D, and 
tell me, which signature in C++ should correspond to D? I'm not 
talking about ABI, I'm talking about semantics.


foo(std::move(bar)); // C++
foo(move(bar)); // D

remembering that D's move() doesn't call postblit.

void foo(Bar bar) wouldn't satisfy that last bit, would it?

Yes, the semantics are different, as in D the move occurs before 
the call. But in D right now you *must* assume that the argument 
may have been moved, with all the consequences the language 
currently entails (and some of which the DIP attempts to 
resolve), whereas in C++ you can be explicit about it via 
overloading for rvalue references.



It's impossible to perform copy elision when passing an lvalue
by-value *to* a function, but that's a case where you depend on 
move semantics.


Of course it's impossible. I'm not sure I understand your point 
here.


Also, within the function that receives an argument by value, 
you

depend on move to construct something with that argument.

Type&& passes a reference, which means you can perform the move 
direct from source to destination, without a stop at the middle 
man.


Yup, no argument there.

Re: DIP 1014

[Shachar Shemesh via Digitalmars-d]

On 03/10/18 20:43, Stanislav Blinov wrote:

On Wednesday, 3 October 2018 at 15:33:00 UTC, Shachar Shemesh wrote:
I.e. - I am asserting if a move was not caught. The program fails to 
run on either ldc or dmd. To me, this makes perfect sense as for the 
way D is built. In essence, opAssign isn't guaranteed to run. Feel 
free to build a struct where that assert passes to convince me.


That's a slightly different issue here.


Well, I view this issue as a deal breaker. If you need to move the 
object *in order* to pass it to your move hook, then anything that 
requires knowing the address of the old instance will, by definition, 
not work.


Look at the output. The operator is being run, it can't *not* run, 


Sure it can. Just look at the example I posted on the other thread 
(https://forum.dlang.org/post/pp2v16$1014$1...@digitalmars.com). The hook 
you mention is downright @disabled there.


In fact, had that not been the case, this DIP would never have happened.




Here is the flaw in your logic:

    void opAssign(Tracker rhs)

rhs is passed by value. This means that already at the point opAssign 
is called, rhs *already* has a different address than the one it was 
passed in with.


Currently that is only true if you define a destructor.


No, that's not true. Try printing the instance's address in the 
constructor and again in your operator.


In the presence of a move hook, 'rhs' 
would first have to pass through that hook, which will not take 
destructors into account at all.


I'm sorry, I'm not following. What is the difference between what you're 
proposing and opPostMove as defined?


Consider your own DIP: what you're suggesting is the ability to take the 
address of the original when a move is taking place. My example shows 
that in the simplest case even today, address of the original is already 
the address of the argument.


This is the run I got:
$ ./movetest2
Address of temporary is 'b382e390', counter points to 'b382e390'
... which is '0' bytes from the address of temporary.
Address of temporary is 'b382e390', counter points to '8eb82b60'
... which is '-966984906800' bytes from the address of temporary.
Address of temporary is 'b382e390', counter points to 'b382e390'
... which is '0' bytes from the address of temporary.
Address of temporary is 'b382e390', counter points to '8eb82b60'
... which is '-966984906800' bytes from the address of temporary.

I'm not sure what I should have seen, or what I should have concluded 
from it. This is your original program, unmodified.





The changes are literally the same as the ones you're proposing:

"When moving a struct's instance, the compiler MUST call __move_post_blt 
giving it both new and old instances' addresses."


That is the same that would have to happen with this(typeof(this) rhs), 
where  is the address of new instance, and  is the address of 
old instance, but there's no need for opPostMove then. I guess what I 
should've said from the start is that the semantics you're proposing fit 
nicely within one special function, instead of two.


Except, like I said, it's not working for me, and I find it hard to 
understand how it *can* work (inlining notwithstanding), which is why I 
did not propose it.




this(typeof(this)), of course, would need to be special in the ABI, but 
again, that's one special function instead of two.


No. My proposal requires one amendment to argument passing in the ABI, 
but no special cases at all. Changes to the ABI are not the same as 
changes to the run time library.




Let's take a step back for a moment and look at what should actually be 
happening for this hook to work (which you briefly mention in the DIP):


1. The compiler constructs the value. In your case, it constructs two:


It does not. It copies the bits from one to the other.

This also means that a struct where some of its members have a hook is 
copied wholesale and patched, which is typically faster than copying in 
parts.


the original and the new one. In my case, it constructs the original and 
then passes it over to the move ctor (one blit potentially avoided).

2. It calls the hook (move ctor).


I'm not sure I follow you on that one. What did you mean?


3. In your case, it calls the opPostMove.


In all cases, you need to call the hook, whatever it is, for those 
structs that have it, and do some default handling for those that don't.


4. In any case, it *doesn't* destruct the original. Ever. The 
alternative would be to force the programmer to put the original back 
into valid state, and suddenly we're back to C++ with all it's 
pleasantries.


That last part is quite different from the current model, in which the 
compiler always destructs function arguments. That's why my example 
fails when a destructor is present.


Like I said above, I don't think that's correct.



The other thing to note (again something that you mention but don't 
expand on), and that's nodding back to my comment about making move() 
and emplace() 

Re: DIP 1014

[Manu via Digitalmars-d]

On Wed, Oct 3, 2018 at 7:00 AM Stanislav Blinov via Digitalmars-d
 wrote:
>
> Shachar, as I don't see a better place of discussing that DIP at
> the moment, I'll pour some observations and thoughts in here if
> you don't mind, will add some comments on GitHub later.
> As I see it right now, it's a case of over-engineering of a quite
> simple concept.
>
> 1. A new function, called __move_post_blt, will be added to
> DRuntime.
>
> That's unnecessary, if not downright harmful for the language. We
> should strive to remove things from DRuntime, not add to it. The
> core language should deal with type memory, not a .so or dll. And
> it's extraneous, because...
>
> 2 and 3. onPostMove and __move_post_blt:
>
> They're unnecessary as well. All that's required is to allow a
> by-value constructor, e.g:
>
> struct S {
>  this(S rhs);
> }
>
> Any function in D that has a signature of the form
>
> ReturnType foo(Type x);
>
> in C++ would have an equivalent signature of
>
> ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);

What are you talking about? Equivalent C++ is:

ReturnType foo(Type x);

It's impossible to perform copy elision when passing an lvalue
by-value *to* a function, but that's a case where you depend on move
semantics.
Also, within the function that receives an argument by value, you
depend on move to construct something with that argument.

Type&& passes a reference, which means you can perform the move direct
from source to destination, without a stop at the middle man.

This all has nothing to do with Walter's surprising claim that "as the
DMD compiler doesn't actually move structs"... I'm still trying to
understand this statement.

Re: DIP 1014

[Stanislav Blinov via Digitalmars-d]

On Wednesday, 3 October 2018 at 08:21:38 UTC, Manu wrote:

Okay, so copy elision is working... but moves otherwise are 
not? That's still not what we've been peddling all these years. 
A whole lot of design surface area is dedicated to implicit 
move semantics... and they don't work? What does it do? 
postblit unnecessarily?


No. The problem is that the language is under-specified. It is 
built on the *assumption* that no one ever should create 
self-referencing data. But it does not enforce that. Which 
eventually leads to someone trying to make such data and then run 
into a wall, or worse, a heisenbug.


Thing is, there isn't anything wrong with self-referencing data 
per se. It's that the language plumbing should either disallow it 
wholesale (i.e. Rust) or allow a graceful way of handling it. 
Neither is present in D. The latter could be added though, that's 
what the DIP is about.

Re: DIP 1014

[Stanislav Blinov via Digitalmars-d]

On Wednesday, 3 October 2018 at 17:43:08 UTC, Stanislav Blinov 
wrote:


But IMHO, it's something that should be fixed by not making 
these facilities built into the language.


s/not//

Re: DIP 1014

[Stanislav Blinov via Digitalmars-d]

On Wednesday, 3 October 2018 at 15:33:00 UTC, Shachar Shemesh 
wrote:

On 03/10/18 17:29, Stanislav Blinov wrote:

OMG, that's so simple!!! Why didn't I think of it?

Oh wait, I did.


Now I see why sometimes your posts are greeted with hostility.


Yes. I am actually sorry about that. I was responding to your 
assumption that I'm wrong. Had your post been phrased as "why 
didn't you", instead of "you're wrong wrong wrong" I wouldn't 
have responded that way.


Like I said, I am sorry.


I am sorry as well since I wasn't clear in my initial post.


> Allow me to further illustrate with something that can be
written in D > today:

I am not sure what you were trying to demonstrate, so instead I 
wanted to see if you succeeded. I added the following to your 
Tracker struct:


 ~this() {
writefln("%s destructed", );
assert(counter is null || counter is );
}

I.e. - I am asserting if a move was not caught. The program 
fails to run on either ldc or dmd. To me, this makes perfect 
sense as for the way D is built. In essence, opAssign isn't 
guaranteed to run. Feel free to build a struct where that 
assert passes to convince me.


That's a slightly different issue here.
Look at the output. The operator is being run, it can't *not* 
run, unlike postblit (ironically, right now it doesn't run on 
fixed-size arrays though). In fact, as soon as you define a 
destructor, the compiler will generate a by-value opAssign if you 
haven't defined one.
That's a separate problem. Currently, presence of a destructor 
makes the compilers generate different code, because it cannot 
elide destruction of arguments, because explicit move semantics 
do not exist in the language. That's why I haven't included a 
destructor in the example to begin with.



Here is the flaw in your logic:

void opAssign(Tracker rhs)

rhs is passed by value. This means that already at the point 
opAssign is called, rhs *already* has a different address than 
the one it was passed in with.


Currently that is only true if you define a destructor. That 
would not be true, however, if a move hook in any form existed in 
the language. That was my point.
I only used opAssign as something resembling the supposed new 
behavior, not as a "look, it already works". In the presence of a 
move hook, 'rhs' would first have to pass through that hook, 
which will not take destructors into account at all.
Consider your own DIP: what you're suggesting is the ability to 
take the address of the original when a move is taking place. My 
example shows that in the simplest case even today, address of 
the original is already the address of the argument. Except it 
cannot be enforced in any way right now. A move hook will have to 
enforce that, as it will have to be called for every move.


I did not follow your logic on why this isn't so, but I don't 
see how you can make it not so without changing the ABI quite 
drastically.


The changes are literally the same as the ones you're proposing:

"When moving a struct's instance, the compiler MUST call 
__move_post_blt giving it both new and old instances' addresses."


That is the same that would have to happen with this(typeof(this) 
rhs), where  is the address of new instance, and  is the 
address of old instance, but there's no need for opPostMove then. 
I guess what I should've said from the start is that the 
semantics you're proposing fit nicely within one special 
function, instead of two.


this(typeof(this)), of course, would need to be special in the 
ABI, but again, that's one special function instead of two.


Let's take a step back for a moment and look at what should 
actually be happening for this hook to work (which you briefly 
mention in the DIP):


1. The compiler constructs the value. In your case, it constructs 
two: the original and the new one. In my case, it constructs the 
original and then passes it over to the move ctor (one blit 
potentially avoided).

2. It calls the hook (move ctor).
3. In your case, it calls the opPostMove.
4. In any case, it *doesn't* destruct the original. Ever. The 
alternative would be to force the programmer to put the original 
back into valid state, and suddenly we're back to C++ with all 
it's pleasantries.


That last part is quite different from the current model, in 
which the compiler always destructs function arguments. That's 
why my example fails when a destructor is present.


The other thing to note (again something that you mention but 
don't expand on), and that's nodding back to my comment about 
making move() and emplace() intrinsics, is that creating such a 
hook *will* invalidate current behavior of move(). Which is 
perhaps more easily fixed with your implementation, actually, 
*except* for the part about eliding destruction. Unions are 
unreliable for that unless we also change the spec that talks 
about them.
But IMHO, it's something that should be fixed by not making these 
facilities built into the language.

Re: DIP 1014

[Shachar Shemesh via Digitalmars-d]

On 03/10/18 12:48, Corel wrote:
The fact that in D the structures to date are not moved, is known for 
years ... take advantage of this fact, and move on.


I have no idea where you got this fact:

import std.stdio;

struct MoveTest {
static uint counter=1;
uint id;

@disable this(this);
@disable this(MoveTest);

this(uint dummy) {
id = counter++;
writefln("Constructed %s id %s", , id);
}

~this() {
writefln("Id %s destroyed at %s", id, );
}
}

MoveTest func1() {
return MoveTest(3);
}

void func2(MoveTest m) {
}

int main() {
func2(func1());

return 0;
}

$ rdmd movetest.d
Constructed 7FFDC7A663E0 id 1
Id 1 destroyed at 7FFDC7A66400

Our instance was constructed at one address, but destroyed at another. 
In other words, it was moved.


Can we, please, put that myth to rest?

Shachar

Re: DIP 1014

[Manu via Digitalmars-d]

On Wed, Oct 3, 2018 at 2:50 AM Corel via Digitalmars-d
 wrote:
>
> On Wednesday, 3 October 2018 at 08:21:38 UTC, Manu wrote:
> > On Tue, Oct 2, 2018 at 6:15 PM Walter Bright via Digitalmars-d
> >  wrote:
> >>
> >> On 10/2/2018 4:30 PM, Adam D. Ruppe wrote:
> >> > On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis
> >> > wrote:
> >> >> Yeah. IIRC, it was supposed to be _guaranteed_ that the
> >> >> compiler moved structs in a number of situations - e.g.
> >> >> when the return value was an rvalue. Something like
> >> >
> >> > Eh, I don't think that moves it, but rather just constructs
> >> > it in-place for the next call.
> >>
> >> The technical term for that is "copy elision".
> >
> > Okay, so copy elision is working... but moves otherwise are
> > not? That's still not what we've been peddling all these years.
> > A whole lot of design surface area is dedicated to implicit
> > move semantics... and they don't work? What does it do?
> > postblit unnecessarily?
>
> The impression is that you are complaining about the continuous
> lack of "things" based on an incomplete knowledge of how D works
> in detail ... tragically you invoke low-level features, and you
> do not know the question.
>
> The fact that in D the structures to date are not moved, is known
> for years ... take advantage of this fact, and move on.
>
> Work on an implementation that works, AFTER profile it, and
> possibly complain about performance.

O_o .. this is one of the stranger replies I've ever gotten here.

Re: DIP 1014

[Shachar Shemesh via Digitalmars-d]

On 03/10/18 18:33, Shachar Shemesh wrote:

  ~this() {
     writefln("%s destructed", );
     assert(counter is null || counter is );
     }


You might also want to add @disable this(this); and remove the dead code 
(i.e. - the case where the pointer is global) to reduce noise. I 
verified that neither one changes anything in the outcome.


Shachar

Re: DIP 1014

[Shachar Shemesh via Digitalmars-d]

On 03/10/18 17:29, Stanislav Blinov wrote:

OMG, that's so simple!!! Why didn't I think of it?

Oh wait, I did.


Now I see why sometimes your posts are greeted with hostility.


Yes. I am actually sorry about that. I was responding to your assumption 
that I'm wrong. Had your post been phrased as "why didn't you", instead 
of "you're wrong wrong wrong" I wouldn't have responded that way.


Like I said, I am sorry.


> Allow me to further illustrate with something that can be written in 
D > today:


I am not sure what you were trying to demonstrate, so instead I wanted 
to see if you succeeded. I added the following to your Tracker struct:


 ~this() {
writefln("%s destructed", );
assert(counter is null || counter is );
}

I.e. - I am asserting if a move was not caught. The program fails to run 
on either ldc or dmd. To me, this makes perfect sense as for the way D 
is built. In essence, opAssign isn't guaranteed to run. Feel free to 
build a struct where that assert passes to convince me.


Here is the flaw in your logic:

void opAssign(Tracker rhs)

rhs is passed by value. This means that already at the point opAssign is 
called, rhs *already* has a different address than the one it was passed 
in with. I did not follow your logic on why this isn't so, but I don't 
see how you can make it not so without changing the ABI quite drastically.


Shachar

Re: DIP 1014

[Stanislav Blinov via Digitalmars-d]

On Wednesday, 3 October 2018 at 14:07:58 UTC, Shachar Shemesh 
wrote:


If you read the DIP, you will notice that the *address* in 
which the old instance resides is quite important...


Allow me to further illustrate with something that can be written 
in D today:


import std.stdio;

struct Tracker {
static int globalCounter;
int localCounter;
int* counter;

this (bool local) {
if (local) counter = 
else counter = 
}

// this should be this(Tracker rhs)
void opAssign(Tracker rhs) {
// note: taking address of local parameter
// note: LDC will already 'optimize' the move which in 
the absence
// of any move hooks will mess up the address; try with 
DMD
printf("Address of temporary is '%x', counter points to 
'%x'\n", , rhs.counter);

auto d = cast(void*) rhs.counter - cast(void*) 
printf("... which is '%ld' bytes from the address of 
temporary.\n", d);

localCounter = rhs.localCounter;
counter = rhs.counter;
if (counter is )
counter = 
}
}

auto createCounter(bool local = true) {
Tracker result = Tracker(local);
return result;
}

auto createCounterNoNRV(bool local = true) {
return Tracker(local);
}

void main() {

Tracker stale1, stale2;

stale1 = createCounter();
stale2 = createCounter(false);

Tracker stale3, stale4;

stale3 = createCounterNoNRV();
stale4 = createCounterNoNRV(false);
}


If you run the above with DMD, you'll see what I mean about 
obviating the address. If we get this(typeof(this)) (that is 
*always* called on move) into the language, the behavior would be 
set in stone regardless of compiler.

Re: DIP 1014

[Stanislav Blinov via Digitalmars-d]

On Wednesday, 3 October 2018 at 14:07:58 UTC, Shachar Shemesh 
wrote:

On 03/10/18 16:56, Stanislav Blinov wrote:

struct S {
     this(S rhs);


OMG, that's so simple!!! Why didn't I think of it?

Oh wait, I did.


Now I see why sometimes your posts are greeted with hostility.


And this simply and utterly doesn't work.

If you read the DIP, you will notice that the *address* in 
which the old instance resides is quite important for 
performing the actual move. This is not available with the 
interface you're suggesting, mainly because by the time you 
have rhs, it has already moved.


In other words, for the interface above to work, the type must 
already be movable, which kinda contradict what we're trying to 
achieve here.


In the presence of such a constructor, the compiler will have to 
call it every time it moves the value, same as what you're 
proposing for __move_post_blt. This obviates the need of an 
address: address of the argument will always already be 
sufficient, even though it's not ref, as the chain of calls for 
this(S) will inevitably start with the address of something 
constructed in place.



in C++ would have an equivalent signature of

ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);


No, it is not. You see, in C++, x is an "rvalue *reference*". x 
has not moved by this point in the run, it has simply had its 
address passed to foo.


You've misunderstood me. Yes, in C++ there's an obvious 
difference between pass-by-value and pass-by-rvalue-reference, 
and it is always user's responsibility to write a move ctor. Not 
so in D. In D, you can always assume that anything passed by 
value *is* an rvalue reference, precisely because of D's take on 
move semantics. I.e. any argument passed by value can assumed to 
be moved or constructed in place (that's the main difference from 
C++, where it must be explicitly specified).

Re: DIP 1014

[Shachar Shemesh via Digitalmars-d]

On 03/10/18 16:56, Stanislav Blinov wrote:

struct S {
     this(S rhs);


OMG, that's so simple!!! Why didn't I think of it?

Oh wait, I did.

And this simply and utterly doesn't work.

If you read the DIP, you will notice that the *address* in which the old 
instance resides is quite important for performing the actual move. This 
is not available with the interface you're suggesting, mainly because by 
the time you have rhs, it has already moved.


In other words, for the interface above to work, the type must already 
be movable, which kinda contradict what we're trying to achieve here.



in C++ would have an equivalent signature of

ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);


No, it is not. You see, in C++, x is an "rvalue *reference*". x has not 
moved by this point in the run, it has simply had its address passed to 
foo. Please see 
https://stackoverflow.com/questions/28483250/rvalue-reference-is-treated-as-an-lvalue


Shachar

Re: DIP 1014

[Stanislav Blinov via Digitalmars-d]

On Wednesday, 3 October 2018 at 13:56:29 UTC, Stanislav Blinov 
wrote:


Aendment, this should of course be:


this(Tracker oldLocation) {
localCounter = oldLocation.locaclCounter;
counter = oldLocation.counter;
if( counter is  )
counter = 
}

Re: DIP 1014

[Stanislav Blinov via Digitalmars-d]

Shachar, as I don't see a better place of discussing that DIP at 
the moment, I'll pour some observations and thoughts in here if 
you don't mind, will add some comments on GitHub later.
As I see it right now, it's a case of over-engineering of a quite 
simple concept.


1. A new function, called __move_post_blt, will be added to 
DRuntime.


That's unnecessary, if not downright harmful for the language. We 
should strive to remove things from DRuntime, not add to it. The 
core language should deal with type memory, not a .so or dll. And 
it's extraneous, because...


2 and 3. onPostMove and __move_post_blt:

They're unnecessary as well. All that's required is to allow a 
by-value constructor, e.g:


struct S {
this(S rhs);
}

Any function in D that has a signature of the form

ReturnType foo(Type x);

in C++ would have an equivalent signature of

ReturnType foo(Type&& x); // NOT ReturnType foo(Type x);

because passing by value in D always implies a possible move. The 
'x' in such functions can be safely cannibalized without any 
repercussions, as it is either a temporary on the call site, or, 
which is especially pertaining to the original bugzilla 
discussion, constructed in place via copy elision.


Thus in effect this(S) would be an equivalent of C++'s move 
constructor. We already have a de-facto move-assignment in the 
form of opAssign(S), this(S) would be a natural extension to that.


Note, per above, that it is NOT a copy constructor, although user 
code may want to create a copy *before* calling it, to create the 
temporary.


Such approach reduces added complexity. The only potential 
problem with it would be a need to "special-case" initialization 
from .init, although at the moment, I think even that may be 
unnecessary: this is a hook after all.


Your example from the DIP would become:

struct Tracker {
static uint globalCounter;
uint localCounter;
uint* counter;

@disable this(this);

this(bool local) {
localCounter = 0;
if( local )
counter = 
else
counter = 
}

this(Tracker oldLocation) {
if( counter is  )
counter = 
}

void increment() {
(*counter)++;
}

}

Usage:

auto old = Tracker(true);
// ...
auto new = move(old); // calls Tracker.this(Tracker);

...this avoids any need to inject special postblits into user 
code.


As I see it, in addition to the above, what would be really 
desirable is for move() and emplace() family of calls to become 
compiler intrinsics instead of library constructs. Those at the 
moment are complete poison: being templates they infect user code 
with dependencies on libc (moveEmplace calls memset and memcpy of 
all things) and unnecessary calls to DRuntime (typeid), and they 
of course blow up the amount of generated code in the form of 
template instantiations. That's despite the fact that the 
compiler possesses ALL the necessary knowledge at the time of 
those calls.

Re: DIP 1014

[Shachar Shemesh via Digitalmars-d]

On 03/10/18 04:10, Walter Bright wrote:

On 10/2/2018 4:30 PM, Adam D. Ruppe wrote:

On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis wrote:
Yeah. IIRC, it was supposed to be _guaranteed_ that the compiler 
moved structs in a number of situations - e.g. when the return value 
was an rvalue. Something like


Eh, I don't think that moves it, but rather just constructs it 
in-place for the next call.


The technical term for that is "copy elision".


I'm not sure I follow.

First of all, you cannot elide the copy if there is more than one 
potential local variable you are returning, ala:


A someFunc() {
  A a, b;
  manipulate(a); manipulate(b);

  if( someRandomCondition )
return a;

  return b;
}

What happens then?

What happens if A has @disable this(this)?

What happens if we explicitly call std.algorithm.move?

Shachar

Re: DIP 1014

[Corel via Digitalmars-d]

On Wednesday, 3 October 2018 at 08:21:38 UTC, Manu wrote:
On Tue, Oct 2, 2018 at 6:15 PM Walter Bright via Digitalmars-d 
 wrote:


On 10/2/2018 4:30 PM, Adam D. Ruppe wrote:
> On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis 
> wrote:
>> Yeah. IIRC, it was supposed to be _guaranteed_ that the 
>> compiler moved structs in a number of situations - e.g. 
>> when the return value was an rvalue. Something like

>
> Eh, I don't think that moves it, but rather just constructs 
> it in-place for the next call.


The technical term for that is "copy elision".


Okay, so copy elision is working... but moves otherwise are 
not? That's still not what we've been peddling all these years. 
A whole lot of design surface area is dedicated to implicit 
move semantics... and they don't work? What does it do? 
postblit unnecessarily?


The impression is that you are complaining about the continuous 
lack of "things" based on an incomplete knowledge of how D works 
in detail ... tragically you invoke low-level features, and you 
do not know the question.


The fact that in D the structures to date are not moved, is known 
for years ... take advantage of this fact, and move on.


Work on an implementation that works, AFTER profile it, and 
possibly complain about performance.

Re: DIP 1014

[Manu via Digitalmars-d]

On Tue, Oct 2, 2018 at 6:15 PM Walter Bright via Digitalmars-d
 wrote:
>
> On 10/2/2018 4:30 PM, Adam D. Ruppe wrote:
> > On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis wrote:
> >> Yeah. IIRC, it was supposed to be _guaranteed_ that the compiler moved 
> >> structs
> >> in a number of situations - e.g. when the return value was an rvalue.
> >> Something like
> >
> > Eh, I don't think that moves it, but rather just constructs it in-place for 
> > the
> > next call.
>
> The technical term for that is "copy elision".

Okay, so copy elision is working... but moves otherwise are not?
That's still not what we've been peddling all these years. A whole lot
of design surface area is dedicated to implicit move semantics... and
they don't work? What does it do? postblit unnecessarily?

Re: DIP 1014

[Walter Bright via Digitalmars-d]

On 10/2/2018 4:30 PM, Adam D. Ruppe wrote:

On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis wrote:
Yeah. IIRC, it was supposed to be _guaranteed_ that the compiler moved structs 
in a number of situations - e.g. when the return value was an rvalue. 
Something like


Eh, I don't think that moves it, but rather just constructs it in-place for the 
next call.


The technical term for that is "copy elision".

Re: DIP 1014

[Adam D. Ruppe via Digitalmars-d]

On Tuesday, 2 October 2018 at 22:30:38 UTC, Jonathan M Davis 
wrote:
Yeah. IIRC, it was supposed to be _guaranteed_ that the 
compiler moved structs in a number of situations - e.g. when 
the return value was an rvalue. Something like


Eh, I don't think that moves it, but rather just constructs it 
in-place for the next call.

Re: DIP 1014

[Jonathan M Davis via Digitalmars-d]

On Tuesday, October 2, 2018 11:54:57 AM MDT Manu via Digitalmars-d wrote:
> On Tue, Oct 2, 2018 at 2:40 AM Walter Bright via Digitalmars-d
>
>  wrote:
> > On 10/2/2018 2:17 AM, Walter Bright wrote:
> > > 1. Don't allow moving of C++ structs
> > > 2. Add a struct attribute that means "not moveable"
> > > 3. DIP 1014, which is add a __move_post_blit() function (most complex
> > > solution) 4. Use copy/destruct for C++ structs that have copy
> > > constructors (this is the old C++ solution, and is less efficient
> > > than the move constructor)>
> > The postblit solution can also work today,
> >
> > https://issues.dlang.org/show_bug.cgi?id=17448#c37
> >
> > as the DMD compiler doesn't actually move structs.
>
> Sorry... what?! DMD doesn't move? O_O
> We've been talking endlessly about move semantics for like, 10 years...
> Do the other compilers move?
>
> I don't understand... I've missed something.

Yeah. IIRC, it was supposed to be _guaranteed_ that the compiler moved
structs in a number of situations - e.g. when the return value was an
rvalue. Something like

A foo();
void bar(A);

bar(foo());

is supposed to be guaranteed to move and not copy the return value. Maybe it
manages to place the return value in a way that doesn't require actually
moving it or copying it, but structs being moved was supposed to be a key
thing that D could do. Are we currently getting a bunch of copies that we
shouldn't be getting because the compiler isn't yet doing moves when it
should be? A number of us have been answering questions for years based on
what the spec and TDPL say indicating that the compiler moves structs.

And while I'm all for having objects be moveable by default, I confess that
I don't understand what the problem is with the opPostMove idea that the DIP
is presenting. I don't think that it should be the norm by any means, but it
sure seems like it's cleanly solving a major problem with interacting with
C++, and it makes it possible to do stuff like have pointers and dynamic
arrays refer to a structs internals in a way that we can't @safely do right
now (even though occasionally, it really would be useful to be able to do
it). And it sure seems like opPostMove fits in cleanly with the current
design, though maybe I'm missing something. Either way, without something
like it, we're clearly missing some key functionality for a systems language
- particularly one that wants to interoperate with C++.

- Jonathan M Davis

Re: DIP 1014

[Andrei Alexandrescu via Digitalmars-d]

On 10/2/18 1:51 PM, Manu wrote:

But dangling pointer is an instant crash/memory corruption... it's a
pretty bad 'bug'.


Yeah doesn't sound very brilliant. I think such a workaround wouldn't 
fare well. To keep momentum while we mull over a solution to this I 
suggest you look at porting other data structures.

Re: DIP 1014

[Manu via Digitalmars-d]

On Tue, Oct 2, 2018 at 2:40 AM Walter Bright via Digitalmars-d
 wrote:
>
> On 10/2/2018 2:17 AM, Walter Bright wrote:
> > 1. Don't allow moving of C++ structs
> > 2. Add a struct attribute that means "not moveable"
> > 3. DIP 1014, which is add a __move_post_blit() function (most complex 
> > solution)
> > 4. Use copy/destruct for C++ structs that have copy constructors (this is 
> > the
> > old C++ solution, and is less efficient than the move constructor)
>
> The postblit solution can also work today,
>
> https://issues.dlang.org/show_bug.cgi?id=17448#c37
>
> as the DMD compiler doesn't actually move structs.

Sorry... what?! DMD doesn't move? O_O
We've been talking endlessly about move semantics for like, 10 years...
Do the other compilers move?

I don't understand... I've missed something.

>  So you're OK for the time
> being until DMD does, or a copying garbage collector is implemented.

Ummm...

Re: DIP 1014

[Manu via Digitalmars-d]

On Tue, Oct 2, 2018 at 2:20 AM Walter Bright via Digitalmars-d
 wrote:
>
> On 9/29/2018 9:34 PM, Manu wrote:
> > Who knows about DIP 1014? (struct move hook)
> > Is it well received? Is it likely to be accepted soon?
> >
> > I'm working on the std::string binding, it's almost finished... but
> > then I hit a brick wall.
> > GNU's std::string implementation stores an interior pointer! >_<
>
>
> The rationale behind D allowing structs to be moveable is to enable a copying
> garbage collector.

Do we do that? What's the use of that?
Does the opPostMove() concept not work here? If the GC wanted to move
something, it can call it like any other code?

> Some solutions to this problem:
>
> 1. Don't allow moving of C++ structs

std::string, std::vector, etc are quite impotent without move semantics.
I'm pretty sure the first time I ever started using those containers
was around 2015 when we became secure we could use C++11 in our code
(supported by all compiler vendors). Before that, they were banned and
we had alternative solutions.

> 2. Add a struct attribute that means "not moveable"

They must be movable though.

> 3. DIP 1014, which is add a __move_post_blit() function (most complex 
> solution)

It's alleged you're working through the DIP... what's the story there?
Are you unhappy with it?

> 4. Use copy/destruct for C++ structs that have copy constructors (this is the
> old C++ solution, and is less efficient than the move constructor)

If that's where we land on this, I'll struggle to find value in my
work. Containers become as impotent as back before 2011 :/

> Pragmatically, I suggest for the moment just ignore the problem, file a bug
> report for std::string, and move on.

But dangling pointer is an instant crash/memory corruption... it's a
pretty bad 'bug'.

Re: DIP 1014

[Walter Bright via Digitalmars-d]

On 10/2/2018 2:17 AM, Walter Bright wrote:

1. Don't allow moving of C++ structs
2. Add a struct attribute that means "not moveable"
3. DIP 1014, which is add a __move_post_blit() function (most complex solution)
4. Use copy/destruct for C++ structs that have copy constructors (this is the 
old C++ solution, and is less efficient than the move constructor)


The postblit solution can also work today,

https://issues.dlang.org/show_bug.cgi?id=17448#c37

as the DMD compiler doesn't actually move structs. So you're OK for the time 
being until DMD does, or a copying garbage collector is implemented.

Re: DIP 1014

[Walter Bright via Digitalmars-d]

On 9/29/2018 9:34 PM, Manu wrote:

Who knows about DIP 1014? (struct move hook)
Is it well received? Is it likely to be accepted soon?

I'm working on the std::string binding, it's almost finished... but
then I hit a brick wall.
GNU's std::string implementation stores an interior pointer! >_<



The rationale behind D allowing structs to be moveable is to enable a copying 
garbage collector.


Some solutions to this problem:

1. Don't allow moving of C++ structs
2. Add a struct attribute that means "not moveable"
3. DIP 1014, which is add a __move_post_blit() function (most complex solution)
4. Use copy/destruct for C++ structs that have copy constructors (this is the 
old C++ solution, and is less efficient than the move constructor)


A discussion of the rationale for the C++ move constructor is:

https://akrzemi1.wordpress.com/2011/08/11/move-constructor/


> Anyway, I'm blocked until this DIP is accepted; so, is it looking promising?

Pragmatically, I suggest for the moment just ignore the problem, file a bug 
report for std::string, and move on.

Re: DIP 1014

[Walter Bright via Digitalmars-d]

On 9/29/2018 9:34 PM, Manu wrote:

Who knows about DIP 1014? (struct move hook)


When discussing DIP 1014, a link is helpful:

https://github.com/dlang/DIPs/blob/38cec74a7471735559e3b8a7553f55102d289d28/DIPs/DIP1014.md

Re: DIP 1014

[Jonathan M Davis via Digitalmars-d]

On Sunday, September 30, 2018 1:35:28 AM MDT Shachar Shemesh via 
Digitalmars-d wrote:
> On 30/09/18 10:26, Manu wrote:
> > Other implementations make much better use of that built-in space by
> > not wasting 8 bytes on an interior pointer for small-strings.
>
> I will point out that a pointer that *sometimes* points to an internal
> member was one of the use cases I documented when I submitted the DIP.
>
> Starting a long discussion about the merits of the design is a bit
> off-topic. I will point out that branch prediction considerations
> *might* make this a wise choice, despite the loss of 8 bytes of
> potential storage.
>
> Either way, this is a design that is highly sensitive to precise use
> pattern, which, admitably, GNU's std::string probably can't know.

I think that the key thing here is that if GNU's std::string is using a
design like this, it's that much more critical that we have a way to hook
into moves to do stuff like adjust pointers. It's one more example of a real
world use case where the DIP (or something like it) is needed, or there are
things that we simply won't be able to do in D - and given the push to
interface with C++, it's that much more important. And while a discussion
could certainly be had as to whether GNU's design decision was a good one or
not, it's secondary to what really matters here, which is what the state of
the DIP is how we're going to deal with interfacing with this C++ code. We
need to worry about how to interface with it whether it's the best design
ever or whether it's the worst design ever - and we have to take into
account the fact the its design could actually change in future versions if
they decide that a different way is better (e.g. GNU could change to match
other implementations, or other implementations could change to match GNU,
depending on what actually turned out to be best in practice when all of the
various factors were taken into account - including developers making future
decisions that aren't necessarily good ones; we have to interface with the
code whether it's good or bad).

All in all though, if anything, I have to think that this issue increases
the chances of the DIP being accepted given the importance that Walter and
Andrei have been placing on interfacing with C++. And having it come up
while they're in the middle of discussing it probably doesn't hurt - though
maybe they were already going to accept it. I don't know. Personally, while
I tend to think that it's generally better to avoid designs where opPostMove
is necessary if possible, I think that the case was well made that we need a
solution like it in certain cases, and if we want to interface with C++,
which can do more or less arbitrary stuff in its move constructors, I don't
see how we can avoid having an analogue unless we want to give up on
interfacing with that code without an extra compatibility layer.

- Jonathan M Davis

Re: DIP 1014

[Shachar Shemesh via Digitalmars-d]

On 30/09/18 10:26, Manu wrote:


Other implementations make much better use of that built-in space by
not wasting 8 bytes on an interior pointer for small-strings.



I will point out that a pointer that *sometimes* points to an internal 
member was one of the use cases I documented when I submitted the DIP.


Starting a long discussion about the merits of the design is a bit 
off-topic. I will point out that branch prediction considerations 
*might* make this a wise choice, despite the loss of 8 bytes of 
potential storage.


Either way, this is a design that is highly sensitive to precise use 
pattern, which, admitably, GNU's std::string probably can't know.


Shachar

Re: DIP 1014

[Manu via Digitalmars-d]

On Sat, Sep 29, 2018 at 11:50 PM Walter Bright via Digitalmars-d
 wrote:
>
> On 9/29/2018 9:34 PM, Manu wrote:
> > GNU's std::string implementation stores an interior pointer! >_<
> >
> > No other implementation does this. It's a really bad implementation
> > actually, quite inefficient. It could make better use of its space for
> > small-strings if it wasn't wasting 8-bytes for an interior pointer to
> > a small string buffer...
>
> Could you post a synopsis of the layout of std::string?

The code's all in the PR if you wanna dig into it.

The synopsis is:
struct string
{
  char* ptr;
  size_t len;
  union
  {
char[16] localBuffer;
size_type allocatedCapacity;
  }

  bool isAllocated() { return ptr != [0]; }
  bool capacity() { return isAllocated() ? allocatedCapacity :
localBuffer.length; }

  this(DefaultCtor) { ptr = [0]; }  // <- and here it is.
interior pointer that breaks move semantics
}

Other implementations make much better use of that built-in space by
not wasting 8 bytes on an interior pointer for small-strings.

Re: DIP 1014

[Walter Bright via Digitalmars-d]

On 9/29/2018 9:34 PM, Manu wrote:

GNU's std::string implementation stores an interior pointer! >_<

No other implementation does this. It's a really bad implementation
actually, quite inefficient. It could make better use of its space for
small-strings if it wasn't wasting 8-bytes for an interior pointer to
a small string buffer...


Could you post a synopsis of the layout of std::string?

Re: DIP 1014

[Jonathan M Davis via Digitalmars-d]

On Saturday, September 29, 2018 10:34:20 PM MDT Manu via Digitalmars-d 
wrote:
> Who knows about DIP 1014? (struct move hook)
> Is it well received? Is it likely to be accepted soon?
>
> I'm working on the std::string binding, it's almost finished... but
> then I hit a brick wall.
> GNU's std::string implementation stores an interior pointer! >_<
>
> No other implementation does this. It's a really bad implementation
> actually, quite inefficient. It could make better use of its space for
> small-strings if it wasn't wasting 8-bytes for an interior pointer to
> a small string buffer...
>
> Anyway, I'm blocked until this DIP is accepted; so, is it looking
> promising?

Per https://github.com/dlang/DIPs/tree/master/DIPs, it's at the "Formal
Assessement" stage, whatever that means (I'm not well enough versed in all
of the stages of the current DIP process to say what that means). However, I
would think that if Walter and Andrei were aware of this particular problem
with std::string, given the importance they place on interfacing with C++,
it would greatly increase the odds of that DIP being accepted.

- Jonathan M Davis

Re: DIP 1014

[Mike Parker via Digitalmars-d]

On Sunday, 30 September 2018 at 04:34:20 UTC, Manu wrote:

Who knows about DIP 1014? (struct move hook)
Is it well received? Is it likely to be accepted soon?

I'm working on the std::string binding, it's almost finished... 
but

then I hit a brick wall.
GNU's std::string implementation stores an interior pointer! >_<

No other implementation does this. It's a really bad 
implementation actually, quite inefficient. It could make 
better use of its space for small-strings if it wasn't wasting 
8-bytes for an interior pointer to a small string buffer...


Anyway, I'm blocked until this DIP is accepted; so, is it 
looking promising?


It's pending a decision from Walter & Andrei, which I hope to 
hear soon.

Re: DIP 1014--Hooking D's struct move semantics--Final Review

[Shachar Shemesh via Digitalmars-d]

On 17/07/18 16:29, aliak00 wrote:


A postblit on a class issues a compiler error. And an identity opAssign 
on a class also issues a compiler error. So I'm not sure how this would 
be different. And the page In 
https://dlang.org/spec/operatoroverloading.html also explicitly mentions 
differences between ops on classes or structs.


That's actually a good argument. It should definitely be handled the 
same way the corresponding opAssign is handled.


Thanks,
Shachar

Re: DIP 1014--Hooking D's struct move semantics--Final Review

[aliak00 via Digitalmars-d]

On Thursday, 12 July 2018 at 10:24:40 UTC, Shachar Shemesh wrote:

On 29/06/18 15:35, aliak wrote:

On Wednesday, 27 June 2018 at 07:24:05 UTC, Mike Parker wrote:

On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote:


Thanks in advance for your participation.


For those of you using the NNTP or mailing list interfaces, 
this is the thread to respond in. Thanks!


Alo!

This is great!

Just a clarification about the last paragraph phrasing

The last line: "We can further reduce this problem by calling 
the function opPostMove." seemed to imply that an alternate 
name to opPostMove would be mentioned, but am I correct in 
understanding that it is just saying that "naming this second 
function as op* will keep code breakage to a minimum" ?


This is a left over from a previous draft, where the operator 
was called "opMove". It should be removed.


Also, what should happen if someone defines an opPostMove for 
a class. Compile error or? Should something about that be 
mentioned?


I think nothing should happen. The function would be ignored, 
just like it is today. I am open to hear other ideas, however.


I'm not sure whether it should be explicitly mentioned or not.

Shachar


A postblit on a class issues a compiler error. And an identity 
opAssign on a class also issues a compiler error. So I'm not sure 
how this would be different. And the page In 
https://dlang.org/spec/operatoroverloading.html also explicitly 
mentions differences between ops on classes or structs.


Cheers,
- Ali

Re: DIP 1014--Hooking D's struct move semantics--Final Review

[Shachar Shemesh via Digitalmars-d]

On 14/07/18 15:56, Johan Engelen wrote:
First off: I am trying to wear a strict language lawyer hat. D spec is 
already very much ill specced which is _very_ problematic for language 
and compiler development. I am not attacking the proposal in order to 
kill it. I am merely commenting on points that I feel should be improved.


Wouldn't these comments be better suited for the language spec's PR, then?

I'm asking seriously. There is nothing in this DIP (or just about any 
other one) that can go, unmodified, into the specs. If that's the DIP's 
purpose, then so be it. If not, then I don't see the value.




OK, so make _that_ explicit. I think there is value in prescribing the 
precise order of moves (like for construction of members), such that 
reasoning about code becomes easier.


I disagree.

The member variables at the point of the move are all:
1. Fully initialized.
2. Either move agnostic or know how to handle their own move
3. Oblivious to one another

Obviously, you might wish to spite me by creating a case where any one 
of the above is not true, but the compiler has every right to assume all 
three points above are true. In your hypothetical spite case, your best 
course of action is to leave the members with no opPostMove at all, and 
handle their move in the containing struct's opPostMove, in which case 
you are fully defined.


Assuming all three are correct, the order in which they are moved makes 
no difference.


If you want the same semantic effect (as I wrote above), then the text 
should say that the ordering is relaxed.


I have no objection to explicitly stating that exact move order of 
members is undefined.





Why "SHOULD" and not "MUST"?


Precisely for the reason you stated above. So that if you want to do 
something else, you may.


Why is that freedom needed?


Because compiler implementers might have a good reason to do something 
besides this. For example, a compiler writer might choose to place the 
actual moving code inside __move_post_blt, and I don't think we should 
stop her.


The freedom is already provided by 
user-defined opPostMove?


Different audiences. opPostMove serves the D programmer, __move_post_blt 
serves the compiler.




I think the language spec doesn't say when a "move" is performed?


I think Walter sees that as an advantage, but I'm not sure.

Either way, the current language spec says structs must have semantics 
that remain correct even if the struct suddenly changes the memory 
address it resides in. The specs + DIP 1014 say that the above is true, 
or the struct must supply an opPostMove that fixes the semantics post-move.


In both cases, *when* the move takes place is irrelevant.

Or is it enough to define what a "move" is ? (didn't check but I guess 
the DIP already explains that)

Only implicitly.


(D's "move" is different from C++'s right?

Yes.

D's move after exiting a 
struct's constructor doesn't lead to a destructor call, but D's 
std.algorithm.mutation.move _does_ call the destructor of the moved 
source object.)


Depends on which version of move you're referring to.

For example, moveEmplace does not.

I think the correct way to phrase this is to say that D's move *never* 
calls a destructor, but if the move's destination had a valid object in 
it, that one gets destructed.


In a way, C++'s move is the same, except the actual moving of the data 
from the source location to the destination one is up to the programmer, 
and accordingly, so is destructing. Since, logically, a C++ move 
operator always copies, it also has to destruct the source.


Technically, however, it doesn't always. A move assignment operator 
typically just swaps the content of the structs (i.e. - moves the source 
to the destination and the destination to the source), and lets the 
usual rvalue elimination code destruct it.


Shachar

Re: DIP 1014--Hooking D's struct move semantics--Final Review

[Johan Engelen via Digitalmars-d]

On Thursday, 12 July 2018 at 10:22:33 UTC, Shachar Shemesh wrote:

On 11/07/18 20:04, Johan Engelen wrote:

On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote:
DIP 1014, "Hooking D's struct move semantics", is now ready 
for final review.


after quick read:

(would be much easier to do inline commenting, but it appears 
that's not supported)


### Section "Code emitted by the compiler on move"
Dangerous to talk about what "code is emitted" by the 
compiler. I think this DIP doesn't need that, and semantics is 
enough. "the compiler MUST call " should be reworded, because 
an _actual_ call to the function should not be mandatory, 
because it would limit the optimizer in eliding it or inlining 
it (note that it will be hard to _prevent_ the optimizer from 
eliding/inlining the call as currently specified by the DIP).


I don't draw the same conclusions from the text as you.

I'm perfectly fine with specifying that nothing here is 
mandatory if the compiler ensures that *the end effect* is as 
if it happened.


AFAIK, C++ has a standing order to that effect, and it greatly 
simplifies documenting what you want to do.


First off: I am trying to wear a strict language lawyer hat. D 
spec is already very much ill specced which is _very_ problematic 
for language and compiler development. I am not attacking the 
proposal in order to kill it. I am merely commenting on points 
that I feel should be improved.


My point was to remove things like "compiler" and "emitted code" 
from the DIP / spec. In this case, the simple rewording can be: 
"When moving a struct's instance, an implicit call to 
__move_post_blt is inserted with both new and old instances' 
addresses as arguments."





### "__move_post_blt SHOULD be defined in a manner that is 
compatible"

What does "compatible" mean?


"Has the same effect as".

It's a little as if you're complaining about something not 
being explicit in one section, and again about that same thing 
being explicit in the next. Precisely why such standing order 
would be a good idea.


Being explicit about generated machine is not good in a language 
spec. Being explicit about the semantic meaning of something 
_is_.  ;-)
"compatible" is very vague. It may mean that just the function 
signature should match.
"has the same semantic effect" would be a much better description 
of what you want.


Some things should be made more explicit, such as the order of 
calls for compound structs.


I don't think it makes sense to specify the order, except to 
say that member's opPostMove must be called before the 
instance's opPostMove (which the code already says).


OK, so make _that_ explicit. I think there is value in 
prescribing the precise order of moves (like for construction of 
members), such that reasoning about code becomes easier.
If you want the same semantic effect (as I wrote above), then the 
text should say that the ordering is relaxed.



Why "SHOULD" and not "MUST"?


Precisely for the reason you stated above. So that if you want 
to do something else, you may.


Why is that freedom needed? The freedom is already provided by 
user-defined opPostMove? I think the implicit call to 
__move_post_blt is a MUST, like calls to dtors.





### "This MUST return true iff a struct or any of its members 
have an opPostMove defined."
Doesn't cover struct A containing struct B containing struct C 
with opPostMove. Reword e.g. into: "hasElaborateMove!S MUST 
return true iff `S` has an `opPostMove` defined or if 
hasElaborateMove!X is true for any member of S of type X.


Yes, I'm sorry. I worded the spec for humans.


Please don't ;-)


I can suggest a recursive definition:

hasElaborateMove for a struct MUST return true iff at least one 
of the following is true:

* The struct has opPostMove defined
* hasElaborateMove returns true for at least one of the 
struct's members.


Great.




### What is lacking is a clear list of exactly in which cases 
`opPostMove` will be called (needed for user-facing 
documentation of the function).


I don't think I understand this point. Can you suggest what 
that list might contain?


I think the language spec doesn't say when a "move" is performed? 
So I don't know when exactly the opPostMove is called. Things 
that come to mind:

* exiting from struct ctor
* std.algorithm.mutation.move
Or is it enough to define what a "move" is ? (didn't check but I 
guess the DIP already explains that)
(D's "move" is different from C++'s right? D's move after exiting 
a struct's constructor doesn't lead to a destructor call, but D's 
std.algorithm.mutation.move _does_ call the destructor of the 
moved source object.)



I now realize that the DIP is a mix between language semantic 
changes (opPostMove) and implementation suggestions/details 
("__move_post_blt"). I think it would have been clearer to split 
the two in the DIP (it's valuable to have implementation 
suggestions in addition to spec changes), but by now that's too 
late. Part of my comments 

Re: DIP 1014--Hooking D's struct move semantics--Final Review

[Dukc via Digitalmars-d]

On Thursday, 12 July 2018 at 10:38:52 UTC, Shachar Shemesh wrote:
On 12/07/18 04:17, Jonathan M Davis wrote:> I'm also> not sure 
if going to copy constructors means that we should do 
something> different with this. It don't think that it's 
affected by it, but I could be> missing something.


I actually had that very same concern myself. Andrei does not 
seem to share it (I talked to him about it during DConf, and he 
even mentioned it in his talk). He seems to think the two are 
completely independent.


That's already thought through? Great, no need to "lockstep" the 
DIPs then and one curve less to worry about.

Re: DIP 1014--Hooking D's struct move semantics--Final Review

[Shachar Shemesh via Digitalmars-d]

On 12/07/18 04:17, Jonathan M Davis wrote:> I'm also> not sure if going 
to copy constructors means that we should do something> different with 
this. It don't think that it's affected by it, but I could be> missing 
something.


I actually had that very same concern myself. Andrei does not seem to 
share it (I talked to him about it during DConf, and he even mentioned 
it in his talk). He seems to think the two are completely independent.


Like I said, I'm not sure I completely agree. Some of the problems 
postblit have will also affect us here (specifically, what happens if 
you want to override a member's behavior, specifically when @disabled.


What tipped the scale for me was this: This DIP is relatively simple. It 
requires a few changes to all three (compiler, run time and phobos), but 
small changes to all three. Best of all, this change does not introduce 
what Andrei called "magic". The changes are easily lowered to existing D 
code.


Switching to a move-constructor type implementation will make all of 
those advantages disappear in a puff of bits.




However, I don't agree that opPostMove needs to be nothrow on the basis that
it might be confusing if it's not.


The problem here is that D moves *everywhere*, and it is often quite 
impossible to make it not move (believe me, I've tried).


With that said, downgrading this to a very hearty recommendation instead 
of a requirement is fine by me.



Also, as soon as we discuss overriding what happens when an object is moved,
that opens up the question of whether it should be allowed to be @disabled.
What happens when someone does

@disable opPostMove();


What happens today is that if you actually try to move the struct, you 
will get a compilation error. IMHO, this is the behavior we actually 
want to keep.


I can actually see some uses for such a step - if you simply cannot have 
that struct move, a compilation error is preferable to things breaking.


With that said, the moves are so integral to D's code generation that a 
struct with opPostMove @disabled would not be very useful. I am of the 
opinion that the programmer can make up their own mind on what to do 
about it.



However, I would argue that if we're going to take the step of allowing
the programmer to hook into the move process to do custom stuff that we
should also allow it to be outright disabled - the use case that comes to
mind at the moment being stuff like mutexes where the object can't safely be
moved, because it's not only not thread-safe, but it's probably not possible
with the mutex's C API, since you handed a pointer off to it and have no
control over what it does with it.


Like I said above, I completely agree.


Of course, that also opens the question of what should happen with a shared
opPostMove, since presumably a mutex would be in a shared object, and that
would then require that we finish working out shared - though I'd argue that
we would have to disallow moves on a shared object in order to be
thread-safe anyway.


That one is actually handled by the DIP.

opPostMove is called by the compiler when it has just moved the object. 
This means that, at least as far as the compiler is concerned, the 
object has no external references to it (or it couldn't move it).


If a struct's pointer is shared, and there are external pointers to 
update, it is up to the programmer to decide how to handle it (and, yes, 
@disable might be the only safe option).



By no means do I think that this DIP should be contingent on anything to do
with disallowing moving, but I do think that it if we're going to allow
mucking around with moves like this rather than simply leaving it up to the
compiler that we should seriously consider allowing it to be disabled -


I disagree. I think we should allow it to be disabled whether this DIP 
goes in or not. On the contrary: this DIP would allow cases to be 
handled where today they are simply not safe, no matter what you do. In 
other words, the cases where you'd want to disable move are only reduced 
by adopting this DIP.


Shachar

Re: DIP 1014--Hooking D's struct move semantics--Final Review

[Shachar Shemesh via Digitalmars-d]

On 29/06/18 15:35, aliak wrote:

On Wednesday, 27 June 2018 at 07:24:05 UTC, Mike Parker wrote:

On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote:


Thanks in advance for your participation.


For those of you using the NNTP or mailing list interfaces, this is 
the thread to respond in. Thanks!


Alo!

This is great!

Just a clarification about the last paragraph phrasing

The last line: "We can further reduce this problem by calling the 
function opPostMove." seemed to imply that an alternate name to 
opPostMove would be mentioned, but am I correct in understanding that it 
is just saying that "naming this second function as op* will keep code 
breakage to a minimum" ?


This is a left over from a previous draft, where the operator was called 
"opMove". It should be removed.


Also, what should happen if someone defines an opPostMove for a class. 
Compile error or? Should something about that be mentioned?


I think nothing should happen. The function would be ignored, just like 
it is today. I am open to hear other ideas, however.


I'm not sure whether it should be explicitly mentioned or not.

Shachar

Re: DIP 1014--Hooking D's struct move semantics--Final Review

[Shachar Shemesh via Digitalmars-d]

On 11/07/18 20:04, Johan Engelen wrote:

On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote:
DIP 1014, "Hooking D's struct move semantics", is now ready for final 
review.


after quick read:

(would be much easier to do inline commenting, but it appears that's not 
supported)


### Section "Code emitted by the compiler on move"
Dangerous to talk about what "code is emitted" by the compiler. I think 
this DIP doesn't need that, and semantics is enough. "the compiler MUST 
call " should be reworded, because an _actual_ call to the function 
should not be mandatory, because it would limit the optimizer in eliding 
it or inlining it (note that it will be hard to _prevent_ the optimizer 
from eliding/inlining the call as currently specified by the DIP).


I don't draw the same conclusions from the text as you.

I'm perfectly fine with specifying that nothing here is mandatory if the 
compiler ensures that *the end effect* is as if it happened.


AFAIK, C++ has a standing order to that effect, and it greatly 
simplifies documenting what you want to do.




### "__move_post_blt SHOULD be defined in a manner that is compatible"
What does "compatible" mean?


"Has the same effect as".

It's a little as if you're complaining about something not being 
explicit in one section, and again about that same thing being explicit 
in the next. Precisely why such standing order would be a good idea.


Some things should be made more explicit, 
such as the order of calls for compound structs.


I don't think it makes sense to specify the order, except to say that 
member's opPostMove must be called before the instance's opPostMove 
(which the code already says).



Why "SHOULD" and not "MUST"?


Precisely for the reason you stated above. So that if you want to do 
something else, you may.





### "This MUST return true iff a struct or any of its members have an 
opPostMove defined."
Doesn't cover struct A containing struct B containing struct C with 
opPostMove. Reword e.g. into: "hasElaborateMove!S MUST return true iff 
`S` has an `opPostMove` defined or if hasElaborateMove!X is true for any 
member of S of type X.


Yes, I'm sorry. I worded the spec for humans.

I can suggest a recursive definition:

hasElaborateMove for a struct MUST return true iff at least one of the 
following is true:

* The struct has opPostMove defined
* hasElaborateMove returns true for at least one of the struct's members.

I'm fine with this definition, and it resolves your concern, but I think 
it is less readable.


You are free to suggest a better way of phrasing this.




### What is lacking is a clear list of exactly in which cases 
`opPostMove` will be called (needed for user-facing documentation of the 
function).


I don't think I understand this point. Can you suggest what that list 
might contain?


Thank you,
Shachar

Re: DIP 1014--Hooking D's struct move semantics--Final Review

[Jonathan M Davis via Digitalmars-d]

On Wednesday, 27 June 2018 01:13:14 MDT Mike Parker via Digitalmars-d wrote:
> DIP 1014, "Hooking D's struct move semantics", is now ready for
> final review. This is a last chance for community feedback before
> the DIP is handed off to Walter and Andrei for the Formal
> Assessment. Please read the procedures document for details on
> what is expected in this review stage:
>
> https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#final-review
>
> The current revision of the DIP for this review is located here:
>
> https://github.com/dlang/DIPs/blob/01af4bb1e81993066bd1e7f77263002d3883fab
> f/DIPs/DIP1014.md
>
> In it you'll find a link to and summary of the previous review
> round. This round of review will continue until 11:59 pm ET on
> July 11th unless I call it off before then.
>
> Thanks in advance for your participation.


Looks pretty good overall, though the name of the __move_post_blt arguably
should be different if we're going to move to copy constructors. I'm also
not sure if going to copy constructors means that we should do something
different with this. It don't think that it's affected by it, but I could be
missing something.

However, I don't agree that opPostMove needs to be nothrow on the basis that
it might be confusing if it's not. Based on that argument, postblit
constructors should be required to be nothrow, and they don't have to be
nothrow. Copying and moving frequently look pretty much the same to the
programmer and may even depend on how the compiler is able to optimize the
code (e.g. it figures out that it can use a move instead of a copy). So,
while it probably is generally better for opPostMove to be nothrow, it
doesn't seem to me like it really should be required that it be nothrow. For
better or worse, we don't even require that destructors be nothrow. So, it
doesn't really fit for opPostMove to have to be nothrow. The motivation for
it is not at all technical in nature and the social aspect of it is already
contradicted by similar features which are allowed to throw.

Also, as soon as we discuss overriding what happens when an object is moved,
that opens up the question of whether it should be allowed to be @disabled.
What happens when someone does

@disable opPostMove();

For better or worse, @disable works on any function in D (even if it's kind
of pointless for functions that aren't constructors or operators). So, it's
going to need to be handled even if it's just making it an error to @disable
it. However, I would argue that if we're going to take the step of allowing
the programmer to hook into the move process to do custom stuff that we
should also allow it to be outright disabled - the use case that comes to
mind at the moment being stuff like mutexes where the object can't safely be
moved, because it's not only not thread-safe, but it's probably not possible
with the mutex's C API, since you handed a pointer off to it and have no
control over what it does with it.

Of course, that also opens the question of what should happen with a shared
opPostMove, since presumably a mutex would be in a shared object, and that
would then require that we finish working out shared - though I'd argue that
we would have to disallow moves on a shared object in order to be
thread-safe anyway. So, maybe the mutex use case doesn't ultimately matter,
since we're likely going to have to disable moves for shared anyway, but
shared and mutexes aside, any use case where you hand the pointer off to
something outside the object won't work with opPostMove, whereas it could be
made to work if we had opPostMove and were then allowed to @disable it.

By no means do I think that this DIP should be contingent on anything to do
with disallowing moving, but I do think that it if we're going to allow
mucking around with moves like this rather than simply leaving it up to the
compiler that we should seriously consider allowing it to be disabled -
especially when opPostMove gives us a really obvious syntax for it, and
while some of the motivations for this DIP are solved by it, others would
really require disallowing moves.

- Jonathan M Davis

Re: DIP 1014--Hooking D's struct move semantics--Final Review

[Johan Engelen via Digitalmars-d]

On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote:
DIP 1014, "Hooking D's struct move semantics", is now ready for 
final review.


after quick read:

(would be much easier to do inline commenting, but it appears 
that's not supported)


### Section "Code emitted by the compiler on move"
Dangerous to talk about what "code is emitted" by the compiler. I 
think this DIP doesn't need that, and semantics is enough. "the 
compiler MUST call " should be reworded, because an _actual_ call 
to the function should not be mandatory, because it would limit 
the optimizer in eliding it or inlining it (note that it will be 
hard to _prevent_ the optimizer from eliding/inlining the call as 
currently specified by the DIP). So this should be reworded such 
that the semantic effect is as if the function is called. Also 
unspecified _when_ the function needs to be called.



### "__move_post_blt SHOULD be defined in a manner that is 
compatible"
What does "compatible" mean? Some things should be made more 
explicit, such as the order of calls for compound structs.

Why "SHOULD" and not "MUST"?


### "This MUST return true iff a struct or any of its members 
have an opPostMove defined."
Doesn't cover struct A containing struct B containing struct C 
with opPostMove. Reword e.g. into: "hasElaborateMove!S MUST 
return true iff `S` has an `opPostMove` defined or if 
hasElaborateMove!X is true for any member of S of type X.



### What is lacking is a clear list of exactly in which cases 
`opPostMove` will be called (needed for user-facing documentation 
of the function).


-Johan

Re: DIP 1014--Hooking D's struct move semantics--Final Review

[Dukc via Digitalmars-d]

On Thursday, 5 July 2018 at 10:27:52 UTC, Dukc wrote:

The DIP looks well written. I'm in favour of it.


However, we need to consider how well this interacts with 
Razvan's DIP [1]. We should consider this together with it, so 
the implementations do not end up messing each other.


1: 
https://forum.dlang.org/thread/hhdtliynfwckcgafl...@forum.dlang.org

Re: DIP 1014--Hooking D's struct move semantics--Final Review

[Dukc via Digitalmars-d]

On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote:
DIP 1014, "Hooking D's struct move semantics", is now ready for 
final review.


Structs are a low level feature that should be able to be used in 
any way programmer sees fit. This is just what is wrong with C# 
structs: In principle they're the same as D structs but disallow 
many things for no obvious reason, thus limiting their usability. 
See my question and it's answers at Stack overflow to see what I 
mean: 
https://stackoverflow.com/questions/51098690/assigning-value-to-member-of-nullable-struct-in-c-sharp


This is a feature that is likely to be useful for low level 
programming, but is zero cost for those who don't need it. The 
DIP looks well written. I'm in favour of it.

Re: DIP 1014--Hooking D's struct move semantics--Final Review

[aliak via Digitalmars-d]

On Wednesday, 27 June 2018 at 07:24:05 UTC, Mike Parker wrote:

On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote:


Thanks in advance for your participation.


For those of you using the NNTP or mailing list interfaces, 
this is the thread to respond in. Thanks!


Alo!

This is great!

Just a clarification about the last paragraph phrasing

The last line: "We can further reduce this problem by calling the 
function opPostMove." seemed to imply that an alternate name to 
opPostMove would be mentioned, but am I correct in understanding 
that it is just saying that "naming this second function as op* 
will keep code breakage to a minimum" ?


Also, what should happen if someone defines an opPostMove for a 
class. Compile error or? Should something about that be mentioned?


Cheers
- Ali

Re: DIP 1014--Hooking D's struct move semantics--Final Review Begins

[Mike Parker via Digitalmars-d-announce]

On Wednesday, 27 June 2018 at 07:28:07 UTC, Mike Parker wrote:

On Wednesday, 27 June 2018 at 07:26:20 UTC, Mike Parker wrote:



https://forum.dlang.org/post/hwscnvmtrzfcfnppd...@forum.dlang.org


And I've pasted the wrong link here. The correct thread link is 
this one:


https://forum.dlang.org/post/ipidsodqhgazrfvzh...@forum.dlang.org

Thanks!


Nope. Wrong one again. This is it:

https://forum.dlang.org/post/rrdtjzjwanjlcnhhk...@forum.dlang.org

It's days like this I wish we had a proper forum.

Re: DIP 1014--Hooking D's struct move semantics--Final Review Begins

[Mike Parker via Digitalmars-d-announce]

On Wednesday, 27 June 2018 at 07:26:20 UTC, Mike Parker wrote:



https://forum.dlang.org/post/hwscnvmtrzfcfnppd...@forum.dlang.org


And I've pasted the wrong link here. The correct thread link is 
this one:


https://forum.dlang.org/post/ipidsodqhgazrfvzh...@forum.dlang.org

Thanks!

Re: DIP 1014--Hooking D's struct move semantics--Final Review

[Mike Parker via Digitalmars-d]

On Wednesday, 27 June 2018 at 07:13:14 UTC, Mike Parker wrote:


Thanks in advance for your participation.


For those of you using the NNTP or mailing list interfaces, this 
is the thread to respond in. Thanks!

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[Shachar Shemesh via Digitalmars-d]

On 18/05/18 22:57, kinke wrote:
I checked, and the reason is that D and C++ use a different ABI wrt. 
by-value passing of non-POD arguments. C++ indeed passes a reference to 
a caller-allocated rvalue, not just on Win64; that makes it trivial, as 
there are no moves across call boundaries. But your proposal may imply 
changing the D ABI accordingly.


That seems to be the case.

Assuming https://dlang.org/spec/abi.html is the ABI you refer to, there 
is very little in way of argument calling there:


https://dlang.org/spec/abi.html#function_calling_conventions

"
The extern (C) and extern (D) calling convention matches the C calling 
convention used by the supported C compiler on the host system. Except 
that the extern (D) calling convention for Windows x86 is described here.

"

So the current state is, in essence, that in C everything is PoD, and 
that's why that's also the case in D. Yes, something will need to change 
there.


Shachar

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[kinke via Digitalmars-d]

On Thursday, 17 May 2018 at 19:11:27 UTC, Shachar Shemesh wrote:

On 17/05/18 18:47, kinke wrote:
Since clang is able to compile this struct and do everything 
with it, and since the existence of the move constructor 
requires the precise same type of hooking as is needed in this 
case, I tend to believe that an IR representation of DIP 1014 
is possible.


I checked, and the reason is that D and C++ use a different ABI 
wrt. by-value passing of non-POD arguments. C++ indeed passes a 
reference to a caller-allocated rvalue, not just on Win64; that 
makes it trivial, as there are no moves across call boundaries. 
But your proposal may imply changing the D ABI accordingly.

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[Rubn via Digitalmars-d]

On Thursday, 17 May 2018 at 20:25:26 UTC, Shachar Shemesh wrote:
Obviously, Something can be an enum or a boolean. If it is, 
however, then we have to perform a condition to select the 
correct value. The problem with conditionals is that if the CPU 
misses a guess about what they are (and in our case, the CPU is 
going to miss about 50% of the time), they are extremely 
expensive to evaluate.


Performance wise, a much saner approach is:
alias Something = int*;

Of course, this means our struct now has a self referencing 
pointer.


What I'm getting at is that even if there are alternatives to 
structs pointing at themselves, they may not be performance 
wise comparable to pointers.



It's possible to do a branchless condition that chooses between 
two pointers. I think if the hardware (and compiler) support it 
it'll just optimize down to a "cmov".

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[Kagamin via Digitalmars-d]

On Thursday, 17 May 2018 at 19:13:48 UTC, Shachar Shemesh wrote:
The only inherent non @safe thing we advocate here is if you 
want to be able to move const/immutable structs, in which case 
DIP 1014 advocates casting the constness away. That will, of 
course, have to be either @system or @trusted.


There's an idea to give const postblit ability to write, but it's 
difficult to make such exception for operator method, a possible 
syntax can be `this(this, const ref S prev)`

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[Manu via Digitalmars-d]

On 17 May 2018 at 13:25, Shachar Shemesh via Digitalmars-d
 wrote:
> On 17/05/18 22:29, Manu wrote:
>>
>>
>> This is great!
>> I've wanted this on numerous occasions when interacting with C++ code.
>> This will make interaction more complete.
>>
>> Within self-contained D code, I have avoided self-pointers by using
>> self-offsets instead in the past (a bit hack-ey). But this nicely
>> tidies up one more little rough-edge in the language.
>>
>> :+1:
>>
>
> Following Andrei's advice, I've actually started writing a couple of
> examples to illustrate why this is needed.
>
> The first was this:
>
> struct S {
>   static int global;
>   int local;
>
>   Something selector; // Decides whether we want local or global.
> }
>
> Let's further assume that we have an array of S instances with random
> uniform distribution between local and global.
>
> Obviously, Something can be an enum or a boolean. If it is, however, then we
> have to perform a condition to select the correct value. The problem with
> conditionals is that if the CPU misses a guess about what they are (and in
> our case, the CPU is going to miss about 50% of the time), they are
> extremely expensive to evaluate.
>
> Performance wise, a much saner approach is:
> alias Something = int*;
>
> Of course, this means our struct now has a self referencing pointer.
>
> What I'm getting at is that even if there are alternatives to structs
> pointing at themselves, they may not be performance wise comparable to
> pointers.
>
> Of course, the second example was a struct that has no internal pointers,
> but rather maintains global pointers pointing to it. This problem is quite a
> bit harder to solve.
>
> Shachar

Oh yeah, I totally recognise that there are instances where a local
offset is not a solution, I was just saying how I've managed to
avoided dealing with this issue before, and not that I loved doing so
that way ;)
This would be a very welcome fix, particularly in code where I
interact with C++!

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[Shachar Shemesh via Digitalmars-d]

On 17/05/18 22:29, Manu wrote:


This is great!
I've wanted this on numerous occasions when interacting with C++ code.
This will make interaction more complete.

Within self-contained D code, I have avoided self-pointers by using
self-offsets instead in the past (a bit hack-ey). But this nicely
tidies up one more little rough-edge in the language.

:+1:



Following Andrei's advice, I've actually started writing a couple of 
examples to illustrate why this is needed.


The first was this:

struct S {
  static int global;
  int local;

  Something selector; // Decides whether we want local or global.
}

Let's further assume that we have an array of S instances with random 
uniform distribution between local and global.


Obviously, Something can be an enum or a boolean. If it is, however, 
then we have to perform a condition to select the correct value. The 
problem with conditionals is that if the CPU misses a guess about what 
they are (and in our case, the CPU is going to miss about 50% of the 
time), they are extremely expensive to evaluate.


Performance wise, a much saner approach is:
alias Something = int*;

Of course, this means our struct now has a self referencing pointer.

What I'm getting at is that even if there are alternatives to structs 
pointing at themselves, they may not be performance wise comparable to 
pointers.


Of course, the second example was a struct that has no internal 
pointers, but rather maintains global pointers pointing to it. This 
problem is quite a bit harder to solve.


Shachar

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[Manu via Digitalmars-d]

On 17 May 2018 at 01:12, Mike Parker via Digitalmars-d
 wrote:
> This is the review thread for the first Community Review round for DIP 1014,
> "Hooking D's struct move semantics".
>
> All review-related feedback on and discussion of the DIP should occur in
> this thread. The review period will end at 11:59 PM ET on May 31, or when I
> make a post declaring it complete.
>
> At the end of Round 1, if further review is deemed necessary, the DIP will
> be scheduled for another round. Otherwise, it will be queued for the formal
> review and evaluation by the language maintainers.
>
> Please familiarize yourself with the documentation for the Community Review
> before participating.
>
> https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#community-review
>
> Thanks in advance to all who participate.
>
> Destroy!

This is great!
I've wanted this on numerous occasions when interacting with C++ code.
This will make interaction more complete.

Within self-contained D code, I have avoided self-pointers by using
self-offsets instead in the past (a bit hack-ey). But this nicely
tidies up one more little rough-edge in the language.

:+1:

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[Shachar Shemesh via Digitalmars-d]

On 17/05/18 19:08, Kagamin wrote:

On Thursday, 17 May 2018 at 13:50:26 UTC, Shachar Shemesh wrote:
There is no such use case. Please remember that at the time opPostMove 
is called, both new and old memory are still allocated.


That's an interesting moment too. A struct that was supposed to be moved 
is copied instead and exists in two places simultaneously. Can't tell it 
yet, but it can have a hole in type system and opPostMove can only be 
trusted, not safe.


It is a hole (of sorts) in the type system, in that "old" is not going 
to have a destructor run on its code.


With that said, just because the code is not safe, does not mean it is 
not @safe.


The only inherent non @safe thing we advocate here is if you want to be 
able to move const/immutable structs, in which case DIP 1014 advocates 
casting the constness away. That will, of course, have to be either 
@system or @trusted.


Shachar

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[Shachar Shemesh via Digitalmars-d]

On 17/05/18 18:47, kinke wrote:

On Thursday, 17 May 2018 at 15:23:50 UTC, kinke wrote:

See IR for https://run.dlang.io/is/1JIsk7.


I should probably emphasize that the LLVM `byval` attribute is strange 
at first sight. Pseudo-IR `void foo(S* byval param); ... foo(S* byarg 
arg);` doesn't mean that the IR callee gets the S* pointer from the IR 
callsite; it means 'memcpy(param, arg, S.sizeof)', with `param` being an 
*implicit* address in foo's parameters stack (calculated by LLVM and so 
exposed to the callee only). That's the difficulty for LDC I mentioned 
earlier.


I understand there might be difficulty, but I strongly protest the idea 
that it is not possible, for one very simple reason: C++.


class Movable {
  int member;

public:
  Movable();

  Movable( const Movable  ); // Copy constructor
  Movable( Movable & ); // Move constructor
}

Since clang is able to compile this struct and do everything with it, 
and since the existence of the move constructor requires the precise 
same type of hooking as is needed in this case, I tend to believe that 
an IR representation of DIP 1014 is possible.


Shachar

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[Kagamin via Digitalmars-d]

On Thursday, 17 May 2018 at 13:50:26 UTC, Shachar Shemesh wrote:
There is no such use case. Please remember that at the time 
opPostMove is called, both new and old memory are still 
allocated.


That's an interesting moment too. A struct that was supposed to 
be moved is copied instead and exists in two places 
simultaneously. Can't tell it yet, but it can have a hole in type 
system and opPostMove can only be trusted, not safe.

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[kinke via Digitalmars-d]

On Thursday, 17 May 2018 at 15:23:50 UTC, kinke wrote:

See IR for https://run.dlang.io/is/1JIsk7.


I should probably emphasize that the LLVM `byval` attribute is 
strange at first sight. Pseudo-IR `void foo(S* byval param); ... 
foo(S* byarg arg);` doesn't mean that the IR callee gets the S* 
pointer from the IR callsite; it means 'memcpy(param, arg, 
S.sizeof)', with `param` being an *implicit* address in foo's 
parameters stack (calculated by LLVM and so exposed to the callee 
only). That's the difficulty for LDC I mentioned earlier.

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[kinke via Digitalmars-d]

On Thursday, 17 May 2018 at 12:36:29 UTC, Shachar Shemesh wrote:
Again, as far as I know, structs are not copied when passed as 
arguments. They are allocated on the caller's stack and a 
reference is passed to the callee. If that's the case, no move 
(of any kind) is done.


That's the exception to the rule (LDC's `ExplicitByvalRewrite`), 
and true for structs > 64 bit on Win64 (and some more structs) 
and something similar for AArch64. No other ABIs supported by LDC 
pass a low-level pointer to a caller-allocated copy for 
high-level pass-argument-by-value semantics; the argument is 
normally moved to the function parameter (in the callEE 
parameters stack).


```
struct S
{
size_t a, b;
this(this) {} // no POD anymore
}

void foo(S param);

void bar()
{
// allocate a temporary on the caller's stack and move it to 
the callee

foo(S(1, 2));

S lvalue;
// copy lvalue to a temporary on the caller's stack (incl. 
postblit call)

// and then move that temporary to the callee
foo(lvalue);

import std.algorithm.mutation : move;
// move move()-rvalue-result to the callee
foo(move(lvalue));
}
```

'Move to callee' for most ABIs meaning a bitcopy/blit to the 
callee's memory parameters stack, for LDC via LLVM `byval` 
attribute.


See IR for https://run.dlang.io/is/1JIsk7.

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[Shachar Shemesh via Digitalmars-d]

On 17/05/18 16:42, Kagamin wrote:
Looks like requirement for @nogc @safe has no consequence as the DIP 
suggests to infer them anyway. On ideological side safety can't be a 
requirement because it would contradict its purpose of providing 
guarantee.


I think you are confusing __move_post_blt's implementation (by druntime) 
with opPostMove's implementation by the user.


For the former, these attributes are deducted by the compiler. For the 
later, the user may choose to include them for all of the usual reasons 
for including them, not least of which is that if she does not include 
@nogc, then trying to move a struct's instance from @nogc code will 
cause compilation failure.


> Especially if the suggested use case is handling of dangling
> pointers.

There is no such use case. Please remember that at the time opPostMove 
is called, both new and old memory are still allocated.


Shachar

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[Kagamin via Digitalmars-d]

Looks like requirement for @nogc @safe has no consequence as the 
DIP suggests to infer them anyway. On ideological side safety 
can't be a requirement because it would contradict its purpose of 
providing guarantee. Especially if the suggested use case is 
handling of dangling pointers.

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[Shachar Shemesh via Digitalmars-d]

I'm not sure I follow all of your comments.

For the rest my comments, let's assume that the compiler may assume that 
__move_post_blt is a no-op if hasElaborateMove returns false.


On 17/05/18 14:33, kinke wrote:
3. When deciding to move a struct instance, the compiler MUST emit a 
call to the struct's __move_post_blt after blitting the instance and 
before releasing the memory containing the old instance. 
__move_post_blt MUST receive references to both the pre- and post-move 
instances.


This implies that such structs must not be considered PODs, i.e., cannot 
be passed in registers and must be passed on the stack.


I'm not familiar with passing structs in registers. I am familiar with 
passing pointer to the structs in registers, which should not be 
affected by this.


If actual (I'm assuming short) structs can be passed in registers, then, 
yes, structs with elaborate move are not PoDs.


It also means 
that the compiler will have to insert a __move_post_blt call right 
before the call (as the callee has no idea about the old address),


Again, as far as I know, structs are not copied when passed as 
arguments. They are allocated on the caller's stack and a reference is 
passed to the callee. If that's the case, no move (of any kind) is done.


I might be missing something. Can you write a code snippet to which you 
are referring?


after 
blitting the arg to the callee params stack; this may be tricky to 
implement for LDC, as that last blit is implicit in LLVM IR (LLVM byval 
attribute).


And yet, C++ clang managed to do it in classes with && constructors.


As a side note, when passing a postblit-struct lvalue arg by value,


Just to be clear, are we talking about passing by reference an instance 
of a struct that has postblit?


the 
compiler first copies the lvalue



to a temporary on the caller's stack, 
incl. postblit call, and then moves that copy to the callee. So this 
requires either a postblit+postmove combo on the caller side before the 
actual call, or a single postblit call for the final address (callee's 
param).


Again, that does not align with my familiarity of the ABI. If I do:

struct S {
  ...

  this(this) {
// some code
  }

  void opPostMove(ref S new, const ref S old) {
// some code
  }
}

void func1(ref S arg) {
}

void func2(S arg) {
}


As far as I know the ABI, in func1, a pointer to S is passed. In func2, 
a pointer to caller stack allocate instance is passed, and the original 
is copied in. It sounds to me like you are talking about the case of:


S s;

func2(s);

in which case you need to copy s to a temporary, and then pass a pointer 
to that temporary to func2. I don't see where a move enters here.


However, if a move does enter here (and one is necessary if, for 
example, func2's frame needs to be dynamically allocated because an 
escaping delegate references it), then, yes, an opPostMove will also 
need to be called.


Again, if hasElaborateMove returns false, then no change from current 
behavior is needed.


Shachar

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[kinke via Digitalmars-d]

3. When deciding to move a struct instance, the compiler MUST 
emit a call to the struct's __move_post_blt after blitting the 
instance and before releasing the memory containing the old 
instance. __move_post_blt MUST receive references to both the 
pre- and post-move instances.


This implies that such structs must not be considered PODs, i.e., 
cannot be passed in registers and must be passed on the stack. It 
also means that the compiler will have to insert a 
__move_post_blt call right before the call (as the callee has no 
idea about the old address), after blitting the arg to the callee 
params stack; this may be tricky to implement for LDC, as that 
last blit is implicit in LLVM IR (LLVM byval attribute).


As a side note, when passing a postblit-struct lvalue arg by 
value, the compiler first copies the lvalue to a temporary on the 
caller's stack, incl. postblit call, and then moves that copy to 
the callee. So this requires either a postblit+postmove combo on 
the caller side before the actual call, or a single postblit call 
for the final address (callee's param).

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[Shachar Shemesh via Digitalmars-d]

On 17/05/18 11:22, rikki cattermole wrote:


What is the benefit of opPostMove over copy constructors (not postblit)?


The two are unrelated. A copy is a very different operation from a move. 
With a copy, you have to figure out how to duplicate the resources used 
by the object. With a move, no such duplication is needed.


People are somewhat conditioned to treat a move as a "copy+destroy". One 
certainly may implement it that way. There is a lot of power in not 
having to do it, though. Think a struct with "@disable this(this)".


The D compiler does moves of structs, whether they are copyable or not. 
This DIP is about being able to hook this move and fix external and 
internal references.


Shachar

Re: DIP 1014: Hooking D's struct move semantics -- Community Review Round 1 Begins

[rikki cattermole via Digitalmars-d-announce]

On 17/05/2018 8:15 PM, Mike Parker wrote:
The first round of Community Review for DIP 1014, "Hooking D's struct 
move semantics", has begun. To participate, please visit the review 
thread for the details and leave all feedback there rather than here:


https://forum.dlang.org/post/zfyfhqczkrfdpfkca...@forum.dlang.org


What is the benefit of opPostMove over copy constructors (not postblit)?

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[rikki cattermole via Digitalmars-d]

On 17/05/2018 8:12 PM, Mike Parker wrote:
This is the review thread for the first Community Review round for DIP 
1014, "Hooking D's struct move semantics".


All review-related feedback on and discussion of the DIP should occur in 
this thread. The review period will end at 11:59 PM ET on May 31, or 
when I make a post declaring it complete.


At the end of Round 1, if further review is deemed necessary, the DIP 
will be scheduled for another round. Otherwise, it will be queued for 
the formal review and evaluation by the language maintainers.


Please familiarize yourself with the documentation for the Community 
Review before participating.


https://github.com/dlang/DIPs/blob/master/PROCEDURE.md#community-review

Thanks in advance to all who participate.

Destroy!


What is the benefit of opPostMove over copy constructors (not postblit)?

Re: DIP 1014:Hooking D's struct move semantics--Community Review Round 1

[Mike Parker via Digitalmars-d]

On Thursday, 17 May 2018 at 08:12:50 UTC, Mike Parker wrote:
This is the review thread for the first Community Review round 
for DIP 1014, "Hooking D's struct move semantics".


And the link to the DIP:

https://github.com/dlang/DIPs/blob/38cec74a7471735559e3b8a7553f55102d289d28/DIPs/DIP1014.md