Re: Implicit conversions through purity

[Steve Teale]

On Tuesday, 15 April 2014 at 18:02:00 UTC, bearophile wrote:

Steve Teale:

Since this is D-Learn, I can be indignant, and say that D 
needs to get its act together, and have a clean definition of 
'pure'. What you describe is not only undocumented, but also 
far too complicated - pure weak nothrow dontpiss kissmyass 
@never, and so on if the direction continues.


There is a nice article on D purity that I suggest you to read. 
Unfortunately I don't remember the link.


To design a system language as complex as C++/D you need lot of 
mathematics and theory. Andrei is good in computer science 
theory and in complex programming too, but Walter is less 
mathematically-minded and this could be visible in the design 
of some parts of D. Anyway, everyone is doing their best, 
Walter is very good and he has all my respect. And in a 
real-world system language a clean definition of pure (like the 
original pure implementation of D) is not very useful. The 
current definition of pure of D is good enough and it's a part 
very near to be completely implemented (unlike many other parts 
of D like synchronized, scope, SIMD, vector operations, 
operator overloading, dynamic libraries, GC, and more). So 
please don't be too much indignant, we don't have the research 
laboratories of Microsoft :-)



Nontheless, thank you for your assiduous efforets to make D 
internally consistent.


In the end I am not doing much. At best I can only hope to spot 
some sharp corners of the language, and ask for them to be 
smoothed. But it's very hard to make progress. And lately 
Andrei (perhaps rightfully) has raised the bar regarding the 
acceptable breaking changes. And people like Kenji have a mind 
better than mine (able to think about tens of corner cases, 
able to keep in mind many complex interactions, etc) even when 
they are sleeping :-)


Bye,
bearophile


Like I said, I allowed myself the luxury of being indignant 
because we were hidden away on D Learn. I have total respect for 
Walter. For one thing he is a self-taught programmer like me, 
though much more advanced, and I've known him on and off since 
the Zortech days in 1988. I don't much about Andrei and Kenji, 
except that they are out of my league.


I guess that rather than complain that the compiler does not 
appear to conform to the definition, I should help by trying to 
improve the documentation.


Anyway my view on the original subject is that the implicit 
conversion should be allowed for the out case, but not the ref.


Steve

Re: Implicit conversions through purity

[Jonathan M Davis]

On Tuesday, April 15, 2014 18:01:59 bearophile wrote:
 Steve Teale:
  Since this is D-Learn, I can be indignant, and say that D needs
  to get its act together, and have a clean definition of 'pure'.
  What you describe is not only undocumented, but also far too
  complicated - pure weak nothrow dontpiss kissmyass @never, and
  so on if the direction continues.
 
 There is a nice article on D purity that I suggest you to read.
 Unfortunately I don't remember the link.

http://klickverbot.at/blog/2012/05/purity-in-d/

The key thing to understand about pure though is that _all_ it guarantees is 
that the function cannot access mutable, global variables or call any other 
functions which are not pure. The only variables that it can access which are 
either global and guaranteed to never change their values after initialization 
(which generally means that they're immutable) and the ones which are passed 
in directly to the function or which can be accessed via the variables that 
are passed in.

So, really, pure has _nothing_ to do with mathematical purity. It's just 
protecting against access to global variables. Now, that restriction allows a 
number of other things to be derived from that, which allows us to do cool 
stuff - including optimizing functions which _are_ pure in the mathematical 
sense. The most basic distinction would be what is typically called weak or 
strong purity, where strongly pure functions have immutable parameters (and 
thus multiple calls with the same arguments can be optimized to a single 
call), and weakly pure functions are pure functions which aren't strongly 
pure. Strongly pure functions generally can be optimized, whereas weakly pure 
functions cannot, but the actual distinctions can get more complicated than 
that, so even talking about weak vs strong purity isn't actually clear enough.

So, it's certainly true that things get a bit complicated when discussing what 
can be inferred due to the fact that a function is pure. And it doesn't always 
come down to strong vs weak purity either. So, I wouldn't expect very many 
people to understand under what circumstances the compiler is able to do 
certain things with a pure function, and that's not necessarily a good thing.

However, the key thing to understand with pure itself is that all it means is 
that the function can't access global, mutable state or call other functions 
which aren't pure. Pretty much everything else about it is essentially either 
a compiler optimization based on that information or an implicit type 
conversion which is made legal based on that information.

Regardless, I'd advise reading David's blog post on the matter, which I linked 
to above.

- Jonathan M Davis

Re: Implicit conversions through purity

[Steve Teale]

On Monday, 14 April 2014 at 15:16:07 UTC, bearophile wrote:

Steven Schveighoffer:

For that reason, I would disallow out parameters from casting 
implicitly.


A number of things perplex me here.

1) If I attempt to compile foo2() more or less as presented with 
2.065, the compiler tells me:


Error: '_adDupT' is not nothrow
Error: function 'mu.foo2' is nothrow yet may throw

What version is the discussion about?

2) If I have a simple program as follows, the compiler does not 
complain about me altering the value of global a.


import std.stdio;
string a = aaa;

void foo2(in string s, ref string sOut) pure {
auto s2 = s.dup;
s2[0] = 'a';
sOut = cast(string) s2; // Error: cannot implicitly convert
}

void main() {
{
   foo2(xyz, a);
   writeln(a);
}

3) Using a ref parameter for a pure function seems to me to be a 
clear indication of intended side effect. Wikipedia on pure 
functions says it's not allowed. Out does seem somehow different 
to me, since it's initial value is by definition throw-away.


Steve

Re: Implicit conversions through purity

[bearophile]

Steve Teale:

1) If I attempt to compile foo2() more or less as presented 
with 2.065, the compiler tells me:


Error: '_adDupT' is not nothrow
Error: function 'mu.foo2' is nothrow yet may throw

What version is the discussion about?


This rather fastidious limitation was finally removed in the 
latest D alpha (and if you look in Bugzilla you see it has 
created few regressions).



2) If I have a simple program as follows, the compiler does not 
complain about me altering the value of global a.


import std.stdio;
string a = aaa;

void foo2(in string s, ref string sOut) pure {
auto s2 = s.dup;
s2[0] = 'a';
sOut = cast(string) s2; // Error: cannot implicitly convert
}

void main() {
{
   foo2(xyz, a);
   writeln(a);
}


Yes, foo2 is weakly pure, but main is not tagged as pure, so main 
is free to use a global reference. If you mark main pure, your 
code will not compile even if you comment out the writeln. D is 
working as designed here.



3) Using a ref parameter for a pure function seems to me to be 
a clear indication of intended side effect.


In D you can also have const ref. A mutable ref makes the 
function weakly pure at best.




Wikipedia on pure functions says it's not allowed.


D has both strongly pure and weakly pure functions (and later the 
compiler has added const-ly pure and another category, all 
invisible.




Out does seem somehow different to me, since it's initial
value is by definition throw-away.


Yes, this is true in theory. I don't know if this is also true in 
D in practice, because D out has some problems.


Bye,
bearophile

Re: Implicit conversions through purity

[Steve Teale]

On Tuesday, 15 April 2014 at 09:41:57 UTC, bearophile wrote:
Yes, foo2 is weakly pure, but main is not tagged as pure, so 
main is free to use a global reference. If you mark main pure, 
your code will not compile even if you comment out the writeln. 
D is working as designed here.



3) Using a ref parameter for a pure function seems to me to be 
a clear indication of intended side effect.


In D you can also have const ref. A mutable ref makes the 
function weakly pure at best.




Wikipedia on pure functions says it's not allowed.


D has both strongly pure and weakly pure functions (and later 
the compiler has added const-ly pure and another category, 
all invisible.




Out does seem somehow different to me, since it's initial
value is by definition throw-away.


Yes, this is true in theory. I don't know if this is also true 
in D in practice, because D out has some problems.


Bye,
bearophile


Since this is D-Learn, I can be indignant, and say that D needs 
to get its act together, and have a clean definition of 'pure'. 
What you describe is not only undocumented, but also far too 
complicated - pure weak nothrow dontpiss kissmyass @never, and so 
on if the direction continues.


Nontheless, thank you for your assiduous efforets to make D 
internally consistent.


Steve

Re: Implicit conversions through purity

[bearophile]

Steve Teale:

Since this is D-Learn, I can be indignant, and say that D needs 
to get its act together, and have a clean definition of 'pure'. 
What you describe is not only undocumented, but also far too 
complicated - pure weak nothrow dontpiss kissmyass @never, and 
so on if the direction continues.


There is a nice article on D purity that I suggest you to read. 
Unfortunately I don't remember the link.


To design a system language as complex as C++/D you need lot of 
mathematics and theory. Andrei is good in computer science theory 
and in complex programming too, but Walter is less 
mathematically-minded and this could be visible in the design of 
some parts of D. Anyway, everyone is doing their best, Walter is 
very good and he has all my respect. And in a real-world system 
language a clean definition of pure (like the original pure 
implementation of D) is not very useful. The current definition 
of pure of D is good enough and it's a part very near to be 
completely implemented (unlike many other parts of D like 
synchronized, scope, SIMD, vector operations, operator 
overloading, dynamic libraries, GC, and more). So please don't be 
too much indignant, we don't have the research laboratories of 
Microsoft :-)



Nontheless, thank you for your assiduous efforets to make D 
internally consistent.


In the end I am not doing much. At best I can only hope to spot 
some sharp corners of the language, and ask for them to be 
smoothed. But it's very hard to make progress. And lately Andrei 
(perhaps rightfully) has raised the bar regarding the acceptable 
breaking changes. And people like Kenji have a mind better than 
mine (able to think about tens of corner cases, able to keep in 
mind many complex interactions, etc) even when they are sleeping 
:-)


Bye,
bearophile

Re: Implicit conversions through purity

[Jonathan M Davis]

On Sunday, April 13, 2014 01:52:13 bearophile wrote:
 Jonathan M Davis:
  Honestly, I would have considered that to be a bug. Converting
  the return type
  to a different level of mutability based on purity is one
  thing. Automatically
  casting the return value just because the function is pure is
  another matter
  entirely. Clearly, it can work, but it seems incredibly sloppy
  to me.
 
 In foo1 D is working as designed, as this was a desired feature,
 it has passed the Kenji and Walter review, and it was implemented
 several months ago. It's a very handy way to create immutable
 data with pure functions and it's safe, it's safer than
 assumeUnique that is just a convention. Very recently Walter has
 further improved this feature, allowing more implicit conversion
 cases. So it's the opposite of a bug, it saves you from bugs in
 three different ways.

Well, it's the first I've heard of it, and I certainly don't like the idea, 
but if it's intended and implemented, then that's the way it is, I guess. 
Maybe I'll come to agree after thinking about it more.

 But my question was about the successive foo functions :-)

Well, if I don't like the first example, I'm not about to be in favor of 
making more examples follow suite.

- Jonathan M Davis

Re: Implicit conversions through purity

[Jonathan M Davis]

On Sunday, April 13, 2014 21:21:02 Daniel Murphy wrote:
 Jonathan M Davis  wrote in message
 news:mailman.112.1397351369.5999.digitalmars-d-le...@puremagic.com...
 
  Honestly, I would have considered that to be a bug. Converting the return
  type
  to a different level of mutability based on purity is one thing.
  Automatically
  casting the return value just because the function is pure is another
  matter
  entirely. Clearly, it can work, but it seems incredibly sloppy to me.
 
 It's not a bug, and it's not another matter entirely - it's the same thing
 as converting a call expression, just on the inside instead of the outside.

Doing the conversion on the caller's side enables code that wouldn't otherwise 
work - particularly if you're not intimately familiar with what guarantees 
pure really gives you. Doing the conversion inside the function doesn't really 
buy you much of anything IMHO and promotes being lazy with types. It may not 
be a big deal, but I don't think that it's really a good idea either.

So, while from the perspective of what the compiler can guarantee, it may be 
the same from both sides, I don't think that it's the same at all with regards 
to how it affects the programmer or what code they need to write (or should 
write).

- Jonathan M Davis

Re: Implicit conversions through purity

[Steven Schveighoffer]

On Mon, 14 Apr 2014 06:37:18 -0400, Jonathan M Davis jmdavisp...@gmx.com  
wrote:



On Sunday, April 13, 2014 01:52:13 bearophile wrote:

Jonathan M Davis:
 Honestly, I would have considered that to be a bug. Converting
 the return type
 to a different level of mutability based on purity is one
 thing. Automatically
 casting the return value just because the function is pure is
 another matter
 entirely. Clearly, it can work, but it seems incredibly sloppy
 to me.

In foo1 D is working as designed, as this was a desired feature,
it has passed the Kenji and Walter review, and it was implemented
several months ago. It's a very handy way to create immutable
data with pure functions and it's safe, it's safer than
assumeUnique that is just a convention. Very recently Walter has
further improved this feature, allowing more implicit conversion
cases. So it's the opposite of a bug, it saves you from bugs in
three different ways.


Well, it's the first I've heard of it, and I certainly don't like the  
idea,

but if it's intended and implemented, then that's the way it is, I guess.
Maybe I'll come to agree after thinking about it more.


Here is how I consider it...

The foo1 function for reference:


string foo1(in string s) pure nothrow {
 auto s2 = s.dup;
 s2[0] = 'a';
 return s2; // OK.
}


The compiler accepts only immutable references. Any mutable references  
inside a pure function that only accepts immutable references can ONLY be  
data that is uniquely referenced from this function. In other words,  
there's no possible way that data is referenced outside this function,  
because a pure function cannot access globals.


So it's logical to assume that any mutable data inside the function can be  
implicitly cast to immutable. In fact, I would say in general, mutable  
data can be cast to immutable inside any pure function that only accepts  
immutable parameters. However, it cannot use the mutable references after  
casting. This would confuse the optimizer, which thinks that immutable  
data is still immutable.


For that reason, I would disallow out parameters from casting implicitly.

e.g.:

pure string mkcopy(string s) pure nothrow { return s.idup; }

void foo2(in string s, ref string r, ref string r2) pure nothrow {
auto s2 = s.dup;
r2 = s2;
auto s3 = mkcopy(r2); // should be a pure copy of s
s2[0] = 'a'; // modified immutable data referenced by r2!
r = mkcopy(r2); // ???
}

At the line marked ???, note that the compiler has already called mkcopy  
on r2, which hasn't itself changed, and data r2 references is immutable.  
The compiler is fully allowed to change that line to:


r = s3;

which would mean that r != r2, even though the last line says otherwise.

I think the point of restricting to the return value is not only that you  
are sure nothing else can refer to the data, but also that nothing else  
happens to the mutable copy after the cast.


I wonder if scope(exit) usage could compromise the current rule...

-Steve

Re: Implicit conversions through purity

[bearophile]

Steven Schveighoffer:

For that reason, I would disallow out parameters from casting 
implicitly.


If you think the proposal is not good, can you please copy what 
you have written here in the issue thread?

https://issues.dlang.org/show_bug.cgi?id=12573

Bye,
bearophile

Re: Implicit conversions through purity

[Daniel Murphy]

Jonathan M Davis  wrote in message 
news:mailman.112.1397351369.5999.digitalmars-d-le...@puremagic.com...


Honestly, I would have considered that to be a bug. Converting the return 
type
to a different level of mutability based on purity is one thing. 
Automatically
casting the return value just because the function is pure is another 
matter

entirely. Clearly, it can work, but it seems incredibly sloppy to me.


It's not a bug, and it's not another matter entirely - it's the same thing 
as converting a call expression, just on the inside instead of the outside. 

Re: Implicit conversions through purity

[Daniel Murphy]

bearophile  wrote in message news:hxdrbyqrhwvuochlh...@forum.dlang.org... 

Is it possible and a good idea to allow code like the function 
foo2?


It seems reasonable.

Re: Implicit conversions through purity

[bearophile]

Daniel Murphy:


It seems reasonable.


OK, added:
https://issues.dlang.org/show_bug.cgi?id=12573

Bye,
bearophile

Implicit conversions through purity

[bearophile]

Is it possible and a good idea to allow code like the function 
foo2?



string foo1(in string s) pure nothrow {
auto s2 = s.dup;
s2[0] = 'a';
return s2; // OK.
}

void foo2(in string s, ref string sOut) pure nothrow {
auto s2 = s.dup;
s2[0] = 'a';
sOut = s2; // Error: cannot implicitly convert
}

void foo3(in string s, out string sOut) pure nothrow {
auto s2 = s.dup;
s2[0] = 'a';
sOut = s2; // Error: cannot implicitly convert
}

void main() {}


Bye,
bearophile

Re: Implicit conversions through purity

[Jonathan M Davis]

On Saturday, April 12, 2014 21:26:01 bearophile wrote:
 Is it possible and a good idea to allow code like the function
 foo2?
 
 
 string foo1(in string s) pure nothrow {
  auto s2 = s.dup;
  s2[0] = 'a';
  return s2; // OK.
 }

Honestly, I would have considered that to be a bug. Converting the return type 
to a different level of mutability based on purity is one thing. Automatically 
casting the return value just because the function is pure is another matter 
entirely. Clearly, it can work, but it seems incredibly sloppy to me.

- Jonathan M Davis

Re: Implicit conversions through purity

[bearophile]

Jonathan M Davis:

Honestly, I would have considered that to be a bug. Converting 
the return type
to a different level of mutability based on purity is one 
thing. Automatically
casting the return value just because the function is pure is 
another matter
entirely. Clearly, it can work, but it seems incredibly sloppy 
to me.


In foo1 D is working as designed, as this was a desired feature,
it has passed the Kenji and Walter review, and it was implemented
several months ago. It's a very handy way to create immutable
data with pure functions and it's safe, it's safer than
assumeUnique that is just a convention. Very recently Walter has
further improved this feature, allowing more implicit conversion
cases. So it's the opposite of a bug, it saves you from bugs in
three different ways.

But my question was about the successive foo functions :-)

Bye,
bearophile