Re: From slices to perfect imitators: opByValue
On 05/09/14 01:05, Sönke Ludwig via Digitalmars-d wrote:
Am 09.05.2014 00:02, schrieb Timon Gehr:
On 05/08/2014 06:30 PM, Sönke Ludwig wrote:
Am 08.05.2014 18:10, schrieb Timon Gehr:
On 05/08/2014 06:02 PM, monarch_dodra wrote:
If you have const data referencing mutable data, then yes, you can cast
away all the const you want, but at that point, it kind of makes the
whole const thing moot.
This is not guaranteed to work. I guess the only related thing that is
safe to do is casting away const, but then not modifying the memory.
For what practical reason would that be the case? I know that the spec
states undefined behavior,
Case closed.
Bonus points for getting the difference between practical and
theoretical...
It's not just theoretical. This D program:
void main() {
const a = 1;// The same program works w/ auto a = 1;
auto p = cast(int *)a;
++*p;
import std.stdio;
writeln(a); // Oops. What if this was the refcount?
writeln(*p);
}
is enough to see that 'const' vs 'auto' actually affects the result;
no need for a mythical sufficiently advanced compiler.
Yes, todays compilers often do not take advantage of the const info
when an indirection is involved - this is because it's then harder to
prove that there are no other (mutable) aliases to the data. (This is
one reason for the malloc function attribute; unique-expressions will
also help)
artur
Re: From slices to perfect imitators: opByValue
void main()
{
DemoStruct m;
test(m);
acceptor(immutable) i;
I mean:
acceptor(immutable) DemoStruct i
test(i);
}
Re: From slices to perfect imitators: opByValue
On Thursday, 8 May 2014 at 21:08:36 UTC, bearophile wrote: I think T1 and T2 should be equivalent for built-in tuples. There are no built-in tuples in D.
Re: From slices to perfect imitators: opByValue
On Thursday, 8 May 2014 at 11:05:20 UTC, monarch_dodra wrote:
On Thursday, 8 May 2014 at 07:09:24 UTC, Sönke Ludwig wrote:
Just a general note: This is not only interesting for
range/slice types, but for any user defined reference type
(e.g. RefCounted!T or Isolated!T).
Not necessarily: As soon as indirections come into play, you
are basically screwed, since const is turtles all the way
down.
So for example, the conversion from const RefCounted!T to
RefCounted!(const T) is simply not possible, because it
strips the const-ness of the ref count.
What we would *really* need here is NOT:
const RefCounted!T = RefCounted!(const T)
But rather
RefCounted!T = RefCounted!(const T)
The idea is to cut out the head const directly. This also
applies to most ranges too BTW.
Skip paragraph.
Okay daedalnix. Second attempt. Started with
Container!(const(T)). Thought about
separating the the const. Container!(T, const) and then only one
const. None of
that Container!(A,B, immutable, const). Then thought about int
qual(*) * a.
With qual as entry point. Then decided to go tail const only,
single head mutable. But then seeing that there are two cases
above, decided to
go acceptor, copy.
---
a) I'm going to call this the copying case where the value types
are copied
const RefCounted!T = RefCounted!(const T)
immutable int [] a = immutable (int) [] a;
immutable to mutable
b) Acceptor case.
RefCounted!T = RefCounted!(const T)
e.g a const field accepts the mutable or immutable field.
---
struct DemoStruct
{
int * * a;
acceptor int * * b;
void demonstrate() acceptor
{
assert(typeof(a).stringof == int * *);
assert(typeof(b).stringof == acceptor(int *) *);
}
}
void test(acceptor(const) DemoStruct v)
{
assert(typeof(v.a).stringof == int * *);
assert(typeof(v.b).stringof == const(int *) *);
}
void main()
{
DemoStruct m;
test(m);
acceptor(immutable) i;
test(i);
}
Like having acceptor behave like inout or something. The acceptor
field can
receive the following.
int * * a;
immutable (int *) * b;
const(int *) * acceptorfield = a;
acceptorfield = b;
const(int) * * oops = b; // not valid acceptor.
const(int * *) meh = b; // will choose the first pointer mutable
since
// is a copy so meh.
The acceptor field is tail const or something with the first
entry being mutable.
---
struct DemoStruct
{
int * * a;
acceptor int * * b;
void demonstrate() copy
{
assert(typeof(a).stringof == copy(int *) *);
assert(typeof(b).stringof == copy(int *) *);
}
}
void test(copy(const) DemoStruct v)
{
assert(typeof(v.a).stringof == const(int *) *);
assert(typeof(v.b).stringof == const(int *) *);
}
void main()
{
immutable DemoStruct i;
test(i);
}
For the copying version, immutable == mutable the acceptor is
applied to all
fields.
Please forgive me for pressing the send button.
Sclytrack
Re: From slices to perfect imitators: opByValue
Andrei Alexandrescu:
making user-defined types as powerful as built-in types is a
Good Thing(tm).
An example of something useful that I think is not currently easy
to do with user-defined types (but I think this could be done by
future built-in tuples):
Tuple!(ref int, bool) foo(ref int x) pure {
x++;
return tuple(x, true);
}
In order to bring about parity, we'd need to
introduce opByValue which (if present) would be automatically
called whenever the object is passed by value into a function.
I suggest to add some usage examples, to help focus the
discussion.
Currently only the slices decay in mutables, while an immutable
int doesn't become mutable:
import std.stdio;
void foo(T)(T x) {
writeln(typeof(x).stringof);
}
void main() {
immutable a = [1, 2];
writeln(typeof(a).stringof);
foo(a);
immutable y = 10;
foo(y);
}
Output:
immutable(int[])
immutable(int)[]
immutable(int)
Bye,
bearophile
Re: From slices to perfect imitators: opByValue
On Wed, 07 May 2014 20:58:21 -0700
Andrei Alexandrescu via Digitalmars-d digitalmars-d@puremagic.com
wrote:
So there's this recent discussion about making T[] be refcounted if
and only if T has a destructor.
That's an interesting idea. More generally, there's the notion that
making user-defined types as powerful as built-in types is a Good
Thing(tm).
Which brings us to something that T[] has that user-defined types
cannot have. Consider:
import std.stdio;
void fun(T)(T x)
{
writeln(typeof(x).stringof);
}
void main()
{
immutable(int[]) a = [ 1, 2 ];
writeln(typeof(a).stringof);
fun(a);
}
This program outputs:
immutable(int[])
immutable(int)[]
which means that the type of that value has subtly and silently
changed in the process of passing it to a function.
This change was introduced a while ago (by Kenji I recall) and it
enabled a lot of code that was gratuitously rejected.
This magic of T[] is something that custom ranges can't avail
themselves of. In order to bring about parity, we'd need to introduce
opByValue which (if present) would be automatically called whenever
the object is passed by value into a function.
This change would allow library designers to provide good solutions
to making immutable and const ranges work properly - the way T[]
works.
There are of course a bunch of details to think about and figure out,
and this is a large change. Please chime in with thoughts. Thanks!
As far as I can see, opByValue does the same thing as opSlice, except
that it's used specifically when passing to functions, whereas this code
immutable int [] a = [1, 2, 3];
immutable(int)[] b = a[];
or even
immutable int [] a = [1, 2, 3];
immutable(int)[] b = a;
compiles just fine. So, I don't see how adding opByValue helps us any.
Simply calling opSlice implicitly for user-defined types in the same
places that it's called implicitly on arrays would solve that problem.
We may even do some of that already, though I'm not sure.
The core problem in either case is that const(MyStruct!T) has no
relation to MyStruct!(const T) or even const(MyStruct!(const T)).
They're different template instantations and therefore can have
completely different members. So, attempts to define opSlice such that
it returns a tail-const version of the range tends to result in
recursive template instantiations which then blow the stack (or maybe
error out due to too many levels - I don't recall which at the moment -
but regardless, it fails). I think that careful and clever use of
static ifs could resolve that, but that's not terribly pleasant. At
best, it would result in an idiom that everyone would have to look up
exactly how to do correctly every time they needed to define opSlice.
Right now, you'd have to declare something like
struct MyRange(T)
{
...
static if(isMutable!T)
MyRange!(const T) opSlice() const {...}
else
MyRange opSlice() const {...}
...
}
and I'm not even sure that that quite works, since I haven't even
attempted to define a tail-const opSlice recently. Whereas ideally,
you'd just do something mroe like
struct MyRange(T)
{
...
MyRange!(const T) opSlice() const {...}
...
}
but that doesn't currently work due to recursive template instantations.
I don't know quite how we can make it work (maybe making the compiler detect
when MyRange!T and MyRange!(const T) are effectively identical), but I think
that that's really the problem that we need to solve, not coming up with a new
function, because opSlice is already there to do what we need (though it may
need to have some additional implicit calls added to it to make it match when
arrays are implicitly sliced).
Regardless, I concur that this is a problem that sorely needs solving it.
Without it, const and ranges really don't mix at all.
- Jonathan M Davis
Re: From slices to perfect imitators: opByValue
On Thu, 08 May 2014 06:48:57 + bearophile via Digitalmars-d digitalmars-d@puremagic.com wrote: Currently only the slices decay in mutables, while an immutable int doesn't become mutable: That's because what's happening is that the slice operator for arrays is defined to return a tail-const slice of the array, and then any time you slice it - be it explicit or implicit - you get a tail-const slice. It really has nothing to do with passing an argument to a function beyond the fact that that triggers an implicit call to the slice operator. For feature parity here, what we really should be looking it is how to make opSlice have feature parity, not adding a new function. And ints can't be sliced, so there is no situation where you end up with a tail-const slice of an int. - Jonathan M Davis
Re: From slices to perfect imitators: opByValue
Just a general note: This is not only interesting for range/slice types, but for any user defined reference type (e.g. RefCounted!T or Isolated!T).
Re: From slices to perfect imitators: opByValue
On 05/08/2014 05:58 AM, Andrei Alexandrescu wrote:
...
import std.stdio;
void fun(T)(T x)
{
writeln(typeof(x).stringof);
}
void main()
{
immutable(int[]) a = [ 1, 2 ];
writeln(typeof(a).stringof);
fun(a);
}
This program outputs:
immutable(int[])
immutable(int)[]
which means that the type of that value has subtly and silently changed in the
process of passing it to a function.
...
This way of stating it is imprecise: What happened is that during
implicit function template instantiation, T was determined to be
immutable(int)[] instead of immutable(int[]). Then 'a' was implicitly
converted from immutable(int[]) to immutable(int)[] just as it would be
done for any function call with those argument and parameter types.
This change was introduced a while ago (by Kenji I recall) and it enabled a lot
of code that was gratuitously rejected.
This magic of T[] is something that custom ranges can't avail themselves
of. In order to bring about parity, we'd need to introduce opByValue
which (if present) would be automatically called whenever the object is
passed by value into a function.
...
There are two independent pieces of magic here, and this proposal
removes none of them in a satisfactory way:
struct S(T){
HeadUnqual!(typeof(this)) opByValue(){ ... }
...
}
void fun(in S!int a){}
void main(){
const S!int s;
fun(s); // error, cannot implicitly convert expression s.opByValue
// of type S!(const(int)) to const(S!int).
}
A better ad-hoc way of resolving this is opImplicitCast and a special
opIFTIdeduce member or something like that.
struct S(T){
alias opIFTIdeduce = HeadUnqual!(typeof(this));
S opImplicitCast(S)(typeof(this) arg){ ... }
...
}
But this just means that now every author of a datatype of suitable kind
has to manually re-implement implicit conversion rules of T[].
A probably even better way is to just allow to specify by annotation
that some templated datatype should mimic T[]'s implicit conversion
rules (and without necessarily providing a direct and overpowered hook
into the IFTI resolution process, though both could be done.)
This change would allow library designers to provide good solutions to
making immutable and const ranges work properly - the way T[] works.
...
Questionable.
const(T)[] b = ...;
const(T[]) a = b; // =)
Range!(const(T)) s = ...;
const(Range!T) r = s; // =(
Re: From slices to perfect imitators: opByValue
On 2014-05-08 03:58:21 +, Andrei Alexandrescu seewebsiteforem...@erdani.org said: So there's this recent discussion about making T[] be refcounted if and only if T has a destructor. That's an interesting idea. More generally, there's the notion that making user-defined types as powerful as built-in types is a Good Thing(tm). ... This magic of T[] is something that custom ranges can't avail themselves of. In order to bring about parity, we'd need to introduce opByValue which (if present) would be automatically called whenever the object is passed by value into a function. Will this solve the problem that const(MyRange!(const T)) is a different type from const(MyRange!(T))? I doubt it. But they should be the same type if we want to follow the semantics of the language's slices, where const(const(T)[]) is the same as const(T[]). Perhaps this is an orthogonal issue, but I wonder whether a solution to the above problem could make opByValue unnecessary. -- Michel Fortin michel.for...@michelf.ca http://michelf.ca
Re: From slices to perfect imitators: opByValue
On 05/08/2014 08:55 AM, Jonathan M Davis via Digitalmars-d wrote: As far as I can see, opByValue does the same thing as opSlice, except that it's used specifically when passing to functions, whereas this code immutable int [] a = [1, 2, 3]; immutable(int)[] b = a[]; or even immutable int [] a = [1, 2, 3]; immutable(int)[] b = a; compiles just fine. So, I don't see how adding opByValue helps us any. Simply calling opSlice implicitly for user-defined types in the same places that it's called implicitly on arrays would solve that problem. We may even do some of that already, though I'm not sure. Automatic slicing on function call is not what actually happens. You can see this more clearly if you pass an immutable(int*) instead of an immutable(int[]): there is no way to slice the former and the mechanism that determines the parameter type to be immutable(int)* is the same. (And indeed doing the opSlice would have undesirable side-effects, for example, your pet peeve, implicit slicing of stack-allocated static arrays would happen on any IFTI call with a static array argument. :o) Also, other currently idiomatic containers couldn't be passed to functions.)
Re: From slices to perfect imitators: opByValue
On 05/08/2014 12:14 PM, Michel Fortin wrote: Will this solve the problem that const(MyRange!(const T)) is a different type from const(MyRange!(T))? No, but as stated it aggravates this problem. I doubt it. But they should be the same type if we want to follow the semantics of the language's slices, where const(const(T)[]) is the same as const(T[]). Perhaps this is an orthogonal issue, but I wonder whether a solution to the above problem could make opByValue unnecessary. Not necessarily automatically, because there would still need to be a way to figure out that actually const(S!T) - S!(const(T)) is the way to remove top-level constness. (Because sometimes it is actually const(S!(T[])) - S!(const(T)[]), for example, for most ranges in std.algorithm.) But I think the above problem is the fundamental one.
Re: From slices to perfect imitators: opByValue
On Thu, 08 May 2014 12:38:44 +0200 Timon Gehr via Digitalmars-d digitalmars-d@puremagic.com wrote: On 05/08/2014 08:55 AM, Jonathan M Davis via Digitalmars-d wrote: As far as I can see, opByValue does the same thing as opSlice, except that it's used specifically when passing to functions, whereas this code immutable int [] a = [1, 2, 3]; immutable(int)[] b = a[]; or even immutable int [] a = [1, 2, 3]; immutable(int)[] b = a; compiles just fine. So, I don't see how adding opByValue helps us any. Simply calling opSlice implicitly for user-defined types in the same places that it's called implicitly on arrays would solve that problem. We may even do some of that already, though I'm not sure. Automatic slicing on function call is not what actually happens. You can see this more clearly if you pass an immutable(int*) instead of an immutable(int[]): there is no way to slice the former and the mechanism that determines the parameter type to be immutable(int)* is the same. (And indeed doing the opSlice would have undesirable side-effects, for example, your pet peeve, implicit slicing of stack-allocated static arrays would happen on any IFTI call with a static array argument. :o) Also, other currently idiomatic containers couldn't be passed to functions.) Ah, you're right. The fact that slicing an array results in tail-const array is part of the equation, but implicit slicing isn't necessarily (though in the case of IFTI, you still get an implicit slice - it's just that that's a side effect of what type the parameter is inferred as). Still, the core problem is that MyRange!(const T) is not the same as const(MyRange!T), and that needs to be solved for opSlice to work properly. I'd still be inclined to try and just solve the problem with opSlice rather than introducing opByValue (maybe by having a UDA on opSlice which indicates that it should be implicitly sliced in the same places that a dynamic array would?), but as you point out, the problem is unfortunately a bit more complicated than that. - Jonathan M Davis
Re: From slices to perfect imitators: opByValue
On 05/08/2014 09:01 AM, Jonathan M Davis via Digitalmars-d wrote:
It really has nothing to do with passing an argument to a function
beyond the fact that that triggers an implicit call to the slice operator.
module check_claim;
import std.stdio;
auto foo(immutable(int)[] a){writeln(Indeed, this is what happens.);}
auto foo(immutable(int[]) a){writeln(No, this is not what happens.);}
void main(){
immutable(int[]) x;
foo(x);
}
$ dmd -run check_claim
No, this is not what happens.
Re: From slices to perfect imitators: opByValue
On Thursday, 8 May 2014 at 03:58:16 UTC, Andrei Alexandrescu
wrote:
This change would allow library designers to provide good
solutions to making immutable and const ranges work properly -
the way T[] works.
There are of course a bunch of details to think about and
figure out, and this is a large change. Please chime in with
thoughts. Thanks!
Andrei
The only thing I'd be afraid about is calling a run-time
*function* when you pass something by value. It seems like it
creates a *huge* hole for abuse.
I'd be OK if opByValue was allowed only as an alias type. EG,
something like:
struct S(T)
{
alias opByValue(const) = S(const(T));
}
Which would (statically) mean that const(S(T)) may (and should)
be value converted to S(const(T));
This would still need a bit of work, but I think having a hidden
function call for pass by value is a Bad Thing (tm)
Re: From slices to perfect imitators: opByValue
On Thursday, 8 May 2014 at 07:09:24 UTC, Sönke Ludwig wrote: Just a general note: This is not only interesting for range/slice types, but for any user defined reference type (e.g. RefCounted!T or Isolated!T). Not necessarily: As soon as indirections come into play, you are basically screwed, since const is turtles all the way down. So for example, the conversion from const RefCounted!T to RefCounted!(const T) is simply not possible, because it strips the const-ness of the ref count. What we would *really* need here is NOT: const RefCounted!T = RefCounted!(const T) But rather RefCounted!T = RefCounted!(const T) The idea is to cut out the head const directly. This also applies to most ranges too BTW. We'd be much better of if we never used `const MyRange!T` to begin with, but simply had a conversion from `MyRange!T` to `MyRange!(const T)`, which references the same data. In fact, I'm wondering if this might not be a more interesting direction to explore.
Re: From slices to perfect imitators: opByValue
Am 08.05.2014 13:05, schrieb monarch_dodra: On Thursday, 8 May 2014 at 07:09:24 UTC, Sönke Ludwig wrote: Just a general note: This is not only interesting for range/slice types, but for any user defined reference type (e.g. RefCounted!T or Isolated!T). Not necessarily: As soon as indirections come into play, you are basically screwed, since const is turtles all the way down. So for example, the conversion from const RefCounted!T to RefCounted!(const T) is simply not possible, because it strips the const-ness of the ref count. What we would *really* need here is NOT: const RefCounted!T = RefCounted!(const T) But rather RefCounted!T = RefCounted!(const T) The idea is to cut out the head const directly. This also applies to most ranges too BTW. We'd be much better of if we never used `const MyRange!T` to begin with, but simply had a conversion from `MyRange!T` to `MyRange!(const T)`, which references the same data. In fact, I'm wondering if this might not be a more interesting direction to explore. The reference count _must_ be handled separate from the payload's const-ness, or a const(RefCount!T) would be completely dysfunctional. Also, if you take the other example, Isolated!T, there is no reference count involved and const(Isolated!T) - Isolated!(const(T)) would be exactly what is needed.
Re: From slices to perfect imitators: opByValue
On Thursday, 8 May 2014 at 12:48:17 UTC, Sönke Ludwig wrote: Am 08.05.2014 13:05, schrieb monarch_dodra: Not necessarily: As soon as indirections come into play, you are basically screwed, since const is turtles all the way down. So for example, the conversion from const RefCounted!T to RefCounted!(const T) is simply not possible, because it strips the const-ness of the ref count. What we would *really* need here is NOT: const RefCounted!T = RefCounted!(const T) But rather RefCounted!T = RefCounted!(const T) The idea is to cut out the head const directly. This also applies to most ranges too BTW. We'd be much better of if we never used `const MyRange!T` to begin with, but simply had a conversion from `MyRange!T` to `MyRange!(const T)`, which references the same data. In fact, I'm wondering if this might not be a more interesting direction to explore. The reference count _must_ be handled separate from the payload's const-ness, or a const(RefCount!T) would be completely dysfunctional. Right, which is my point: const(RefCount!T) *is* dysfunctional, which is why you'd want to skip it out entirely in the first place.This holds true for types implemented with RefCount, such as Array and Array.Range.
Re: From slices to perfect imitators: opByValue
On 05/08/2014 01:05 PM, monarch_dodra wrote: ... In fact, I'm wondering if this might not be a more interesting direction to explore. http://forum.dlang.org/thread/ljrm0d$28vf$1...@digitalmars.com?page=3#post-ljrt6t:242fpc:241:40digitalmars.com http://forum.dlang.org/thread/ljrm0d$28vf$1...@digitalmars.com?page=7#post-ljt0mc:24cto:241:40digitalmars.com
Re: From slices to perfect imitators: opByValue
On Thu, 08 May 2014 14:48:18 +0200 Sönke Ludwig via Digitalmars-d digitalmars-d@puremagic.com wrote: Am 08.05.2014 13:05, schrieb monarch_dodra: On Thursday, 8 May 2014 at 07:09:24 UTC, Sönke Ludwig wrote: Just a general note: This is not only interesting for range/slice types, but for any user defined reference type (e.g. RefCounted!T or Isolated!T). Not necessarily: As soon as indirections come into play, you are basically screwed, since const is turtles all the way down. So for example, the conversion from const RefCounted!T to RefCounted!(const T) is simply not possible, because it strips the const-ness of the ref count. What we would *really* need here is NOT: const RefCounted!T = RefCounted!(const T) But rather RefCounted!T = RefCounted!(const T) The idea is to cut out the head const directly. This also applies to most ranges too BTW. We'd be much better of if we never used `const MyRange!T` to begin with, but simply had a conversion from `MyRange!T` to `MyRange!(const T)`, which references the same data. In fact, I'm wondering if this might not be a more interesting direction to explore. The reference count _must_ be handled separate from the payload's const-ness, or a const(RefCount!T) would be completely dysfunctional. Unless the reference count is completely separate from const(RefCount!T) (which would mean that the functions which accessed the reference couldn't pure - otherwise they couldn't access the reference count), const(RefCount!T) _is_ completely dysfunctional. The fact that D's const can't be cast away in to do mutation without violating the type system means that pretty much anything involving references or pointers is dysfunctional with const if you ever need to mutate any of it (e.g. for a mutex or a reference count). std.typecons.RefCounted is completely broken if you make it const, and I would expect that of pretty much any wrapper object intended to do reference counting. Being able to do RefCounted!T - RefCounted!(const T) makes some sense, but const(RefCounted!T) pretty much never makes sense. That being said, unlike monarch_dodra, I think that it's critical that we find a way to do the equivalent of const(T[]) - const(T)[] with ranges - i.e. const(Range!T) or const(Range!(const T)) - Range!(const T). I don't think that Range!T - Range!(const T) will be enough at all. It's not necessarily the case that const(Range!T) - Range!(const T) would always work, but it's definitely the case that it would work if the underlying data was in an array, and given what it takes for a forward range to work, it might even be the case that a forward range could do be made to do that conversion by definition. The problem is the actual mechanism of converting const(Range!T) to Range!(const T) in the first place (due to recursive template instantiations and the fact that the compiler doesn't understand that they're related). The concept itself is perfectly sound in many cases - unlike with const(RefCounted!T) - because in most cases, with a range, it's the data being referred to which needs to stay const, whereas the bookkeeping stuff for it can be copied and thus be made mutable. With a reference count, however, you have to mutate what's actually being pointed to rather than being able to make a copy and mutate that. So, const(RefCounted!T) - RefCounted!(const T) will never work - not unless the reference is outside of const(RefCounted!T), in which case, it can't be pure, which can be just as bad as not working with const. - Jonathan M Davis
Re: From slices to perfect imitators: opByValue
On Thu, May 08, 2014 at 11:05:19AM +, monarch_dodra via Digitalmars-d wrote: On Thursday, 8 May 2014 at 07:09:24 UTC, Sönke Ludwig wrote: Just a general note: This is not only interesting for range/slice types, but for any user defined reference type (e.g. RefCounted!T or Isolated!T). Not necessarily: As soon as indirections come into play, you are basically screwed, since const is turtles all the way down. So for example, the conversion from const RefCounted!T to RefCounted!(const T) is simply not possible, because it strips the const-ness of the ref count. What we would *really* need here is NOT: const RefCounted!T = RefCounted!(const T) But rather RefCounted!T = RefCounted!(const T) The idea is to cut out the head const directly. This also applies to most ranges too BTW. We'd be much better of if we never used `const MyRange!T` to begin with, but simply had a conversion from `MyRange!T` to `MyRange!(const T)`, which references the same data. In fact, I'm wondering if this might not be a more interesting direction to explore. +1. I don't like opByValue because it seems to be too special-cased. The *real* issue we need to grapple with is head- vs. tail-const, and how to interconvert between them in user-defined types. As others have already pointed out, opByValue is really not that much different from opSlice (arguably not different at all). It seems such a waste to spend so much effort on it when we have much bigger fish to fry -- tail-const. T -- You are only young once, but you can stay immature indefinitely. -- azephrahel
Re: From slices to perfect imitators: opByValue
Am 08.05.2014 15:57, schrieb monarch_dodra: On Thursday, 8 May 2014 at 12:48:17 UTC, Sönke Ludwig wrote: Am 08.05.2014 13:05, schrieb monarch_dodra: Not necessarily: As soon as indirections come into play, you are basically screwed, since const is turtles all the way down. So for example, the conversion from const RefCounted!T to RefCounted!(const T) is simply not possible, because it strips the const-ness of the ref count. What we would *really* need here is NOT: const RefCounted!T = RefCounted!(const T) But rather RefCounted!T = RefCounted!(const T) The idea is to cut out the head const directly. This also applies to most ranges too BTW. We'd be much better of if we never used `const MyRange!T` to begin with, but simply had a conversion from `MyRange!T` to `MyRange!(const T)`, which references the same data. In fact, I'm wondering if this might not be a more interesting direction to explore. The reference count _must_ be handled separate from the payload's const-ness, or a const(RefCount!T) would be completely dysfunctional. Right, which is my point: const(RefCount!T) *is* dysfunctional, which is why you'd want to skip it out entirely in the first place.This holds true for types implemented with RefCount, such as Array and Array.Range. Okay, I didn't know that. For various reasons (mostly weak ref support) I'm using my own RefCount template, which casts away const-ness of the reference counter internally.
Re: From slices to perfect imitators: opByValue
Am 08.05.2014 16:22, schrieb Jonathan M Davis via Digitalmars-d: On Thu, 08 May 2014 14:48:18 +0200 Sönke Ludwig via Digitalmars-d digitalmars-d@puremagic.com wrote: Am 08.05.2014 13:05, schrieb monarch_dodra: On Thursday, 8 May 2014 at 07:09:24 UTC, Sönke Ludwig wrote: Just a general note: This is not only interesting for range/slice types, but for any user defined reference type (e.g. RefCounted!T or Isolated!T). Not necessarily: As soon as indirections come into play, you are basically screwed, since const is turtles all the way down. So for example, the conversion from const RefCounted!T to RefCounted!(const T) is simply not possible, because it strips the const-ness of the ref count. What we would *really* need here is NOT: const RefCounted!T = RefCounted!(const T) But rather RefCounted!T = RefCounted!(const T) The idea is to cut out the head const directly. This also applies to most ranges too BTW. We'd be much better of if we never used `const MyRange!T` to begin with, but simply had a conversion from `MyRange!T` to `MyRange!(const T)`, which references the same data. In fact, I'm wondering if this might not be a more interesting direction to explore. The reference count _must_ be handled separate from the payload's const-ness, or a const(RefCount!T) would be completely dysfunctional. Unless the reference count is completely separate from const(RefCount!T) (which would mean that the functions which accessed the reference couldn't pure - otherwise they couldn't access the reference count), const(RefCount!T) _is_ completely dysfunctional. The fact that D's const can't be cast away in to do mutation without violating the type system means that pretty much anything involving references or pointers is dysfunctional with const if you ever need to mutate any of it (e.g. for a mutex or a reference count). std.typecons.RefCounted is completely broken if you make it const, and I would expect that of pretty much any wrapper object intended to do reference counting. Being able to do RefCounted!T - RefCounted!(const T) makes some sense, but const(RefCounted!T) pretty much never makes sense. That being said, unlike monarch_dodra, I think that it's critical that we find a way to do the equivalent of const(T[]) - const(T)[] with ranges - i.e. const(Range!T) or const(Range!(const T)) - Range!(const T). I don't think that Range!T - Range!(const T) will be enough at all. It's not necessarily the case that const(Range!T) - Range!(const T) would always work, but it's definitely the case that it would work if the underlying data was in an array, and given what it takes for a forward range to work, it might even be the case that a forward range could do be made to do that conversion by definition. The problem is the actual mechanism of converting const(Range!T) to Range!(const T) in the first place (due to recursive template instantiations and the fact that the compiler doesn't understand that they're related). The concept itself is perfectly sound in many cases - unlike with const(RefCounted!T) - because in most cases, with a range, it's the data being referred to which needs to stay const, whereas the bookkeeping stuff for it can be copied and thus be made mutable. With a reference count, however, you have to mutate what's actually being pointed to rather than being able to make a copy and mutate that. So, const(RefCounted!T) - RefCounted!(const T) will never work - not unless the reference is outside of const(RefCounted!T), in which case, it can't be pure, which can be just as bad as not working with const. - Jonathan M Davis Unless I'm completely mistaken, it's safe to cast away const when it is known that the original reference was constructed as mutable. Anyway, this is what I do in my own RefCount struct. But my main point was that any user defined reference type is affected by the head vs. tail const issue, not just range types. So a decent solution should solve it for all of those types. BTW, since RefCount would usually do manual memory management, it can't be used in pure contexts anyway. Proper support of scope/lent pointers would solve that, though. Ideally, there would be a way to allow a RefCount!T to implicitly cast to T if passed as a scoped parameter.
Re: From slices to perfect imitators: opByValue
On Thursday, 8 May 2014 at 15:39:24 UTC, Sönke Ludwig wrote: Unless I'm completely mistaken, it's safe to cast away const when it is known that the original reference was constructed as mutable. Depends. If the original type referencing the (originally mutable) allocated data happens to be *im*-mutable, then it *would* be illegal to modify it, even if you *can*. EG: immutable myFirstRC = RefCounted!int(1); immutable myReference = myFirstRC; In this particular case, you'd be modifying a ref count that can only be accessed via an immutable pointer. As such, the compiler is free to assume the value has never been changed, and avoid reading it all together, destroying your payload at the end of myFirstRC's life-cycle. I honestly don't think (given D's transitive constness mechanics), that a const RefCount *could* make any sense. If you have const data referencing mutable data, then yes, you can cast away all the const you want, but at that point, it kind of makes the whole const thing moot.
Re: From slices to perfect imitators: opByValue
On 05/08/2014 06:02 PM, monarch_dodra wrote: If you have const data referencing mutable data, then yes, you can cast away all the const you want, but at that point, it kind of makes the whole const thing moot. This is not guaranteed to work. I guess the only related thing that is safe to do is casting away const, but then not modifying the memory.
Re: From slices to perfect imitators: opByValue
On Thursday, 8 May 2014 at 16:30:13 UTC, Sönke Ludwig wrote: For what practical reason would that be the case? I know that the spec states undefined behavior, but AFAICS, there is neither an existing, nor a theoretical reason, why this should fail: Compiler optimizations based on immutability. David
Re: From slices to perfect imitators: opByValue
Am 08.05.2014 18:10, schrieb Timon Gehr: On 05/08/2014 06:02 PM, monarch_dodra wrote: If you have const data referencing mutable data, then yes, you can cast away all the const you want, but at that point, it kind of makes the whole const thing moot. This is not guaranteed to work. I guess the only related thing that is safe to do is casting away const, but then not modifying the memory. For what practical reason would that be the case? I know that the spec states undefined behavior, but AFAICS, there is neither an existing, nor a theoretical reason, why this should fail: int* i = new int; const(int)* j = i; int* k = cast(int*)j; *k = 0; Anyway this is going pretty much off topic...
Re: From slices to perfect imitators: opByValue
Am 08.05.2014 18:33, schrieb David Nadlinger: On Thursday, 8 May 2014 at 16:30:13 UTC, Sönke Ludwig wrote: For what practical reason would that be the case? I know that the spec states undefined behavior, but AFAICS, there is neither an existing, nor a theoretical reason, why this should fail: Compiler optimizations based on immutability. David My point was just about const in particular, because RefCount doesn't mix well with immutable anyway (see previous post).
Re: From slices to perfect imitators: opByValue
Am 08.05.2014 18:02, schrieb monarch_dodra: On Thursday, 8 May 2014 at 15:39:24 UTC, Sönke Ludwig wrote: Unless I'm completely mistaken, it's safe to cast away const when it is known that the original reference was constructed as mutable. Depends. If the original type referencing the (originally mutable) allocated data happens to be *im*-mutable, then it *would* be illegal to modify it, even if you *can*. EG: immutable myFirstRC = RefCounted!int(1); immutable myReference = myFirstRC; This currently wouldn't compile, because all the RefCounted construction/destruction/postblit operations are not pure. But I agree that immutable is a different thing and would potentially break the cast - I was just talking about const references, because this is what frequently happens, even when *not* working with immutable data. In this particular case, you'd be modifying a ref count that can only be accessed via an immutable pointer. As such, the compiler is free to assume the value has never been changed, and avoid reading it all together, destroying your payload at the end of myFirstRC's life-cycle. I honestly don't think (given D's transitive constness mechanics), that a const RefCount *could* make any sense. If you have const data referencing mutable data, then yes, you can cast away all the const you want, but at that point, it kind of makes the whole const thing moot. There is of course always the alternative of using a global array of reference counts to implement this in a clean way, but ideally, there would be a way to semantically separate reference management data from payload data, so that immutable(RefCount!T) can be defined without working around the type system. But let's not clutter up this particular thread with RefCount related issues, it's a separate issue (AFAICS), and I really just brought it up as one example.
Re: From slices to perfect imitators: opByValue
Andrei Alexandrescu: That's an interesting idea. More generally, there's the notion that making user-defined types as powerful as built-in types is a Good Thing(tm). Regarding the management of const for library-defined types, sometimes I'd like the type T1 to be seen as equal to the type T2, this could save me some hassles during the usage of tuples: alias T1 = const Tuple!(int, int); alias T2 = Tuple!(const int, const int); I think T1 and T2 should be equivalent for built-in tuples. Bye, bearophile
Re: From slices to perfect imitators: opByValue
On 05/08/2014 06:30 PM, Sönke Ludwig wrote: Am 08.05.2014 18:10, schrieb Timon Gehr: On 05/08/2014 06:02 PM, monarch_dodra wrote: If you have const data referencing mutable data, then yes, you can cast away all the const you want, but at that point, it kind of makes the whole const thing moot. This is not guaranteed to work. I guess the only related thing that is safe to do is casting away const, but then not modifying the memory. For what practical reason would that be the case? I know that the spec states undefined behavior, Case closed.
Re: From slices to perfect imitators: opByValue
Am 09.05.2014 00:02, schrieb Timon Gehr: On 05/08/2014 06:30 PM, Sönke Ludwig wrote: Am 08.05.2014 18:10, schrieb Timon Gehr: On 05/08/2014 06:02 PM, monarch_dodra wrote: If you have const data referencing mutable data, then yes, you can cast away all the const you want, but at that point, it kind of makes the whole const thing moot. This is not guaranteed to work. I guess the only related thing that is safe to do is casting away const, but then not modifying the memory. For what practical reason would that be the case? I know that the spec states undefined behavior, Case closed. Bonus points for getting the difference between practical and theoretical... I'd rather say that the spec needs clarification there. It explicitly states the immutable-mutable case, but it at least makes the strong impression that const-mutable is only left undefined (implicitly, AFAICS) because a const pointer might point to immutable data. Anyway, I feel like I'm dragged into an artificial argument here that has nothing to do either with the original topic, or with what my original statement was about. I agree that a RefCount struct currently has more issues than the head/tail/transitivity of const. My statement on this (off) topic is simply that payload and reference count must be handled separately WRT to const to make sense (because const(RefCount) *does* occur in practice) and that it can be practically achieved using an internal cast now (immutable being a different beast). Nothing more, nothing less.
Re: From slices to perfect imitators: opByValue
On Thu, 08 May 2014 16:33:06 + David Nadlinger via Digitalmars-d digitalmars-d@puremagic.com wrote: On Thursday, 8 May 2014 at 16:30:13 UTC, Sönke Ludwig wrote: For what practical reason would that be the case? I know that the spec states undefined behavior, but AFAICS, there is neither an existing, nor a theoretical reason, why this should fail: Compiler optimizations based on immutability. Or even just based on const. Optimizations based on const are going to be rarer, because other objects of the same type but which are mutable could refer to the same object and thus mutate it, but if the object is thread-local (as is the default), then there will still be some cases where the compiler will be able to assume that the object isn't mutated even if immutable isn't involved at all. If you're even attempting to cast away const and then mutate the object, you need to have a really good understanding of how the compiler could even theoretically optimize based on const (especially since even if an optimization isn't done now, and your code works, it could be added later and break your code). So, I'd strongly argue that casting away const from an object and mutating it is a fundamentally broken idiom in D. You may have a better chance of avoiding blowing your foot off if immutable isn't involve, but you still risk serious bugs. - Jonathan M Davis
Re: From slices to perfect imitators: opByValue
On Thu, 08 May 2014 17:18:03 +0200 Sönke Ludwig via Digitalmars-d digitalmars-d@puremagic.com wrote: Right, which is my point: const(RefCount!T) *is* dysfunctional, which is why you'd want to skip it out entirely in the first place.This holds true for types implemented with RefCount, such as Array and Array.Range. Okay, I didn't know that. For various reasons (mostly weak ref support) I'm using my own RefCount template, which casts away const-ness of the reference counter internally. Which technically violates the type system and isn't something that should be done - though you _should_ be able to get away with it as long as immutable isn't involved. Still, the compiler is permitted to assume that const objects aren't mutated (because that's what const is supposed to guarantee), so you're risking subtle bugs due to compiler optimizations and whatnot. - Jonathan M Davis
Re: From slices to perfect imitators: opByValue
On Thu, 08 May 2014 17:39:25 +0200 Sönke Ludwig via Digitalmars-d digitalmars-d@puremagic.com wrote: Am 08.05.2014 16:22, schrieb Jonathan M Davis via Digitalmars-d: On Thu, 08 May 2014 14:48:18 +0200 Sönke Ludwig via Digitalmars-d digitalmars-d@puremagic.com wrote: Am 08.05.2014 13:05, schrieb monarch_dodra: On Thursday, 8 May 2014 at 07:09:24 UTC, Sönke Ludwig wrote: Just a general note: This is not only interesting for range/slice types, but for any user defined reference type (e.g. RefCounted!T or Isolated!T). Not necessarily: As soon as indirections come into play, you are basically screwed, since const is turtles all the way down. So for example, the conversion from const RefCounted!T to RefCounted!(const T) is simply not possible, because it strips the const-ness of the ref count. What we would *really* need here is NOT: const RefCounted!T = RefCounted!(const T) But rather RefCounted!T = RefCounted!(const T) The idea is to cut out the head const directly. This also applies to most ranges too BTW. We'd be much better of if we never used `const MyRange!T` to begin with, but simply had a conversion from `MyRange!T` to `MyRange!(const T)`, which references the same data. In fact, I'm wondering if this might not be a more interesting direction to explore. The reference count _must_ be handled separate from the payload's const-ness, or a const(RefCount!T) would be completely dysfunctional. Unless the reference count is completely separate from const(RefCount!T) (which would mean that the functions which accessed the reference couldn't pure - otherwise they couldn't access the reference count), const(RefCount!T) _is_ completely dysfunctional. The fact that D's const can't be cast away in to do mutation without violating the type system means that pretty much anything involving references or pointers is dysfunctional with const if you ever need to mutate any of it (e.g. for a mutex or a reference count). std.typecons.RefCounted is completely broken if you make it const, and I would expect that of pretty much any wrapper object intended to do reference counting. Being able to do RefCounted!T - RefCounted!(const T) makes some sense, but const(RefCounted!T) pretty much never makes sense. That being said, unlike monarch_dodra, I think that it's critical that we find a way to do the equivalent of const(T[]) - const(T)[] with ranges - i.e. const(Range!T) or const(Range!(const T)) - Range!(const T). I don't think that Range!T - Range!(const T) will be enough at all. It's not necessarily the case that const(Range!T) - Range!(const T) would always work, but it's definitely the case that it would work if the underlying data was in an array, and given what it takes for a forward range to work, it might even be the case that a forward range could do be made to do that conversion by definition. The problem is the actual mechanism of converting const(Range!T) to Range!(const T) in the first place (due to recursive template instantiations and the fact that the compiler doesn't understand that they're related). The concept itself is perfectly sound in many cases - unlike with const(RefCounted!T) - because in most cases, with a range, it's the data being referred to which needs to stay const, whereas the bookkeeping stuff for it can be copied and thus be made mutable. With a reference count, however, you have to mutate what's actually being pointed to rather than being able to make a copy and mutate that. So, const(RefCounted!T) - RefCounted!(const T) will never work - not unless the reference is outside of const(RefCounted!T), in which case, it can't be pure, which can be just as bad as not working with const. - Jonathan M Davis Unless I'm completely mistaken, it's safe to cast away const when it is known that the original reference was constructed as mutable. Anyway, this is what I do in my own RefCount struct. But my main point was that any user defined reference type is affected by the head vs. tail const issue, not just range types. So a decent solution should solve it for all of those types. Part of my point was that getting a tail-const slice of a range is fundamentally different from trying to get a tail-const with a ref-counted struct. With a range, the bookkeeping that would end up being mutable in the tail-const slice can usually be copied in order to be mutable and still work properly. In a ref-counted struct, however, the ref-count is a reference or pointer and copying it wouldn't help. You need to be able to mutate the same count that every other reference to the same object is using. And that requires casting away const (thus violating the type system) rather than being able to make a copy. So, while it might be the case that the issue of tail-constness can be generalized beyond slices in a useful way, and I don't think that it applies to reference
Re: From slices to perfect imitators: opByValue
On Thu, 08 May 2014 18:10:28 +0200 Timon Gehr via Digitalmars-d digitalmars-d@puremagic.com wrote: On 05/08/2014 06:02 PM, monarch_dodra wrote: If you have const data referencing mutable data, then yes, you can cast away all the const you want, but at that point, it kind of makes the whole const thing moot. This is not guaranteed to work. I guess the only related thing that is safe to do is casting away const, but then not modifying the memory. Exactly. It's effectively illegal to cast away const and then mutate the object. The compiler lets you do it, because D is a systems language, but the compiler is free to assume that the object wasn't modified, so unless you know what you're doing and are very, very careful, you're risking subtle bugs. Really, casting away const and then mutating the now-mutable object is not something that you should ever be doing. - Jonathan M Davis
Re: From slices to perfect imitators: opByValue
This looks like interesting.
Re: From slices to perfect imitators: opByValue
On 8.5.2014. 5:58, Andrei Alexandrescu wrote: This magic of T[] is something that custom ranges can't avail themselves of. In order to bring about parity, we'd need to introduce opByValue which (if present) would be automatically called whenever the object is passed by value into a function. This change would allow library designers to provide good solutions to making immutable and const ranges work properly - the way T[] works. Looks very similar to some kind of opImplicitConvert. http://forum.dlang.org/thread/teddgvbtmrxumffrh...@forum.dlang.org http://forum.dlang.org/thread/gq0fj7$4av$1...@digitalmars.com Maybe it would be better to have a more general solution instead of special case solution, if there is no reason against implicit conversion of course.
