Re: Struct that destroys its original handle on copy-by-value
On Monday, 3 August 2015 at 08:54:32 UTC, Kagamin wrote: On Saturday, 1 August 2015 at 11:47:29 UTC, Joseph Rushton Wakeling wrote: Yea, but that's not what I'm trying to achieve. I know how I can pass something to `take` so as to e.g. obtain reference semantics or whatever; what I'm trying to achieve is a range that _doesn't rely on the user knowing the right way to handle it_. Wrapping a reference to a stack-allocated struct is also unsafe, so no way for the user to not know about it. It is now verified as safe by `return ref`.
Re: Struct that destroys its original handle on copy-by-value
On Saturday, 1 August 2015 at 11:47:29 UTC, Joseph Rushton Wakeling wrote: Yea, but that's not what I'm trying to achieve. I know how I can pass something to `take` so as to e.g. obtain reference semantics or whatever; what I'm trying to achieve is a range that _doesn't rely on the user knowing the right way to handle it_. Wrapping a reference to a stack-allocated struct is also unsafe, so no way for the user to not know about it. On Saturday, 1 August 2015 at 12:10:43 UTC, Joseph Rushton Wakeling wrote: On the other hand, what you want to disallow is this: auto sample = iota(100).randomSample(10, gen); sample.take(5).writeln; sample.take(5).writeln; // statistical correlations result, // probably unwanted Try auto sample = iota(100).randomSample(10, gen);
Re: Struct that destroys its original handle on copy-by-value
On Monday, 3 August 2015 at 09:01:51 UTC, Dicebot wrote: It is now verified as safe by `return ref`. Yes, until you pointed this out to me I'd been convinced that classes were the way forward for RNGs. I think that `return ref` is going to be a _very_ powerful tool for facilitating stack-allocated RNG functionality.
Re: Struct that destroys its original handle on copy-by-value
On 02/08/15 03:38, Dicebot via Digitalmars-d-learn wrote: On Saturday, 1 August 2015 at 17:50:28 UTC, John Colvin wrote: I'm not sure how good an idea it is to totally enforce a range to be non-copyable, even if you could deal with the function call chain problem. Even in totally save-aware code, there can still be valid assignment of a range type. I'm pretty sure a lot of phobos ranges/algorithms would be unusable. This is exactly why I proposed to Joe design with destructive copy originally - that would work with any algorithms expecting implicit pass by value but prevent from actual double usage. Sadly, this does not seem to be implementable in D in any reasonable way. Yup. This work is follow-up on a really creative bunch of suggestions Dicebot made to me on our flight back from DConf, and which we followed up on at the recent Berlin meetup. The design principle of destructive copy is great -- it really cuts through a bunch of potential nastinesses around random number generation -- but it really doesn't look like it's straightforwardly possible :-(
Re: Struct that destroys its original handle on copy-by-value
On 31/07/15 19:21, Ali Çehreli via Digitalmars-d-learn wrote: On 07/26/2015 04:29 AM, Joseph Rushton Wakeling via Digitalmars-d-learn wrote: is this design idea even feasible in principle, or just a bad idea from the get-go? As I understand it, it is against one of fundamental D principles: structs are value types where any copy can be used in place of any other. I expect there are examples where even Phobos violates it but the struct documentation still says so: A struct is defined to not have an identity; that is, the implementation is free to make bit copies of the struct as convenient. http://dlang.org/struct.html That really feels very bad for the problem domain I have in mind -- random number generation. No implementation should be free to make copies of a random number generator as convenient, that should be 100% in the hands of the programmer! And if feasible -- how would I go about it? Disallowing automatic copying and providing a function comes to mind. Yes, I considered that, but I don't think it really delivers what's needed :-( Let me give a concrete example of why I was thinking in this direction. Consider RandomSample in std.random. This is a struct (a value type, instantiated on the stack). However, it also wraps a random number generator. It needs to be consumed once and once only, because otherwise there will be unintended statistical correlations in the program. Copy-by-value leads to a situation where you can accidentally consume the same sequence twice (or possibly, only _part_ of the sequence). Now, indeed, one way is to just @disable this(this) which prevents copy-by-value. But then you can't do something natural and desirable like: iota(100).randomSample(10, gen).take(5).writeln; ... because you would no longer be able to pass the RandomSample instance into `take`. On the other hand, what you want to disallow is this: auto sample = iota(100).randomSample(10, gen); sample.take(5).writeln; sample.take(5).writeln; // statistical correlations result, // probably unwanted The first situation is still possible, and the second disallowed (or at least, guarded against), _if_ a copy-by-value is finalized by tweaking the source to render it an empty range. I would happily hear alternative solutions to the problem, but that's why I was interested in a struct with the properties I outlined in my original post.
Re: Struct that destroys its original handle on copy-by-value
On Saturday, 1 August 2015 at 12:10:43 UTC, Joseph Rushton
Wakeling wrote:
On 31/07/15 19:21, Ali Çehreli via Digitalmars-d-learn wrote:
On 07/26/2015 04:29 AM, Joseph Rushton Wakeling via
Digitalmars-d-learn wrote:
is this design idea even feasible in principle, or just a
bad
idea from the get-go?
As I understand it, it is against one of fundamental D
principles: structs are
value types where any copy can be used in place of any other.
I expect there are examples where even Phobos violates it but
the struct
documentation still says so: A struct is defined to not have
an identity; that
is, the implementation is free to make bit copies of the
struct as convenient.
http://dlang.org/struct.html
That really feels very bad for the problem domain I have in
mind -- random number generation. No implementation should be
free to make copies of a random number generator as
convenient, that should be 100% in the hands of the programmer!
And if feasible -- how would I go about it?
Disallowing automatic copying and providing a function comes
to mind.
Yes, I considered that, but I don't think it really delivers
what's needed :-(
Let me give a concrete example of why I was thinking in this
direction. Consider RandomSample in std.random. This is a
struct (a value type, instantiated on the stack). However, it
also wraps a random number generator. It needs to be consumed
once and once only, because otherwise there will be unintended
statistical correlations in the program. Copy-by-value leads
to a situation where you can accidentally consume the same
sequence twice (or possibly, only _part_ of the sequence).
Now, indeed, one way is to just @disable this(this) which
prevents copy-by-value. But then you can't do something
natural and desirable like:
iota(100).randomSample(10, gen).take(5).writeln;
... because you would no longer be able to pass the
RandomSample instance into `take`.
On the other hand, what you want to disallow is this:
auto sample = iota(100).randomSample(10, gen);
sample.take(5).writeln;
sample.take(5).writeln; // statistical correlations result,
// probably unwanted
The first situation is still possible, and the second
disallowed (or at least, guarded against), _if_ a copy-by-value
is finalized by tweaking the source to render it an empty range.
I would happily hear alternative solutions to the problem, but
that's why I was interested in a struct with the properties I
outlined in my original post.
Naïve compromise solution?
struct S(bool noCopy = true)
{
//replace with real state
int state = 0;
static if(noCopy) @disable this(this);
@property auto copyable()
{
//did the move manually because I got
//weird results std.algorithm.move
auto ret = cast(S!false)this;
this.state = this.state.init;
return ret;
}
}
auto s(int state)
{
return S!()(state);
}
void main()
{
import std.stdio, std.algorithm;
auto s = s(42);
auto s1 = s.move;
assert(s == S!().init);
s1.copyable.writeln;
assert(s1 == S!().init);
}
Then at least the simplest mistakes are avoided. Also, it means
people are more likely to read important docs i.e. Why do I have
to call this copyable thing? Oh, I see, I'll be careful.
I'm not sure how good an idea it is to totally enforce a range to
be non-copyable, even if you could deal with the function call
chain problem. Even in totally save-aware code, there can still
be valid assignment of a range type. I'm pretty sure a lot of
phobos ranges/algorithms would be unusable.
Re: Struct that destroys its original handle on copy-by-value
On 31/07/15 13:40, Kagamin via Digitalmars-d-learn wrote: On Sunday, 26 July 2015 at 12:16:30 UTC, Joseph Rushton Wakeling wrote: Example: Unique!Random rng = new Random(unpredictableSeed); rng.take(10).writeln; My aim by contrast is to _allow_ that kind of use, but render the original handle empty when it's done. `take` stores the range, you can try to use some sort of a weak reference. Yea, but that's not what I'm trying to achieve. I know how I can pass something to `take` so as to e.g. obtain reference semantics or whatever; what I'm trying to achieve is a range that _doesn't rely on the user knowing the right way to handle it_. I'll expand on this more responding to Ali, so as to clarify the context of what I'm aiming for and why.
Re: Struct that destroys its original handle on copy-by-value
On Saturday, 1 August 2015 at 17:50:28 UTC, John Colvin wrote: I'm not sure how good an idea it is to totally enforce a range to be non-copyable, even if you could deal with the function call chain problem. Even in totally save-aware code, there can still be valid assignment of a range type. I'm pretty sure a lot of phobos ranges/algorithms would be unusable. This is exactly why I proposed to Joe design with destructive copy originally - that would work with any algorithms expecting implicit pass by value but prevent from actual double usage. Sadly, this does not seem to be implementable in D in any reasonable way.
Re: Struct that destroys its original handle on copy-by-value
On 07/26/2015 04:29 AM, Joseph Rushton Wakeling via Digitalmars-d-learn wrote: is this design idea even feasible in principle, or just a bad idea from the get-go? As I understand it, it is against one of fundamental D principles: structs are value types where any copy can be used in place of any other. I expect there are examples where even Phobos violates it but the struct documentation still says so: A struct is defined to not have an identity; that is, the implementation is free to make bit copies of the struct as convenient. http://dlang.org/struct.html And if feasible -- how would I go about it? Disallowing automatic copying and providing a function comes to mind. Ali
Re: Struct that destroys its original handle on copy-by-value
On 07/31/2015 11:01 AM, Dicebot wrote: On Friday, 31 July 2015 at 17:21:40 UTC, Ali Çehreli wrote: On 07/26/2015 04:29 AM, Joseph Rushton Wakeling via Digitalmars-d-learn wrote: is this design idea even feasible in principle, or just a bad idea from the get-go? As I understand it, it is against one of fundamental D principles: structs are value types where any copy can be used in place of any other. I expect there are examples where even Phobos violates it but the struct documentation still says so: A struct is defined to not have an identity; that is, the implementation is free to make bit copies of the struct as convenient. I believe this restriction should be banned. Considering classes have inherent reference + heap semantics (and you can only bail out of that with hacks) saying struct can't be anything but data bags makes impossible to implement whole class of useful designs. The fact that Phobos has to violate it itself to get stuff done is quite telling. To make sure, I didn't mean that I know of structs in Phobos that behave that way. Although, it would be interesting to identify them. :) Ali
Re: Struct that destroys its original handle on copy-by-value
On Fri, Jul 31, 2015 at 06:01:44PM +, Dicebot via Digitalmars-d-learn wrote: On Friday, 31 July 2015 at 17:21:40 UTC, Ali Çehreli wrote: On 07/26/2015 04:29 AM, Joseph Rushton Wakeling via Digitalmars-d-learn wrote: is this design idea even feasible in principle, or just a bad idea from the get-go? As I understand it, it is against one of fundamental D principles: structs are value types where any copy can be used in place of any other. I expect there are examples where even Phobos violates it but the struct documentation still says so: A struct is defined to not have an identity; that is, the implementation is free to make bit copies of the struct as convenient. I believe this restriction should be banned. Considering classes have inherent reference + heap semantics (and you can only bail out of that with hacks) saying struct can't be anything but data bags makes impossible to implement whole class of useful designs. The fact that Phobos has to violate it itself to get stuff done is quite telling. On the other hand, though, structs that are *not* mere data bags usually get into the dark area of compiler bugs and other unexpected side-effects or unclear areas of the language, often with questionable or inconsistent behaviour. My suspicion that these problems aren't just because of compiler quality or the language spec being incomplete, but it's because they are caused by this underlying dichotomy between structs being defined to be purely data, and structs being used for things that are more than just pure data. One example of this is how long it took for std.stdio.File to get to its present form. It started out as a simple wrapper around the C library FILE*, passed through stages of being pure data yet showing buggy corner cases, then stepping into dtor territory with its associated unexpected behaviours, then now into a wrapper around a ref-counted type (and IIRC, it may still show buggy / odd behaviour when you stick it inside a container like an array). Along the way, we had issues with dtors, postblit, and a whole slew of issues, almost all of which can be traced back to the dichotomy between a struct wanting to be just a value yet at the same time wanting / needing to be more than that. It seems that what the language (originally) defines structs to be, is not entirely consistent with how it has come to be used (which also entailed later extensions to the struct definition), and this has been a source of problems. T -- Bare foot: (n.) A device for locating thumb tacks on the floor.
Re: Struct that destroys its original handle on copy-by-value
On Friday, 31 July 2015 at 18:13:04 UTC, Ali Çehreli wrote: To make sure, I didn't mean that I know of structs in Phobos that behave that way. Although, it would be interesting to identify them. :) Ali Things like Unique, Scoped, RefCounted - pretty much everything which wraps reference type with additional semantics.
Re: Struct that destroys its original handle on copy-by-value
On Friday, 31 July 2015 at 18:23:39 UTC, H. S. Teoh wrote: It seems that what the language (originally) defines structs to be, is not entirely consistent with how it has come to be used (which also entailed later extensions to the struct definition), and this has been a source of problems. Yes, and it wasn't because people started to misuse structs - it was because certain designs would be simply impossible otherwise. The very language design assumption that structs should always be dumb copyable values was not practical. It may come from times when GC, heaps and classes were considered good enough for everyone and RAII intended to be completely replaces by scope(exit). Which didn't work.
Re: Struct that destroys its original handle on copy-by-value
On 7/31/15 1:21 PM, Ali Çehreli wrote: Disallowing automatic copying and providing a function comes to mind. Isn't that what std.algorithm.move is for? -Steve
Re: Struct that destroys its original handle on copy-by-value
On Friday, 31 July 2015 at 17:21:40 UTC, Ali Çehreli wrote: On 07/26/2015 04:29 AM, Joseph Rushton Wakeling via Digitalmars-d-learn wrote: is this design idea even feasible in principle, or just a bad idea from the get-go? As I understand it, it is against one of fundamental D principles: structs are value types where any copy can be used in place of any other. I expect there are examples where even Phobos violates it but the struct documentation still says so: A struct is defined to not have an identity; that is, the implementation is free to make bit copies of the struct as convenient. I believe this restriction should be banned. Considering classes have inherent reference + heap semantics (and you can only bail out of that with hacks) saying struct can't be anything but data bags makes impossible to implement whole class of useful designs. The fact that Phobos has to violate it itself to get stuff done is quite telling.
Re: Struct that destroys its original handle on copy-by-value
On 07/31/2015 12:18 PM, Steven Schveighoffer wrote: On 7/31/15 1:21 PM, Ali Çehreli wrote: Disallowing automatic copying and providing a function comes to mind. Isn't that what std.algorithm.move is for? -Steve Sounds great and I like it! :) Ali
Re: Struct that destroys its original handle on copy-by-value
On Sunday, 26 July 2015 at 12:16:30 UTC, Joseph Rushton Wakeling wrote: Example: Unique!Random rng = new Random(unpredictableSeed); rng.take(10).writeln; My aim by contrast is to _allow_ that kind of use, but render the original handle empty when it's done. `take` stores the range, you can try to use some sort of a weak reference.
Re: Struct that destroys its original handle on copy-by-value
On Wednesday, 29 July 2015 at 19:10:36 UTC, Adam D. Ruppe wrote: On Sunday, 26 July 2015 at 12:16:30 UTC, Joseph Rushton Wakeling wrote: My aim by contrast is to _allow_ that kind of use, but render the original handle empty when it's done. I don't think D offers any way to do that. With the disabled postblit, you can force people into a method you write that returns a new copy and clears the original, but that won't just work with assignment. The ref assign might not be forbidden by the written doc but I'm guessing that is just an oversight - struct assignment in D never clears the original, it is always a simple copy (perhaps plus other code) Hmm... are you implying that `opAssign(ref T other)` should be disallowed? Why? I find this the obvious way to implement move semantics, and it is also what std.algorithm.move does: http://dlang.org/phobos/std_algorithm_mutation.html#.move
Re: Struct that destroys its original handle on copy-by-value
On Wednesday, 29 July 2015 at 19:10:36 UTC, Adam D. Ruppe wrote: On Sunday, 26 July 2015 at 12:16:30 UTC, Joseph Rushton Wakeling wrote: My aim by contrast is to _allow_ that kind of use, but render the original handle empty when it's done. I don't think D offers any way to do that. With the disabled postblit, you can force people into a method you write that returns a new copy and clears the original, but that won't just work with assignment. The ref assign might not be forbidden by the written doc but I'm guessing that is just an oversight - struct assignment in D never clears the original, it is always a simple copy (perhaps plus other code) Slightly OT, but this is something that would be possible with a copy constructor, I think?
Re: Struct that destroys its original handle on copy-by-value
On Sunday, 26 July 2015 at 12:16:30 UTC, Joseph Rushton Wakeling wrote: My aim by contrast is to _allow_ that kind of use, but render the original handle empty when it's done. I don't think D offers any way to do that. With the disabled postblit, you can force people into a method you write that returns a new copy and clears the original, but that won't just work with assignment. The ref assign might not be forbidden by the written doc but I'm guessing that is just an oversight - struct assignment in D never clears the original, it is always a simple copy (perhaps plus other code)
Re: Struct that destroys its original handle on copy-by-value
On Sunday, 26 July 2015 at 11:30:16 UTC, Joseph Rushton Wakeling wrote: Hello all, A design question that came up during the hackathon held during the last Berlin D Meetup. [...] Ping on the above -- nobody has any insight...?
Re: Struct that destroys its original handle on copy-by-value
On Sunday, 26 July 2015 at 11:30:16 UTC, Joseph Rushton Wakeling wrote: Hello all, A design question that came up during the hackathon held during the last Berlin D Meetup. [...] Sounds like unique_ptr (so UniqueRange might be a nice name). Maybe you can get some ideas from that.
Struct that destroys its original handle on copy-by-value
Hello all,
A design question that came up during the hackathon held during the last Berlin
D Meetup.
I was trying to come up with a range that can be copied by value, but when this
is done, destroys the original handle. The idea would be behaviour something
like this:
auto originalRange = myWeirdRange(whatever);
originalRange.take(10).writeln;
assert(originalRange.empty,
The original handle points to an empty range after copy-by-value.);
A very minimal prototype (missing out the details of front and popFront as
irrelevant) would be something like:
struct MyWeirdRange (R)
if (isInputRange!R)
{
private:
R* input_; // Assumed to never be empty, FWIW. I'm missing out
// a template check on that for brevity's sake.
public:
this (return ref R input)
{
/* return ref should guarantee the pointer
* is safe for the lifetime of the struct,
* right ... ?
*/
this.input_ = input;
}
bool empty () @property
{
return this.input_ is null;
}
auto front () @property { ... }
void popFront () { ... }
void opAssign (ref typeof(this) that)
{
/* copy the internal pointer, then
* set that of the original to null
*/
this.input_ = that.input_;
that.input_ = null;
}
}
Basically, this is a range that would actively enforce the principle that its
use is a one-shot. You copy it by value (whether by direct assignment or by
passing it to another function), you leave the original handle an empty range.
However, the above doesn't work; even in the event of a direct assignment, i.e.
newRange = originalRange;
... the opAssign is never called. I presume this is because of the
ref-parameter input, but it's not obvious to me according to the description
here why this should be: http://dlang.org/operatoroverloading.html#assignment
Can anyone clarify what's going on here?
Anyway, my main question is: is this design idea even feasible in principle, or
just a bad idea from the get-go? And if feasible -- how would I go about it?
Thanks best wishes,
-- Joe
Re: Struct that destroys its original handle on copy-by-value
On 26/07/15 13:45, Martijn Pot via Digitalmars-d-learn wrote: Sounds like unique_ptr (so UniqueRange might be a nice name). Maybe you can get some ideas from that. There is already a Unique in std.typecons. However, I'm not sure that it's doing what I require. Example: Unique!Random rng = new Random(unpredictableSeed); rng.take(10).writeln; ... will fail with an error: Error: struct std.typecons.Unique!(MersenneTwisterEngine!(uint, 32LU, 624LU, 397LU, 31LU, 2567483615u, 11LU, 7LU, 2636928640u, 15LU, 4022730752u, 18LU)).Unique is not copyable because it is annotated with @disable My aim by contrast is to _allow_ that kind of use, but render the original handle empty when it's done.
