Re: Flaw in DIP1000? Returning a Result Struct in DIP1000
On Wednesday, March 21, 2018 19:50:52 Jack Stouffer via Digitalmars-d wrote: > On Wednesday, 21 March 2018 at 18:50:59 UTC, Jonathan M Davis > > wrote: > > The struct being returned would need to be marked with scope > > (or its members marked with scope) such that the compiler > > treated the result as containing values from the function > > arguments. I don't know whether that's possible with DIP 1000 > > as-is > > Not as far as I can tell. Marking it as scope in the function > body means it can't be returned, and marking the array in > GetoptResult as scope results in a syntax error. > > > In this particular case, it may make more sense to just let > > getopt be @safe on its own and just let the caller mark all of > > the uses of & as @trusted. > > This is thankfully the case currently. > > > If it's possible to mark GetoptResult with scope such that we > > can use scope without copying, then great, but if it's not, > > then I'm inclined to argue that we should just make sure that > > getopt itself is @safe and not worry about whether the caller > > is doing anything @system to call getopt or not. > > > > Regardless, this does raise a potential issue with scope and > > user-defined return types, and we should explore how possible > > it is for DIP 1000 to solve that problem without forcing copies > > that wouldn't be necessary in @system code. > > My cause for alarm with this limitation is this is one of the > issues (taking address of locals safely) that DIP1000 was > designed to solve. If, in practice, DIP1000 code can't be used > for this case when the code become sufficiently complex, then we > have a real problem on our hands. Well, it certainly looks like once user-defined types are being returned, DIP 1000 falls short. And even if it were improved to allow for member variables to be marked with scope such that the compiler allowed for arguments to escape via a return value (which may or may not make sense with how DIP 1000 works), we then have the problem with potentially needing to copy the member variables (or the entire struct) in the caller - either that or to cast and have it be @trusted. I don't know how a big a problem this is really going to be in practice, but it certainly shows that DIP 1000 isn't a silver bullet. - Jonathan M Davis
Re: Flaw in DIP1000? Returning a Result Struct in DIP1000
On Wednesday, 21 March 2018 at 17:13:40 UTC, Jack Stouffer wrote:
[snip]
How can we return non-scoped result variables constructed from
scope variables without copies?
If you re-wrote this so that it just had pointers, would it be
simpler?
Below is my attempt, not sure it's the same...
struct Foo
{
int b;
}
struct Bar
{
Foo* a;
}
Bar bar(scope int* a) @safe
{
Bar res;
Foo x = Foo(*a);
res.a =
return res;
}
void main() @safe
{
int x = 1;
bar();
}
Re: Flaw in DIP1000? Returning a Result Struct in DIP1000
On Wednesday, 21 March 2018 at 18:50:59 UTC, Jonathan M Davis wrote: The struct being returned would need to be marked with scope (or its members marked with scope) such that the compiler treated the result as containing values from the function arguments. I don't know whether that's possible with DIP 1000 as-is Not as far as I can tell. Marking it as scope in the function body means it can't be returned, and marking the array in GetoptResult as scope results in a syntax error. In this particular case, it may make more sense to just let getopt be @safe on its own and just let the caller mark all of the uses of & as @trusted. This is thankfully the case currently. If it's possible to mark GetoptResult with scope such that we can use scope without copying, then great, but if it's not, then I'm inclined to argue that we should just make sure that getopt itself is @safe and not worry about whether the caller is doing anything @system to call getopt or not. Regardless, this does raise a potential issue with scope and user-defined return types, and we should explore how possible it is for DIP 1000 to solve that problem without forcing copies that wouldn't be necessary in @system code. My cause for alarm with this limitation is this is one of the issues (taking address of locals safely) that DIP1000 was designed to solve. If, in practice, DIP1000 code can't be used for this case when the code become sufficiently complex, then we have a real problem on our hands.
Re: Flaw in DIP1000? Returning a Result Struct in DIP1000
On Wednesday, 21 March 2018 at 19:15:41 UTC, Meta wrote:
But the compiler doesn't like that. However, I _did_ get it
working by doing this:
GetoptResult getopt(T...)(scope T opts) @safe
{
return GetoptResult([Option(opts[0], opts[1])]);
}
Which is not ideal, obviously, but the notion that some code
has to be rewritten to accomodate ownership semantics is not a
new one; one of the major complaints I've seen about Rust is
that it requires you to adjust your coding style to satisfy the
borrow checker.
The problem here is that it's impossible to apply this to the
actual getopt code :/
Re: Flaw in DIP1000? Returning a Result Struct in DIP1000
On Wednesday, 21 March 2018 at 17:13:40 UTC, Jack Stouffer wrote:
Consider this example simplified from this PR
https://github.com/dlang/phobos/pull/6281
--
struct GetoptResult
{
Option[] options;
}
struct Option
{
string optShort;
string help;
}
GetoptResult getopt(T...)(scope T opts) @safe
{
GetoptResult res;
auto o = Option(opts[0], opts[1]);
res.options ~= o;
return res;
}
void main() @safe
{
bool arg;
getopt("arg", "info", );
}
--
$ dmd -dip1000 -run main.d
--
main.d(16): Error: scope variable o assigned to non-scope res
main.d(23): Error: template instance `onlineapp.getopt!(string,
string, bool*)` error instantiating
--
The only way I've found to make the code compile and retain the
pre-dip1000 behavior is to change the Option construction to
--
auto o = Option(opts[0].idup, opts[1].idup);
--
How can we return non-scoped result variables constructed from
scope variables without copies?
I thought that maybe adding a function to Option and marking it
as `scope` would work:
struct GetoptResult
{
Option[] options;
void addOptions(scope Option opt) @safe scope
{
//Error: scope variable opt assigned to non-scope this
options ~= opt;
}
}
But the compiler doesn't like that. However, I _did_ get it
working by doing this:
GetoptResult getopt(T...)(scope T opts) @safe
{
return GetoptResult([Option(opts[0], opts[1])]);
}
Which is not ideal, obviously, but the notion that some code has
to be rewritten to accomodate ownership semantics is not a new
one; one of the major complaints I've seen about Rust is that it
requires you to adjust your coding style to satisfy the borrow
checker.
Re: Flaw in DIP1000? Returning a Result Struct in DIP1000
On Wednesday, March 21, 2018 17:13:40 Jack Stouffer via Digitalmars-d wrote:
> Consider this example simplified from this PR
> https://github.com/dlang/phobos/pull/6281
>
> --
> struct GetoptResult
> {
> Option[] options;
> }
>
> struct Option
> {
> string optShort;
> string help;
> }
>
> GetoptResult getopt(T...)(scope T opts) @safe
> {
> GetoptResult res;
> auto o = Option(opts[0], opts[1]);
> res.options ~= o;
> return res;
> }
>
> void main() @safe
> {
> bool arg;
> getopt("arg", "info", );
> }
> --
>
> $ dmd -dip1000 -run main.d
>
> --
> main.d(16): Error: scope variable o assigned to non-scope res
> main.d(23): Error: template instance `onlineapp.getopt!(string,
> string, bool*)` error instantiating
> --
>
> The only way I've found to make the code compile and retain the
> pre-dip1000 behavior is to change the Option construction to
>
> --
> auto o = Option(opts[0].idup, opts[1].idup);
> --
>
> How can we return non-scoped result variables constructed from
> scope variables without copies?
The struct being returned would need to be marked with scope (or its members
marked with scope) such that the compiler treated the result as containing
values from the function arguments. I don't know whether that's possible
with DIP 1000 as-is, and that could have some pretty nasty consquences when
you consider how that then limits what can be done with the return value.
Even if we _can_ mark GetoptResult as scope in some manner so that the
copying isn't necessary, you're then just pushing the problem a level up. In
this case, that's probably not a big deal, since this is stuff that's just
going to be used in main and thrown away, but in the general case, having a
struct that won't let you escape any of its members means that you're going
to either need a struct with the same layout but without scope that you can
copy the scope on to, or you're going to need to pull out each of the
members individually to copy them or do whatever you need to do to work
around scope.
My gut reaction is that issues along these lines will either prevent the use
of scope when returning user-defined types as opposed to pointers or dynamic
arrays, and/or they'll force the kind of copying that you're complaining
about. But I don't think that I understand DIP 1000 well enough to know what
it's limitations really are in cases like this. Hopefully, Walter has an
answer.
However, I suspect that we're going to find cases where scope is a blunt
enough instrument that we're going to be forced to either drop it or use
@trusted in places (though in this case, using @trusted would be completely
unreasonable, because you can't guarantee that it _isn't_ a problem for
something from the caller to escape - not without examining the caller code,
which would translate to marking the caller as @trusted, not the function
being called). Actually, I'd be very surprised if we _didn't_ have cases
like that. The question is how frequent those cases are and whether issues
like this are going to pop up enough that it's going to usually make more
sense to simply use pure and manually examine code to mark it as @trusted
rather than use scope so that the compiler can mark a bunch of stuff as
@safe for us. As long as the function doesn't return scope, we're probably
fine, but once it starts returning scope, things start getting interesting.
In this particular case, it may make more sense to just let getopt be @safe
on its own and just let the caller mark all of the uses of & as @trusted. I
know that the bug report that sparked this is trying to make using getopt
completely @safe without requiring the use of @trusted at all, but I don't
think that being forced to allocate memory is worth that, and I don't think
that mucking around with getopt to make it work with ref is worth that.
There are advantages to getopt taking pointers, and I'd rather not see
getopt's API change in a way that breaks code. If getopt itself is @safe,
then it's trivial for the caller to determine that their code is @safe in
spite of the use of & and thus either use @trusted appropriately or just not
bother, since it's in main, which usually isn't doing much fancy. So, I
_really_ don't think that making calling getopt inherently @safe is worth
code breakage if it comes to that, and I don't think that it's worth
allocating memory that we otherwise wouldn't have to allocate.
If it's possible to mark GetoptResult with scope such that we can use scope
without copying, then great, but if it's not, then I'm inclined to argue
that we should just make sure that getopt itself is @safe and not worry
about whether the caller is doing anything @system to call getopt or not.
Regardless, this does raise a potential issue with scope and user-defined
return types, and we should explore how possible it is for DIP 1000 to solve
that problem without forcing copies that wouldn't be necessary in @system
code.
- Jonathan M Davis
Flaw in DIP1000? Returning a Result Struct in DIP1000
Consider this example simplified from this PR
https://github.com/dlang/phobos/pull/6281
--
struct GetoptResult
{
Option[] options;
}
struct Option
{
string optShort;
string help;
}
GetoptResult getopt(T...)(scope T opts) @safe
{
GetoptResult res;
auto o = Option(opts[0], opts[1]);
res.options ~= o;
return res;
}
void main() @safe
{
bool arg;
getopt("arg", "info", );
}
--
$ dmd -dip1000 -run main.d
--
main.d(16): Error: scope variable o assigned to non-scope res
main.d(23): Error: template instance `onlineapp.getopt!(string,
string, bool*)` error instantiating
--
The only way I've found to make the code compile and retain the
pre-dip1000 behavior is to change the Option construction to
--
auto o = Option(opts[0].idup, opts[1].idup);
--
How can we return non-scoped result variables constructed from
scope variables without copies?
