[Issue 18788] New: static arrays with a length specified at runtime should dynamically allocate on the stack

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18788

  Issue ID: 18788
   Summary: static arrays with a length specified at runtime
should dynamically allocate on the stack
   Product: D
   Version: D2
  Hardware: All
OS: All
Status: NEW
  Severity: enhancement
  Priority: P1
 Component: dmd
  Assignee: nob...@puremagic.com
  Reporter: slavo5...@yahoo.com

Consider the following contrived example in C:

---main.c
#include 
#include 

int writeln(char* s)
{
size_t len = strlen(s);
size_t newLen = len + 1;
char buf[newLen + 1]; // dynamically allocated on the stack
memcpy(buf, s, len);
buf[len] = '\n';
buf[newLen] = '\0';
fprintf(stdout, buf);
}

int main()
{
writeln("Hello, World!");
return 0;
}

I think D should do something similar with static arrays, but currently the
following fails to compile:

---main.d
import core.stdc.string;
import core.stdc.stdio;

void writeln(string s)
{
size_t len = s.length;
size_t newLen = len + 1;
char[newLen + 1] buf; // Error: variable newLen cannot be read at
compile time
memcpy(buf.ptr, s.ptr, len);
buf[len] = '\n';
buf[newLen] = '\0';
fprintf(stdout, buf.ptr);
}

int main()
{
writeln("Hello, World!");
return 0;
}

Templates or mixins may be able to be used to workaround this limitation, but
would ultimately result in code bloat if used extensively, and one of the use
cases for this is in the domain of resource constrained microcontrollers.

A possible implementation would be to have an alloca-equivalent druntime hook
with hopefully a much better name, and have the compiler generate a call to it
when appropriate.  The d runtime hook could then forward to the platform's
`alloca`, if one exists, or a custom D implementation could be implemented and
optimized for each platform.

Perhaps `scope` could also even play an interesting roll here.

--

[Issue 18788] static arrays with a length specified at runtime should dynamically allocate on the stack

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18788

Nicholas Wilson  changed:

   What|Removed |Added

 CC||iamthewilsona...@hotmail.co
   ||m

--- Comment #1 from Nicholas Wilson  ---
this is the role of std.experimental.allocator.StackFront and friends (e.g.
InSituRegion), there is probably room for runtime choice of StackFront and
InSituRegion such that there is no stack allocation if the length exceeds the
InSituRegion size.

Perhaps the compiler should point users to this once it is made
non-experimental, but this should not be a language feature as it is unsafe and
may inadvertently blow the stack.

--

[Issue 3925] Missed escaping reference of a local variable

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=3925

Nick Treleaven  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 CC||n...@geany.org
 Resolution|--- |FIXED

--- Comment #8 from Nick Treleaven  ---
All code samples here fail to compile with @safe and -dip1000, dmd v2.079.1,
except this one:

(In reply to Michel Fortin from comment #3)
> There is a similar problem with delegates:

I looked to see if this is filed, the closest I could find is Issue 18738,
which is strongly related. Closing this.

--

[Issue 18738] [scope] scope delegates can be escaped via closure

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18738

Nick Treleaven  changed:

   What|Removed |Added

 CC||n...@geany.org

--- Comment #1 from Nick Treleaven  ---
Closures don't seem to be checked for scope pointers at all:

@safe:
void delegate() foo(ref int a) {
return { a++; };
}

void delegate() bar() {
int a;
return foo(a); // leaking reference to `a`
}

--

[Issue 18789] New: std.stdio messes up UTF conversions on output

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18789

  Issue ID: 18789
   Summary: std.stdio messes up UTF conversions on output
   Product: D
   Version: D2
  Hardware: All
OS: All
Status: NEW
  Severity: normal
  Priority: P1
 Component: phobos
  Assignee: nob...@puremagic.com
  Reporter: ag0ae...@gmail.com


import std.stdio;
void main()
{   
// converting to char
{
auto f = File("test.txt", "w");
f.writeln("\U0001F608"w); // UTFException
}

// converting to wchar_t
{
auto f = File("test.txt", "w,ccs=UTF16LE");
// from char
f.writeln("ö"); // writes garbage
f.writeln("\U0001F608"); // ditto
// from wchar
f.writeln("\U0001F608"w); // leads to ErrnoException
}
}


PR follows.

--

[Issue 18789] std.stdio messes up UTF conversions on output

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18789

ag0aep6g  changed:

   What|Removed |Added

   Keywords||pull
   Assignee|nob...@puremagic.com|ag0ae...@gmail.com

--- Comment #1 from ag0aep6g  ---
(In reply to ag0aep6g from comment #0)
> PR follows.

https://github.com/dlang/phobos/pull/6469

--

[Issue 3024] Array slicing allows returning an escaping reference to a local stack variable

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=3024

Nick Treleaven  changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 CC||n...@geany.org
 Resolution|--- |FIXED

--- Comment #3 from Nick Treleaven  ---
(In reply to david from comment #0)
> byte[] func2()
> {
>   byte[16] v= [65,65,65,65,
>65,65,65,65,
>65,65,65,65,
>65,65,65,65];
>   return v[0..16];
> }

This is now fixed (dmd v2.079.1):

onlineapp.d(7): Error: returning v[0..16] escapes a reference to local variable
v

--

[Issue 18788] static arrays with a length specified at runtime should dynamically allocate on the stack

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18788

--- Comment #2 from Mike Franklin  ---
Yes it is unsafe, but not any less safe than

void unsafeRecursion(size_t iterations)
{
if(iterations)
{
byte[256] buffer;
unsafeRecursion(--iterations);
}
}

As a compromise it could be restricted to @system code.

--

[Issue 15348] std.stdio.writef format specifier error message

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15348

Nick Treleaven  changed:

   What|Removed |Added

 CC||n...@geany.org

--- Comment #1 from Nick Treleaven  ---
Just for reference, you can now use a compile-time format string - you get an
error for the right source line (after the first error):

writef!"%*10d"(100);

/dlang/dmd/src/phobos/std/stdio.d(3878): Error: static assert:  "$ expected"
onlineapp.d(5):instantiated from here: writef!("%*10d", int)

Fixing this issue could still be useful for runtime format strings.

--

[Issue 18788] static arrays with a length specified at runtime should dynamically allocate on the stack

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18788

Adam D. Ruppe  changed:

   What|Removed |Added

 CC||destructiona...@gmail.com

--- Comment #3 from Adam D. Ruppe  ---
I'd actually just like to redefine `new` to mean the compiler is free to stack
allocate it when it can prove the lifetime does not exceed that of the function
(and the scope annotations can hint/prove/override this, as it already does for
classes[!])

--

[Issue 18790] can't put a const(char)[] into a char[]

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18790

--- Comment #3 from Steven Schveighoffer  ---
(In reply to Jonathan M Davis from comment #1)
> (In reply to Steven Schveighoffer from comment #0)
> > The most basic string output range is char[]. Why can't I put a string in
> > there?
> 
> Because char[] isn't an output range at all. isOutputRange!(char[], char) is
> false. No array of char or wchar is an output range, as annoying as that may
> be.

Well, actually, it's isOutputRange!(char[], dchar), and right it's not an
output range. But not because it can't be done. Even with decoding/encoding it
could be done. It just can't be done through the general 'assign-to-front'
option. It would have to be a special case inside the put function itself.

The issue I have is that I want to build a string on the stack that I can use
for example for looking up a value in an AA. I don't want to use Appender
because I'm trying to keep it on the stack.

I have figured out a workaround, I can use byCodeUnit to make it "work", but I
have to do some flips around the type to make it back to a char[].

> > At the very least, the message should be clearer (obviously, we can put a
> > string into a char[]!).
> 
> Well, actually, you _can't_ "put" string into a char[]. put doesn't work
> with char[]. So, it's perfectly accurate. However, it's also a terrible
> message, because all it's saying is that put doesn't compile with the given
> arguments, not why.

There's another factor: the term 'put' is also an english verb. I didn't even
notice that it's talking about put the function, I thought it was just telling
me in general something that's not possible that clearly is. Note that I first
saw the error when trying to use formattedWrite, so I wasn't even calling put.
In the bug report, though, I figured I would boil it down to the lowest level.

> It really should say something more like "char[] isn't
> an output range", and if there's a similar static assert for types that
> _are_ output ranges but don't accept the given type, then they should have a
> different message for that that makes it clear that the output range doesn't
> have a put method that accepts the given argument type, and
> std.range.primitives.put can't convert it for you.

That would have been a better, yet still frustrating, error message. But at
least I would know what the real problem is!

--

[Issue 18790] can't put a const(char)[] into a char[]

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18790

hst...@quickfur.ath.cx changed:

   What|Removed |Added

 CC||hst...@quickfur.ath.cx

--

[Issue 18790] can't put a const(char)[] into a char[]

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18790

--- Comment #5 from Steven Schveighoffer  ---
(In reply to Jonathan M Davis from comment #1)
> Because char[] isn't an output range at all. isOutputRange!(char[], char) is
> false.

Something to consider: isOutputRange!(char[], char) is false because
put(char[], char) doesn't compile.

So it's not a good explanation as to why put doesn't work. We can make put
work, and then isOutputRange will be true. Fun fact: put has NO template
constraints. Because it IS the source of the template constraints.

This is something so arbitrary, and I'm working on the code now to get it to
work, there are some really self-defeating reasons why this isn't working. This
should be a no brainer. The reasons for char[] not being an input range of char
don't translate to output ranges.

--

[Issue 18525] Constraint on std.algorithm.mutation.remove fails with char[]

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18525

github-bugzi...@puremagic.com changed:

   What|Removed |Added

 Status|NEW |RESOLVED
 Resolution|--- |FIXED

--

[Issue 18525] Constraint on std.algorithm.mutation.remove fails with char[]

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18525

--- Comment #2 from github-bugzi...@puremagic.com ---
Commits pushed to master at https://github.com/dlang/phobos

https://github.com/dlang/phobos/commit/8d314b5ffbeb67adf27c68e11976a1d63917717b
Fix Issue 18525: std.algorithm.mutate.remove should work on mutable strings

https://github.com/dlang/phobos/commit/7ba6fbb66c960953d61dc334eea1dc9be5220df1
Merge pull request #6328 from John-Colvin/string_remove

Fix Issue 18525: std.algorithm.mutate.remove should work on mutable strings
merged-on-behalf-of: Nathan Sashihara 

--

[Issue 18790] New: can't put a const(char)[] into a char[]

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18790

  Issue ID: 18790
   Summary: can't put a const(char)[] into a char[]
   Product: D
   Version: D2
  Hardware: All
OS: All
Status: NEW
  Severity: normal
  Priority: P1
 Component: phobos
  Assignee: nob...@puremagic.com
  Reporter: schvei...@yahoo.com

The most basic string output range is char[]. Why can't I put a string in
there?

auto buf = new char[100];

put(buf, "hello"); // static assert: "Cannot put a string into a char[]."

I found this because I'm trying to write to a stack buffer string to use later
using formattedWrite.

The answer to my question is (as usual) auto-decoding. ugh.

At the very least, the message should be clearer (obviously, we can put a
string into a char[]!).

--

[Issue 18790] can't put a const(char)[] into a char[]

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18790

Jonathan M Davis  changed:

   What|Removed |Added

 CC||issues.dl...@jmdavisprog.co
   ||m
   Severity|normal  |enhancement

--- Comment #1 from Jonathan M Davis  ---
(In reply to Steven Schveighoffer from comment #0)
> The most basic string output range is char[]. Why can't I put a string in
> there?

Because char[] isn't an output range at all. isOutputRange!(char[], char) is
false. No array of char or wchar is an output range, as annoying as that may
be.

> The answer to my question is (as usual) auto-decoding. ugh.

Yes. And I don't know what the solution to that is, not without actually
getting rid of autodecoding, and I don't see how we can do that without being
willing to silently break a lot of code or doing something like make it so that
strings aren't ranges anymore (at least temporarily), both of which would be
_really_ disruptive.

> At the very least, the message should be clearer (obviously, we can put a
> string into a char[]!).

Well, actually, you _can't_ "put" string into a char[]. put doesn't work with
char[]. So, it's perfectly accurate. However, it's also a terrible message,
because all it's saying is that put doesn't compile with the given arguments,
not why. It really should say something more like "char[] isn't an output
range", and if there's a similar static assert for types that _are_ output
ranges but don't accept the given type, then they should have a different
message for that that makes it clear that the output range doesn't have a put
method that accepts the given argument type, and std.range.primitives.put can't
convert it for you.

--

[Issue 18790] can't put a const(char)[] into a char[]

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18790

--- Comment #4 from hst...@quickfur.ath.cx ---
Given that std.range.put goes out of its way to support putting single
elements, multiple elements, and in some cases even dchars into strings, I'm
shocked that putting dchar into char[] doesn't work. This needs to be made to
work.

--

[Issue 15869] RVO can overwrite argument

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15869

Walter Bright  changed:

   What|Removed |Added

 CC||bugzi...@digitalmars.com

--- Comment #9 from Walter Bright  ---
The most pragmatic solution is to not allow taking a reference to an
unconstructed field.

--

[Issue 18790] can't put a const(char)[] into a char[]

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18790

--- Comment #6 from Steven Schveighoffer  ---
PR: https://github.com/dlang/phobos/pull/6471

--

[Issue 15869] RVO can overwrite argument

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15869

--- Comment #10 from Walter Bright  ---
https://github.com/dlang/dmd/pull/8200

--

[Issue 18790] can't put a const(char)[] into a char[]

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18790

--- Comment #7 from Jonathan M Davis  ---
(In reply to hsteoh from comment #4)
> Given that std.range.put goes out of its way to support putting single
> elements, multiple elements, and in some cases even dchars into strings, I'm
> shocked that putting dchar into char[] doesn't work. This needs to be made
> to work.

I think that what it comes down to is that because of the whole auto-decoding
fiasco, when Andrei originally did the work on output ranges, he made it so
that char[] and wchar[] were not output ranges for all of the reasons that they
weren't input ranges. However, with the improvements to put over the years, it
becomes increasingly silly for it to not work. And if we can make the changes
for output ranges without breaking any existing code, then we definitely
should.

--

[Issue 18790] can't put a const(char)[] into a char[]

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18790

Nicholas Wilson  changed:

   What|Removed |Added

 CC||iamthewilsona...@hotmail.co
   ||m

--- Comment #2 from Nicholas Wilson  ---
dupe of https://issues.dlang.org/show_bug.cgi?id=18472 (see comment 10).

@JMDavies the problem is not that you can't `put(buf, "hello");`
but that formattedWrite does not force that to happen. Irrespective of the
intricacies of unicode and auto decoding 
---
char[] buf = new char[100];
buf.formattedWrite("This is a number: %s", 5); // Error cannot put a
const(char)[] into a char[]
---

should work.

--

[Issue 18788] static arrays with a length specified at runtime should dynamically allocate on the stack

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18788

--- Comment #5 from Mike Franklin  ---
Another way of achieving the same result would be to make a `scope`d dynamic
array allocate on the stack

---
class C { }

void main()
{
scope C c = new C();   // `c` is allocated on the stack
scope int[] i = [1, 2, 3]; // `i` is inconsistently allocated on the heap
}

I think this should be consistent between classes and dynamic arrays; `i`
should be allocated on the stack, and appending to it should allocate
`realloc`ate it on the stack.

--

[Issue 15869] RVO can overwrite argument

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15869

--- Comment #11 from Ketmar Dark  ---
ahem. i was under impression that "constructors" in D are actually
"post-initializers". i.e. that object must be fully initialized with default
values before calling ctor, and can't have "unconstructed" anything (unless it
has `=void` as default value, of course). that is, in the given case, compiler
should detect that `a` is actually used, and fully initialize it before calling
ctor of `XX`. or generate error on *any* "use-before-init" access, including
things like "v++" and such.

that is, it looks to me that "pragmatic solution" is not really solving
anything in this case: it just hacks around one very specific case, and in the
same time makes D behavior even more unintuitive.

--

[Issue 15869] RVO can overwrite argument

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=15869

--- Comment #12 from Walter Bright  ---
(In reply to Walter Bright from comment #9)
> The most pragmatic solution is to not allow taking a reference to an
> unconstructed field.

That proved unworkable.

One that does work is to regard taking the address of a field as "initializing"
it. Then,

a = clobber();

is regarded as an assignment, rather than a construction, and the RVO is set to
a temporary, not `a`.

Nothing else I could think of was palatable.

--

[Issue 18788] static arrays with a length specified at runtime should dynamically allocate on the stack

[d-bugmail--- via Digitalmars-d-bugs]

https://issues.dlang.org/show_bug.cgi?id=18788

Jonathan M Davis  changed:

   What|Removed |Added

 CC||issues.dl...@jmdavisprog.co
   ||m

--- Comment #4 from Jonathan M Davis  ---
(In reply to Adam D. Ruppe from comment #3)
> I'd actually just like to redefine `new` to mean the compiler is free to
> stack allocate it when it can prove the lifetime does not exceed that of the
> function (and the scope annotations can hint/prove/override this, as it
> already does for classes[!])

Well, in principle, with DIP 1000, the case where scope on classes allocates
the class on the stack simply becomes a compiler optimization rather than the
specific purpose of scope (rather, the purpose of scope at that point is to
give the compiler the information it needs to guarantee that no pointers or
references or whatnot to the variable or object escape). As such, I don't see
any reason why that optimization couldn't be expanded to cover other types
allocated with new.

--