Re: How to make a function table?
On Saturday, 16 February 2019 at 05:06:55 UTC, mnar53 wrote:
Absolutely a newbie to D. This code mimics c, but unfortunately
does not work:
import std.typecons;
import std.math;
alias double function(double) UNARY;
UNARY[] FCNS = [sin, cos, tan];
double[][] MAP (int idx, double[][] args) nothrow {
UNARY f = FCNS[idx];
foreach(i; 0 .. args.length)
foreach(j; 0 .. args[i].length)
args[i][j] = f (args[i][j]);
return args;
}
what's wrong with it?
I assume, this would do its job:
´´´
import std.typecons;
import std.math;
alias UNARY = double function(double);
UNARY[] FCNS = [, , ];
double[][] MAP (size_t idx, double[][] args) {
UNARY f = FCNS[idx];
foreach(i; 0 .. args.length)
foreach(j; 0 .. args[i].length)
args[i][j] = f (args[i][j]);
return args;
}
void main(){
double[][] args;
args.length = 3;
foreach(ref el; args)
el.length = 3;
import std.random : uniform01, dice;
foreach(l; args)
foreach(ref el; l)
el = uniform01;
import std.range : iota;
args = MAP(dice(FCNS.length.iota), args);
import std.stdio : writeln;
writeln(args);
}
´´´
Re: How does Rebindable suppress the compiler's optimizations for immutable?
Thanks for the detailed answers! Yes, I accept that immutable guarantees should be implemented only during @safe that doesn't call into @trusted. On Friday, 15 February 2019 at 18:59:36 UTC, H. S. Teoh wrote: At the very least, such [union] code should be automatically @system. Sensible. Honestly, I'm pretty sure that Rebindable technically violates the type system to do what it does. Hmm, I remember we discussed this, and I feel the same now about Rebindable. Either the spec gets extra rules for @trusted or unions, or Rebindable generates latent bugs. Think of immutable as hint for the programmer, not for the compiler. Right, if the compilers don't use it yet, I'm fine with that interpretation. It's merely strange that we have this very restrictive const/immutable that is advertized to help optimization, but then the compiler won't take advantage. Let's see how this develops in the long term, whether the spec gets clearer on the allowed optimization.
How to make a function table?
Absolutely a newbie to D. This code mimics c, but unfortunately
does not work:
import std.typecons;
import std.math;
alias double function(double) UNARY;
UNARY[] FCNS = [sin, cos, tan];
double[][] MAP (int idx, double[][] args) nothrow {
UNARY f = FCNS[idx];
foreach(i; 0 .. args.length)
foreach(j; 0 .. args[i].length)
args[i][j] = f (args[i][j]);
return args;
}
what's wrong with it?
Re: How does Rebindable suppress the compiler's optimizations for immutable?
On Fri, Feb 15, 2019 at 03:50:33AM -0700, Jonathan M Davis via Digitalmars-d-learn wrote: > On Friday, February 15, 2019 3:06:34 AM MST Kagamin via Digitalmars-d-learn > wrote: > > Union is just a pretty cast, type system guarantees don't hold for > > it. > > Well, technically, what's supposed to be the case is that when you > cast, the type system guarantees still hold but it's up to the > programmer to make sure that they do - just like how it's up to the > programmer to ensure that @trusted code is really @safe. Currently the compiler will automatically mark any code as @system that tries to access a union field that involves a pointer overlapping with a non-pointer. So you cannot just get away from @safe just by using a union. In the same vein, I think any attempt to access a union that contains overlapping immutable / non-immutable fields should automatically taint the immutable value somehow so that no unsafe optimizations happen based on immutability. But that gets into the tricky territory of what happens if you assign an overlapping immutable field to an immutable local variable, and then access the local variable -- will the compiler optimize based on immutable, which may be invalid because of the union? I'm tempted to say that a union that mixes immutable / mutable ought to be illegal. How can you possibly guarantee anything about immutability if it overlaps with mutable fields?? It makes no sense. At the very least, such code should be automatically @system. At the @system level sometimes you have to do exactly this -- e.g., the GC may allocate a segment of memory for an immutable variable, but during the pointer scan the GC has a mutable reference to the immutable memory (it has to, since at some point when the memory is collected the GC has to be able to reassign it to mutable data again), and it's basically a matter of trust that that GC doesn't go about modifying immutable data. (Which brings up interesting questions about how immutability might interact with a moving / compacting GC that may need to rewrite potentially immutable pointers. But that's tangential to this discussion.) So anyway, all of this seems to suggest that optimizations based on immutable really only can take place meaningfully within @safe code, provided we make accessing immutables in union with mutables a @system operation. All interactions between immutable-optimized @safe code and potentially immutable-breaking @system code would then be forced to take place through @trusted APIs, which would seem to be the correct design. > Rebindable is a pretty weird case with what it's doing, but I'm pretty > sure that it's actually violating the type system with what it does in > that you can't rely on the value of the immutable reference itself not > being mutated even though it's immutable. The type system guarantees > for what the reference refers to are maintained but not the guarantees > for the actual reference. In practice, I don't think that it's > actually a problem, but in principle, it's violating the type system, > and a particularly aggressive optimizer could make a bad assumption. > However, given that a union is involved, and as such the "cast" is > built into the type, I question that any optimizer would ever make the > wrong assumption about Rebindable. [...] It probably only means that current D compilers aren't optimizing based on immutable like they're supposedly able to. If/when optimizers start taking advantage of this, we're going to see UB in many more places than we may realize currently. I think the correct approach is to restrict immutable optimizations to only @safe code, and force all immutable casts, including accessing immutables in unions where they might overlap with mutables, a @system operation, thus mandating any interaction with immutable optimizations via @trusted interfaces. T -- In order to understand recursion you must first understand recursion.
Re: Casting int[] to ubyte[]: element wise cast or slice cast
On Friday, 15 February 2019 at 17:18:28 UTC, H. S. Teoh wrote: Why? because that's the only case where you can reinterpret the .sizeof*.length bytes as an array of a different type. There is no conversion, the cast works as a reinterpretation. It's by design. But my problem is that it *isn't* reinterpreted in the case of an enum / array literal, the cast actually transcribes similar to your .map example. Look carefully at the difference of case A and B in my example code: the only difference is whether the int array uses auto or enum. I'm not going from ubyte to int, I'm going from int to ubyte which should always be possible by multiplying the length by 4, in my case: int.sizeof * 2 == ubyte.sizeof * 8 == 8 This happens in case B of my example code, but not in case A. There the cast goes from 4*2 bytes to 1*2 instead of 1*8 and I don't get a runtime error.
Re: Generalizing over function pointers and delegates
On Friday, 15 February 2019 at 17:28:45 UTC, H. S. Teoh wrote:
On Fri, Feb 15, 2019 at 05:40:39PM +0100, ag0aep6g via
Digitalmars-d-learn wrote:
Your fun_to_dlg fails when the function has parameters.
Yes. Delegates are basically syntactic sugar for a function
pointer with an implicit first parameter. I.e., a delegate like:
int delegate(string) dg;
is under the hood implemented as the equivalent of:
struct _delegate {
int function(T* context, string) funcptr;
T* context;
int opCall(string s) { return funcptr(context, s); }
}
where T is an appropriate context type, whether an aggregate
(struct / class) or an anonymous struct that closes over
whatever variables the delegate accesses in its containing
scope.
For this reason, casting a function pointer to a delegate will
not work properly, because the first arguments and number of
parameters would not match.
T
I love this forum!
Re: Generalizing over function pointers and delegates
On Fri, Feb 15, 2019 at 05:40:39PM +0100, ag0aep6g via Digitalmars-d-learn
wrote:
> On 15.02.19 15:20, Bastiaan Veelo wrote:
> > Exploiting this, it is possible to explicitly convert a function
> > pointer into a delegate [2]:
> > ```
> > Ret delegate(Args args) fun_to_dlg(Ret, Args...)(Ret function(Args args)
> > fun)
> > {
> > Ret delegate(Args) dlg;
> > dlg.funcptr = fun;
> > return dlg;
> > }
> > ```
[...]
> Your fun_to_dlg fails when the function has parameters.
Yes. Delegates are basically syntactic sugar for a function pointer with
an implicit first parameter. I.e., a delegate like:
int delegate(string) dg;
is under the hood implemented as the equivalent of:
struct _delegate {
int function(T* context, string) funcptr;
T* context;
int opCall(string s) { return funcptr(context, s); }
}
where T is an appropriate context type, whether an aggregate
(struct / class) or an anonymous struct that closes over whatever
variables the delegate accesses in its containing scope.
For this reason, casting a function pointer to a delegate will not work
properly, because the first arguments and number of parameters would not
match.
T
--
Without outlines, life would be pointless.
Re: Generalizing over function pointers and delegates
On Friday, 15 February 2019 at 16:40:39 UTC, ag0aep6g wrote: Your fun_to_dlg fails when the function has parameters. Hah ok. std.functional.toDelegate() does work in its place though. As far as I see, it would be possible make the conversion would work by changing how a delegate's context is passed [1]. But I didn't pursue that idea further, and no one else picked it up either. [1] https://forum.dlang.org/post/ofc0lj$2u4h$1...@digitalmars.com Interesting. Thanks for the link.
Re: Casting int[] to ubyte[]: element wise cast or slice cast
On Fri, Feb 15, 2019 at 04:17:12PM +, Dennis via Digitalmars-d-learn wrote: > I assumed that casting an int[] to a ubyte[] would keep all bytes and > quadruple the length of the original array. But when the array is a > literal, it keeps the same length but truncates every int element to a > ubyte: That's correct. If you want to *transcribe* a ubyte[] to an int[], what you want is probably something like this: import std.conv : to; ubyte[] data = ...; int[] result = data.map!(b => b.to!int).array; [...] > I looked at the spec and found this: > > https://dlang.org/spec/expression.html#cast_expressions > "Casting a dynamic array to another dynamic array is done only if the > array lengths multiplied by the element sizes match. The cast is done > as a type paint, with the array length adjusted to match any change in > element size. If there's not a match, a runtime error is generated." > > So is this a bug or am I missing something? Read it again carefully: the casting is done "only if the array lengths multiplied by the element sizes match". In other words: T.sizeof * T[].length == U.sizeof * U[].length Why? because that's the only case where you can reinterpret the .sizeof*.length bytes as an array of a different type. There is no conversion, the cast works as a reinterpretation. It's by design. T -- I'm still trying to find a pun for "punishment"...
Re: Generalizing over function pointers and delegates
On 15.02.19 15:20, Bastiaan Veelo wrote:
Exploiting this, it is possible to explicitly convert a function pointer
into a delegate [2]:
```
Ret delegate(Args args) fun_to_dlg(Ret, Args...)(Ret function(Args args)
fun)
{
Ret delegate(Args) dlg;
dlg.funcptr = fun;
return dlg;
}
```
[...]
This may be preferable to the overload, depending on the number of calls
like this. But since this is allowed and working, why can't it be done
automatically? Or, when implicit conversion is dangerous, why doesn't a
cast exist to the effect of my template? Have I overlooked an easier way
of doing this?
Your fun_to_dlg fails when the function has parameters.
As far as I see, it would be possible make the conversion would work by
changing how a delegate's context is passed [1]. But I didn't pursue
that idea further, and no one else picked it up either.
[1] https://forum.dlang.org/post/ofc0lj$2u4h$1...@digitalmars.com
Casting int[] to ubyte[]: element wise cast or slice cast
I assumed that casting an int[] to a ubyte[] would keep all bytes
and quadruple the length of the original array. But when the
array is a literal, it keeps the same length but truncates every
int element to a ubyte:
```
import std.stdio;
void main()
{
// enum:
enum litA = [0x10203040, 0x50607080];
pragma(msg, typeof(litA));
ubyte[] a = cast(ubyte[]) litA;
// auto:
auto litB = [0x10203040, 0x50607080];
pragma(msg, typeof(litB));
ubyte[] b = cast(ubyte[]) litB;
writeln(a);
writeln(b);
}
```
Prints:
```
int[]
int[]
[64, 128]
[64, 48, 32, 16, 128, 112, 96, 80]
```
I looked at the spec and found this:
https://dlang.org/spec/expression.html#cast_expressions
"Casting a dynamic array to another dynamic array is done only if
the array lengths multiplied by the element sizes match. The cast
is done as a type paint, with the array length adjusted to match
any change in element size. If there's not a match, a runtime
error is generated."
So is this a bug or am I missing something?
Re: Generalizing over function pointers and delegates
On Friday, 15 February 2019 at 14:30:45 UTC, Alex wrote:
There is
https://dlang.org/library/std/functional/to_delegate.html
Ah, there it is :-) Thanks.
A templated function also works.
```
int genfun(F)(F dg) {return dg();}
int top_level() {return -1;}
void main()
{
int nested() {return -2;}
assert(genfun(_level) == -1);
assert(genfun() == -2);
}
```
Re: Generalizing over function pointers and delegates
On Friday, 15 February 2019 at 14:20:44 UTC, Bastiaan Veelo wrote:
Given a function taking a delegate, for example
```
int fun(int delegate() dg) {return dg();}
```
Sometimes we need to call `fun` with a pointer to a nested
function and other times with a pointer to a top level
function. As function pointers do not implicitly convert to
delegates, this does not work without jumping through hoops.
One option is to define an overload:
```
int fun(int function() fn)
{
int nested_fn()
{
return fn();
}
return fun(_fn);
}
```
This is clunky and rather a lot of extra lines to work around a
language limitation. The limitation seems somewhat artificial,
because a delegate /can/ be initialized referencing a top level
function (the spec [1] limits that to delegate declarations at
module scope, but this limit does not seem to apply [2]). But
you cannot /assign/ it a top level function. You can however
explicitly assign the `.funcptr` referencing the top level
function, leaving the stack frame pointer null.
Exploiting this, it is possible to explicitly convert a
function pointer into a delegate [2]:
```
Ret delegate(Args args) fun_to_dlg(Ret, Args...)(Ret
function(Args args) fun)
{
Ret delegate(Args) dlg;
dlg.funcptr = fun;
return dlg;
}
```
allowing
```
int top_level() {return -1;}
void main()
{
assert(fun(fun_to_dlg(_level)) == -1);
}
```
This may be preferable to the overload, depending on the number
of calls like this. But since this is allowed and working, why
can't it be done automatically? Or, when implicit conversion is
dangerous, why doesn't a cast exist to the effect of my
template? Have I overlooked an easier way of doing this?
Thanks for any input!
Bastiaan.
P.S. For the record, the spec also says this: "Future
directions: Function pointers and delegates may merge into a
common syntax and be interchangeable with each other." I wish
that was the case already.
[1] Point 9 in https://dlang.org/spec/function.html#closures
[2] https://run.dlang.io/is/x8HJaW
There is
https://dlang.org/library/std/functional/to_delegate.html
Generalizing over function pointers and delegates
Given a function taking a delegate, for example
```
int fun(int delegate() dg) {return dg();}
```
Sometimes we need to call `fun` with a pointer to a nested
function and other times with a pointer to a top level function.
As function pointers do not implicitly convert to delegates, this
does not work without jumping through hoops.
One option is to define an overload:
```
int fun(int function() fn)
{
int nested_fn()
{
return fn();
}
return fun(_fn);
}
```
This is clunky and rather a lot of extra lines to work around a
language limitation. The limitation seems somewhat artificial,
because a delegate /can/ be initialized referencing a top level
function (the spec [1] limits that to delegate declarations at
module scope, but this limit does not seem to apply [2]). But you
cannot /assign/ it a top level function. You can however
explicitly assign the `.funcptr` referencing the top level
function, leaving the stack frame pointer null.
Exploiting this, it is possible to explicitly convert a function
pointer into a delegate [2]:
```
Ret delegate(Args args) fun_to_dlg(Ret, Args...)(Ret
function(Args args) fun)
{
Ret delegate(Args) dlg;
dlg.funcptr = fun;
return dlg;
}
```
allowing
```
int top_level() {return -1;}
void main()
{
assert(fun(fun_to_dlg(_level)) == -1);
}
```
This may be preferable to the overload, depending on the number
of calls like this. But since this is allowed and working, why
can't it be done automatically? Or, when implicit conversion is
dangerous, why doesn't a cast exist to the effect of my template?
Have I overlooked an easier way of doing this?
Thanks for any input!
Bastiaan.
P.S. For the record, the spec also says this: "Future directions:
Function pointers and delegates may merge into a common syntax
and be interchangeable with each other." I wish that was the case
already.
[1] Point 9 in https://dlang.org/spec/function.html#closures
[2] https://run.dlang.io/is/x8HJaW
Re: Should D file end with newline?
On Wednesday, 13 February 2019 at 05:13:12 UTC, sarn wrote: On Tuesday, 12 February 2019 at 20:03:09 UTC, Jonathan M Davis wrote: So, I'd say that it's safe to say that dmd The whole thing just seems like a weird requirement that really shouldn't be there, Like I said in the first reply, FWIW, it's a POSIX requirement. Turns out most tools don't care (and dmd is apparently one of them). If you want an easy counterexample, try the wc command (it miscounts lines for non-compliant files). I've never seen that break an actual build system, which is why I said you could mostly get away with it. On the other hand, being POSIX-compliant always works. it matters even less if text editors are automatically appending newlines to files if they aren't there whether they show them or not, since if that's the case, you'd have to really work at it to have files not ending with newlines anyway. There are definitely broken text editors out there that won't add the newline (can't think of names). Like Jacob Carlborg said, Github flags the files they generate. hexdump shows a newline followed by a null character followed by a newline after the carriage return. hexdump is printing little-endian 16b by default, so I think that's just two newlines followed by a padding byte from hexdump. Try using the -c or -b flag and you probably won't see any null byte. Curiously, if I create a .cpp or .c file with vim and have it end with a curly brace, vim _does_ append a newline followed by a null character followed by a newline at the end of the file. So, I guess that vim looks at the extension and realizes that C/C++ has such a requirement and takes care of it for you, but it does not think that .d files need them and adds nothing extra for them. It doesn't add anything for a .txt file when I tried it either. Are you sure? vim is supposed to add the newline for all text files because that's POSIX. It does on my (GNU/Linux) machine. A lots of fgets() based tools on Unix systems fail to read the last line if it doesn't contain a line feed character at the end. Afaicr glibc implementation does not have that problem but a lot of other standard C libs do. When we were still on Solaris we had to be very careful with that, as strange things could happen when using sed, awk, wc and a lot of other standard Unix commands. Now that we have switched to Linux we don't have the issue anymore.
Re: How does Rebindable suppress the compiler's optimizations for immutable?
On Friday, February 15, 2019 3:06:34 AM MST Kagamin via Digitalmars-d-learn wrote: > Union is just a pretty cast, type system guarantees don't hold > for it. Well, technically, what's supposed to be the case is that when you cast, the type system guarantees still hold but it's up to the programmer to make sure that they do - just like how it's up to the programmer to ensure that @trusted code is really @safe. Rebindable is a pretty weird case with what it's doing, but I'm pretty sure that it's actually violating the type system with what it does in that you can't rely on the value of the immutable reference itself not being mutated even though it's immutable. The type system guarantees for what the reference refers to are maintained but not the guarantees for the actual reference. In practice, I don't think that it's actually a problem, but in principle, it's violating the type system, and a particularly aggressive optimizer could make a bad assumption. However, given that a union is involved, and as such the "cast" is built into the type, I question that any optimizer would ever make the wrong assumption about Rebindable. - Jonathan M Davis
Re: How does Rebindable suppress the compiler's optimizations for immutable?
Union is just a pretty cast, type system guarantees don't hold for it.
Re: How does Rebindable suppress the compiler's optimizations for immutable?
On Thursday, February 14, 2019 11:59:31 PM MST Stefan Koch via Digitalmars-
d-learn wrote:
> On Thursday, 14 February 2019 at 23:55:18 UTC, SimonN wrote:
> > std.typecons.Rebindable!(immutable A) is implemented as:
> > private union {
> >
> > immutable(A) original;
> > A stripped;
> >
> > }
> >
> > ...@trusted assignment operators...
> >
> > @property inout(immutable(A)) get() @trusted pure nothrow
> >
> > @nogc inout
> >
> > {
> >
> > return original;
> >
> > }
> >
> > alias get this;
> >
> > This conforms with the D safety spec: All access to the unsafe
> > union goes through the @trusted get() and the trusted
> > assignment operators.
> >
> > Rebindable!(immutable A) r = a1;
> > // r.original is a1
> > r = a2;
> > // r.original is a2
> >
> > But the compiler may assume that immutable variables -- such as
> > the immutable(A) original -- never change and thus may optimize
> > code. Since immutable(A) original is assignable in the union,
> > such optimization would produce wrong behavior: In the final
> > line, the compiler could think that r.original is a1 without
> > examining r.original.
> >
> > How does Rebindable prevent the compiler from optimizing
> > according to immutable's rules?
>
> It's easy. You cannot use immutable as the only basis of
> optimization.
> You need to proof actual immutability via data-flow-analysis over
> the whole life-time of your immutable.
> When you cannot guarantee actual immutability (within the frame
> of interest) don't perform optimizations which are dependent on
> it.
>
> So much for the theory, in practice I think that most
> optimizations based on immutable are disabled.
> Think of immutable as hint for the programmer, not for the
> compiler.
The type system is supposed to guarantee and be able to rely on immutable
never mutating. That's pretty much the whole point. Honestly, I'm pretty
sure that Rebindable technically violates the type system to do what it
does. However, I would expect that the use of a union would tend to kill
compiler optimizations given how tricky things get with those, so I wouldn't
expect that to be a problem. But yeah, I suspect that the compiler doesn't
do a lot of optimizations based on immutable anyway. I don't know for sure,
but it usually seems to be the case that the optimizations that D could get
from its type system end up being theoretical rather than actual whenever
anyone looks into it.
- Jonathan M Davis
