Heterogeneous Variadic Arguments with Other Arguments
I wanted to do this:
package int serialize(byte[] destination, int offset, T...)(T
values)
{
static foreach (i, value; values)
{
*cast(T[i] *)(destination.ptr + offset) = value;
offset += typeid(T[i]).tsize();
}
return offset;
}
But I got this error:
main.d(8): Error: template main.serialize cannot deduce function
from argument types !()(byte[], int, byte, int), candidates are:
main.d(20):main.serialize(byte[] destination, int offset,
T...)(T values)
Eventually, I assumed that you couldn't do heterogeneous
arguments with other arguments first.
I was able to do this and it works:
package int serialize(T...)(T values)
{
static assert(values.length >= 2);
auto ptr = (cast(byte[])values[0]).ptr;
int offset = cast(int)values[1];
static foreach (i, value; values)
{
static if (i > 1)
{
*cast(T[i] *)(ptr + offset) = value;
offset += typeid(T[i]).tsize();
}
}
return offset;
}
That is obviously a lot uglier. Can you have a function with
non-variadic arguments followed by heterogeneous variadic
arguments?
Re: Binary Serialization
On Sunday, 14 July 2019 at 00:18:02 UTC, Adam D. Ruppe wrote:
On Saturday, 13 July 2019 at 23:52:38 UTC, harakim wrote:
class MoveCommand
{
byte serialNumber;
int x;
int y;
}
When I do MoveCommand.sizeof, it returns 4.
It is important to understand a class in D is a reference type,
so `MoveCommand` here is actually a pointer internally, meaning
that 4 is the size of a pointer. Trying to directly read or
write that to a socket is a mistake.
With struct though, there's potential. The reason you get 12
there though is that the byte is padded. The struct looks like
0: serialNumber
1: padding
2: padding
3: padding
4: x
5: x
6: x
7: x
8: y
9: y
10: y
11: y
To get move the padding to the end, you can add `align(1):`
inside, so given:
struct MoveCommand
{
align(1):
byte serialNumber;
int x;
int y;
}
The layout will look like this:
0: serialNumber
1: x
2: x
3: x
4: x
5: y
6: y
7: y
8: y
9: padding
10: padding
11: padding
The align(1) is kinda like the __packed__ thing in C compilers.
The size of will still read 12 there, but you can read and
write an individual item direct off a binary thing reliably.
But an array of them will have that padding still. To get rid
of that, you put an align(1) on the *outside* of the struct:
align(1) // this one added
struct MoveCommand
{
align(1): // in ADDITION to this one
byte serialNumber;
int x;
int y;
}
And now the sizeof will read 9, with the padding cut off fromt
the end too. You can do an array of these now totally packed.
This behavior is consistent across D compilers; it is defined
by the spec.
Just remember types like `string` have a pointer embedded and
probably shouldn't be memcpyed!
Awesome. I kind of figured the D language would put this type of
thing in the spec. Since the layout in memory will be consistent,
I think I'm just going to let D do its normal thing and keep the
members aligned in the default manner at a cost of a few extra
bytes.
It's nice to know about align, though. In the unlikely event I
end up running into this a lot, I will revisit that decision. I
can just order my members for efficient space usage for now. As a
side note, it would be cool if there were blank pages in the end
of the The D Programming Language book to put addendums like this
in there. I'll have to keep that in mind if I ever write a book.
Re: Binary Serialization
On Saturday, 13 July 2019 at 23:52:38 UTC, harakim wrote:
class MoveCommand
{
byte serialNumber;
int x;
int y;
}
When I do MoveCommand.sizeof, it returns 4.
It is important to understand a class in D is a reference type,
so `MoveCommand` here is actually a pointer internally, meaning
that 4 is the size of a pointer. Trying to directly read or write
that to a socket is a mistake.
With struct though, there's potential. The reason you get 12
there though is that the byte is padded. The struct looks like
0: serialNumber
1: padding
2: padding
3: padding
4: x
5: x
6: x
7: x
8: y
9: y
10: y
11: y
To get move the padding to the end, you can add `align(1):`
inside, so given:
struct MoveCommand
{
align(1):
byte serialNumber;
int x;
int y;
}
The layout will look like this:
0: serialNumber
1: x
2: x
3: x
4: x
5: y
6: y
7: y
8: y
9: padding
10: padding
11: padding
The align(1) is kinda like the __packed__ thing in C compilers.
The size of will still read 12 there, but you can read and write
an individual item direct off a binary thing reliably. But an
array of them will have that padding still. To get rid of that,
you put an align(1) on the *outside* of the struct:
align(1) // this one added
struct MoveCommand
{
align(1): // in ADDITION to this one
byte serialNumber;
int x;
int y;
}
And now the sizeof will read 9, with the padding cut off fromt
the end too. You can do an array of these now totally packed.
This behavior is consistent across D compilers; it is defined by
the spec.
Just remember types like `string` have a pointer embedded and
probably shouldn't be memcpyed!
Binary Serialization
I am receiving packets of data over the network from a
C#/Java/dlang/etc. client.
I'm writing the data directly to a socket, one primitive value at
a time in little endian format. I would like to receive this
super easily in d. Here is an example of roughly what I want to
do.
class MoveCommand
{
byte serialNumber;
int x;
int y;
}
public void processMoveCommand(byte[] buffer, offset)
{
MoveCommand command = *(cast(MoveCommand *)(buffer.ptr +
offset));
}
When I do MoveCommand.sizeof, it returns 4. When I changed
MoveCommand to a struct, it returns 12. Is that going to be
reliable on every machine and with every compiler? It seems like
I had to use __attributed_packed__ (something like this, it's
been 10 years) to guarantee the actual byte layout of a struct in
C when working with files. Do I have to even worry about this in
D? If so, is there something comparable to attribute packed in D?
The alternative that I'm fearing would be that some compiler
would choose word-based boundaries, it would pack that data or
something else (I would hope not these). I would rather know how
this works now and not have it come back to bite me when I least
suspect it on some other machine.
I am happy to read about this stuff, but I don't know where to
start. I have the book "The D Programming Language" if there is
something in there that I missed.
Re: q about slices, strings
On 13.07.19 22:35, Kevin Bailey wrote: int[char[2]] aa; auto arr = "ABBBA"; foreach (i; 0 .. arr.length - 1) aa[arr[i .. $][0 .. 2]]++; which I changed to: aa[arr[i .. i+2][0 .. 2]]++; in the hopes of only doing: aa[arr[i .. i+2]]++; The question is: Why is the [0..2] still necessary? [...] So why isn't arr[i .. i+2] convertible? Is dmd behind the spec here? Looks like it. It works in the most basic context: char[2] sa = arr[i .. i + 2]; And it works in the AA context when you use the most basic kind of expression: aa[arr[0 .. 2]]++; So as far as I can tell, it's just a bug that more complex expressions fail in more complex contexts. It should work the same.
q about slices, strings
I saw on the dlang.org homepage a sample of code that looks a little strange to me: int[char[2]] aa; auto arr = "ABBBA"; foreach (i; 0 .. arr.length - 1) aa[arr[i .. $][0 .. 2]]++; which I changed to: aa[arr[i .. i+2][0 .. 2]]++; in the hopes of only doing: aa[arr[i .. i+2]]++; The question is: Why is the [0..2] still necessary? Without it, dmd (on that homepage) complains: cannot implicitly convert expression arr[i..i + 2LU] of type string to char[2] The spec says that: 10.22 Slices ... 6. If the slice bounds can be known at compile time, the slice expression is implicitly convertible to an lvalue of static array. 7. The following forms of slice expression can be convertible to a static array type: e An expression that contains no side effects. a, b Integers (that may be constant-folded). Form The length calculated at compile time ... arr[e .. e+b] b So why isn't arr[i .. i+2] convertible? Is dmd behind the spec here?
Re: Windows segfault, need brief help
On Saturday, 13 July 2019 at 06:36:06 UTC, Boris Carvajal wrote: On Friday, 12 July 2019 at 22:46:11 UTC, Anonymouse wrote: I'm suddenly getting segfaults when running tests on Windows. It works fine on Linux. I reduced it to a few lines (plus a dependency) with dustmite, but they don't really make sense[1]. Nevertheless they do trigger the segfault. Can someone with Windows 10 and dmd 2.087.0 try the following and see if it crashes please? git clone https://github.com/zorael/tests -b wintestcrash && cd tests && dub test I can reproduce it on Win10/x64 (qemu). But it's really hard to debug on windows (at least with dmd and no V.Studio), I could only get a readable backtrace with "WinDbg Preview" debugger: Thank you! Filed as https://issues.dlang.org/show_bug.cgi?id=20048.
Re: Alias function with arguments
On Thursday, 11 July 2019 at 03:13:43 UTC, Jamie wrote:
Is it possible to alias a function and its arguments, but for
that function to only be evaluated when the alias is used? For
example
alias pragma(inline, true) inline
inline
void func(){}
What you're asking for is possible (see
https://run.dlang.io/is/2Lh4Dr for example).
However I don't believe the code sample you're showing is
possible, because aliases always evaluate to types or values and
you can't use either of those as a function attribute. Also
pragmas are not functions.
WeakRef implementation
Is this implementation of WeakRef correct? Is it portable? I
tested it and it seems to work, but I'm not real familiar with
the runtime or the GC. Also is there any way to convert this into
a struct instead of class? (Not possible this way due to frame
pointer for onDispose)
extern (C) void rt_attachDisposeEvent(Object, void
delegate(Object));
extern (C) void rt_detachDisposeEvent(Object, void
delegate(Object));
class WeakRef(T) if (is(T == class)) {
this(T element) {
address = *cast(size_t*) - offset;
rt_attachDisposeEvent(element, );
}
private extern (C) void onDispose(Object _) {
assert(_ is element);
address = 0 - offset;
}
alias element this;
inout(T) element() inout {
size_t result = address + offset;
return *cast(typeof(return)*)
}
private static immutable size_t offset = 16_785_407;
private size_t address;
}
auto weakRef(T)(T element) {
return new WeakRef!T(element);
}
Re: arsd terminal with ConsoleOutputType.cellular
Thanks for the answers Adam - I can now proceed !
Re: arsd terminal with ConsoleOutputType.cellular
On Saturday, 13 July 2019 at 13:41:41 UTC, Jani Hur wrote:
Other arsd.terminal related question. How to clear a line when
it is re-used ?
You need to print a number of spaces at the end to clear what's
after it. Maybe like a loop from string.length .. terminal.width
-1 and write(" ").
(maybe i should add a function for that)
Re: arsd terminal with ConsoleOutputType.cellular
Other arsd.terminal related question. How to clear a line when it
is re-used ? The following code snipped is expected to print:
important text !
but instead it prints:
important text !gless mambo-jambo (33, 0)
import arsd.terminal;
void main() {
auto term = Terminal(ConsoleOutputType.linear);
term.clear;
term.write("plenty of meaningless mambo-jambo");
term.writef(" (%s, %s)", term.cursorX, term.cursorY);
term.moveTo(0, 0);
term.write("important text !");
term.writeln;
}
Re: arsd terminal with ConsoleOutputType.cellular
On Saturday, 13 July 2019 at 13:30:47 UTC, Jani Hur wrote:
void main() {
auto term = Terminal(ConsoleOutputType.cellular);
term.clear;
term.writefln("(%s, %s)", term.cursorX, term.cursorY);
}
In cellular mode, it switches the terminal to an alternate screen
(like vim - notice that when you exit vim, it goes back to the
same screen view you had when you went into it, instead of being
leftover stuff from the full screen application), and terminal's
destructor switches back.
So what happened here is:
1) it switched to the alternate screen.
2) cleared and printed text to that
3) switched back to the normal screen at program exit
Meaning you don't see the output!
Simply add some kind of wait to the end of main() so the program
doesn't exit.
Like
term.getline();
right at teh end so it waits for you to press enter before
exiting and you should see it.
arsd terminal with ConsoleOutputType.cellular
What might be wrong with the following code below as it doesn't
clear the screen and print "(0, 0)" as expected:
import arsd.terminal;
void main() {
auto term = Terminal(ConsoleOutputType.cellular);
term.clear;
term.writefln("(%s, %s)", term.cursorX, term.cursorY);
}
If I change to the ConsoleOutputType.linear then the screen is
cleared and "(0, 0)" is printed as expected.
I'm running Gnome Terminal in Red Hat Linux and compiling with
DMD:
[jani@red arsd]$ echo $TERM
xterm-256color
[jani@red arsd]$ cat /etc/redhat-release
Red Hat Enterprise Linux Server release 7.6 (Maipo)
[jani@red arsd]$ dmd --version
DMD64 D Compiler v2.083.0
Copyright (C) 1999-2018 by The D Language Foundation, All Rights
Reserved written by Walter Bright
[jani@red arsd]$
Re: Windows segfault, need brief help
On Friday, 12 July 2019 at 23:05:41 UTC, Stefan Koch wrote: [1]: https://github.com/zorael/tests/blob/wintestcrash/source/app.dx maybe you haven't pushed? I can't see it. Sorry, typo in the link. https://github.com/zorael/tests/blob/wintestcrash/source/app.d
Re: Memory allocation failed in CT
On Saturday, 13 July 2019 at 06:25:37 UTC, Andrey wrote: On Tuesday, 9 July 2019 at 19:04:53 UTC, Max Haughton wrote: Is this a 64 or 32 bit compiler? Also could you post the source code if possible? You could try "--DRT-gcopt=profile:1" druntime flag to see if the compiler is running out of memory for real Thanks for help. I solved my issue by rewritting code. can u describe how u rewrote the code? so it can be solution for all of us: when u generate long string in CT for mixins and u get an exception u can do next ...
Re: Windows segfault, need brief help
On Friday, 12 July 2019 at 22:46:11 UTC, Anonymouse wrote: I'm suddenly getting segfaults when running tests on Windows. It works fine on Linux. I reduced it to a few lines (plus a dependency) with dustmite, but they don't really make sense[1]. Nevertheless they do trigger the segfault. Can someone with Windows 10 and dmd 2.087.0 try the following and see if it crashes please? git clone https://github.com/zorael/tests -b wintestcrash && cd tests && dub test I can reproduce it on Win10/x64 (qemu). But it's really hard to debug on windows (at least with dmd and no V.Studio), I could only get a readable backtrace with "WinDbg Preview" debugger: [0x0] msvcr100!_output_l + 0x41e [0x1] msvcr100!printf + 0x7c [0x2] test!int core.runtime.runModuleUnitTests().__foreachbody1(object.ModuleInfo*) + 0xd1 [0x3] test!int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)).__lambda2(immutable(object.ModuleInfo*)) + 0x27 [0x4] test!int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))).__foreachbody2(ref rt.sections_win64.SectionGroup) + 0x54 [0x5] test!int rt.minfo.moduleinfos_apply(scope int delegate(immutable(object.ModuleInfo*))) + 0x1f [0x6] test!int object.ModuleInfo.opApply(scope int delegate(object.ModuleInfo*)) + 0x27 [0x7] test!runModuleUnitTests + 0xfe ... msvcr100!_output_l+0x41e: `62e0346e 443811 cmp byte ptr [rcx],r10b ds:`0010=?? Registers are: rcx = 0x0010, r10 = 0x it seems like ( str[i] == '\0') So the program try to print from an invalid pointer and the StackTrace/Throwable object is somewhat related.
Re: Memory allocation failed in CT
On Tuesday, 9 July 2019 at 19:04:53 UTC, Max Haughton wrote: Is this a 64 or 32 bit compiler? Also could you post the source code if possible? You could try "--DRT-gcopt=profile:1" druntime flag to see if the compiler is running out of memory for real Thanks for help. I solved my issue by rewritting code.
