Re: How to pack a struct to a ubyte[] in a more "The D way" style ?

2017-12-21 Thread Binghoo Dang via Digitalmars-d-learn

hi Davis,

Thanks for your great help to me!

Yeah, the library may had a design principle when it was 
designed, as you can see the buffer appender is not that suitable 
for an application-defined structured data packing.


And after I turn to the bitfield, I then got another trouble:

The bitfield template only allows (8, 64) bits to be packed, so 
in a real application, and obviously, in my application, the 
packet is greater than 8 bytes.


So, the appender and bitfield are not practical for my situation. 
And now, I realized that the problem what I have done is only 
using bit-shift other than using direct cast operator.


In C, one can directly cast a struct to memory bytes, for D, the 
struct can have methods, and the alignment is also more complex. 
So, I think that handy packing and unpacking elements in a struct 
to D array is just OK. And better than C, the D array is 
autoincremented, this feature already makes the thing more 
simpler to understand.


So, before I can write my own template for generating the packing 
and unpacking code, I would just remove the bit shit.


Again, Thank you very much!


On Thursday, 21 December 2017 at 12:21:55 UTC, Jonathan M Davis 
wrote:
On Monday, December 18, 2017 08:45:32 Binghoo Dang via 
Digitalmars-d-learn wrote:

hi Davis,

I read the std.bitmanip, and I tried to test the code like 
below:


```
import std.stdio;
import std.array;
import std.bitmanip;
void main(string[] args)
{
 align(1) struct c {
 ushort c1;
 uint c2;
 //ubyte[8] c3;
 }
 ubyte[] buffer;
 auto ap = appender();
 uint a = 0x11223344;
 ushort b = 0x6677;
 ap.append!uint(a);
 ap.append!ushort(b);

 c cobj;
 cobj.c1 = 0xEEFF;
 cobj.c2 = 0xDEADBEAF;
 //cobj.c3 = [0x12, 0x34, 0x56, 0x78];
 ap.append(cobj);

 ubyte[3] d = [0xAA, 0xBB, 0xCC];
 ap.append(d);

 foreach(e; buffer) {
 writef("%02X ", e);
 }
}
```
For compiling this code, I got error like this

> onlineapp.d(22): Error: template std.bitmanip.append cannot
> deduce function from >argument types 
> !()(RefAppender!(ubyte[]),

>
> c), candidates are:
>/dlang/dmd/linux/bin64/../../src/phobos/std/bitmanip.d(3623):
>std.bitmanip.append(T, Endian endianness = 
> Endian.bigEndian,

>
>R)(R range, T value) >if (canSwapEndianness!T &&
>isOutputRange!(R, ubyte))
>onlineapp.d(25): Error: template std.bitmanip.append cannot
>deduce function from >argument types 
>!()(RefAppender!(ubyte[]),

>ubyte[3]), candidates are:
>
>/dlang/dmd/linux/bin64/../../src/phobos/std/bitmanip.d(3623):
>std.bitmanip.append(T, Endian endianness = 
> Endian.bigEndian,

>
>R)(R range, T value) >if (canSwapEndianness!T &&
>isOutputRange!(R, ubyte))

It seems that the appending is just allow using the 
fundamental types like uint, ushort, it does not support 
struct, and can't support dynamic array too. As the hardware 
protocol is a dynamic-length style, so I also need to support 
append array or another buffer directly.


It seems that I can't get the point how to using the bitmanip 
to do the packing I need, and also I have no idea what the 
unpacking will look like.


write, append, peek, and read all operate on integral types 
only. If you have an array of int or whatnot that you want to 
put into an array of ubyte, then just use a loop, and if you 
have a struct, then write or append each of the members 
individually. Those functions are for making it clean and easy 
to write integral types to an array or range of ubyte (or to 
read them back out again) while properly taking endianness into 
account, not for directly serializing objects.


I really don't know what to tell you if the examples in the 
docs aren't enough to figure out how to use the functions, 
because if I were trying to show you how, I'd give very similar 
examples, and I don't know why the docs wouldn't be clear as 
they are or how they could be made clearer for you.


- Jonathan M Davis





Re: How to pack a struct to a ubyte[] in a more "The D way" style ?

2017-12-21 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, December 18, 2017 08:45:32 Binghoo Dang via Digitalmars-d-learn 
wrote:
> hi Davis,
>
> I read the std.bitmanip, and I tried to test the code like below:
>
> ```
> import std.stdio;
> import std.array;
> import std.bitmanip;
> void main(string[] args)
> {
>  align(1) struct c {
>  ushort c1;
>  uint c2;
>  //ubyte[8] c3;
>  }
>  ubyte[] buffer;
>  auto ap = appender();
>  uint a = 0x11223344;
>  ushort b = 0x6677;
>  ap.append!uint(a);
>  ap.append!ushort(b);
>
>  c cobj;
>  cobj.c1 = 0xEEFF;
>  cobj.c2 = 0xDEADBEAF;
>  //cobj.c3 = [0x12, 0x34, 0x56, 0x78];
>  ap.append(cobj);
>
>  ubyte[3] d = [0xAA, 0xBB, 0xCC];
>  ap.append(d);
>
>  foreach(e; buffer) {
>  writef("%02X ", e);
>  }
> }
> ```
> For compiling this code, I got error like this
>
> > onlineapp.d(22): Error: template std.bitmanip.append cannot
> > deduce function from >argument types !()(RefAppender!(ubyte[]),
> >
> > c), candidates are:
> >/dlang/dmd/linux/bin64/../../src/phobos/std/bitmanip.d(3623):
> >std.bitmanip.append(T, Endian endianness = Endian.bigEndian,
> >
> >R)(R range, T value) >if (canSwapEndianness!T &&
> >isOutputRange!(R, ubyte))
> >onlineapp.d(25): Error: template std.bitmanip.append cannot
> >deduce function from >argument types !()(RefAppender!(ubyte[]),
> >ubyte[3]), candidates are:
> >
> >/dlang/dmd/linux/bin64/../../src/phobos/std/bitmanip.d(3623):
> >std.bitmanip.append(T, Endian endianness = Endian.bigEndian,
> >
> >R)(R range, T value) >if (canSwapEndianness!T &&
> >isOutputRange!(R, ubyte))
>
> It seems that the appending is just allow using the fundamental
> types like uint, ushort, it does not support struct, and can't
> support dynamic array too. As the hardware protocol is a
> dynamic-length style, so I also need to support append array or
> another buffer directly.
>
> It seems that I can't get the point how to using the bitmanip to
> do the packing I need, and also I have no idea what the unpacking
> will look like.

write, append, peek, and read all operate on integral types only. If you
have an array of int or whatnot that you want to put into an array of ubyte,
then just use a loop, and if you have a struct, then write or append each of
the members individually. Those functions are for making it clean and easy
to write integral types to an array or range of ubyte (or to read them back
out again) while properly taking endianness into account, not for directly
serializing objects.

I really don't know what to tell you if the examples in the docs aren't
enough to figure out how to use the functions, because if I were trying to
show you how, I'd give very similar examples, and I don't know why the docs
wouldn't be clear as they are or how they could be made clearer for you.

- Jonathan M Davis



Re: How to pack a struct to a ubyte[] in a more "The D way" style ?

2017-12-21 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, December 18, 2017 05:32:02 Binghoo Dang via Digitalmars-d-learn 
wrote:
> Thanks for your idea.
>
> On Monday, 18 December 2017 at 05:08:24 UTC, Jonathan M Davis
>
> wrote:
> >>  ubyte[] makeCMDPacket(RCPCmd cmd, in ubyte[] data)
> >>  {
> >>
> >>  this.preamble = RCPPKT_PRE;
> >>  this.hdr.msgtype = RCPMSG_CMD;
> >>  this.hdr.cmdcode =  cast (ubyte) cmd;
> >>  this.hdr.len = 0x & data.length;
> >
> > Why are you using & instead of simply casting to ushort?
> > Casting would be more idiomatic. Or you can use to!ushort if
> > you want to verify the size at runtime and have a ConvException
> > thrown if the value is too large.
>
> I'm using & because I'm using a C-Style. and the length needs to
> be packed into 2bytes, for casting, I don't know what will happen
> if data.length is greater. In C & C++, this kind of the cast will
> cause errors. I don't know what D will do for this.

This assertion never fails:

void main()
{
foreach(i; 0 .. uint.max)
assert((i & 0x) == cast(ushort)i);
}

- Jonathan M Davis



Re: How to pack a struct to a ubyte[] in a more "The D way" style ?

2017-12-18 Thread Binghoo Dang via Digitalmars-d-learn

hi Davis,

I read the std.bitmanip, and I tried to test the code like below:

```
import std.stdio;
import std.array;
import std.bitmanip;
void main(string[] args)
{
align(1) struct c {
ushort c1;
uint c2;
//ubyte[8] c3;
}
ubyte[] buffer;
auto ap = appender();
uint a = 0x11223344;
ushort b = 0x6677;
ap.append!uint(a);
ap.append!ushort(b);

c cobj;
cobj.c1 = 0xEEFF;
cobj.c2 = 0xDEADBEAF;
//cobj.c3 = [0x12, 0x34, 0x56, 0x78];
ap.append(cobj);

ubyte[3] d = [0xAA, 0xBB, 0xCC];
ap.append(d);

foreach(e; buffer) {
writef("%02X ", e);
}
}
```
For compiling this code, I got error like this

onlineapp.d(22): Error: template std.bitmanip.append cannot 
deduce function from >argument types !()(RefAppender!(ubyte[]), 
c), candidates are:
/dlang/dmd/linux/bin64/../../src/phobos/std/bitmanip.d(3623):
   std.bitmanip.append(T, Endian endianness = Endian.bigEndian, 
R)(R range, T value) >if (canSwapEndianness!T && 
isOutputRange!(R, ubyte))
onlineapp.d(25): Error: template std.bitmanip.append cannot 
deduce function from >argument types !()(RefAppender!(ubyte[]), 
ubyte[3]), candidates are:
/dlang/dmd/linux/bin64/../../src/phobos/std/bitmanip.d(3623):
   std.bitmanip.append(T, Endian endianness = Endian.bigEndian, 
R)(R range, T value) >if (canSwapEndianness!T && 
isOutputRange!(R, ubyte))


It seems that the appending is just allow using the fundamental 
types like uint, ushort, it does not support struct, and can't 
support dynamic array too. As the hardware protocol is a 
dynamic-length style, so I also need to support append array or 
another buffer directly.


It seems that I can't get the point how to using the bitmanip to 
do the packing I need, and also I have no idea what the unpacking 
will look like.


Thanks!


Re: How to pack a struct to a ubyte[] in a more "The D way" style ?

2017-12-17 Thread Binghoo Dang via Digitalmars-d-learn

Thanks for your idea.

On Monday, 18 December 2017 at 05:08:24 UTC, Jonathan M Davis 
wrote:

 ubyte[] makeCMDPacket(RCPCmd cmd, in ubyte[] data)
 {
 this.preamble = RCPPKT_PRE;
 this.hdr.msgtype = RCPMSG_CMD;
 this.hdr.cmdcode =  cast (ubyte) cmd;
 this.hdr.len = 0x & data.length;


Why are you using & instead of simply casting to ushort? 
Casting would be more idiomatic. Or you can use to!ushort if 
you want to verify the size at runtime and have a ConvException 
thrown if the value is too large.




I'm using & because I'm using a C-Style. and the length needs to 
be packed into 2bytes, for casting, I don't know what will happen 
if data.length is greater. In C & C++, this kind of the cast will 
cause errors. I don't know what D will do for this.



```

It's basically a C-style code, and it works perfect, But I 
know that this is not a "D-way", So, any suggestions?


I'd suggest that you check out std.bitmanip if you haven't.

https://dlang.org/phobos/std_bitmanip.html

append, peek, read, and write in particular are useful if 
you're putting various integral values into an array of ubyte[] 
or reading them from it. Glancing over what you have, I'm 
pretty sure that all of the bitshifts and bitwise &'s and can 
be removed by using append or write to write the data to the 
array and peek or read to read from it.


Also, as far as D style goes, variables and enum values are 
usually given camelCase names, whereas you've named your enum 
values in all uppercase and your variables names in all 
lowercase, and you've underscores in the middle of names. But 
of course, you can do what you want on that. It's just that if 
you make any libraries public (e.g. on code.dlang.org), it's 
better to follow the D naming guidelines for anything in the 
public API:


https://dlang.org/dstyle.html

- Jonathan M Davis


Yeah, thanks for the suggestion, I will check bit manipulation 
more in detail. I just heard of that, but I just did a 
misunderstanding that I think that is for packing and unpacking 
of bits.


Re: How to pack a struct to a ubyte[] in a more "The D way" style ?

2017-12-17 Thread Jonathan M Davis via Digitalmars-d-learn
On Monday, December 18, 2017 03:00:47 Binghoo Dang via Digitalmars-d-learn 
wrote:
> hi,
>
> I'm a dlang newbie, and I recently made a program that interacts
> with an RFID Reader, it's basically a Serial-Port communication
> model. And after 2 days, I finally got a working code like these:
>
> ```
>  enum RCPCmd{
>  RCPCMD_GET_REGION = 0x06,
>  RCPCMD_SET_REGION = 0x07,
>  RCPCMD_SYS_RESET = 0x08,
>  ...
>
>  RCPCMD_START_AUTOREAD2 = 0x36,
>  RCPCMD_STOP_AUTOREAD2 = 0x37,
>
>  RCPCMD_WRITE_TYPEC_TAGDATA = 0x46,
>
>  RCPCMD_FAIL = 0xFF
>  }
>
>  enum {
>  RCPMSG_CMD = 0x00,
>  RCPMSG_RESP = 0x01,
>  RCPMSG_NOTIFY = 0x02
>  }
>
>  private struct RCProtocol
>  {
>  enum {
>  RCPPKT_PRE = 0xBB,
>  RCPPKT_END = 0x7E
>  }
>
>  align(1) struct RCPacketHdr {
>  ubyte msgtype;
>  ubyte cmdcode;
>  ushort len;
>  }
>
>  /**
>   * These kind of members will be need only for private
> var for making a cmd packet.
>   * But access-able as a parsed result for a incomming
> packet.
>   */
>  ubyte preamble;
>  RCPacketHdr hdr;
>  ubyte[] payload; //ubyte[len] of payload.
>  ubyte end;
>  ushort crc16;
>
>  ubyte[] makeCMDPacket(RCPCmd cmd, in ubyte[] data)
>  {
>  this.preamble = RCPPKT_PRE;
>  this.hdr.msgtype = RCPMSG_CMD;
>  this.hdr.cmdcode =  cast (ubyte) cmd;
>  this.hdr.len = 0x & data.length;

Why are you using & instead of simply casting to ushort? Casting would be
more idiomatic. Or you can use to!ushort if you want to verify the size at
runtime and have a ConvException thrown if the value is too large.

>  this.end = RCPPKT_END;
>  this.payload = data.dup;
>
>  ubyte [] pkt;
>  pkt ~= this.hdr.msgtype;
>  pkt ~= this.hdr.cmdcode;
>  pkt ~= this.hdr.len >> 8;
>  pkt ~= this.hdr.len & 0xff;
>  pkt = this.preamble ~ pkt ~ this.payload ~ this.end;
>
>  // calc crc16 for Preamble + HDR + Data + END
>  this.crc16 = new RCPCRC16().calc(pkt);
>
>  pkt ~= this.crc16 >> 8;
>  pkt ~= this.crc16 & 0xff;
>
>  return pkt;
>  }
>
>  int parseRespPacket(in ubyte[] recv)
>  {
>  const size_t len = recv.length;
>  if (recv is null)
>  return PARSE_ERR;
>
>  if (len < 8)
>  return PARSE_ERR_PKTLEN;
>
>  if (recv[0] !is RCPPKT_PRE)
>  return PARSE_ERR_PKTHEAD;
>
>  this.preamble = recv[0];
>  this.hdr.msgtype = recv[1];
>  this.hdr.cmdcode = recv[2];
>  this.hdr.len = (recv[3] << 8) + 0xff & recv[4];
>
>  if ( this.hdr.len > (len - 8))
>  return PARSE_ERR_PKT;
>
>  this.end = recv[5+this.hdr.len];
>
>  if (this.end != RCPPKT_END)
>  return PARSE_ERR_PKT;
>
>  this.payload = recv[5 .. (5+this.hdr.len)].dup;
>
>  ubyte [] pkt;
>  pkt ~= this.hdr.msgtype;
>  pkt ~= this.hdr.cmdcode;
>  pkt ~= this.hdr.len >> 8;
>  pkt ~= this.hdr.len & 0xff;
>  pkt = this.preamble ~ pkt ~ this.payload ~ this.end;
>
>  // calc crc16 for Preamble + HDR + Data + END
>  const ushort desired_crc = new RCPCRC16().calc(pkt);
>
>  this.crc16 = (recv[6+this.hdr.len] << 8) +
> recv[7+this.hdr.len];
>  if (this.crc16 != desired_crc) {
>  writefln("-- PARSE ERR: CRC, desired = %04X,
> actuall is %04X", this.crc16, desired_crc);
>  return PARSE_ERR_CRC;
>  }
>
>  return PARSE_OK;
>  }
>  }
>
> ```
>
> It's basically a C-style code, and it works perfect, But I know
> that this is not a "D-way", So, any suggestions?

I'd suggest that you check out std.bitmanip if you haven't.

https://dlang.org/phobos/std_bitmanip.html

append, peek, read, and write in particular are useful if you're putting
various integral values into an array of ubyte[] or reading them from it.
Glancing over what you have, I'm pretty sure that all of the bitshifts and
bitwise &'s and can be removed by using append or write to write the data to
the array and peek or read to read from it.

Also, as far as D style goes, variables and enum values are usually given
camelCase names, whereas you've named your enum values in all uppercase and
your variables names in all lowercase, and you've underscores in the middle
of names. But of course, you can do what you want on that. It's just that if
you make any libraries public (e.g. on code.dlang.org), it's better to
follow the D 

How to pack a struct to a ubyte[] in a more "The D way" style ?

2017-12-17 Thread Binghoo Dang via Digitalmars-d-learn

hi,

I'm a dlang newbie, and I recently made a program that interacts 
with an RFID Reader, it's basically a Serial-Port communication 
model. And after 2 days, I finally got a working code like these:


```
enum RCPCmd{
RCPCMD_GET_REGION = 0x06,
RCPCMD_SET_REGION = 0x07,
RCPCMD_SYS_RESET = 0x08,
...

RCPCMD_START_AUTOREAD2 = 0x36,
RCPCMD_STOP_AUTOREAD2 = 0x37,

RCPCMD_WRITE_TYPEC_TAGDATA = 0x46,

RCPCMD_FAIL = 0xFF
}

enum {
RCPMSG_CMD = 0x00,
RCPMSG_RESP = 0x01,
RCPMSG_NOTIFY = 0x02
}

private struct RCProtocol
{
enum {
RCPPKT_PRE = 0xBB,
RCPPKT_END = 0x7E
}

align(1) struct RCPacketHdr {
ubyte msgtype;
ubyte cmdcode;
ushort len;
}

/**
 * These kind of members will be need only for private 
var for making a cmd packet.
 * But access-able as a parsed result for a incomming 
packet.

 */
ubyte preamble;
RCPacketHdr hdr;
ubyte[] payload; //ubyte[len] of payload.
ubyte end;
ushort crc16;

ubyte[] makeCMDPacket(RCPCmd cmd, in ubyte[] data)
{
this.preamble = RCPPKT_PRE;
this.hdr.msgtype = RCPMSG_CMD;
this.hdr.cmdcode =  cast (ubyte) cmd;
this.hdr.len = 0x & data.length;
this.end = RCPPKT_END;
this.payload = data.dup;

ubyte [] pkt;
pkt ~= this.hdr.msgtype;
pkt ~= this.hdr.cmdcode;
pkt ~= this.hdr.len >> 8;
pkt ~= this.hdr.len & 0xff;
pkt = this.preamble ~ pkt ~ this.payload ~ this.end;

// calc crc16 for Preamble + HDR + Data + END
this.crc16 = new RCPCRC16().calc(pkt);

pkt ~= this.crc16 >> 8;
pkt ~= this.crc16 & 0xff;

return pkt;
}

int parseRespPacket(in ubyte[] recv)
{
const size_t len = recv.length;
if (recv is null)
return PARSE_ERR;

if (len < 8)
return PARSE_ERR_PKTLEN;

if (recv[0] !is RCPPKT_PRE)
return PARSE_ERR_PKTHEAD;

this.preamble = recv[0];
this.hdr.msgtype = recv[1];
this.hdr.cmdcode = recv[2];
this.hdr.len = (recv[3] << 8) + 0xff & recv[4];

if ( this.hdr.len > (len - 8))
return PARSE_ERR_PKT;

this.end = recv[5+this.hdr.len];

if (this.end != RCPPKT_END)
return PARSE_ERR_PKT;

this.payload = recv[5 .. (5+this.hdr.len)].dup;

ubyte [] pkt;
pkt ~= this.hdr.msgtype;
pkt ~= this.hdr.cmdcode;
pkt ~= this.hdr.len >> 8;
pkt ~= this.hdr.len & 0xff;
pkt = this.preamble ~ pkt ~ this.payload ~ this.end;

// calc crc16 for Preamble + HDR + Data + END
const ushort desired_crc = new RCPCRC16().calc(pkt);

this.crc16 = (recv[6+this.hdr.len] << 8) + 
recv[7+this.hdr.len];

if (this.crc16 != desired_crc) {
writefln("-- PARSE ERR: CRC, desired = %04X, 
actuall is %04X", this.crc16, desired_crc);

return PARSE_ERR_CRC;
}

return PARSE_OK;
}
}

```

It's basically a C-style code, and it works perfect, But I know 
that this is not a "D-way", So, any suggestions?


Thanks!