[Resend from correct address] On Mon, 2007-09-03 at 22:20 +0930, Tim Ansell wrote: > Hello, > > I had a chance to look over tpe_util_parse_packet in the dev branch > ( > http://git.thousandparsec.net/gitweb/gitweb.cgi?p=galaxie.git;a=shortlog;h=dev > ). I'm assuming that is what you where talking about.
Actually I was more refering to tpe_util_parse_array(). Parse_packet() is the older one which has hard coded structure formats. It's not particularly scalable. http://git.thousandparsec.net/gitweb/gitweb.cgi?p=galaxie.git;a=blob;f=tpe_util.c;h=c81d11befdc395b06a452d1c84d5e9c65aa351a6;hb=2fdbebf7acb9d2ae18b6d8289975b32c3d156b9b#l538 > I was thinking something which is closer to a direct port of the python > xstruct.py > > On a side note, shouldn't you be using specific type sizes in the file. > A int might be 64 bits on a AMD 64 machine (while int32_t will always be > 32 bits)? a) Parse_packet does. b) Probably. c) Yet to find a IP64 arch in general use, since most emt64 (aka amd64) alpha, mips and sparcs are LLP64 or occasionally LP64; it's not a major issue practically. > The various formats I want to support, from > http://git.thousandparsec.net/gitweb/gitweb.cgi?p=libtpproto-py.git;a=blob;f=tp/netlib/xstruct.py > > c Char Why is 'c' different from 'b' at a data level? From a protocol perspective they are the same. > b Int8 (8 bit integer) > B UInt8 (8 bit unsigned integer) The only byte size data I could find in the TP protocol was for the header. This needs to be handled specially anyway (the tpe_msg code handles it). Did I miss these? I originally started with these, but dropped them as I couldn't see them in use. > h Int16 (16 bit integer) > H UInt16 (16 bit unsigned integer) Are there any in TP protocol? > i Int32 (32 bit integer) > I UInt32 (32 bit unsigned integer) There is no need to differentiate between signed and unsigned at the wire level. So I don't ;-) > q Int64 (64 bit integer) > Q UInt64 (64 bit unsigned integer) Handle this: See 'l'. /me notes 'q' is a bit flaky - it's used extensively on the BSD based unices, but unfortunately the size is not always consistent. > f float (32 bit floating point number) > d double (64 bit floating point number) What format? Floats and exponents aren't always handled that cleanly. The number of bits in either aren't really portable. As the K&R enumerates: "Any of single precision floating point (float), double precision floating point (double), and extra precision floating point (long double) may be synonymous, but the ones later in the list are at least as precise as those before." So without a wire format there is not much I can do here. Heck I can't even be sure that a 64 bit float exists. And there isn't a float32_t that I am aware of. > S String Done. > [ List Start (unsigned int32 length) > ] List End The main thing this function does is the list. Needs work on lists in lists... but that is, as they say... another matter. > { List Start (unsigned int64 length) > } List End Is this used anywhere? And in what case are we going to be sending more then 4 billion items whose total capacity is less then 1 meg? > n SInt16 (16 bit semi-signed integer) > j SInt32 (32 bit semi-signed integer) > p SInt64 (64 bit semi-signed integer) > t timestamp (32 bit unsigned integer) > T timestamp (64 bit unsigned integer) All synonyms for one of short, int or long. > Some examples would be, > Board - "ISSIT" > Category - "ITSS" > Component - "IT[I]SSS[IS]" > Connect - "S" > Design - "jT[I]SSjj[II]S[IS]" > > However, lists of structures don't really map well to C as they are not > even constant size thanks to things like "[IS]". Not really - we can recurse and return a pointer to go into another struct. And a size pointer... It is the plan. > -------------- > > Maybe something like, > > /* In parse.h */ > typedef struct { > uint32_t len; > char[] s; > } string_t; > > typedef sint32_t uint32_t; > > /* Defined locally */ > struct my_list { > int32_t i; > string_t s; > }; > > void do_something() { > sint32_t arg1; > struct my_list* arg2; > > p = parse_packet(p, "j[IS]", &arg1, &arg2); The problem with this is it requires a particular struct layout and a predefined structure - this is way tpe_util_parse_packet currently does. It works, but it's pants. tpe_util_parse_array is a little more sophisticated. So it takes something like: ioff = (char *)&tut_il->i - (char *)tut_il; loff = (char *)&tut_il->l - (char *)tut_il; tut_il = tpe_util_parse_array(tutildata, tutildata + sizeof(tutildata), sizeof(struct tut_il),"[il]", ioff,loff); (I do intent to put some wrappers in to make this a little easier). This allows you to point to parts of structures and the like. > Another question is, should we return pointers to the bits in the string > (and then let the person copy/malloc the bits they need) or should you > malloc for them? Malloc for them; the string itself is in the wire format and endianess. So don't really want higher level code to need to deal with byte swapping. If they want to copy again, it's all fine. tpe_util_parse_packet currently will reallocate buffers and copy data up for things like arrays of resources, so it will manage the C memory for you. I intend to do similar for the _parse_array function. Regards, nash _______________________________________________ tp-devel mailing list [email protected] http://www.thousandparsec.net/tp/mailman.php/listinfo/tp-devel
