are you sick of hearing about the wire format by now? :) my latest thinking is the we will using S-expressions on the wire. S-expressions are LISP-like expressions that can be recursive and as complex (or as simple) as we need them to be.
remember my original thinking of an ascii string like cpu::user:10.0:nice:0.0:system:1.0:idle:89.0 [45 bytes] the same S-expression would look like... ---------- S expression 1 test output --------------- msg size is 61 S-expression is... (cpu ((user "10.0") (system "10.0") (nice "10.0") (idle "70.0"))) real 0m7.174s user 0m7.131s sys 0m0.014s ---------- end test output ----------- this test uses an S expression that is a little easier to read but is parsed a little slower (than the alternate expression below). keep in mind that to parse 1,000,000 messages with XDR it took xdr message size is 80 real 0m6.976s user 0m6.939s sys 0m0.014s so even the easy to read S expression is about the same speed as XDR but about 25% smaller in size. there is an alternate way of expression the same data using S-expressions that is a little more compact and faster to parse... ---------- S expression 2 test output ----------------- msg size is 57 S-expression is... (cpu ((user4:10.0) (system4:10.0) (nice4:10.0) (idle4:70.0))) real 0m5.613s user 0m5.568s sys 0m0.012s ---------- end test output ---------------------------- this is significantly smaller and faster than XDR. the message size is 57 bytes which is 29% smaller and 22% faster than XDR. i like the way S expressions represent the data and the parser http://sexpr.sourceforge.net/ is really well written and easy to incorporate into the g3 library. i spent the data running through the code with valgrind and mpatrol and it's solid. it even runs on cygwin. it has great memory management which prevents unecessary mallocs and frees. in short.. i'm sold right now. i don't want to abandon xml at all (just to be clear). i think the way we export the data to applications should be in xml (btw, it's trivial to convert from S-expressions to xml). it's also very easy to get the encapsulation and hierarchical space that i wanted before... say host A sends the message (cpu(number "2")) when host B gets the message it becomes (A(cpu(number "2"))) the sexp parser converts that string to a C struct which is passed to my tree data structure. easy. here is the S-expression test code.. in case you want to see it... #include <stdio.h> #include <string.h> #include "sexp.h" char buf[1024]; char wire[1024]; int main ( void ) { register int i; sexp_t *sx; int len; pcont_t *cc = NULL; /* We'll get the data from a socket */ strcpy( wire, "(cpu((user4:10.0)(system4:10.0)(nice4:10.0)(idle4:70.0)))"); len = strlen(wire); fprintf(stderr,"msg size is %d\n", len); for(i=0; i< 1000000; i++) { /* Convert the sexp string... */ cc = cparse_sexp(wire, 1024, cc); /* ... to a C struct */ sx = cc->last_sexp; cc->lastPos = NULL; /* Convert the C struct back to a string */ print_sexp( buf, 1024, sx ); /* The beauty is.. no real free's thanks to caching */ destroy_sexp( sx ); } sexp_cleanup(); destroy_continuation(cc); fprintf(stderr,"buffer has %s\n", buf); return 0; } pretty simple stuff. what say you? -- matt
