> Sorry about not replying sooner, but we have chatted about it on IRC > which has made me slack.
That's cool. > > This affects no other type in tp03, so existing code can continue to be > > used for everything _but_ strings. Newly written code can take > > advantage of the stronger guarantees. > > tp03 is "frozen", so it won't be added to tp03. Absolutely. That would be foolish. > tp04 is suppose to be an > incremental increase over tp03 so changing a fundamental type doesn't > seem the right time. Based on the number changes and extensions for tp04, it doesn't seem very incremental. One of the things is protocol specification in XML, so generation of code using an IDL or similar system is quite feasible... This seems to break the idea of a minor incremental increase. >From a little website I found called blog.mithis.net: Changes in the protcol Full XML protocol specification Meta Protocol definition Filter Support Difference Support Dynamic Objects Old Data support Multiple Instruction queue support Media support Research support EOT Support Frame Type Versions Now quite a few of these are going to involve updates to server and client code. > > Other advantages are allowing better use of data in the message - it > > can be read and directly responded too, which is particularly useful for > > smaller messages in certain cases. Essentially at the moment the entire > > message needs to be copied out piece-by-piece. This allows either block > > copying or direct access to relevant pieces. Also processing the header > > of message can now be done in place (since fragmentation won't happen > > below a 4bytes). > > I'm not sure I understand this point, care to explain further? So using the client I am most familiar with - mine - in a couple of places - eg Time remaining updates for instance. I just need to set the value in the display system. The value in the packet (if aligned) can essentially be used in place, without an issue. If not aligned, I need to copy it into a second buffer, then pass it around. The second buffer requires another allocation, and a memcpy. More subtly this increases my programs working set, which may involve extra TLB misses, cache misses, more (small) allocations increase internal heap fragmentation further degrading performance. I can build you pathological showing large > <snip> > > tp04 has support for "filter negotiation" I say that we add a new filter > type called "word aligned" filter. This would pad strings out so that > they always aligned with word boundaries. However, for this to be useful > it needs to be a required filter. Helps clients, but not the server. I'd be happy with that, but I think there is a benefit for the server as well. Hence why I'd like to see it everywhere. > Another argument is that it would decrease server load, however I think > most servers will be bound by the backend speed and not protocol > decoding speed. > > The next question is, you have already implemented the protocol and is > it going to be worth adding when the gains are lost in the noise of most > other languages (such as Python, Ruby, C# or Java). Well Java is going to give you lots of grief anyway, at least if you want to implement things correctly (the word unsigned comes to mind). Other languages I'd have to know more about to comment clearly, but I'm willing to bet such guarantees could improve there performance as well (eg Use int arrays for incoming data, not byte arrays). Anyway that argument is kinda dead - improvements are improvements. Profiling Java code used to show type conversions and copying and allocations where a major performance bottleneck. So surely makeing it easier would be a good thing? > So far the only real advantages I see are: > - easy of maintenance of the C (and possible C++) libraries > - possibly small speed increases - Reduction in allocations. - Reduction in fragmentation. - Reduction in working set. - Avoidence of emulated instructions on certain CPU families (eg byte instructions on Alpha ev4) [Very slow]. As for the penalty: In pathological case: 4000% penalty: http://www-128.ibm.com/developerworks/library/pa-dalign/ And that is letting the CPU do it itself. We are manually doing it - which is even slower (since we don't do it in microcode). Microsoft has a bit to say too: http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dv_vstechart/html/vcconwindowsdataalignmentonipfx86x86-64.asp And comp compilers has a bit to say too: http://compilers.iecc.com/comparch/article/98-02-008 > Disadvantages > - Breaks backwards compatibility with tp03/tp02 strings > - Requires extra slow level code be written for the Ruby/Python/C++ > libraries Not sure I understand this point. Extra code to be written? Yes a little if it already exists (unless you have a 'packed' required filter), or less if if it does exist. > - Extra bandwidth requirements Well I think arguments about a 0-3 byte increase in packet size are a bit stupid - especially when you are talking about sending animated gifs and other media across the network. Let me add this is also a protocol where I must send empty NULL word in certain packets for no good reason... > Why not just define a macro? Something like, Well the macros presented are C++, not C... But copying out is slower then using directly. And is still more annoying: if (ntohl(packet[pos]) == SOME_VAL) is less easy to write then: int tmp tmp = my_ntoh(packet[pos]); if (tmp == SOME_VAL).. Even if it is defined so I can do: if (myntohl(packet[pos]) == SOME_VAL) then I'm still copying the data around for no good reason... > Then you don't have memcpy's anywhere except in those functions. I'm > sure you could even reduce those to like really fast ASM code or > something ;) Which is hardly portable. If I wasn't worried about portability, I'd just write it for x86, and make sure that unaligned loads are allowed, and just use ntohl(packet[pos]) and ignore possibly unaligned loads. > This isn't a definite "NO", just what I'm pondering at 3am in the > morning. What do others think? I'd be happy with a required filter (either way), but I think the server can gain as well ;-) Regards, nash _______________________________________________ tp-devel mailing list [email protected] http://www.thousandparsec.net/tp/mailman.php/listinfo/tp-devel
