Executive level objectives: * Find and fix known performance bottlenecks; * Remove barriers to uptake by developer community.
Metaphorically, this is a spinal resection: Surgeons will tell you this is one of the more intensive surgical interventions that can be performed on a human being, because of all the inter-system coupling involved. The last round of metrics from Ohloh.net, indicate we're around the 400 KLOCs of C++ mark for XORP. This includes libxipc, which is around 40 KLOCs. This is sizeable by any project's standards. It's obvious that the solution SHOULD NOT require a rewrite of existing process code. This rules out Boost.ASIO right off the bat, and forces the scope to get constrained to just implementing Thrift. Before this project fully commenced, there was research into the available alternatives. There aren't many; a list can be produced, if that's something people care to see. Where Thrift stands out above all the others, is in cross-language interop. You can use it from just about anything (except C, although that's being worked on). As time goes on, this is probably going to become more important, because of who's using it, and why: Spotify, Last.FM, Facebook and others. If you look at http://thriftpuzzle.facebook.com/ -- Thrift is cunningly used as a Facebook recruitment tool. In this project, we're using it to streamline the RPC marshaling in a (possibly distributed) router. There is plenty of common ground here, because of overlapping requirements for low latency and compact representation. The Thrift developers make it pretty clear that they expect Thrift to run on a reliable, ordered, stream protocol (e.g. UNIX file handles, pipes, HTTP sessions, TCP sessions, UNIX domain stream sockets) -- for now. Of course the price you pay for using a stream to deliver messages, is head-of-line blocking. This dependency comes about largely because Thrift was developed for web services, although because Thrift is itself message-oriented, there's no reason why a message-oriented transport cannot be used for the RPC calls in future. Of course, the trade-off with cross-language interop, is that not all languages/frameworks implement asynchronous dispatch, or do so in the same way. It is the proverbial 'garden rake on the ground' -- as we've seen with the Net-SNMP code, if you keep stepping on it, it will hit you in the face time and time again. The above named startups all do slightly different things to implement the server side scalability in their service offerings, which is where the value is. So, Thrift has left the async programming model as an unanswered question, to date. Although what's in Facebook's tree, is in Facebook's tree, and not something we get to see, just yet. Esteve Fernandez (who I had the pleasure of meeting at LShift, Ltd. over the summer) has done some excellent work on getting Thrift to work on top of AMQP in Twisted Python, with asynchronous dispatch. It's likely that we can borrow from this conceptually for AMQP to be implemented in future, but that's far off at the moment. It's an open question. thanks, BMS _______________________________________________ Xorp-hackers mailing list [email protected] http://mailman.ICSI.Berkeley.EDU/mailman/listinfo/xorp-hackers
