[this was originally part of a private E-mail exchange, which I'm
forwarding to the list with Cyrus's permission, because I thought it
might be of interest to other people who might wonder why I used Qt
in the SST prototype, or who are considering alternatives for
asynchronous network programming in C++... Apologies if this rant
diverges a bit from core P2P issues...]
Hi Cyrus,
On Jul 12, 2007, at 2:34 PM, Cyrus Hall wrote:
This library looks extremely interesting for the P2P community as a
whole, but the QT dependency may be an impediment to adoption. Since
you are already in C++, have you thought about using the boost asio
library? It was formally accepted earlier this year into boost,
and is
a pleasure to use. See http://asio.sourceforge.net/ for more
information. If you've looked at asio before, and decided to go with
the QT libs instead, I'd be interested to know why.
I haven't used of asio, and hadn't heard of it until now, but I've
used a libasync variant called yam (Yet Another Mainloop, written by
my colleague Sean Rhea), which implements a similar async programming
model in C++ using Boost's "bind" module instead of libasync's "wrap"
to provide function closures in C++. It appears to me that libasync,
yam, and boost.asio are all part of the same general family of
closure-based async toolkits for C++.
Although I didn't know about asio, I consciously decided against
either libasync for yam when I started SST for two reasons: one that
applies generally to any "async library X versus async library Y"
argument, one specific to Qt versus libasync/yam/asio.
First, the general issue: because async programming is not native to
the API of any Unix-based system (including Mac OS X), and there is
no "standard" OS-level async API that everything else can build on,
every Unix-based application that wants to do async programming has
to have a main loop, and has to decide among the many possible async
libraries that want to provide this main loop. If you want to use
two different async libraries (or two different higher-level
libraries that in turn depend on different async libraries), you
can't just link them both in and go, because each wants to provide
its own main loop. So you have to either: (a) build some kind of
custom "master main loop" that replaces each async library's main
loop, (b) pick one of the libraries' main loops to use and somehow
hack the other libraries to use the selected library's main loop
instead of their own, or (c) run each library's main loop on a
separate thread, and deal with all the inter-thread communication and
synchronization problems that creates between the different parts of
your application. As much as it personally pains me to say anything
nice about the Windows platform, this is one way in which the Windows
API beats every flavor of Unix I know of hands down: it provides a
standard basic async programming facilities that diverse libraries
can take advantage of without affecting the application's main loop.
So the upshot is that no matter what async programming framework I
were to choose to build SST on, any application developers that
prefer a _different_ async programming framework will have to work a
bit to integrate SST's chosen async programming framework with their
own. On any platform except Windows, there's just no way around
that. If I had built SST on boost.asio, then Qt/Kde developers would
no doubt make exactly the opposite complaint as you did. At least
with Qt I know there is a clean way to integrate Qt's event loop into
any other (by subclassing QAbstractEventDispatcher), and I intend to
add some functionality in the SST library to make this even easier to
do. Upon a quick perusal of boost.asio's documentation, it's not
obvious to me how you might integrate asio's "io::run()" event loop
into another main loop such as Qt's or Gtk's, although I would hope
that the asio developers have thought of this and provided some
reasonable method. In any case, given the state of the world today,
any choice of async framework will make some people happy and piss
others off, and Qt is already a number of years more well-established
than boost.asio and at least as well-established as libasync.
The second issue specifically concerns the async programming model
that Qt provides versus the libasync/yam/asio model. I've used both
models, and while both are technically usable, I find Qt's model
orders of magnitude more robust and less error-prone. In my
experience, the difference is night and day: bugs in Qt programs tend
to be obvious and easy to find by looking at stack backtraces and
such, whereas libasync-style code even by the most competent
programmer just oozes tricky race conditions leading to subtle
dangling pointer bugs that can take weeks to track down, if they are
ever found at all. There is a clear and simple reason for this. The
libasync/yam/asio model is based on function closures (what boost's
"bind" module does), a concept taken from functional languages with
automatic memory management, and shoehorned into a very non-
functional language (C++) without garbage collection. In libasync/
yam/asio, whenever you register an asynchronous callback for any
purpose, you generally create and pass a function closure that
"wraps" some arguments that the callback will need when it eventually
gets called. Those function arguments almost invariably have to
include pointers or references to objects in the heap that represent
parts of the system's state - otherwise the callback couldn't know
exactly what it is supposed to do or with which objects. But those
function closures, and the pointer/reference arguments they contain,
don't automatically have any connection to the lifetimes of the
specific heap objects those arguments refer to. (You can wrap
reference-counted pointers in closures, of course, but those are
likely to create cyclic references...) The closure model essentially
puts the programmer into a "fire and forget" frame of mind - that's
why it _seems_ like such "a pleasure to use" as you say - but this
frame of mind is really a giant sand trap because if you _do_
actually "forget" what async actions you've actually fired off and
accidentally delete an object they refer to before all async actions
referring to them have completed, you're hosed. And those are
precisely the kind of bugs that almost never happen except at random
times when your system comes under heavy load, objects start getting
created and deleted more frequently, and the race conditions start
emerging from the woodwork like cockroaches.
In Qt's signal/slot event delivery model, in contrast, the lifetime
of any asynchronous notification is tied to the lifetimes of both the
object originating the signal and the object to which it is
delivered: if either object is destroyed, Qt silently and harmlessly
severs the connection between them. Qt's model does not tempt the
programmer with the false promise of a "fire and forget" model that
doesn't actually work in a language in which objects change state and
can be deleted at any time leaving dangling pointers. If you want to
get an asynchronous notification of an event in Qt, you have to have
an object around to receive it; if you delete the object, you won't
see the event. Which is far more robust and almost always exactly
what you want. In summary, I think that use of the wrap/bind/
function closure model for async programming in C++ is a terrible
idea, and frankly something like asio built on that model should
never have been accepted into a library that purports to encourage
clean, robust programming practices.
These are just my personal opinions, of course - no doub others will
disagree. As I said in my earlier E-mail, though, I'm happy in any
case to work with anyone interested in using SST regardless of what
async toolkit they've chosen (or rolled their own), and will do my
best to make the prototype easy to incorporate into applications in
the future.
Cheers,
Bryan
_______________________________________________
p2p-hackers mailing list
[email protected]
http://lists.zooko.com/mailman/listinfo/p2p-hackers
