Hello! We didn't realize that interest in our efforts was so high among this group, so we were a bit surprised to see the discussion we've prompted. Thanks for your interest! Thanks also to Peter Saint-Andre and Anders Conbere, who called this to our attention and suggested that we send this message.
Now that we're aware of the interest on this list, we want to give you the full story. What I'd like to do is provide a summary of what we're actually doing with Android, what technical speedbumps we encountered, and what our current plans are. Note that I am merely on the Android team, so please don't get the impression that I am speaking for other Google teams using XMPP; I can only speak to what we're doing specifically with XMPP for Android. First, our goals. We are working on integrating Google Talk into our suite of applications for the upcoming Android platform. For Android, we have two (technically unrelated) needs: send instant messages via our Google Talk servers, and send machine-readable data messages between apps running on different handsets. Obviously XMPP provides a way to do both of these, so we started defining a way to use our Google Talk servers to kill two birds with one stone. Unfortunately, we ran into a few roadblocks pretty early on in this process. We reached the conclusion that unmodified (that is, standard) XMPP is not well-suited to mobile applications. (Unfortunately we still shipped a class named XMPPService in our earlier SDK pre-release, even though it wasn't actually using standard XMPP. This was the source of some confusion, which was bad and wasn't our intention.) I've provided summaries of the reasons below. Of course I haven't been a subscriber to this list for long, but from looking at your archives I don't think any of these are going to be much of a surprise to you. Overhead incurred on disconnects ******************************************** Our first issue was the overhead that occurs during connection. For a high-reliability network that's reasonably high-speed, this isn't a big deal. However, mobile networks are frequently quite a bit slower than many other networks, so connection startup overhead is critical to startup time. Also, since mobiles get temporarily disconnected a lot due to transient network issues, we incur this overhead fairly frequently. In our tests, we saw fairly high connection-startup overhead with plain XMPP. (When I say "connection startup", I am referring to everything up to the roster transmission.) High connection overhead multiplied by frequent disconnections multiplied by millions of users is also quite a bandwidth load for a mobile carrier, and these costs are frequently passed on to the user. I see that there has been some discussion about a spec to support resumable sessions in XMPP: that is along the lines of what we need here. Bandwidth and CPU overhead ******************************************** Bandwidth used means radio transmissions sent, and overhead means more work done by the processor, both of which take battery power and reduce battery life. Meanwhile, compression turned out to not be very helpful. Since it's negotiated during connection startup, it doesn't help with startup overhead. It does help somewhat with steady-state bandwidth, but at the expense of additional CPU cycles. The result is that enabling compression actually reduced battery life in our tests -- it took more power for the CPU to do compression than we saved on radio power. In other words, zlib can save you bandwidth, but only at a significant net cost of battery life. This left us stuck between a rock and a hard place. Unreliable Connectivity ******************************************** Mobile data networks tend to be less reliable (in the TCP sense, not the high-availability sense) than terrestrial and even 802.11 networks. First there's the obvious problem of devices traveling in and out of range, meaning the entire network interface will go up and down fairly frequently. In some cases, even simply receiving a phone call disables the network connection. Such connectivity issues aren't a separate problem on their own, but they make the problems above much worse. Unfortunately carrier networks are all different (for instance some use proxies and others don't), and while it may be possible to tweak a protocol like XMPP to work correctly on one carrier's network, we need something that's robust enough to function usefully on them all with no (or at least minimal) changes. So, those are the key reasons why vanilla XMPP wasn't working for us. Once we realized that, we did consider some alternatives before deciding to blaze our own trail. For binary XML, we looked at WBXML and EXI. WBXML does not easily support namespaces, which makes it difficult to use with XMPP. Meanwhile, EXI is promising, but as we understand it's not yet a formal standard. We considered XMPP over BOSH, but felt that it wasn't a good fit for Android since it adds overhead in the form of TCP connection and HTTP headers, and we were already looking to minimize overhead. In the absence of another standard, we went with a custom implementation based on our existing internal tools. So for the 1.0 release of Android, we are taking a conservative approach. We are implementing a custom client and server protocol that implements enough functionality to meet our needs. Structurally, that protocol started out similar to XMPP, although it's drifting a bit as we optimize it to work with our infrastructure. For example, some structures that are nested in XMPP have been "promoted" to separate messages to make them work better with our tools. It will work for our purposes, but it will be inevitably linked to our internal tools so it's not something that will be suitable for consideration by a standards process. All that said, we definitely plan to keep an eye on your work on the actual XMPP-for-mobile standard. As it matures, we can revisit the system we've built for Android to see if it makes sense to switch over to the standard. In the meantime, we have renamed our service GTalkService, since we recognize that what we're implementing is not actually XMPP and we don't want to confuse other developers. So, that's the background to the story. Thanks for reading this far! A standard for XMPP optimized for mobile would obviously be a very big deal for more than just Android, and we will be excited to see one come together. Since we are deep in our 1.0 release cycle, I don't know how actively we'll be able to contribute to this process. However, you folks seem to already be aware of the problems we faced -- and hopefully now we've made them even clearer. Best, - Dan Morrill
