Yay, it's big commit time!
I've landed my XMPP output queuing work in 0.9.x, running on my public
test site at http://status.leuksman.com/ Do let me know if XMPP
subscription or i/o is flaky on it!
Should be ready to merge to testing & master and deploy when we're
content to do so... there is a database change necessary for the
DB-based queueing system, so I want to confirm that's not a problem
before pushing out.
Big thanks to Craig Andrews for his work on generalizing the DB queues
which I've started integrating; we should be able to land the rest
including the IM pluginization for 1.0.
Commit summary:
XMPP queued output & initial retooling of DB queue manager to support
non-Notice objects.
Queue handlers for XMPP individual & firehose output now send their XML
stanzas to another output queue instead of connecting directly to the
chat server. This lets us have as many general processing threads as we
need, while all actual XMPP input and output go through a single daemon
with a single connection open.
This avoids problems with multiple connected resources:
* multiple windows shown in some chat clients (psi, gajim, kopete)
* extra load on server
* incoming message delivery forwarding issues
Database changes:
* queue_item drops 'notice_id' in favor of a 'frame' blob. This is based
on Craig Andrews' work branch to generalize queues to take any object,
but conservatively leaving out the serialization for now. Table updater
(preserves any existing queued items) in db/rc3to09.sql
Code changes to watch out for:
* Queue handlers should now define a handle() method instead of
handle_notice()
* QueueDaemon and XmppDaemon now share common i/o (IoMaster) and
respawning thread management (RespawningDaemon) infrastructure.
* The polling XmppConfirmManager has been dropped, as the message is
queued directly when saving IM settings.
* Enable $config['queue']['debug_memory'] to output current memory usage
at each run through the event loop to watch for memory leaks
To do:
* Adapt XMPP i/o to component connection mode for multi-site support.
* XMPP input can also be broken out to a queue, which would allow the
actual notice save etc to be handled by general queue threads.
* Make sure there are no problems with simply pushing serialized Notice
objects to queues.
* Find a way to improve interactive performance of the database-backed
queue handler; polling is pretty painful to XMPP.
* Possibly redo the way QueueHandlers are injected into a QueueManager.
The grouping used to split out the XMPP output queue is a bit awkward.
-- brion vibber (brion @ status.net)
_______________________________________________
StatusNet-dev mailing list
StatusNet-dev@lists.status.net
http://lists.status.net/mailman/listinfo/statusnet-dev
