On 03/20/2015 06:54 AM, Robert Norris wrote: > Hi all, > > I've got a prototype for a Prosody feature that I want/need. I'm keen to > get your input on it. > > What I want to do is distribute users for a single domain across > multiple Prosody instances while having them look like a single server > on the outside. The reasons for this are partly to do with load - I'm > preparing for a time where there are thousands of users online - but > also to fit neatly into our existing infrastructure, where we spread > email users across multiple servers. I like the idea of mirroring our > email architecture and doing redundancy the same way. > > Note that I'm expressly not talking about real live clustering, where a > server can fail and the user remains connected. A Prosody instance (or > its underlying machine) can fail and the sessions will be disconnected. > That's totally fine with me - most clients will quietly reconnect > without issue.
Great. I'm was working on that already so maybe we can share ideas. Be aware that this work was done several months ago for 0.9.4 and I had to stop working on this due to work load. > It seems that all that's necessary to make this happen is to hook into > the stanza delivery code. If we have a session for a user on the current > server, we deliver it to them. If not, we do a lookup (via an external > daemon) to see where the "home" server for the user is. If we find one, > we pass the stanza over to that server and let it deliver it. If not, we > continue as normal. > > So I have a very rough prototype in this three plugins: > > https://github.com/robn/prosody/blob/shard/plugins/mod_shard.lua > https://github.com/robn/prosody/blob/shard/plugins/mod_shard_client.lua > https://github.com/robn/prosody/blob/shard/plugins/mod_shard_server.lua > > mod_shard is the shard lookup hook. mod_shard_client and > mod_shard_server are the bits that pass stanzas between servers. They're > not especially interesting, and are mostly hacked from mod_component and > mod_component_client. I'd implemented a module to store the info of what jid is where over redis here: https://github.com/sipwise/prosody/blob/master/plugins/mod_sipwise_redis_sessions.lua You just have to ask redis_sessions.get(jid) to know where to send the stanza > I'm assuming a very controlled environment here - secure networks (no > auth required between shards), a user->shard lookup daemon, common > storage and auth and a proxy that can get incoming user connections to > the right shard (in my case that's nginx-xmpp). I'm expecting s2s to be > covered by a par of Prosody instances that accept stanzas and pass them > to the right shard. I should be able to run multiple of those because > they're basically stateless. But did you try that? My approach was to extend mod_s2s as mod_s2sc to connect the nodes: https://github.com/sipwise/prosody/blob/master/core/s2scmanager.lua https://github.com/sipwise/prosody/tree/master/plugins/mod_s2s_cluster https://github.com/sipwise/prosody/blob/master/plugins/mod_s2sc_dialback.lua And this module is the one that hooks and decides if the stanza needs to be routed somewhere else: https://github.com/sipwise/prosody/blob/master/plugins/mod_sipwise_cluster.lua > So that's the whole description. Hopefully it all made sense! What I'm > looking for is some feedback on whether the way I've implemented this > makes sense (half-arsed prototype code notwithstanding) and any thoughts > about what user-visible differences in XMPP it might present (stanza > delivery order comes to mind). And any other thoughts you might have! > So I think we are trying to do the same thing. I have to point that I didn't finish to polish this and for sure it could not work as it is right now. But you can get the idea. Looking forward your comments Victor Seva -- You received this message because you are subscribed to the Google Groups "prosody-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to prosody-dev+unsubscr...@googlegroups.com. To post to this group, send email to email@example.com. Visit this group at http://groups.google.com/group/prosody-dev. For more options, visit https://groups.google.com/d/optout.
Description: OpenPGP digital signature