Hi Tomasz,
I've thought (again ;-) ) about an implementation (at router side, not
yet at other components' sides) that could work mixing partly what you said
(i.e. doing things according to your different scenarios) and what I said
before (e.g. about the routers used as components).
Please read my comments interspersed between yours and at the end of
this email I'll detail my ideas and ask you some questions.
Le 06/09/2012 10:36, Tomasz Sterna a écrit :
Dnia 2012-09-06, czw o godzinie 00:29 +0200, Alexandre Jousset pisze:
2) Is is OK to assume that all non full-JIDs are of priority 0 (or,
better said, have no priority at all)?
Obviously, yes.
I'm thinking:
- domain level - priority configured in component.xml (ie. sm.xml)
configuration file, passed to router on connection.
This allows to have load balancing (same priority) or one main
component (higher priority) and backup(s) (lower priority)
- bare-JID level - highest priority bound to user resources on SM.
SM keeps a notion of "top" resource used to deliver messages directed
to bare JID. Using this resource priority will make the component
catch messages passed to bare-JID
See: https://github.com/Jabberd2/jabberd2/blob/master/sm/pres.c#L40
- full-JID - just use the bound resource priority
Ok for this.
There is one caveat here.
With current implementation, if user has two bound resources with the
same priority, we pass the message to this priority to both connections.
If we do it on router level, with current "select one randomly" it will
break this implementation.
In my ideas, one could send messages to both (i.e. all) connections
with same priority.
We may need to think what would happen if we would implement "pass to
both connections" at the router level. Maybe it would work even better
than the random approach. :-)
As told above, I can do it at router level. What I don't know is
whether that could lead to problems (e.g. about ACKs...?)
3) I have a doubt when you say at different places that a component needs to bind
bare-JIDs / full-JIDs "from now on". Does that mean that it needs to send bind
requests for all sessions it already has, or just make *new* bind requests of the type
you mentioned?
I think the component has to send all its already online sessions.
Yes. If a component gets a request to change the "binding level" it
needs to bind/unbind all pending resources (ie. bind all known bare JIDs
if upgrading from domain level bind) and keep binding new connections at
the current "binding level".
Ok for this.
And I think this can be a hint for the routers' synchronisation in multi-router
implementation. In a multi-router implementation, why not consider a router as a more or
less "normal" component (when connected to other routers)? With one exception,
that when a random has to be chosen between components, only the local components have a
chance. I have to think more about this...
We cannot as router does not have a domain it keeps all other
connections under. You can have every possible combination of resources
at any router.
Router just have names like every other component on the network, that
may be used to pass packets to.
What could work? Let's imagine a routing table like this:
JID(prio) | destination | where did I get it from
ad...@example.com/deedbeef(1) sm31 router7
.... .
null(4) | sm31 | router7
So, when you get packet directed to ad...@example.com/deedbeef, you know
that it needs to go to sm31.
Since sm31 is not your local component, you need to find sm31 component.
You find that souter7 component knows it, so you pass the packet to
router7 connection and you're done.
My idea differs here, and with it I can see the router (when connected
to another router) as an almost normal component. See below.
4) In this process, what if component1 disconnects? I suppose that the router needs
to crawl through its tree to update it, but that can be CPU intensive and can cause
lags... I know that this event is not a "normal" event anyway and is not
supposed to happen often, so it can be negligible.
In case of local component, just find the node containing the
disconnected component, and iterate through all nodes and "disconnect"
them. It's not that hard or slow as you may think. It's fast and happen
in jabberd2 code many, many times. We have a very fast hash-table
implementation. :-)
See: https://github.com/Jabberd2/jabberd2/blob/master/util/xhash.h
If you have remote router connections, you just broadcast the component
disconnections to them, the same way it is broadcasted to local
components.
In case of remote component disconnection, you just remove the entries
for the component from your routing table. Simple.
Ok for this.
5) A Jabber protocol question, I know I could find the answer in the online docs,
but as I have you at hand ;-) Is it possible to have 2 full-JIDs connected at the same
time? With equal or different priorities? I suppose the answer is "no" to both
questions, but just to be sure...
Obviously, no.
Full-JIDs need to be unique.
See: https://github.com/Jabberd2/jabberd2/blob/master/sm/sess.c#L141
Ok. That leads me to a question about conflicts, see below.
6) The tree structure I'm thinking about uses hashes to find the next node (root => domain =>
user => resource), and finally a pointer or an array of pointers to components for the leaves. If the
answer to previous questions at 5) is "no", there is only one case where there can be more than
one leaf: at each node is a "default route" leading to one or more components. If I use a static
sized array to store them, that will use more memory. So the best would be to use a linked list, but that
would make the process slower. I would tend to use linked lists because the cases where one has to crawl
through them are (relatively) less frequent. What do you think?
Stupid question: the case where there is multiple choice for components is only
the "domain" case. So I can use a reasonnably configurable-at-compile-time
sized array.
Not really. You can have multiple bounds on bare-jid level also. There
can be several components handling 'ad...@example.org' connections. Each
with separate set of 'ad...@example.org/something' full-JIDs.
Ok for this. I'll use linked lists, which will be pretty small after
all (and ordered, too).
Look at the current implementation:
https://github.com/Jabberd2/jabberd2/blob/master/router/router.c#L159
And just extend it to e tree structure, with current implementation at
each node. :-)
Of course I'll try to minimize the changes. Anyway, as in my previous
attempt I've set up 2 options for the configure script: --enable-adaptive-routing
and --enable-cluster (<= should I change this last one to --enable-router-mesh?
;-) ) that activate these functionalities that should be marked as experimental
for a while.
Now, some explanations about my (new) ideas. I hope they will be doable.
At first I thought about a real tree structure, as I explained in a previous email, with "root
=> domains => nodes => resources" structure. But after seeing your example routing table
above, I thought "why not use a single flat hash table that could be used as a tree as well?".
It's a detail but I think it simplifies some implementation details.
The differences with your hash table is at the pointed structures level. After
all, why the router should care (and even know!) that sm31, connected on router7, must be
used to route packets for "ad...@example.com/deedbeef(1)"? Why not just store
the component_t (of router7 if remote, or of sm31 if local) in the data structure pointed
by the hash value?
This way a router connected to another router can be seen, for the
routing purpose, as a normal component. What do you think about this?
So, in the hash table values structure, one can store a linked list of possible
routes (which is a structure containing a component_t, a priority and a "next"
pointer) sorted descendant by priorities, and an enum containing the request type for
this entry (domain, bare JID or full JID).
About the default route(s), they could be stored in the hash table using the same
schema (using a special hash key), so there could be more than one "default
route" (with, e.g., a constraint about not being possible to have 2 default routes
with same priority).
Components like sm and c2s could be stored the same, using just the
"domain" part of the JID, as it is the case now.
I've started to write some pseudo code about how binding and routing
could be done along with a quick description of data structures that could
possibly be used. It's a pretty low level (but somewhat short) pseudo code that
could be implemented very quickly in the router.
I can post this pseudo code to illustrate my ideas and give a clue
about how it could be implemented. If you don't like my ideas and propose some
better ideas, I'll trash mine just like my previous attempt :-p It's not a
problem as I liked spending some time to think about these topics that are
quite new for me.
About possible conflicts: I haven't spent a lot of time thinking about the router
tables synchronization but I see them as just forwarding binding/unbinding requests,
except that the component_t stored in the remote router would be the local router one (as
seen by the remote) instead of the real local component one (which is actually not
connected to the remote router). In my preliminary thoughts I see one reason to have a
conflict: when one user use 2 computers at the same time to bind using the same full JID.
Actually not necessarily at the exact same time because of possible binding of more
sessions after a domain or bare JID binding (you know, when the component has to send all
its sessions). If a router could say "JID already bound" and force a resource
change (after the session started), that's ok. If it is not possible to change the
resource after the session is started, we should find another way to resolve conflicts.
This is a Jabber protocol question again, I'm sor
ry :-p
Just a note about the routing table sharing, when a component is
requested to send binds for all its sessions, this could be done in a bulk way.
Same for unbinding. And this bulk could be forwarded as-is to other connected
routers.
Well, these are my thoughts for the moment :-) I hope I'm clear enough
in my explanations.
What do you think? :-)
--
-- \^/ --
-- -/ O \--------------------------------------- --
-- | |/ \| Alexandre (Midnite) Jousset | --
-- -|___|--------------------------------------- --
