Re: [jabberd2] Re: Rate limiting non-functional?
Hmm, two people have told me my last email was empty, so here it is again: FYI I just added a stanza rate limit. Instead of limiting based on the number of bytes it limits based on the combined number of message, presence and iq requests (regardless of their size). It's configurable in c2s.xml -Mark -- To unsubscribe send a mail to [EMAIL PROTECTED]
Re: [jabberd2] Re: Rate limiting non-functional?
On Fri, Jul 18, 2008 at 2:23 AM, Tomasz Sterna [EMAIL PROTECTED] wrote: Dnia 2008-07-11, pią o godzinie 13:48 -0700, Mark Doliner pisze: 1. Don't disconnect people when they hit the rate limit. Instead, stop reading from their socket until the limit has expired. Do we have a way to call a function after a certain amount of time has passed? We would need one for this. There is another problem with this reported to me today. If user has a lot of messages in offline storage, these get bursted to the client after connection, hitting the rate limit and effectively disconnecting the user. :-/ That seems a little odd... I think only data read from the client counts toward the limit. -Mark
[jabberd2] External libraries: libevent and GLib
I just thought I'd throw this idea out there... has anyone ever thought of changing jabberd to use libevent or GLib? libevent is an event notification library that abstracts the APIs for poll, select, epoll, kqueue and maybe a few other things. I guess we would really only gain support for kqueue (the fast event polling mechanism used on FreeBSD (and other BSDs?)), since jabberd already has implementations for poll, select and epoll. And I guess we would lose support for wasync? It's probably not worth the effort, but if libevent existed when jabberd was first written it seems like someone could have saved themselves a good bit of time. And I'm always in favor of re-using shared libraries. I haven't used libevent, but it seems nice. You can find more information at http://monkey.org/~provos/libevent/ There's also libev, which does basically the same thing but is purportedly a tad bit faster. GLib is a C library with lots of random and useful functions. It provides lots of handy string manipulation functions, a generic linked list structure, a queue, a hash table that grows and shrinks automatically, base64 encoding/decoding (recently), etc. Again, we probably wouldn't gain a whole lot from it immediately, since jabberd has already been written with it's own implementations of a lot of this stuff, but I'm always in favor of re-using shared libraries. I've used GLib a lot and I love it. It's fantastic. You can see the documentation at http://library.gnome.org/devel/glib/stable/ Anyway, probably not something we want to dive into now. I mostly just wanted to make people aware of the existence of the two libraries. -Mark -- To unsubscribe send a mail to [EMAIL PROTECTED]
Re: [jabberd2] Re: Rate limiting non-functional?
On Sun, Jul 13, 2008 at 1:58 AM, Tomasz Sterna [EMAIL PROTECTED] wrote: Dnia 2008-07-11, pią o godzinie 13:48 -0700, Mark Doliner pisze: Do we have a way to call a function after a certain amount of time has passed? We would need one for this. We have time checks in main loop, but this is pretty hacky. Using/implementing more sophisticated event loop is a good idea. Yeah, I agree that that sounds like a good idea. I guess it makes sense to leave the rate limiting in router and it will become more useful when/if someone adds some sort of timeout functionality and then improves the rate limiting. I don't think I'll have time to attempt to do that right now. But that's ok, I'm not sure my implementation would be very good :-) Thanks for the response, Mark
[jabberd2] Re: Rate limiting non-functional?
On Mon, Jul 7, 2008 at 10:25 AM, Mark Doliner [EMAIL PROTECTED] wrote: I've been looking at the rate limiting options in c2s.xml, and looking at the code that intends to enforce those limits. It looks like the connection rate limit would work (but I haven't tested it), but it looks like the byte limit code has a few major problems that would keep it from doing anything useful. Has anyone successfully used byte limiting? For starters there's no call to rate_add(), which means jabberd checks if the rate has been exceeded, but it always checks if 0 rate_limit, which will never happen. Then there's a call to rate_left(), which I think is intended to have the recv() call only attempt to read the number of bytes that are allowed before you would be rate limited, but the return value from rate_left() is never used. Those two problems are fairly easy to fix. The third problem I've noticed, which is more difficult, is what to do once someone has been rate limited. Ideally jabberd would just stop reading data from that user until the desired number of seconds has elapsed. But that's hard to do because I think our mio code uses readiness change notification/edge-triggered readiness notification instead of level-triggered readiness notification. So we can't just not read from the socket during this function call because _c2s_client_sx_callback() won't be called again. We could disconnect the user pretty easily, but that's a bit clunky. I noticed that router has similar rate limiting, but there the byte rate limit is a little more effective, but I think it has the same problem with ceasing to read from the socket once it gets limited. It seems weird to me that the router would limit the traffic sent between the components and the rate at which new components can connect. I'm in favor of removing this limiting from router for the sake of simplifying the code and simplifying router.xml. How do other people feel? I noticed another flaw in the rate implementation. rate_t-bad is set when rate_add() is called, and rate_t-time is only reset in rate_reset(). What ends up happening is that the rate count gradually increases and approaches the limit. It eventually calls rate_add() which pushes the count over the limit and you get rate limited. This isn't so bad, except that it happens regardless of the amount of time that has elapsed since the first message. -Mark -- To unsubscribe send a mail to [EMAIL PROTECTED]
[jabberd2] Rate limiting non-functional?
I've been looking at the rate limiting options in c2s.xml, and looking at the code that intends to enforce those limits. It looks like the connection rate limit would work (but I haven't tested it), but it looks like the byte limit code has a few major problems that would keep it from doing anything useful. Has anyone successfully used byte limiting? For starters there's no call to rate_add(), which means jabberd checks if the rate has been exceeded, but it always checks if 0 rate_limit, which will never happen. Then there's a call to rate_left(), which I think is intended to have the recv() call only attempt to read the number of bytes that are allowed before you would be rate limited, but the return value from rate_left() is never used. Those two problems are fairly easy to fix. The third problem I've noticed, which is more difficult, is what to do once someone has been rate limited. Ideally jabberd would just stop reading data from that user until the desired number of seconds has elapsed. But that's hard to do because I think our mio code uses readiness change notification/edge-triggered readiness notification instead of level-triggered readiness notification. So we can't just not read from the socket during this function call because _c2s_client_sx_callback() won't be called again. We could disconnect the user pretty easily, but that's a bit clunky. -Mark -- To unsubscribe send a mail to [EMAIL PROTECTED]
Re: [jabberd2] asynchronous DNS and resolver component
On Nov 2, 2007 at 2:42 AM Tomasz Sterna wrote: I had started implementing asynchronous DNS resolving in jabberd, to really fix problem of resolver timing out requests by accumulation. If there are 20 requests to process at the moment, and each takes 5 seconds to resolve (default libresolv timeout), then with 30 seconds request timeout (default jabberd timeout), resolver will process only 6 first requests, with 14 failing with timeout when resolver didn't even try to resolv it. I tried to fix it by prolonging timeouts, but it hurts on the user experience point of view - when a request takes more than 30 seconds to finish, users tend to cancel it and retry. Answer after 5 mins of waiting is not an option. So, the real solution is to implement asynchronous DNS resolving. I'm going to use UDNS http://www.corpit.ru/mjt/udns.html library. Is everyone OK with it? I've never used a library for asynchronous DNS, but this one looks good to me. FYI Pidgin has two homegrown async DNS implementations, one forks child processes and talks to them with pipes, and one spawns threads. Using this library seems like a better idea than implementing either of those methods yourself. And the second question arises. With async DNS, we are able to ditch the whole resolver component completely - s2s will be perfectly able to do the resolution by itself. Resolver component was only needed to delegate the blocking DNS resolution requests to another process. I do see a use for resolver component in big, distributed deployments, to do central name-resolution for all present s2s components. But as for now, the resolver does only resolving. The name-2-address mappings are cached... but in the s2s itself! So, with the current design, we could move the resolution to s2s and loose nothing. Or we could move the caching to resolver. The s2s based resolving could also help in split-horizon DNS cases. I personally am after the first option. Am I loosing something, or we could just go with ditching the resolver? Getting rid of resolver seems like a good idea to me. If big, distributed deployments really want to do central name-resolution they could easily run a caching DNS server and have all DNS queries proxied through that. And then, if we do left resolver behind, does it justify raising the minor number of jabberd version. It sure is architectural change, and I think we should release the ADNS branch under 2.2.x line. Raising the minor number sounds like a good idea to me. And it'll help people see that jabberd really is under active development. -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
Re: [jabberd2] Server to Server can't connet for DNS reason
On Oct 20, 2007 at 4:20 AM, ?? wrote: Dear Sir To test the S2S connection function, I installed two jabberd2 servers in LAN, that are Server1 and Server2. Because lack of DNS, I modfied the /ets/hosts in each of Linux as follows: == 192.168.82.123 server1 192.168.82.122 server2. == The first client, its jid [EMAIL PROTECTED] , logon the server1, while the second one [EMAIL PROTECTED] logon the server2. Then the first client invited the second, but nothing happened. In the server1's log write: server2 can't be resolved. I don't think the method have any problem, Why can't the two server communication? Is it nessary to have a DNS server? jabberd2 currently does not use the /etc/hosts file for resolving domain names--it ONLY uses DNS. There are some notes about this http://jabberd2.xiaoka.com/ticket/63 and http://jabberd2.xiaoka.com/wiki/OldTODO So right now, yes, it is necessary to have a DNS server. -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
Re: [jabberd2] s2s.xml id
On Oct 21, 2007 at 7:57 PM, ?? wrote: What does the id in s2s.xml represent? The comment writes Our ID on the network. I think the id should be the hostname. As does sm.xml and c2s.xml, I also changed the default to my hostname. But It's wrong . sm died. Please explpain more details Thanks The id is just a name that jabberd2 uses internally to refer to the server to server process. Leaving it as the default s2s should be fine. Each one of the five jabberd2 processes should have their own unique id--it's what they use to talk to each other. Changing the s2s id to the hostname will break things because the sm id is supposed to be set to the hostname. -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
Re: [jabberd2] Clustering
-Original Message- From: Tomasz Sterna Sent: Wednesday, October 17, 2007 1:34 PM To: jabberd2 development Subject: Re: [jabberd2] Clustering Dnia 17-10-2007, Śr o godzinie 13:05 -0700, Mark Doliner pisze: Doesn't this make the router a bottleneck? Router has almost no logic - it only passes packets through. I'm worried more about the fact that it is a single point of failure. We may introduce router mesh. Set of interconnected routers that exchange their routing information. A router receiving bound name advertisement records it in the routing table. Same for routing method (domain/bare-jid/full-jid) change requests when connected router detects conflict. Example: sm1 - router1 - router2 - sm2 router1 table: sm1 router2 sm2* - advertisement record (presence from component) router2 table: sm2 1234 router1 2345 sm1* 2345 - advertisement record (presence from component) Creating a clustering component [...] I don't like the idea of introducing yet another component in the puzzle. The clustering functionality should be handled in the router. Hmm, it DOES make sense to have the router do the routing... And the router mesh idea seems cleaner than having a cluster component route packets from one cluster to another using the s2s component... Also, when would sm bind to a bare jid? When the router detects bind conflict. It tells the conflicting components to go to the lower level. And there would be a bind conflict when two sm instances try to connect to router with the id jabber.org, right? And it would also happen when a single user logs in from two locations using two different resources? how does c2s decide which sm instance to forward the login to? C2S needs changes in the SM tracking logic. When it receives advertisements of bare-jids, it starts routing by bare-jids, not domain. Same for full-jids. When a user logs in they get assigned to one of the sm instances, right? How would c2s decide which sm instance they should be assigned to? Would the router make that decision somehow? -Mark ___ Jabberd2 mailing list Jabberd2@lists.xiaoka.com http://lists.xiaoka.com/listinfo.cgi/jabberd2-xiaoka.com
