On Fri, Oct 07, 2005 at 05:42:31PM -0400, Rick Mohr wrote: > > On Fri, 7 Oct 2005, michael chang wrote: > > > So the key here is that UDP may not be reliable an enough of a > > solution for your situation, and in that case, a custom TCP may be a > > better idea. > > > > Now, there is one thing to note -- if the packets are being sent every > > second, if one packet it lost, it should be noted there'll be another > > one in the next second. That said, because ganglia may send simple > > differential packets, if one gets lost, then the whole results > > recieved by e.g. the web interface may drift off the actual value. > > (That said, the full value is resent every hour or few hours or so, > > IIRC.) > > I agree that a TCP solution would probably be needed for this particular > application. Basically, it boils down to understanding the application > well enough to know whether UDP loss is acceptable. > > Because Ganglia allows you to do things like set thresholds for sending > data, expiring data if it gets too old, etc. we have been able to work > around such limitations. For my applications, I just assume that some > data will be lost. Then I just have to figure out how much loss is > acceptable, and send some sort of notification via Nagios when it passes > the acceptable limit. > > Although it would be really cool if Ganglia implemented some TCP support > for reliable communications. But if there are a large number of TCP > connections, then there might have to be some sort of "tree" structure. > Rather than having 1000 nodes trying to open connections to the a single > gmond to send their metrics, there would probably need to be "collectors" > so that these 100 nodes sent data to collector #1, then next 100 sent it > to collector #2, etc. Then there could be a main collector that collects > metrics from the sub-collectors....
At the data rates ganglia produces, 1000+ TCP sessions should be a complete non-issue other than the absurdly low default values many OSes use for maximum numner of descriptors per process and possiably overly large default window sizes. -- Brooks -- Any statement of the form "X is the one, true Y" is FALSE. PGP fingerprint 655D 519C 26A7 82E7 2529 9BF0 5D8E 8BE9 F238 1AD4
pgpWfNZGKTa5E.pgp
Description: PGP signature
