On Fri, 7 Oct 2005, michael chang wrote: > So the key here is that UDP may not be reliable an enough of a > solution for your situation, and in that case, a custom TCP may be a > better idea. > > Now, there is one thing to note -- if the packets are being sent every > second, if one packet it lost, it should be noted there'll be another > one in the next second. That said, because ganglia may send simple > differential packets, if one gets lost, then the whole results > recieved by e.g. the web interface may drift off the actual value. > (That said, the full value is resent every hour or few hours or so, > IIRC.)
I agree that a TCP solution would probably be needed for this particular application. Basically, it boils down to understanding the application well enough to know whether UDP loss is acceptable. Because Ganglia allows you to do things like set thresholds for sending data, expiring data if it gets too old, etc. we have been able to work around such limitations. For my applications, I just assume that some data will be lost. Then I just have to figure out how much loss is acceptable, and send some sort of notification via Nagios when it passes the acceptable limit. Although it would be really cool if Ganglia implemented some TCP support for reliable communications. But if there are a large number of TCP connections, then there might have to be some sort of "tree" structure. Rather than having 1000 nodes trying to open connections to the a single gmond to send their metrics, there would probably need to be "collectors" so that these 100 nodes sent data to collector #1, then next 100 sent it to collector #2, etc. Then there could be a main collector that collects metrics from the sub-collectors.... But that would be quite a bit of work. --Rick -------------------------- Rick Mohr Systems Developer Ohio Supercomputer Center
