Hello everyone, we've been running equalizer for a while in a large visualization cluster. Our architecture has 18 nodes, each with 4 GPUs. The Equalizer configuration file actually defines 72 nodes, each with one pipe, one window and finally one channel (with the pipe assigned to the corresponding GPU on the system and the OpenGL context getting properly created there). Furthermore, for each GPU, we define one canvas and one segment. So we end up with 72 nodes and 72 canvases.
What I am observing is best described as jittering or frame stuttering, happening about every 1 second. The frame rate will drop from 100s of FPS down to 1-2 very briefly and then recover, only to happen soon there after. This is not related to rendering complexity (it happens with very simple scenes and also with the various eq samples). I did some profiling on the AppNode driving the cluster in order to narrow down the source of the issue. I am noticing hotspots in co::LocalNode::_runReceiverThread (38.17% of all samples). In particular, there seems to be a bunch of time spent within co::LocalNode::_handleData (26.6% of all CPU time) and approximately 12.7% for the call to ConnectionSet::select() within the same function (_runReceiverThread). The second hotspot that I've noticed is in the ServerThread::run() function and more specifically in _cmdStartFrame() (roughly 25% of CPU time spent there). Our application is relatively simple with a basic distributed object for application state (a few kb in size). This object gets commited 2-3 times during a single event/frame loop. I've tried a number of things to work around this: -Forcing swap-sync to off through out the cluster -Trying different pipe threading modes -Setting up RSP (which seems to work but makes no difference) -Played around with swap barries -Disabled statistics collection None of the above made any significant difference in terms of performance. My next step is to simplify the equalizer configuration by fixing the mess that I currently have with the 72 canvases and actually use 4 canvases (one per wall) with properly defined segments. Meanwhile, I'd love to get people's input on the above! Thank you all, Harris -- View this message in context: http://software.1713.n2.nabble.com/Jittery-performance-with-large-cluster-tp7585928.html Sent from the Equalizer - Parallel Rendering mailing list archive at Nabble.com. _______________________________________________ eq-dev mailing list [email protected] http://www.equalizergraphics.com/cgi-bin/mailman/listinfo/eq-dev http://www.equalizergraphics.com
