on the server, %wa value is going above 10, if I watch it for a minute it will always jump above 10 at least once, sometimes more. on the other hand it is mostly between 0-2 the rest of the time and will often drop back to sero right after peaking for 1 refresh cycle. the highest I saw was 32. CAED is always at the top of the list, which doesn't surprise me, I would think it should be consuming the most resources, it is typicaly taking 10-15% of the CPU. It's hard to catch it but when the wa is up it seems like mysql or nfs are in the number 2 position. AFAIK there were no issues with playout at this time, at least I didn't hear anything.
When it does affect playout it is often on the order of minutes, not brief dropouts or anything like that, but minute(s) of silence. I need to go observe the client now for a little while though I suspect it will be better of, it is doing virtually nothing compared to the server. Nathaniel C. Steele Assistant Chief Engineer/Technical Director WTRM-FM / TheCrossFM On 3/15/2013 12:33 PM, Nathan Steele wrote: >> This can be a nasty to find, as there are many potential culprits -- server, >> client and the network in-between are all possibilities. > You're telling me....I've been fighting this one for a while. problem Is > I'm not sure when it started because it's so intermittent I might not > have noticed it for weeks. > >> I'd start by taking a good look at top(1) on both client and server. In >> addition to the obvious things like load averages, keep an eye on the wait >> percentage (the 'wa' field). This is an indication of how hard the I/O >> system is being worked. This should normally be hovering around zero; >> anything consistently higher than around 10% is cause for concern. > I'll take a look at this after lunch and let you know what I find. > >> Is this a gigabit network? > Yes it is, and as of this week the client and server are on their own > switch. I suppose I could start sniffing the network, But I'll have to > get that all setup. > > I"m half tempted to just come in in the middle of the night and reboot > everything, but I know I shouldn't need to do that and I'd rather > Identify the problem and know what it was. > > Thanks Fred, I"ll let you know what I find out. > > Nathaniel C. Steele > Assistant Chief Engineer/Technical Director > WTRM-FM / TheCrossFM > > On 3/15/2013 12:04 PM, Fred Gleason wrote: >> On Mar 14, 2013, at 19:43 41, Nathan Steele wrote: >> >>> Nothing appears in /var/log/messages when it happens sound output stops, >>> meters freeze, counter keeps counting, when it resumes playback, it >>> seems to pick up where it left off, but when the counter reaches zero it >>> transitions (because the counter kept counting). Rdairplay is on one >>> machine, Mysql, /var/snd , and CAE running on the server with 2 ASI >>> cards for output. >> Data starvation between the server and the client. >> >> >>> What do I need to check here? >> This can be a nasty to find, as there are many potential culprits -- server, >> client and the network in-between are all possibilities. >> >> I'd start by taking a good look at top(1) on both client and server. In >> addition to the obvious things like load averages, keep an eye on the wait >> percentage (the 'wa' field). This is an indication of how hard the I/O >> system is being worked. This should normally be hovering around zero; >> anything consistently higher than around 10% is cause for concern. >> >> If the systems are clean, then it's time to take a close look at the >> network. Is this a gigabit network? >> >> Cheers! >> >> >> |-------------------------------------------------------------------------| >> | Frederick F. Gleason, Jr. | Chief Developer | >> | | Paravel Systems | >> |-------------------------------------------------------------------------| >> | A room without books is like a body without a soul. | >> | -- Cicero | >> |-------------------------------------------------------------------------| >> >> _______________________________________________ >> Rivendell-dev mailing list >> [email protected] >> http://lists.rivendellaudio.org/mailman/listinfo/rivendell-dev >> >> >> > _______________________________________________ > Rivendell-dev mailing list > [email protected] > http://lists.rivendellaudio.org/mailman/listinfo/rivendell-dev > > > _______________________________________________ Rivendell-dev mailing list [email protected] http://lists.rivendellaudio.org/mailman/listinfo/rivendell-dev
