On Tue, Oct 30, 2012 at 09:54:36AM -0500, Romain Beauxis wrote: > Hi Ken! > > 2012/10/29 Ken Restivo <[email protected]>: > > Hello folks, thanks for liquidsoap. I'm very glad someone created a > > streaming engine written in a functional language. > > Thanks! > > > Earlier this summer I set up a server for a micropower radio station in a > > remote location. > > > > It's running an old 3Ghz P4 with 2GB RAM, Ubuntu 10.04, and airtime. > > > > They wanted a jukebox for times there was no scheduled show playing, so, > > rather than tweak airtime, I set up a separate liquidsoap instance that ran > > a custom script. The script is below. > > > > It basically plays a jukebox, but falls forward to a live stream or to > > airtime, if either of those become available. It worked for a month or two > > flawlessly. > > > > The problem is, after a few months running, that it'd periodically stutter > > approximately 400ms of audio endlessly, then after 10-60 seconds move on to > > another frame, stutter that one, then it'd do a scrambled-sounding insane > > thing that sounds like an mp3 stream seeking past its key frame, then go > > back to stuttering, for hours. The problem would clear itself after some > > hours had passed. CPU usage was minimal at those times. There's no RT > > kernel, but I had liquidsiap nice'd -20 so it ran with higher priority than > > anything. The problem seems to happen randomly, intermittendly, typically > > once or twice a day, at no predictable time, and regardless of whether the > > active source is the jukebox, the livestream, or the airtime stream. > > > > The logs all look completely normal, no errors, no overruns, etc. At one > > point I did see an overrun, at the point where the error STOPPED happening, > > not when it started, which was odd. At the time that it's stuttering, the > > jukebox (or livestream, or airtime), looks like it's just cruising along > > doing what it's supposed to be doing. > > > > This was with Ubuntu's (old) liquidsoap package. I built a custom > > liquidsoap-1.0.1 from source, installed it, and ran that.... and it did not > > solve the problem. > > > > I've since ditched the separate liquidsoap instance completely, and now run > > these functions from inside airtime. It did not solve the problem. > > > > I asked on the linux audio list, and one of the ALSA devs pointed out that > > stuttering is caused by the application not sending data to the driver, so > > the hardware just keeps playing the same frame over and over again until it > > gets a new one from the application. But this isn't one frame (~40ms) it's > > a 400ms chunk. And the error indicates something more complex. > > > > Since this is airtime, I'm certain I'll get at least one reply telling me > > to go ask the Airtime people. They, in turn, sent me here. I did just today > > upgrade to airtime 2.2 which uses liquidsoap 1.0.1, but I didn't see any > > issues related to this problem, or anything in the changelog indicating it > > fixed it. Though, they are now using mp3replaygain in this new version, so > > CPU usage has gone sky high. I'll know by tomorrow if the upgrade fixed > > anything. > > > > My questions are: > > > > 1) Are there any things in the below script that look like the could cause > > liquidsoap to do this crazy thing? I am not new to functional programming > > or linux audio, but I am new to liquidsoap. Am I doing anything obviously > > wrong in the code below? > > > > 2) How would I find out when is happening, or that it had happened? I found > > the rms() command, which looks like it could be helpful for detecting > > silence. But what unit of sound is this 400ms chunk? What cuts it up into > > those size hunks? Is there some way to compare whether two or more > > sequential blocks like that are exactly equal to each other (and flag it > > perhaps as an error)? > > > > 3) How could I get more data as to what exactly is wrong? > > Logs could help, particularly in verbose mode -- set("log.level",4) >
Thanks, I didn't think to increase log verbosity, doh. Could try that. > As for your issue, I would suspect some bug with the alsa > driver/output. First thing to try to confirm that would be to use an > alternative output. I'd suggest output.ao which is generally the most > robust. We tried it with AO. Same problem. I should also note that I misspoke in my original report: at the time the stuttering would happen, liquidsoap would inded be at minimal 5% load, but the rest of the system was quite loaded down! Apache would be at 70% load or something similarly ridiculous, for short periods, and average about 40% CPU. So, perhaps this is just resource starvation? > > If output.ao works and you want to help us fixing the bug, even though > your jukebox should be now working, then we could try to see what The jukebox is not working. Interestingly, we tried just straight stock airtime, removing my jukebox modifications completely, and... same exact problem. So it's not my script causing it. HOWEVER. We've been running mplayer on a live http mp3 stream for 24 hours now, no glitches, no problems. But also, no load. So the problem is not ALSA. It's liquidsoap, or, perhaps liquidsoap + load. Another question: what's the significance of the 400ms stutter period? It's very consistent. Is that the size of an mp3 frame? The length of a liquidsoap buffer? Is it the interval of a poll or select somewhere? Or a system timer of some kind? Finally, how well does liquidsoap run under RT priority? I've used Ingo kernels for 6 years now, perhaps it might be worthwhile to try installing one on this box and giving liquidsoap rt prio > audio card you are using. However, debugging alsa on a card that we do > not own is kinda tricky.. You may also want to test with or without > the bufferize option. > It was screwing up using an HDA-INTEL card. So they put in an ES1371/1 card and tried with that... same problem. We also tried moving from the build in ethernet card to a PCI card, and then scooting the cards around so that the network was on its own interrupt. Also, same problem. -ken ------------------------------------------------------------------------------ Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct _______________________________________________ Savonet-users mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/savonet-users
