Hi guys, Just a thought for Romain: how about pipes used as locks inside duppy? Could there be a feature here that makes specific use of them?
Cheers, David On Thu, Jan 30, 2014 at 5:26 PM, Romain Beauxis <to...@rastageeks.org> wrote: > Hi all, > > I think the open pipes point to the problem. Beside external encoders, > calling external processes may also open pipes and it's possible that > that would be the issue. > > I see that you are using an external command for your php bridge. Can > you test without it and, if so, do you observe the same amount of open > pipes? Also, same remark for any external process that you may be > calling in your script. > > Romain > > 2014-01-30 Alexander <truenr...@gmail.com>: >> Hello! >> >> No, we don't use external encoder. But, as we support mp3/ogg/aac with big >> variety of bitrates, we have a lot of definitions for each format. I will >> send library we use to create/remove outputs (we allow to change output >> formats/bitrates on the fly) in a separate main directly to you, just in >> case. >> >> To be honest, sometimes i use "kill $pid" to stop some liquidsoap processes, >> but i don't use "-9" parameter. As i understand, there is a difference >> between "kill $pid" and "kill -9 $pid" (in first case process is able to >> close all connections, remove temp files, etc). But it is _very_ rare. Usual >> process of shutting station down liquidsoap is to execute server command via >> socket, which executes "shutdown()" function to perform clean shutdown. >> >> But, i have checked what happens when i execute "kill $pid". I have removed >> all files from /tmp, started all liquidsoap processes and then killed them >> with "kill $pid1 $pid2 $pid3". All callbacks are executed correctly. All >> files from /tmp are removed. Everything looks fine. >> >> I can still see 344 mp3 files in /tmp folder. But i am 100% sure those are >> files, that were left after liquidsoap processes had crashed. >> >> Also, i can't see any reports about dead stations after yesterday's changes >> (i have commented out cue cuts). But we shoud keep in mind that i have >> rebooted all processes yesterday. Perhaps collapse time just have not come >> (see report about "ls -l /proc/$pid/fd" below). >> >> I tried to execute server command request.alive on several stations. i can >> still see 3 requests per station. >> >> I executed "ls -l /proc/$pid/fd" for several stations, picture is quite >> different. I will give output of "ls -l /proc/$pid/fd | wc -l" (as a >> summary) for all stations: 393, 374, 28, 27, 48, 411, 400, 321, 109, 301, >> 331, 371, 28, 337, 37 >> >> Hope you will find this useful. >> >> >> >> 2014-01-30 David Baelde <david.bae...@gmail.com>: >> >>> Hi Alexander, >>> >>> Thanks for the extra info. Nothing to worry about regarding liquidget, >>> it's a harmless wrapper around wget. The logs are useful, as they >>> point to pipes rather than downloaded files. You must be using some >>> feature (external encoder, maybe) that uses pipes for interprocess >>> communication, and that does not close them properly. That being said, >>> 372 temporary files is still a lot, but that must be a different >>> problem... do you kill -9 liquidsoap instances, by any chance? >>> >>> Cheers, >>> >>> David >>> >>> On Wed, Jan 29, 2014 at 12:46 PM, Alexander <truenr...@gmail.com> wrote: >>> > Hello! >>> > I've also added your email directly to recipients list (in case >>> > sourcefourge >>> > lists will crop attachments). Please find answers on your questions >>> > below: >>> > >>> >> Are you using the standard resolver for http, that is wget? >>> > Hmm.. i suppose that we use wget. ./configure script shows in table >>> > following data (and i have wget installed): >>> > * Additional libraries >>> > - wget URI resolver : requires wget at runtime >>> > >>> > But, in liquidsoap logs i found following line: '[protocols.external:3] >>> > Found "/usr/local/lib/liquidsoap/scm/liquidget"'. So, i am not sure. >>> > >>> > >>> >> Do you have a reasonable number of alive requests? You can see this >>> >> using >>> >> the "request.alive" server command. On the server you can also perform >>> >> "request.trace" to see what happened to a request. >>> > I had to enable telnet and to restart stations (usually we use only >>> > sockets). So, i am not sure it would be informative enough. Anyway, i am >>> > attaching tcp.log file with server commands execution log. >>> > >>> > >>> >> Can you identify the too many files in "ls -l /proc/<liq pid>/fd"? If >>> >> they >>> >> are the downloaded files, you should see temporary file names >>> > starting with "liq". It would be useful to check if these files are all >>> > the >>> > downloaded files so far or only some. >>> > I see only one downloaded file there. All others are pipes (oh god, how >>> > many >>> > of them i can see here!). Please find "station-11-fd.log" file attached >>> > with >>> > output for 1 liquidsoap process. >>> > >>> > One suggestion: as liquidsoap puts all files in /tmp folder, i just >>> > checked >>> > how many files are there. 372 audio files (that are certainly donwloaded >>> > via >>> > liquidsoap). It should be too many files for 15 liquidsoap processes >>> > working >>> > at one time. >>> > >>> > >>> >> Just to be sure, can you reproduce the issue without cue_cut? >>> > Of course, i have disabled it and restarted all stations, i will >>> > continue >>> > reporting if liquidsoap processes will die again. >>> > >>> > >>> > Hope this information helps. Thanks for your response! >>> > >>> > >>> > 2014-01-29 David Baelde <david.bae...@gmail.com> >>> >> >>> >> Hi, >>> >> >>> >> No clue so far, so I'll ask a few questions instead. >>> >> >>> >> * Are you using the standard resolver for http, that is wget? >>> >> >>> >> * Do you have a reasonable number of alive requests? You can see this >>> >> using the "request.alive" server command. On the server you can also >>> >> perform "request.trace" to see what happened to a request. >>> >> >>> >> * Can you identify the too many files in "ls -l /proc/<liq pid>/fd"? >>> >> If they are the downloaded files, you should see temporary file names >>> >> starting with "liq". It would be useful to check if these files are >>> >> all the downloaded files so far or only some. >>> >> >>> >> * Just to be sure, can you reproduce the issue without cue_cut? >>> >> >>> >> Thanks, >>> >> >>> >> David >>> >> >>> >> >>> >> >>> >> ------------------------------------------------------------------------------ >>> >> WatchGuard Dimension instantly turns raw network data into actionable >>> >> security intelligence. It gives you real-time visual feedback on key >>> >> security issues and trends. Skip the complicated setup - simply import >>> >> a virtual appliance and go from zero to informed in seconds. >>> >> >>> >> >>> >> http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk >>> >> _______________________________________________ >>> >> Savonet-users mailing list >>> >> Savonet-users@lists.sourceforge.net >>> >> https://lists.sourceforge.net/lists/listinfo/savonet-users >>> > >>> > >>> > >>> > >>> > -- >>> > Kind regards, >>> > Alexander. >>> >>> >>> >>> -- >>> David >> >> >> >> >> -- >> Kind regards, >> Alexander. >> >> ------------------------------------------------------------------------------ >> WatchGuard Dimension instantly turns raw network data into actionable >> security intelligence. It gives you real-time visual feedback on key >> security issues and trends. Skip the complicated setup - simply import >> a virtual appliance and go from zero to informed in seconds. >> http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk >> _______________________________________________ >> Savonet-users mailing list >> Savonet-users@lists.sourceforge.net >> https://lists.sourceforge.net/lists/listinfo/savonet-users >> -- David ------------------------------------------------------------------------------ WatchGuard Dimension instantly turns raw network data into actionable security intelligence. It gives you real-time visual feedback on key security issues and trends. Skip the complicated setup - simply import a virtual appliance and go from zero to informed in seconds. http://pubads.g.doubleclick.net/gampad/clk?id=123612991&iu=/4140/ostg.clktrk _______________________________________________ Savonet-users mailing list Savonet-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/savonet-users
