Hi Marty,
Thanks for pointing this out. I'm gonna be exploring the poll() function
instead of select() to remedy this situation in future releases.
Romain
2017-11-25 21:14 GMT-06:00 Marty Gauvin <[email protected]>:
> This one has been resolved. A few points.
>
>
>
> 1. Increasing the number of file descriptors above 1024 doesn’t
> help. Select can’t use them.
>
> 2. Solved the problem by splitting the work into a number of
> different scripts each running in a separate liquidsoap process. Each
> process can run about 60 input.harbor and 30 output.icecast before things
> start to break.
>
>
>
>
>
> *From:* Marty Gauvin [mailto:[email protected]]
> *Sent:* Friday, 17 November 2017 9:17 PM
> *To:* [email protected]
> *Subject:* [Savonet-users] liquidsoap 1.3.3 crashs with 'exception
> invalid argument in select()
>
>
>
> I’m running in an environment where I need to have lots of harbor
> endpoints.
>
>
>
> I have found that I get this error when I go above a certain number (about
> 150 or so).
>
>
>
> The script is
>
> # =========================================
>
> # SCRIPT CONSTANTS
>
> # =========================================
>
> # Load in the env variables including ICE_HOST, ICE_PASS, home dir, etc
>
> %include ".env"
>
>
>
> # =========================================
>
> # RUNTIME SETTINGS
>
> # http://savonet.sourceforge.net/doc-svn/in_production.html
>
> # =========================================
>
> # Harbor settings
>
> set("harbor.verbose",true) # make sure Harbor shows EVERYTHING
>
> set("harbor.bind_addr", HARBOR_IP) # bind Harbor to nominated IP
>
> set("harbor.reverse_dns",false) # tweak to improve network slightly
>
>
>
> # Setup logging - default is /var/log/liquidsoap/<script.log>
>
> set("log.file.path","/var/log/liquidsoap/<script>.log")
>
> set("log.file",true)
>
> set("log.file.append",true) # don't just overwrite
>
> set("log.level",LOG_LEVEL)
>
> set("log.stdout",true) # show in console, too
>
> set("log.unix_timestamps",false) # show UNIX timestamps instead
>
> set("init.daemon.pidfile.path","/var/run/liquidsoap")
>
>
>
> # Setup decoder order
>
> set("decoder.file_decoders",["MP3","OGG","AAC","MP4","META","WAV","MIDI","FLAC"])
> # set order according to likelihood
>
> set("decoder.stream_decoders",["MP3","OGG","AAC","WAV","PCM/BASIC","FLAC"])
> # set order according to likelihood
>
>
>
> # =========================================
>
> # OUTPUT APPLICATIONS
>
> # =========================================
>
> # Build the output control - Vorbis format by default
>
> #out = output.icecast(%vorbis(samplerate=44100, channels=2, quality=0),
>
> out = output.icecast(%mp3.vbr(samplerate=44100,quality=6),
>
> host=ICE_HOST,
>
> port=ICE_PORT,
>
> password=ICE_PASS,
>
> fallible=true,
>
> icy_metadata="true")
>
>
>
> # =========================================
>
> # INCLUDES
>
> # =========================================
>
> # Load all the custom mount point includes - these need to be loaded last
>
> %include "macmount.inc"
>
> %include "mounts.inc"
>
>
>
> The includes are:
>
> *.env*
>
> # Liquidsoap home directory
>
> # lshomedir = "/home/liquidsoap/liquidsoap"
>
>
>
> # Icecast host settings
>
> ICE_HOST = "###############" # Icecast host DNS
>
> #ICE_USER = "source" # icecast default user is always "source" anyway
>
> ICE_PASS = "#########" # default Icecast password
>
> ICE_PORT = 7034 # Icecast listen port (not the streaming port)
>
>
>
> # Basic script settings
>
> LOG_LEVEL = 5 # 4 = debug-level logs; 5 = extra
>
>
>
> # Harbor config
>
> HARBOR_IP = "0.0.0.0" # Harbor to listen on all IPs
>
>
>
> *macmount.inc*
>
> def macmount(num,mntname)
>
> rport = num * 2
>
> mport = rport + 7500
>
> vport = rport + 7800
>
> vname=mntname ^ "voice"
>
> harbom = audio_to_stereo(input.harbor(mntname, icy=true, port=mport,
> password=ICE_PASS, max=20.))
>
> harbov = strip_blank(max_blank=5.,
> mksafe(audio_to_stereo(input.harbor(vname,
> icy=true, port=vport, password=ICE_PASS, max=20.))))
>
> output=out(mount=vname,harbov)
>
> radio = fallback(track_sensitive=false, [harbov, harbom,
> single("media/silence.m4a")])
>
> output=out(mount=mntname,radio)
>
> end
>
>
>
> *mounts.inc*
>
> macmount(1,"2IdxhCR0SLF8ks")
>
> macmount(2,"osmQm9AHTPxHbs")
>
> # further 98 rows following the same pattern
>
>
>
> The log file produced looks like this before the crash:
>
>
>
> liquidsoap@bluejay-up-pr-01:~/liquidsoap$ tail -n 100 log1.log
>
> 2017/11/17 10:28:33 [source:4] Source fallback_7507 gets up.
>
> 2017/11/17 10:28:33 [strip_blank_7504:4] Activations changed:
> static=[apr5eXRHAnjs49voice:apr5eXRHAnjs49voice, strip_blank_7504],
> dynamic=[fallback_7507:apr5eXRHAnjs49:apr5eXRHAnjs49].
>
> 2017/11/17 10:28:33 [source:4] Source audio_to_stereo_7499 gets up.
>
> 2017/11/17 10:28:33 [audio_to_stereo_7499:4] Content kind is
> {audio=2;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [source:4] Source input.harbor_7498 gets up.
>
> 2017/11/17 10:28:33 [input.harbor_7498:4] Content kind is
> {audio=1+;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [harbor:4] Opening port 7536 with icy = false
>
> 2017/11/17 10:28:33 [harbor:4] Opening port 7537 with icy = true
>
> 2017/11/17 10:28:33 [harbor:3] Adding mountpoint '/apr5eXRHAnjs49' on port
> 7536
>
> 2017/11/17 10:28:33 [input(dot)harbor_7498:4] Activations changed:
> static=[audio_to_stereo_7499:fallback_7507:apr5eXRHAnjs49:apr5eXRHAnjs49],
> dynamic=[].
>
> 2017/11/17 10:28:33 [audio_to_stereo_7499:4] Activations changed:
> static=[], dynamic=[fallback_7507:apr5eXRHAnjs49:apr5eXRHAnjs49].
>
> 2017/11/17 10:28:33 [source:4] Source single_7506 gets up.
>
> 2017/11/17 10:28:33 [single_7506:3] "media/silence.m4a" is static,
> resolving once for all...
>
> 2017/11/17 10:28:33 [single_7506:4] Content kind is
> {audio=2;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [single_7506:4] Activations changed: static=[],
> dynamic=[fallback_7507:apr5eXRHAnjs49:apr5eXRHAnjs49].
>
> 2017/11/17 10:28:33 [fallback_7507:4] Activations changed:
> static=[apr5eXRHAnjs49:apr5eXRHAnjs49], dynamic=[].
>
> 2017/11/17 10:28:33 [apr5eXRHAnjs49:4] Activations changed:
> static=[apr5eXRHAnjs49], dynamic=[].
>
> 2017/11/17 10:28:33 [apr5eXRHAnjs49:4] Enabling caching mode: active
> source.
>
> 2017/11/17 10:28:33 [source:4] Source strip_blank_7493 gets up.
>
> 2017/11/17 10:28:33 [strip_blank_7493:4] Content kind is
> {audio=2;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [source:4] Source mksafe gets up.
>
> 2017/11/17 10:28:33 [source:4] Source audio_to_stereo_7490 gets up.
>
> 2017/11/17 10:28:33 [audio_to_stereo_7490:4] Content kind is
> {audio=2;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [source:4] Source input.harbor_7489 gets up.
>
> 2017/11/17 10:28:33 [input.harbor_7489:4] Content kind is
> {audio=1+;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [harbor:4] Opening port 7834 with icy = false
>
> 2017/11/17 10:28:33 [harbor:4] Opening port 7835 with icy = true
>
> 2017/11/17 10:28:33 [harbor:3] Adding mountpoint '/nxYjWRxEteOgunvoice' on
> port 7834
>
> 2017/11/17 10:28:33 [input(dot)harbor_7489:4] Activations changed:
> static=[audio_to_stereo_7490:mksafe:strip_blank_7493:strip_blank_7493],
> dynamic=[].
>
> 2017/11/17 10:28:33 [audio_to_stereo_7490:4] Activations changed:
> static=[], dynamic=[mksafe:strip_blank_7493:strip_blank_7493].
>
> 2017/11/17 10:28:33 [source:4] Source safe_blank gets up.
>
> 2017/11/17 10:28:33 [safe_blank:4] Content kind is
> {audio=2;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [safe_blank:4] Activations changed: static=[],
> dynamic=[mksafe:strip_blank_7493:strip_blank_7493].
>
> 2017/11/17 10:28:33 [mksafe:4] Activations changed:
> static=[strip_blank_7493:strip_blank_7493], dynamic=[].
>
> 2017/11/17 10:28:33 [strip_blank_7493:4] Activations changed:
> static=[strip_blank_7493], dynamic=[].
>
> 2017/11/17 10:28:33 [strip_blank_7493:4] Enabling caching mode: active
> source.
>
> 2017/11/17 10:28:33 [source:4] Source output.icecast_7494 gets up.
>
> 2017/11/17 10:28:33 [strip_blank_7493:4] Activations changed:
> static=[nxYjWRxEteOgunvoice:nxYjWRxEteOgunvoice, strip_blank_7493],
> dynamic=[].
>
> 2017/11/17 10:28:33 [nxYjWRxEteOgunvoice:4] Activations changed:
> static=[nxYjWRxEteOgunvoice], dynamic=[].
>
> 2017/11/17 10:28:33 [nxYjWRxEteOgunvoice:4] Enabling caching mode: active
> source.
>
> 2017/11/17 10:28:33 [source:4] Source output.icecast_7497 gets up.
>
> 2017/11/17 10:28:33 [source:4] Source fallback_7496 gets up.
>
> 2017/11/17 10:28:33 [strip_blank_7493:4] Activations changed:
> static=[nxYjWRxEteOgunvoice:nxYjWRxEteOgunvoice, strip_blank_7493],
> dynamic=[fallback_7496:nxYjWRxEteOgun:nxYjWRxEteOgun].
>
> 2017/11/17 10:28:33 [source:4] Source audio_to_stereo_7488 gets up.
>
> 2017/11/17 10:28:33 [audio_to_stereo_7488:4] Content kind is
> {audio=2;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [source:4] Source input.harbor_7487 gets up.
>
> 2017/11/17 10:28:33 [input.harbor_7487:4] Content kind is
> {audio=1+;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [harbor:4] Opening port 7534 with icy = false
>
> 2017/11/17 10:28:33 [harbor:4] Opening port 7535 with icy = true
>
> 2017/11/17 10:28:33 [harbor:3] Adding mountpoint '/nxYjWRxEteOgun' on port
> 7534
>
> 2017/11/17 10:28:33 [input(dot)harbor_7487:4] Activations changed:
> static=[audio_to_stereo_7488:fallback_7496:nxYjWRxEteOgun:nxYjWRxEteOgun],
> dynamic=[].
>
> 2017/11/17 10:28:33 [audio_to_stereo_7488:4] Activations changed:
> static=[], dynamic=[fallback_7496:nxYjWRxEteOgun:nxYjWRxEteOgun].
>
> 2017/11/17 10:28:33 [source:4] Source single_7495 gets up.
>
> 2017/11/17 10:28:33 [single_7495:3] "media/silence.m4a" is static,
> resolving once for all...
>
> 2017/11/17 10:28:33 [single_7495:4] Content kind is
> {audio=2;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [single_7495:4] Activations changed: static=[],
> dynamic=[fallback_7496:nxYjWRxEteOgun:nxYjWRxEteOgun].
>
> 2017/11/17 10:28:33 [fallback_7496:4] Activations changed:
> static=[nxYjWRxEteOgun:nxYjWRxEteOgun], dynamic=[].
>
> 2017/11/17 10:28:33 [nxYjWRxEteOgun:4] Activations changed:
> static=[nxYjWRxEteOgun], dynamic=[].
>
> 2017/11/17 10:28:33 [nxYjWRxEteOgun:4] Enabling caching mode: active
> source.
>
> 2017/11/17 10:28:33 [source:4] Source strip_blank_7482 gets up.
>
> 2017/11/17 10:28:33 [strip_blank_7482:4] Content kind is
> {audio=2;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [source:4] Source mksafe gets up.
>
> 2017/11/17 10:28:33 [source:4] Source audio_to_stereo_7479 gets up.
>
> 2017/11/17 10:28:33 [audio_to_stereo_7479:4] Content kind is
> {audio=2;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [source:4] Source input.harbor_7478 gets up.
>
> 2017/11/17 10:28:33 [input.harbor_7478:4] Content kind is
> {audio=1+;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [harbor:4] Opening port 7832 with icy = false
>
> 2017/11/17 10:28:33 [harbor:4] Opening port 7833 with icy = true
>
> 2017/11/17 10:28:33 [harbor:3] Adding mountpoint '/vqyXzbCzVjHLrcvoice' on
> port 7832
>
> 2017/11/17 10:28:33 [input(dot)harbor_7478:4] Activations changed:
> static=[audio_to_stereo_7479:mksafe:strip_blank_7482:strip_blank_7482],
> dynamic=[].
>
> 2017/11/17 10:28:33 [audio_to_stereo_7479:4] Activations changed:
> static=[], dynamic=[mksafe:strip_blank_7482:strip_blank_7482].
>
> 2017/11/17 10:28:33 [source:4] Source safe_blank gets up.
>
> 2017/11/17 10:28:33 [safe_blank:4] Content kind is
> {audio=2;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [safe_blank:4] Activations changed: static=[],
> dynamic=[mksafe:strip_blank_7482:strip_blank_7482].
>
> 2017/11/17 10:28:33 [mksafe:4] Activations changed:
> static=[strip_blank_7482:strip_blank_7482], dynamic=[].
>
> 2017/11/17 10:28:33 [strip_blank_7482:4] Activations changed:
> static=[strip_blank_7482], dynamic=[].
>
> 2017/11/17 10:28:33 [strip_blank_7482:4] Enabling caching mode: active
> source.
>
> 2017/11/17 10:28:33 [source:4] Source output.icecast_7483 gets up.
>
> 2017/11/17 10:28:33 [strip_blank_7482:4] Activations changed:
> static=[vqyXzbCzVjHLrcvoice:vqyXzbCzVjHLrcvoice, strip_blank_7482],
> dynamic=[].
>
> 2017/11/17 10:28:33 [vqyXzbCzVjHLrcvoice:4] Activations changed:
> static=[vqyXzbCzVjHLrcvoice], dynamic=[].
>
> 2017/11/17 10:28:33 [vqyXzbCzVjHLrcvoice:4] Enabling caching mode: active
> source.
>
> 2017/11/17 10:28:33 [source:4] Source output.icecast_7486 gets up.
>
> 2017/11/17 10:28:33 [source:4] Source fallback_7485 gets up.
>
> 2017/11/17 10:28:33 [strip_blank_7482:4] Activations changed:
> static=[vqyXzbCzVjHLrcvoice:vqyXzbCzVjHLrcvoice, strip_blank_7482],
> dynamic=[fallback_7485:vqyXzbCzVjHLrc:vqyXzbCzVjHLrc].
>
> 2017/11/17 10:28:33 [source:4] Source audio_to_stereo_7477 gets up.
>
> 2017/11/17 10:28:33 [audio_to_stereo_7477:4] Content kind is
> {audio=2;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [source:4] Source input.harbor_7476 gets up.
>
> 2017/11/17 10:28:33 [input.harbor_7476:4] Content kind is
> {audio=1+;video=0;midi=0}.
>
> 2017/11/17 10:28:33 [harbor:4] Opening port 7532 with icy = false
>
> 2017/11/17 10:28:33 [harbor:4] Opening port 7533 with icy = true
>
> 2017/11/17 10:28:33 [harbor:3] Adding mountpoint '/vqyXzbCzVjHLrc' on port
> 7532
>
> 2017/11/17 10:28:33 [threads:2] Queue generic queue #1 crashed with
> exception Invalid argument in select()
>
> Raised by primitive operation at file "duppy.ml", line 181, characters
> 20-51
>
> Re-raised at file "duppy.ml", line 199, characters 11-18
>
> Called from file "duppy.ml", line 201, characters 4-8
>
> Called from file "duppy.ml", line 295, characters 12-25
>
> Called from file "tools/tutils.ml", line 213, characters 19-55
>
>
>
> 2017/11/17 10:28:33 [threads:1] PANIC: Liquidsoap has crashed, exiting.,
>
> Please report at: [email protected]
>
>
>
> Is this a bug, or am I doing something wrong? I upgraded from liquidsoap
> 1.1.1 but the error I’m getting is the same.
>
>
>
> The server is on DigitalOcean running Ubuntu 16.04.03 LTS.
>
>
>
> The open files limit in ulimit has been increased to 65536.
>
>
>
> [image: cid:[email protected]]
>
>
>
>
>
> *Marty Gauvin*
>
> *President and CEO*
>
>
>
> Office +61 8 72000800 <+61%208%207200%200800>
>
> Mobile +61 418843550 <+61%20418%20843%20550>
>
> email: [email protected]
>
> skype: marty.gauvin.virtualark
>
> web: www.virtualark.com
>
>
>
>
>
> ------------------------------------------------------------
> ------------------
> Check out the vibrant tech community on one of the world's most
> engaging tech sites, Slashdot.org! http://sdm.link/slashdot
> _______________________________________________
> Savonet-users mailing list
> [email protected]
> https://lists.sourceforge.net/lists/listinfo/savonet-users
>
>
------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
Savonet-users mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/savonet-users
