Hi,
o Robert Szokovacs on 07/15/2011 12:50 PM:
Hi,
We came about a situation, where we keep (some of) our audio files on NFS,
which can die. (other filesystems can do that also, so I think this topic is
important to people not using NFS as well)
Here's how I broke down the possible scenarios:
a1,
all your files on the same fs, all your calls use at least one of the files:
thats too bad, nothing SEMS can do for you
a2,
all your files on the same fs, you have calls that don't use files: you're OK,
the signaling is done in separate threads if you don't use threadpools, so
don't, the file-less calls will work in this case.
UNLESS
a2-python,
you're using IVR and python, in which case you'll be suprised to see you're
not receiving any new calls if you have a session blocking in opening the
file: that's because python (and IVR) holds the GIL during file operations. I
have attached a patch that addresses this problem, I hope I covered every
case.
thanks, its applied in master and 1.4.
b,
some of your files on NFS, some are not:
if you're extremely lucky (under reasonable load, you're not), none of the
sessions running when the NFS goes down, so the involved sessions will get
stuck in open(), in this case the non-NFS using sessions will work
undisturbed. But if you have a session getting stuck in read(), the
AmMediaProcessor thread(s) will get stuck, and even the non-NFS sessions will
lose audio. This could be solved if we have more then one AmMediaProcessor
thread, different callgroup to NFS and non-NFS sessions and mechanism to
assign callgroups to a specific AmMediaProcessor thread. The last part is
missing, addSession() adds the callgroup to the thread with the lowest load.
Do you think there would be interest if I implemented this mechanism? Any
yes. The best solution would be to have (optionally) several pools of
threads.
suggestions?
This is the obvious, but don't use NFS:
- cache files in memory on startup/reload (AmCachedAudioFile,
PrompSets for DSM)
- use local fs (which behaves better in case of errors, and they
are less frequent) and synchronize after recording/changing the files
I think that you are spot on in trying to reduce the impact on the
other calls. If the FS blocks, there's not much one can do for the
calls that necessarily need data there, but it could be handled more
gracefully; as at least the read operation can be buffered in
AmBufferedAudio, it could also be possible to fan out the IO to a
different thread (possibly using async io). But I am not sure it's
worth the performance impact in normal operation, especially as with
the above measures you get a very reliable system already.
Stefan
_______________________________________________
Semsdev mailing list
[email protected]
http://lists.iptel.org/mailman/listinfo/semsdev
