On Thu, Jun 18, 2009 at 11:16 AM, Patrick Stinson<[email protected]> wrote: > I wrote a scripting engine for a pro audio plugin by embedding a > CPython interpreter. Since audio sequencing hosts use separate threads > for each audio track, loading more than one instance of my plugin > while running scripting code causes contention on Python's GIl and > results in CPU spikes at low latencies. > > While CPython is more than capable of running fast enough in the audio > thread for control-rate (MIDI) work, the GIL is killing us. Using > process migration to move calls to CPython to daemon processes would > take less code than forking python itself, but the scripting engine > includes a python extension module that exposes pointers to the C++ > classes in the audio engine. This complicates things quite a bit. The > calls look like this: > > size = engine.getAudioBufferSize() > engine.loadPatchFile(SOME_PATH) > instrument.loadAudioFile(SOME_PATH_2) > instrument.setVolume(.5) > > ...where 'engine' and 'instrument' are C++ classes in the audio engine > that I wrapped in a python extension. > > I think that the biggest problem for me is tackling the complexity of > managing new real time daemon processes for each audio track, finding > an IPC method, and also implementing a middle-ware layer that would > allow those synchronous calls to the engine be made back to the host. > This is the sort of overhead that one can expect when trying to use > process migration to work around the GIL. I consider myself an > experienced coder, but just thinking about finding a clean, rock-solid > daemon management and IPC mechanism that works on Mac and Windows > freaks me out. > > Does anyone have any comments on this topic? > > cheers, > -P > _______________________________________________ > Linux-audio-dev mailing list > [email protected] > http://lists.linuxaudio.org/mailman/listinfo/linux-audio-dev >
My previous experience of getting around the GIL (not for audio stuff, networking related) the multiprocessing module in python 2.6, or pyprocessing before that is great. I had very good performance improvements and the effort wasn't that extensive. The IPC / management mechanisms included where enough for my needs. They are really easy to use. For my problem performance basically scaled linearly by number of cpus using the module. On 2-8 core machines running many mostly io bound processes. Have you looked at stackless as well? pyprocessing is really easy though if it works. Feel free to give me a yell with any issues. Loki I am having an awesome day ;) _______________________________________________ Linux-audio-dev mailing list [email protected] http://lists.linuxaudio.org/mailman/listinfo/linux-audio-dev
