Thanks for the feedback, but my problem cannot be solved by creative python code, because in order to import and use those modules my real time C++ audio threads would have to still use the same python GIL.
On Wed, Jun 17, 2009 at 6:08 PM, Loki Davison<[email protected]> wrote: > On Thu, Jun 18, 2009 at 11:16 AM, Patrick > Stinson<[email protected]> wrote: >> I wrote a scripting engine for a pro audio plugin by embedding a >> CPython interpreter. Since audio sequencing hosts use separate threads >> for each audio track, loading more than one instance of my plugin >> while running scripting code causes contention on Python's GIl and >> results in CPU spikes at low latencies. >> >> While CPython is more than capable of running fast enough in the audio >> thread for control-rate (MIDI) work, the GIL is killing us. Using >> process migration to move calls to CPython to daemon processes would >> take less code than forking python itself, but the scripting engine >> includes a python extension module that exposes pointers to the C++ >> classes in the audio engine. This complicates things quite a bit. The >> calls look like this: >> >> size = engine.getAudioBufferSize() >> engine.loadPatchFile(SOME_PATH) >> instrument.loadAudioFile(SOME_PATH_2) >> instrument.setVolume(.5) >> >> ...where 'engine' and 'instrument' are C++ classes in the audio engine >> that I wrapped in a python extension. >> >> I think that the biggest problem for me is tackling the complexity of >> managing new real time daemon processes for each audio track, finding >> an IPC method, and also implementing a middle-ware layer that would >> allow those synchronous calls to the engine be made back to the host. >> This is the sort of overhead that one can expect when trying to use >> process migration to work around the GIL. I consider myself an >> experienced coder, but just thinking about finding a clean, rock-solid >> daemon management and IPC mechanism that works on Mac and Windows >> freaks me out. >> >> Does anyone have any comments on this topic? >> >> cheers, >> -P >> _______________________________________________ >> Linux-audio-dev mailing list >> [email protected] >> http://lists.linuxaudio.org/mailman/listinfo/linux-audio-dev >> > > My previous experience of getting around the GIL (not for audio stuff, > networking related) the multiprocessing module in python 2.6, or > pyprocessing before that is great. I had very good performance > improvements and the effort wasn't that extensive. The IPC / > management mechanisms included where enough for my needs. They are > really easy to use. For my problem performance basically scaled > linearly by number of cpus using the module. On 2-8 core machines > running many mostly io bound processes. Have you looked at stackless > as well? pyprocessing is really easy though if it works. Feel free to > give me a yell with any issues. > > Loki > I am having an awesome day ;) > _______________________________________________ Linux-audio-dev mailing list [email protected] http://lists.linuxaudio.org/mailman/listinfo/linux-audio-dev
