As far as I can see there's no documentation on AudioServerPlugIn besides the
headers, and the comments on the various mix operations are lacking. I know
this is a lot to read, but hopefully my questions and the answers to them will
help others in the future too.
I have a couple of questions near the end. Please correct me if I'm wrong at
any point.
~~~~
When talking about the operations which get audio from clients *into* my
device, the relevant operations are kAudioServerPlugInIOOperationMixOutput,
ProcessMix, ConvertMix, and WriteMix.
MixOutput works just as I thought it would. The main buffer in MixOutput has
the audio (in the canonical format correct? - despite not explicitly saying so)
for a single client only, requiring me to copy the audio out of that buffer,
and mix it into my device's ring buffer, converting if necessary. Using this
operation, if I only handle IO operations coming from a single particular
client, then I get a clean audio signal from just that one client.
The one thing that's confusing is why does it say "this operation always
happens in-place" if the entire point of the operation is to copy the audio out
of the main buffer and into my device? My understanding of "in-place" means
that some audio is put *into* the main buffer, and yet I should only be copying
*out* of it. I suppose it's simply trying to convey that the seconday buffer is
unnecessary?
~~~~
Now, for the other operations: ProcessMix, ConvertMix, and WriteMix. It seems
to me that these can only used if you don't say you do MixOutput (in
WillDoIOOperation). Without using MixOutput, WriteMix is required, but
ProcessMix and ConvertMix are both optional?
My understanding of how they work together is that the output of ProcessMix is
handed into ConvertMix, and the output of ConvertMix is handed into WriteMix,
as such:
ProcessMix:
Input: The main buffer contains in canonical format, a mix of
all clients' audio.
Output: processed (volume, gain, filters, whatever?) mix of all
clients' audio, into either the main buffer or secondary (depending on WillDo's
return value for "in-place"), canonical format
ConvertMix:
Input: the output of the ProcessMix operation, in the main
buffer, still canonical format.
Output: Simply converted input into the device's native format.
Put into either the main buffer or secondary buffer, (again depending on
WillDo's in-place value)
WriteMix:
Input: The output of ConvertMix, which is the fully mixed,
device-native format audio.
Output: none. (Copying into my device's ring buffer.)
Only WriteMix is required (according to the header), so I imagine it's possible
that I could not implement/support ConvertMix, and Core Audio would do the
conversion for me, but still let me use ProcessMix and WriteMix?
~~~~
Now, assuming the above is all correct, then I'm really confused about
WriteMix's behavior because according to the above (based on my interpretation
of the header comments), during a WriteMix operation, the main buffer passed to
DoIOOperation contains the fully mixed audio from all of the various clients.
So if there are two applications sending audio to my device, I expect to get
once DoIOOperation/WriteMix call per sample time range, where the main buffer
contains the mix of the two apps' audio, in my device's native format.
Instead, DoIOOperation/WriteMix is called *for each client*, and they have
overlapping sample time ranges. For example (sample times are on the end:)
WriteMix: Client 205 -- 2091040.000000
WriteMix: Client 348 -- 2091500.000000
WriteMix: Client 205 -- 2091552.000000
WriteMix: Client 348 -- 2092012.000000
Notice that they completely overlap, so that I always have two WriteMix calls
for each sample. This doesn't make any sense to me. I'd expect just one, and
the client id for the operation to be 0 since the buffer contains a full mix of
multiple clients.
What's really odd is if I only copy the audio from the WriteMix operations that
are "from" just one of these clients (either one), it sounds garbled and
terrible. If I copy the audio from both (despite the sample times completely
overlapping), it sounds correct.
And a second thing, why when only recording from my device (which is both an
input and an output device) does WriteMix happen with the client id matching
that of the client which is recording from my device (aka ReadInput is
happening)? (The main io buffer during these WriteMix calls is all zeros.)
~~~~
I know that's a lot to digest. Hopefully someone can help me out. :-)
--
Seth Willits
--
Seth Willits
_______________________________________________
Do not post admin requests to the list. They will be ignored.
Coreaudio-api mailing list ([email protected])
Help/Unsubscribe/Update your Subscription:
https://lists.apple.com/mailman/options/coreaudio-api/archive%40mail-archive.com
This email sent to [email protected]
