Hi Martin, Thanks for the suggestion, but unfortunately it’s not applicable in this case.
Firstly, when creating an Audio Queue processing tap, you have no control over the maximum number of frames per slice; rather, AudioQueueProcessingTapNew determines that for you based on the AudioQueue’s configuration (see the „outMaxFrames” parameter below): > extern OSStatus > AudioQueueProcessingTapNew( AudioQueueRef inAQ, > AudioQueueProcessingTapCallback > inCallback, > void * __nullable > inClientData, > AudioQueueProcessingTapFlags inFlags, > UInt32 * > outMaxFrames, // <——— here > AudioStreamBasicDescription * > outProcessingFormat, > AudioQueueProcessingTapRef __nullable * > __nonnull outAQTap) Source: AudioToolbox/AudioQueue.h Secondly, AudioQueueProcessingTapNew is already returning 4096 as the max frames per slice, regardless of the sample format of the Audio Queue… I’ve spent last night fiddling with this issue, and came up with an hypothesis of what’s going on: 1. The documentation for "kAudioUnitProperty_MaximumFramesPerSlice" (and QA1606) says that upon locking the screen, iOS will increase the I/O buffer size to 4096 samples. 2. The Audio Queue is not necessarily operating at the same frequency as the audio hardware, in which case a sample rate conversion has to take place before passing on the audio to the hardware. E.g. you can create an AudioQueue configured with AudioStreamBasicDescription.mSampleRate = 96khz, but the hardware will be operating only at 48khz. 3. Even if such a sample rate conversion is taking place, the Audio Queue processing tap receives the data _before_ the sample rate conversion. We can verify this by checking the AudioStreamBasicDescription returned from AudioQueueProcessingTapNew as "outProcessingFormat". In this ASBD, the sample rate seems to be always the same as the sample rate of the data fed into the AudioQueue, regardless of the actual hardware sample rate. This is confirmed by the header documentation for AudioQueueProcessingTapNew too: > @param outProcessingFormat > The format in which the client will receive the audio > data to be processed. > This will always be the same sample rate as the client > format and usually > the same number of channels as the client format of the > audio queue. 4. This means that if the audio hardware is operating at a _lower_ rate than the AudioQueue, then 4096 samples at the hardware side correspond to _more_ than 4096 samples on the producer side! E.g. if the hardware is operating at 44.1khz, but the AudioQueue is fed with 48khz material, then whenever the hardware is asking for 4096 samples the Audio Queue must produce 4096 * 48000 / 44100 = 4458 pre-conversion samples. 5. The fact that the processing tap callback only ever gets called with 4096 samples means that we're gonna be short of a few hundred samples, which explains why the audio is skipping. If my hypothesis is true, then the problem would become worse as the downsampling ratio increases. And indeed, if I try playing a 96khz file, then the audio becomes completely silent when the screen is locked, and even with the built-in speakers, not just with headphones! One might also ask why 48khz audio doesn't cause issues with the built-in speaker: my suspicion is that the DAC for the built-in speakers is running at 48khz, while the DAC in the lightning adapter is running at 44.1khz. We can also calculate the sample rate at which the default (when the screen is on) I/O buffer of 1024 exceeds the upstream max frames per slice of 4096. Let's say the hardware is running at 44.1khz, then 1024 samples at the hardware side will exceed 4096 max frames per slice for sample rates beyond 4096 / (1024 / 44100) = 176.4khz. Indeed, a 192khz file plays smoothly with the built-in speakers (they run at 48khz) _as long as the screen is on_, but through lightning headphones the audio stutters even if the screen is on! It would be nice if someone from Apple could chime in, because it seems to me that this is a bug in the Audio Queue Services… The way the Audio Queue calculates the maximum frames per slice for its processing tap doesn't take into account the downsampling ratio… It's a nasty bug, because in order to fix it the Audio Queue would have to take into account the _lowest_ possible hardware rate, as it can change while the queue is running (e.g. plugging/unplugging headphones). E.g. if the hardware is capable of running at 8khz, and the Audio Queue is consuming 192khz audio, then the maximum frames per slice should be 4096 * 192000 / 8000 = 98304. Also, it only occurs when using processing taps - the "happy path" (i.e. no processing taps) plays everything smoothly, regardless of sample rate (I've tried up to 192khz). Best regards, Tamás Zahola > On 2018. Oct 8., at 13:12 , Martin Man <[email protected]> wrote: > > Hi Tomas, > > Never tried with the tap, but make sure to properly configure the maximum > number of frames per slice to support any audio stuff when screen is locked. > > Here is the excerpt from the docs, google for the property to find more > >> By default, the kAudioUnitProperty_MaximumFramesPerSlice property is set to >> a value of 1024, which is not sufficient when the screen locks and the >> display sleeps. If your app plays audio with the screen locked, you must >> increase the value of this property unless audio input is active. Do as >> follows: >> >> • If audio input is active, you do not need to set a value for the >> kAudioUnitProperty_MaximumFramesPerSlice property. >> >> • If audio input is not active, set this property to a value of 4096. > > Hope it helps, > Martin > >> On 7 Oct 2018, at 22:51, Tamás Zahola <[email protected] >> <mailto:[email protected]>> wrote: >> >> Hi, >> >> I have an iOS audio player app built on Audio Queue Services [1], that >> incorporates an equalizer using the n-band EQ from AudioToolbox >> (kAudioUnitSubType_NBandEQ). >> >> I’ve received complaints about playback starting to stutter for certain >> files when the device is locked (or if the screen turns off automatically). >> First I thought the stuttering was caused by doing something inappropriate >> in the processing tap’s real-time callback, but I couldn’t find anything >> suspicious… Also, I’ve found that the stuttering/skipping occurs ONLY if the >> following hold: >> >> - the AudioQueue is configured with a sample rate of 48khz >> - the audio is playing through headphones >> - the screen is off/locked >> >> After further experiments, it turned out that it doesn’t matter what I do in >> the processing tap’s callback, the problem persists even if the processing >> tap is the „empty": >> >> static void ProcessingTapCallback(void * inClientData, >> AudioQueueProcessingTapRef inAQTap, >> UInt32 inNumberFrames, >> AudioTimeStamp * ioTimeStamp, >> AudioQueueProcessingTapFlags * ioFlags, >> UInt32 * outNumberFrames, >> AudioBufferList * ioData) { >> OSStatus status = AudioQueueProcessingTapGetSourceAudio(inAQTap, >> inNumberFrames, ioTimeStamp, ioFlags, outNumberFrames, ioData); >> NSCAssert(status == noErr && inNumberFrames == *outNumberFrames, @"%d vs >> %d", (int)inNumberFrames, (int)*outNumberFrames); >> } >> >> The assert on the last line never fails, so the buffer should always be >> completely filled. (inNumberFrames is 4096 by the way) >> >> I’ve found an old QA writeup [2] that discusses AUGraph behaviour when the >> screen is locked: the system increases the "maximum frames per slice” >> property to 88ms (4096 samples at 44.1khz). However, the maximum frames per >> slice returned by AudioQueueProcessingTapNew is already 4096, so there >> should be no mismatch here… >> >> I’ve created a minimal demonstration project here: >> https://github.com/tzahola/AudioQueueProcessingTapSample >> <https://github.com/tzahola/AudioQueueProcessingTapSample> >> By default it plays a 48khz AIFF file, which will stutter if the screen is >> off and you're using headphones. If you unlock the device, or unplug the >> headphones, or change „piano_48k” to „piano_44.1k” (in ViewController.mm), >> then there will be no stutter… (needless to say, it works perfectly without >> the processing tap, too) >> >> Can anybody shed some light on why this might be happening, and how to >> circumvent it? The documentation on Audio Queue processing taps is a bit >> scarce... >> >> [1] >> https://developer.apple.com/documentation/audiotoolbox/audio_queue_services >> <https://developer.apple.com/documentation/audiotoolbox/audio_queue_services> >> [2] https://developer.apple.com/library/archive/qa/qa1606/_index.html >> <https://developer.apple.com/library/archive/qa/qa1606/_index.html> >> >> Best regards, >> Tamás Zahola >> _______________________________________________ >> Do not post admin requests to the list. They will be ignored. >> Coreaudio-api mailing list ([email protected] >> <mailto:[email protected]>) >> Help/Unsubscribe/Update your Subscription: >> https://lists.apple.com/mailman/options/coreaudio-api/mman%40martinman.net >> <https://lists.apple.com/mailman/options/coreaudio-api/mman%40martinman.net> >> >> This email sent to [email protected] >
_______________________________________________ Do not post admin requests to the list. They will be ignored. Coreaudio-api mailing list ([email protected]) Help/Unsubscribe/Update your Subscription: https://lists.apple.com/mailman/options/coreaudio-api/archive%40mail-archive.com This email sent to [email protected]
