Accessing Audio Samples in Cocoa
I am interested in doing some signal processing on the audio channels of a QuickTime Movie. I can retrieve the buffer using MovieAudioExtractionFillBuffer, but the available examples specify the buffer as type Byte*. I am interested in decomposing the buffer into the left and right channel data sample arrays. I assume that these are 16 bit audio samples, but I have yet to find a method that returns the two separate (left and right) arrays or the appropriate Cocoa data type to use with such sample data. Any ideas or examples? Thanks, -- Joseph Ayers, Professor Department of Biology and Marine Science Center Northeastern University East Point, Nahant, MA 01908 Phone (781) 581-7370 x309(office), x335(lab) Cellular (617) 755-7523, FAX: (781) 581-6076 Boston Office 444RI, (617) 373-4044 eMail: [EMAIL PROTECTED] http://www.neurotechnology.neu.edu/ ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Accessing Audio Samples in Cocoa
Le 25 août 08 à 16:02, Joseph Ayers a écrit : I am interested in doing some signal processing on the audio channels of a QuickTime Movie. I can retrieve the buffer using MovieAudioExtractionFillBuffer, but the available examples specify the buffer as type Byte*. I am interested in decomposing the buffer into the left and right channel data sample arrays. I assume that these are 16 bit audio samples, Assumptions are evil. It should be either documented as the default value, or you have to force it to this format. Personnaly, I use the second choice. but I have yet to find a method that returns the two separate (left and right) arrays or the appropriate Cocoa data type to use with such sample data. Any ideas or examples? Thanks, AFAK, there is no Cocoa API to do audio processing. You can have stereo channels by setting the appropriate output format and channel layout format on your session: MovieAudioExtractionSetProperty(ay_session, kQTPropertyClass_MovieAudioExtraction_Audio, kQTMovieAudioExtractionAudioPropertyID_AudioStreamBasicDescription, sizeof(*aFormat), aFormat); MovieAudioExtractionSetProperty(ay_session, kQTPropertyClass_MovieAudioExtraction_Audio, kQTMovieAudioExtractionAudioPropertyID_AudioChannelLayout, bytes, aLayout); Your format should have two channels: AudioStreamBasicDescription inasbd; inasbd.mChannelsPerFrame = 2; // stereo /* 16 bits linear PCM interleaved */ inasbd.mBitsPerChannel = 16; inasbd.mFormatID = kAudioFormatLinearPCM; inasbd.mFormatFlags = kLinearPCMFormatFlagIsSignedInteger; #if TARGET_RT_BIG_ENDIAN inasbd.mFormatFlags |= kAudioFormatFlagIsBigEndian; #else inasbd.mFormatFlags = ~kAudioFormatFlagIsBigEndian; #endif /* 1 frame = 1 packet when using PCM */ inasbd.mFramesPerPacket = inasbd.mChannelsPerFrame; inasbd.mBytesPerFrame = inasbd.mBytesPerPacket = 2 * inasbd.mChannelsPerFrame; inasbd.mSampleRate = 44100; // choose whatever you want, or retreive the default value from your session. and your layout format should describe a stereo layout: AudioChannelLayout layout; layout.mChannelLayoutTag = kAudioChannelLayoutTag_Stereo; Your output buffer will contains interleaved audio data, that is one integer for the first channel, one integer for the second channel, one for the first, … Note that you have to set the output format before setting the layout format. I think it will failed if you do it otherwise. ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Accessing Audio Samples in Cocoa
Hi Jean-Daniel: My issue is with the structure of the actual buffer and how to access the individual audio data samples. For example, is each data sample a UInt16 and can one count on the first sample pointed at by the buffer pointer being the left channel, the second sample the right channel and the next sample being the subsequent left sample. Thanks, Joseph Jean-Daniel Dupas wrote: Le 25 août 08 à 16:02, Joseph Ayers a écrit : I am interested in doing some signal processing on the audio channels of a QuickTime Movie. I can retrieve the buffer using MovieAudioExtractionFillBuffer, but the available examples specify the buffer as type Byte*. I am interested in decomposing the buffer into the left and right channel data sample arrays. I assume that these are 16 bit audio samples, Assumptions are evil. It should be either documented as the default value, or you have to force it to this format. Personnaly, I use the second choice. but I have yet to find a method that returns the two separate (left and right) arrays or the appropriate Cocoa data type to use with such sample data. Any ideas or examples? Thanks, AFAK, there is no Cocoa API to do audio processing. You can have stereo channels by setting the appropriate output format and channel layout format on your session: MovieAudioExtractionSetProperty(ay_session, kQTPropertyClass_MovieAudioExtraction_Audio, kQTMovieAudioExtractionAudioPropertyID_AudioStreamBasicDescription, sizeof(*aFormat), aFormat); MovieAudioExtractionSetProperty(ay_session, kQTPropertyClass_MovieAudioExtraction_Audio, kQTMovieAudioExtractionAudioPropertyID_AudioChannelLayout, bytes, aLayout); Your format should have two channels: AudioStreamBasicDescription inasbd; inasbd.mChannelsPerFrame = 2; // stereo /* 16 bits linear PCM interleaved */ inasbd.mBitsPerChannel = 16; inasbd.mFormatID = kAudioFormatLinearPCM; inasbd.mFormatFlags = kLinearPCMFormatFlagIsSignedInteger; #if TARGET_RT_BIG_ENDIAN inasbd.mFormatFlags |= kAudioFormatFlagIsBigEndian; #else inasbd.mFormatFlags = ~kAudioFormatFlagIsBigEndian; #endif /* 1 frame = 1 packet when using PCM */ inasbd.mFramesPerPacket = inasbd.mChannelsPerFrame; inasbd.mBytesPerFrame = inasbd.mBytesPerPacket = 2 * inasbd.mChannelsPerFrame; inasbd.mSampleRate = 44100; // choose whatever you want, or retreive the default value from your session. and your layout format should describe a stereo layout: AudioChannelLayout layout; layout.mChannelLayoutTag = kAudioChannelLayoutTag_Stereo; Your output buffer will contains interleaved audio data, that is one integer for the first channel, one integer for the second channel, one for the first, … Note that you have to set the output format before setting the layout format. I think it will failed if you do it otherwise. -- Joseph Ayers, Professor Department of Biology and Marine Science Center Northeastern University East Point, Nahant, MA 01908 Phone (781) 581-7370 x309(office), x335(lab) Cellular (617) 755-7523, FAX: (781) 581-6076 Boston Office 444RI, (617) 373-4044 eMail: [EMAIL PROTECTED] http://www.neurotechnology.neu.edu/ ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
Re: Accessing Audio Samples in Cocoa
That's why you have to set the output format. If you set it to be Interleaved 16 bits Integer, it will be UInt16 interleaved. And if you set the channel layout to kAudioChannelLayoutTag_Stereo, it will be as you expect: first left, second right, third left, etc… You cannot make any asusumption about the output format. You have either to retreive it from the session or set it to what you expect. See http://developer.apple.com/quicktime/audioextraction.html for other examples and explanation about extraction session configuration. Le 25 août 08 à 19:35, Joseph Ayers a écrit : Hi Jean-Daniel: My issue is with the structure of the actual buffer and how to access the individual audio data samples. For example, is each data sample a UInt16 and can one count on the first sample pointed at by the buffer pointer being the left channel, the second sample the right channel and the next sample being the subsequent left sample. Thanks, Joseph Jean-Daniel Dupas wrote: Le 25 août 08 à 16:02, Joseph Ayers a écrit : I am interested in doing some signal processing on the audio channels of a QuickTime Movie. I can retrieve the buffer using MovieAudioExtractionFillBuffer, but the available examples specify the buffer as type Byte*. I am interested in decomposing the buffer into the left and right channel data sample arrays. I assume that these are 16 bit audio samples, Assumptions are evil. It should be either documented as the default value, or you have to force it to this format. Personnaly, I use the second choice. but I have yet to find a method that returns the two separate (left and right) arrays or the appropriate Cocoa data type to use with such sample data. Any ideas or examples? Thanks, AFAK, there is no Cocoa API to do audio processing. You can have stereo channels by setting the appropriate output format and channel layout format on your session: MovieAudioExtractionSetProperty(ay_session, kQTPropertyClass_MovieAudioExtraction_Audio, kQTMovieAudioExtractionAudioPropertyID_AudioStreamBasicDescription, sizeof(*aFormat), aFormat); MovieAudioExtractionSetProperty(ay_session, kQTPropertyClass_MovieAudioExtraction_Audio, kQTMovieAudioExtractionAudioPropertyID_AudioChannelLayout, bytes, aLayout); Your format should have two channels: AudioStreamBasicDescription inasbd; inasbd.mChannelsPerFrame = 2; // stereo /* 16 bits linear PCM interleaved */ inasbd.mBitsPerChannel = 16; inasbd.mFormatID = kAudioFormatLinearPCM; inasbd.mFormatFlags = kLinearPCMFormatFlagIsSignedInteger; #if TARGET_RT_BIG_ENDIAN inasbd.mFormatFlags |= kAudioFormatFlagIsBigEndian; #else inasbd.mFormatFlags = ~kAudioFormatFlagIsBigEndian; #endif /* 1 frame = 1 packet when using PCM */ inasbd.mFramesPerPacket = inasbd.mChannelsPerFrame; inasbd.mBytesPerFrame = inasbd.mBytesPerPacket = 2 * inasbd.mChannelsPerFrame; inasbd.mSampleRate = 44100; // choose whatever you want, or retreive the default value from your session. and your layout format should describe a stereo layout: AudioChannelLayout layout; layout.mChannelLayoutTag = kAudioChannelLayoutTag_Stereo; Your output buffer will contains interleaved audio data, that is one integer for the first channel, one integer for the second channel, one for the first, … Note that you have to set the output format before setting the layout format. I think it will failed if you do it otherwise. -- Joseph Ayers, Professor Department of Biology and Marine Science Center Northeastern University East Point, Nahant, MA 01908 Phone (781) 581-7370 x309(office), x335(lab) Cellular (617) 755-7523, FAX: (781) 581-6076 Boston Office 444RI, (617) 373-4044 eMail: [EMAIL PROTECTED] http://www.neurotechnology.neu.edu/ ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/devlists%40shadowlab.org This email sent to [EMAIL PROTECTED] ___ Cocoa-dev mailing list (Cocoa-dev@lists.apple.com) Please do not post admin requests or moderator comments to the list. Contact the moderators at cocoa-dev-admins(at)lists.apple.com Help/Unsubscribe/Update your Subscription: http://lists.apple.com/mailman/options/cocoa-dev/archive%40mail-archive.com This email sent to [EMAIL PROTECTED]
