> On Mar 11, 2016, at 1:11 PM, Garrett Smith <dhtmlkitc...@gmail.com> wrote: > > > > On Tuesday, March 8, 2016, Jer Noble <jer.no...@apple.com > <mailto:jer.no...@apple.com>> wrote: > > > On Mar 8, 2016, at 4:42 PM, Garrett Smith <dhtmlkitc...@gmail.com <>> wrote: > > > > On Fri, Mar 4, 2016 at 3:43 PM, Jer Noble <jer.no...@apple.com <>> wrote: > >> > >>> On Mar 4, 2016, at 3:19 PM, Garrett Smith <dhtmlkitc...@gmail.com <>> > >>> wrote: > >>> > >>> On Fri, Mar 4, 2016 at 1:55 PM, Jer Noble <jer.no...@apple.com <>> wrote: > >>>> > >>>>> On Mar 1, 2016, at 8:00 PM, Philip Jägenstedt <phil...@opera.com <>> > >>>>> wrote: > >>>>> > >>>>> On Wed, Mar 2, 2016 at 9:19 AM, Garrett Smith <dhtmlkitc...@gmail.com > >>>>> <>> wrote: > >>>>>> On Thu, Nov 12, 2015 at 11:32 AM, Philip Jägenstedt <phil...@opera.com > >>>>>> <>> wrote: > >>>>>>> On Thu, Nov 12, 2015 at 10:55 AM, Garrett Smith > >>>>>>> <dhtmlkitc...@gmail.com <>> > >>>>>>> wrote: > >>>>>>>> On 11/12/15, Philip Jägenstedt <phil...@opera.com <>> wrote: > >>>>>>>>> On Thu, Nov 12, 2015 at 9:07 AM, Garrett Smith > >>>>>>>>> <dhtmlkitc...@gmail.com <>> > >>>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>> On 10/19/15, Philip Jägenstedt <phil...@opera.com <>> wrote: > >>>>>>>>>>> On Tue, Sep 1, 2015 at 11:21 AM, Philip Jägenstedt > >>>>>>>>>>> <phil...@opera.com <>> > >>>>>>>>>>> wrote: > >>>>>>>>>>>> On Mon, Aug 31, 2015 at 9:48 PM, Domenic Denicola > >>>>>>>>>>>> <d...@domenic.me <>> > >>>>>>>>>>>> wrote: > >>>>>>>>>>>>> From: Eric Carlson [mailto:eric.carl...@apple.com <>] > > > >> > >> The Web Audio equivalent would be: > >> > >> var video = document.querySelector(‘video’); > >> video.preservesPitch = false; > >> var context = new AudioContext(); > >> var sourceNode = context.createMediaElementSource(video); > >> var pitchShiftNode = context.createPitchShift(); > >> pitchShiftNode.shiftAmount = 1.023; > >> sourceNode.connect(pitchShiftNode); > >> pitchShiftNode.connect(context.destination); > >> > > > > Which implementations does that work in? > > None, because there is no such thing as a PitchShiftNode. > > > I see. > > > That code is more complex than should be necessary. I see where you're > > coming from on separating the audio. Could we move the media decorator > > behind the scenes, and replace it with a simple getter/setter property > > like `videoElement.audio` so that that can happen automagically? > > Reminds me of createElement, createRange, document.implementation, > > etc. Warts! > > I’m not entirely sure what you’re asking here. If it’s that you don’t like > the `context.createMediaElementSource()` or `context.createPitchShift()` > syntax and would rather a constructor syntax, Issue #250 > <https://github.com/WebAudio/web-audio-api/issues/250 > <https://github.com/WebAudio/web-audio-api/issues/250>> in the Web Audio spec > is the issue for you. > > > But then again, you also just said that there are no APIs on OS X that > > allow an arbitrary pitch shift to be added to audio. If that is true, > > then your `createPitchShift` code would be possible anyway, is that > > There is no such API for such post-processing built into the OS X media > frameworks. > > Oh. Poor hardware integration, and now this…. Being Apple CEO is not Tim > Cook's greatest gift…
I’m not sure how that’s relevant. > As an example, there is an API for preserving pitch across rate changes: > -[AVPlayerItem setAudioTimePitchAlgorithm:] > <https://developer.apple.com/library/prerelease/ios/documentation/AVFoundation/Reference/AVPlayerItem_Class/index.html#//apple_ref/occ/instp/AVPlayerItem/audioTimePitchAlgorithm > > <https://developer.apple.com/library/prerelease/ios/documentation/AVFoundation/Reference/AVPlayerItem_Class/index.html#//apple_ref/occ/instp/AVPlayerItem/audioTimePitchAlgorithm>>. > So implementing the HTMLMediaElement “preservesPitch" property is trivial. > There is no such API for applying a specific and arbitrary amount of pitch > shifting. It would have to be implemented entirely by hand, the same way we > implement processing of media element audio with the Web Audio API. Which > gets back to: at this point, we should just implement a Pitch Shift node in > Web Audio. > > > I see. > > >> var video = document.querySelector(‘video’); > >> var context = new AudioContext(); > >> var sourceNode = context.createMediaElementSource(video); > >> var pitchShiftNode = context.createPitchShift(); > >> pitchShiftNode.shiftAmount = 1.023; > >> sourceNode.connect(pitchShiftNode); > >> pitchShiftNode.connect(context.destination); > >> > > Too many steps and implementation details. It's complicated. It's asking for > more js libraries and wrappers for that, and other situations related to > videos' audio. Make it simpler. Hide the implementation details. > > Hide:— > * AudioContext creation and subsequent video connection. > * createPitchShift — use audio.pitchShift to get a pitchShiftNode > * pitchShiftNode.shiftAmount — make pitchShiftNode a getter/setter (like > video.playbackRate) > > >> var video = document.querySelector(‘video’); > >> var context = new AudioContext(); > >> var sourceNode = context.createMediaElementSource(video); > >> var context = new AudioContext(); > > // Hidden AudioContext construction > var audio = video.audio; > > >> var pitchShiftNode = context.createPitchShift(); > >> pitchShiftNode.shiftAmount = 1.023; > > // Hidden AudioContext createPitchShift factory. > // Setter on pitchShift > audio.pitchShift = 1.023; > > >> sourceNode.connect(pitchShiftNode); > >> pitchShiftNode.connect(context.destination); > >> > > Combined:— > > var video = document.querySelector(‘video’); > video.audio.pitchShift = 1.023; > > The way you have it now is overkill when client doesn't want to care about a > pitch shift node or other implementation details. And in that case, wouldn't > it make sense to add preservesPitch on the audio and deprecate it from the > video? No, because preservePitch requires knowledge from the video element, namely the speed of playback. An arbitrary pitch shift node needs no such information, and is useful outside of the context of a HTMLMediaElement. > I also found:— > https://developer.apple.com/library/prerelease/ios/documentation/AVFoundation/Reference/AVAudioUnitTimePitch_Class/ > > <https://developer.apple.com/library/prerelease/ios/documentation/AVFoundation/Reference/AVAudioUnitTimePitch_Class/> > ("preliminary document for an API or technology in development.”) You’ll be interested to know that the design of the Web Audio API was heavily influenced by the design of the OS X Core Audio library, and that is class you’ve found is an Objective-C wrapper around a Core Audio node. So yes, this class could well be used to implement a Web Audio PitchShift node. -Jer > > Thank you,
