Re: [whatwg] VIDEO and pitchAdjustment

2016-03-11 Thread Jer Noble 

> On Mar 11, 2016, at 1:11 PM, Garrett Smith  wrote:
> 
> 
> 
> On Tuesday, March 8, 2016, Jer Noble  > wrote:
> 
> > On Mar 8, 2016, at 4:42 PM, Garrett Smith > wrote:
> >
> > On Fri, Mar 4, 2016 at 3:43 PM, Jer Noble > wrote:
> >>
> >>> On Mar 4, 2016, at 3:19 PM, Garrett Smith > 
> >>> wrote:
> >>>
> >>> On Fri, Mar 4, 2016 at 1:55 PM, Jer Noble > wrote:
> 
> > On Mar 1, 2016, at 8:00 PM, Philip Jägenstedt > 
> > wrote:
> >
> > On Wed, Mar 2, 2016 at 9:19 AM, Garrett Smith  > <>> wrote:
> >> On Thu, Nov 12, 2015 at 11:32 AM, Philip Jägenstedt  >> <>> wrote:
> >>> On Thu, Nov 12, 2015 at 10:55 AM, Garrett Smith 
> >>> >
> >>> wrote:
>  On 11/12/15, Philip Jägenstedt > wrote:
> > On Thu, Nov 12, 2015 at 9:07 AM, Garrett Smith 
> > >
> > wrote:
> >>
> >> On 10/19/15, Philip Jägenstedt > wrote:
> >>> On Tue, Sep 1, 2015 at 11:21 AM, Philip Jägenstedt 
> >>> >
> >>> wrote:
>  On Mon, Aug 31, 2015 at 9:48 PM, Domenic Denicola 
>  >
>  wrote:
> > From: Eric Carlson [mailto:eric.carl...@apple.com <>]
> 
>  
> >>
> >> The Web Audio equivalent would be:
> >>
> >> var video = document.querySelector(‘video’);
> >> video.preservesPitch = false;
> >> var context = new AudioContext();
> >> var sourceNode = context.createMediaElementSource(video);
> >> var pitchShiftNode = context.createPitchShift();
> >> pitchShiftNode.shiftAmount = 1.023;
> >> sourceNode.connect(pitchShiftNode);
> >> pitchShiftNode.connect(context.destination);
> >>
> >
> > Which implementations does that work in?
> 
> None, because there is no such thing as a PitchShiftNode.
> 
> 
> I see. 
>  
> > That code is more complex than should be necessary. I see where you're
> > coming from on separating the audio. Could we move the media decorator
> > behind the scenes, and replace it with a simple getter/setter property
> > like `videoElement.audio` so that that can happen automagically?
> > Reminds me of createElement, createRange, document.implementation,
> > etc. Warts!
> 
> I’m not entirely sure what you’re asking here. If it’s that you don’t like 
> the `context.createMediaElementSource()` or `context.createPitchShift()` 
> syntax and would rather a constructor syntax, Issue #250 
>  > in the Web Audio spec 
> is the issue for you.
> 
> > But then again, you also just said that there are no APIs on OS X that
> > allow an arbitrary pitch shift to be added to audio. If that is true,
> > then your `createPitchShift` code would be possible anyway, is that
> 
> There is no such API for such post-processing built into the OS X media 
> frameworks. 
> 
> Oh. Poor hardware integration, and now this…. Being Apple CEO is not Tim 
> Cook's greatest gift… 

I’m not sure how that’s relevant.

> As an example, there is an API for preserving pitch across rate changes: 
> -[AVPlayerItem setAudioTimePitchAlgorithm:] 
>   
> >.
>   So implementing the HTMLMediaElement “preservesPitch" property is trivial.  
> There is no such API for applying a specific and arbitrary amount of pitch 
> shifting. It would have to be implemented entirely by hand, the same way we 
> implement processing of media element audio with the Web Audio API.  Which 
> gets back to: at this point, we should just implement a Pitch Shift node in 
> Web Audio.
> 
> 
>  I see.
> 
> >> var video = document.querySelector(‘video’);
> >> var context = new AudioContext();
> >> var sourceNode = context.createMediaElementSource(video);
> >> var pitchShiftNode = context.createPitchShift();
> >> pitchShiftNode.shiftAmount = 1.023;
> >> sourceNode.connect(pitchShiftNode);
> >> pitchShiftNode.connect(context.destination);
> >>
> 
> Too many steps and implementation details. It's complicated. It's asking for 
> more js libraries and wrappers for that, and other situations related to 
> videos' audio.  Make it simpler. Hide the implementation details.
> 
> Hide:—
>  * AudioContext creation and subsequent video connection.
>  * createPitchShift — use audio.pitchShift to get a pitchShiftNode
>  * pitchShiftNode.shiftAmount —

Re: [whatwg] VIDEO and pitchAdjustment

2016-03-11 Thread Garrett Smith 
On Tuesday, March 8, 2016, Jer Noble  wrote:

>
> > On Mar 8, 2016, at 4:42 PM, Garrett Smith 
> wrote:
> >
> > On Fri, Mar 4, 2016 at 3:43 PM, Jer Noble  wrote:
> >>
> >>> On Mar 4, 2016, at 3:19 PM, Garrett Smith 
> wrote:
> >>>
> >>> On Fri, Mar 4, 2016 at 1:55 PM, Jer Noble  wrote:
> 
> > On Mar 1, 2016, at 8:00 PM, Philip Jägenstedt 
> wrote:
> >
> > On Wed, Mar 2, 2016 at 9:19 AM, Garrett Smith <
> dhtmlkitc...@gmail.com> wrote:
> >> On Thu, Nov 12, 2015 at 11:32 AM, Philip Jägenstedt <
> phil...@opera.com> wrote:
> >>> On Thu, Nov 12, 2015 at 10:55 AM, Garrett Smith <
> dhtmlkitc...@gmail.com>
> >>> wrote:
>  On 11/12/15, Philip Jägenstedt  wrote:
> > On Thu, Nov 12, 2015 at 9:07 AM, Garrett Smith <
> dhtmlkitc...@gmail.com>
> > wrote:
> >>
> >> On 10/19/15, Philip Jägenstedt  wrote:
> >>> On Tue, Sep 1, 2015 at 11:21 AM, Philip Jägenstedt <
> phil...@opera.com>
> >>> wrote:
>  On Mon, Aug 31, 2015 at 9:48 PM, Domenic Denicola <
> d...@domenic.me>
>  wrote:
> > From: Eric Carlson [mailto:eric.carl...@apple.com]


>
>
>>
> >> The Web Audio equivalent would be:
> >>
> >> var video = document.querySelector(‘video’);
> >> video.preservesPitch = false;
> >> var context = new AudioContext();
> >> var sourceNode = context.createMediaElementSource(video);
> >> var pitchShiftNode = context.createPitchShift();
> >> pitchShiftNode.shiftAmount = 1.023;
> >> sourceNode.connect(pitchShiftNode);
> >> pitchShiftNode.connect(context.destination);
> >>
> >
> > Which implementations does that work in?
>
> None, because there is no such thing as a PitchShiftNode.
>
>
I see.


> > That code is more complex than should be necessary. I see where you're
> > coming from on separating the audio. Could we move the media decorator
> > behind the scenes, and replace it with a simple getter/setter property
> > like `videoElement.audio` so that that can happen automagically?
> > Reminds me of createElement, createRange, document.implementation,
> > etc. Warts!
>
> I’m not entirely sure what you’re asking here. If it’s that you don’t like
> the `context.createMediaElementSource()` or `context.createPitchShift()`
> syntax and would rather a constructor syntax, Issue #250 <
> https://github.com/WebAudio/web-audio-api/issues/250> in the Web Audio
> spec is the issue for you.
>
> > But then again, you also just said that there are no APIs on OS X that
> > allow an arbitrary pitch shift to be added to audio. If that is true,
> > then your `createPitchShift` code would be possible anyway, is that
>
> There is no such API for such post-processing built into the OS X media
> frameworks.


Oh. Poor hardware integration, and now this…. Being Apple CEO is not Tim
Cook's greatest gift…

As an example, there is an API for preserving pitch across rate changes:
> -[AVPlayerItem setAudioTimePitchAlgorithm:] <
> https://developer.apple.com/library/prerelease/ios/documentation/AVFoundation/Reference/AVPlayerItem_Class/index.html#//apple_ref/occ/instp/AVPlayerItem/audioTimePitchAlgorithm>.
> So implementing the HTMLMediaElement “preservesPitch" property is trivial.
> There is no such API for applying a specific and arbitrary amount of pitch
> shifting. It would have to be implemented entirely by hand, the same way we
> implement processing of media element audio with the Web Audio API.  Which
> gets back to: at this point, we should just implement a Pitch Shift node in
> Web Audio.
>
>
 I see.

>> var video = document.querySelector(‘video’);
>> var context = new AudioContext();
>> var sourceNode = context.createMediaElementSource(video);
>> var pitchShiftNode = context.createPitchShift();
>> pitchShiftNode.shiftAmount = 1.023;
>> sourceNode.connect(pitchShiftNode);
>> pitchShiftNode.connect(context.destination);
>>

Too many steps and implementation details. It's complicated. It's asking
for more js libraries and wrappers for that, and other situations related
to videos' audio.  Make it simpler. Hide the implementation details.

Hide:—
 * AudioContext creation and subsequent video connection.
 * createPitchShift — use audio.pitchShift to get a pitchShiftNode
 * pitchShiftNode.shiftAmount — make pitchShiftNode a getter/setter (like
video.playbackRate)

>> var video = document.querySelector(‘video’);
>> var context = new AudioContext();
>> var sourceNode = context.createMediaElementSource(video);
>> var context = new AudioContext();

// Hidden AudioContext construction
var audio = video.audio;

>> var pitchShiftNode = context.createPitchShift();
>> pitchShiftNode.shiftAmount = 1.023;

//  Hidden AudioContext createPitchShift factory.
//  Setter on pitchShift
audio.pitchShift = 1.023;

>> sourceNode.connect(pitchShiftNode);
>>