[whatwg] VIDEO and pitchAdjustment
It would be useful to have pitch adjustment for VIDEO element. There is playbackRate, to control playback speed — useful!* And there is vv.mozPreservesPitch, in Firefox, which can be set to false, so that pitch will adjust to the speed of the video, sort of like old analog gear (tapes and records). Negative playbackRate, to watch videos backwards, currently crashes Safari 8; Firefox 40 says not implemented. I think it would be entertaining for example, to watch things like cars uncrashing. Should this work? And if so, shouldn't audio match the video speed for backwards playing? I think yes, and yes. But variable pitch control it would be useful for music adjustments like over the mountain, Black Star, Take your Whiskey Home, all originally recorded at half-step detuning (A=415), none of which match standard tuning commonly used in schools, industry, etc. Recordings of Baroque era music often use instruments tuned lower (and also different scale temperment, but that is a different issue). So it would be nice to adjust the pitch with a `pitchAdjustment` property, as a double, to adjust pitch in cents. Currently, there are applications to do this, such as the Amazing Slow Downer. Could similar API functionality be realized for VIDEO elements, to make such apps in-browser? For that, there must be input from browser vendors to say yes, we see the value in that, and it's possible. YouTube Speed https://gist.githubusercontent.com/GarrettS/c7bf3f49e9ce43f6bd25/raw/c9dc1d3e42529fb5bd2b5b283a9190fc233cd5c8/Variable%2520Speed%2520for%2520YouTube Over the Mountain https://www.youtube.com/watch?v=wO2BDiSHTh8feature=youtu.be Black Star https://www.youtube.com/watch?v=blNQZc84Q5c Take your whiskey home https://www.youtube.com/watch?v=4ckFuURIWXc Amazing Slow Downer http://www.amazing-slow-downer.com/ -- Garrett @xkit ChordCycles.wordpress.com garretts.github.io personx.tumblr.com
[whatwg] HTML spec now on GitHub
Just a heads-up to those of you who like to follow the nitty gritty of spec development here: The HTML spec has joined the other WHATWG specs on GitHub, rather than being alone in the svn.whatwg.org SVN repository. This will enable other WHATWG editors to also contribute to the HTML spec, so you might get replies from Anne, Domenic, or Philip rather than just me. As part of this we are also going to move to using GitHub issues rather than bugs in the W3C Bugzilla. (This also means the widget in the lower left is gone for now. Hopefully we'll figure out a way to add that back.) Don't worry though, we're still going to track the bugs in the W3C Bugzilla, there's no need to manually go and move them over or anything. Let me know if you have any questions. In theory this shouldn't really have any practical impact on anyone except us editors and those of you filing bugs. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] VIDEO and pitchAdjustment
On Thu, Aug 27, 2015 at 12:02 PM, Garrett Smith dhtmlkitc...@gmail.com wrote: Negative playbackRate, to watch videos backwards, currently crashes Safari 8; Firefox 40 says not implemented. I think it would be entertaining for example, to watch things like cars uncrashing. Should this work? And if so, shouldn't audio match the video speed for backwards playing? I think yes, and yes. It would be much easier to re-encode the videos in reverse and provide two versions of the video. Videos are made to be played in one direction, and they use this assumption to improve their compression efficiency. Basically, videos have two different types of frames; let's call them Full frames and Partial frames. A Full frame contains all of the information that a stand-alone image would have. If a video had only a single Full frame, it would be able to display that image and show it for the length of the video (incidentally, this is what a WebP image is). Partial frames, on the other hand, do NOT have enough information to produce a full image. They merely contain the information representing the difference between the previous frame and the current frame. This obviously saves a lot of space if the frames are similar. A video will contain a mixture of Full frames and Partial frames, with many more Partial frames than Full frames (ex. 1 Full frame followed by 10 seconds of Partial frames (~240 partial frames)). What this means is that when you play a video in the proper direction, it must first decode a Full frame, but then it only has to decode Partial frames (until the next Full frame). However, if you try to play a video in reverse, it must decode a Full frame and ALL of the Partial frames up to the current frame before it can display the current frame. To display the frame before that, it must decode all of those frames again, and so on for every frame of the video. You could of course store the decoded frames in memory, but that will take a lot of space. 240 uncompressed frames at 720p is over half a gigabyte of information. It's just not practical.
Re: [whatwg] VIDEO and pitchAdjustment
On Fri, Aug 28, 2015 at 6:02 AM, Garrett Smith dhtmlkitc...@gmail.com wrote: It would be useful to have pitch adjustment for VIDEO element. There is playbackRate, to control playback speed — useful!* And there is vv.mozPreservesPitch, in Firefox, which can be set to false, so that pitch will adjust to the speed of the video, sort of like old analog gear (tapes and records). We should standardize mozPreservesPitch. I'm embarrassed that we haven't done so already. Negative playbackRate, to watch videos backwards, currently crashes Safari 8; Firefox 40 says not implemented. I think it would be entertaining for example, to watch things like cars uncrashing. Should this work? And if so, shouldn't audio match the video speed for backwards playing? I think yes, and yes. According to the spec it should work, but it's very low priority for us and implementing it would be very inefficient as Yay295 describes. So I don't think it's going to happen in Firefox in the forseeable future. But variable pitch control it would be useful for music adjustments like over the mountain, Black Star, Take your Whiskey Home, all originally recorded at half-step detuning (A=415), none of which match standard tuning commonly used in schools, industry, etc. Recordings of Baroque era music often use instruments tuned lower (and also different scale temperment, but that is a different issue). So it would be nice to adjust the pitch with a `pitchAdjustment` property, as a double, to adjust pitch in cents. I'd prefer to focus on making sure that Web Audio's Audio Workers are powerful enough for Web developers to implement this themselves. Rob -- lbir ye,ea yer.tnietoehr rdn rdsme,anea lurpr edna e hnysnenh hhe uresyf toD selthor stor edna siewaoeodm or v sstvr esBa kbvted,t rdsme,aoreseoouoto o l euetiuruewFa kbn e hnystoivateweh uresyf tulsa rehr rdm or rnea lurpr .a war hsrer holsa rodvted,t nenh hneireseoouot.tniesiewaoeivatewt sstvr esn
Re: [whatwg] VIDEO and pitchAdjustment
On 8/27/15, Robert O'Callahan rob...@ocallahan.org wrote: On Fri, Aug 28, 2015 at 6:02 AM, Garrett Smith dhtmlkitc...@gmail.com wrote: It would be useful to have pitch adjustment for VIDEO element. There is playbackRate, to control playback speed — useful!* And there is vv.mozPreservesPitch, in Firefox, which can be set to false, so that pitch will adjust to the speed of the video, sort of like old analog gear (tapes and records). We should standardize mozPreservesPitch. I'm embarrassed that we haven't done so already. Negative playbackRate, to watch videos backwards, currently crashes Safari 8; Firefox 40 says not implemented. I think it would be entertaining for example, to watch things like cars uncrashing. Should this work? And if so, shouldn't audio match the video speed for backwards playing? I think yes, and yes. According to the spec it should work, but it's very low priority for us and implementing it would be very inefficient as Yay295 describes. So I don't think it's going to happen in Firefox in the forseeable future. But variable pitch control it would be useful for music adjustments like over the mountain, Black Star, Take your Whiskey Home, all originally recorded at half-step detuning (A=415), none of which match standard tuning commonly used in schools, industry, etc. Recordings of Baroque era music often use instruments tuned lower (and also different scale temperment, but that is a different issue). So it would be nice to adjust the pitch with a `pitchAdjustment` property, as a double, to adjust pitch in cents. I'd prefer to focus on making sure that Web Audio's Audio Workers are powerful enough for Web developers to implement this themselves. I don't know enough about Web Audio to comment on that. Most of what I'm using is on YouTube. And there are some video issues that I'd like to comment on because it's such a common situation to have audio as video MP4's. So here's two video user scenarios: Want hear the audio at the desired pitch, while simultaneously watching the video at the desired speed, to learn it. For fine-grained adjustment for music performances that are in between keys (off by a few cents) or need to be transcribed. This guy's cover of Over the Mountain is pretty good. https://www.youtube.com/watch?v=Ze6h1t7Z734 Right now, I can get that up a half step to standard tuning by increasing the speed and setting mozPreservesPitch:— var vv = document.querySelector(video); vv.mozPreservesPitch = false; vv.playbackRate = 1.05; But it's a little sped up. Not ideal for learning! I'd rather slow down it a bit. Enter the Amazing Slow Downer … does what I want. OK, there's my two user scenarios with an example. Thanks for reading! -- Garrett @xkit ChordCycles.wordpress.com garretts.github.io personx.tumblr.com
