Re: [whatwg] HTML5 video: frame accuracy / SMPTE
On Wed, Jan 12, 2011 at 2:49 AM, Philip Jägenstedt phil...@opera.com wrote: (Also, it might be useful to be able to chose whether seeking should be fast or exact, as frame-accurate seeking is hardly necessary in most normal playback situations.) In an audio engine I worked on I had a seek hint like that, to indicate whether the priority was accuracy or speed. It matters even more with video: when seeking with a seek bar, you may want to snap to keyframes, whereas bookmarks, jump to chapter features, etc. will often want to jump precisely. A fast seek option would be particularly useful for files with infrequent keyframes. -- Glenn Maynard
Re: [whatwg] HTML5 video: frame accuracy / SMPTE
On Wed, 12 Jan 2011 09:16:59 +0100, Glenn Maynard gl...@zewt.org wrote: On Wed, Jan 12, 2011 at 2:49 AM, Philip Jägenstedt phil...@opera.com wrote: (Also, it might be useful to be able to chose whether seeking should be fast or exact, as frame-accurate seeking is hardly necessary in most normal playback situations.) In an audio engine I worked on I had a seek hint like that, to indicate whether the priority was accuracy or speed. It matters even more with video: when seeking with a seek bar, you may want to snap to keyframes, whereas bookmarks, jump to chapter features, etc. will often want to jump precisely. A fast seek option would be particularly useful for files with infrequent keyframes. For the record, this is the solution I've been imagining: * add HTMLMediaElement.seek(t, [exact]), where exact defaults to false if missing * make setting HTMLMediaElement.currentTime be a non-exact seek, as that seems to be the most common case -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] HTML5 video: frame accuracy / SMPTE
2011-01-12 00:40 EEST: Rob Coenen: Hi David- that is b/c in an ideal world I'd want to seek to a time expressed as a SMPTE timecode (think web apps that let users step x frames back, seek y frames forward etc.). In order to convert SMPTE to the floating point value for video.seekTime I need to know the frame rate. It seems to me that such an application really requires a method for querying the timestamp for previous and next frames when given a timestamp. If such an application requires FPS value, it can then compute it by itself it such a value is assumed meaningful. (Simply get next frame timestamp from zero timestamp and continue for a couple of frames to compute FPS and check if the FPS seems to be stable.) Perhaps there should be a method getRelativeFrameTime(timestamp, relation) where timestamp is the current timestamp and relation is one of previousFrame, nextFrame, previousKeyFrame, nextKeyFrame? Use of this method could be allowed only for paused video if needed for simple implementation. -- Mikko
Re: [whatwg] Make radio button group suffering from being missing
2011-01-08 00:06 EEST: Ian Hickson: The basic idea behind this design is that type=radio seems to have been designed to keep each control as independent as possible -- before we started fiddling with it in WF2, the only way type=radio controls had any relationship to other type=radio controls was with their behaviour when they were clicked. Now, for required=, we needed a little more than that, because otherwise required= was essentially useless for radio buttons, but I tried to be faithful to the original design and kept it as independent as possible. [...] In the end I'll spec whatever gets implemented. What do other browser vendors feel about this? Should we make required= on any one type=radio control affect the validity of all the other controls? Or should we stick to the independent basis of this API? I'm not a browser vendor but my first opinion is that radio buttons should use any @required attribute in the radio button group. Rationale: 1. [In case all inputs should have @required set instead of any input in the radio button group:] From an author point of view, having a bunch of radio input elements with @required attribute set is not intuitive at all because end user cannot ever select all @required inputs in that case. 2. [In case any input having @required is enough:] I agree with Ian that currently type=radio controls are almost independent. However, the way @selected already works in a radio button group, the @required attribute would match the behavior quite well. From author point of view, this would seem a bit illogical, too: the end user is free to select some other input but @required. However... Saying that, I'd prefer having explicit not-allowed selection in a radio button group to represent the fact that something must be selected. That way the end user could intentionally reset the radio button group to the initial state. Should there also be support for UI style where radio button group has an item initially selected that makes the group suffering from being missing (logically one input type=radio in a radio group would have both @selected and @not-allowed set)? In the end, a reasonable compromise solution to these use cases would be to add @required attribute to any acceptable radio button item. In that case, the case where nothing is initially selected in a radio button group would be represented with a source where every item in radio button group had @required and none had @selected. On the other hand, the style where one input had @selected initially could be represented with markup where initial input had @selected but not @required and all the other inputs had @required. That would be interpreted as a initial selection of radio button item that is not allowed for the radio button group (at least one other item has @required and the currently selected item does not have @required, in other words, should not be considered acceptable). That would be a nice match for UI sometimes seen in web service EULAs: [type=radio @selected] I do not accept license terms. [type=radio @required] I accept the license terms. -- Mikko
Re: [whatwg] HTML5 video: frame accuracy / SMPTE
On Jan 12, 2011, at 11:04 AM, whatwg-requ...@lists.whatwg.org wrote: Date: Wed, 12 Jan 2011 11:54:47 +0200 From: Mikko Rantalainen mikko.rantalai...@peda.net To: whatwg@lists.whatwg.org Subject: Re: [whatwg] HTML5 video: frame accuracy / SMPTE Message-ID: 4d2d7a67.7090...@peda.net Content-Type: text/plain; charset=ISO-8859-1 2011-01-12 00:40 EEST: Rob Coenen: Hi David- that is b/c in an ideal world I'd want to seek to a time expressed as a SMPTE timecode (think web apps that let users step x frames back, seek y frames forward etc.). In order to convert SMPTE to the floating point value for video.seekTime I need to know the frame rate. It seems to me that such an application really requires a method for querying the timestamp for previous and next frames when given a timestamp. If such an application requires FPS value, it can then compute it by itself it such a value is assumed meaningful. (Simply get next frame timestamp from zero timestamp and continue for a couple of frames to compute FPS and check if the FPS seems to be stable.) Perhaps there should be a method getRelativeFrameTime(timestamp, relation) where timestamp is the current timestamp and relation is one of previousFrame, nextFrame, previousKeyFrame, nextKeyFrame? Use of this method could be allowed only for paused video if needed for simple implementation. Alternatively, one could look at a step() function instead of a seek(pos,exact) function. The step function can be used for frame-accurate controls. e.g. step(2) or step(-1). The advantage over a seek(pos,exact) function (and the playback rate controls) is that the viewer really knows the video is X frames offset. This is very useful for both artistic/editing applications and for video analysis applications (think sports, medical or experiments). The downside of a step() to either always accurate seeking or a seek(pos,exact) is that it requires two steps in situations like bookmarking or chaptering. It seems like the framerate / SMPTE proposals done here are all a means to end up with frame-accurate seeking. With a step() function in place, there's no need for such things. In fact, one could do a step(10) or so and then use the difference in position to calculate framerate. - Jeroen
Re: [whatwg] HTML5 video: frame accuracy / SMPTE
On Wed, Jan 12, 2011 at 10:15 AM, Jeroen Wijering jer...@longtailvideo.comwrote: Alternatively, one could look at a step() function instead of a seek(pos,exact) function. The step function can be used for frame-accurate controls. e.g. step(2) or step(-1). The advantage over a seek(pos,exact) function (and the playback rate controls) is that the viewer really knows the video is X frames offset. This is very useful for both artistic/editing applications and for video analysis applications (think sports, medical or experiments). glad that you are mentioning these artistic/editing/video analysis type of applications. I'd like to add video archiving/logging/annotating/subtitling to the list of potential applications. But also experiments and time-based interaction. Most online ad-agencies have been using Flash to design eg. highly interactive mini-sites where banners, etc. are shown or hidden based up on the exact timing of the video. Also think projects such as http://www.thewildernessdowntown.com/ The downside of a step() to either always accurate seeking or a seek(pos,exact) is that it requires two steps in situations like bookmarking or chaptering. I like the idea of a stepping-function, similar to how Flash Media Server/Flash Player 10.1 does frame-accurate stepping. It seems like the framerate / SMPTE proposals done here are all a means to end up with frame-accurate seeking. With a step() function in place, there's no need for such things. In fact, one could do a step(10) or so and then use the difference in position to calculate framerate. - Jeroen The need for SMPTE still remains as I want to be able to do things such as video.seekTo(smpte_timecode_converted_to_seconds, seek_exact=true); so that my video goes to exactly the exact frame as indicated by smpte_timecode_converted_to_seconds. Think chapter bookmarking, scene indexing, etc. -Rob
Re: [whatwg] HTML5 video: frame accuracy / SMPTE
On Jan 12, 2011, at 2:05 PM, Rob Coenen wrote:
The need for SMPTE still remains as I want to be able to do things such as
video.seekTo(smpte_timecode_converted_to_seconds, seek_exact=true); so that
my video goes to exactly the exact frame as indicated by
smpte_timecode_converted_to_seconds. Think chapter bookmarking, scene
indexing, etc.
With the step() in place, this would be a simple convenience function. This
pseudo-code is not ideal and making some assumptions, but the approach should
work:
function seekToTimecode(timecode) {
var seconds = convert_timecode_to_seconds(timecode);
videoElement.seek(seconds);
var delta = seconds - videoElement.currentTime;
while (delta 0) {
videoElement.step(1);
delta = seconds - videoElement.currentTime;
}
};
Its basically stepping to the frame that's closest to the timecode (as
elaborated by others, there's no such thing as timecode in MP4/WebM. It's just
timestamps).
Note you actually do want to have this conversion taking place in javascript,
since there are many reasons to adjust/offset the conversion (sync issues,
timecode base differences, ...). If it's locked up inside the browser API you
have to do duplicate work around it if the input files / conversion assumptions
don't align.
- Jeroen
Re: [whatwg] HTML5 video: frame accuracy / SMPTE
On Jan 12, 2011, at 12:42 AM, Philip Jägenstedt wrote: On Wed, 12 Jan 2011 09:16:59 +0100, Glenn Maynard gl...@zewt.org wrote: On Wed, Jan 12, 2011 at 2:49 AM, Philip Jägenstedt phil...@opera.com wrote: (Also, it might be useful to be able to chose whether seeking should be fast or exact, as frame-accurate seeking is hardly necessary in most normal playback situations.) In an audio engine I worked on I had a seek hint like that, to indicate whether the priority was accuracy or speed. It matters even more with video: when seeking with a seek bar, you may want to snap to keyframes, whereas bookmarks, jump to chapter features, etc. will often want to jump precisely. A fast seek option would be particularly useful for files with infrequent keyframes. For the record, this is the solution I've been imagining: * add HTMLMediaElement.seek(t, [exact]), where exact defaults to false if missing * make setting HTMLMediaElement.currentTime be a non-exact seek, as that seems to be the most common case That is a very interesting idea!. Precise seeking in some video files can be quite slow, greater than a second is not unlikely on some devices. FWIW, the media playback framework on iOS has a seek method with parameters for the tolerance allowed before and after the seek time [1] to allow the programmer to choose. eric [1] http://developer.apple.com/library/ios/#documentation/AVFoundation/Reference/AVPlayer_Class/Reference/Reference.html%23//apple_ref/occ/instm/AVPlayer/seekToTime:toleranceBefore:toleranceAfter:
Re: [whatwg] HTML5 video: frame accuracy / SMPTE
I implemented frame-accurate seeking in Chrome (mostly as an experiment) and it does have the drawback of potentially being very slow. Depending on the type of video there can be a noticeable difference in seek time if you seek to just-before-a-keyframe versus just-after-a-keyframe. I do like the idea, however :) Andrew On Wed, Jan 12, 2011 at 5:32 AM, Eric Carlson eric.carl...@apple.comwrote: On Jan 12, 2011, at 12:42 AM, Philip Jägenstedt wrote: On Wed, 12 Jan 2011 09:16:59 +0100, Glenn Maynard gl...@zewt.org wrote: On Wed, Jan 12, 2011 at 2:49 AM, Philip Jägenstedt phil...@opera.com wrote: (Also, it might be useful to be able to chose whether seeking should be fast or exact, as frame-accurate seeking is hardly necessary in most normal playback situations.) In an audio engine I worked on I had a seek hint like that, to indicate whether the priority was accuracy or speed. It matters even more with video: when seeking with a seek bar, you may want to snap to keyframes, whereas bookmarks, jump to chapter features, etc. will often want to jump precisely. A fast seek option would be particularly useful for files with infrequent keyframes. For the record, this is the solution I've been imagining: * add HTMLMediaElement.seek(t, [exact]), where exact defaults to false if missing * make setting HTMLMediaElement.currentTime be a non-exact seek, as that seems to be the most common case That is a very interesting idea!. Precise seeking in some video files can be quite slow, greater than a second is not unlikely on some devices. FWIW, the media playback framework on iOS has a seek method with parameters for the tolerance allowed before and after the seek time [1] to allow the programmer to choose. eric [1] http://developer.apple.com/library/ios/#documentation/AVFoundation/Reference/AVPlayer_Class/Reference/Reference.html%23//apple_ref/occ/instm/AVPlayer/seekToTime:toleranceBefore:toleranceAfter :
[whatwg] Control over selection direction
Hi, I'd like to propose a minor addition to 4.10.20 APIs for the text field selections. When programmatically setting the selection of a text input, it is currently impossible to create a range with the 'anchor' at the bottom and the 'base' at the top. Concretely, this means that, after a selection has been set by a program, if the user presses shift and moves the cursor, it is always the bottom of the selection that is moved. When doing heavy scripting on such input element, it is often necessary to restore a previous selection exactly as the user made it. This is currently not possible, and I'd say the HTML5 standard is our only hope of finally getting something like this widely implemented. The most obvious way to handle this would be to allow selectionStart to be greater than selectionEnd. This is, however, unacceptable for various reasons -- it's not easily feature-detectable, it breaks older code that reads these properties and expects start to never be greater than end, and it makes the names more confusing than they have to be. So I propose a selectionAnchor property, which holds either top or bottom, and can be set to one of these strings to modify the direction. top would mean the anchor lies after the base of the selection, so further shift-movement modifies the bottom, whereas botton means the inverse, with movement modifying the top. (I think the top/bottom terminology shouldn't break with any languages that order characters differently. Does anyone write bottom to top? If someone sees a problem, we could choose other terms, but these are the most obvious I could come up with.) So, in standard-ese: - element.selectionAnchor [= value] Returns either top or bottom, indicating the side of the selection that is anchored. Can be set to change the side of the selection that counts as the anchor. [...] The selectionAnchor attribute must, on getting, return one of the strings top or bottom. bottom is returned when the anchor, the fixed end, of the selection lies before the base, the movable end. top is returned in all other cases. When set to top when its current value is bottom, or set to bottom when its current value is top, is must flip the roles of the ends of the selection, so that what used to be the anchor now becomes the base. When set to any other value, this is ignored. -- Let me know what you think. Best, Marijn Haverbeke
Re: [whatwg] Control over selection direction
I agree that something like this is a valuable addition. I'm not sure selectionAnchor is ideal. I was thinking selectionDirection=forward/backward. It's equivalent really, but that just seems more intuitive to me. On Wed, Jan 12, 2011 at 11:35 AM, Marijn Haverbeke mari...@gmail.comwrote: Hi, I'd like to propose a minor addition to 4.10.20 APIs for the text field selections. When programmatically setting the selection of a text input, it is currently impossible to create a range with the 'anchor' at the bottom and the 'base' at the top. Concretely, this means that, after a selection has been set by a program, if the user presses shift and moves the cursor, it is always the bottom of the selection that is moved. When doing heavy scripting on such input element, it is often necessary to restore a previous selection exactly as the user made it. This is currently not possible, and I'd say the HTML5 standard is our only hope of finally getting something like this widely implemented. The most obvious way to handle this would be to allow selectionStart to be greater than selectionEnd. This is, however, unacceptable for various reasons -- it's not easily feature-detectable, it breaks older code that reads these properties and expects start to never be greater than end, and it makes the names more confusing than they have to be. So I propose a selectionAnchor property, which holds either top or bottom, and can be set to one of these strings to modify the direction. top would mean the anchor lies after the base of the selection, so further shift-movement modifies the bottom, whereas botton means the inverse, with movement modifying the top. (I think the top/bottom terminology shouldn't break with any languages that order characters differently. Does anyone write bottom to top? If someone sees a problem, we could choose other terms, but these are the most obvious I could come up with.) So, in standard-ese: - element.selectionAnchor [= value] Returns either top or bottom, indicating the side of the selection that is anchored. Can be set to change the side of the selection that counts as the anchor. [...] The selectionAnchor attribute must, on getting, return one of the strings top or bottom. bottom is returned when the anchor, the fixed end, of the selection lies before the base, the movable end. top is returned in all other cases. When set to top when its current value is bottom, or set to bottom when its current value is top, is must flip the roles of the ends of the selection, so that what used to be the anchor now becomes the base. When set to any other value, this is ignored. -- Let me know what you think. Best, Marijn Haverbeke
Re: [whatwg] Control over selection direction
I completely agree, and have been lobbying for similar functionality for the main document selection object, resulting in the current ongoing discussion in this bug: http://www.w3.org/Bugs/Public/show_bug.cgi?id=10624. Rather than a single string property, how about integer selectionAnchor and selectionFocus properties? This is then analogous to Selection's anchorNode, anchorOffset, focusNode and focusOffset properties and avoids an awkward string property. Tim On 12 January 2011 19:35, Marijn Haverbeke mari...@gmail.com wrote: Hi, I'd like to propose a minor addition to 4.10.20 APIs for the text field selections. When programmatically setting the selection of a text input, it is currently impossible to create a range with the 'anchor' at the bottom and the 'base' at the top. Concretely, this means that, after a selection has been set by a program, if the user presses shift and moves the cursor, it is always the bottom of the selection that is moved. When doing heavy scripting on such input element, it is often necessary to restore a previous selection exactly as the user made it. This is currently not possible, and I'd say the HTML5 standard is our only hope of finally getting something like this widely implemented. The most obvious way to handle this would be to allow selectionStart to be greater than selectionEnd. This is, however, unacceptable for various reasons -- it's not easily feature-detectable, it breaks older code that reads these properties and expects start to never be greater than end, and it makes the names more confusing than they have to be. So I propose a selectionAnchor property, which holds either top or bottom, and can be set to one of these strings to modify the direction. top would mean the anchor lies after the base of the selection, so further shift-movement modifies the bottom, whereas botton means the inverse, with movement modifying the top. (I think the top/bottom terminology shouldn't break with any languages that order characters differently. Does anyone write bottom to top? If someone sees a problem, we could choose other terms, but these are the most obvious I could come up with.) So, in standard-ese: - element.selectionAnchor [= value] Returns either top or bottom, indicating the side of the selection that is anchored. Can be set to change the side of the selection that counts as the anchor. [...] The selectionAnchor attribute must, on getting, return one of the strings top or bottom. bottom is returned when the anchor, the fixed end, of the selection lies before the base, the movable end. top is returned in all other cases. When set to top when its current value is bottom, or set to bottom when its current value is top, is must flip the roles of the ends of the selection, so that what used to be the anchor now becomes the base. When set to any other value, this is ignored. -- Let me know what you think. Best, Marijn Haverbeke
Re: [whatwg] HTML5 video: frame accuracy / SMPTE
On Wed, Jan 12, 2011 at 3:42 AM, Philip Jägenstedt phil...@opera.com wrote: * add HTMLMediaElement.seek(t, [exact]), where exact defaults to false if missing Boolean parameters are evil, since it's impossible to guess what they do from reading the code. Make it a two-value enum instead. The second argument could be extended to a bitfield later if desired, too.
Re: [whatwg] HTML5 video: frame accuracy / SMPTE
On Wed, Jan 12, 2011 at 9:42 PM, Philip Jägenstedt phil...@opera.comwrote: For the record, this is the solution I've been imagining: * add HTMLMediaElement.seek(t, [exact]), where exact defaults to false if missing * make setting HTMLMediaElement.currentTime be a non-exact seek, as that seems to be the most common case I think setting currentTime should be exact, since a) exact seeking is simpler from the author's point of view and b) it would be unfortunate to set currentTime to value T and then discover that getting currentTime gives you a value other than T (assuming you're paused). Rob -- Now the Bereans were of more noble character than the Thessalonians, for they received the message with great eagerness and examined the Scriptures every day to see if what Paul said was true. [Acts 17:11]
Re: [whatwg] Control over selection direction
On Wed, Jan 12, 2011 at 2:35 PM, Marijn Haverbeke mari...@gmail.com wrote: So I propose a selectionAnchor property, which holds either top or bottom, and can be set to one of these strings to modify the direction. top would mean the anchor lies after the base of the selection, so further shift-movement modifies the bottom, whereas botton means the inverse, with movement modifying the top. This sounds good to me. I used the term forwards and backwards for the DOM Range spec, which I think make more sense. I also think selectionDirection is a better property name, as Ojan suggests.
Re: [whatwg] HTML5 video: frame accuracy / SMPTE
On Jan 12, 2011, at 4:04 PM, Robert O'Callahan wrote: On Wed, Jan 12, 2011 at 9:42 PM, Philip Jägenstedt phil...@opera.comwrote: For the record, this is the solution I've been imagining: * add HTMLMediaElement.seek(t, [exact]), where exact defaults to false if missing * make setting HTMLMediaElement.currentTime be a non-exact seek, as that seems to be the most common case I think setting currentTime should be exact, since a) exact seeking is simpler from the author's point of view and b) it would be unfortunate to set currentTime to value T and then discover that getting currentTime gives you a value other than T (assuming you're paused). I agree that precise seeking follows the principle of least surprise, based partly on the bugs files against the video element on iOS where this hasn't always been the behavior. eric
[whatwg] Structured clones of Files
Does the same underlying data of a structured clone of a File refer to a reference to the file (eg. the path and filename, which is the real underlying data of a File object), or the contents of the file? As it's used by web worker messaging, it's obviously the pathname. However, HTML Storage (in theory, anyway) and IndexedDB also use structured clones, and I'm guessing you're not meant to be able to store a reference to a File (or FileList) persistently and reopen it later, which suggests that the actual contents of the file should be stored. Personally, I think it's both acceptable from a security perspective and incredibly useful to allow the user to grant access to a file (or--more importantly--a directory, eventually) just once, and then access it from then on without the user having to manually grant access each and every time he uses the application, perhaps requiring one-time user permission at some point to do so. Storing a reference in an IndexedDB would allow that. But from what I recall, that's not currently meant to be allowed. -- Glenn Maynard
Re: [whatwg] HTML5 video: frame accuracy / SMPTE
On Thu, 13 Jan 2011 01:23:57 +0100, Eric Carlson eric.carl...@apple.com wrote: On Jan 12, 2011, at 4:04 PM, Robert O'Callahan wrote: On Wed, Jan 12, 2011 at 9:42 PM, Philip Jägenstedt phil...@opera.comwrote: For the record, this is the solution I've been imagining: * add HTMLMediaElement.seek(t, [exact]), where exact defaults to false if missing * make setting HTMLMediaElement.currentTime be a non-exact seek, as that seems to be the most common case I think setting currentTime should be exact, since a) exact seeking is simpler from the author's point of view and b) it would be unfortunate to set currentTime to value T and then discover that getting currentTime gives you a value other than T (assuming you're paused). I agree that precise seeking follows the principle of least surprise, based partly on the bugs files against the video element on iOS where this hasn't always been the behavior. Changing the default at this point wouldn't really hurt since not all browsers are doing exact seeking anyway, right? I think that keyframe seeking is more often what you want and it'd be better to let the few who want frame-exact seeking jump through hoops than having everyone who makes a simple seeking UI discover that seeking is a bit slow and 20% of them figuring out that it can be fixed by using seek(). -- Philip Jägenstedt Core Developer Opera Software
Re: [whatwg] HTML5 video: frame accuracy / SMPTE
On Thu, 13 Jan 2011 01:03:03 +0100, Aryeh Gregor simetrical+...@gmail.com wrote: On Wed, Jan 12, 2011 at 3:42 AM, Philip Jägenstedt phil...@opera.com wrote: * add HTMLMediaElement.seek(t, [exact]), where exact defaults to false if missing Boolean parameters are evil, since it's impossible to guess what they do from reading the code. Make it a two-value enum instead. The second argument could be extended to a bitfield later if desired, too. WFM -- Philip Jägenstedt Core Developer Opera Software
