Re: [whatwg] Full Screen API Feedback
On Wed, May 11, 2011 at 11:27 AM, Jer Noble jer.no...@apple.com wrote: WebKit is in the process of implementing Mozilla's proposed Full Screen API https://wiki.mozilla.org/Gecko:FullScreenAPI. Basic full screen support is available in WebKit Nightlies http://nightly.webkit.org/ on Mac and Windows (other ports are adding support as well), and can be enabled through user defaults (WebKitFullScreenEnabled=1). To test the feasibility of this API, we have mapped the full screen button in the default controls in video elements to this new API. The webkit-only webkitenterfullscreen() method on HTMLMediaElement has also been mapped to this new API. In so doing, we have been able to collect test case results from live websites. In this process, I believe we have uncovered a number of issues with the API proposal as it currently stands that I'd like to see addressed. 1. Z-index as the primary means of elevating full screen elements to the foreground. The spec suggests that a full screen element is given a z-index of BIGNUM in order to cause the full screen element to be visible on top of the rest of page content. The spec also notes that it is possible for a document to position content over an element with the :full-screen pseudo-class, for example if the :full-screen element is in a container with z-index not 'auto'. In our testing, we have found that this caveat causes extreme rendering issues on many major video-serving websites, including Vimeo and Apple.com. In order to fix rendering under the new full-screen API to be on par with WebKit's existing full-screen support for video elements, we chose to add a new pseudo-class and associated style rule to forcibly reset z-index styles and other stacking-context styles. This is of course not ideal, and we have only added this fix for full screen video elements. This rendering quirk makes it much more difficult for authors to elevate a single element to full-screen mode without modifying styles on the rest of their page. Proposal: the current API proposal simply recommends a set of CSS styles. The proposal should instead require that no other elements render above the current full-screen element and its children, and leave it up to implementers to achieve that requirement. (E.g., WebKit may implement this by walking up the ancestors of the full-screen element disabling any styles which create stacking contexts.) That does seem more reasonable. A conceptual way to think of it is that the fullscreen element and all its children get a temporary z-index boost. 2. Animating into and out of full screen. WebKit's current video full-screen support will animate an element between its full-screen and non-full-screen states. This has both security and user experience benefits. However, with the current z-index-based rendering technique recommended by the proposed Full Screen API, animating the full-screen transition is extremely difficult. Proposal: The full-screen element should create a new view, separate from its parent document's view. This would allow the UA to resize and animate the view separate from the parent document's view. This would also solve issue 1 above. I'm not sure what view means but I can see what I think are Robert's issues. The DOM still has to be connected and CSS still has to flow. So if I have body(a(b(c))) and I fullscreen c then changes to CSS of body, a, and b should all still effect c even when c is in fullscreen mode. Of course maybe your idea of detaching into a separate view means doesn't imply that other elements. I guess conceptually I thought all fullscreen does is (1) make the browser window fullscreen with no chrome and (2) stretch the fullscreen element to fill that space. That implies that if I set the background color of the fullscreen element to rgba(0,0,0,0.5) I can see back to the non-fullscreen elements behind it. 3. fullscreenchange events and their targets. The current proposal states that a fullscreenchange event must be dispatched when a document enters or leaves full-screen. Additionally, when the event is dispatched, if the document's current full-screen element is an element in the document, then the event target is that element, otherwise the event target is the document. This has the side effect that, if an author adds an event listener for this event to an element, he will get notified when an element enters full screen, but never when that element exits full-screen (if the current full screen element is cleared, as it should be, before the event is dispatched.) In addition, if the current full-screen element is changed while in full screen mode (e.g. by calling requestFullScreen() on a different element) then an event will be dispatched to only one of the two possible targets. Proposal: split the fullscreenchange events into two: fullscreenentered and fullscreenexited (or some variation thereof) and fire each at the
Re: [whatwg] Embedding custom hierarchical data
Is X3DOM an example of solution to what you are trying to do? They embed hierarchical data directly in HTML docs through namespaces and have JavaScript use that data http://www.x3dom.org/ I believe Angular does this as well. http://angularjs.org/
Re: [whatwg] Canvas and drawWindow
Someone pointed out that once you have HTML5-Canvas-WebGL, even though you can't call readPixels or toDataURL or getImageData because of cross origin issues you can write a shader that takes longer depending on the color and then just time draw calls to figure out what's in the texture. In other words, if you want to prevent security issues you could only do this on same origin content. But then you open another can of worms. Once you can put content in a texture you want to be able to let the user interact with it (like they can with 3d css) but then you run into the issue that you don't know what the user's shaders are doing so you have to let JavaScript translate mouse coordinates which is probably another security issue on top of being a PITA to implement. On Fri, Mar 11, 2011 at 8:35 AM, Erik Möller emol...@opera.com wrote: I bet this has been discussed before, but I'm curious as to what people think about breathing some life into a more general version of Mozillas canvas.drawWindow() that draws a snapshot of a DOM window into the canvas? https://developer.mozilla.org/en/drawing_graphics_with_canvas#section_9 I know there are some security considerations (for example listed in the source of drawWindow): // We can't allow web apps to call this until we fix at least the // following potential security issues: // -- rendering cross-domain IFRAMEs and then extracting the results // -- rendering the user's theme and then extracting the results // -- rendering native anonymous content (e.g., file input paths; // scrollbars should be allowed) I'm no security expert, but it seems to me there's an easy way to at least cater for some of the use-cases by always setting origin-clean to false when you use drawWindow(). Sure it's a bit overkill to always mark it dirty, but it's simple and would block you from reading any of the pixels back which would address most (all?) of the security concerns. I'm doing a WebGL demo, so the use-case I have for this would be to render a same-origin page to a canvas and smack that on a monitor in the 3d-world. Intercept mouse clicks, transform them into 2d and passing them on would of course be neat as well and probably opens up the use-cases you could dream up. So, I'm well aware its a tad unconventional, but perhaps someone has a better idea of how something like this could be accomplished... i.e. via SVG and foreignObject or punching a hole in the canvas and applying a transform etc. I'd like to hear your thoughts. -- Erik Möller Core Developer Opera Software
Re: [whatwg] Workers feedback
On Mon, Feb 14, 2011 at 1:37 AM, Ian Hickson i...@hixie.ch wrote: On Fri, 11 Feb 2011, Gregg Tavares (wrk) wrote: On Fri, Feb 11, 2011 at 5:45 PM, Ian Hickson i...@hixie.ch wrote: On Fri, 11 Feb 2011, Gregg Tavares (wrk) wrote: On Fri, 7 Jan 2011, Berend-Jan Wever wrote: 1) To give WebWorkers access to the DOM API so they can create their own elements such as img, canvas, etc...? It's the API itself that isn't thread-safe, unfortunately. I didn't see the original thread but how is a WebWorker any different from another webpage? Those run just fine in other threads and use the DOM API. Web pages do not run in a different thread. Oh, sorry. I meant they run in a different process. At least in some browsers. The goal here is interoperability with all browsers, not just some. I guess I don't understand. There are lots of things all browsers didn't do at some point in the past. The video tag. CSS animation. The canvas tag. Etc. We don't say because it hasn't been done yet therefore we can't try or can't spec it. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Workers feedback
On Fri, Feb 4, 2011 at 3:43 PM, Ian Hickson i...@hixie.ch wrote: On Sat, 16 Oct 2010, Samuel Ytterbrink wrote: *What is the problem you are trying to solve?* To create sophisticated single file webpages. That's maybe a bit vaguer than I was hoping for when asking the question. :-) Why does it have to be a single file? Would multipart MIME be acceptable? A single file is a solution, not a problem. What is the problem? [...] trying to build a more optimal standalone DAISY player (would be nice if i could rewrite it with web workers). Now that's a problem. :-) It seems like what you need is a package mechanism, not necessarily a way to run workers without an external script. On Fri, 15 Oct 2010, Jonas Sicking wrote: Allowing both blob URLs and data URLs for workers sounds like a great idea. I expect we'll add these in due course, probably around the same time we add cross-origin workers. (We didn't add them before because exactly how we do them depends on how we determine origins.) On Sat, 16 Oct 2010, Samuel Ytterbrink wrote: But then i got another problem, why is not file:///some_directory_where_the_html_are/ not the same domain as file:///some_directory_where_the_html_are/child_directory_with_ajax_stuff/. I understand if it was not okay to go closer to root when ajax, file:///where_all_secrete_stuff_are/ or /../../. That's not a Web problem. I recommend contacting your browser vendor about it. (It's probably security-related.) On Thu, 30 Dec 2010, Glenn Maynard wrote: On Thu, Dec 30, 2010 at 7:11 PM, Ian Hickson i...@hixie.ch wrote: Unfortunately we can't really require immediate failure, since there'd be no way to test it or to prove that it wasn't implemented -- a user agent could always just say oh, it's just that we take a long time to launch the worker sometimes. (Performance can be another hardware limitation.) Preferably, if a Worker is successfully created, the worker thread starting must not block on user code taking certain actions, like closing other threads. How can you tell the difference between the thread takes 3 seconds to start and the thread waits for the user to close a thread, if it takes 3 seconds for the user to close a thread? My point is from a black-box perspective, one can never firmly say that it's not just the browser being slow to start the thread. And we can't disallow the browser from being slow. That doesn't mean it needs to start immediately, but if I start a thread and then do nothing, it's very bad for the thread to sit in limbo forever because the browser expects me to take some action, without anything to tell me so. I don't disagree that it's bad. Hopefully browser vendors will agree and this problem will go away. If queuing is really necessary, please at least give us a way to query whether a worker is queued. It's queued if you asked it to start and it hasn't yet started. On Fri, 31 Dec 2010, Aryeh Gregor wrote: I've long thought that HTML5 should specify hardware limitations more precisely. We can't, because it depends on the hardware. For example, we can't say you must be able to allocate a 1GB string because the system might only have 500MB of storage. Clearly it can't cover all cases, and some sort of general escape clause will always be needed -- but in cases where limits are likely to be low enough that authors might run into them, the limit should really be standardized. It's not much of a standardised limit if there's still an escape clause. I'm happy to put recommendations in if we have data showing certain specific limits are needed for interop with real content. Unfortunately we can't really require immediate failure, since there'd be no way to test it or to prove that it wasn't implemented -- a user agent could always just say oh, it's just that we take a long time to launch the worker sometimes. (Performance can be another hardware limitation.) In principle this is so, but in practice it's not. In real life, you can easily tell an algorithm that runs the first sixteen workers and then stalls any further ones until one of the early ones exit, from an algorithm that just takes a while to launch workers sometimes. I think it would be entirely reasonable and would help interoperability in practice if HTML5 were to require that the UA must run all pending workers in some manner that doesn't allow starvation, and that if it can't do so, it must return an error rather than accepting a new worker. Failure to return an error should mean that the worker can be run soon, in a predictable timeframe, not maybe at some indefinite point in the future. All workers should run soon, not maybe in the future. Not running a worker should be an unusual circumstance. Errors that occur in unusual circumstances aren't errors that authors will check for. This dicussion comes from Chrome
Re: [whatwg] Workers feedback
On Fri, Feb 11, 2011 at 5:45 PM, Ian Hickson i...@hixie.ch wrote: On Fri, 11 Feb 2011, Gregg Tavares (wrk) wrote: On Fri, 7 Jan 2011, Berend-Jan Wever wrote: 1) To give WebWorkers access to the DOM API so they can create their own elements such as img, canvas, etc...? It's the API itself that isn't thread-safe, unfortunately. I didn't see the original thread but how is a WebWorker any different from another webpage? Those run just fine in other threads and use the DOM API. Web pages do not run in a different thread. Oh, sorry. I meant they run in a different process. At least in some browsers. -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Canvas element image scaling
As others have pointed out, canvas scaling algorithm is not specified and is different in each browser. http://greggman.com/downloads/examples/canvas-test/test-01/canvas-test-01-results.html http://greggman.com/downloads/examples/canvas-test/test-01/canvas-test-01.html On Sat, Sep 18, 2010 at 7:51 PM, Rob Evans r...@mtn-i.com wrote: Thanks I'll give that a go in the morning! All the best, Rob On 19 Sep 2010 03:42, Boris Zbarsky bzbar...@mit.edu wrote: On 9/18/10 9:57 PM, Rob Evans wrote: Thanks for the reply. I’m already using high resolution ima... Gecko will scale canvas images in one of two ways: either using a nearest-neighbor algorithm or using a more complicated (bilinear, bicubic, may depend on other details) algorithm which is slower but usually gives better results. You can control which is happening by setting mozImageSmoothingEnabled on the canvas 2d context (set to false to get nearest-neighbor and set to true to get the other). The default value there is true. Does setting it to false give you the Chrome 6 behavior, perchance? I'd be a little surprised if it does, but worth trying. -Boris
Re: [whatwg] Scriptable interface for video element FullScreen mode
On Wed, Sep 22, 2010 at 9:09 AM, Shiv Kumar sku...@exposureroom.com wrote: I’ve changed the subject of this post in the hopes that it receives the correct attention… As per the current spec: quote WARNING! User agents should not provide a public API to cause videos to be shown full-screen. A script, combined with a carefully crafted video file, could trick the user into thinking a system-modal dialog had been shown, and prompt the user for a password. There is also the danger of mere annoyance, with pages launching full-screen videos when links are clicked or pages navigated. Instead, user-agent-specific interface features may be provided to easily allow the user to obtain a full-screen playback mode. /quote In order for anyone to be able to provide their own skin/player, we’ll need to provide a scriptable way to switch the video element to full screen and out including events to support the same. I know Webkit folks have provided, webkitEnterFullScreen/webkitExitFullScreen methods to allow this. I don’t believe there are any events supporting the change in state, however I think it’s important that the video element provide a scriptable way to do this. Internally, the UA can determine if the call was made using a user gesture as I believe Webkit is doing. Can we agree to change the current spec to allow for this? Shiv http://exposureroom.com Is this proposal not good enough? https://wiki.mozilla.org/index.php?title=Gecko:FullScreenAPI It handles the case I think you want. I also it's also useful for other things like HTML5 games. It's also secure or at least as secure as flash. The browser will not go fullscreen without either a prompt or a mouse click.
Re: [whatwg] Canvas: clarification of compositing operations needed
Even Firefox's implementation is inconsistent. drawShape uses the infinite transparent black bitmap but drawImage does not. I believe even many at Mozilla would like Firefox to switch to the Chrome/Safari method because it's more easily GPU accelerated. In that direction it would be nice if 2 things in the spec changed #1) Get rid of the infinite transparent black bitmap stuff and change it to something that say only pixels inside the shape/image are effected #2) Change the globalCompositingOperation spec from referencing PORTER-DUFF to referencing OpenGL source-over glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_ALPHA); source-in glBlendFunc(GL_DST_ALPHA, GL_ZERO); source-out glBlendFunc(GL_ONE_MINUS_DST_ALPHA, GL_ZERO); source-atop glBlendFunc(GL_DST_ALPHA, GL_ONE_MINUS_SRC_ALPHA); destination-over glBlendFunc(GL_ONE_MINUS_DST_ALPHA, GL_ONE); destination-in glBlendFunc(GL_ZERO, GL_SRC_ALPHA) destination-out glBlendFunc(GL_ZERO, GL_ONE_MINUS_SRC_ALPHA); destination-atop glBlendFunc(GL_ONE_MINUS_DST_ALPHA, GL_SRC_ALPHA); lighter glBlendFunc(GL_ONE, GL_ONE); darker deprecated copy glBlendFunc(GL_ONE, GL_ZERO); xor glBlendFunc(GL_ONE_MINUS_DST_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
Re: [whatwg] An API to resize and rotate images client-side
On Mon, Jul 26, 2010 at 6:34 PM, Ian Hickson i...@hixie.ch wrote: On Thu, 20 May 2010, David Levin wrote: Twice when this was brought up on whatwg developers out of the blue mentioned that the image resizing was a useful thing for them (once early in this thread and once long ago when canvas in workers was brought up). In addition to that anecdotal evidence, here are several other places this comes up which I can list quickly: - For example, take Facebook. If I upload a huge photo to Facebook, it seems to upload the whole thing and then resizes it on the server (down to something much smaller than 1600 X 1200). - This is similar for other social sites like dating sites or Orkut that only allow a maximum size of photo. Typically, either they force the user to resize the image (which is a horrible experience) or they resize the image on the client using gears (with workers and canvas) or flash, etc. (or canvas but for more than one browser that may hang the UI). - Similarly Gmail now allows dragging images into email (http://gmailblog.blogspot.com/2010/05/drag-images-into-messages.html). The full resolution image isn't necessary for this. It would be better to have a resized image. - Something like Google Docs or Wave which show real time participation of other people typing would benefit from getting a thumbnail of an inserted image to other people in the conversation. (One could envision this for any real time chat/communication website.) - When you upload photos to picasaweb from the Picasa client, it offers to resize them to 1600X1200 before uploading them. Also, it offers an option to upload a thumbnail first before uploading the bigger picture, so the album can appear even quicker (just at a really low resolution). Ideally, a website could do something similar. On Tue, 25 May 2010, David Levin wrote: http://webkit.org/demos/canvas-perf/canvas.html Firefox 3.7a4 (no D2D) Direct image copy: 39ms Indirect copy with (via ImageData): 160ms Copy with 2x scale: 646.5ms Copy with 0.5x scale: 42.5ms Copy with rotate: 358ms Firefox 3.7a4 (D2D) Direct image copy: 115ms Indirect copy with (via ImageData): 365.5ms Copy with 2x scale: 246ms Copy with 0.5x scale: 48.5ms Copy with rotate: 100.5ms Chrome 4.1.249.1064 (45376) Direct image copy: 32.5ms Indirect copy with (via ImageData): 207.5ms Copy with 2x scale: 378.5ms Copy with 0.5x scale: 27.5ms Copy with rotate: 367ms While the GPU does help in some scenarios, unfortunately it must still take some time to do its work, so it doesn't enable us to do sync apis that don't hang the UI. The logical conclusion is that we should make one of the following choices: 1. Provide dedicated asynchronous APIs for this use case. For example, a method to go from an image URL to a Blob representing that image at a different size and/or rotation. 2. Provide generic APIs for that can handle this use case amongst others. For example, porting the 2D canvas API to workers. I'd like to add I think there's a lot more enthusiasm for #2 because there are many other situations that need to do CPU intensive computing without bogging down the main browser thread. I'm not knocking #1. I'm just suggesting that there are more champions for #2 since it solves problems for more people. Any pointers to why this hasn't happened already? I'm guessing there have already been giant debates on how to share data between workers and the main thread and or how to expose more services to workers. 3. Not address this use case (yet). For #1, my preference would be to add two methods to img, one to make img resize and rotate the image and then fire 'load' again, and one to obtain the current image as a Blob, much like toDataURL() on canvas (where a similar toBlob() would also be useful). For #2, we'd need to provide an Image object (a non-DOM version of HTMLImageElement) and a Canvas object (a non-DOM version of HTMLCanvasElement) in workers. But unless we have a critical mass of browser vendors agreed on one of these approaches, we have to default to #3. Currently it seems only the Chrome team is especially interested in either #1 or #2. My recommendation, in the absence of enthusiasm from other browser vendors, would be for the Chrome team to experiment with #1 or #2. If I have misread the current situation and there is a willingness to implement #1 or #2 in more browsers, then please let me know. I'd be happy to spec one of those options. (#1 would be easy to do; #2 might take longer, since it is significantly more work.) -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
