Re: [whatwg] Workers
On Wed, Nov 12, 2008 at 3:35 PM, Ian Hickson [EMAIL PROTECTED] wrote: (cc'ed whatwg -- sorry if that wasn't what you intended) On Wed, 27 Aug 2008, Michael Nordman wrote: Is it possible for a worker (shared or dedicated) to reload itself? Not currently. For dedicated workers, this can effectively be accomplished already by one of the two means of close()ing a worker and then starting a new one and then handing out ports as needed. But for shared workers, this is a more interesting question. Provided clients of the shared worker (those that have a reference to it) can be relied on to restart it upon onclose(), reload() would effectively happen. Are there any windows of time where a shared worker executes w/o a client having a reference it? Yes. Any worker can survive, e.g. if it has timers running, until its parent window (or any window that it ever spoke to) closes the document it is associated with. I don't really see the use case for self-reloading. Do scripts self-reload in general? For people who want to update script in SharedWorker, could they do it by making the SharedWorker merely a shell? The SharedWorker would create a Worker to do everything. If a reload is needed, then the SharedWorker closes the Worker and starts it again (just like the dedicated worker method described above). Dave
Re: [whatwg] Questions on Workers spec
It looks like all of this is there in a just published update of the spec. On Wed, Nov 12, 2008 at 11:33 AM, Dmitry Titov [EMAIL PROTECTED] wrote: Hi, I have couple of questions about Workers: 1. The sample code looks as if setTimeout/clearTimeout/setInterval/clearInterval should be available to Workers (as methods of WorkerUtils?) but they are not explicitly specified on any interface. Should they be there? 2. It seems workers should be able to create workers (including creating 'themselves' in case of SharedWorker). It is especially useful for a SharedWorker to create dedicated workers - since one of the popular scenarios for it is likely a state container that communicates with UI pages while using dedicated workers to do other operations. Will spec include this? Thanks Dmitry
Re: [whatwg] AppCache and SharedWorkers?
On Wed, Mar 25, 2009 at 3:01 PM, Drew Wilson atwil...@google.com wrote: On Wed, Mar 25, 2009 at 2:11 PM, Michael Nordman micha...@google.comwrote: The appcache spec has changed since the ian and i sent these old messages. Child browsing contexts (nested iframes) no longer inherit the appcache of their parent context (frame) by default. How's this for a starting point for how these things intereract... * Dedicated worker contexts should be associated with an appcache according to the same resource loading and cache selection logic used for child browsing contexts. (So just like navigating an iframe). Since dedicated workers are tightly tied (1:1) with a specific top-level browsing context, I'd say that they should use the same appcache as the document that started them. * Shared (or persistent) worker contexts should be associated with an appcache according to the same resource loading and cache selection logic used for top-level browsing contexts. (So just like navigating a window.) That may make sense for Shared workers, I think. For persistent workers I think this is a problem - persistent workers need a way to manage their own app cache, since they are not guaranteed to have any open windows/documents associated with them. My concern about this is that app cache manifests are only specified via manifest html tags, which makes them only applicable to HTML documents (you can't associate a manifest with a worker since there's no document to put the manifest tag in). At least one question, I'm sure there are others... What does a shared (or persistent) worker do when the appcache its associated with is updated? Is there a way to reload itself with the new script in the latest version of the appcache? What about message ports between the worker and other contexts? One could imagine that the worker would reload its javascript via importScripts(). It kind of assumes that the script is idempotent, though. Similarly one could use nested workers (which I like because it gives the new script a new global object). The shared/persistent worker would start a nested worker. Then for a reload, it could shut down the current nested worker and start up a new one. Regarding message ports, it would be up to the implementation to decide if the shared/persistent worker followed a pointer to implementation pattern or if it handed out message ports directly to the nested worker. Dave -atw
Re: [whatwg] Installed Apps
It sounds like most of the concerns are about the 2nd part of this proposal: allowing a background page to continue running after the visible page has been closed. However, the first part sounds like it alone would be useful to web applications like GMail: The first, which should begenerally useful, is the ability to have a hidden HTML/JS page running in the background that can access the DOM of visible windows. This page should be accessible from windows that the user navigates to. We call this background Javascript window a shared context or a background page. This will enable multiple instances of a web app (e.g. tearoff windows in Gmail) to cleanly access the same user state no matter which windows are open. + restrict things to the same security origin. It sounds similar in concept to a share worker except that it runs in the main thread and is more concerned with dom manipulation/state while workers have typically been thought of as allowing background processing. It seems that the lifetime of this could be scoped, so that it dies when it isn't referenced (in a similar way to how shared worker lifetime is scoped). Dave On Mon, Jul 27, 2009 at 6:39 PM, Maciej Stachowiak m...@apple.com wrote: On Jul 27, 2009, at 7:13 PM, Aryeh Gregor wrote: I'm not clear how the UI requirements here are different from persistent workers, though. Those also persist after the user navigates away, right? Persistent workers are even more of a security risk, since they are supposed to persist even after the browser has been restarted, or after the system has been rebooted. Persistent workers should be renamed to BotNet Construction Kit. Regards, Maciej
Re: [whatwg] Installed Apps
It feels like this has become a discussion of which dangerous feature is more dangerous Several browsers (or browser like things) have mechanisms for allowing the installation of potentially dangerous things. For example, FireFox has the extension install mechanism. Google Chrome has/must have something for extensions. There is Prism and various html desktop gadget engines. So far, it sounds like many folks are saying these persistent contexts belong more to this domain. Is it interesting discussing the api/behavior for things exposed in that domain? dave On Tue, Jul 28, 2009 at 9:58 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Wed, Jul 29, 2009 at 4:47 PM, Michael Davidson m...@google.com wrote: I agree 100%. I'm only trying to argue that from a user perspective, access that we currently have acceptable UI for, e.g. camera hardware, is about as scary as agreeing to let a web app run in the background. The consequences of a malicious app that gets either permission are dire. One difference is that photos of you in your underwear are probably not easily monetizable, whereas botnets are, so you'll see a lot more exploitation of features that let you create botnets. Rob -- He was pierced for our transgressions, he was crushed for our iniquities; the punishment that brought us peace was upon him, and by his wounds we are healed. We all, like sheep, have gone astray, each of us has turned to his own way; and the LORD has laid on him the iniquity of us all. [Isaiah 53:5-6]
Re: [whatwg] Security risks of persistent background content (Re: Installed Apps)
On Thu, Jul 30, 2009 at 10:18 AM, Michael Davidson m...@google.com wrote: On Tue, Jul 28, 2009 at 10:58 PM, Maciej Stachowiakm...@apple.com wrote: Here's some security risks I've thought about, for persistent workers and persistent background pages: great list of risks Thanks for the list, Maciej. However, Firefox extensions today have all of the same problems. Do you consider the permission UI in Firefox insufficient? Given Safari's extension model, I'm going to guess the answer is yes. The fact that FF has extensions, however, at least shows that one browser vendor believes that sufficient permission UI exists. Maciej said this at the end of original email. I do think offering a feature like this in the context of an application or extension style install experience might be acceptable - specifically an experience that is explicitly initiated by the user with multiple affirmative steps. But web features are not usually designed around such an expectation, usually this is the hallmark of a proprietary platform, at times also including central vetting and revocation capabilities. Does that answer your question?
Re: [whatwg] HTMLCanvasElement.toFile()
It seems like it the method should be toBlob. This doesn't address my concern that you won't know the mime type of the blob returned. This makes a good case to move the readonly attrbiute DOMString type from File to Blob. dave
Re: [whatwg] Notification API
On Wed, Feb 3, 2010 at 3:25 PM, Jonas Sicking jo...@sicking.cc wrote: On Wed, Feb 3, 2010 at 3:18 PM, Robert O'Callahan rob...@ocallahan.org wrote: On Thu, Feb 4, 2010 at 11:34 AM, Drew Wilson atwil...@google.com wrote: 2) Allow more full-featured HTML notifications on the overwhelming majority of platforms that support them. Given that Mac and Linux don't support HTML notifications, what platforms are those? Additionally, how full fledged HTML are we talking? Do you want script to be running in the notification? Should forms work? Plugins? HTML is a very powerful format, not really the best choice if all you need is some basic text formatting. I was about to comment, but then I realized this is all on the wrong alias (whatwg instead of public-webapps) as Drew pointed out a little earlier. dave
[whatwg] Offscreen canvas (or canvas for web workers).
I've talked with some other folks on WebKit (Maciej and Oliver) about having
a canvas that is available to workers. They suggested some nice
modifications to make it an offscreen canvas, which may be used in the
Document or in a Worker.
Proposal:
Introduce an OffscreenCanvas which may be created from a Document or a
Worker context.
interface OffscreenCanvas {
attribute unsigned long width;
attribute unsigned long height;
DOMString toDataURL (in optional DOMString type, in any... args);
object getContext(in DOMString contextId);
};
When it is created in the Worker context, OffscreenCanvas.getContext(2d)
returns a CanvasWorkerContext2D. In the Document context, it returns a
CanvasRenderingContext2D.
The base class for both CanvasWorkerContext2D and CanvasRenderingContext2D
is CanvasContext2D. CanvasContext2D is just like a CanvasRenderingContext2D
except for omitting the font methods and any method which uses HTML
elements. It does have some replacement methods for createPattern/drawImage
which take an OffscreenCanvas. The canvas object attribute is either a
HTMLCanvasElement or an OffscreenCanvas depending on what the canvas context
came from.
interface CanvasContext2D {
readonly attribute object canvas;
void save();
void restore();
void scale(in float sx, in float sy);
void rotate(in float angle);
void translate(in float tx, in float ty);
void transform(in float m11, in float m12, in float m21, in float
m22, in float dx, in float dy);
void setTransform(in float m11, in float m12, in float m21, in float
m22, in float dx, in float dy);
attribute float globalAlpha;
attribute [ConvertNullToNullString] DOMString
globalCompositeOperation;
CanvasGradient createLinearGradient(in float x0, in float y0, in
float x1, in float y1)
raises (DOMException);
CanvasGradient createRadialGradient(in float x0, in float y0, in
float r0, in float x1, in float y1, in float r1)
raises (DOMException);
CanvasPattern createPattern(in OffscreenCanvas image, in DOMString
repetition);
attribute float lineWidth;
attribute [ConvertNullToNullString] DOMString lineCap;
attribute [ConvertNullToNullString] DOMString lineJoin;
attribute float miterLimit;
attribute float shadowOffsetX;
attribute float shadowOffsetY;
attribute float shadowBlur;
attribute [ConvertNullToNullString] DOMString shadowColor;
void clearRect(in float x, in float y, in float width, in float
height);
void fillRect(in float x, in float y, in float width, in float
height);
void strokeRect(in float x, in float y, in float w, in float h);
void beginPath();
void closePath();
void moveTo(in float x, in float y);
void lineTo(in float x, in float y);
void quadraticCurveTo(in float cpx, in float cpy, in float x, in
float y);
void bezierCurveTo(in float cp1x, in float cp1y, in float cp2x, in
float cp2y, in float x, in float y);
void arcTo(in float x1, in float y1, in float x2, in float y2, in
float radius);
void rect(in float x, in float y, in float width, in float height);
void arc(in float x, in float y, in float radius, in float
startAngle, in float endAngle, in boolean anticlockwise);
void fill();
void stroke();
void clip();
boolean isPointInPath(in float x, in float y);
void drawImage(in OffscreenCanvas image, in float dx, in float dy,
in optional float dw, in optional float dh);
void drawImage(in OffscreenCanvas image, in float sx, in float sy,
in float sw, in float sh, in float dx, in float dy, in float dw, in float
dh);
// pixel manipulation
ImageData createImageData(in float sw, in float sh)
raises (DOMException);
ImageData getImageData(in float sx, in float sy, in float sw, in
float sh)
raises(DOMException);
void putImageData(in ImageData imagedata, in float dx, in float dy,
in optional float dirtyX, in optional float dirtyY, in optional float
dirtyWidth, in optional float dirtyHeight]);
};
interface CanvasWorkerContext2D : CanvasContext2D {
};
interface CanvasRenderingContext2D : CanvasContext2D {
CanvasPattern createPattern(in HTMLImageElement image, in DOMString
repetition);
CanvasPattern createPattern(in HTMLCanvasElement image, in
DOMString repetition);
CanvasPattern createPattern(in HTMLVideoElement image, in DOMString
repetition);
// focus management
boolean drawFocusRing(in Element element, in float xCaret, in float
yCaret, in optional boolean canDrawCustom);
// text
attribute DOMString font;
attribute DOMString textAlign;
Re: [whatwg] Offscreen canvas (or canvas for web workers).
On Mon, Feb 22, 2010 at 11:57 AM, Drew Wilson atwil...@google.com wrote: Do we feel that text APIs are, in general, difficult to implement in a multi-thread safe manner? On Mon, Feb 22, 2010 at 11:51 AM, Michael Nordman micha...@google.com wrote: The lack of support for text drawing in the worker context seems like a short sighted mistake. On Mon, Feb 22, 2010 at 2:35 PM, Jeremy Orlow jor...@chromium.org wrote: It does indeed seem pretty short sighted. On Mon, Feb 22, 2010 at 2:48 PM, Maciej Stachowiak m...@apple.com wrote: 1) Like others, I would recommend not omitting the text APIs. I can see why they are a bit trickier to implement than the others, but I don't see a fundamental barrier. I did want to add the text apis but it does add implementation difficulties for WebKit (and as I understand Firefox). However, they are part of what people want and it does simplify the interfaces, so done. On Mon, Feb 22, 2010 at 2:48 PM, Maciej Stachowiak m...@apple.com wrote: 2) I would propose adding createPattern and drawImage overloads that take an OffscreenCanvas. The other overloads would in practice not be usefully callable in the worker case since you couldn't get an image, canvas or video element. 3) This would leave the only difference between the two interfaces as the drawFocusRing method. This would not be usefully callable in a worker, since there would be no way to get an Element. But it doesn't seem worth it to add an interface just for one method's worth of difference. Sounds good. On Mon, Feb 22, 2010 at 3:10 PM, Jonas Sicking jo...@sicking.cc wrote: What is the use case for this? It seems like in most cases you'll want to display something on screen to the user, and so the difference comes down to shipping drawing commands across the pipe, vs. shipping the pixel data. Apologies for not including this at the start. As now mentioned in several places in the thread, the simplest use case is resize/rotate of images. However, more complex use cases may involve heavy users of canvas who would like to either prerender the canvas and/or move the time taken to generate the canvas off of the main thread. While it is true that simple uses of canvas would not get a performance win out of this, if you are doing many canvas operations, (like many complex operations) it is faster to copy a result than it is to generate it. Ideally canvas will get some form of toBlob which will allow the image to be appropriately compressed as well which will also reduce the number of bits that need to be copied from the worker to the main thread (as well as make for a smaller upload than the raw bits). Lastly, in the future, one can see other uses for the OffscreenCanvas including WebGL for workers which several folks (Maciej and Jonas) have expressed interest about on this thread. dave
Re: [whatwg] canvas feedback
On Thu, Mar 11, 2010 at 9:16 PM, Ian Hickson i...@hixie.ch wrote: On Mon, 22 Feb 2010, David Levin wrote: I've talked with some other folks on WebKit (Maciej and Oliver) about having a canvas that is available to workers. They suggested some nice modifications to make it an offscreen canvas, which may be used in the Document or in a Worker. What are the use cases? The simplest is resize/rotate for large images. However there are more advanced uses in which the page heavily uses canvas. Since there are many canvas operations, they take a noticeable amount of time, and it would be better than they are done on a different thread than the main one. Another use related use case is when a page needs to render multiple canvases and each one is involved. (One could think of some sort of animation.) It would also be nice if they could do multiple operations in parallel (using multiple workers for instance.)
Re: [whatwg] Offscreen canvas (or canvas for web workers).
On Fri, Mar 12, 2010 at 12:16 PM, Jonas Sicking jo...@sicking.cc wrote: On Fri, Mar 12, 2010 at 11:57 AM, David Levin le...@google.com wrote: On Mon, Feb 22, 2010 at 3:10 PM, Jonas Sicking jo...@sicking.cc wrote: What is the use case for this? It seems like in most cases you'll want to display something on screen to the user, and so the difference comes down to shipping drawing commands across the pipe, vs. shipping the pixel data. Apologies for not including this at the start. As now mentioned in several places in the thread, the simplest use case is resize/rotate of images. As Hixie pointed out, resize/rotate images do not seem solved by this API. In order to resize an image with this API you need to: 1. Load the image into an img 2. Copy the image into a canvas 3. Extract an ImageData from the canvas 4. Send the ImageData to the worker thread 5. Import the ImageData into the worker thread canvas 6. Resize/rotate the image using the worker thread canvas 7. Extract an ImageData from the worker thread canvas 8. Send the ImageData to the main thread 9. Import the ImageData into a main thread canvas And if you want to send the resized image to the server: 10. Extract the data in a serialized format from the canvas 11. Send using XHR. Just looking at just the work happening on the main thread it sounds like just resizing/rotating on the main thread is faster. Not to mention much less complex. I'm not saying that the proposed API is bad. It just doesn't seem to solve the (seemingly most commonly requested) use case of rotating/scaling images. So if we want to solve those use cases we need to either come up with a separate API for that, or extend this proposal to solve that use case somehow. If fromBlob and toBlob were on canvas, it gets rid of steps 1-3 and changes step 4 to be send file to worker thread. I simply didn't include fromBlob/toBlob because toBlob was already being discussed in another thread. I thought it best to let that topic get discussed in parallel, but it is part of this whole thing, so I am interested in that happening (and discussing those apis further). So it looks like this: 1. Send the File to the worker thread 2. Import the File/blob into the worker thread canvas 3. Resize/rotate the image using the worker thread canvas (to thumbnail for instance) 4. Extract a blob from the worker thread canvas Either 5. Send the blob using XHR in the worker. or 5. Send the Blob to the main thread 6. Import the Blob into a main thread canvas (or both). Given the blob support this would be overall a better user experience because the loading of the image is done in the worker as well as the resize to a much smaller size, so the i/o happening on the main thread is much lower overall. dave
Re: [whatwg] Offscreen canvas (or canvas for web workers).
On Fri, Mar 12, 2010 at 2:35 PM, Jonas Sicking jo...@sicking.cc wrote: On Fri, Mar 12, 2010 at 12:46 PM, Oliver Hunt oli...@apple.com wrote: On Mar 12, 2010, at 12:16 PM, Jonas Sicking wrote: I'm not saying that the proposed API is bad. It just doesn't seem to solve the (seemingly most commonly requested) use case of rotating/scaling images. So if we want to solve those use cases we need to either come up with a separate API for that, or extend this proposal to solve that use case somehow. Just for reference I think one thing that people are forgetting that there is a difference between being computationally faster, and being more responsive. As I mentioned in my email, if you look at the steps listed, enough of them happen *on the main thread* that you're spending far more of the main threads CPU cycles than you'd like. Possibly even more than doing all the resizing on the main thread. With the other improvements suggested by David things do definitely look different, but those are not in a proposal yet. There is the other scenario I mentioned, but I'll see what I can do about separately working up a proposal for adding those methods because they were next on my list to deal with. (fromBlob/load may be enough for this.) dave
Re: [whatwg] Image resize API proposal
Thanks for your feedback Gregg. On Thu, May 13, 2010 at 3:32 PM, Gregg Tavares g...@google.com wrote: This really seems like the wrong solution. Taken to an extreme next you'll need to add VideoRisizer, AudioRecompresser, and any thing else JavaScript can't do without freezing the browser. It seems like it would be better to figure out a way to get Web Workers to be able to do this. This has been explored. See Alternatives considered in the original proposal. Even if they have to XHR the binary down, decompress into a TypeArray (see WebGL) and read the data themsevles so they can keep the EXIF stuff, bloating the browser for one small use doesn't seem like the right solution. Small use is a relative term. In that there are a lot of web properties that either do this or want to do this right now. You can post an open source library and the usage can be as simple as this proposal. Then you would have to include the open source library as part of your web application which is suboptimal. On Tue, May 11, 2010 at 11:58 AM, Sterling Swigart sswig...@google.comwrote: 2. Or, limit the size of an image file before uploading it to a web server. This use case is already handled (minus the EXIF). How? (Note limit the size. If you mean using canvas, it may hang the UI for the user which is a really bad experience. dave
Re: [whatwg] Image resize API proposal
On Thu, May 20, 2010 at 11:30 AM, Maciej Stachowiak m...@apple.com wrote: I'm not clear on why this API is needed. ... This API seems much less general than offscreen canvas, so it's subject to the same criticism and you can't even make the argument that it also serves other use cases. The criticism for the OffscreenCanvas was 1. it made the core use case of image resizing rather complicated and 2. depending on the browser's implementation, there may be faster ways to do the image resizing than doing it on a worker. The net suggestion that This supports the idea of specialized API... -- http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-March/025590.html This is exactly what we are doing (and it addresses those criticisms). Thanks, dave
Re: [whatwg] Image resize API proposal
On Thu, May 20, 2010 at 7:55 PM, Maciej Stachowiak m...@apple.com wrote:
On May 20, 2010, at 6:24 PM, Robert O'Callahan wrote:
On Fri, May 21, 2010 at 12:50 PM, Maciej Stachowiak m...@apple.com wrote:
I'd also love to hear from Mike Shaver and others from the original thread
what they think of this API proposal.
I think Shaver's feedback still applies: on any device with a GPU, we can
optimize canvas and/or rendering enough that scaling images is no problem on
the main thread. So this API would have a short useful lifetime ... possibly
negative.
Coming thread full circle, I now think there are good use cases for
off-main-thread canvas, in particular with WebGL, but I guess that deserves
its own thread again :-).
As far as I can tell, the proposed API doesn't enable resizing off the main
thread. It is an API on HTMLImageElement so you can't call it from a worker.
And it returns synchronously so the main thread has to block until the
resize is done regardless. It returns a Blob, so I suppose it may be
possible that the intent is to make a magical blob that's actually backed by
a background thread asynchronous computation instead of data. That wasn't
clear to me from the proposal. It would be a little weird.
The proposed api has a callback for the blob, so it is totally async. We
attempted to return the blob sync (and do everything hidden in the
background), but blob.size wouldn't be known (so it would need to throw or
block, etc.)
Sample code from the proposal:
var successEvt = function (newBlob) { myDiv.innerHTML += img src=' +
newBlob.url + ' /; };
i.getBlob(image/jpeg, 300, 350, successEvt);
dave
Regards,
Maciej
Re: [whatwg] Image resize API proposal
On Thu, May 20, 2010 at 5:48 PM, Maciej Stachowiak m...@apple.com wrote: On May 20, 2010, at 1:00 PM, David Levin wrote: On Thu, May 20, 2010 at 11:30 AM, Maciej Stachowiak m...@apple.com wrote: I'm not clear on why this API is needed. ... This API seems much less general than offscreen canvas, so it's subject to the same criticism and you can't even make the argument that it also serves other use cases. The criticism for the OffscreenCanvas was 1. it made the core use case of image resizing rather complicated and 2. depending on the browser's implementation, there may be faster ways to do the image resizing than doing it on a worker. The net suggestion that This supports the idea of specialized API... -- http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-March/025590.html This is exactly what we are doing (and it addresses those criticisms). Is the purpose of this API performance or convenience? Ideally performance and convenience. It seems like the proposed API is so specialized that it's only really useful to resize an image immediately before transferring it over the network in some way. Since you get back a blob, you could use blob.url to put the result in into css. As a css background, it may be nice to have a resized image (and one that has been rotated/flipped according to the exif info). Is that really so much more common than any other resizing that it needs a specialized convenience API? * * Twice when this was brought up on whatwg developers out of the blue mentioned that the image resizing was a useful thing for them (once early in this thread and once long ago when canvas in workers was brought up). In addition to that anecdotal evidence, here are several other places this comes up which I can list quickly: - For example, take Facebook. If I upload a huge photo to Facebook, it seems to upload the whole thing and then resizes it on the server (down to something much smaller than 1600 X 1200). - This is similar for other social sites like dating sites or Orkut that only allow a maximum size of photo. Typically, either they force the user to resize the image (which is a horrible experience) or they resize the image on the client using gears (with workers and canvas) or flash, etc. (or canvas but for more than one browser that may hang the UI). - Similarly Gmail now allows dragging images into email ( http://gmailblog.blogspot.com/2010/05/drag-images-into-messages.html). The full resolution image isn't necessary for this. It would be better to have a resized image. - Something like Google Docs or Wave which show real time participation of other people typing would benefit from getting a thumbnail of an inserted image to other people in the conversation. (One could envision this for any real time chat/communication website.) - When you upload photos to picasaweb from the Picasa client, it offers to resize them to 1600X1200 before uploading them. Also, it offers an option to upload a thumbnail first before uploading the bigger picture, so the album can appear even quicker (just at a really low resolution). Ideally, a website could do something similar. dave
Re: [whatwg] Image resize API proposal
On Thu, May 20, 2010 at 6:24 PM, Robert O'Callahan rob...@ocallahan.orgwrote: On Fri, May 21, 2010 at 12:50 PM, Maciej Stachowiak m...@apple.com wrote: I'd also love to hear from Mike Shaver and others from the original thread what they think of this API proposal. I think Shaver's feedback still applies: on any device with a GPU, we can optimize canvas and/or rendering enough that scaling images is no problem on the main thread. So this API would have a short useful lifetime ... possibly negative. There are a few issues here: 1. This only applies when you can accelerate with a GPU. Not all devices may support this. 2. This only applies to browsers that implement the acceleration with a GPU. When Mike Shaver mentioned this, he referred to a Windows version of Firefox. It is unclear if Firefox supports this on any other platform nor does it seem that all other browsers will support the accelerated canvas in the near-ish future. 3. The gpu results are due to the fact that the operation is done async from the call (which is great as far as not hanging the UI until you try to get the data out of the canvas, which leads to...). 4. Even with gpu acceleration, in order to use the result in an xhr, one has to get back the result from the gpu and this is a more expensive operation (because getting the data out of the gpu is slow) as indicated by the indirect copy results from Firefox and forces the completion of all of the operations that were being done async. In short the gpu accelerated canvas seems like it may be a ways away still on all platforms, and it is unclear if it will provide the desired benefit. dave
Re: [whatwg] Image resize API proposal
Thanks for all the feedback. We've gotten into a lot of details about this proposal for image resizing (without hanging the UI), so I'd like to step back to a summary of the current state: 1. We've presented several use cases which demonstrate many websites which would benefit from this feature. In fact, twice when this has come up, there have been web developers who mentioned a need for this. This mirrors my experience in which I've talked to at least 4 different teams that would use this functionality now if it were available. 2. We've presented a canvas solution that would utilize workers, but as Mike Shaver ( http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-March/025590.html) indicated, different trade-offs among UAs supports the idea of specialized API. 3. We explored 7 possible approaches for a more specialized api and then presented the one which seemed best (as well as mentioned the other 6 and the reasons that they were not our primary). 4. We've discussed the leading alternate proposal optimized canvas (plus js to read the exif information) and then get the bits out of canvas, but there are several issues with this proposal including - that not all browsers will have an implementation using the gpu that allows web sites to use this and not hang the UI - that even if it was implemented everywhere, this solution involves readback from the GPU which, as Chris mentioned, is generally evil and should be avoided at all costs. At this point, it seems clear that the image resizing scenario is worth solving due to the number of places that it will benefit and that the api presented at the beginning of this thread solves it in a manner that allows the UA to optimize the operation. dave PS The proposed api is narrow (but it is just one api so perhaps that is good). In essence, that was part of the intent (following the guidance of the specialized api). For example, this api doesn't allow for getting an image for a pdf or word document. Also, the api makes it hard pull the thumbnail right out of the jpg when applicable (and this is a really nice optimization to avoid doing lots of unnecessary slow I/O). If folks have ideas about how to fix this, it would be interesting to hear.
Re: [whatwg] Image resize API proposal
On Mon, May 24, 2010 at 1:40 PM, Aryeh Gregor simetrical+...@gmail.comsimetrical%2b...@gmail.com wrote: On Mon, May 24, 2010 at 1:21 PM, David Levin le...@google.com wrote: We've discussed the leading alternate proposal optimized canvas (plus js to read the exif information) and then get the bits out of canvas, but there are several issues with this proposal including that not all browsers will have an implementation using the gpu that allows web sites to use this and not hang the UI This is a nonissue. There's no point in speccing one feature to work around the fact that browsers haven't implemented another -- it makes more sense to just get the browsers to implement the latter feature, making the former moot. Browsers look like they're moving toward GPU acceleration for everything now, and that has many more benefits, so we should assume that by the time they'd implement this API, they'll already be GPU-accelerated. that even if it was implemented everywhere, this solution involves readback from the GPU which, as Chris mentioned, is generally evil and should be avoided at all costs. This I'm not qualified to comment on, though. To the best of my knowledge, GPUs are magical boxes that make things go faster via pixie dust. ;) Thanks for your opinion. :) Chris is qualified so are other people whom I've spoken to who have said the same thing, so using the gpu is not pixie dust in this particular scenario even though folks would like to be believe it so.
Re: [whatwg] Image resize API proposal
I've improved the accuracy of the canvas (resize/copy/rotate) perf test: http://webkit.org/demos/canvas-perf/canvas.html Here are some results in a similar format to what Mike posted ( http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-March/025590.html): Firefox 3.7a4 (no D2D) Direct image copy: 39ms Indirect copy with (via ImageData): 160ms Copy with 2x scale: 646.5ms Copy with 0.5x scale: 42.5ms Copy with rotate: 358ms Firefox 3.7a4 (D2D) Direct image copy: 115ms Indirect copy with (via ImageData): 365.5ms Copy with 2x scale: 246ms Copy with 0.5x scale: 48.5ms Copy with rotate: 100.5ms Chrome 4.1.249.1064 (45376) Direct image copy: 32.5ms Indirect copy with (via ImageData): 207.5ms Copy with 2x scale: 378.5ms Copy with 0.5x scale: 27.5ms Copy with rotate: 367ms While the GPU does help in some scenarios, unfortunately it must still take some time to do its work, so it doesn't enable us to do sync apis that don't hang the UI. Feel free to run the test yourself and look at the code. Best wishes, Dave
Re: [whatwg] Image resize API proposal
On Tue, May 25, 2010 at 3:58 AM, Kornel Lesinski kor...@geekhood.netwrote: JPEG can be efficiently decoded at fraction of its size — without full decode and scale process. This process also needs only fraction of memory required for full scaling, which might matter on low-end mobile devices. Letting UA utilize this feature may give huge performance gains. Agreed. I could see this being done. Scaling isn't the only operation desirable — in some cases users might also want to crop the image (e.g., to upload only their face as an avatar), and cropping interface needs to be platform-specific — on touchscreen devices I'd rather use gestures than select-by-click'n'drag interface typical for desktop. ok. I think scaling of images before upload might be left completely up to UA. From site's perspective it could look like user simply selected scaled-down file. It could even be done declaratively — site could define desired size and whether user should be asked to crop the image: input type=file accept=image/* max-image-size=1000x1000 crop=allowed This appears to be so limited that it doesn't allow for the web page to display a preview of what is being uploaded. Take the gmail example from that I gave in this thread, where the image is dragged into an email and then the image is resized before uploading. Ideally, the image shown in the email would be the image that is being sent. Actually, I wish UAs offered scaling even for plain input type=file, because I don't expect every site with image upload to add extra code for resizing. I agree that having it be totally declarative would be simpler to write in a web page. dave
Re: [whatwg] Image resize API proposal
Summarizing We've found that a synchronous solution will likely lead to a bad user experience. As an alternative, we've presented an async apihttp://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-May/026292.htmlwhich solves this frequent use casehttp://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-May/026435.htmlfor web pages (which may involve many images at once) and allows the UA to optimize the operation (by using the GPU and/or multiple threads): image. getBlob(mimeType /* req */, width /* req */, height /* req */, successEvent /* req */, errorEvent /* op */, qualityLevel /* op */, preserveAspectRatio /* op */, rotateExif /* op */); dave
Re: [whatwg] whatwg Digest (and how it is full of political BS)
I believe this has already been addressed in that thread which I guess you'll get in the next digest. Now back to getting things done. On Mon, Jul 12, 2010 at 7:58 PM, Joseph Thomas jthom...@calpoly.edu wrote: Occasionally, I notice that a single conversation completely dominates my digest emails for this group. At the moment, it is Growing Pains, in the past it has been called other things, but it all amounts to the same thing: useless politics. Can we migrate these discussions to another list? Whatwg-politburro or some such? Maybe then there will be a place to discuss, you know, spec writing and features being worked on in peace. Instead, I have to scan through pages upon pages of WC3'ers whining about Hixie actually getting shit done.
Re: [whatwg] WebWorkers and images
On Fri, Jan 7, 2011 at 3:24 AM, Berend-Jan Wever skyli...@chromium.orgwrote: I ask because I wanted to port an image manipulation script to a WebWorker, but found out that WebWorkers have no way to use elements such as canvas and img. fwiw, ImageData can be used in a worker. Many folks have argued that canvas isn't that useful in a worker and that the gpu acceleration will make it less useful (and that most image manipulation would be able to use ImageData for its needs). dave
[whatwg] Proposal for IsSearchProviderInstalled / AddSearchProvider
Problem Although the default search provider may have a significant impact on a user’s web experience, it isn’t easy for users to set this. Ideally, a search engine should be able to offer the user the ability to easily use it as the default. Currently, there are two obstacles to this: 1. The search engine may not be able to detect that it already is the default, so it would offer this too broadly. 2. When a user decides to use it, they have to follow a set of complex instructions (http://www.google.com/search?q=switch+default+search+engines) Proposal Add two apis to window.external: IsSearchProviderInstalled AddSearchProvider IsSearchProviderInstalled(string url) returns * 2 if the origin in the given url is the default search provider * 1 if the origin in the given url is a search provider but not the default. * 0 otherwise If the given url doesn’t have the same security origin as the page, this api throws an access denied exception. (This makes the url redundant but it is kept to be consistent with the method exposed by IE 7.) AddSearchProvider(string openSearchUrl, [optional] bool asDefault) retrieves the open search document from openSearchUrl and decides in a UA specific manner whether to prompt the user about the change or addition. FAQ Doesn’t this already exist? Both of these api’s are present in IE7 (and later) except for the additional argument to AddSearchProvider (asDefault) and a change in the url's that may be queried. The pair of apis aren’t in a standard or many of the other browsers. Won’t the InstallSearchProvider api get abused? The home page setting for users has a similar value. Yet, the SetHomePage api available in IE does not seem to be abused significantly. Largely this will depend on the UA providing appropriate measures to protect users. In Chromium, we rely on a combination of user gesture (click, etc.) plus a dialog which clearly explains what is going to happen with the default being that nothing is changed ( https://sites.google.com/a/chromium.org/dev/developers/design-documents/chromium-search-provider-js-support ). Thanks, dave
Re: [whatwg] Proposal for IsSearchProviderInstalled / AddSearchProvider
On Tue, Feb 15, 2011 at 9:42 AM, Bjartur Thorlacius svartma...@gmail.comwrote: 2. When a user decides to use it, they have to follow a set of complex instructions (http://www.google.com/search?q=switch+default+search+engines ) Annoying implementation issue. http://bugzilla.mozilla.com/enter_bug.cgi mailto:implement...@lists.whatwg.org It is still complicated in all browsers, so I wasn't trying to point out flaws in any particular browser -- only that it is complicated and hard for users to do. On Tue, Feb 15, 2011 at 9:42 AM, Bjartur Thorlacius svartma...@gmail.com wrote: IsSearchProviderInstalled(string url) This seems like a slight privacy violation. Not a serious one, but nothing I'd like to be explicitly exposed. Note that it only tells a search engine if they are the default. They cannot query about other search engines. On Tue, Feb 15, 2011 at 1:48 PM, Kornel Lesiński kor...@geekhood.net wrote: There are many many sites that dream they were used as a default search engine, but their use of this API is only going to annoy or confuse users. This should be rare as annoying and confusing your users typically isn't a good business strategy. On Tue, Feb 15, 2011 at 9:42 AM, Bjartur Thorlacius svartma...@gmail.com wrote: https://sites.google.com/a/chromium.org/dev/developers/design-documents/chromium-search-provider-js-support Frankly, this reminds me of the security UI of Windows Vista. ... On Tue, Feb 15, 2011 at 1:48 PM, Kornel Lesiński kor...@geekhood.net wrote: Change of default search engine may have security implications I understand your concern. In this case, the UI is similar to what happens in many browsers when AddSearchProvider is called in response to a user action. In addition, the default is to not change anything. Yet, we are discussing one possible UI of many that I simply gave as one possible example. It is completely up to the UA to decide what to do in this case. I look forward to others proposing other solutions too in this regard. On Tue, Feb 15, 2011 at 1:48 PM, Kornel Lesiński kor...@geekhood.net wrote: You don't change that often, and there are only few search engines that make sense to be set as the default one. Browsers can simply ship with predefined set of engines, and that may be easiest and safest option for users. If a UA wanted to limit this to only be available to a predefined set of search engines, that would be possible. It is up to the UA to decide what to do. Technically a AddSearchProvider that did nothing ever would be conforming though certainly against the spirit of the api. Best wishes, dave
Re: [whatwg] Proposing canvas.toBlob(contentType)
Shouldn't this api be async? Returning a blob means that the size is available which implies a sync operation. dave On Wed, Apr 13, 2011 at 3:02 PM, Kyle Huey m...@kylehuey.com wrote: Hello All, Gecko 2.0 ships with a non-standard method on canvas named mozGetAsFile(contentType, fileName). We added this for internal use in our UI. It retrieves the contents of the canvas as a File object (at the time Gecko did not supports Blobs) encoded in the contentType according to the same rules toDataURL uses. I propose adding a toBlob(contentType) method to the canvas element in the style of toDataURL. This would greatly increase the options available to developers for extracting data from a canvas element (a Blob can be saved to disk, XHRed, etc.) - Kyle
Re: [whatwg] window.cipher HTML crypto API draft spec
Should it also be available in Web Workers? On Fri, May 20, 2011 at 11:45 AM, Adam Barth w...@adambarth.com wrote: Why window.cipher and not just window.crypto? Adam On Fri, May 20, 2011 at 8:04 AM, David Dahl dd...@mozilla.com wrote: Hello WHATWG members, With user control and privacy in mind, I have created a spec and an implementation for an easy to use cryptography API called DOMCrypt. This API will provide each web browser window with a 'cipher' property that facilitates: * asymmetric encryption key pair generation * public key encryption * decryption * signature generation * signature verification * hashing * easy public key discovery via meta tags I have created a Firefox extension that implements all of the above, and am working on an experimental patch that integrates this API into Firefox. The draft spec is here: https://wiki.mozilla.org/Privacy/Features/DOMCryptAPISpec/Latest The project originated in an extension I wrote, the home page is here: http://domcrypt.org The source code for the extension is here: https://github.com/daviddahl/domcrypt The Mozilla bugs are here: https://bugzilla.mozilla.org/show_bug.cgi?id=649154 https://bugzilla.mozilla.org/show_bug.cgi?id=657432 You can test the API by installing the extension hosted at domcrypt.organd addons.mozilla.org, and going to http://domcrypt.org Best Regards, David Dahl Firefox Engineer, Mozilla Corp.
Re: [whatwg] Questions on error handling in web workers
On Wed, Jul 15, 2009 at 5:17 PM, Ian Hickson i...@hixie.ch wrote: On Wed, 15 Jul 2009, Jian Li wrote: I have a couple questions regarding error handling in HTML 5 workers. In the spec, it says Whenever a runtime script error occurs in one of the worker's scripts, if the error did not occur while handling a previous script error, the user agent must report the error using the WorkerGlobalScope object's onerror attribute.. Does it mean that onerror attribute of WorkerContext would be called? In addition to the errors being thrown? The term report the error is defined here: http://www.whatwg.org/specs/web-apps/current-work/#report-the-error The wording in the Web Workers part of the spec about what to do for onerror has confused a number of people. Recently, when asked about it, I thought that it should send through an ErrorEvent when I was pointed at the spec (even though I had seen this issue before). Owen Merkling added a repro case here under that assumption: https://bugs.webkit.org/show_bug.cgi?id=35302 Separately, Yury Semikhatsky got it mixed up here: https://bugs.webkit.org/show_bug.cgi?id=36375 And a number of us (Jian Li, etc.) were confused about it. We can see that Ben Turner also found it confusing: https://bugzilla.mozilla.org/show_bug.cgi?id=664650 Hopefully, the spec can be clarified in this regard to avoid incompatibilities and confusion among implementors. One possible clarification would be a link in the onerror section for Web Workers to indicate that they use this definition: http://www.whatwg.org/specs/web-apps/current-work/#report-the-error Thanks, dave The spec also says For dedicated workers, if the error is still not handled afterwards, or if the error occurred while handling a previous script error, the user agent must queue a task to fire a worker error event at the Worker object associated with the worker.. What does it mean by the error is still not handled afterwards? Does it mean that the onerror attribute is not set for WorkerContext? The term not handled is defined here: http://www.whatwg.org/specs/web-apps/current-work/#concept-error-nothandled -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Questions on error handling in web workers
On Wed, Jun 15, 2011 at 11:02 PM, Ian Hickson i...@hixie.ch wrote: On Wed, 15 Jun 2011, David Levin wrote: The wording in the Web Workers part of the spec about what to do for onerror has confused a number of people. Recently, when asked about it, I thought that it should send through an ErrorEvent when I was pointed at the spec (even though I had seen this issue before). Owen Merkling added a repro case here under that assumption: https://bugs.webkit.org/show_bug.cgi?id=35302 Separately, Yury Semikhatsky got it mixed up here: https://bugs.webkit.org/show_bug.cgi?id=36375 And a number of us (Jian Li, etc.) were confused about it. We can see that Ben Turner also found it confusing: https://bugzilla.mozilla.org/show_bug.cgi?id=664650 Hopefully, the spec can be clarified in this regard to avoid incompatibilities and confusion among implementors. One possible clarification would be a link in the onerror section for Web Workers to indicate that they use this definition: http://www.whatwg.org/specs/web-apps/current-work/#report-the-error I spoke to bent earlier this evening, and tried to make some minor modifications to the spec to help make this clearer. Can you take a look and see if it's any better? I see that there is a link for report the error in the section 10.2.6 Runtime script errors I'm guessing that was part of your modifications. I think that does the trick. Thanks, dave If it's not, can you elaborate on what exactly is confusing? I tried to answer a question bent was asking but I am so familiar with the style that the spec is written in that I found the answer straight away and didn't see anything confusing, which doesn't help me make it better! :-) -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: [whatwg] Proposal for IsSearchProviderInstalled / AddSearchProvider
On Fri, Dec 16, 2011 at 3:08 PM, Peter Kasting pkast...@google.com wrote: On Tue, 17 May 2011, Bjartur Thorlacius wrote: Then why add an API when we've already got (IMO superior) declarative markup? In the case of adding the API to the spec, because it's already implemented. As to why it was added to the browsers, no idea. Certainly there's no declarative markup for IsSearchProviderInstalled(). As for AddSearchProvider(), I know one reason it was added to Chrome was explicitly to expose the and make default functionality. Of course one could argue that a UA could give users the option to make any provider default for whatever UI it exposes for the declarative case. But there are a couple fine points worth mentioning: one is that since users rarely want to make an engine default, adding that option to the UI all the time would be even more annoying than adding [ ] And set as my homepage to whatever UI is shown for bookmark this page, and thus UAs may shy away from this idea. Another is that engines may wish to explicitly request to be made default in response to some explicit user action on the page, e.g. clicking a Make this my default search engine button. Creating this UI and making it work smoothly is difficult with the existing mechanisms. Note that I am not the one who proposed, specced, or implemented this in Chrome; I'm just trying to convey a few things that seem apparent to me as another Chrome UI engineer :) Just to clarify, it was clear that many including other browsers didn't suppport the asDefault parameter, and Google Chrome no longer supports it. dave
Re: [whatwg] onclose events for MessagePort
On Tue, Oct 1, 2013 at 11:13 AM, Boris Zbarsky bzbar...@mit.edu wrote: On 10/1/13 2:11 PM, Ian Hickson wrote: How often do we expect two tabs to be talking to each other though? Or a page to an out-of-process subframe? How often do we expect MessageChannel to be used at all? Speaking as an web developer, I've found MessageChannel to be pretty useful when writing a complex web application because it allows you to isolate different communication pathways and hop across multiple iframes in a single bound (while only knowing about the next step at any one layer... it is hard to explain why this is useful without having to dive into the app). You can do this all on top of window.postMessage but it is more complicated and fragile. (Due to this, I've had to write a a layer above these that allows me to use either so that my code can work on browsers that don't support MessageChannel but it will be easier to debug on those that do.) dave Is that even possible on phone browsers? Sure is in Firefox and FirefoxOS. Anyway, I'm happy to support this in principle. OK, good. ;) -Boris
