date:20120514

On Fri, May 11, 2012 at 11:40 AM, Odin Hørthe Omdal odi...@opera.com wrote:
 I remembered this thread when I read Hixies outburst on #whatwg yesterday:

    [10:29] Hixie we really still have no way for JS to access the form
 data set on submission?
    [10:30] Hixie who's running this show, a clown?

  src: http://krijnhoetmer.nl/irc-logs/whatwg/20120510#l-483

For the person that defines form submission that's a fun thing to say :-)


http://wiki.whatwg.org/wiki/FormData has some ideas on how we could
update FormData, XMLHttpRequest, and HTMLFormElement to address the
various use cases presented thus far. Feedback welcome.


-- 
Anne — Opera Software
http://annevankesteren.nl/
http://www.opera.com/

Re: Working with FormData as an object

On Mon, May 14, 2012 at 2:00 PM, Anne van Kesteren ann...@annevk.nl wrote:
 http://wiki.whatwg.org/wiki/FormData has some ideas on how we could
 update FormData, XMLHttpRequest, and HTMLFormElement to address the
 various use cases presented thus far. Feedback welcome.

The one potentially problematic thing is exposing Blob passed to
FormData as File since they always have a filename. This might give
you distinct references to the underlying system file object. I'm not
sure if that is problematic. If it is problematic, I am not sure how
to keep the FormData API simple. I guess we should then introduce
FormEntry objects and change the API proposal to work with those.


-- 
Anne — Opera Software
http://annevankesteren.nl/
http://www.opera.com/

[Bug 17046] New: Implement redirect audit functionality for XHR

2012-05-14 Thread bugzilla

https://www.w3.org/Bugs/Public/show_bug.cgi?id=17046

   Summary: Implement redirect audit functionality for XHR
   Product: WebAppsWG
   Version: unspecified
  Platform: All
OS/Version: All
Status: NEW
  Severity: enhancement
  Priority: P2
 Component: XHR
AssignedTo: ann...@annevk.nl
ReportedBy: baranov...@yandex-team.ru
 QAContact: public-webapps-bugzi...@w3.org
CC: m...@w3.org, public-webapps@w3.org


Most of the time people talk (e.g. Stackoverflow) about controlling XHR
redirect, they don't mean really _controlling_ redirects, they mean 2 simple
things:
 - figure out, whether it was redirect or was not.
 - get final redirect url (or maybe interstitial).

From
http://lists.w3.org/Archives/Public/public-webapps/2010JulSep/thread.html#msg474
it is seen that everything is not so simple with implementation of
user-controlled redirects. But also, there was a phrase 

 However, if the use case is only
 auditing redirects, then
 ...

So maybe for most of the needs there it will be enough to implement such audit,
without touching current redirect behaviour. It may satisfy more than 90% of
the requests about controlling redirects.

For example XMLHttpRequest object may have a property called redirects or
smth, containing array of objects:
{url: whatever-no-matter-relative-or-absolute, {headers*}?}
Semantically this array will be a linked list, i.e. first element will be
initial resource request details and last - final one's.

Full implementation of XHR redirects is not so straightforward, but
implementation of such redirect audit is quite simple although very useful,
and it will be good to have such thing in XHR standard.

Problem:
As almost always, the problem rose from chrome extensions development. We have
to follow redirect and detect that fact as well as the final URL.

-- 
Configure bugmail: https://www.w3.org/Bugs/Public/userprefs.cgi?tab=email
--- You are receiving this mail because: ---
You are on the CC list for the bug.

Re: XHR's setRequestHeader and the Do Not Track (DNT) header

On Tue, May 8, 2012 at 9:34 PM, Ian Melven imel...@mozilla.com wrote:
 i'd like to propose that the Do Not Track header (see 
 http://www.w3.org/TR/tracking-dnt/#dnt-header-field) DNT
 be added to the list of request headers not allowed to be set via XHR's 
 setRequestHeader method (see
 http://dvcs.w3.org/hg/xhr/raw-file/tip/Overview.html#the-setrequestheader%28%29-method)

Done: http://dvcs.w3.org/hg/xhr/rev/a4a35861a49d


 there is an additional question of whether a user's Do Not Track setting in 
 the user agent should
 also be added to requests made by XHR - the DNT spec says A user agent must 
 send the DNT header field on all
 HTTP requests if (and only if) a tracking preference is enabled which would 
 seem to include XHR...

Unless specified otherwise XMLHttpRequest follows the requirements
from and on HTTP so I don't think anything needs changing here.


Per Adam's comment I have not removed the Sec- handling.


-- 
Anne — Opera Software
http://annevankesteren.nl/
http://www.opera.com/

Re: App Manifest API Proposal

2012-05-14 Thread Anant Narayanan


On 5/13/12 2:17 PM, SULLIVAN, BRYAN L wrote:

For (1) we can expect a text change, right?


Yes, I will make them as soon as I able to.


For (2), If the app manifest if obtained over non-secure HTTP, it is subject to 
modification. If the app is delivered over non-secure HTTP, even more can be 
modified. So is the plan to provide some kind of user warning when the manifest 
and/or app (including assets from the same origin) are delivered via non-secure 
HTTP (in the absence of a manifest signature)? And even if a manifest signature 
is provided how does it ensure protection of the assets (e.g. JS, CSS, and 
HTML) if they are delivered over non-secure HTTP? Does HTTPS need to be 
enforced, and cert domain validation as well?


We've previously discussed enforcing serving manifests over HTTPS, but 
it may not be appropriate to put this into the spec itself. Different 
user agents may choose to do different things, ranging from disallowing 
installs over HTTP or warning the user before proceeding.


Regards,
-Anant

Re: App Manifest API Proposal

2012-05-14 Thread Marcos Caceres




On Monday, 14 May 2012 at 17:44, Anant Narayanan wrote:

 
 We've previously discussed enforcing serving manifests over HTTPS, but
 it may not be appropriate to put this into the spec itself. Different
 user agents may choose to do different things, ranging from disallowing
 installs over HTTP or warning the user before proceeding.

I agree - trying to enface this in the spec would be unhelpful. Would make, for 
instance, testing really annoying because devs would need to shell out a bunch 
of money for a static IP and an SSL cert.  

-- 
Marcos Caceres
http://datadriven.com.au

Re: App Manifest API Proposal

2012-05-14 Thread Anant Narayanan

Hi Scott,

Thanks for your comments, more inline.

On 5/13/12 12:06 PM, Scott Wilson wrote:

On 12 May 2012, at 19:02, Anant Narayanan wrote:

Q. Why not simply reuse the widgets spec [2]?

A. Aside from naming (we're talking about apps, the word widget seems to
imply an artificial limitation),

To be fair, you can call your implementation anything you want even if it implements the
Widget specs. Maybe we could rename the Widget specs Widgets, Apps, Gadgets or
Whatever specs.

If you really, really hate the word that much you could decide to call the TWI widget object app
instead in your own documentation, and just silently convert window.widget to
window.app whenever you come across it. To reciprocate, I could add a line somewhere in Apache
Wookie and Apache Cordova that does the exact opposite. Interoperability FTW!

I'm trying to understand how building on the widget spec would work in
practice. I'm not opposed to it on principle, but we (Mozilla) have
chosen not to implement the widget spec in the past, but we have already
implemented the JSON manifest and API spec. If we rework this proposal
as an extension to the widget spec, does it mean we will have to
implement the entirety of the widget spec too?

Essentially, I'd like to make both spec independently implementable,
even if we chose to extend some objects defined in the widget spec.

and replacing XML with JSON;

No objections to representing the manifest in JSON either. Would a
serialization of The Widget Interface as a JSON manifest file obviate the need
for defining basically the same metadata in a different spec? We can then just
focus on the things that definitely aren't part of existing specs, such as the
security model, installation events, and default orientation, all of which look
like interesting extensions.

Rich Tibbett from Opera did precisely that, you can see a mapping here:
http://people.opera.com/richt/release/specs/manifests/widgets_to_app_manifest.html

It looks good to me in general, but I'm a little wary of committing to
all fields that are valid keys in the XML schema. Is there a way we can
take a subset instead?

the other fundamental difference is that the widget spec describes packaged
apps, whereas our manifest describes hosted apps.

Widgets is also used for hosted as well as packaged apps e.g. Apache Wookie +
Apache Rave...

Ah, that's really good to know; I hadn't come across a widget that was
hosted before, but looks like it is possible.

We think hosted apps have several interesting and unique web-like properties
that are worth retaining. Hosted apps can be made to work offline just as well
as packaged apps with AppCache (which is in need of some improvement, but can
be made to work!).

Which are the bits of this proposal that are important for this and which
aren't found in Widgets? Can we add those to the existing specs to fill any
gaps?

The manifests in the proposal don't have an id field, because an app
is simply identified by the domain from which the manifest for it was
fetched. This is the key difference, but I'll have to look deeper at the
Widget spec to see if there are any more.

Packaged apps do have their own advantages though, which we acknowledge, and
are open to extending the spec to support both types of apps.

Hmm, that does kind of negate the previous point... but moving on..!

We don't support packaged apps yet, either in the specification or the
implementation. If possible we'd like to go hosted + appcache as far as
we can. I mentioned this because I don't want packaged apps to be a
reason for this spec to be rejected.

I'm very positive about this proposal and would love to see it merged into
Widgets:PC TWI, with perhaps a separate spec on web app/widget installation
including the work Mozilla has done on installation APIs and events.

I'm glad you like the proposal! However, I would really like to see the
API and manifest in the same document, because, as I mentioned earlier,
at-least in the context of browsers they are dependent on each other.
What does it mean for a browser to only implement the manifest spec
but not the installation API (or vice-versa)?

On the other hand, there might be other User-Agents that won't have the
installation API though, because they don't have a DOM or support
JavaScript; in which case we could seperate them but write additional
text that recommends implementing both for environments that have a DOM.
I'm not sure if that's in scope for the working group.

I'd be interested in implementing those in Apache Wookie, Apache Rave and
related projects and initiatives that build on them, as web app installation
and app store APIs are something thats come up in quite a few implementations
and it would be great to have a spec for that.

Just don't tie it to another competing manifest format, please!

The current widget spec doesn't allow for a JSON representation. We will
have to come up with a

Re: App Manifest API Proposal

2012-05-14 Thread Anant Narayanan


On 5/14/12 10:47 AM, Mounir Lamouri wrote:

I don't think we can justify the choice of one app per origin just
because it's how browsers work nowadays regarding security and
permissions. This is an implementation detail and we shouldn't write
specs based on implementation details. And that might be true only for
some browsers and not for others.


Agree wholeheartedly!


In addition, how permissions are currently handled already have serious
limitations which would make the app security model hard to implement if
we stick to a simple origin definition. For example, if I go to
foo.example.com from APP1 (because APP1 is from that origin or because
I'm browsing that website from that app) and then I go to
foo.example.com from APP2, should permissions given when I was using
APP1 given to APP2? I don't think we should. Concretely that means that
a website accessed from it's dedicated app and the same website browsed
from a browser app would have the same permissions or that a website
browsed from browser foo and the same website browsed from browser bar
would also have the same permissions.


I don't understand this scenario fully, permissions given to an app are 
tied to its origin and cannot be extended to any other domain, 
irrespective of whether that domain was accessed via the app (an iframe 
or something else). We do not allow an app to navigate outside its 
origin at the top-level window.



IMO, the manifest URL should be the unique identifier for an application
so we should not restricts applications per origin and the security
model should consider that permissions where given to a specific origin
inside a specific application.


I'm not sure how this would work. Not only do we have to isolate API 
permissions between two apps from the same domain, but also other things 
like cookie jars, localStorage, indexedDB, XHR, and so on.


How will you enforce App1 from being unable to XHR to App2 even though 
they're both from the same domain? In particular, how do you know what 
pages from a given domain belong to which app? One possibility is to 
allow suffixes: (example.org/app1, example.org/app2), but the manifest 
URL by itself is insufficient.


-Anant

exposing CANVAS or something like it to Web Workers

I'd like to work on exposing something like CANVAS to web workers.

Ideally how over it works I'd like to be able to

*) get a 2d context in a web worker
*) get a WebGL context in a web worker
*) download images in a web worker and the images with both 2d contexts and
WebGL contexts

Any thoughts?

Re: exposing CANVAS or something like it to Web Workers

On Mon, May 14, 2012 at 10:01 PM, Gregg Tavares (勤) g...@google.com wrote:
 I'd like to work on exposing something like CANVAS to web workers.

 Ideally how over it works I'd like to be able to

 *) get a 2d context in a web worker
 *) get a WebGL context in a web worker
 *) download images in a web worker and the images with both 2d contexts and
 WebGL contexts

 Any thoughts?

Have we gotten any further with use cases? See
http://lists.w3.org/Archives/Public/public-whatwg-archive/2010Mar/thread.html#msg144
for an old use case thread that went nowhere. Or
http://lists.w3.org/Archives/Public/public-whatwg-archive/2011Feb/thread.html#msg274
where you requested general DOM access which cannot be implemented
everywhere.


-- 
Anne — Opera Software
http://annevankesteren.nl/
http://www.opera.com/

Re: exposing CANVAS or something like it to Web Workers


On 5/14/2012 1:01 PM, Gregg Tavares (勤) wrote:

I'd like to work on exposing something like CANVAS to web workers.

Ideally how over it works I'd like to be able to

*) get a 2d context in a web worker
*) get a WebGL context in a web worker
*) download images in a web worker and the images with both 2d 
contexts and WebGL contexts


Any thoughts?


As far as implementation, I'd love to be able to pass webkit's 
document.getCSSCanvasContext('2d') around.

It seems like a safe place to experiment.

I can get a lot done with CanvasPattern as a transferable, without 
needing to add Image (or video) into the worker context.


Notes:
1. getCSSCanvasContext is non-standard. It works with CSS image 
-webkit-canvas.
2. I heard that a more generic -moz-element() paint server is supposed 
to replace -webkit-canvas in time.
3. Passing the CSS canvas context would let me render off-frame and 
update a canvas visible on the document automatically.


Canvas -should- have toBlob and a typed array buffer for ImageData.
They are both useful for passing image data back to the main frame.


From my experience with WebGL, I think it should be considered with 
added care and lower priority.
There are stability, speed and memory issues. WebGL in workers seems to 
augment 2d, are there other big benefits?



-Charles

Re: exposing CANVAS or something like it to Web Workers

2012-05-14 Thread Jonas Sicking

On Mon, May 14, 2012 at 1:16 PM, Anne van Kesteren ann...@annevk.nl wrote:
 On Mon, May 14, 2012 at 10:01 PM, Gregg Tavares (勤) g...@google.com wrote:
 I'd like to work on exposing something like CANVAS to web workers.

 Ideally how over it works I'd like to be able to

 *) get a 2d context in a web worker
 *) get a WebGL context in a web worker
 *) download images in a web worker and the images with both 2d contexts and
 WebGL contexts

 Any thoughts?

 Have we gotten any further with use cases? See
 http://lists.w3.org/Archives/Public/public-whatwg-archive/2010Mar/thread.html#msg144
 for an old use case thread that went nowhere. Or
 http://lists.w3.org/Archives/Public/public-whatwg-archive/2011Feb/thread.html#msg274
 where you requested general DOM access which cannot be implemented
 everywhere.

One of the strongest use-cases we have found within mozilla is to
enable compiling applications written in other languages and for other
platforms to the web platform. For example emscripten [1] supports
compiling C/C++/OpenGL programs to javascript + WebGL.

Such programs generally use things like synchronous IO and have their
own event loops, thus not returning to the platform event loop.

So a good strategy for tools like emscripten is to create code which
is intended to run in a worker. That way you don't have to return to
the event loop, and you have access to things like full featured
synchronous XHR (including support for .responseType) without
degrading the user experience.

However a big problem is that you can't do graphics inside a worker.
It would be great to expose canvas since that is the graphics model
these programs have today.

[1] https://github.com/kripken/emscripten

/ Jonas

Re: exposing CANVAS or something like it to Web Workers


On 5/14/2012 1:16 PM, Anne van Kesteren wrote:

On Mon, May 14, 2012 at 10:01 PM, Gregg Tavares (勤)g...@google.com  wrote:

I'd like to work on exposing something like CANVAS to web workers.

Ideally how over it works I'd like to be able to

*) get a 2d context in a web worker
*) get a WebGL context in a web worker
*) download images in a web worker and the images with both 2d contexts and
WebGL contexts

Any thoughts?

Have we gotten any further with use cases? See
http://lists.w3.org/Archives/Public/public-whatwg-archive/2010Mar/thread.html#msg144
for an old use case thread that went nowhere. Or


1. Speeding up onmousemove-based drawing:
In my drawing projects (based on mouse/pen input), we lose mouse events 
/ pen pressure information when the main thread is busy rendering what 
the user is drawing.

Processing the drawing commands off-thread would lighten the load.

2. Avoiding blocking during redrawing of complex scenes or pre-rendering 
of animations.


With complex scenes, where we're repainting, we don't particularly want 
to block the main thread while a scene is loading.
But, we'd also like to use as much horsepower as the user's machine can 
lend.


A complex scene may block for a few seconds -- we can of course use 
green threading approaches, but that adds quite a bit of extra 
guess-work and

does not fully exploit the user's machine for speed.

Re: App Manifest API Proposal

2012-05-14 Thread Scott Wilson


On 14 May 2012, at 18:12, Anant Narayanan wrote:

 Hi Scott,
 
 Thanks for your comments, more inline.
 
 On 5/13/12 12:06 PM, Scott Wilson wrote:
 On 12 May 2012, at 19:02, Anant Narayanan wrote:
 Q. Why not simply reuse the widgets spec [2]?
 
 A. Aside from naming (we're talking about apps, the word widget seems to 
 imply an artificial limitation),
 
 To be fair, you can call your implementation anything you want even if it 
 implements the Widget specs. Maybe we could rename the Widget specs 
 Widgets, Apps, Gadgets or Whatever specs.
 
 If you really, really hate the word that much you could decide to call the 
 TWI widget object app instead in your own documentation, and just silently 
 convert window.widget to window.app whenever you come across it. To 
 reciprocate, I could add a line somewhere in Apache Wookie and Apache 
 Cordova that does the exact opposite. Interoperability FTW!
 
 I'm trying to understand how building on the widget spec would work in 
 practice. I'm not opposed to it on principle, but we (Mozilla) have chosen 
 not to implement the widget spec in the past, but we have already implemented 
 the JSON manifest and API spec. If we rework this proposal as an extension to 
 the widget spec, does it mean we will have to implement the entirety of the 
 widget spec too?

Entirety of the widget spec isn't much - you've done most of it already. If 
you mean would you have to support an XML manifest, or support packaged apps as 
well as naked manifests? No, I can't see a reason you would.

Comparing TWI with the proposal, the only things in TWI that are additional are 
shortName, authorEmail, and preferences. Preferences may not make sense for 
Mozilla's implementation - if so, don't use them, or autowire into WebStorage.

 Essentially, I'd like to make both spec independently implementable, even if 
 we chose to extend some objects defined in the widget spec.

 
 and replacing XML with JSON;
 
 No objections to representing the manifest in JSON either. Would a 
 serialization of The Widget Interface as a JSON manifest file obviate the 
 need for defining basically the same metadata in a different spec? We can 
 then just focus on the things that definitely aren't part of existing specs, 
 such as the security model, installation events, and default orientation, 
 all of which look like interesting extensions.
 
 Rich Tibbett from Opera did precisely that, you can see a mapping here: 
 http://people.opera.com/richt/release/specs/manifests/widgets_to_app_manifest.html
 
 It looks good to me in general, but I'm a little wary of committing to all 
 fields that are valid keys in the XML schema. Is there a way we can take a 
 subset instead?

The spec defines the set of widget metadata. If you only choose to use a subset 
in implementation, thats fine. 

Also, don't worry about the XML - I think the main point of comparison is the 
Widget Interface spec which already maps a subset of the XML manifest 
properties to JS properties that make sense in the browsing context of a 
running widget. 

So manifest properties that only really apply to a packaged widget wouldn't 
necessarily be used in a JSON representation for a hosted widget.

 
 the other fundamental difference is that the widget spec describes packaged 
 apps, whereas our manifest describes hosted apps.
 
 Widgets is also used for hosted as well as packaged apps e.g. Apache Wookie 
 + Apache Rave...
 
 Ah, that's really good to know; I hadn't come across a widget that was hosted 
 before, but looks like it is possible.
 
 We think hosted apps have several interesting and unique web-like 
 properties that are worth retaining. Hosted apps can be made to work 
 offline just as well as packaged apps with AppCache (which is in need of 
 some improvement, but can be made to work!).
 
 Which are the bits of this proposal that are important for this and which 
 aren't found in Widgets? Can we add those to the existing specs to fill any 
 gaps?
 
 The manifests in the proposal don't have an id field, because an app is 
 simply identified by the domain from which the manifest for it was fetched. 
 This is the key difference, but I'll have to look deeper at the Widget spec 
 to see if there are any more.

The id property is optional in widgets anyway.

 
 Packaged apps do have their own advantages though, which we acknowledge, 
 and are open to extending the spec to support both types of apps.
 
 Hmm, that does kind of negate the previous point... but moving on..!
 
 We don't support packaged apps yet, either in the specification or the 
 implementation. If possible we'd like to go hosted + appcache as far as we 
 can. I mentioned this because I don't want packaged apps to be a reason for 
 this spec to be rejected.
 
 I'm very positive about this proposal and would love to see it merged into 
 Widgets:PC  TWI, with perhaps a separate spec on web app/widget 
 installation including the work Mozilla has done on installation APIs and 
 events.

Re: App Manifest API Proposal

2012-05-14 Thread Marcos Caceres


Hi Anant,

On 14/05/2012 18:12, Anant Narayanan wrote:

Hi Scott,

Thanks for your comments, more inline.

On 5/13/12 12:06 PM, Scott Wilson wrote:

On 12 May 2012, at 19:02, Anant Narayanan wrote:

Q. Why not simply reuse the widgets spec [2]?

A. Aside from naming (we're talking about apps, the word widget 
seems to imply an artificial limitation),


To be fair, you can call your implementation anything you want even 
if it implements the Widget specs. Maybe we could rename the Widget 
specs Widgets, Apps, Gadgets or Whatever specs.


If you really, really hate the word that much you could decide to 
call the TWI widget object app instead in your own documentation, 
and just silently convert window.widget to window.app whenever 
you come across it. To reciprocate, I could add a line somewhere in 
Apache Wookie and Apache Cordova that does the exact opposite. 
Interoperability FTW!


I'm trying to understand how building on the widget spec would work in 
practice. I'm not opposed to it on principle, but we (Mozilla) have 
chosen not to implement the widget spec in the past, but we have 
already implemented the JSON manifest and API spec. If we rework this 
proposal as an extension to the widget spec, does it mean we will have 
to implement the entirety of the widget spec too?
Absolutely not: there would be no need to support the XML format. You 
just support a particular serialization (JSON) and not the packaging 
side (which is also orthogonal). But the semantics and data model remain 
the same across the specs where appropriate.
Essentially, I'd like to make both spec independently implementable, 
even if we chose to extend some objects defined in the widget spec.
Yes, that would be ideal. Historically, we renamed the Widget spec to 
Packaging and *XML Configuration* on purpose, because we've been 
anticipating the JSON format for a long long time (our requirement 
document has mentioned JSON as a target for many years [1]). We sat on 
it because we were waiting for Google or you guys to jump on it :)


Ideal situation: JSON format should be usable by UAs that currently 
support XML format (if there is support for that from implementers like 
Opera). If you guys have new/interesting use cases, then we should keep 
the specs in sync. I'm happy to help keep them in synchornized.

and replacing XML with JSON;


No objections to representing the manifest in JSON either. Would a 
serialization of The Widget Interface as a JSON manifest file obviate 
the need for defining basically the same metadata in a different 
spec? We can then just focus on the things that definitely aren't 
part of existing specs, such as the security model, installation 
events, and default orientation, all of which look like interesting 
extensions.


Rich Tibbett from Opera did precisely that, you can see a mapping 
here: 
http://people.opera.com/richt/release/specs/manifests/widgets_to_app_manifest.html


It looks good to me in general, but I'm a little wary of committing to 
all fields that are valid keys in the XML schema. Is there a way we 
can take a subset instead?
Absolutely! That would totally be ok. There is nothing required in the 
widget XML format at all. So, if you don't have a use case for something 
in the Widgets PC spec, then you can simply ignore it. This is by design.


the other fundamental difference is that the widget spec describes 
packaged apps, whereas our manifest describes hosted apps.


Widgets is also used for hosted as well as packaged apps e.g. Apache 
Wookie + Apache Rave...


Ah, that's really good to know; I hadn't come across a widget that was 
hosted before, but looks like it is possible.

There is a long legacy, dating back to the original Google Gadgets.
We think hosted apps have several interesting and unique web-like 
properties that are worth retaining. Hosted apps can be made to work 
offline just as well as packaged apps with AppCache (which is in 
need of some improvement, but can be made to work!).


Which are the bits of this proposal that are important for this and 
which aren't found in Widgets? Can we add those to the existing specs 
to fill any gaps?


The manifests in the proposal don't have an id field, because an app 
is simply identified by the domain from which the manifest for it was 
fetched. This is the key difference, but I'll have to look deeper at 
the Widget spec to see if there are any more.


Looking forward to seeing what you find. And remember, we are open to 
changes so feel free to discuss.
Packaged apps do have their own advantages though, which we 
acknowledge, and are open to extending the spec to support both 
types of apps.


Hmm, that does kind of negate the previous point... but moving on..!


We don't support packaged apps yet, either in the specification or the 
implementation. If possible we'd like to go hosted + appcache as far 
as we can. I mentioned this because I don't want packaged apps to be a 
reason for this spec to be rejected.

I think that is

Re: exposing CANVAS or something like it to Web Workers

2012-05-14 Thread Glenn Maynard

On Mon, May 14, 2012 at 3:01 PM, Gregg Tavares (勤) g...@google.com wrote:

 I'd like to work on exposing something like CANVAS to web workers.

 Ideally how over it works I'd like to be able to

 *) get a 2d context in a web worker


I'd recommend not trying to tackle 2d and 3d contexts at once, and only
worrying about WebGL to start.

Another issue: rendering in a worker thread onto a canvas which is
displayed in the main thread.  This needs to be solved in a way that
doesn't cause the asynchronous nature of what's happening to be visible to
scripts.  toDataURL and toBlob would probably need to be prohibited on the
canvas element.  I'm not sure what the actual API would look like.  This
would also require some equivalent to requestAnimationFrame in the worker
thread.

*) download images in a web worker and the images with both 2d contexts and
 WebGL contexts


Don't forget to point people to the recent discussion on this, so the
conversation doesn't reboot:

https://www.khronos.org/webgl/public-mailing-list/archives/1205/msg00059.html

In summary, the idea is to provide a much more basic interface for images,
which would represent an image in an opaque, non-DOM way, to make it
implementable in workers; you'd retrieve it with a method on
HTMLImageElement after the image is completely available.  It'd be
structured clonable, to hand it to workers.  There should also be an XHR2
responseType mode to retrieve an object like this directly, to allow
loading images from workers (again without using the DOM), since workers
shouldn't need to rely on a helper in the UI thread just to load textures.

This seems like it would solve this sub-problem pretty well, at least.


On Mon, May 14, 2012 at 3:16 PM, Anne van Kesteren ann...@annevk.nl wrote:

 Have we gotten any further with use cases? See

 http://lists.w3.org/Archives/Public/public-whatwg-archive/2010Mar/thread.html#msg144
 for an old use case thread that went nowhere. Or

 http://lists.w3.org/Archives/Public/public-whatwg-archive/2011Feb/thread.html#msg274
 where you requested general DOM access which cannot be implemented
 everywhere.


The use case is being able to draw without blocking the UI thread.  WebGL,
like OpenGL, tries to be nonblocking by queuing draw operations, but it's
impossible to do this in all cases: as soon as you call any method that
returns data (eg. getError), WebGL has to synchronously flush the draw
queue.  This isn't as severe as synchronous XHR, but it may be done every
frame--a 10ms delay at 60Hz is going to have visible effects on the UI.

WebGL is also useful for computational tasks, which may perform much longer
drawing operations.  If you're doing this, it's even more important that it
not block the UI thread.

Compiling shaders can also take a long time for complex shaders.  Since
this is almost always followed by a synchronous call to check the result,
it's currently hard to load shaders without affecting the browser UI.

-- 
Glenn Maynard

Re: exposing CANVAS or something like it to Web Workers

2012-05-14 Thread Jonas Sicking

On Mon, May 14, 2012 at 3:28 PM, Glenn Maynard gl...@zewt.org wrote:
 On Mon, May 14, 2012 at 3:01 PM, Gregg Tavares (勤) g...@google.com wrote:

 I'd like to work on exposing something like CANVAS to web workers.

 Ideally how over it works I'd like to be able to

 *) get a 2d context in a web worker


 I'd recommend not trying to tackle 2d and 3d contexts at once, and only
 worrying about WebGL to start.

 Another issue: rendering in a worker thread onto a canvas which is displayed
 in the main thread.  This needs to be solved in a way that doesn't cause the
 asynchronous nature of what's happening to be visible to scripts.  toDataURL
 and toBlob would probably need to be prohibited on the canvas element.  I'm
 not sure what the actual API would look like.

If/when we do this, I think it should be done in such a way that the
main window can't access the canvas object at all. Similar to what
happens when an ArrayBuffer is transferred to a Worker using
structured cloning. Once a canvas is transferred to a Worker, any
access to it should throw or return null/0/. If you want to transfer
pixel data to the main thread, it seems less racy to do that by
getting the pixel data in the Worker which owns the canvas and then
transfer that to the main thread using postMessage.

 This would also require some
 equivalent to requestAnimationFrame in the worker thread.

Agreed!

/ Jonas

Re: exposing CANVAS or something like it to Web Workers

2012-05-14 Thread Glenn Maynard

On Mon, May 14, 2012 at 6:42 PM, Jonas Sicking jo...@sicking.cc wrote:

 If/when we do this, I think it should be done in such a way that the
 main window can't access the canvas object at all. Similar to what
 happens when an ArrayBuffer is transferred to a Worker using
 structured cloning. Once a canvas is transferred to a Worker, any
 access to it should throw or return null/0/.


Sort of.  You want to disable the methods on Canvas--toDataURL, toBlob and
getContext--but you don't want to disable inherited methods (you should
still be able to access eg. parentNode).  I think handling this on a
method-by-method basis is fine, since there aren't many of them--just those
three, I think.

I still don't have much intuition about how to do this, but another
thought: assuming you create a context in the worker which is attached to
the canvas, you need to be able to create new contexts in the future if the
worker dies, since workers can be killed by the browser at any time.  A
mechanism that says you can create a worker-context for a canvas, but then
the canvas element is locked to that worker isn't good enough (but it's
fine to say you can only have one worker associated with a canvas at a
time).

If you want to transfer
 pixel data to the main thread, it seems less racy to do that by
 getting the pixel data in the Worker which owns the canvas and then
 transfer that to the main thread using postMessage.


Right.


   This would also require some
  equivalent to requestAnimationFrame in the worker thread.

 Agreed!


A tricky bit: you need to know which element to sync to, so the browser
knows which monitor's vsync to use.  According to [1] only WebKit's
requestAnimationFrame actually takes an element.  (That's surprising; this
seems obvious.  Are multi-monitor systems always vsync-locked to each
other, which would make this not matter?)  I mention this because this
method would need to accept a context in lieu of an element (assuming
whatever mechanism we land on it results in having a WebGLContext in the
worker but no HTMLCanvasElement, whcih seems likely).

This in mind, it would be good if all Canvas context types had a common
base class, even if it's an empty interface.  The return value of
HTMLCanvasElement.getContext could be changed to it, instead of object.

[1] https://developer.mozilla.org/en/DOM/window.requestAnimationFrame

-- 
Glenn Maynard

Re: exposing CANVAS or something like it to Web Workers

On Mon, May 14, 2012 at 4:42 PM, Jonas Sicking jo...@sicking.cc wrote:

 On Mon, May 14, 2012 at 3:28 PM, Glenn Maynard gl...@zewt.org wrote:
  On Mon, May 14, 2012 at 3:01 PM, Gregg Tavares (勤) g...@google.com
  wrote:
 
  I'd like to work on exposing something like CANVAS to web workers.
 
  Ideally how over it works I'd like to be able to
 
  *) get a 2d context in a web worker
 
 
  I'd recommend not trying to tackle 2d and 3d contexts at once, and only
  worrying about WebGL to start.
 
  Another issue: rendering in a worker thread onto a canvas which is
 displayed
  in the main thread.  This needs to be solved in a way that doesn't cause
 the
  asynchronous nature of what's happening to be visible to scripts.
  toDataURL
  and toBlob would probably need to be prohibited on the canvas element.
  I'm
  not sure what the actual API would look like.

 If/when we do this, I think it should be done in such a way that the
 main window can't access the canvas object at all. Similar to what
 happens when an ArrayBuffer is transferred to a Worker using
 structured cloning. Once a canvas is transferred to a Worker, any
 access to it should throw or return null/0/. If you want to transfer
 pixel data to the main thread, it seems less racy to do that by
 getting the pixel data in the Worker which owns the canvas and then
 transfer that to the main thread using postMessage.


How about separating the canvasy parts of canvas from canvas and the imagy
parts of image from image.

In other words, Imagine canvas is implemented like this

class Canvas : public HTMLElement {
  private:
CanvasSurface* m_surface;  // everything about canvas that is not
HTMLElement
};

And that Image is similarly implemented as

class Image : public HTMLElement {
  private:
Picture* m_picture;  // everything about Image that is not HTMLElement
}

now imagine you can instantiate inner implementation of these things. The
parts that are not HTMLElement

var canvasSurface = new CanvasSurface();
var ctx = canvasSurface.getContext(2d);
var pic = new Picture;
pic.src = http://someplace.com/someimage.jpg;;
pic.onload = function() {
   ctx.drawImage(pic, 0, );
}

Let's assume you can instantiate these things in either the page or a
worker. Nether can be transfered.

Would that work? What problems would that have?






  This would also require some
  equivalent to requestAnimationFrame in the worker thread.

 Agreed!

 / Jonas

Re: exposing CANVAS or something like it to Web Workers


On 5/14/12 7:56 PM, Glenn Maynard wrote:

A tricky bit: you need to know which element to sync to, so the browser
knows which monitor's vsync to use.  According to [1] only WebKit's
requestAnimationFrame actually takes an element.  (That's surprising;
this seems obvious.


Does WebKit actually use the element to determine vsync?  How do they 
handle cases when the element spans monitors?


As far as I know WebKit's implementation uses the element to optimize 
out callbacks when the element is not visible, but that's it.


Note that Gecko, for example, does not tie requestAnimationFrame 
callbacks to vsync.  I can't speak for other UAs.



I mention this because
this method would need to accept a context in lieu of an element


What would the context be used for?

-Boris

Re: exposing CANVAS or something like it to Web Workers

On May 14, 2012, at 4:42 PM, Jonas Sicking jo...@sicking.cc wrote:

 On Mon, May 14, 2012 at 3:28 PM, Glenn Maynard gl...@zewt.org wrote:
 On Mon, May 14, 2012 at 3:01 PM, Gregg Tavares (勤) g...@google.com wrote:
 
 I'd like to work on exposing something like CANVAS to web workers.
 
 Ideally how over it works I'd like to be able to
 
 *) get a 2d context in a web worker
 
 
 I'd recommend not trying to tackle 2d and 3d contexts at once, and only
 worrying about WebGL to start.
 
 Another issue: rendering in a worker thread onto a canvas which is displayed
 in the main thread.  This needs to be solved in a way that doesn't cause the
 asynchronous nature of what's happening to be visible to scripts.  toDataURL
 and toBlob would probably need to be prohibited on the canvas element.  I'm
 not sure what the actual API would look like.
 
 If/when we do this, I think it should be done in such a way that the
 main window can't access the canvas object at all. Similar to what
 happens when an ArrayBuffer is transferred to a Worker using
 structured cloning. Once a canvas is transferred to a Worker, any
 access to it should throw or return null/0/. If you want to transfer
 pixel data to the main thread, it seems less racy to do that by
 getting the pixel data in the Worker which owns the canvas and then
 transfer that to the main thread using postMessage.
 
 This would also require some
 equivalent to requestAnimationFrame in the worker thread.
 
 Agreed!
 
 / Jonas



I'm a bit lost-- wouldn't we just postMessage from the document over to the web 
worker when we want a refresh?

I agree that we ought to be transferring pixel data not Canvas contexts; with 
the possible exception of CSS context.

We could just create new canvas instances inside the worker thread. I'd still 
prefer to clone CanvasPattern as a means of transferring paint over to the 
worker, though sending pixel data would work too.

I heard Picture come up-- it seems like that object might have additional 
semantics for high resolution alternatives that may need to be considered.

-Charles

Re: exposing CANVAS or something like it to Web Workers


On 5/14/12 8:03 PM, Gregg Tavares (勤) wrote:

var canvasSurface = new CanvasSurface();
var ctx = canvasSurface.getContext(2d);
var pic = new Picture;
pic.src = http://someplace.com/someimage.jpg;;
pic.onload = function() {
ctx.drawImage(pic, 0, );
}

Let's assume you can instantiate these things in either the page or a
worker. Nether can be transfered.

Would that work? What problems would that have?


Two things that come to mind immediately:

1)  Various canvas 2d context methods depend on the styles of the canvas 
to define how they actually behave.  Clearly this would need some sort 
of changes for Workers anyway; the question is what those changes would 
need to be.


2)  How (or maybe whether) this should work with pic.src = 
http://someplace.com/someimage.svg; is an interesting question.


3)  This leaves open the question of how to get data from the worker 
back to the main thread to paint there.


#1 is an issue with any proposal that puts a 2d context in a worker.

#2 is only an issue if we do the image loads in workers somehow; if we 
wanted to just rasterize the SVG and pass the resulting data to the 
worker there would be no problem, obviously.


For #3, being able to transfer a canvas context from the page to a 
worker but leave it painting to the canvas may still be desirable...


-Boris

Re: exposing CANVAS or something like it to Web Workers

2012-05-14 Thread James Robinson

On Mon, May 14, 2012 at 5:03 PM, Boris Zbarsky bzbar...@mit.edu wrote:

 On 5/14/12 7:56 PM, Glenn Maynard wrote:

 A tricky bit: you need to know which element to sync to, so the browser
 knows which monitor's vsync to use.  According to [1] only WebKit's
 requestAnimationFrame actually takes an element.  (That's surprising;
 this seems obvious.


 Does WebKit actually use the element to determine vsync?  How do they
 handle cases when the element spans monitors?


 As far as I know WebKit's implementation uses the element to optimize out
 callbacks when the element is not visible, but that's it.


The element isn't used for anything currently in WebKit.  Which vsync is
determined by the monitor the tab/window/whatever lands on.  When this
spans monitors, something random happens (there aren't many good options).


 Note that Gecko, for example, does not tie requestAnimationFrame callbacks
 to vsync.  I can't speak for other UAs.


I think you'll want to.

- James




  I mention this because
 this method would need to accept a context in lieu of an element


 What would the context be used for?

 -Boris

Re: exposing CANVAS or something like it to Web Workers


On 5/14/12 8:21 PM, Charles Pritchard wrote:

SVG and animated gif would render the same as it would in an
Image that has not been added to the dom or is otherwise display: none.


I'm not sure that would be workable in a worker for SVG, for the same 
reasons that responseXML is not available in workers: it involves DOM 
elements.  Unless the SVG rasterization happens on the main thread under 
the hood and the raster data is then sent over to the worker.  This 
might have ... surprising performance characteristics.


-Boris

Re: exposing CANVAS or something like it to Web Workers

On Mon, May 14, 2012 at 5:12 PM, Boris Zbarsky bzbar...@mit.edu wrote:

 On 5/14/12 8:03 PM, Gregg Tavares (勤) wrote:

 var canvasSurface = new CanvasSurface();
 var ctx = canvasSurface.getContext(2d)**;
 var pic = new Picture;
 pic.src = 
 http://someplace.com/**someimage.jpghttp://someplace.com/someimage.jpg
 ;
 pic.onload = function() {
ctx.drawImage(pic, 0, );
 }

 Let's assume you can instantiate these things in either the page or a
 worker. Nether can be transfered.

 Would that work? What problems would that have?


 Two things that come to mind immediately:

 1)  Various canvas 2d context methods depend on the styles of the canvas
 to define how they actually behave.  Clearly this would need some sort of
 changes for Workers anyway; the question is what those changes would need
 to be.


Which methods are these?




 2)  How (or maybe whether) this should work with pic.src = 
 http://someplace.com/**someimage.svg http://someplace.com/someimage.svg
 is an interesting question.

 3)  This leaves open the question of how to get data from the worker back
 to the main thread to paint there.


I feel that's a somewhat orthogonal issue to getting a Canvas or
CanvasSurface in a worker. At least for the WebGL case, WebGL already
defines ways to share data safely between contexts so that use case is
already covered.

For a 2d context I can certainly imagine lots of ways. Some fast, some slow

slow) pass the result of toDataURL back

slow) pass an ImageData.data back (transfer of ownership)

fast) some how use a CanvasSurface as the source of a video tag

maybe there are some other ideas? But, even if we didn't resolve this, it's
still massively useful, at least for WebGL, to be able to instantiate a
WebGL context in a worker.






 #1 is an issue with any proposal that puts a 2d context in a worker.

 #2 is only an issue if we do the image loads in workers somehow; if we
 wanted to just rasterize the SVG and pass the resulting data to the worker
 there would be no problem, obviously.

 For #3, being able to transfer a canvas context from the page to a worker
 but leave it painting to the canvas may still be desirable...

 -Boris

Re: exposing CANVAS or something like it to Web Workers

On May 14, 2012, at 5:50 PM, Boris Zbarsky bzbar...@mit.edu wrote:

 On 5/14/12 8:21 PM, Charles Pritchard wrote:
 SVG and animated gif would render the same as it would in an
 Image that has not been added to the dom or is otherwise display: none.
 
 I'm not sure that would be workable in a worker for SVG, for the same reasons 
 that responseXML is not available in workers: it involves DOM elements.  
 Unless the SVG rasterization happens on the main thread under the hood and 
 the raster data is then sent over to the worker.  This might have ... 
 surprising performance characteristics.

I agree... Can we get this off the main thread? Svg via image is not quite the 
same as svg via HTMLDocument (I guess I mean, embedded).

Afaik, svg via image does not have any script controls but it does have xsl 
things. 

I've never tried to abuse the distinction via embedded blob Uris and such.

Put in other words: img src=image.svg may use an entirely different 
implementation than svg in an HTML document.

Re: exposing CANVAS or something like it to Web Workers


On 5/14/12 8:55 PM, Gregg Tavares (勤) wrote:

1)  Various canvas 2d context methods depend on the styles of the
canvas to define how they actually behave.  Clearly this would need
some sort of changes for Workers anyway; the question is what those
changes would need to be.

Which methods are these?


Anything involving setting color (e.g. the strokeStyle setter, the 
fillStyle setter), due to currentColor.  Anything involving text 
because font styles come from the element or document.


Those are the ones that come to mind offhand, but I haven't looked at 
the various recent additions to the 2d context closely.


-Boris

Re: exposing CANVAS or something like it to Web Workers


On 5/14/12 8:58 PM, Charles Pritchard wrote:

I agree... Can we get this off the main thread?


Maybe.

It would be pretty nontrivial in Gecko; last I looked it would be pretty 
painful in WebKit too.  Can't speak for other UAs.



Svg via image is not quite the same as svg via HTMLDocument (I guess I mean, 
embedded).


Sure, but that doesn't matter.


Afaik, svg via image does not have any script controls but it does have xsl 
things.


And CSS.


Put in other words:img src=image.svg  may use an entirely different implementation 
thansvg  in an HTML document.


It _may_, but it would likely be pretty buggy.  And at least in Gecko, I 
don't think we're willing to duplicate enough of the rendering engine to 
do this.


-Boris

Re: exposing CANVAS or something like it to Web Workers

On Mon, May 14, 2012 at 6:07 PM, Boris Zbarsky bzbar...@mit.edu wrote:

 On 5/14/12 8:55 PM, Gregg Tavares (勤) wrote:

1)  Various canvas 2d context methods depend on the styles of the
canvas to define how they actually behave.  Clearly this would need
some sort of changes for Workers anyway; the question is what those
changes would need to be.

 Which methods are these?


 Anything involving setting color (e.g. the strokeStyle setter, the
 fillStyle setter), due to currentColor.  Anything involving text because
 font styles come from the element or document.


Good to know.

That doesn't sound like a showstopper though. If a canvas/CanvasSurface is
available in workers the simplest solution would just be that
currentColor defaults to something black? or nothing . Pick one.



 Those are the ones that come to mind offhand, but I haven't looked at the
 various recent additions to the 2d context closely.

 -Boris

Re: exposing CANVAS or something like it to Web Workers


On 5/14/2012 6:08 PM, Boris Zbarsky wrote:

On 5/14/12 8:58 PM, Charles Pritchard wrote:

I agree... Can we get this off the main thread?


Maybe.

It would be pretty nontrivial in Gecko; last I looked it would be 
pretty painful in WebKit too.  Can't speak for other UAs.


Can it be pumped through what's essentially an iframe on a null origin?

I don't know enough about browser internals to help on this one.
Yes, loading an SVG image is a heavy call.

For 90% of the SVG content out there, it'd probably be faster to parse 
and draw the SVG via Canvas and JS.

Still it's a hell of a lot nicer to load it via img tag.

I'd just assumed that img src=pic.svg / was loaded off-thread / 
async much like img calls.


There's nothing that gets carried from the document to the img other 
than the width/height, which I believe is carried through.

Which is a good thing, of course.


-Charles

Re: exposing CANVAS or something like it to Web Workers


On 5/14/12 9:15 PM, Charles Pritchard wrote:

On 5/14/2012 6:08 PM, Boris Zbarsky wrote:

On 5/14/12 8:58 PM, Charles Pritchard wrote:

I agree... Can we get this off the main thread?


Maybe.

It would be pretty nontrivial in Gecko; last I looked it would be
pretty painful in WebKit too. Can't speak for other UAs.


Can it be pumped through what's essentially an iframe on a null origin?


The issue at least in Gecko is that the DOM uses various 
effectively-global variables (shared caches of various sorts, 
non-threadsafe services, etc).  The origin is irrelevant.



I'd just assumed that img src=pic.svg / was loaded off-thread /
async much like img calls.


The _loading_ is async.  In Gecko the parsing of the XML, DOM 
construction, style computations, layout are on the main thread, like 
they are for HTML.


It's the DOM construction, style computations, layout parts that are 
not so easy to push off to a background thread...


-Boris

Re: exposing CANVAS or something like it to Web Workers


On 5/14/2012 6:14 PM, Gregg Tavares (勤) wrote:



On Mon, May 14, 2012 at 6:07 PM, Boris Zbarsky bzbar...@mit.edu 
mailto:bzbar...@mit.edu wrote:


On 5/14/12 8:55 PM, Gregg Tavares (勤) wrote:

   1)  Various canvas 2d context methods depend on the styles
of the
   canvas to define how they actually behave.  Clearly this
would need
   some sort of changes for Workers anyway; the question is
what those
   changes would need to be.

Which methods are these?


Anything involving setting color (e.g. the strokeStyle setter, the
fillStyle setter), due to currentColor.  Anything involving text
because font styles come from the element or document.


Good to know.

That doesn't sound like a showstopper though. If a 
canvas/CanvasSurface is available in workers the simplest solution 
would just be that currentColor defaults to something black? or 
nothing . Pick one.


Font is still a little tricky from loading fonts via CSS. Font is tricky 
anyway, though, so it wouldn't be that much of a step backward.


Would we assume that if a font is available from the parent context it's 
going to be available to the worker?


currentColor would just default to black, as we're not talking about a 
color inherited from the DOM.




Those are the ones that come to mind offhand, but I haven't looked
at the various recent additions to the 2d context closely.



The recent additions are more proposals than additions. They're 
proposals from Tab and Ian and not yet implemented.


They revolve around a Path object, which we've not yet discussed.
Otherwise, they include lightweight nodes and DOM fallback content, 
which isn't relevant to workers.


My thinking is the same as yours: fillStyle/strokeStyle and font are the 
ones that come to mind.
Pattern and Gradient are items that could conceivably be cloned and/or 
shared.

They can both be as efficient as sending Blob via postMessage.

I think Gregg was just settling on not-sending Canvas over postMessage 
but rather creating the instance inside of each Worker.

Re: exposing CANVAS or something like it to Web Workers

2012-05-14 Thread Jonas Sicking

On Mon, May 14, 2012 at 5:10 PM, Charles Pritchard ch...@jumis.com wrote:
 On May 14, 2012, at 4:42 PM, Jonas Sicking jo...@sicking.cc wrote:

 On Mon, May 14, 2012 at 3:28 PM, Glenn Maynard gl...@zewt.org wrote:
 On Mon, May 14, 2012 at 3:01 PM, Gregg Tavares (勤) g...@google.com wrote:

 I'd like to work on exposing something like CANVAS to web workers.

 Ideally how over it works I'd like to be able to

 *) get a 2d context in a web worker


 I'd recommend not trying to tackle 2d and 3d contexts at once, and only
 worrying about WebGL to start.

 Another issue: rendering in a worker thread onto a canvas which is displayed
 in the main thread.  This needs to be solved in a way that doesn't cause the
 asynchronous nature of what's happening to be visible to scripts.  toDataURL
 and toBlob would probably need to be prohibited on the canvas element.  I'm
 not sure what the actual API would look like.

 If/when we do this, I think it should be done in such a way that the
 main window can't access the canvas object at all. Similar to what
 happens when an ArrayBuffer is transferred to a Worker using
 structured cloning. Once a canvas is transferred to a Worker, any
 access to it should throw or return null/0/. If you want to transfer
 pixel data to the main thread, it seems less racy to do that by
 getting the pixel data in the Worker which owns the canvas and then
 transfer that to the main thread using postMessage.

 This would also require some
 equivalent to requestAnimationFrame in the worker thread.

 Agreed!

 / Jonas



 I'm a bit lost-- wouldn't we just postMessage from the document over to the 
 web worker when we want a refresh?

 I agree that we ought to be transferring pixel data not Canvas contexts; with 
 the possible exception of CSS context.

 We could just create new canvas instances inside the worker thread. I'd still 
 prefer to clone CanvasPattern as a means of transferring paint over to the 
 worker, though sending pixel data would work too.

 I heard Picture come up-- it seems like that object might have additional 
 semantics for high resolution alternatives that may need to be considered.

I was saying that I think we should transfer the context from the main
thread to the worker thread. The worker thread should then be able to
use the context to draw directly to the screen without interacting
with the main thread.

You should generally not need to transfer pixel data. But I think it
should be possible to grab pixel data on whichever thread is currently
owning a context. This will implicitly make it possible to transfer
pixel data if anyone wants to do it.

/ Jonas

Re: exposing CANVAS or something like it to Web Workers


On 5/14/2012 6:23 PM, Jonas Sicking wrote:

  This would also require some
  equivalent to requestAnimationFrame in the worker thread.


  Agreed!

  / Jonas




  I'm a bit lost-- wouldn't we just postMessage from the document over to the 
web worker when we want a refresh?

  I agree that we ought to be transferring pixel data not Canvas contexts; 
with the possible exception of CSS context.

  We could just create new canvas instances inside the worker thread. I'd 
still prefer to clone CanvasPattern as a means of transferring paint over to the 
worker, though sending pixel data would work too.

  I heard Picture come up-- it seems like that object might have additional 
semantics for high resolution alternatives that may need to be considered.

I was saying that I think we should transfer the context from the main
thread to the worker thread. The worker thread should then be able to
use the context to draw directly to the screen without interacting
with the main thread.

You should generally not need to transfer pixel data. But I think it
should be possible to grab pixel data on whichever thread is currently
owning a context. This will implicitly make it possible to transfer
pixel data if anyone wants to do it.


OK, that's the same concept as I was hoping for with 
document.getCSSCanvasContext.


Mozilla is a bit ahead of webkit, going with  element(any-element) -- 
webkit has -webkit-canvas(css-canvas-id).


It's a very different approach. I'd still go ahead and just have 
requestAnimationFrame pump events from the main frame.
There are so many things that could block the main frame to where an rAF 
repaint just isn't necessary from the worker-end.


It may also keep some animations continuing. I'm not positive, but 
something like window.prompt may block execution on the main thread, but 
if rAF were running on the worker, its repaints would still happen.
Generally, while the main thread is blocked, repainting on it is not 
going to be that useful; we're shuffling the repaints off so they aren't 
the cause of blocking.


Those are the two sides of it that I see... but it's still a very 
different proposal than the idea of just having non-transferable canvas 
contexts.


...

Boris brought up some good points about possible blocking in 
implementations with loading SVG images in the worker thread via Picture 
(or a chopped down Image). At present, those sound more like 
implementation-issues, not particularly issues in the validity of a spec.

http://lists.w3.org/Archives/Public/public-webapps/2012AprJun/0744.html

They do sound a little unfortunate, but I'd sooner trade some possible 
blocking between a Worker and the main thread, than having to carry a 
full SVG parser written in JS in my worker threads.
As an author, I've always had to take precautions for SVG and decide 
whether I want to render or give myself up to the browser implementation.


-Charles

Re: exposing CANVAS or something like it to Web Workers

On Mon, May 14, 2012 at 6:23 PM, Jonas Sicking jo...@sicking.cc wrote:

 On Mon, May 14, 2012 at 5:10 PM, Charles Pritchard ch...@jumis.com
 wrote:
  On May 14, 2012, at 4:42 PM, Jonas Sicking jo...@sicking.cc wrote:
 
  On Mon, May 14, 2012 at 3:28 PM, Glenn Maynard gl...@zewt.org wrote:
  On Mon, May 14, 2012 at 3:01 PM, Gregg Tavares (勤) g...@google.com
 wrote:
 
  I'd like to work on exposing something like CANVAS to web workers.
 
  Ideally how over it works I'd like to be able to
 
  *) get a 2d context in a web worker
 
 
  I'd recommend not trying to tackle 2d and 3d contexts at once, and only
  worrying about WebGL to start.
 
  Another issue: rendering in a worker thread onto a canvas which is
 displayed
  in the main thread.  This needs to be solved in a way that doesn't
 cause the
  asynchronous nature of what's happening to be visible to scripts.
  toDataURL
  and toBlob would probably need to be prohibited on the canvas element.
  I'm
  not sure what the actual API would look like.
 
  If/when we do this, I think it should be done in such a way that the
  main window can't access the canvas object at all. Similar to what
  happens when an ArrayBuffer is transferred to a Worker using
  structured cloning. Once a canvas is transferred to a Worker, any
  access to it should throw or return null/0/. If you want to transfer
  pixel data to the main thread, it seems less racy to do that by
  getting the pixel data in the Worker which owns the canvas and then
  transfer that to the main thread using postMessage.
 
  This would also require some
  equivalent to requestAnimationFrame in the worker thread.
 
  Agreed!
 
  / Jonas
 
 
 
  I'm a bit lost-- wouldn't we just postMessage from the document over to
 the web worker when we want a refresh?
 
  I agree that we ought to be transferring pixel data not Canvas contexts;
 with the possible exception of CSS context.
 
  We could just create new canvas instances inside the worker thread. I'd
 still prefer to clone CanvasPattern as a means of transferring paint over
 to the worker, though sending pixel data would work too.
 
  I heard Picture come up-- it seems like that object might have
 additional semantics for high resolution alternatives that may need to be
 considered.

 I was saying that I think we should transfer the context from the main
 thread to the worker thread. The worker thread should then be able to
 use the context to draw directly to the screen without interacting
 with the main thread.


That's a great goal but is transferring the right way to do it?

context.canvas is currently a reference to the canvas from which the
context as created. Will that just go NULL on transfer? Will it change to
some other object that is not a Canvas but acts similarly?

What happens to the context on the main page? All of it's methods become
no-ops and it's properties become non-live?

How will things synchronize if I call canvas.toDataURL after passing the
context to a work? Exception? nothing? It seems hard to define how that
would synchronize

If there was a way to allow a worker to generate a media stream and one of
those ways involves an in worker canvas/canvassurface, then it seems like
none of the above questions have to be answered. Instead the worker creates
the surface, and then through some other API attached it to a video tag.
You can call video.toDataURL, that's well defined (I assume)

Synchronization is also somewhat defined in that direction. Doesn't that
seem like the path of least resistance?






 You should generally not need to transfer pixel data. But I think it
 should be possible to grab pixel data on whichever thread is currently
 owning a context. This will implicitly make it possible to transfer
 pixel data if anyone wants to do it.

 / Jonas

Re: exposing CANVAS or something like it to Web Workers


On 5/14/2012 6:07 PM, Boris Zbarsky wrote:

On 5/14/12 8:55 PM, Gregg Tavares (勤) wrote:

1)  Various canvas 2d context methods depend on the styles of the
canvas to define how they actually behave.  Clearly this would need
some sort of changes for Workers anyway; the question is what those
changes would need to be.

Which methods are these?


Anything involving setting color (e.g. the strokeStyle setter, the 
fillStyle setter), due to currentColor.  Anything involving text 
because font styles come from the element or document.


Those are the ones that come to mind offhand, but I haven't looked at 
the various recent additions to the 2d context closely.


What would web fonts do in this situation, in Mozilla? If I've confirmed 
that a font is loaded in the main thread, would it be available to a 
worker for use in rendering?


Some implementations of Canvas font methods have been buggy, but 
dropping fillText/strokeText altogether would be a loss and may look 
strange in the specs.
I've seen font not working correctly with other canvas state variables 
in WebKit + Chrome; so it's not as though font is fully supported in the 
main thread, currently.


It may be easier to postpone Image / Picture semantics in workers. I 
think patterns could still be salvaged from createPattern.

It sounds like there are practical issues in current implementations.

When authoring,
With a picture, I can still xhr request it in the worker, then I send it 
to the main thread, and load it in the main thread, then send it back to 
the worker.
As an author, if I'm loading images I expect them to be a normal part of 
the page's load time and responsiveness I can still off-load my 
js-heavy work onto workers.


In Canvas 2d anyway, I can easily get by without picture in workers: (a 
= new Picture()).src = ; a.onload { ctx.drawImage(a,0,0); };  if I 
can use fillRect with pattern.
Otherwise, I'd have to do the extra steps of pushing pixel arrays back 
and forth.


All of the extra work is still worth it to get work done in workers, in 
cases where this level of work is needed. It's just a few extra lines of 
JS code (every time).


-Charles

Re: exposing CANVAS or something like it to Web Workers


On 5/14/12 10:00 PM, Charles Pritchard wrote:

What would web fonts do in this situation, in Mozilla?


Probably cry.  ;)


If I've confirmed that a font is loaded in the main thread, would it be 
available to a
worker for use in rendering?


Not without some pretty serious reworking.  Which might need to happen.

Of course basic text layout would also not be available without some 
serious reworking (e.g. making the textrun cache threadsafe or creating 
per-thread textrun caches or something), so the question of web fonts is 
somewhat academic at the moment.


-Boris

Re: exposing CANVAS or something like it to Web Workers


On 5/14/2012 7:09 PM, Boris Zbarsky wrote:

On 5/14/12 10:00 PM, Charles Pritchard wrote:

What would web fonts do in this situation, in Mozilla?


Probably cry.  ;)

If I've confirmed that a font is loaded in the main thread, would it 
be available to a

worker for use in rendering?


Not without some pretty serious reworking.  Which might need to happen.

Of course basic text layout would also not be available without some 
serious reworking (e.g. making the textrun cache threadsafe or 
creating per-thread textrun caches or something), so the question of 
web fonts is somewhat academic at the moment.




I meant solely for Canvas 2d.

I can live with staying away from fillText/strokeText on a worker thread 
if I'm loading fonts.
It's been broken on the main thread anyway, requiring intermediate 
Canvas surfaces for some operations.


...

SVG image and drawImage is mixed anyway; we can't transfer the data 
between threads as drawImage SVG will usually flag the Canvas as dirty 
in implementations.


We could just use Canvas 2d to handle pattern uploads for WebGL. Seems 
like that'd work without requiring fancy footwork to gain Picture/Image 
support in the worker.

SVG images would get fixed some other day.


-Charles

Re: exposing CANVAS or something like it to Web Workers


On 5/14/12 10:18 PM, Charles Pritchard wrote:

On 5/14/2012 7:09 PM, Boris Zbarsky wrote:

On 5/14/12 10:00 PM, Charles Pritchard wrote:

What would web fonts do in this situation, in Mozilla?


Probably cry. ;)


If I've confirmed that a font is loaded in the main thread, would it
be available to a
worker for use in rendering?


Not without some pretty serious reworking. Which might need to happen.

Of course basic text layout would also not be available without some
serious reworking (e.g. making the textrun cache threadsafe or
creating per-thread textrun caches or something), so the question of
web fonts is somewhat academic at the moment.



I meant solely for Canvas 2d.


Yes, I understand that.  Canvas 2d text still needs to be able to do 
things like font fallback, shaping, bidi, etc, etc, etc. last I checked.


-Boris

Re: exposing CANVAS or something like it to Web Workers


On 5/14/2012 7:24 PM, Boris Zbarsky wrote:

On 5/14/12 10:18 PM, Charles Pritchard wrote:

On 5/14/2012 7:09 PM, Boris Zbarsky wrote:

On 5/14/12 10:00 PM, Charles Pritchard wrote:

What would web fonts do in this situation, in Mozilla?


Probably cry. ;)


If I've confirmed that a font is loaded in the main thread, would it
be available to a
worker for use in rendering?


Not without some pretty serious reworking. Which might need to happen.

Of course basic text layout would also not be available without some
serious reworking (e.g. making the textrun cache threadsafe or
creating per-thread textrun caches or something), so the question of
web fonts is somewhat academic at the moment.



I meant solely for Canvas 2d.


Yes, I understand that.  Canvas 2d text still needs to be able to do 
things like font fallback, shaping, bidi, etc, etc, etc. last I checked.


Oh, the rendering isn't thread safe either? Yes, Canvas 2d text does [is 
supposed to] use all of those items.


Well, I'll give up strokeText/fillText entirely in workers if it'll get 
me the goods faster.
For a11y, I'm going to need to track my text in the main thread anyway. 
I can pre-render there if need be.


-Charles

Re: exposing CANVAS or something like it to Web Workers