PointerLock and Gamepad APIs

[Darin Fisher]

Back in September, it was proposed to expand the charter of the WebEvents
WG to include PointerLock (formerly known as MouseLock) and Gamepad APIs
[1].  This seemed like a logical home for them given that both of these
APIs pertain to input event systems.

However, one thing that became apparent was that Apple was not willing to
join the WebEvents WG [2].  Since we, the Chrome team at Google, work on
WebKit alongside Apple engineers, it is a bit undesirable for us to not be
able to work together with Apple in the WGs, which develop the specs that
we implement in WebKit.  From an efficiency point of view, it is far easier
if we can discuss APIs in a single forum instead of having to relay
concerns between forums.

I'd like to therefore propose that instead of expanding the charter of
WebEvents to include PointerLock and Gamepad, that we instead add those
APIs to another WG such as WebApps.  I believe they make sense in WebApps
given the scope of work being done there and the parties involved.

Thoughts?
-Darin

[1]
http://lists.w3.org/Archives/Public/public-webevents/2011JulSep/0087.html

[2] Recently, Apple disclosed some patents for the Touch Events
specification [
http://lists.w3.org/Archives/Public/public-webevents/2011OctDec/0118.html],
which is currently the only deliverable for the WebEvents WG.

Re: [Pointer Lock] Few comments

[Darin Fisher]

On Thu, Dec 15, 2011 at 1:39 PM, Olli Pettay wrote:

> On 12/15/2011 11:27 PM, Vincent Scheib wrote:
>
>>
>>
>> On Thu, Dec 15, 2011 at 6:16 AM, Olli Pettay > > wrote:
>>
>>Hi all,
>>
>>few comments about the API
>>
>>(1)
>>currently
>>http://dvcs.w3.org/hg/__**webevents/raw-file/default/__**
>> mouse-lock.html
>>
>>
>> 
>> >
>>uses VoidCallback which isn't defined anywhere.
>>
>>I guess there should be something like
>>
>>void lock (in Element target,
>>   optional in LockSuccessCallback successCallback,
>>   optional in LockErrorCallback failureCallback);
>>
>>
>>[Callback,NoInterfaceObject]
>>interface LockSuccessCallback {
>>  void pointerLockSuccess();
>>};
>>
>>[Callback,NoInterfaceObject]
>>interface LockErrorCallback {
>>  void pointerLockFailure();
>>};
>>
>>Or if the new proposed callback syntax is used:
>>callback LockSuccessCallback = void pointerLockSuccess();
>>callback LockErrorCallback = void pointerLockFailure();
>>
>>
>> I used the concept of VoidCallback from other implemented specs. Are
>> there any issues with it other than that the spec should define
>> VoidCallback? e.g.
>> http://www.w3.org/TR/file-**system-api/#the-voidcallback-**interface
>> http://www.w3.org/TR/2009/WD-**DataCache-20091029/#**VoidCallback
>>
>
> well, in those specs VoidCallback is FunctionOnly= which it probably
> shouldn't be.
> But there is ongoing discussion about removing FunctionOnly= from
> WebIDL
>
>
>
>
>>
>>(2)
>>"If another element is locked a user agent must transfer the mouse
>>lock to the new target and call the pointerlocklost callback for the
>>previous target."
>>There is no such thing as 'pointerlocklost callback'
>>
>>
>> Spec typo, it should read "pointerlocklost event".
>>
>
> dispatch pointerlocklost event...
>
>
>
>
>
>>(3)
>>"Mouse lock must succeed only if the window is in focus and the
>>user-agent is the active application of the operating system"
>>What window? window object as in web page? Or OS level window?
>>What if lock is called in some iframe?
>>
>>
>> The intent is the user-agent window and tab (if tabbed UA). Other than
>> UA security considerations, I propose there be no difference between
>> lock calls from a top level document or an iframe. Suggestions welcome
>> for a way to make this more clear than rewriting to be, "... succeed
>> only if the user-agent window (and tab, if a tabbed browser) is in focus
>> ..."
>>
>
> I'm just worried about iframes being able to lock mouse.
> But perhaps that should be allowed only if the iframe is in
> the same domain as the top level document.
>
>
The fullscreen API requires that the IFRAME tag have an "allowfullscreen"
attribute:
http://dvcs.w3.org/hg/fullscreen/raw-file/tip/Overview.html#security-and-privacy-considerations

Perhaps a similar approach would work for pointer lock?

-Darin



>
>
>
>
>>(4)
>>"If the target is removed from the DOM tree after mouse lock is
>>entered then mouse lock will be lost."
>>Should 'pointerlocklost' event be dispatched?
>>
>>
>> I'm not yet certain about the implementation practicalities, and need to
>> research more, but is seems we have these options:
>> a- don't send the event
>>
> A bit strange
>
>
>
>  b- send to the element after it has been detached
>>
> I would assume this. At least it would be consistent.
>
>
>
>  c- send to the nearest ancestor of the element that remains in the tree
>>
> Or perhaps send to the document. Should pointerlocklost always be
> dispatched to the document? If really needed, the event could have
> property .unlockedElement or some such.
>
>
>
>  d- send to the element before it is detached
>>
> this is not possible. Well, possible, but would bring in all the
> problems there are with mutation events
>
>
> -Olli
>
>
>

Re: [FileAPI] createObjectURL isReusable proposal

[Darin Fisher]

On Wed, Dec 14, 2011 at 4:40 PM, Ian Hickson  wrote:

> On Wed, 14 Dec 2011, Adrian Bateman wrote:
> >
> > [...] the first dereference of the URL revokes it.
> >
> > This means that you can do something like:
> >
> > imgElement.src = URL.createObjectURL(blob,false)
> >
> > and not worry about having to call URL.revokeObjectURL to release the
> > Blob.
>
> I think it's dangerous to assume that the URL will only be dereferenced
> once. For example, it would mean that the above image would break if the
> user toggled images off and back on in a browser that discarded image data
> when the user toggles images off. It would mean that you couldn't clone
> the element and get the image data in the clone. It would mean that you
> couldn't read the image's .src URL and have anything useful.
>

I share your concerns.  This is enough reason to make me not want to
support the one-shot flag.  That said, I'm greatly bothered by how easily
createObjectURL makes it to create web apps that accumulate mass
amounts of unreleased and unused memory.  Most examples gloss over
revokeObjectURL entirely.

The idea of just letting these blobs accumulate until the document is
destroyed is really unsatisfying.  A growing number of apps have very
long-lived main documents.  What if createObjectURL were somehow
scoped to an Element in the DOM (not just the Document itself)?

For AJAX apps that are creating and tearing down DOM snippets
frequently, if the blob URLs needed for a particular view were bound to
elements in that view, then perhaps there would be a more natural
cleansing of the blob URL registry.

I'm not sure what a concrete proposal would look like.  Maybe
Element.URL.createObjectURL or just Element.createObjectURL?

-Darin


>
> I think the better solution is to have implementations make keeping object
> URLs defined be very cheap, so that nobody needs to ever release them.
>

Re: [xhr] responseType for sync requests in window context

[Darin Fisher]

Please keep in mind that multiple web apps share the same main thread.
 While browser's can keep their own UI responsive in the face of
synchronous XHRs, they cannot do the same for other web app, which may
share their main thread with a web app that uses synchronous XHR.
 Unfortunately, the web platform is a cooperatively multi-tasking system :-(

This applies even to browsers like Chrome that create multiple processes.
 There you can still end up with unrelated web apps in the same process,
sharing the same main thread.

Now, I have to admit that I'm quite sympathetic to your concerns.  I think
it is always bad form for a platform to give an API and then take it away.

At least one nice thing about switching to async XHRs is that you can now
display a progress meter to your users.  There's no telling how long a
network load may take afterall.

Regards,
-Darin


On Thu, Jan 26, 2012 at 7:47 PM, Mark Callow wrote:

>  I do not think you should be in the business of brute-forcing authors
> into converting their applications to use async XHRs. As far as I
> understand it, it is the application's UI that may be unresponsive during a
> sync XHR. In that case it should be the app. authors choice which to use.
> If it is the browser's UI, that is a bug in the browser.
>
> Since this change principally affects WebGL app's it would have been nice
> of someone to have mentioned this change in the public-webgl mailing list
> while it was still at the proposal stage.
>
> For the record the change has broken all our WebGL applications. Forcing
> us to jump through the hoops of using async XHRs is going to have zero
> impact on the user experience in our case because the 3D itself is the UI.
> Until its loaded there is nothing the user can do.
>
> If sync XHRs are so bad, why to they exist?
>
> Regards
>
> -Mark
>
>

Re: FileSystem API: Adding file size field to Metadata?

[Darin Fisher]

On Tue, Feb 28, 2012 at 10:47 AM, Kinuko Yasuda  wrote:

> Hi,
>
> While looking at the FileSystem API draft I noticed that we only expose
> 'modificationTime' in 'Metadata' object.  Since FileEntry itself doesn't
> have 'size' field unlike File object maybe it's reasonable to add 'size'
> field to Metadata?
>
> http://www.w3.org/TR/file-system-api/#the-metadata-interface
>
> Without adding this we can indirectly get the file size by creating 'File'
> object via FileEntry.file() method and accessing File.size, but when an app
> wants to display the modificationTime and file size at once (and it sounds
> very plausible use case) the app needs to call two different async
> methods-- which doesn't sound very nice.  WDYT?
>
> Thanks,
> Kinuko
>
>
>

I think this is a nice improvement.  File size is very obviously something
one might expect to be included in meta data for a file :-)

-Darin

Re: File API "oneTimeOnly" is too poorly defined

[Darin Fisher]

On Wed, Mar 28, 2012 at 5:49 PM, Jonas Sicking  wrote:

> On Wed, Mar 28, 2012 at 4:36 PM, Glenn Maynard  wrote:
> > Here's another proposal, which is an iteration of the previous.  It's
> based
> > on the "microtask" concept, which is creeping up here and there but
> hasn't
> > yet been properly defined.  The idea is that microtasks can be queued
> (call
> > it "queue a microtask"), and the microtask queue is executed by the event
> > loop as soon as the current task completes, so it executes as soon as the
> > outermost task returns to the event loop.
> >
> > oneTimeOnly (a poor name in this proposal) would simply queue a
> microtask to
> > revoke the URL.
> >
> > This is simpler, and answers a lot of questions.  It means you can use
> the
> > URL as many times as you want synchronously, since it's not released
> until
> > the script returns.  Any cases where the ordering may not be strictly
> > defined (eg. the  case in
> > http://lists.w3.org/Archives/Public/public-webapps/2012JanMar/1265.htmlmay
> > be like this; I don't know how innerHTML works, exactly) are now defined:
> > both  elements would get the object.
> >
> > It has another nice side-effect: it's much less prone to leaks.  For
> > example, under previous approaches, the following code would leak the
> blob:
> >
> > function updateProgressMeter() { throw "obscure error"; }
> > url = URL.createObjectURL(blob, {oneTimeOnly: true});
> > updateProgressMeter();
> > img.src = url; // never happens
> >
> > Since the URL is never actually used, the blob reference leaks.  You'd
> have
> > to work around this with careful exception handling, which is precisely
> the
> > sort of thing oneTimeOnly is supposed to avoid.  With this proposal, the
> URL
> > would always be revoked when the script returns to the event loop,
> whether
> > or not it was actually used.
> >
> > This would still require work in each URL-consuming spec, to define
> taking a
> > reference to the underlying blob's data when it receives an object URL.
>  I
> > think this is inherent to the feature.
> >
> > "oneTimeOnly" would be the wrong name with this approach; it should be
> > something like "autoRelease".
> >
> > This has one drawback: it doesn't work nicely in long-running Workers,
> which
> > may never return to the event loop at all.  I think that's probably an
> > acceptable tradeoff.
>
> This is an interesting idea for sure. It doesn't solve any of the
> issues I brought up, so we still need to define when dereferencing
> happens. But it does solve the problem of the URL leaking if it never
> gets dereferenced, which is nice.


I've never been terribly happy with createObjectURL and the requirement for
folks to remember to call revokeObjectURL.  I really like that we're talking
about ways to minimize this pain :-)

I noticed the WeakRefs proposal:
http://wiki.ecmascript.org/doku.php?id=strawman:weak_refs

It also makes use of the micro-task concept, and it does so to avoid
revealing
details about garbage collection.

What if we were to adopt a similar approach here.  Instead of introducing a
second parameter to createObjectURL, what if createObjectURL returned a
WeakObjectURL object instead of a String object?  WeakObjectURL could
be converted to a String to reveal the Blob URL.

Suppose WeakObjectURL if retained would keep the Blob URL valid.  Else,
when WeakObjectURL gets deleted, its Blob URL would remain alive up
until the next micro-task.

Crazy idea?  Too crazy?

I agree that it is valuable to define "dereference" points for APIs that
receive
Blob URLs.

-Darin

Re: Moving File API: Directories and System API to Note track?

[Darin Fisher]

Note: The {File,Directory}Entry types are also separately useful for
multi-file input and drag-n-drop applications:
http://wiki.whatwg.org/wiki/DragAndDropEntries

To summarize, it solves several problems with FileList:

1-  A DirectoryEntry can be lazily resolved by the developer.  With
FileList, the browser has to fully traverse a potentially deep directory
structure before it can provide the application with the FileList.  That
can be very slow.  DirectoryEntry enables applications to provide progress
UI.

2-  FileList is not good at representing empty directories.  Empty
directories can be important for cases where a user wishes to upload a
directory structure to a server.

(This has been implemented in Chrome for a little while now.)

My understand of the sysapps working group was that it was for APIs that
are not generally suitable for use in the conventional web sandbox.  They
are for use in a different ("system apps") sandbox.  What I've described
above is useful in both, and so standardizing in webapps seems to make
sense.

Regards,
-Darin


On Tue, Sep 18, 2012 at 4:11 AM, Robin Berjon  wrote:

> On 18/09/2012 13:04 , Olli Pettay wrote:
>
>> I think we should discuss about moving File API: Directories and
>> System API from Recommendation track to Note. Mainly because the API
>> hasn't been widely accepted nor implemented and also because there
>> are other proposals which handle the same use cases. The problem with
>> keeping the API in recommendation track is that people outside
>> standardization world think that the API is the one which all the
>> browsers will implement and as of now that doesn't seem likely.
>>
>
> I wonder if a good case couldn't be made to move it to the upcoming
> SysApps WG 
> (http://www.w3.org/2012/05/**sysapps-wg-charter.html
> )?
>
> It's very late to sneak anything into the charter, but I reckon it could
> fall under their Media Storage API deliverable.
>
> --
> Robin Berjon - http://berjon.com/ - @robinberjon
>
>

Re: Moving File API: Directories and System API to Note track?

[Darin Fisher]

On Wed, Sep 19, 2012 at 11:50 PM, James Graham  wrote:

>
>
> On Wed, 19 Sep 2012, Adam Barth wrote:
>
>  On Wed, Sep 19, 2012 at 1:46 PM, James Graham  wrote:
>>
>>> On Wed, 19 Sep 2012, Edward O'Connor wrote:
>>>
 Olli wrote:

> I think we should discuss about moving File API: Directories and
> System API from Recommendation track to Note.
>

 Sounds good to me.

>>>
>>> Indeed. We are not enthusiastic about implementing an API that has to
>>> traverse directory trees as this has significant technical challenges, or
>>> may expose user's path names, as this has security implications. Also
>>> AIUI
>>> this API is not a good fit for all platforms.
>>>
>>
>> There's nothing in the spec that exposes user paths.  That's just FUD.
>>
>
> I was thinking specifically of the combination of this and Drag and Drop
> and this API. I assumed that at some level one would end up with a bunch on
> Entry objects which seem to expose a path. It then seems that then a user
> who is tricked into dragging their root drive onto a webapp would expose
> all their paths.
>
> It is quite possible that this is a horrible misunderstanding of the spec,
> and if so I apologise. Nevertheless I think it's poor form to immediately
> characterise an error as a deliberate attempt to spread lies.
>
> In any case my central point remains which is that would support this spec
> moving off the Rec. track at this time.
>
>
File path information is already exposed via .

"File names may contain partial paths."
http://www.whatwg.org/specs/web-apps/current-work/multipage/states-of-the-type-attribute.html#concept-input-type-file-selected

-Darin

Re: Moving File API: Directories and System API to Note track?

[Darin Fisher]

No comment on the value of DirectoryEntry for enabling asynchronous listing
of files in ?

-Darin

On Thu, Sep 20, 2012 at 4:48 PM, Maciej Stachowiak  wrote:

>
> +1
>
> I don't see an indication of any major browser but Chrome planning to
> implement this and expose it to the Web.
>
>  - Maciej
>
> On Sep 18, 2012, at 4:04 AM, Olli Pettay  wrote:
>
> > Hi all,
> >
> >
> > I think we should discuss about moving File API: Directories and System
> API
> > from Recommendation track to Note. Mainly because the API hasn't been
> widely accepted nor
> > implemented and also because there are other proposals which handle the
> same use cases.
> > The problem with keeping the API in recommendation track is that people
> outside
> > standardization world think that the API is the one which all the
> browsers will implement and
> > as of now that doesn't seem likely.
> >
> >
> >
> >
> > -Olli
> >
>
>
>

Re: Paths exposed in input type=file (was: Re: Moving File API: Directories and System API to Note track?)

[Darin Fisher]

On Fri, Sep 21, 2012 at 1:14 AM, James Graham  wrote:

> On 09/20/2012 11:45 PM, Darin Fisher wrote:
>
>  File path information is already exposed via .
>>
>> "File names may contain partial paths."
>> http://www.whatwg.org/specs/**web-apps/current-work/**
>> multipage/states-of-the-type-**attribute.html#concept-input-**
>> type-file-selected<http://www.whatwg.org/specs/web-apps/current-work/multipage/states-of-the-type-attribute.html#concept-input-type-file-selected>
>>
>
> I couldn't get any actual browser to expose paths in this way. Did I miss
> something? It also seems to be encouraging wilful violation of the FileAPI
> spec which clearly states that File objects don't return path info [1].
>
> Having a may-level conformance criterion that supposes to override a
> must-level criterion in another spec, but no strong compat. requirement,
> doesn't seem like the optimum path to interoperability or sanity of people
> trying to understand and implement the platform. Is there any reason this
> paragraph shouldn't be considered a bug in HTML?
>

I'm not sure.

Note: Chrome exposes relative file path information via
File.webkitRelativePath.  I haven't reviewed the history of this property.

Exposing partial path information like this is useful in conjunction with
 [*].  It enables people to upload directories
of files to a web server.  The web server can re-create the exact directory
structure, including empty directories.

Regards,
-Darin

[*]
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2010-April/025764.html


>
> [1] 
> http://dev.w3.org/2006/webapi/**FileAPI/#dfn-file<http://dev.w3.org/2006/webapi/FileAPI/#dfn-file>
>
>
>

Re: Moving File API: Directories and System API to Note track?

[Darin Fisher]

Or.. drag-n-drop receipt of a folder or set of folders.

On Fri, Sep 21, 2012 at 3:50 PM, Darin Fisher  wrote:

> No comment on the value of DirectoryEntry for enabling asynchronous
> listing of files in ?
>
> -Darin
>
>
> On Thu, Sep 20, 2012 at 4:48 PM, Maciej Stachowiak  wrote:
>
>>
>> +1
>>
>> I don't see an indication of any major browser but Chrome planning to
>> implement this and expose it to the Web.
>>
>>  - Maciej
>>
>> On Sep 18, 2012, at 4:04 AM, Olli Pettay  wrote:
>>
>> > Hi all,
>> >
>> >
>> > I think we should discuss about moving File API: Directories and System
>> API
>> > from Recommendation track to Note. Mainly because the API hasn't been
>> widely accepted nor
>> > implemented and also because there are other proposals which handle the
>> same use cases.
>> > The problem with keeping the API in recommendation track is that people
>> outside
>> > standardization world think that the API is the one which all the
>> browsers will implement and
>> > as of now that doesn't seem likely.
>> >
>> >
>> >
>> >
>> > -Olli
>> >
>>
>>
>>
>

Re: Streams and Blobs

[Darin Fisher]

On Tue, Feb 26, 2013 at 9:40 AM, Aaron Colwell wrote:

>
> On Tue, Feb 26, 2013 at 9:12 AM, Anne van Kesteren wrote:
>
>> On Tue, Feb 26, 2013 at 4:50 PM, Aaron Colwell 
>> wrote:
>> > One other data point is that we are using the Stream as an opaque
>> handle for
>> > routing data to the Media Source Extensions. (See
>> > SourceBuffer.appendStream()). The idea here is that it allows the data
>> from
>> > an XHR to be transferred to the SourceBuffer object w/o having to
>> surface
>> > the data to JavaScript. The Stream object seemed like a nice
>> abstraction for
>> > this since it doesn't have the immutable properties implied by Blobs. It
>> > also hides the source of the data from MSE which is nice. I'm not saying
>> > that Stream is the only way to handle this, but it is why parts of the
>> > Streams API are attractive to MSE.
>>
>> Okay, so we want to keep something like Stream around. Do you need a
>> Blob without size kind of object? E.g. that would mean it cannot have
>> "chunked" semantics (the stuff you read is thrown away), which is
>> something we probably want for XMLHttpRequest. It seems like these are
>> the types we've been discussing:
>>
>> * Blob fixed in size
>> * Blob that grows over time
>> * Blob that grows over time and shrinks from the start on read
>>
>
> This last one is the use case I'm primarily interested in and I believe it
> is what Stream was intended to model. As long as XHR can return one of
> these types of Blobs and there is a way for JavaScript to create and feed
> data into a Blob of this type, then I'm happy.
>

In other environments w/ stream concepts, a non-seekable stream, looks like
the last one too.  It seems really useful for the web platform to provide a
way to refer to a stream of data.  I can imagine many applications outside
of just XHR response handling.

Being able to efficiently pipe data from a data provider to a data consumer
seems useful.  XHR seems like just one example of a data provider.

-Darin




>
>
>>
>> Not entirely sure which one Stream represents at the moment. Also not
>> entirely sure if we want to support this as discrete objects or if we
>> want another approach as outlined in earlier emails.
>>
>
> I like having a single object representing the data flow like Stream or a
> "growable Blob" does. That is just personal preference though and I'm
> interested in hearing arguments for the sequence of Blobs if people have a
> particular preference for that model.
>
> Aaron
>

Re: Streams and Blobs

[Darin Fisher]

On Tue, Feb 26, 2013 at 5:10 PM, Glenn Maynard  wrote:

> On Tue, Feb 26, 2013 at 11:12 AM, Anne van Kesteren wrote:
>
>>  Okay, so we want to keep something like Stream around. Do you need a
>> Blob without size kind of object? E.g. that would mean it cannot have
>> "chunked" semantics (the stuff you read is thrown away), which is
>> something we probably want for XMLHttpRequest. It seems like these are
>> the types we've been discussing:
>>
>> * Blob fixed in size
>> * Blob that grows over time
>> * Blob that grows over time and shrinks from the start on read
>>
>
> For taking an XHR response and handing it off to some native API, the
> Stream interface does seem like the right abstraction: it represents a
> logical stream, not a block of data like Blob.  The only parts of the
> Streams API needed to make this work are the simple Stream interface (which
> is nothing but an object with a MIME type) and createObjectURL support for
> it.  You don't need StreamReader or StreamBuilder.
>
> You'd want XHR itself to behave a bit differently if it returns a Stream,
> though.  It should never enter LOADING, instead finishing as soon as
> headers complete, as if XHR hands off the file descriptor to the stream
> object as soon as the headers finish.
>
>
Interesting idea, but what is the motivation for this change?  Why would
the XHR (data provider) state machine need to change?



> That said, I'm not fully convinced about the overall picture of Stream:
>
> - What happens if you start a video stream, then the user pauses the video
> and goes away?  The browser (or server) will want to shut down the TCP
> stream and reopen it again if the user unpauses a day later.
>

This seems like a problem for the application.  If an error is detected,
then the application can do what it needs to restart the stream.
 Presumably, the application would be built to only create streams that can
be restarted if pausing is a mode it wants to support.

-Darin



> If the stream comes from an XHR POST, it's not allowed to do that.
> Similarly, you have to deal with restarting the stream after a dropped
> connection.
> - What about seekable streams (eg. streaming large but static videos,
> rather than a live feed)?  You can't do it, for the same reason.
>
> If you structure services with the GET, you don't have any of these
> problems; the browser can stop, restart and seek the request whenever it
> wants; and we don't need any new APIs.  What are the actual use cases for
> POST (or other custom XHR-based requests) that are so hard to structure as
> a GET (or an initial authenticating POST that replies with a GET URL) to
> warrant all this?
>
> --
> Glenn Maynard
>
>

Re: Streams and Blobs

[Darin Fisher]

On Tue, Feb 26, 2013 at 5:55 PM, Glenn Maynard  wrote:

> On Tue, Feb 26, 2013 at 7:16 PM, Darin Fisher  wrote:
>
>> Interesting idea, but what is the motivation for this change?  Why would
>> the XHR (data provider) state machine need to change?
>>
>
> XHR is a consumer, not a provider: it consumes data from the HTTP stream
> and returns it in one form or another.
>

I was thinking of XHR as the proxy between the underlying HTTP stream and
the Stream object.  Since XHR is used to setup the HTTP stream, it seemed
reasonable to think of it as the producer.  But, I see your point...
especially when you think about progress events and such.



>   The provider is the HTTP stream itself.  If XHR continues to run and
> read data from the HTTP connection after creating the Stream, that
> essentially means we have two consumers for the same provider.
>
> That's confusing.  For example, if nobody is reading from the Stream, you
> want to stop reading from the TCP socket so data buffers and the stream
> backs up.  If XHR continues to be a consumer of the HTTP stream (as it is
> in other modes), then it would keep reading full-speed from the socket even
> if the Stream isn't being read.  (That, or it means it "ghosts" along as
> the Stream is read by some other API, which is weird.)  It also means we
> have two objects returning things like progress events for the same stream,
> which is trickier to get right--you'd need to define the ordering for the
> two sets of events, which probably happen in different task sources.  By
> getting XHR out of the picture as soon as its work is done, all of this
> goes away.
>
>
I didn't mean to suggest that XHR would continue consuming data from the
HTTP stream.  I just assumed that XHR would remain a controller for the
Stream.  For instance, I assumed that XHR.abort() would continue to work,
but that doesn't seem like a requirement.  The approach you outline seems
reasonable.

Hmm, now I wonder how window.stop() should work with a Stream generated
from XHR.  Should window.stop() interrupt the Stream?  My assumption is
that it should since it is backed by a network request.

I assume that even if the Stream is not done downloading that it should be
safe to reuse the XHR object that provided the Stream, right?  Hmm, I can
imagine some implementation complexity arising from saying that a XHR is
done before the Stream is done.

-Darin



> Think of it as if XHR always reads from a Stream object, and all it's
> doing in "stream" mode is stopping after headers and giving you the
> underlying stream to do something else with.
>
>
>
>>  That said, I'm not fully convinced about the overall picture of Stream:
>>>
>>> - What happens if you start a video stream, then the user pauses the
>>> video and goes away?  The browser (or server) will want to shut down the
>>> TCP stream and reopen it again if the user unpauses a day later.
>>>
>>
>> This seems like a problem for the application.  If an error is detected,
>> then the application can do what it needs to restart the stream.
>>  Presumably, the application would be built to only create streams that can
>> be restarted if pausing is a mode it wants to support.
>>
>
> It's not just pausing, it's resuming the stream after the TCP connection
> is closed for any reason, like a network hiccup.  Everyone should want to
> get that right, not just people who want to support pausing.  This is a
> problem for every application using this API, and it seems tricky to get
> right, which in a web API suggests there's something wrong with the API
> itself.  Handling seeking seems even harder.  If people use GET requests,
> and don't interject XHR in at all, the whole category of problems goes away.
>
> What are some actual use cases?  I don't think any have been mentioned on
> this thread.
>
> --
> Glenn Maynard
>
>

Re: Streams and Blobs

[Darin Fisher]

On Wed, Feb 27, 2013 at 7:24 AM, Glenn Maynard  wrote:

> On Wed, Feb 27, 2013 at 12:02 AM, Darin Fisher  wrote:
>
>> I assume that even if the Stream is not done downloading that it should
>> be safe to reuse the XHR object that provided the Stream, right?  Hmm, I
>> can imagine some implementation complexity arising from saying that a XHR
>> is done before the Stream is done.
>>
>
> Out of curiosity, what's the complexity?  Once XHR enters DONE, doesn't
> calling open() again pretty much throw away the whole internal state of the
> client and start from scratch?
>

This assumes a refactoring of the guts of XHR to separate the loader
portion so that it can be connected to a Stream and divorced from the XHR
that created it.  The point, I assume, is so that a subsequent XHR.open()
doesn't interrupt an existing Stream.

-Darin


>
>
> On Wed, Feb 27, 2013 at 12:59 AM, Tillmann Karras wrote:
>
>> On 2013-02-27 02:55, Glenn Maynard wrote:
>>
>>> What are some actual use cases?  I don't think any have been
>>> mentioned on this thread.
>>>
>>
>> - I would like to stream realtime TV. Pausing shouldn't be a problem
>> because I don't rely on POST requests and I would just buffer up to a
>> certain limit.
>>
> - Another use case that comes to mind is starting to watch a video file
>> before it is fully downloaded.
>>
>
> You don't need XHR or Stream for this.  Just point video @src at the
> stream.
>
>
>> - Another one would be downloading only the beginning of a file, e.g. for
>> file type identification / thumbnails/ MP3 tags and so on. (How much data
>> is needed is probably not known up front and just using increasing Range
>> headers would require multiple round trips.)
>> - Jonas mentioned incremental rendering of large Word or PDF documents.
>>
>
> These are separate.  We're talking about taking the result of an XHR and
> handing it off to a native API like .  There's a separate discussion
> for incremental reads in JavaScript, which doesn't involve Stream at all
> (search the thread for clearResponse for my rough proposal).
>
> The use cases I'm asking for are where you want to hand a stream off to a
> native API like , but where you can't arrange it so it's a simple
> URL--that is, where you need to make a POST request or set custom headers
> in the stream request.  I can't think of any good reason to do this.  For
> example, you might need to authenticate with a POST before starting to
> stream, but that'd be better done by doing a separate POST, receiving an
> authentication token in response, then putting the token in a simple URL to
> be used for the actual stream, or in a cookie depending on security needs.
> The stream itself shouldn't be POST.
>
> --
> Glenn Maynard
>
>

Re: Streams and Blobs

[Darin Fisher]

On Wed, Feb 27, 2013 at 8:54 AM, Aaron Colwell wrote:

>
>
>
> On Tue, Feb 26, 2013 at 5:55 PM, Glenn Maynard  wrote:
>
>> On Tue, Feb 26, 2013 at 7:16 PM, Darin Fisher  wrote:
>>
>>> Interesting idea, but what is the motivation for this change?  Why would
>>> the XHR (data provider) state machine need to change?
>>>
>>
>> XHR is a consumer, not a provider: it consumes data from the HTTP stream
>> and returns it in one form or another.  The provider is the HTTP stream
>> itself.  If XHR continues to run and read data from the HTTP connection
>> after creating the Stream, that essentially means we have two consumers for
>> the same provider.
>>
>
> Yes it consume the data from the server, but it provides it to the Stream.
>
> On Tue, Feb 26, 2013 at 5:55 PM, Glenn Maynard  wrote:
>
>> That's confusing.  For example, if nobody is reading from the Stream, you
>> want to stop reading from the TCP socket so data buffers and the stream
>> backs up
>>
>
> That is not how I was assuming it would work. I assumed it would keep
> reading & buffer the data just like a normal XHR would do in the other
> modes.
>
> On Tue, Feb 26, 2013 at 5:55 PM, Glenn Maynard  wrote:
>
>> .  If XHR continues to be a consumer of the HTTP stream (as it is in
>> other modes), then it would keep reading full-speed from the socket even if
>> the Stream isn't being read.
>>
>
> Yes. This is what I would expect.
>
> On Tue, Feb 26, 2013 at 5:55 PM, Glenn Maynard  wrote:
>
>>   (That, or it means it "ghosts" along as the Stream is read by some
>> other API, which is weird.)  It also means we have two objects returning
>> things like progress events for the same stream, which is trickier to get
>> right--you'd need to define the ordering for the two sets of events, which
>> probably happen in different task sources.  By getting XHR out of the
>> picture as soon as its work is done, all of this goes away.
>>
>
> I'm viewing XHR as controlling the connection to the server and
> representing the progress of the data reaching the client. Progress on the
> Stream just represents when the Stream received data, from XHR or whatever
> other source created the object.
>
> In my opinion keeping XHR in the picture the whole time allows it to act
> just like it would if it was being used in all the other modes. I don't
> really see a need to special case this.
>

+1


>
> On Tue, Feb 26, 2013 at 5:55 PM, Glenn Maynard  wrote:
>
>> Think of it as if XHR always reads from a Stream object, and all it's
>> doing in "stream" mode is stopping after headers and giving you the
>> underlying stream to do something else with.
>>
>
> I think this may be where we have a difference of opinion. I'm thinking of
> XHR as the source of the stream not a reader of it. This is mainly because
> the stream represents  the body of the request, not the socket itself.
>
> On Tue, Feb 26, 2013 at 5:55 PM, Glenn Maynard  wrote:
>
> On Tue, Feb 26, 2013 at 7:16 PM, Darin Fisher  wrote:
>>
>  That said, I'm not fully convinced about the overall picture of Stream:
>>>>
>>>>
>>>> - What happens if you start a video stream, then the user pauses the
>>>> video and goes away?  The browser (or server) will want to shut down the
>>>> TCP stream and reopen it again if the user unpauses a day later.
>>>>
>>>
>>> This seems like a problem for the application.  If an error is detected,
>>> then the application can do what it needs to restart the stream.
>>>  Presumably, the application would be built to only create streams that can
>>> be restarted if pausing is a mode it wants to support.
>>>
>>
>> It's not just pausing, it's resuming the stream after the TCP connection
>> is closed for any reason, like a network hiccup.  Everyone should want to
>> get that right, not just people who want to support pausing.  This is a
>> problem for every application using this API, and it seems tricky to get
>> right, which in a web API suggests there's something wrong with the API
>> itself.  Handling seeking seems even harder.  If people use GET requests,
>> and don't interject XHR in at all, the whole category of problems goes away.
>>
>
> I don't expect XHR to have to deal with pausing. In most cases where the
> video tag would use Stream it would be for a live stream where pausing
> isn't an option. Even if it was, since all the video tag has is a
> non-seekable stream of bytes, it would have to either do its own caching or
> just give up and signal an error. It can't make any assumptions about the
> request because it doesn't know where the stream of bytes are coming from.
> Here we are only talking about XHR, but it could be from JavaScript, a
> WebSocket, or something else. The application knows that it is restricting
> the video tag in this way when it chooses to pass in a Stream. If pausing
> needs to be supported then Stream shouldn't be passed to the tag directly.
> Something like MSE or a standard video.src = URL should be used.
>
> Hope this helps,
> Aaron
>
>

Re: Streams and Blobs

[Darin Fisher]

On Wed, Feb 27, 2013 at 5:26 PM, Glenn Maynard  wrote:

> On Wed, Feb 27, 2013 at 10:39 AM, Aaron Colwell wrote:
>
>>  - I would like to stream realtime TV. Pausing shouldn't be a problem
 because I don't rely on POST requests and I would just buffer up to a
 certain limit.
>>>
>>>   - Another use case that comes to mind is starting to watch a video
 file before it is fully downloaded.

>>>
>>> You don't need XHR or Stream for this.  Just point video @src at the
>>> stream.
>>>
>>
>> In the Media Source Extensions context it is valuable to be able to
>> consume the data as it arrives instead of waiting for the whole transfer to
>> complete. The normal use case for MSE involves chunks of media several
>> seconds long and it is important to be able to process the data as it
>> arrives to help minimize startup time and media prefetching.
>>
>
> (This reply doesn't seem related to the text you're quoting.  Tillmann
> said he wants to be able to stream TV.  You don't need any of this to do
> that;  can do it already.)
>
> Sending data to MSE, or any other API, doesn't mean it has to be done with
> XHR.  Simply handing over a URL is much simpler.  Why do you want to go
> through XHR in any of these cases?
>
>  On Wed, Feb 27, 2013 at 7:24 AM, Glenn Maynard  wrote:
>>
>>> These are separate.  We're talking about taking the result of an XHR and
>>> handing it off to a native API like .  There's a separate discussion
>>> for incremental reads in JavaScript, which doesn't involve Stream at all
>>> (search the thread for clearResponse for my rough proposal).
>>>
>>
>> Why do these need to be thought about differently? It seems to me that
>> Stream simply represents a non-seekable stream of bytes. It seems
>> convenient to be able to hand this object to MSE, video, or any other
>> object native or otherwise that wants to consume such a stream. If
>> JavaScript needs access to the bytes then something like the StreamReader
>> seems reasonable to me just like FileReader seems reasonable for accessing
>> Blob data.
>>
>
> The whole initial point of this thread is that the Streams spec is a lot
> of API for streaming to script.  In response, the only argument made for
> Stream is for handing streams off to native (video or anything else).  If
> that can't be justified with use cases, then we're back where we
> started--if the only use cases we have are for streaming to script, we
> don't need Stream for that.  All we need to have to support that is one new
> method (and maybe another property, depending on web-compatibility).
>
> "It seems convenient" isn't good enough to justify adding something to the
> web, if there's a much simpler, more reliable way to do the same thing.
>
> I don't see why the difference between POST or GET usage matters. The XHR
>> is just being used to fetch data. I don't think we should bake any
>> assumptions into which methods people use.
>>
>
> The difference, more precisely, is between using XHR to initiate a fetch
> and just handing in a URL for the resource.  ("POST" vs. "GET" is a mild
> oversimplification of that, since POST is one thing you can't do by passing
> in a URL.)
>
> There's a huge, fundamental difference between a URL and an XHR-provided
> POST.  If you have a simple URL, eg. , the browser can open,
> close and seek the URL in any way it needs to.  The browser can handle
> pausing, seeking, and recovering interrupted streams, completely
> automatically.  If you have a POST initiated from XHR, the browser can't do
> any of that and it all gets pushed onto the developer.  (Even if it's a
> GET, it's not clear that the browser could restart it if it comes from XHR.)
>
> On Wed, Feb 27, 2013 at 10:54 AM, Aaron Colwell 
>  wrote:
>
>> That is not how I was assuming it would work. I assumed it would keep
>> reading & buffer the data just like a normal XHR would do in the other
>> modes.
>>
>
> I think this may be a more complicated model that will be harder to
> precisely define.  But, I don't think the need for the feature has been
> established, so for now I'll focus on that part of the discussion.
>
>  It's not just pausing, it's resuming the stream after the TCP connection
>>> is closed for any reason, like a network hiccup.  Everyone should want to
>>> get that right, not just people who want to support pausing.  This is a
>>> problem for every application using this API, and it seems tricky to get
>>> right, which in a web API suggests there's something wrong with the API
>>> itself.  Handling seeking seems even harder.  If people use GET requests,
>>> and don't interject XHR in at all, the whole category of problems goes away.
>>>
>>
>> I don't expect XHR to have to deal with pausing. In most cases where the
>> video tag would use Stream it would be for a live stream where pausing
>> isn't an option.
>>
>
> But again: what cases are those?  Why would you use XHR, create a Stream
> and pass the Stream to another API, instead of just giving the API the URL
> in 

Re: File API: Blob.type

[Darin Fisher]

On Wed, Mar 6, 2013 at 6:29 AM, Anne van Kesteren  wrote:

> On Wed, Mar 6, 2013 at 2:21 PM, Glenn Maynard  wrote:
> > Blob.type is a MIME type, not a Content-Type header.  It's a string of
> > codepoints, not a series of bytes.  XHR is a protocol-level API, so
> maybe it
> > makes sense there, but it doesn't make sense for Blob.
>
> It's a Content-Type header value and should have those restrictions.
> Making it a ByteString plus additional restrictions will make it do as

required.
>
>
So the intent is to allow specifying attributes like "charset"?  That
sounds useful.

-Darin



>
> --
> http://annevankesteren.nl/
>
>

Re: XMLHttpRequest.responseBlob

[Darin Fisher]

On Wed, Apr 28, 2010 at 11:57 AM, Michael Nordman wrote:

>
>
> On Wed, Apr 28, 2010 at 11:21 AM, Jonas Sicking  wrote:
>
>> Ugh, sent this originally to just Darin. Resending to the list.
>>
>> On Wed, Apr 28, 2010 at 10:11 AM, Darin Fisher 
>> wrote:
>> > On Tue, Apr 27, 2010 at 2:04 PM, Jonas Sicking 
>> wrote:
>> >>
>> >> On Tue, Apr 27, 2010 at 1:59 PM, Darin Fisher 
>> wrote:
>> >> > On Tue, Apr 27, 2010 at 1:33 PM, Jonas Sicking 
>> wrote:
>> >> >>
>> >> >> On Tue, Apr 27, 2010 at 1:26 PM, Darin Fisher 
>> >> >> wrote:
>> >> >> >> It would be nice to be able to allow streaming such that every
>> time
>> >> >> >> a
>> >> >> >> progress event is fired only the newly downloaded data is
>> available.
>> >> >> >> The UA is then free to throw away that data once the event is
>> done
>> >> >> >> firing. This would be useful in the cases when the page is able
>> to
>> >> >> >> do
>> >> >> >> incremental parsing of the resulting document.
>> >> >> >>
>> >> >> >> If we add a 'load mode' flag on XMLHttpRequest, which can't be
>> >> >> >> modified after send() is called, then streaming to a Blob could
>> >> >> >> simply
>> >> >> >> be another enum value for such a flag.
>> >> >> >>
>> >> >> >> There is still the problem of how the actual blob works. I.e.
>> does
>> >> >> >> .responseBlob return a new blob every time more data is returned?
>> Or
>> >> >> >> should the same Blob be constantly modifying? If modifying, what
>> >> >> >> happens to any in-progress reads when the file is modified? Or do
>> >> >> >> you
>> >> >> >> just make the Blob available once the whole resource has been
>> >> >> >> downloaded?
>> >> >> >>
>> >> >> >
>> >> >> >
>> >> >> > This is why I suggested using FileWriter.  FileWriter already has
>> to
>> >> >> > deal with
>> >> >> > most of the problems you mentioned above,
>> >> >>
>> >> >> Actually, as far as I can tell FileWriter is write-only so it
>> doesn't
>> >> >> deal with any of the problems above.
>> >> >
>> >> > When you use createWriter, you are creating a FileWriter to an
>> existing
>> >> > File.
>> >> > The user could attempt to create a FileReader to the very same File
>> >> > while
>> >> > a FileWriter is open to it.
>> >> > It is true that for  there is no way to get at the
>> >> > underlying
>> >> > File object.  That is perhaps a good thing for the use case of
>> >> > downloading
>> >> > to
>> >> > a location specified by the user.
>> >>
>> >> Ah. But as far as I can tell (and remember), it's still fairly
>> >> undefined what happens when the OS file under a File/Blob object is
>> >> mutated.
>> >>
>> >> / Jonas
>> >
>> > Agreed.  I don't see it as a big problem.  Do you?  The application
>> > developer is
>> > in control.  They get to specify the output file (via FileWriter) that
>> XHR
>> > sends its
>> > output to, and they get to know when XHR is done writing.  So, the
>> > application
>> > developer can avoid reading from the file until XHR is done writing.
>>
>> Well, it seems like a bigger deal here since the file is being
>> constantly modified as we're downloading data into it, no? So for
>> example if you grab a File object after the first progress event, what
>> does that File object contain after the second? Does it contain the
>> whole file, including the newly downloaded data? Or does it contain
>> only the data after the first progress event? Or is the File object
>> now invalid and can't be used?
>>
>
> What gears did about that was to provide a 'snapshot' of the
> downloaded data each time responseBlob was called, with
> the 'snapshot' being consistent with the progress events
> having been seen by the caller. The 'snapshot' would rema

Re: XMLHttpRequest.responseBlob

[Darin Fisher]

On Wed, Apr 28, 2010 at 2:30 PM, Eric Uhrhane  wrote:

> On Wed, Apr 28, 2010 at 12:45 PM, Darin Fisher  wrote:
> > On Wed, Apr 28, 2010 at 11:57 AM, Michael Nordman 
> > wrote:
> >>
> >>
> >> On Wed, Apr 28, 2010 at 11:21 AM, Jonas Sicking 
> wrote:
> >>>
> >>> Ugh, sent this originally to just Darin. Resending to the list.
> >>>
> >>> On Wed, Apr 28, 2010 at 10:11 AM, Darin Fisher 
> >>> wrote:
> >>> > On Tue, Apr 27, 2010 at 2:04 PM, Jonas Sicking 
> >>> > wrote:
> >>> >>
> >>> >> On Tue, Apr 27, 2010 at 1:59 PM, Darin Fisher 
> >>> >> wrote:
> >>> >> > On Tue, Apr 27, 2010 at 1:33 PM, Jonas Sicking 
> >>> >> > wrote:
> >>> >> >>
> >>> >> >> On Tue, Apr 27, 2010 at 1:26 PM, Darin Fisher <
> da...@chromium.org>
> >>> >> >> wrote:
> >>> >> >> >> It would be nice to be able to allow streaming such that every
> >>> >> >> >> time
> >>> >> >> >> a
> >>> >> >> >> progress event is fired only the newly downloaded data is
> >>> >> >> >> available.
> >>> >> >> >> The UA is then free to throw away that data once the event is
> >>> >> >> >> done
> >>> >> >> >> firing. This would be useful in the cases when the page is
> able
> >>> >> >> >> to
> >>> >> >> >> do
> >>> >> >> >> incremental parsing of the resulting document.
> >>> >> >> >>
> >>> >> >> >> If we add a 'load mode' flag on XMLHttpRequest, which can't be
> >>> >> >> >> modified after send() is called, then streaming to a Blob
> could
> >>> >> >> >> simply
> >>> >> >> >> be another enum value for such a flag.
> >>> >> >> >>
> >>> >> >> >> There is still the problem of how the actual blob works. I.e.
> >>> >> >> >> does
> >>> >> >> >> .responseBlob return a new blob every time more data is
> >>> >> >> >> returned? Or
> >>> >> >> >> should the same Blob be constantly modifying? If modifying,
> what
> >>> >> >> >> happens to any in-progress reads when the file is modified? Or
> >>> >> >> >> do
> >>> >> >> >> you
> >>> >> >> >> just make the Blob available once the whole resource has been
> >>> >> >> >> downloaded?
> >>> >> >> >>
> >>> >> >> >
> >>> >> >> >
> >>> >> >> > This is why I suggested using FileWriter.  FileWriter already
> has
> >>> >> >> > to
> >>> >> >> > deal with
> >>> >> >> > most of the problems you mentioned above,
> >>> >> >>
> >>> >> >> Actually, as far as I can tell FileWriter is write-only so it
> >>> >> >> doesn't
> >>> >> >> deal with any of the problems above.
> >>> >> >
> >>> >> > When you use createWriter, you are creating a FileWriter to an
> >>> >> > existing
> >>> >> > File.
> >>> >> > The user could attempt to create a FileReader to the very same
> File
> >>> >> > while
> >>> >> > a FileWriter is open to it.
> >>> >> > It is true that for  there is no way to get at
> >>> >> > the
> >>> >> > underlying
> >>> >> > File object.  That is perhaps a good thing for the use case of
> >>> >> > downloading
> >>> >> > to
> >>> >> > a location specified by the user.
> >>> >>
> >>> >> Ah. But as far as I can tell (and remember), it's still fairly
> >>> >> undefined what happens when the OS file under a File/Blob object is
> >>> >> mutated.
> >>> >>
> >>> >> / Jonas
> >>> >
> >>> > Agreed.  I don&#

Re: XMLHttpRequest.responseBlob

[Darin Fisher]

On Thu, Apr 29, 2010 at 3:24 PM, Eric Uhrhane  wrote:

> On Thu, Apr 29, 2010 at 3:04 PM, Darin Fisher  wrote:
> >
> >
> > On Wed, Apr 28, 2010 at 2:30 PM, Eric Uhrhane  wrote:
> >>
> >> On Wed, Apr 28, 2010 at 12:45 PM, Darin Fisher 
> wrote:
> >> > On Wed, Apr 28, 2010 at 11:57 AM, Michael Nordman <
> micha...@google.com>
> >> > wrote:
> >> >>
> >> >>
> >> >> On Wed, Apr 28, 2010 at 11:21 AM, Jonas Sicking 
> >> >> wrote:
> >> >>>
> >> >>> Ugh, sent this originally to just Darin. Resending to the list.
> >> >>>
> >> >>> On Wed, Apr 28, 2010 at 10:11 AM, Darin Fisher 
> >> >>> wrote:
> >> >>> > On Tue, Apr 27, 2010 at 2:04 PM, Jonas Sicking 
> >> >>> > wrote:
> >> >>> >>
> >> >>> >> On Tue, Apr 27, 2010 at 1:59 PM, Darin Fisher <
> da...@chromium.org>
> >> >>> >> wrote:
> >> >>> >> > On Tue, Apr 27, 2010 at 1:33 PM, Jonas Sicking
> 
> >> >>> >> > wrote:
> >> >>> >> >>
> >> >>> >> >> On Tue, Apr 27, 2010 at 1:26 PM, Darin Fisher
> >> >>> >> >> 
> >> >>> >> >> wrote:
> >> >>> >> >> >> It would be nice to be able to allow streaming such that
> >> >>> >> >> >> every
> >> >>> >> >> >> time
> >> >>> >> >> >> a
> >> >>> >> >> >> progress event is fired only the newly downloaded data is
> >> >>> >> >> >> available.
> >> >>> >> >> >> The UA is then free to throw away that data once the event
> is
> >> >>> >> >> >> done
> >> >>> >> >> >> firing. This would be useful in the cases when the page is
> >> >>> >> >> >> able
> >> >>> >> >> >> to
> >> >>> >> >> >> do
> >> >>> >> >> >> incremental parsing of the resulting document.
> >> >>> >> >> >>
> >> >>> >> >> >> If we add a 'load mode' flag on XMLHttpRequest, which can't
> >> >>> >> >> >> be
> >> >>> >> >> >> modified after send() is called, then streaming to a Blob
> >> >>> >> >> >> could
> >> >>> >> >> >> simply
> >> >>> >> >> >> be another enum value for such a flag.
> >> >>> >> >> >>
> >> >>> >> >> >> There is still the problem of how the actual blob works.
> I.e.
> >> >>> >> >> >> does
> >> >>> >> >> >> .responseBlob return a new blob every time more data is
> >> >>> >> >> >> returned? Or
> >> >>> >> >> >> should the same Blob be constantly modifying? If modifying,
> >> >>> >> >> >> what
> >> >>> >> >> >> happens to any in-progress reads when the file is modified?
> >> >>> >> >> >> Or
> >> >>> >> >> >> do
> >> >>> >> >> >> you
> >> >>> >> >> >> just make the Blob available once the whole resource has
> been
> >> >>> >> >> >> downloaded?
> >> >>> >> >> >>
> >> >>> >> >> >
> >> >>> >> >> >
> >> >>> >> >> > This is why I suggested using FileWriter.  FileWriter
> already
> >> >>> >> >> > has
> >> >>> >> >> > to
> >> >>> >> >> > deal with
> >> >>> >> >> > most of the problems you mentioned above,
> >> >>> >> >>
> >> >>> >> >> Actually, as far as I can tell FileWriter is write-only so it
> >> >>> >> >> doesn't
> >> >>> >> >> deal with any of the problems above.
> >> &g

Re: Updates to File API

[Darin Fisher]

Glad to hear that you didn't intend sync access :-)

Can you define the contentType parameter to slice better?  Is that intended
to correspond to the value of a HTTP Content-Type response header?  For
example, can the contentType value include a charset attribute?  It might be
useful to indicate that a slice of a file should be treated as text/html
with a specific encoding.

I'm also a fan of providing a way to specify optional "Content-Disposition"
parameters in the slice call.  It seems to me that Content-Disposition like
Content-Type impacts the way that Blob.url might be interpreted.  It is
useful to enable Blob.url to be able to replicate what you can do with
http:// URLs.  I think this would make it easier for apps to use
http://URLs while online and Blob.url while offline without changing
the rest of
their code.  I'm specifically thinking of use cases like the "download"
links for attachments in webmail apps.

Regards,
-Darin


On Thu, May 13, 2010 at 8:21 AM, Arun Ranganathan  wrote:

> On 5/13/10 7:37 AM, David Levin wrote:
>
>> On Thu, May 13, 2010 at 5:27 AM, Arun Ranganathan
>>  wrote:
>>
>>
>>
>>> Greetings WebApps WG,
>>>
>>> I have updated the editor's draft of the File API to reflect changes that
>>> have been in discussion.
>>>
>>> http://dev.w3.org/2006/webapi/FileAPI
>>>
>>> Notably:
>>>
>>> 1. Blobs now allow further binary data operations by exposing an
>>> ArrayBuffer property that represents the Blob.
>>>
>>>
>>
>> Does this imply *sync* access to the blob data? "new
>> DataArray(blob.blobBuffer).getInt8(0);"
>>
>>
>
> A more sensible way is an additional asynchronous read method on
> FileReader, which is what I should have done in the first place.  Here,
> partial data is going to be an interesting question.  While partial strings
> makes sense (for readAsBinaryString and readAsText), partial ArrayBuffers
> gets us into a different area altogether.  Any thoughts on partial reads
> here?
>
> For now, I've caveated my (pretty major) mistake with an editor's note.
>  I'll update later today with a better way to expose this, but I'm thinking
> something like readAsArrayBuffer on FileReader (with an open question on
> partial reads).
>
>  Also, does it imply the ability to modify the blob contents? (If so, what
>> does this mean when there is a file backing it?)
>> "new DataArray(blob.blobBuffer).setInt8(0, 0);"
>>
>>
>
> I'll let Eric speak to what BlobBuilder might want to do, but I'll strongly
> disallow it in my draft :)
>
> -- A*
>
>> Thanks, dave
>>
>>
>>
>
>
>

Re: Updates to File API

[Darin Fisher]

On Tue, May 18, 2010 at 2:56 PM, Arun Ranganathan  wrote:

> On 5/18/10 2:35 PM, Eric Uhrhane wrote:
>
>> On Mon, May 17, 2010 at 3:37 PM, Dmitry Titov
>>  wrote:
>>
>>
>>> I have couple of questions, mostly clarifications I think:
>>> 1. FileReader takes Blob but there are multiple hints that the blob
>>> should
>>> be actually a 'file'. As we see Blob concept grows in popularity with
>>> such
>>> specs as FileWriter that defines BlobBuilder. Other proposals include
>>> Image
>>> Resizing that returns a Blob with a compressed image data. Can all types
>>> of
>>> Blobs be 'read' using FileReader? If not, then it would be logical to
>>> only
>>> accept File parameter. If any type of Blob can be read (as I think the
>>> spirit of the spec assumes) then would it be less confusing to cange the
>>> name to BlobReader?
>>>
>>>
>> I'd support that.  I think we always want to allow reading of any type
>> of Blob--it's the interchange container--so calling it BlobReader
>> makes sense.  Arun, how do you feel about that?
>>
>>
>
> The FileReader object accepts File objects for DataURL-reads, and Blob
> objects for binary string, text, and binary reads.  I agree that having a
> name like FileReader is generally a bit confusing, given that we do allow
> Blobs to be read, including Blobs which aren't directly coined from files.
>  Blob itself isn't a great name, though it's a stand-in for Binary Large
> Object.
>
> Aside from the slight bikeshed-ish nature of this discussion, there are
> implementations in circulation that already use the name FileReader (e.g.
> Firefox 3.6.3).  This doesn't mean I'm against changing it, but I do wish
> the name change suggestion came earlier.  Also, I'm keen that the main
> object name addresses the initial use case -- reading file objects.  Perhaps
> in the future Blobs that are not files will be the norm; maybe then, Blob
> APIs will evolve, including implementations with ArrayBuffer and potential
> streaming use cases getting addressed better.
>
> Perhaps it is late to have a name change, and we've added to
> less-than-adequate naming on the Web (example: XMLHttpRequest).
>
>
It doesn't seem too late to change the name.  FF could support both
FileReader and BlobReader.  One could just be an alias for the other.  It
seems like we have situations like this frequently when it comes to new web
platform APIs.  A name only becomes immutable once there is a lot of content
using it since user agents would be compelled to support the existing name
for compat with existing content ;-)

-Darin



>
>  Would FileWriter ever be used to write anything other than a File?  I
>> think not, so it should probably stay as it is, despite the lack of
>> symmetry.
>>
>>
>>
>>> 2. The FileReader.result is a string.
>>>
>>>
>>
> Actually, in my next draft, I will have FileReader.result be of type 'any'
> (WebILD's 'any') since it could also be an ArrayBuffer (using the
> readAsBinary method, which will function like the other asynchrous read
> methods, but read into ArrayBuffers across the ProgressEvent spectrum.
>
> -- A*
>

Re: Lifetime of Blob URL

[Darin Fisher]

On Mon, Jul 12, 2010 at 9:59 AM, David Levin  wrote:

>
>
> On Mon, Jul 12, 2010 at 9:54 AM, Adrian Bateman wrote:
>
>> On Monday, July 12, 2010 9:32 AM, David Levin wrote:
>> > On Mon, Jul 12, 2010 at 8:39 AM, Adrian Bateman > >
>> > wrote:
>> > The behaviour would have to be explicitly specified and not left to
>> depend on
>> > indeterminate browser implementations.
>> >
>> > Yes. Unfortunately, another way of saying that "the url lives as long as
>> the
>> > Blob lives" is "the url lives until the Blob is garbage collected". This
>> > exposes a very indeterminate behavior.
>>
>> Exactly. So what I'm saying is the spec needs to say more than just that.
>> It needs to make further guarantees.
>>
>> >
>> http://lists.w3.org/Archives/Public/public-device-apis/2010Jul/.html
>> > See point #5 basically once a load has started for a url, that
>> > load should succeed and revoke may be called.
>>
>> I read point #5 to be only about surviving the start of a navigation. As a
>> web developer, how can I tell when a load has started for an ? Isn't
>> this similarly indeterminate.
>>
>
> As soon as img.src is set.
>
>
> "the spec could mention that the resource pointed by blob URL should be
> loaded successfully as long as the blob URL is valid at the time when the
> resource is starting to load."
>
>
> Should apply to xhr (after send is called), img, and navigation.
>
>
>
Right, it seems reasonable to say that ownership of the resource referenced
by a Blob can be shared by a XHR, Image, or navigation once it is told to
start loading the resource.

-Darin



>
>> Regards,
>>
>> Adrian.
>>
>
>

Using FileWriter.truncate() to extend the length of a file?

[Darin Fisher]

I noticed that FileWriter.truncate() can only be used to shorten a file, and
there does not seem to be a good way to grow a file using FileWriter without
appending data to it.  By contrast, the POSIX ftruncate function can be used
to grow a file (zero padding it):

>From ftruncate(2):
  "If the file previously was larger than this size, the extra data is lost.
 If  the  file  previously
   was shorter, it is extended, and the extended part reads as null bytes
('\0')."

I think there are use cases for wanting to grow a file in advance of writing
data to it.  For example, you might implement a system that downloads chunks
of a file in parallel using multiple XMLHttpRequest objects.  The chunks may
arrive out-of-order.

A possible alternative API would be to support seeking beyond the end of the
file or writing to a starting offset that is beyond the length fo the file.
 It may also be reasonable to support all of those in addition to truncating
to an offset greater than the length of the file.

Regards,
-Darin

Re: Lifetime of Blob URL

[Darin Fisher]

On Mon, Aug 30, 2010 at 11:14 AM, Jian Li  wrote:

>
>
> On Mon, Aug 30, 2010 at 9:59 AM, Arun Ranganathan wrote:
>
>>  Jian,
>>
>>
>>
>>
>> On 8/28/10 8:59 PM, Jian Li wrote:
>>
>> Adding explicit methods to window and WorkerGlobalScope seems to be a
>> better solution that solves potential problems we currently have with
>> blob.url. Given that, we're going to experiment the proposed new APIs in the
>> WebKit implementation, That is, we will add the following two methods to
>> window and WorkerGlobalScope in the WebKit implementation:
>>  URLString createBlobURL(in Blob blob);
>> void revokeBlobURL(in URLString url);
>>
>>
>> I think we're agreed here, although can anyone tell me whether there's a
>> precedent on the Window object to have a method that's so specific to a
>> top-level object (such as Blob)?  It definitely seems awkward, but I'm
>> willing to add it to the specification given that there isn't really a
>> better solution than adding these methods.  Also, the names seem fine to
>> me.  Can you clarify what your implementation will do if createBlobURL is
>> called twice (or more) for the same Blob object?
>>
>
> Yes, we know that everyone is working hard to try to get the best API out.
> It is just that we do not see any better solution than that.
>
> Our implementation assumes that calling window.createBlobURL() each time
> will return a new blob URL.
>
>>
>>  In addition, BlobError and BlobException sound better because these
>> names are consistent with current "Blob" naming scheme in File API. So we're
>> also going to adopt these new names in the WebKit implementation when we
>> rename FileReader to BlobReader per the spec.
>>
>>
>> *sigh.  Naming continues to be hard.  Not everyone's thrilled with the
>> proliferation of Blob in the API [1] including other major implementors (my
>> co-editor included ;-))  but I changed it mainly due to Darin/Jian/other
>> objections.  I suppose you folks are pretty adamant on the Blob* name?
>>
>
> I do not have any strong preference towards the naming. I just feel that it
> would be good if we could have consistent naming across File API in order to
> avoid the confusion to the developers. This is the reason I ask.
>

We could also rename to IOError/IOException with the idea that these might
be generic enough to be useful in other I/O related interfaces.

I just think it is awkward to raise a FileError/FileException in cases where
a file may not be involved; hence, my preference toward renaming
FileError/FileException to something else... BlobError/BlobException being
the obvious alternative.

-Darin



>
>> -- A*
>>
>> [1]
>> http://lists.w3.org/Archives/Public/public-webapps/2010JulSep/0153.html
>>
>
>

Re: Lifetime of Blob URL

[Darin Fisher]

On Mon, Aug 30, 2010 at 1:08 PM, Jonas Sicking  wrote:

> On Mon, Aug 30, 2010 at 9:59 AM, Arun Ranganathan 
> wrote:
> >> In addition, BlobError and BlobException sound better because these
> names
> >> are consistent with current "Blob" naming scheme in File API. So we're
> also
> >> going to adopt these new names in the WebKit implementation when we
> rename
> >> FileReader to BlobReader per the spec.
> >
> > *sigh.  Naming continues to be hard.  Not everyone's thrilled with the
> > proliferation of Blob in the API [1] including other major implementors
> (my
> > co-editor included ;-))  but I changed it mainly due to Darin/Jian/other
> > objections.  I suppose you folks are pretty adamant on the Blob* name?
>
> I feel pretty strongly that we should name this back to FileReader,
> for several reasons:
>
> 1. Most people that I talk to dislike the name Blob, much less having
> it spread to things like BlobReader.
> 2. My understanding is that the writer counterpart is going to be
> called FileWriter (is this correct?)
>

Yes, that is what we are currently implementing in WebKit.



> 3. While it's certainly possible to read Blobs with this, it seems to
> me like the far most common case will be to read a "real" file, or
> part of a file (i.e. the result from myfile.slice()).

4. There is one shipping implementation of FileReader
>
>
It just seems odd to use an interface named FileReader to read blobs, which
may not correspond to files.  Consider BlobBuilder, which can be used to
construct a Blob from a string.

Another idea (possibly a crazy one) would be to eliminate Blob, and just use
File for everything.  We could rename BlobBuilder to FileBuilder and have it
return a File instead of a Blob.  Same goes for Blob.slice().  Of course,
the File would not necessarily correspond to a real physical file on disk
for performance reasons.

-Darin

Re: Lifetime of Blob URL

[Darin Fisher]

On Mon, Aug 30, 2010 at 10:52 PM, Anne van Kesteren wrote:

> On Tue, 31 Aug 2010 01:22:45 +0200, Darin Fisher 
> wrote:
>
>> Another idea (possibly a crazy one) would be to eliminate Blob, and just
>> use File for everything.  We could rename BlobBuilder to FileBuilder and
>> have it return a File instead of a Blob.  Same goes for Blob.slice().  Of
>> course,
>> the File would not necessarily correspond to a real physical file on disk
>> for performance reasons.
>>
>
> Not having Blob at all works for me!
>
>
>

I gave this some more thought.  Here's some issues I came up with:

1) BlobBuilder -> FileBuilder

This renaming seems to suggest the creation of a file, which is not the
intent at all.


2) XHR.{asBlob,responseBlob} -> XHR.{asFile,responseFile}

This renaming seems to suggest the creation of a file, which is not
necessary for small responses.


3) Combine Blob and File into a single File interface

This merging has the unfortunate side-effect of introducing a "name"
property for the result of a
File.slice() operation.  It also means that the result of FileBuilder and
XHR.responseFile would
have a name.


Considering the above, it seems like there is a place for Blob (or something
like it).

I can see Jonas' points about BlobReader vs. FileReader, and so I'm happy to
backpedal and go with
FileReader, FileException, and FileError, keeping Blob for cases where we
don't promise a file.

-Darin

Re: [XHR] Redirects

[Darin Fisher]

On Fri, Aug 27, 2010 at 5:50 AM, Anne van Kesteren  wrote:

> On Fri, 13 Aug 2010 00:13:16 +0200, João Eiras 
> wrote:
>
>> Between the boolean and an integer, the integer is more useful, although
>> seeing long redirection chains is somewhat rare and overkill.
>>
>
> I went for a boolean followRedirects attribute as that gives sufficient
> low-level control that people can implement whatever behavior they want on
> top of that. (Including every use case mentioned in this thread.) This is an
> XMLHttpRequest Level 2 feature now:
>
> http://dev.w3.org/2006/webapi/XMLHttpRequest-2/
>
>
>
So the idea is that you would have to manually create the redirect request
using a new XHR if you wanted to manually follow the redirect?

One risk with that is that it is easy to construct the redirect request
improperly.  There's the classic example of what happens when you issue a
POST request and it results in a redirect.  303 and 307 had to be invented
because some browsers implemented 302 incorrectly.

Maybe there should be a way to create a new XHR from the existing XHR that
is already configured to follow the redirect?  I'm not sure how to make this
less awkward...

-Darin



>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>
>

Re: [XHR] Redirects

[Darin Fisher]

On Tue, Aug 31, 2010 at 1:16 PM, Anne van Kesteren  wrote:

> On Tue, 31 Aug 2010 19:04:08 +0200, Darin Fisher 
> wrote:
>
>> So the idea is that you would have to manually create the redirect request
>> using a new XHR if you wanted to manually follow the redirect?
>>
>
> Yeah, or you could reset the existing object using open().
>
>
>
>  One risk with that is that it is easy to construct the redirect request
>> improperly.  There's the classic example of what happens when you issue a
>> POST request and it results in a redirect.  303 and 307 had to be invented
>> because some browsers implemented 302 incorrectly.
>>
>
> Yeah, though I think that the people that implement redirects manually have
> put some thought in the matter. Maybe that is naive, though on the other
> hand if people do it wrong it does not matter much either.
>
>
>
>  Maybe there should be a way to create a new XHR from the existing XHR that
>> is already configured to follow the redirect?  I'm not sure how to make this
>> less awkward...
>>
>
> If this is actually common and people start using this a lot and ask this
> question here that would be the point where we should start thinking about
> making this less awkward. For now libraries and developers with "lots of
> time" have all the hooks they need.
>
>
>
>

Hmm, that's a pretty good argument.

Another issue that concerns me is redirect loops.  The proposed API gives us
know way to know that the application is following a redirect chain, which
means that we have little way to know if the application is following an
endless redirect chain.  The classical example is a server that redirects
back to the same URL with a Set-Cookie header, expecting the next request to
have a corresponding Cookie header.  If the Cookie header is not added, then
the redirect is issued again.

Most browsers detect this after N iterations, and then give up.  Of course,
this situation only occurs if the user has disabled cookies in the User
Agent.  I worry that a case like this will almost never be tested by
application developers, and hence, users who like to disable some cookies
will be unable to use the internet.  (I realize that the internet is mostly
broken already if you disable cookies, but there are some variations of
cookie blocking that sort of work.)

A User Agent could try to implement a heuristic to catch certain flavors of
redirect chains, by observing the XHR requests made by an application.  But,
it seems like it would be fairly difficult to avoid false positives.

If instead, we had an API for auditing redirects (perhaps an "onredirect"
event), then we could let developers handle that event and call
preventDefault if they want to stop redirect processing.  Otherwise, the
redirect would be processed normally.  The result being that application
authors would be guided away from following redirects manually, and hence
they would avoid the problems I've described above.

-Darin

Re: [XHR] Redirects

[Darin Fisher]

On Tue, Aug 31, 2010 at 11:26 PM, Anne van Kesteren wrote:

> On Tue, 31 Aug 2010 22:51:19 +0200, Darin Fisher 
> wrote:
>
>> If instead, we had an API for auditing redirects (perhaps an "onredirect"
>> event), then we could let developers handle that event and call
>> preventDefault if they want to stop redirect processing.  Otherwise, the
>> redirect would be processed normally.  The result being that application
>> authors would be guided away from following redirects manually, and hence
>> they would avoid the problems I've described above.
>>
>
> This does not work for synchronous requests in a worker.


Hmm, good point.  An alternative design might be a flag that causes .send()
to complete once a redirect response is received, but then also provide a
method on the XHR object to tell it to now follow the redirect.  This model
would work for both synchronous as well as asynchronous.



> If we are okay with that it seems an acceptable alternative. However,
> redirects are processed before the object reaches HEADERS_RECEIVED (as
> otherwise it would reach that several times) so HTTP headers et cetera are
> not available yet. Given that, and given that you want that information, you
> would always invoke preventDefault(), defeating the purpose of having an
> event.
>
>
It does seem like you'd want to see the headers corresponding to the
redirect.

I thought of another reason to want the original XHR object to be
responsible for following the redirect:  the value of a Location header may
be a relative URL.  It would be nice if application authors did not have to
take care of resolving that manually.  (In the case of a cross-origin
request, the relative URL should be resolved relative to the URL that was
redirected instead of against the Document.)  This seems like something that
could be easy to mess up.

-Darin

Re: [XHR] Redirects

[Darin Fisher]

On Wed, Sep 1, 2010 at 1:16 AM, Anne van Kesteren  wrote:

> On Wed, 01 Sep 2010 10:03:49 +0200, Darin Fisher 
> wrote:
>
>> On Tue, Aug 31, 2010 at 11:26 PM, Anne van Kesteren > >wrote:
>>
>>> This does not work for synchronous requests in a worker.
>>>
>>
>> Hmm, good point.  An alternative design might be a flag that causes
>> .send() to complete once a redirect response is received, but then also
>> provide a
>> method on the XHR object to tell it to now follow the redirect.  This
>> model would work for both synchronous as well as asynchronous.
>>
>
> That seems like the current design, except we currently do not have that
> additional method. I would like to keep it out until it is more clear this
> is a real problem. It would add quite a bit of complexity whereas just
> having this is fairly straightforward.
>
>
The problems I've raised here are real problems that I've observed while
building HTTP implementations for Mozilla and Chromium.  It is easy for good
coders to make these kinds of mistakes.

-Darin



>
>
>  I thought of another reason to want the original XHR object to be
>> responsible for following the redirect:  the value of a Location header
>> may be a relative URL.  It would be nice if application authors did not have
>> to take care of resolving that manually.  (In the case of a cross-origin
>> request, the relative URL should be resolved relative to the URL that was
>> redirected instead of against the Document.)  This seems like something
>> that could be easy to mess up.
>>
>
> Yeah, I thought of that. There's location.resolveURL(), but it does not
> take a base URL at the moment. We could add that. Though note that relative
> URLs are forbidden in theory.
>
>
>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>

Re: [XHR] Redirects

[Darin Fisher]

On Wed, Sep 1, 2010 at 5:19 AM, Julian Reschke wrote:

> On 01.09.2010 10:16, Anne van Kesteren wrote:
>
>> ...
>>
>>  I thought of another reason to want the original XHR object to be
>>> responsible for following the redirect: the value of a Location header
>>> may be a relative URL. It would be nice if application authors did not
>>> have to take care of resolving that manually. (In the case of a
>>> cross-origin
>>> request, the relative URL should be resolved relative to the URL that was
>>> redirected instead of against the Document.) This seems like something
>>> that could be easy to mess up.
>>>
>>
>> Yeah, I thought of that. There's location.resolveURL(), but it does not
>> take a base URL at the moment. We could add that. Though note that
>> relative URLs are forbidden in theory.
>> ...
>>
>
> They are in RFC 2616, but not in HTTPbis (<
> http://greenbytes.de/tech/webdav/draft-ietf-httpbis-p2-semantics-11.html#rfc.section.9.4
> >).
>
> Best regards, Julian
>

What does it mean for them to not be part of HTTPbis?  Relative URLs in
Location headers are not uncommon.

-Darin

Re: [XHR] Redirects

[Darin Fisher]

On Wed, Sep 1, 2010 at 1:16 AM, Anne van Kesteren  wrote:

> On Wed, 01 Sep 2010 10:03:49 +0200, Darin Fisher 
> wrote:
>
>> On Tue, Aug 31, 2010 at 11:26 PM, Anne van Kesteren > >wrote:
>>
>>> This does not work for synchronous requests in a worker.
>>>
>>
>> Hmm, good point.  An alternative design might be a flag that causes
>> .send() to complete once a redirect response is received, but then also
>> provide a
>> method on the XHR object to tell it to now follow the redirect.  This
>> model would work for both synchronous as well as asynchronous.
>>
>
> That seems like the current design, except we currently do not have that
> additional method. I would like to keep it out until it is more clear this
> is a real problem. It would add quite a bit of complexity whereas just
> having this is fairly straightforward.


One more question:  suppose the author requests not to follow redirects.
 Should the .responseText property provide the response body sent with the
redirect?  Is there a use case for exposing the response body of a redirect
page?

(I ask because in Chromium, the cache does not store response bodies for
cached redirects.  IIRC, Mozilla's cache is similar.)

-Darin


>
>
>
>  I thought of another reason to want the original XHR object to be
>> responsible for following the redirect:  the value of a Location header
>> may be a relative URL.  It would be nice if application authors did not have
>> to take care of resolving that manually.  (In the case of a cross-origin
>> request, the relative URL should be resolved relative to the URL that was
>> redirected instead of against the Document.)  This seems like something
>> that could be easy to mess up.
>>
>
> Yeah, I thought of that. There's location.resolveURL(), but it does not
> take a base URL at the moment. We could add that. Though note that relative
> URLs are forbidden in theory.
>
>
>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>

Re: Flags for getFile/getDirectory (Re: New draft of FileSystem API posted)

[Darin Fisher]

On Tue, Sep 7, 2010 at 8:41 PM, Ian Hickson  wrote:

> On Tue, 7 Sep 2010, Adam Barth wrote:
> >
> > I think the bitfield approach is better.  The current approach doesn't
> > work very well in strongly typed languages.  Although we might think
> > that these APIs will be used most-often from JavaScript, these APIs are
> > language neutral and should work in a variety of settings (e.g., as part
> > of the NPAPI).  Baking in assumptions that APIs are used by dynamically
> > typed language isn't good for the web platform in the long term.
>
> The APIs don't have to be identical in each language. For example, I would
> expect a C++ port of ValidityState:
>
>
> http://www.whatwg.org/specs/web-apps/current-work/complete.html#validitystate
>
> ...to be implemented as a bitfield with constants, rather than as an
> object with fields. Should anyone want to implement that interface in such
> a language, then would be a time to provide suitable IDL for that case.
>
> --
> Ian Hickson   U+1047E)\._.,--,'``.fL
> http://ln.hixie.ch/   U+263A/,   _.. \   _\  ;`._ ,.
> Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'
>
>

Slight tangent

Based on the ValidityState example, it seems that the members of Flags
should be camelCase then instead of UPPERCASE?

-Darin

Re: ArrayBuffer and ByteArray questions

[Darin Fisher]

On Tue, Sep 7, 2010 at 5:23 PM, Kenneth Russell  wrote:

> On Tue, Sep 7, 2010 at 4:19 PM, Nathan  wrote:
> > Jian Li wrote:
> >>
> >> Hi,
> >>
> >> Several specs, like File API and WebGL, use ArrayBuffer, while other
> spec,
> >> like XMLHttpRequest Level 2, use ByteArray. Should we change to use the
> >> same
> >> name all across our specs? Since we define ArrayBuffer in the Typed
> Arrays
> >> spec (
> >>
> >>
> https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/doc/spec/TypedArray-spec.html
> ),
> >> should we favor ArrayBuffer?
> >>
> >> In addition, can we consider adding ArrayBuffer support to BlobBuilder,
> >> FormData, and XMLHttpRequest.send()?
> >
> > which reminds me, I meant to ask if the aforementioned TypedArray spec
> > should be brought in to webapps / w3c land? seems to complement the other
> > base types used in webidl etc rather well + my gut reaction was why isn't
> > this standardized within w3c?
>
> There's no particular reason why the Typed Array spec is being
> standardized in the Khronos group, aside from the fact that these
> array-like types originated in the WebGL spec and have evolved to meet
> use cases specified by WebGL. We have been hoping that they would have
> uses outside of WebGL, and some discussions have occurred with working
> groups such as TC39 to see how they might be better specified and
> standardized. We'd be open to hosting the spec development elsewhere.
>
> Vlad mentioned to me off-list that Mozilla has implemented an
> experimental mozResponseArrayBuffer on XHR objects, and will likely do
> the same on the File API to add a readAsArrayBuffer alongside
> readAsBinaryString etc.
>
> -Ken
>
>

It sounds like ArrayBuffer is the name that is gaining traction (to circle
back to Jian's original question about naming).

-Darin

Re: [XHR] Redirects

[Darin Fisher]

On Mon, Sep 13, 2010 at 8:51 AM, James Leigh wrote:

> On Wed, 2010-09-01 at 01:03 -0700, Darin Fisher wrote:
> >
> > I thought of another reason to want the original XHR object to be
> > responsible for following the redirect:  the value of a Location
> > header may be a relative URL.  It would be nice if application authors
> > did not have to take care of resolving that manually.  (In the case of
> > a cross-origin request, the relative URL should be resolved relative
> > to the URL that was redirected instead of against the Document.)  This
> > seems like something that could be easy to mess up.
> >
> >
> > -Darin
>
>
> I want to point out that resolving the Location header is not specific
> to 3XX responses. A 201 (Created) response also includes a Location
> header that may need to be resolved. If the problem of providing a way
> to resolve redirect is addressed it might be good to also address the
> problem of resolving 201 locations as well.
>
> One suggestion might be to provide a getResponseLocation() that returns
> the resolved Location header.
>
> James
>
>

That's a good point.  Note, resolving the Location header is only one of the
issues.  Another is knowing what HTTP method to use in response to a
redirect.

-Darin

Re: [XHR] Redirects

[Darin Fisher]

On Mon, Sep 13, 2010 at 7:35 AM, Anne van Kesteren  wrote:

> On Wed, 01 Sep 2010 23:59:17 +0200, Darin Fisher 
> wrote:
>
>> On Wed, Sep 1, 2010 at 1:16 AM, Anne van Kesteren 
>> wrote:
>>
>>> That seems like the current design, except we currently do not have that
>>> additional method. I would like to keep it out until it is more clear
>>> this
>>> is a real problem. It would add quite a bit of complexity whereas just
>>> having this is fairly straightforward.
>>>
>>
>> The problems I've raised here are real problems that I've observed while
>> building HTTP implementations for Mozilla and Chromium.  It is easy for
>> good coders to make these kinds of mistakes.
>>
>
> I did not mean to downplay the problems you were raising. Do you think we
> should have a method like openPreserveState() that unlike open() does not
> reset a bunch of information? So to handle a redirect you would do
> openPreserveState(); send(); after the initial request if status is one of
> 301/302/303/307. (You might have to do some more.)
>

That's a very interesting idea.  Maybe it should be more specific, like
openRedirectedLocation?  That said, if one still has to call .send()
afterwards, then we are still leaving it up to the caller to pass the right
value to .send().  They have to know when the method is being changed from
POST to GET and take appropriate action.  That is a bit unfortunate.  Maybe
.send() and .openPreserveState() could just be combined into a single
.followRedirect() method?



>
> Do you think I should comment out the followRedirects feature for now? At
> until we address the problem of how to follow it easily?


I don't have strong opinions about that.  It is up to you.  I think it is OK
to iterate :)
-Darin



>
>
>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>

Re: ArrayBuffer and ByteArray questions

[Darin Fisher]

On Wed, Sep 15, 2010 at 6:53 PM, Arun Ranganathan  wrote:

>  On 9/7/10 10:08 PM, Darin Fisher wrote:
>
> On Tue, Sep 7, 2010 at 5:23 PM, Kenneth Russell  wrote:
>
>>  On Tue, Sep 7, 2010 at 4:19 PM, Nathan  wrote:
>> > Jian Li wrote:
>> >>
>> >> Hi,
>> >>
>> >> Several specs, like File API and WebGL, use ArrayBuffer, while other
>> spec,
>> >> like XMLHttpRequest Level 2, use ByteArray. Should we change to use the
>> >> same
>> >> name all across our specs? Since we define ArrayBuffer in the Typed
>> Arrays
>> >> spec (
>> >>
>> >>
>> https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/doc/spec/TypedArray-spec.html
>> ),
>> >> should we favor ArrayBuffer?
>> >>
>> >> In addition, can we consider adding ArrayBuffer support to BlobBuilder,
>> >> FormData, and XMLHttpRequest.send()?
>> >
>> > which reminds me, I meant to ask if the aforementioned TypedArray spec
>> > should be brought in to webapps / w3c land? seems to complement the
>> other
>> > base types used in webidl etc rather well + my gut reaction was why
>> isn't
>> > this standardized within w3c?
>>
>>  There's no particular reason why the Typed Array spec is being
>> standardized in the Khronos group, aside from the fact that these
>> array-like types originated in the WebGL spec and have evolved to meet
>> use cases specified by WebGL. We have been hoping that they would have
>> uses outside of WebGL, and some discussions have occurred with working
>> groups such as TC39 to see how they might be better specified and
>> standardized. We'd be open to hosting the spec development elsewhere.
>>
>> Vlad mentioned to me off-list that Mozilla has implemented an
>> experimental mozResponseArrayBuffer on XHR objects, and will likely do
>> the same on the File API to add a readAsArrayBuffer alongside
>> readAsBinaryString etc.
>>
>> -Ken
>>
>>
>
>  It sounds like ArrayBuffer is the name that is gaining traction (to
> circle back to Jian's original question about naming).
>
>
> In fact, readAsArrayBuffer / ArrayBuffer is used with FileReader, and will
> be the names going forward.  ArrayBuffer is gaining traction as the used
> name :)
>
>
Great, sounds good.
-Darin

Re: [XHR] Redirects

[Darin Fisher]

On Wed, Sep 15, 2010 at 3:07 AM, Julian Reschke wrote:

> On 15.09.2010 11:56, Boris Zbarsky wrote:
>
>> On 9/15/10 2:47 AM, Boris Zbarsky wrote:
>>
>>> So it's possible that the original behavior was just an oversight that
>>> then got copied. Someone with access to a browser version control system
>>> from before 1998 would need to look to make sure...
>>>
>>
>> It's also possible that no UA implementor was willing to implement the
>> MUST NOT requirements below:
>>
>> If the 301 status code is received in response to a request other
>> than GET or HEAD, the user agent MUST NOT automatically redirect the
>> request unless it can be confirmed by the user, since this might
>> change the conditions under which the request was issued.
>>
>> and
>>
>> If the 302 status code is received in response to a request other
>> than GET or HEAD, the user agent MUST NOT automatically redirect
>> the request unless it can be confirmed by the user, since this might
>> change the conditions under which the request was issued.
>>
>> (RFC 2616 sections 10.3.2 and 10.3.3). How do you expect this to work in
>> the XHR context? Is "user" for purposes of those two clauses the script
>> that triggered the XHR, or the person actually represented by the
>> user-agent (browser, say) in question?
>>
>> Then again, I guess they already ignore that MUST NOT clause for 307
>> redirects... So maybe they would just do the same thing here. Gotta love
>> specs that really can't be implemented as written in sane ways.
>>
>
> For XHR I'd argue that once the caller requests control over redirects,
> he's responsible to do it right.
>
> With respect to 2616 and the text you quoted above: the WG is aware of this
> problem, see .
>
> Best regards, Julian
>
>
>
Adding a .followRedirect method means that developers can choose to use it
or not.  If they want to manually construct the redirected request, then
they can do so using a new XHR object.  However, if the use case is only
auditing redirects, then being able to call .followRedirect on the existing
XHR is quite convenient and minimizes the chances of making some common
mistakes.

-Darin

Re: [XHR] Redirects

[Darin Fisher]

On Thu, Sep 16, 2010 at 6:31 AM, Julian Reschke wrote:

>
>>> I think that obtaining the resolved Location for the redirect is the
>>> most tricky part, and we should come up with a solution where that is
>>> also available for people who do not want the default browser behavior.
>>>
>>
>> I don't really like special casing the Location header here. It seems we
>> are ending up with two additional members for dealing with redirects. I
>> do not want another one. There are other headers where resolving a URL
>> might be needed. Adding a parameter to navigator.resolveURL() for a base
>> URL is the way to go I think.
>>
>
> That sounds good to me. In general I think it would be great if there were
> standard APIs for URI/IRI construction, parsing and resolution...
>
>
Yes, that sounds pretty good to me too.

-Darin

Re: A URL API

[Darin Fisher]

On Fri, Sep 17, 2010 at 11:05 AM, Adam Barth  wrote:

> On Fri, Sep 17, 2010 at 10:27 AM, Adam Barth  wrote:
> > On Thu, Sep 16, 2010 at 3:25 PM, Darin Fisher 
> wrote:
> >> On Thu, Sep 16, 2010 at 6:31 AM, Julian Reschke 
> >> wrote:
> >>> That sounds good to me. In general I think it would be great if there
> were
> >>> standard APIs for URI/IRI construction, parsing and resolution...
> >>
> >> Yes, that sounds pretty good to me too.
> >
> > This has annoyed me for a while too.  I'll write up a spec.
>
> Here's a sketch:
>
>
> https://docs.google.com/document/edit?id=1r_VTFKApVOaNIkocrg0z-t7lZgzisTuGTXkdzAk4gLU&hl=en
>
> Adam
>

Nice!  Is there any implicit unescaping done when reading those members?
 I'd hope for the answer to be "no" :-)

When does canonicalization happen?  Only when you call href?  Or, does it
happen when you readback any attribute?

-Darin

Re: A URL API

[Darin Fisher]

On Sun, Sep 19, 2010 at 10:41 PM, Boris Zbarsky  wrote:

> On 9/20/10 1:02 AM, Adam Barth wrote:
>
>> I've updated the document:
>>
>>
>> https://docs.google.com/document/edit?id=1r_VTFKApVOaNIkocrg0z-t7lZgzisTuGTXkdzAk4gLU&hl=en#
>>
>
> General comments based on a quick read (and ignoring various typos that I
> figure we'll fix in due course):
>
> 1)  The reference chain for actually parsing a URI terminates in HTML5
> referencing the IRI RFC, which doesn't seem to define a parsing algorithm.
>  Did I just miss it?
>
> 2)  Why lastPathComponent as opposed to, say, fileName?
>
> 3)  Why is lastPathComponent readonly?
>
> 4)  Things Gecko has on its URI objects that this seems to be missing
>and that might be useful and aren't obviously available via
>existing facilities of this interface:
>
>  * An equals() method that takes another URI object
>  * An 'asciiSpec' property (Host IDN-escaped as needed, other parts
>URL-escaped).
>  * An 'asciiHost' property
>

asciiSpec/Host is the canonical form, which is what Location currently
outputs.  gecko's URI class has the UTF-8 spec/host attributes to be more
friendly to UI code that might prefer the non-IDN version of the strings.



>  * A 'param' property (for things following ';')
>

This one seems unused as best as I can tell.


>  * A 'directory' property (which may be similar to getting the
>pathname and then dropping everything after the last '/'... not
>sure).
>  * fileBaseName and fileExtension (URI might not be the right place
>for these, though).
>

^^^ extracting those is pretty simple given the currently proposed
interface.



>  * A getCommonBaseSpec() method that will take two URIs and return a
>URI string they could both be relative to, if any.
>  * A getRelativeSpec() method that takes two URIs and returns a
>string that represents one as a relative URI relative to the
>other (if this is possible).
>

^^^ these are convenient

-Darin


>
> -Boris
>
>
>

Re: A URL API

[Darin Fisher]

On Mon, Sep 20, 2010 at 11:02 AM, Garrett Smith wrote:

> On 9/20/10, Julian Reschke  wrote:
> > On 20.09.2010 18:56, Garrett Smith wrote:
> [...]
> >> Requests that don't have lot of parameters are often simple one-liners:
> >>
> >> url = "/getShipping/?zip=" + zip + "&pid=" + pid;
> >
> > That's exactly the kind of code that will fail once "pid" and "zip"
> > contain things you don't expecz.
> >
> >> What XHRs have complicated URL with a lot of query parameters?
> >
> > What XHRs?
> >
> IOW, what are the cases where an XHR instance wants to use a lot o query
> params?
>
>
Probably when speaking to a HTTP server designed to take input from an HTML
form.

-Darin

Re: createBlobURL

[Darin Fisher]

On Tue, Oct 19, 2010 at 4:57 PM, Jonas Sicking  wrote:

> On Tue, Oct 19, 2010 at 1:24 AM, Anne van Kesteren 
> wrote:
> > On Mon, 18 Oct 2010 20:15:53 +0200, Jonas Sicking 
> wrote:
> >>
> >> Without revoking the UA has to keep around the URL-string -> resource
> >> mapping for the lifetime of the page. Which in the world of web apps
> >> can be a very long time. Even worse, in the case of dynamically
> >> created blobs (blobbuilder, canvas.toFile/toBlob/getAsFile whatever
> >> we'll call it), the resource has to stay around at least on the users
> >> file system for the lifetime of the page.
> >
> > So we are moving the responsibility to do things right to authors? Oh
> joy.
> > Though I suppose it might just work for the most complex of applications,
> > where they measure things such as memory usage, etc.
>
> Suggestions welcome. The base problem here is that we are doing
> resource management using a string-value rather than using a
> object-reference. It is provably impossible for the implementation to
> know if a given url is going to get used in the future (since it
> requires solving the halting problem).
>
> The only real solution here is to abandon the use of URLs-strings
> ("blob:...") and instead use some type of object which represents a
> reference to the blob/stream/whatever. Then make img.src, iframe.src,
> CSSStyleDeclaration.backgroundImage etc accept this new type in
> addition to a string.
>

I had a similar thought the other day.  However, why not just support
assigning a
Blob/Stream directly to img.src, iframe.src, etc.?  Why does there need to
be any
other representation of the data source?

(I'm not suggesting we abandon blob: URLs completely.)

-Darin



>
> I think the main mitigating factor here is that as far as memory usage
> goes, the only thing "leaked" is an entry in a hash-table, so in the
> order of 50 bytes for each generated url.
>
> / Jonas
>

Re: XHR responseArrayBuffer attribute: suggestion to replace "asBlob" with "responseType"

[Darin Fisher]

On Tue, Oct 26, 2010 at 5:43 PM, Boris Zbarsky  wrote:

> On 10/26/10 5:27 AM, Anne van Kesteren wrote:
>
>> Boris, do you see value in the asBlob / responseBlob proposal currently
>> in the draft?
>>
>
> The idea there being that you don't have to have the data in memory at all
> unless the page explicitly asks for it, and then giving the page the option
> of only mapping some of the data into memory?
>

...where "some" may mean none.  you could just get an URL to the Blob, and
shove that into some HTML element for display, or you could use the
FileWriter or FileSaver interfaces to copy the Blob someplace.



>
> I think that's a good idea, yes.
>
> What I'm not quite sure about is why it needs to be a state flag which
> makes it an open-time all-or-nothing decision, other than implementor
> convenience


I think it's about performance as well as implementor convenience.  If the
implementor knows up front that the data should be stored as a Blob, then
the implementor can write the data to disk without having to waste memory.

-Darin

Re: XHR responseArrayBuffer attribute: suggestion to replace "asBlob" with "responseType"

[Darin Fisher]

On Mon, Oct 25, 2010 at 3:33 PM, Boris Zbarsky  wrote:

> On 10/25/10 6:21 PM, Chris Rogers wrote:
>
>>  People are concerned that it would require keeping two copies of the
>> data around (raw bytes, and unicode text version) since it's unknown
>> up-front whether "responseText", or "responseArrayBuffer" will be
>> accessed.
>>
>
> Note that Gecko does exactly that, and we've seen no problems with it...
>  It's very rare to have really large XHR bodies, for what it's worth.
>
>
>  This approach does seem a little strange because of the mutually
>> exclusive nature of the access.  However, it seems that it would be hard
>> to come up for a reasonable use case where both the raw bytes *and* the
>> text would be needed for the same XHR.
>>
>
> You could have a situation where a library dispatches the XHR for you and
> always looks at the text for some reason (libraries tend to do that sort of
> thing) while you actually want the bytes.
>
> In general, this approach seems really fragile and hostile to web
> developers.
>
>
>  2. Darin Fisher has suggested this approach: Add an "asArrayBuffer"
>> attribute (similar to "asBlob") which *must* be set before sending the
>> request if there will be a subsequent access of the
>> "responseArrayBuffer" attribute.
>>
>
> This make it impossible to decide whether to look at the text or the bytes
> based on the content-type of the response (unless you allow setting the
> attribute in some early-enough onStateChange callback _and_ libraries expose
> XHRs in that early a state to consumers); having that ability seems like a
> desirable feature.


I think XMLHttpRequest is trying to be too much of a kitchen-sink.  It seems
pretty unfortunate that our networking API has XML parsing features, for
example.  That should be a separate component, but alas, we cannot change
history.

Ideally, there'd be separate components that operate on an ArrayBuffer and
produce a decoded string / XML document.  Then, for the use case you are
talking about, people could just ask for the response as an ArrayBuffer,
inspect the response headers, and then optionally invoke a text decoder
interface or a XML parser / DOM builder interface.

As for the performance discussion, we learned the hard way that it was
valuable to only keep one copy of the XHR's data.  There are some sites out
there that load large documents.  Sad, but true.  Maybe James Robinson or
someone else can dig up some examples.  I think we should try to design for
a future where we don't have to compromise performance for capabilities.

If we keep the ArrayBuffer up front and only decode on demand, then we will
be doing more work in the common case (in which someone only wants the
responseText).  That seems bad.

-Darin


>
>  3. Get rid of the "asBlob" attribute and add a new "responseType"
>> attribute which could be:
>> "Text" <--- this is the default
>> "XML"
>> "Bytes"
>> ... other types yet to be defined
>>
>
> I'm not sure I follow this proposal.
>
>
>  I can accept any of the proposed solutions and would like to hear what
>> others think about these or other approaches.
>>
>
> How about:
>
> 4)  Make things easy to use for authors; that means supporting responseText
> and responseArrayBuffer, with access to both on the same XHR object without
> weird restrictions and without clairvoyance required on the part of the
> author.  If UAs feel that they don't want to keep both copies of the data in
> memory on a permanent basis, they can optimize this by dropping the
> responseText (on a timer or via a low-memory detection mechanism, or in
> various other ways) and regenerating it from the raw bytes as needed.
>
> ?
>
> The other suggestions really feel like a case of overoptimizing for
> simplicity and ease of creation of UA code at the expense of authors.
> Sometimes that's warranted, but in this case the UA code needed to produce
> sane behavior is really just not that complicated or hard
>
> -Boris
>

Re: XHR responseArrayBuffer attribute: suggestion to replace "asBlob" with "responseType"

[Darin Fisher]

On Wed, Oct 27, 2010 at 12:04 AM, Boris Zbarsky  wrote:

> On 10/27/10 2:28 AM, Darin Fisher wrote:
>
>> I think it's about performance as well as implementor convenience.  If
>> the implementor knows up front that the data should be stored as a Blob,
>> then the implementor can write the data to disk without having to waste
>> memory.
>>
>
> I have no problem with hinting to the implementor that the result is
> expected to be used as a Blob so they can optimistically store it to disk.
>  My problem is with then making it impossible to get it as a string if
> needed later (which would just involve reading the data from disk into
> memory).
>
> -Boris
>


Yes, but doing so is inefficient since the API requires the data to be in
memory synchronously.  We don't want to block the UI thread on disk IO.

-Darin

Re: XHR responseArrayBuffer attribute: suggestion to replace "asBlob" with "responseType"

[Darin Fisher]

On Wed, Oct 27, 2010 at 12:12 AM, Boris Zbarsky  wrote:

> On 10/27/10 2:39 AM, Darin Fisher wrote:
>
>> I think XMLHttpRequest is trying to be too much of a kitchen-sink.
>>
>
> OK, something we agree on.  ;)
>
>
>  Ideally, there'd be separate components that operate on an ArrayBuffer
>> and produce a decoded string / XML document.  Then, for the use case you
>> are talking about, people could just ask for the response as an
>> ArrayBuffer, inspect the response headers, and then optionally invoke a
>> text decoder interface or a XML parser / DOM builder interface.
>>
>
> My worry is about fragility we introduce in situations where the person
> dispatching the XHR (library author or page author) and the person(s)
> reading the results (library/page authors) are not the same set.  As long as
> the same object provides both views and the views are mutually exclusive,
> it's really easy for the dispatcher to change what they ask for and subtly
> break other consumers he doesn't even know about.
>
> I'd be much happier with either separate objects that provide different
> views and can't be confused for each other or all the possible views being
> available on demand.
>
>
>  As for the performance discussion, we learned the hard way that it was
>> valuable to only keep one copy of the XHR's data.  There are some sites
>> out there that load large documents.  Sad, but true.  Maybe James
>> Robinson or someone else can dig up some examples.
>>
>
> Those would be helpful, yes.
>
>
>  I think we should try to design for a future where we don't have to
>> compromise performance
>> for capabilities.
>>
>
> If we can do that without compromising capabilities for performance too
> much, yes.  Need to strike a balance.
>
>
>  If we keep the ArrayBuffer up front and only decode on demand, then we
>> will be doing more work in the common case (in which someone only wants
>> the responseText).  That seems bad.
>>
>
> Well, sure.  The question is how much more work it is, and how bad the
> other options are.
>
> -Boris
>

So, it sounds like we just need to present you with some concrete examples
of XHRs being used to fetch large responses as Text or XML, and then you
will be convinced?

I wrote a web app that processes a subversion log using responseXML.  Does
that count?  (The subversion log is pretty big.  45MB for all of WebKit.)

-Darin

Re: requestAnimationFrame

[Darin Fisher]

On Thu, Nov 18, 2010 at 11:21 AM, Gregg Tavares (wrk) wrote:

>
>
> On Thu, Nov 18, 2010 at 12:45 AM, Robert O'Callahan 
> wrote:
>
>> On Thu, Nov 18, 2010 at 4:03 PM, Gregg Tavares (wrk) wrote:
>>
>>> On (a)
>>> Take this page (http://boingboing.net) At the time I checked it today
>>> (6:55pm PST) it had 10 instances of flash running. O page load 3 were
>>> animating continuallty, 7 were idle. The 7 idle ones were all video with
>>> custom UIs. Of those 7, 6 of them will animate constantly once the video is
>>> finished advertising other videos to play. That means at any one time, if
>>> this was canvas based, 9 areas will be re-rendered by JavaScript on each
>>> "animationFrame".  It seems like giving the browser more info by tying these
>>> animation frames to specific elements would let the browser not call
>>> animationFrame callbacks unless the element it is tied to is on the screen
>>> and that would be a pretty good thing. If all 10 of those areas were
>>> re-rendering their ads all the time I suspect that page would get pretty
>>> slow, especially on netbooks. As it is, only 1-3 areas are ever visible at
>>> once.
>>>
>>
>> Yeah, that makes sense.
>>
>> Then I suggest adding requestAnimationFrame to elements, so you can call
>> "element.requestAnimationFrame(callback)".
>>
>> I think there needs to be a guarantee that the callback is eventually
>> called even if the element never becomes visible. People sometimes want to
>> take action when an animation finishes. window.mozRequestAnimationFrame
>> fires once per second even for hidden tabs for this reason.
>
>
> I see your point but I'm a little worried about adding that exception. Take
> the boingboing.net example again. There are 10 areas, 1 is on the screen,
> it's playing video. If once a second all 9 areas have their animationFrame
> callback called that video will probably glitch or stutter once a second.  I
> suppose the UA can stagger the offscreen animationFrame calls but I just
> wonder if calling them at all when they are offscreen is really warranted.
> Or how about I've got boingboing.net in one tab (10 areas), techcrunch.comin 
> another (4 areas),
> geekologie.com in a 3rd (7 areas), nytimes.com in a 4th (3 areas), add a
> few more and soon my machine will end up being bogged down making
> animationFrame callbacks to offscreen tabs.  Is the solution to just keep
> calling each one less and less frequently? I'd prefer it if my under powered
> netbook wasn't spending it's time and battery rendering things offscreen
> ever if possible. Not only that, but as I add more and more tabs I'd like to
> be able to swap them out of memory, but if every second or 2 seconds or 10
> seconds each one gets an animationFrame callback then it will be swapped
> back into memory.
>
>
I agree.  What's the use case for animating hidden tabs (or canvases that
are hidden)?

One of the big problems with JavaScript based animations is that they have
no way of knowing they should "go idle" when their window is hidden (in a
background tab or otherwise).  The same API used to rate-limit rendering
could address the problem of hidden tabs too.

-Darin



>
>
>
>
>>
>> It's a little tricky to define exactly what it means for an element to be
>> visible. I think we could probably leave that up to the UA without hurting
>> interop.
>>
>>
>> Rob
>> --
>> "Now the Bereans were of more noble character than the Thessalonians, for
>> they received the message with great eagerness and examined the Scriptures
>> every day to see if what Paul said was true." [Acts 17:11]
>>
>
>

Re: requestAnimationFrame

[Darin Fisher]

How about just running the callback once the tab becomes visible again?  It
will run, but just not unless there is reason to animate/paint.

-Darin



On Thu, Nov 18, 2010 at 6:08 PM, Robert O'Callahan wrote:

> I suppose we could have a variant API that explicitly means "I don't care
> if the callback never gets called". I don't know what to call it, though.
>
>
> Rob
> --
> "Now the Bereans were of more noble character than the Thessalonians, for
> they received the message with great eagerness and examined the Scriptures
> every day to see if what Paul said was true." [Acts 17:11]
>

Re: Structured clone in WebStorage

[Darin Fisher]

Have you guys considered what happens when a Blob is present?  I'm very
concerned about that case since WebStorage is a synchronous API.  If storing
a Blob in LocalStorage involves synchronous disk IO, then I'm pretty sure it
is something I would object to implementing.  I don't think there is a
hand-wavy answer about performing a file copy asynchronously in the
background.  What if the file copy fails?

-Darin



On Thu, Dec 2, 2010 at 1:45 PM, Anne van Kesteren  wrote:

> On Thu, 02 Dec 2010 21:51:29 +0100, Maciej Stachowiak 
> wrote:
>
>> We think this feature would be straightforward to implement in
>> Safari/WebKit, and we think it is a useful feature. We would like to
>> implement it at some point. I can't give a specific timeline.
>>
>
> I am not sure if it is straightforward to implement in Opera, but this
> applies to us as well.
>
>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>
>

Re: Structured clone in WebStorage

[Darin Fisher]

I will also add that I think WebStorage (well LocalStorage) is terrible from
a performance point of view because it is synchronous, and I'd be very happy
if we could discourage its usage and give people more reasons to adopt a
better API like IndexedDB.

-Darin



On Fri, Dec 3, 2010 at 10:41 AM, Darin Fisher  wrote:

> Have you guys considered what happens when a Blob is present?  I'm very
> concerned about that case since WebStorage is a synchronous API.  If storing
> a Blob in LocalStorage involves synchronous disk IO, then I'm pretty sure it
> is something I would object to implementing.  I don't think there is a
> hand-wavy answer about performing a file copy asynchronously in the
> background.  What if the file copy fails?
>
> -Darin
>
>
>
> On Thu, Dec 2, 2010 at 1:45 PM, Anne van Kesteren wrote:
>
>> On Thu, 02 Dec 2010 21:51:29 +0100, Maciej Stachowiak 
>> wrote:
>>
>>> We think this feature would be straightforward to implement in
>>> Safari/WebKit, and we think it is a useful feature. We would like to
>>> implement it at some point. I can't give a specific timeline.
>>>
>>
>> I am not sure if it is straightforward to implement in Opera, but this
>> applies to us as well.
>>
>>
>> --
>> Anne van Kesteren
>> http://annevankesteren.nl/
>>
>>
>

Re: [FileAPI] File.slice spec bug

[Darin Fisher]

On Tue, Apr 12, 2011 at 4:02 PM, Mike Taylor  wrote:

> On 4/12/11 2:05 PM, Jonas Sicking wrote:
>
>> It appears that Opera too implements File.slice. Would be great to
>> know for how long it's been implemented.
>>
> The first public build [1] with File.slice was made available last week.
> It's only been officially supported as of today, however, with the release
> of 11.10.
>
> [1] http://my.opera.com/desktopteam/blog/2011/04/05/stability-gmail-socks
>
> - Mike
>


As Jian mentioned earlier, File.slice has been available in Chrome since
version 6, which was released Sept 2, 2010:
http://googlechromereleases.blogspot.com/2010/09/stable-and-beta-channel-updates.html

It seems like too much time has passed with this feature baked into Chrome
for us to be able to change its behavior now.  While I agree that it would
be nice for ".slice(...)" to work the same way in all contexts, I'm just not
sure that this is something we can remove from Chrome at this point or even
change.

I'm very concerned about breaking existing web content.

A new name for this function might be nice, but once you do that then I'm
not sure that its arguments should really be any different than the current
arguments for Blob.slice().  What's wrong with "start" and "length" other
than that the fact that it differs from the parameters to Array.slice?  Why
should Blob.createSubBlob favor the argument style of Array.slice over
Blob.slice?

I guess I'm leaning toward no change at all and just taking our lumps for
having created an inconsistent API :-/

-Darin

Re: [FileAPI] File.slice spec bug

[Darin Fisher]

On Tue, Apr 12, 2011 at 5:59 PM, Brendan Eich  wrote:

> Darin's position is "leaning toward" not breaking compatibility with what
> Chrome has shipped for a while. That's one consideration. It can't be the
> only consideration, or there's no point having a discussion and whatever
> Chrome ships first is an instant standard.
>

Agreed.  I certainly don't assert that whatever Chrome ships first should be
regarded as standard.  Our rapid release schedule depends on platform
features beginning life with a vendor prefix.  I believe that we goofed in
this case by not vendor-prefixing the slice method.  More on this below...



>
> When we talked to Kenneth Arnold of Google about a typed array method that
> was more subtly misnamed "slice", he quickly worked with other WebGL folks
> to rename it to "subarray". That in spite of some support for the typed
> array "slice" method having shipped in some beta releases (at least; not
> sure if any final releases supported slice).
>

s/Kenneth Arnold/Kenneth Russell/

WebGL was disabled by default (hidden behind a flag) up until Chrome 9.  It
was "trivial" to change the API from the point of view of Chrome prior to it
coming out from behind a flag.



>
> The subtler differences between typed array's subarray (formerly named
> slice) and JS's Array slice:
>
> (a) The subarray method returns a view of mutable ArrayBuffer data, its
> elements alias the elements of the typed array from which it came. This is
> the big reason to avoid using "slice".
>
> (b)  While subarray takes (start, end) parameters and allows end to be
> omitted; and it handles negative indexes; it does not allow start to be
> omitted too, whereas JS (after Python) does.
>
> http://www.khronos.org/registry/typedarray/specs/latest/
>
> http://www.khronos.org/webgl/public-mailing-list/archives/1102/msg00028.html
>
> The DOM is full of crazy inconsistency from insta-standards in the
> mid-'90s, starting with my DOM level 0 work. That is not a good pattern to
> repeat now that we have more balanced market actors and the benefits of
> hindsight. We should try to agree on less inconsistent APIs and prototype
> them, but no single prototype implementation can create a standard. Only
> interoperable de-facto standards can do that.
>
> At this point we need to be completely concrete, no leaning one way or
> another:
>
> 0. Do nothing.
>
> 1. Should we change slice, possibly breaking Chrome-only content that
> detects it by name but does not test its functionality?
>
> 2. Should we rename slice, allowing Chrome to keep the old name for a while
> if it must?
>
> Solution 0 just leaves a noxious inconsistency on two counts: length not
> end as the second parameter, the lack of negative index support. Is there
> another difference with Array slice (which comes from Python and is in Ruby
> too)? What about optionality of the arguments?
>
> Doing 1 is simpler for future users who don't have to deal with the legacy
> that Chrome-only content might represent.
>
> Doing 2 avoids breaking any such legacy, but may increase the cognitive
> load for users who see the legacy and the new renamed method.
>
> What about negative indexes? Is there time to spec and implement these per
> Python/JS/Ruby?
>
> Without negative index support, the case for 2 is stronger again.
>
> If Blob slice returns an aliasing view onto mutable data instead of a copy,
> just as typed arrays' subarray method does, then the case for 2 is extremely
> strong.
>

Blobs are views on immutable data.  WebKit's implementation will reject
reads on a Blob, which points to a file that has since been modified.  (This
should be part of the spec if it is not.)



>
> I do not see any mutator methods in http://dev.w3.org/2006/webapi/FileAPI/,
> but with the typed array precedent I didn't want to assume.
>
> I favor 1 if negative index and optional argument semantics can be made to
> match precedent, and assuming no way to mutate shared Blob/sub-Blob data.
>
> If not, I suggest 2, with "subBlob" or a better name -- that one hurts due
> to the shifted-B, but "subblob" seems also bad for the run-together "bb".
>
> /be



One option we can consider is to rename Blob.slice to Blob.webkitSlice and
adopt the newer arguments.  I think we would want to similarly vendor-prefix
BlobBuilder and FileReader as those too are non-standard.  This change runs
just as much risk of breaking Chrome-specific content; however, the failure
mode (absence of a method) should be easier for apps to cope with.

I'm not stating that we will do this yet.  I'm just proposing it as another
approach to consider.  I'll discuss this idea with others on the team
tomorrow.

By the way, I'm not too concerned about having to fix Google web apps.  I'm
much more concerned about third-party, Chrome-specific content available as
either apps or extensions through the Chrome app store.  It is hard to grep
for instances of Blob.slice!!

-Darin

Re: [FileAPI] File.slice spec bug

[Darin Fisher]

On Tue, Apr 12, 2011 at 11:50 PM, Jonas Sicking  wrote:

> On Tue, Apr 12, 2011 at 10:01 PM, Darin Fisher  wrote:
> > On Tue, Apr 12, 2011 at 5:59 PM, Brendan Eich 
> wrote:
> >>
> >> Darin's position is "leaning toward" not breaking compatibility with
> what
> >> Chrome has shipped for a while. That's one consideration. It can't be
> the
> >> only consideration, or there's no point having a discussion and whatever
> >> Chrome ships first is an instant standard.
> >
> > Agreed.  I certainly don't assert that whatever Chrome ships first should
> be
> > regarded as standard.  Our rapid release schedule depends on platform
> > features beginning life with a vendor prefix.  I believe that we goofed
> in
> > this case by not vendor-prefixing the slice method.  More on this
> below...
> >
> >>
> >> When we talked to Kenneth Arnold of Google about a typed array method
> that
> >> was more subtly misnamed "slice", he quickly worked with other WebGL
> folks
> >> to rename it to "subarray". That in spite of some support for the typed
> >> array "slice" method having shipped in some beta releases (at least; not
> >> sure if any final releases supported slice).
> >
> > s/Kenneth Arnold/Kenneth Russell/
> > WebGL was disabled by default (hidden behind a flag) up until Chrome 9.
>  It
> > was "trivial" to change the API from the point of view of Chrome prior to
> it
> > coming out from behind a flag.
> >
> >>
> >> The subtler differences between typed array's subarray (formerly named
> >> slice) and JS's Array slice:
> >>
> >> (a) The subarray method returns a view of mutable ArrayBuffer data, its
> >> elements alias the elements of the typed array from which it came. This
> is
> >> the big reason to avoid using "slice".
> >>
> >> (b)  While subarray takes (start, end) parameters and allows end to be
> >> omitted; and it handles negative indexes; it does not allow start to be
> >> omitted too, whereas JS (after Python) does.
> >>
> >> http://www.khronos.org/registry/typedarray/specs/latest/
> >>
> >>
> http://www.khronos.org/webgl/public-mailing-list/archives/1102/msg00028.html
> >>
> >> The DOM is full of crazy inconsistency from insta-standards in the
> >> mid-'90s, starting with my DOM level 0 work. That is not a good pattern
> to
> >> repeat now that we have more balanced market actors and the benefits of
> >> hindsight. We should try to agree on less inconsistent APIs and
> prototype
> >> them, but no single prototype implementation can create a standard. Only
> >> interoperable de-facto standards can do that.
> >>
> >> At this point we need to be completely concrete, no leaning one way or
> >> another:
> >>
> >> 0. Do nothing.
> >>
> >> 1. Should we change slice, possibly breaking Chrome-only content that
> >> detects it by name but does not test its functionality?
> >>
> >> 2. Should we rename slice, allowing Chrome to keep the old name for a
> >> while if it must?
> >>
> >> Solution 0 just leaves a noxious inconsistency on two counts: length not
> >> end as the second parameter, the lack of negative index support. Is
> there
> >> another difference with Array slice (which comes from Python and is in
> Ruby
> >> too)? What about optionality of the arguments?
> >>
> >> Doing 1 is simpler for future users who don't have to deal with the
> legacy
> >> that Chrome-only content might represent.
> >>
> >> Doing 2 avoids breaking any such legacy, but may increase the cognitive
> >> load for users who see the legacy and the new renamed method.
> >>
> >> What about negative indexes? Is there time to spec and implement these
> per
> >> Python/JS/Ruby?
> >>
> >> Without negative index support, the case for 2 is stronger again.
> >>
> >> If Blob slice returns an aliasing view onto mutable data instead of a
> >> copy, just as typed arrays' subarray method does, then the case for 2 is
> >> extremely strong.
> >
> > Blobs are views on immutable data.  WebKit's implementation will reject
> > reads on a Blob, which points to a file that has since been modified.
>  (This
> > should be part of the spec if it is not.)
> >
> >
> >>
> >> I do not see any mutator meth

Re: [XHR2] load events on XMHttpRequestUpload

[Darin Fisher]

Hmm... this is curious.  I'm not sure that the Chrome behavior is
intentional.  I need
to investigate further.

Isn't there already a signal to tell you when response headers are
available?  Isn't it a
bit redundant for the upload complete notification to be tied to the same
signal?

To support the use case of showing a progress meter, it seems helpful to
know when
the browser has finished writing bytes on the wire.  It also seems like it
would be useful
for there to be a "resend" event indicating that the upload will be resent.
 Otherwise, one
has to guess this.

-Darin



On Thu, Mar 17, 2011 at 8:43 AM, Boris Zbarsky  wrote:

> http://www.w3.org/TR/**XMLHttpRequest2/section
>  3.6.9 near the end says:
>
>  If the request entity body has been successfully uploaded and the
>  upload complete flag is still false, queue a task to run these
>  substeps:
>
> We have implemented this in Gecko, but the result seems to be some
> confusion on the part of users of this API [1,2].  Furthermore, at least
> Chrome implements what those users expect, not what the spec currently
> specifies should happen.
>
> The fundamental problem is that you don't know whether the upload is
> "successful" until you get the start of the server's response.  In
> particular, if the server responds with a 401 or 307 the upload may well
> have to be redone.  And if the server never responds, then the upload may
> not have been successful: just because we put the bits on the wire doesn't
> mean the server got them.
>
> Therefore we fire "load" on XMHttpRequestUpload after we have gotten the
> initial part of the server response.  This is consistent with the text in
> the spec right now (and is incidentally easiest to do with our networking
> library, which doesn't really notify the consumer when it's done putting
> upload bits on the wire, for the same reasons: it's a misleading metric).
>
> The question is whether the spec should be relaxed to not require the
> upload is successful (in which case we'll change our implementation) or
> whether Chrome's implementation needs to change.  In the latter case the
> spec needs to be clearer about what "successful" means, obviously, since at
> least one implementor misinterpreted it.
>
> [1] 
> https://bugzilla.mozilla.org/**show_bug.cgi?id=637002
> [2] 
> https://bugzilla.mozilla.org/**show_bug.cgi?id=642463
>
>

Re: [XHR2] load events on XMHttpRequestUpload

[Darin Fisher]

On Tue, Jun 21, 2011 at 12:30 PM, Boris Zbarsky  wrote:

> On 6/21/11 3:01 PM, Darin Fisher wrote:
>
>> Isn't there already a signal to tell you when response headers are
>> available?
>>
>
> Yes; I believe the readystate changes at this point and onreadystatechange
> is fired.
>
>
>  Isn't it a bit redundant for the upload complete notification to be tied
>> to the
>> same signal?
>>
>
> Yes, sure.  But that the point when the browser knows the upload is
> complete


It really depends on how you define complete.  I think it is useful to know
when
all of the data has been sent.  That seems useful to applications.  An
application
might wish to switch from a progress meter for upload percentage to a static
notification indicating that the application is now simply waiting for a
response.
Chrome for instance does exactly this in its status bar for HTML form
submissions.
Why shouldn't web apps be able to present the same kind of detailed upload
progress UI?



>
>
>  To support the use case of showing a progress meter, it seems helpful to
>> know when the browser has finished writing bytes on the wire.
>>
>
> If you're willing to restart the meter from 0 in some cases, perhaps.


Yeah :-)


>
>
> > It also seems like it would be useful
>
>> for there to be a "resend" event indicating that the upload will be
>> resent.  Otherwise, one
>> has to guess this.
>>
>
> Yep.
>
> -Boris
>


So, anyways... I think I like how Chrome is behaving.  I haven't checked to
see
if this is a property of WebKit or Chromium's networking layer.  I suspect
it is
a WebKit detail.  Anyways, I think it would be useful to update the spec to
support this behavior.  It seems useful :-)

-Darin

Comment on File API's FileData::slice method

[Darin Fisher]

FileData::slice appears to be spec'd like so:
FileData slice(in long long offset, in long long length); // throws
FileException

This suggests that it may throw a file exception.  I'm wondering if that is
a requirement?  It seems that the rest of the methods are designed to report
success or failure asynchronously.  I'm wondering why slice is not spec'd
similarly.

I think it would be bad for UAs to implement this exception since it would
require a synchronous stat'ing of the filesystem.  That could hang the
calling thread for a long time (consider network file systems).

It seems like it would be better for the spec to encourage
an asynchronous implementation.  For example, it should be the case that the
FileData object returned by slice may or may not be valid.  The user would
only find out when they try to use the FileData object.

Thoughts?
-Darin

Re: Comment on File API's FileData::slice method

[Darin Fisher]

On Fri, Oct 2, 2009 at 9:15 AM, Arun Ranganathan
wrote:

> Darin Fisher wrote:
>
>> FileData::slice appears to be spec'd like so:
>> FileData slice(in long long offset, in long long length); // throws
>> FileException
>>
>> This suggests that it may throw a file exception.  I'm wondering if that
>> is
>> a requirement?  It seems that the rest of the methods are designed to
>> report
>> success or failure asynchronously.  I'm wondering why slice is not spec'd
>> similarly.
>>
>> I think it would be bad for UAs to implement this exception since it would
>> require a synchronous stat'ing of the filesystem.  That could hang the
>> calling thread for a long time (consider network file systems).
>>
>> It seems like it would be better for the spec to encourage
>> an asynchronous implementation.  For example, it should be the case that
>> the
>> FileData object returned by slice may or may not be valid.  The user would
>> only find out when they try to use the FileData object.
>>
>>
> By asynchronous, do you mean, with a callback function for the FileData
> object to be returned?  It sounds like what you are advocating is that this
> method does not raise an exception, but the FileData object is either valid
> or null.  If so, I think that sounds sensible.  Have I understood you
> correctly?
>
> -- A*
>
>
>

Not quite.  My proposal is that:

1) no exception is thrown
2) a non-null FileData object is always returned

The parameters of the returned FileData object are only validated when you
try to use the FileData object.

Note: it is possible that we may still wish to throw an exception if the
inputs to slice can be determined to be invalid without requiring a
filesystem 'stat' call.  For example, if either the offset or length is
negative, then clearly we could consider the slice invalid.

-Darin

Re: Comment on File API's FileData::slice method

[Darin Fisher]

On Fri, Oct 2, 2009 at 4:00 AM, Olli Pettay  wrote:

> On 10/2/09 7:31 AM, Darin Fisher wrote:
>
>> FileData::slice appears to be spec'd like so:
>>
>> FileData slice(in long long offset, in long long length); // throws
>> FileException
>>
>> This suggests that it may throw a file exception.  I'm wondering if that
>> is a requirement?  It seems that the rest of the methods are designed to
>> report success or failure asynchronously.  I'm wondering why slice is
>> not spec'd similarly.
>>
>> I think it would be bad for UAs to implement this exception since it
>> would require a synchronous stat'ing of the filesystem.  That could hang
>> the calling thread for a long time (consider network file systems).
>>
> It doesn't need to stat if we expect that the original FileData's length
> can be used also for sliced FileDatas.
>

That's a good point.  I was reacting to all of the other values for
FileException, namely:
NOT_FOUND_ERR, NOT_READABLE_ERR, etc.



> The API doesn't really handle the case where file length changes, so
> the same problem there is with slice is with .length property.



I think that .length could be determined at the time when the file picker
was run.
The UA could just remember the value and report that without requiring a
'stat'
each time .length is accessed.

Backing up a step, so long as the API does not require synchronous file
system
access from the main thread, I'm fine with it.  Synchronous, blocking IO
from the
main thread is *evil*.

-Darin



>
>
>
>
>> It seems like it would be better for the spec to encourage
>> an asynchronous implementation.  For example, it should be the case that
>> the FileData object returned by slice may or may not be valid.  The user
>> would only find out when they try to use the FileData object.
>>
>
> Synchronous slice seems pretty useful to me.
> Reader methods need to have some error handling anyway, so why to have it
> also with slice. (I guess the exception is thrown only if offset or length
> have out-of-bounds values).
>
> If we want to support streaming or handle changes to file's size, the API
> needs some changes. Or we need a separate API for streaming.
>
> Btw, I'm not sure now which is the latest draft.
> http://dev.w3.org/2006/webapi/FileUpload/publish/FileAPI.html is one, but
> I think there is events based API somewhere.
>
>
> -Olli
>

re: Proposal for sending multiple files via XMLHttpRequest.send()

[Darin Fisher]

I'd like to revive the "Proposal for sending multiple files via
XMLHttpRequest.send()" thread
started by Jian Li back in September.

As pointed out on that thread, sending a JS array of strings and File
references isn't going
to fly due to an array of strings already having meaning.

That thread concluded with Jonas Sicking saying:

> I definitely think that we need to add some way of sending a stream
> that is a concatenation of strings, binary data, and files, without
> requiring the files to be read.

What should this look like?  Should there be a new global constructor for an
object that supports methods like appendText and appendFile?  Like this:

var data = new UploadData;
data.appendText("foo");
data.appendFile(fileRef);
xhr.send(data);

I like the idea of creating a primitive that does not enforce any encoding.

Thoughts?
-Darin

Re: Proposal for sending multiple files via XMLHttpRequest.send()

[Darin Fisher]

On Tue, Oct 20, 2009 at 1:37 AM, Anne van Kesteren  wrote:

> On Tue, 20 Oct 2009 08:55:21 +0200, Darin Fisher 
> wrote:
>
>> I'd like to revive the "Proposal for sending multiple files via
>> XMLHttpRequest.send()" thread
>> started by Jian Li back in September.
>>
>> As pointed out on that thread, sending a JS array of strings and File
>> references isn't going
>> to fly due to an array of strings already having meaning.
>>
>> That thread concluded with Jonas Sicking saying:
>>
>>  I definitely think that we need to add some way of sending a stream
>>> that is a concatenation of strings, binary data, and files, without
>>> requiring the files to be read.
>>>
>>
>> What should this look like?  Should there be a new global constructor for
>> an
>> object that supports methods like appendText and appendFile?  Like this:
>>
>> var data = new UploadData;
>> data.appendText("foo");
>> data.appendFile(fileRef);
>> xhr.send(data);
>>
>> I like the idea of creating a primitive that does not enforce any
>> encoding.
>>
>> Thoughts?
>>
>
> I think for a lot of authors the easiest would be easiest if it was in the
> form of multipart/form-data as then they do not have to do anything special
> to get the data on the server.
>
>

That does address the primary use case, but do you really think that we
should rule out the use case of custom encodings?

(At least for Gecko and WebKit, it is really trivial to support the more
free-form interface since the underlying network layers speak in terms of a
list of strings and file references.)

If we do go the more specific route, then it could look like this:

var data = new FormData;
data.appendText("name1", "value");
data.appendFile("name2", fileData);
xhr.send(data);

The UA could populate the filename attribute of the Content-Disposition
header based on the given fileRef.  Same goes for the Content-Type header.

Hmm... this may indeed be a good way to go since I don't think the proposed
FileData (
http://dev.w3.org/2006/webapi/FileUpload/publish/FileAPI.html#FileData-if)
interface includes a name attribute.

-Darin



>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>

Re: Proposal for sending multiple files via XMLHttpRequest.send()

[Darin Fisher]

On Tue, Oct 20, 2009 at 4:31 AM, Anne van Kesteren  wrote:

> On Tue, 20 Oct 2009 10:56:47 +0200, Darin Fisher 
> wrote:
>
>> On Tue, Oct 20, 2009 at 1:37 AM, Anne van Kesteren 
>> wrote:
>>
>>> I think for a lot of authors the easiest would be easiest if it was in
>>> the form of multipart/form-data as then they do not have to do anything
>>> special to get the data on the server.
>>>
>>
>> That does address the primary use case, but do you really think that we
>> should rule out the use case of custom encodings?
>>
>> (At least for Gecko and WebKit, it is really trivial to support the more
>> free-form interface since the underlying network layers speak in terms of
>> a list of strings and file references.)
>>
>
> I do not want to rule anything out, but a request I repeatedly hear is
> duplicating  functionality with XMLHttpRequest. Support for
> multipart/form-data furthermore is extremely widespread and resembles quite
> closely what is desired so it seems good to use it.
>
>
OK



> If eventually we get native support for octet-arrays and all we can at that
> point add the ability to XMLHttpRequest so you can send anything you want.
>
>
Hmm, OK



>
>  If we do go the more specific route, then it could look like this:
>>
>> var data = new FormData;
>> data.appendText("name1", "value");
>> data.appendFile("name2", fileData);
>> xhr.send(data);
>>
>> The UA could populate the filename attribute of the Content-Disposition
>> header based on the given fileRef.  Same goes for the Content-Type header.
>>
>> Hmm... this may indeed be a good way to go since I don't think the
>> proposed
>> FileData (
>> http://dev.w3.org/2006/webapi/FileUpload/publish/FileAPI.html#FileData-if
>> )
>> interface includes a name attribute.
>>
>
> This looks reasonable to me.
>
>
> I suppose we also want support passing File[Data] and FileList directly as
> I believe WebKit might have already added. Do you happen to know the
> implementation details?
>
>
WebKit presently supports sending File.  It does not support FileData yet.
 I'm not sure about supporting FileList.  If it were to be encoded using
multipart/form-data, then how would the form field names be chosen?  Would
they just be excluded from the encoding?

If we have something like the FormData type proposed above, then we wouldn't
need to support sending a FileList.  People could just iterate a FileList
and call appendFile for each element of the FileList.



>
> (I'm planning to wait with defining this until at least after TPAC. I
> expect more discussion to happen there and I also want to keep focus on the
> CSSOM before switching tasks again. It would be good to have most of the
> details before than though.)
>
>
>
OK

I'm interested in having a WebKit implementation for this soon.

-Darin



>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>

Re: Proposal for sending multiple files via XMLHttpRequest.send()

[Darin Fisher]

On Tue, Oct 20, 2009 at 9:47 AM, Anne van Kesteren  wrote:

> On Tue, 20 Oct 2009 18:27:36 +0200, Darin Fisher 
> wrote:
>
>> On Tue, Oct 20, 2009 at 4:31 AM, Anne van Kesteren 
>> wrote:
>>
>>> If eventually we get native support for octet-arrays and all we can at
>>> that point add the ability to XMLHttpRequest so you can send anything you
>>> want.
>>>
>>
>> Hmm, OK
>>
>
> If you envision things differently, by all means say so :-)



Sorry...  I just meant that I need to read up on discussions about
octet-array support before I can comment ;-)



>
>
>
>   If we do go the more specific route, then it could look like this:
>>>
>>>>
>>>> var data = new FormData;
>>>> data.appendText("name1", "value");
>>>> data.appendFile("name2", fileData);
>>>> xhr.send(data);
>>>>
>>>> The UA could populate the filename attribute of the Content-Disposition
>>>> header based on the given fileRef.  Same goes for the Content-Type
>>>> header.
>>>>
>>>> Hmm... this may indeed be a good way to go since I don't think the
>>>> proposed
>>>> FileData (
>>>>
>>>> http://dev.w3.org/2006/webapi/FileUpload/publish/FileAPI.html#FileData-if
>>>> )
>>>> interface includes a name attribute.
>>>>
>>>>
>>> This looks reasonable to me.
>>>
>>>
>>> I suppose we also want support passing File[Data] and FileList directly
>>> as I believe WebKit might have already added. Do you happen to know the
>>> implementation details?
>>>
>>
>> WebKit presently supports sending File.  It does not support FileData yet.
>>
>
> Is Content-Type set to anything specific if the author has not set it?


// FIXME: Should we set a Content-Type if one is not set.

^^^ looks like it presently depends on the web page setting the C-T header
manually :-/  I think that should be fixed to be based on the system's
knowledge of the file's mime type.



> I assume what is being transferred is the file data, octet-for-octet, as it
> is represented on the disk?


Yes.



> Does it make a straight copy of the file or can modifying the file cause
> potential problems?


Modifying the file can cause potential problems.  The same is true of
ordinary HTML form submissions.  I don't think we should spec this behavior.



>
>
>
>  I'm not sure about supporting FileList.  If it were to be encoded using
>> multipart/form-data, then how would the form field names be chosen?  Would
>> they just be excluded from the encoding?
>>
>> If we have something like the FormData type proposed above, then we
>> wouldn't need to support sending a FileList.  People could just iterate a
>> FileList
>> and call appendFile for each element of the FileList.
>>
>
> True.
>
>
>
>  (I'm planning to wait with defining this until at least after TPAC. I
>>> expect more discussion to happen there and I also want to keep focus on
>>> the CSSOM before switching tasks again. It would be good to have most of the
>>> details before than though.)
>>>
>>
>> OK
>>
>> I'm interested in having a WebKit implementation for this soon.
>>
>
> After TPAC means the week of November 9. Does that work?
>
>
Sure.  We might start on things sooner, but it doesn't mean it can't be
revised.



>
> Meanwhile my basic idea based on your proposal is:
>
>  [Constructor] interface FormData {
>void append(DOMString name, DOMString value);
>void append(DOMString name, FileData value);
>  }
>
> Then when send() is invoked with a FormData object I assume the easiest
> would be to make a copy of the object serialize it per the rules of RFC 2388
> (encoding DOMString values as UTF-8) and done.


Yup, sounds good.  I like the overloaded "append" methods.

Thanks,
-Darin



>
>
>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>

Re: Proposal for sending multiple files via XMLHttpRequest.send()

[Darin Fisher]

On Tue, Nov 10, 2009 at 7:48 AM, Jonas Sicking  wrote:

> Yay!
>
> On Tue, Nov 10, 2009 at 4:06 PM, Anne van Kesteren 
> wrote:
>  WebKit presently supports sending File.  It does not support FileData
>  yet.
> >>>
> >>> Is Content-Type set to anything specific if the author has not set it?
> >>
> >> // FIXME: Should we set a Content-Type if one is not set.
> >>
> >> ^^^ looks like it presently depends on the web page setting the C-T
> header
> >> manually :-/  I think that should be fixed to be based on the system's
> >> knowledge of the file's mime type.
> >
> > So far I have only added support for Blob (the new FileData) of these two
> > and set the Content-Type to application/octet-stream if not provided by
> the
> > Web author. Should I also add File and set the Content-Type to
> > File.mediaType?
>
> I definitely think so yes.
>

Agreed.



>
> Same thing for the send() function.
>
> On a separate note, is sending "application/octet-stream" really
> better than simply sending no Content-Type header at all when sending
> a Blob (or a File with empty mediaType)?
>
>
I think it is probably wise to default to sending a content type that
explicitly says "i don't know the type" on the grounds that being explicit
is better than not.

-Darin

Re: File API: Blob and underlying file changes.

[Darin Fisher]

I don't think we should worry about underlying file changes.

If the app wants to cut a file into parts and copy them separately, then
perhaps the app should first copy the file into a private area.  (I'm
presuming that one day, we'll have the concept of a chroot'd private file
storage area for a web app.)

I think we should avoid solutions that involve file locking since it is bad
for the user (loss of control) if their files are locked by the browser on
behalf of a web app.

It might be reasonable, however, to lock a file while sending it.

-Darin


On Thu, Jan 14, 2010 at 2:41 PM, Jian Li  wrote:

> It seems that we feel that when a File object is sent via either Form or
> XHR, the latest underlying version should be used. When we get a slice via
> Blob.slice, we assume that the underlying file data is stable since then.
>
> So for uploader scenario, we need to cut a big file into multiple pieces.
> With current File API spec, we will have to do something like the following
> to make sure that all pieces are cut from a stable file.
> var file = myInputElement.files[0];
> var blob = file.slice(0, file.size);
> var piece1 = blob.slice(0, 1000);
> var piece2 = blob.slice(1001, 1000);
> ...
>
> The above seems a bit ugly. If we want to make it clean, what Dmitry
> proposed above seems to be reasonable. But it would require non-trivial spec
> change.
>
>
> On Wed, Jan 13, 2010 at 11:28 AM, Dmitry Titov wrote:
>
>> Atomic read is obviously a nice thing - it would be hard to program
>> against API that behaves as unpredictably as a single read operation that
>> reads half of old content and half of new content.
>>
>> At the same note, it would be likely very hard to program against Blob
>> objects if they could change underneath unpredictably. Imagine that we need
>> to build an uploader that cuts a big file in multiple pieces and sends those
>> pieces to the servers so they will be stitched together later. If during
>> this operation the underlying file changes and this changes all the pieces
>> that Blobs refer to (due to clamping and just silent change of content), all
>> the slicing/stitching assumptions are invalid and it's hard to even notice
>> since blobs are simply 'clamped' silently. Some degree of mess is possible
>> then.
>>
>> Another use case could be a JPEG image processor that uses slice() to cut
>> the headers from the image file and then uses info from the headers to cut
>> further JFIF fields from the file (reading EXIF and populating local
>> database of images for example). Changing the file in the middle of that is
>> bad.
>>
>> It seems the typical use cases that will need Blob.slice() functionality
>> form 'units of work' where Blob.slice() is used with likely assumption that
>> underlying data is stable and does not change silently. Such a 'unit of
>> work'  should fail as a whole if underlying file changes. One way to achieve
>> that is to reliably fail operations with 'derived' Blobs and even perhaps
>> have a 'isValid' property on it. 'Derived' Blobs are those obtained via
>> slice(), as opposite to 'original' Blobs that are also File.
>>
>> One disadvantage of this approach is that it implies that the same Blob
>> has 2 possible behaviors - when it is obtained via Blob.slice() (or other
>> methods) vs is a File.
>>
>> It all could be a bit cleaner if File did not derive from Blob, but
>> instead had getAsBlob() method - then it would be possible to say that Blobs
>> are always immutable but may become 'invalid' over time if underlying data
>> changes. The FileReader can then be just a BlobReader and have cleaner
>> semantics.
>>
>> If that was the case, then xhr.send(file) would capture the state of file
>> at the moment of sending, while xhr.send(blob) would fail with exception if
>> the blob is 'invalid' at the moment of send() operation. This would keep
>> compatibility with current behavior and avoid duplicity of Blob behavior.
>> Quite a change to the spec though...
>>
>> Dmitry
>>
>> On Wed, Jan 13, 2010 at 2:38 AM, Jonas Sicking  wrote:
>>
>>> On Tue, Jan 12, 2010 at 5:28 PM, Chris Prince 
>>> wrote:
>>> >> For the record, I'd like to make the read "atomic", such that you can
>>> >> never get half a file before a change, and half after. But it likely
>>> >> depends on what OSs can enforce here.
>>> >
>>> > I think *enforcing* atomicity is difficult across all OSes.
>>> >
>>> > But implementations can get nearly the same effect by checking the
>>> > file's last modification time at the start + end of the API call.  If
>>> > it has changed, the read operation can throw an exception.
>>>
>>> I'm talking about during the actual read. I.e. not related to the
>>> lifetime of the File object, just related to the time between the
>>> first 'progress' event, and the 'loadend' event. If the file changes
>>> during this time there is no way to fake atomicity since the partial
>>> file has already been returned.
>>>
>>> / Jonas
>>>
>>
>>
>

Re: File API: Blob and underlying file changes.

[Darin Fisher]

On Fri, Jan 15, 2010 at 10:19 AM, Dmitry Titov  wrote:

> Nobody proposed locking the file. Sorry for being unclear if that sounds
> like it. Basically it's all about timestamps.
>
> As Chris proposed earlier, a read operation can grab the timestamp of the
> file before and after reading its content and throw exception if the
> timestamps do not match. This is pretty good approximation of 'atomic' read
> - although it can not guarantee success, it can at least provide reliable
> detection of it.
>

but doesn't that imply some degree of unpredictability for web developers?
 must they always handle that exception even though it is an extremely rare
occurrence?  also, what about normal form submission, in which the file
reading happens asynchronously to form.submit().



>
> Same thing with the Blob - the slice() may capture the timestamp of the
> content it's based on. Blob can throw exception later if the modification
> timestamp of underlying data has changed since the time of Blob's creation.
>

also note that we MUST NOT design APIs that involve synchronous file access.
 no "stat" calls allowed on the main UI thread please!  (remember the
network filesystem case.)

in other words, assuming detection of file changes happens asynchronously,
we'll have trouble producing exceptions as you describe.



>
> Both actual OS locking and requiring copying files to a safe location
> before slice() seem to be problematic, for different reasons. Good example
> is youtube uploader that needs to slice and send 1Gb file, while having a
> way to reliably detect the change of the underlyign file, terminate current
> upload and potentially request another one. Copying is hard because of size
> and locking, even if provided by OS, may stay in the way of user's workflow.
>
> Dmitry
>
> On Thu, Jan 14, 2010 at 11:58 PM, Darin Fisher  wrote:
>
>> I don't think we should worry about underlying file changes.
>>
>> If the app wants to cut a file into parts and copy them separately, then
>> perhaps the app should first copy the file into a private area.  (I'm
>> presuming that one day, we'll have the concept of a chroot'd private file
>> storage area for a web app.)
>>
>> I think we should avoid solutions that involve file locking since it is
>> bad for the user (loss of control) if their files are locked by the browser
>> on behalf of a web app.
>>
>> It might be reasonable, however, to lock a file while sending it.
>>
>> -Darin
>>
>>
>> On Thu, Jan 14, 2010 at 2:41 PM, Jian Li  wrote:
>>
>>> It seems that we feel that when a File object is sent via either Form or
>>> XHR, the latest underlying version should be used. When we get a slice via
>>> Blob.slice, we assume that the underlying file data is stable since then.
>>>
>>> So for uploader scenario, we need to cut a big file into multiple pieces.
>>> With current File API spec, we will have to do something like the following
>>> to make sure that all pieces are cut from a stable file.
>>>  var file = myInputElement.files[0];
>>> var blob = file.slice(0, file.size);
>>> var piece1 = blob.slice(0, 1000);
>>> var piece2 = blob.slice(1001, 1000);
>>> ...
>>>
>>> The above seems a bit ugly. If we want to make it clean, what Dmitry
>>> proposed above seems to be reasonable. But it would require non-trivial spec
>>> change.
>>>
>>>
>>> On Wed, Jan 13, 2010 at 11:28 AM, Dmitry Titov wrote:
>>>
>>>> Atomic read is obviously a nice thing - it would be hard to program
>>>> against API that behaves as unpredictably as a single read operation that
>>>> reads half of old content and half of new content.
>>>>
>>>> At the same note, it would be likely very hard to program against Blob
>>>> objects if they could change underneath unpredictably. Imagine that we need
>>>> to build an uploader that cuts a big file in multiple pieces and sends 
>>>> those
>>>> pieces to the servers so they will be stitched together later. If during
>>>> this operation the underlying file changes and this changes all the pieces
>>>> that Blobs refer to (due to clamping and just silent change of content), 
>>>> all
>>>> the slicing/stitching assumptions are invalid and it's hard to even notice
>>>> since blobs are simply 'clamped' silently. Some degree of mess is possible
>>>> then.
>>>>
>>>> Another use case could be a JPEG image processor that uses

Re: Notifications

[Darin Fisher]

On Wed, Feb 10, 2010 at 4:16 PM, Drew Wilson  wrote:

>
>
> On Wed, Feb 10, 2010 at 3:31 PM, Robert O'Callahan 
> wrote:
>
>> On Thu, Feb 11, 2010 at 12:03 PM, Drew Wilson wrote:
>>
>>>  On Wed, Feb 10, 2010 at 2:33 PM, Robert O'Callahan <
>>> rob...@ocallahan.org> wrote:
>>>
>>>
 I think a better way to go would be to support a restricted subset of
 HTML, and then consider how the UA should extract text for a plaintext-only
 notification framework (possibly opting to fall back to non-native
 notifications if the text extraction doesn't work). For example, you could
 allow only  and  elements, and for text-only notifications you 
 would
 strip  and use the alt text for , and if the author didn't provide
 alt text for  then and only then you would fall back to non-native
 notifications.

 It seems unusual to me that in a spec directed at HTML-based UAs, we
>>> would define some sort of domain-specific markup language rather than simply
>>> supporting HTML. Browsers know how to render HTML and can extract text
>>> appropriately if for some reason they need to down-render (for example, see
>>> the textContent and innerText element properties). Why would we define a
>>> separate markup language to allow defining marked up text that can be
>>> rendered differently depending on the display capabilities - isn't that what
>>> HTML is for? Is there an advantage to defining a subset?
>>>
>>
>> If it supports, e.g., scripting, then you have to define what the
>> relationship is between the browsing context for the notification and the
>> browsing context for the opener. Can a notification document navigate itself
>> using document.location = "..."? At that point, this sounds like a
>> specialized version of window.open. If that's your intent, maybe we should
>> just add a flag to window.open?
>>
>
> I agree - the spec needs to define all of the implications of HTML support
> as well as any restrictions as to its use. On the bright side, from a
> security standpoint treating an HTML notification as its own browsing
> context ala window.open() is a well-understood model.
>
> As for using a specialized version of window.open() for notifications -
> that's an interesting suggestion. I think we'd still need to deal with the
> permission grant issue as well as having a good way to generate
> icon+header+body notifications from the HTML for compatibility with
> Growl/NotifyOSD/mobile platforms - I still think that having some way for
> the developer to explicitly specify a title/icon/body is superior to having
> the browser have to infer these pieces from a parsed DOM tree. But it's a
> good point that this starts to feel more like window.open().
>
>
Maybe something like this?

icon  ==>
body  ==>  document.body.innerText





> Stepping back from the HTML issue, I note that one of the things in the
> NotifyOSD/DBUS API is the concept of a client-specified "replace ID" - the
> idea is that client applications can give their notifications a replace ID,
> and when that application displays a notification with that ID it replaces
> any currently displayed notification with that ID. I think that something
> like this would solve one of the core issues with the proposed notification
> API, which is avoiding duplicate notifications in the case where you have
> the same web page open in multiple windows.
>
> If we supported an optional "replace ID" parameter in createNotification(),
> then a page that wanted to have (say) a "you have new mail" notification
> could give that notification an ID of "new_mail", so if any other page under
> the same origin tried to display a notification with a "new_mail" ID it
> would just replace the existing notification rather than displaying a
> duplicate notification. If we don't support this, apps have to jump through
> hoops using SharedWorkers or other methods to make sure they don't annoy the
> user with duplicate notifications.
>


ReplaceID sounds quite useful.

-Darin



>
>
>>
>> Rob
>> --
>> "He was pierced for our transgressions, he was crushed for our iniquities;
>> the punishment that brought us peace was upon him, and by his wounds we are
>> healed. We all, like sheep, have gone astray, each of us has turned to his
>> own way; and the LORD has laid on him the iniquity of us all." [Isaiah
>> 53:5-6]
>>
>
>

Re: FormData with sliced Blob

[Darin Fisher]

Why couldn't FormData.append(Blob) provide optional parameters to allow the
caller to specify the name and type?

-Darin


On Tue, Mar 23, 2010 at 1:06 AM, Anne van Kesteren  wrote:

> On Tue, 23 Mar 2010 02:24:52 +0100, Dmitry Titov 
> wrote:
>
>> Seriously though, it would be nice to have XHR2 spec to have these details
>> spelled out, especially mime type (I think David meant
>> application/octet-stream)
>>
>
> We previously discussed this and then we decided that for Blob Content-Type
> simply would not be present. I don't see why that would be different for
> multipart/form-data. Not really sure what to do about filename.
>
>
>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>
>

[FileAPI] Blob.URN?

[Darin Fisher]

Apologies if this has been discussed before, but I'm curious why URN is not
a property of Blob.  It seems like it would be useful to be able to load a
slice of a File.  For example, this could be used by an application to fetch
all of its subresources out of a single file.

-Darin

Re: [FileAPI] Blob.URN?

[Darin Fisher]

Thanks for the explanation.  The getURN method is an interesting proposal.
 It seems useful even when you do have a File because you might want to
force the file contents to be loaded as a particular media type.

-Darin



On Tue, Mar 23, 2010 at 7:40 PM, Michael Nordman wrote:

> This has been discussed before, not sure what the conclusion was (if any)
> http://www.mail-archive.com/public-webapps@w3.org/msg06137.html
>
> <http://www.mail-archive.com/public-webapps@w3.org/msg06345.html>
>
> In order for the URN to be useful, it would have to have a mediaType
> associated with it, and then there's content-disposition to think
> about, which then wants a file name as well...boy, that's a lot of
> baggage.  However, since these aren't really inherent properties of
> the Blob, just of the way you want the browser to view the Blob, it
> would seem natural to me do to something like this:
>
> interface Blob {
>   ...
>   DOMString getURN(in DOMString mediaType,
>[Optional] in DOMString contentDisposition,
>[Optional] in DOMString name);
> };
>
> 
>
>
> On Tue, Mar 23, 2010 at 7:19 PM, Dmitry Titov  wrote:
>
>> Blob would need a content-type for that, but it could probably easy be
>> added as a property that is assignable.
>>
>> BTW if the Blob could have a urn and mime type, this info could be
>> directly used to form the headers for the Blob when sending it in multipart
>> form using FormData.
>>
>> Dmitry
>>
>> On Tue, Mar 23, 2010 at 2:23 PM, Darin Fisher  wrote:
>>
>>> Apologies if this has been discussed before, but I'm curious why URN is
>>> not a property of Blob.  It seems like it would be useful to be able to load
>>> a slice of a File.  For example, this could be used by an application to
>>> fetch all of its subresources out of a single file.
>>>
>>> -Darin
>>>
>>
>>
>

Re: Adding Content-Disposition header to File.urn response

[Darin Fisher]

The Blob.getURN proposal from Eric Uhrhane might be a better way to solve
the C-D issue:
http://www.mail-archive.com/public-webapps@w3.org/msg06137.html

It would give the user the ability to control whether a URN loaded into an
IFRAME triggers a download or not.

-Darin


On Tue, Mar 23, 2010 at 4:57 PM, Jian Li  wrote:

> Hi,
>
> We probably have already discussed this: adding the Content-Disposition
> header into the response when reading File.urn resource. But since this is
> not currently documented in the spec, I want to ping you guys to make sure
> we are all in the same page.
>
> When the header "Content-Dispositon: attachment" is added, the UA could
> either trigger the inline replacement or initiate the download depending on
> the different element type. For IMG/INPUT/VIDEO/SCRIPT/LINK, our UA is doing
> the replacement inline. For others like IFRAME/LOCATION, our UA will
> initiate the download. Are these behaviors you also expect for your UA?
>
> In addition, do we want to add the file name to the C-D header?
>
> I've heard that there is a discussion on setting C-D header pragmatically.
> Do we want to go along this way?
>
> Thanks,
>
> Jian
>
>
>

Re: FormData with sliced Blob

[Darin Fisher]

On Wed, Mar 24, 2010 at 2:55 AM, Anne van Kesteren  wrote:

> On Tue, 23 Mar 2010 22:22:37 +0100, Darin Fisher 
> wrote:
>
>> Why couldn't FormData.append(Blob) provide optional parameters to allow
>> the caller to specify the name and type?
>>
>
> Why wouldn't you provide a File object instead then? It seems a lot of use
> cases here and elsewhere would be addressed by having a way to get a File
> out of a Blob.
>
>
That's a very interesting idea.  I hadn't considered the possibility of
creating a File from a Blob.  I guess that would imply having a File object
that is not necessarily backed by an on-disk representation.  Obviously,
there was nothing promising that before.  Your proposal sounds reasonable to
me.

-Darin



>
>
> --
> Anne van Kesteren
> http://annevankesteren.nl/
>

Re: [FileAPI] Blob.URN?

[Darin Fisher]

On Tue, Mar 30, 2010 at 12:22 AM, Jonas Sicking  wrote:

> On Tue, Mar 23, 2010 at 2:23 PM, Darin Fisher  wrote:
> > Apologies if this has been discussed before, but I'm curious why URN is
> not
> > a property of Blob.  It seems like it would be useful to be able to load
> a
> > slice of a File.  For example, this could be used by an application to
> fetch
> > all of its subresources out of a single file.
>
> IIRC originally it was placed on File since Blobs do not have a
> content type. However I think there is general agreement that it
> should be moved to Blob.
>
> However it would be great to be able to assign a content type to a
> Blob. Possibly slice() could take a optional argument.
>
>
Adding an optional parameter to slice() sounds attractive indeed.

BlobBuilder [1] should probably also have such an optional argument.

-Darin

[1]
http://dev.w3.org/2009/dap/file-system/file-writer.html#the-blobbuilder-interface

Re: [FileAPI] Blob.URN?

[Darin Fisher]

The only way to get a FileWriter at the moment is from .  What is desired is a way to simulate the load of a resource
with Content-Disposition: attachment that would trigger the browser's
download manager.

-Darin



On Tue, Mar 30, 2010 at 4:32 PM, Jonas Sicking  wrote:

> What's the use case for specifying the Content-Disposition mime type.
> The ones I've heard so far seems better solved using the FileWriter
> [1] API.
>
> [1] http://dev.w3.org/2009/dap/file-system/file-writer.html
>
> / Jonas
>
> On Tue, Mar 30, 2010 at 4:22 PM, Michael Nordman 
> wrote:
> > There is more than just the mime type when dealing with the URLs.
> > There at least two content headers of interest, Content-Type
> > and Content-Disposition.
> > Whatever mechanism involved should allow for both of these content
> headers
> > to be set
> > by the web application.
> >
> > On Tue, Mar 30, 2010 at 3:07 PM, Jonas Sicking  wrote:
> >>
> >> On Tue, Mar 30, 2010 at 2:57 PM, Darin Fisher 
> wrote:
> >> > On Tue, Mar 30, 2010 at 12:22 AM, Jonas Sicking 
> >> > wrote:
> >> >>
> >> >> On Tue, Mar 23, 2010 at 2:23 PM, Darin Fisher 
> >> >> wrote:
> >> >> > Apologies if this has been discussed before, but I'm curious why
> URN
> >> >> > is
> >> >> > not
> >> >> > a property of Blob.  It seems like it would be useful to be able to
> >> >> > load
> >> >> > a
> >> >> > slice of a File.  For example, this could be used by an application
> >> >> > to
> >> >> > fetch
> >> >> > all of its subresources out of a single file.
> >> >>
> >> >> IIRC originally it was placed on File since Blobs do not have a
> >> >> content type. However I think there is general agreement that it
> >> >> should be moved to Blob.
> >> >>
> >> >> However it would be great to be able to assign a content type to a
> >> >> Blob. Possibly slice() could take a optional argument.
> >> >>
> >> >
> >> > Adding an optional parameter to slice() sounds attractive indeed.
> >> > BlobBuilder [1] should probably also have such an optional argument.
> >>
> >> Indeed!
> >>
> >> / Jonas
> >>
> >
> >
>

Re: [FileAPI] Blob.URN?

[Darin Fisher]

On Wed, Mar 31, 2010 at 3:01 PM, Eric Uhrhane  wrote:

> On Wed, Mar 31, 2010 at 2:40 PM, Jonas Sicking  wrote:
> > On Wed, Mar 31, 2010 at 1:55 AM, Robin Berjon  wrote:
> >> On Mar 31, 2010, at 01:56 , Darin Fisher wrote:
> >>> The only way to get a FileWriter at the moment is from  type="saveas">.  What is desired is a way to simulate the load of a resource
> with Content-Disposition: attachment that would trigger the browser's
> download manager.
> >>
> >> I don't think that  is a good solution for this, for
> one it falls back to a text input control, which is less than ideal. I think
> that the File Writer should trigger downloads on an API call since that
> doesn't introduce security issues that aren't already there. I'll make a
> proposal for that.
> >
> > Why not simply allow FileWriters be instantiated using:
> >
> > x = new FileWriter;
> >
> > And then make every call to .write open up the save as dialog.
>
> You wouldn't want to prompt on every write; developers might want to
> append a chunk at a time.
> You could prompt on creation, if you didn't mind that being a synchronous
> call.
>
> The reason I made an html element be the way of getting a FileWriter
> was to make the normal usage pattern not involve modal dialogs popping
> up in front of the user unprompted.  With an inpput control, they can
> choose when to interact with it, rather than having a speedbump shoved
> in front of them.
>


Please no more JS functions that block on modal dialogs! :-)

@sicking:  It seems odd to vary the behavior of .append based on how the
FileWriter was created.  Perhaps we should provide some other asynchronous
means of creating a FileWriter?

I also find it a little odd that we are overloading  to specify a
save-as
location.  Has the idea of an  tag been discussed?

-Darin

XMLHttpRequest.responseBlob

[Darin Fisher]

There is some interest from application developers at Google in being able
to get a Blob corresponding to the response body of a XMLHttpRequest.

The use case is to improve the efficiency of getting a Blob from a binary
resource downloaded via XHR.

The alternative is to play games with character encodings so that
responseText can be used to fetch an image as a string, and then use
BlobBuilder to reconstruct the image file, again being careful with the
implicit character conversions.  All of this is very inefficient.

Is there any appetite for adding a responseBlob getter on XHR?

Regards,
-Darin

Re: XMLHttpRequest.responseBlob

[Darin Fisher]

On Mon, Apr 26, 2010 at 3:29 PM, Jonas Sicking  wrote:

> On Mon, Apr 26, 2010 at 3:21 PM, Darin Fisher  wrote:
> > There is some interest from application developers at Google in being
> able
> > to get a Blob corresponding to the response body of a XMLHttpRequest.
> > The use case is to improve the efficiency of getting a Blob from a binary
> > resource downloaded via XHR.
> > The alternative is to play games with character encodings so that
> > responseText can be used to fetch an image as a string, and then use
> > BlobBuilder to reconstruct the image file, again being careful with the
> > implicit character conversions.  All of this is very inefficient.
> > Is there any appetite for adding a responseBlob getter on XHR?
>
> There has been talk about exposing a responseBody property which would
> contain the binary response. However ECMAScript is still lacking a
> binary type.
>
> Blob does fit the bill in that it represents binary data, however it's
> asynchronous nature is probably not ideal here, right?
>
> / Jonas
>


I think there are applications that do not require direct access to the
response data.

For example,

1- Download a binary resource (e.g., an image) via XHR.
2- Load the resource using Blob.URN (assuming URN moves from File to Blob).

It may be the case that providing direct access to the response data may be
more
expensive than just providing the application with a handle to the data.
 Consider
the case of large files.

-Darin

Re: XMLHttpRequest.responseBlob

[Darin Fisher]

On Mon, Apr 26, 2010 at 3:52 PM, Jonas Sicking  wrote:

> On Mon, Apr 26, 2010 at 3:39 PM, Darin Fisher  wrote:
> > On Mon, Apr 26, 2010 at 3:29 PM, Jonas Sicking  wrote:
> >>
> >> On Mon, Apr 26, 2010 at 3:21 PM, Darin Fisher 
> wrote:
> >> > There is some interest from application developers at Google in being
> >> > able
> >> > to get a Blob corresponding to the response body of a XMLHttpRequest.
> >> > The use case is to improve the efficiency of getting a Blob from a
> >> > binary
> >> > resource downloaded via XHR.
> >> > The alternative is to play games with character encodings so that
> >> > responseText can be used to fetch an image as a string, and then use
> >> > BlobBuilder to reconstruct the image file, again being careful with
> the
> >> > implicit character conversions.  All of this is very inefficient.
> >> > Is there any appetite for adding a responseBlob getter on XHR?
> >>
> >> There has been talk about exposing a responseBody property which would
> >> contain the binary response. However ECMAScript is still lacking a
> >> binary type.
> >>
> >> Blob does fit the bill in that it represents binary data, however it's
> >> asynchronous nature is probably not ideal here, right?
> >>
> >> / Jonas
> >
> >
> > I think there are applications that do not require direct access to the
> > response data.
> > For example,
> > 1- Download a binary resource (e.g., an image) via XHR.
> > 2- Load the resource using Blob.URN (assuming URN moves from File to
> Blob).
> > It may be the case that providing direct access to the response data may
> be
> > more
> > expensive than just providing the application with a handle to the data.
> >  Consider
> > the case of large files.
>
> Ah, so you want the ability to have the XHR implementation stream to
> disk and then use a Blob to read from there? If so, you need more
> modifications as currently the XHR implementation is required to keep
> the whole response in memory anyway in order to be able to implement
> the .responseText property.
>
> So we'll need to add some way for the page to indicate to the
> implementation "I don't care about the .responseText or .responseXML
> properties, just responseBlob"
>
> / Jonas
>


I thought about this more, and I came to the same conclusion as you.  I
agree that we wouldn't want to support .responseText or .responseXML if we
were streaming directly to a file because the implied synchronous readback
from disk would suck.

I'm not sure how to add such a feature to XHR in a way that is not awkward.
 Perhaps if there was a way to bind a FileWriter to an XMLHttpRequest object
prior to calling send?

-Darin

Re: XMLHttpRequest.responseBlob

[Darin Fisher]

On Tue, Apr 27, 2010 at 10:11 AM, Eric Uhrhane  wrote:

> On Mon, Apr 26, 2010 at 11:03 PM, Darin Fisher  wrote:
> > On Mon, Apr 26, 2010 at 3:52 PM, Jonas Sicking  wrote:
> >>
> >> On Mon, Apr 26, 2010 at 3:39 PM, Darin Fisher 
> wrote:
> >> > On Mon, Apr 26, 2010 at 3:29 PM, Jonas Sicking 
> wrote:
> >> >>
> >> >> On Mon, Apr 26, 2010 at 3:21 PM, Darin Fisher 
> >> >> wrote:
> >> >> > There is some interest from application developers at Google in
> being
> >> >> > able
> >> >> > to get a Blob corresponding to the response body of a
> XMLHttpRequest.
> >> >> > The use case is to improve the efficiency of getting a Blob from a
> >> >> > binary
> >> >> > resource downloaded via XHR.
> >> >> > The alternative is to play games with character encodings so that
> >> >> > responseText can be used to fetch an image as a string, and then
> use
> >> >> > BlobBuilder to reconstruct the image file, again being careful with
> >> >> > the
> >> >> > implicit character conversions.  All of this is very inefficient.
> >> >> > Is there any appetite for adding a responseBlob getter on XHR?
> >> >>
> >> >> There has been talk about exposing a responseBody property which
> would
> >> >> contain the binary response. However ECMAScript is still lacking a
> >> >> binary type.
> >> >>
> >> >> Blob does fit the bill in that it represents binary data, however
> it's
> >> >> asynchronous nature is probably not ideal here, right?
> >> >>
> >> >> / Jonas
> >> >
> >> >
> >> > I think there are applications that do not require direct access to
> the
> >> > response data.
> >> > For example,
> >> > 1- Download a binary resource (e.g., an image) via XHR.
> >> > 2- Load the resource using Blob.URN (assuming URN moves from File to
> >> > Blob).
> >> > It may be the case that providing direct access to the response data
> may
> >> > be
> >> > more
> >> > expensive than just providing the application with a handle to the
> data.
> >> >  Consider
> >> > the case of large files.
> >>
> >> Ah, so you want the ability to have the XHR implementation stream to
> >> disk and then use a Blob to read from there? If so, you need more
> >> modifications as currently the XHR implementation is required to keep
> >> the whole response in memory anyway in order to be able to implement
> >> the .responseText property.
> >>
> >> So we'll need to add some way for the page to indicate to the
> >> implementation "I don't care about the .responseText or .responseXML
> >> properties, just responseBlob"
> >>
> >> / Jonas
> >
> >
> > I thought about this more, and I came to the same conclusion as you.  I
> > agree that we wouldn't want to support .responseText or .responseXML if
> we
> > were streaming directly to a file because the implied synchronous
> readback
> > from disk would suck.
> > I'm not sure how to add such a feature to XHR in a way that is not
> awkward.
> >  Perhaps if there was a way to bind a FileWriter to an XMLHttpRequest
> object
> > prior to calling send?
>
> A FileWriter is quite a bit more limited than a File/Blob.  You can't
> slice it, for instance, and you can't write it to multiple files
> without writing, reading back, and writing again.
>
>

All true, but my suggestion was based on the notion that XHR would be
writing to a disk location in order to provide a Blob to that location.  So,
it might make sense to allow the application to specify the underlying file
location.  That seems to be what FileWriter can be used for.  The
application would presumably have access to the File that the FileWriter
modifies.

This could be useful in conjunction with  to download a
file from the cloud to a location of the user's choice without requiring any
intermediate copy of the entire file.

-Darin

Re: XMLHttpRequest.responseBlob

[Darin Fisher]

On Tue, Apr 27, 2010 at 11:01 AM, Jonas Sicking  wrote:

> On Mon, Apr 26, 2010 at 11:03 PM, Darin Fisher  wrote:
> > On Mon, Apr 26, 2010 at 3:52 PM, Jonas Sicking  wrote:
> >>
> >> On Mon, Apr 26, 2010 at 3:39 PM, Darin Fisher 
> wrote:
> >> > On Mon, Apr 26, 2010 at 3:29 PM, Jonas Sicking 
> wrote:
> >> >>
> >> >> On Mon, Apr 26, 2010 at 3:21 PM, Darin Fisher 
> >> >> wrote:
> >> >> > There is some interest from application developers at Google in
> being
> >> >> > able
> >> >> > to get a Blob corresponding to the response body of a
> XMLHttpRequest.
> >> >> > The use case is to improve the efficiency of getting a Blob from a
> >> >> > binary
> >> >> > resource downloaded via XHR.
> >> >> > The alternative is to play games with character encodings so that
> >> >> > responseText can be used to fetch an image as a string, and then
> use
> >> >> > BlobBuilder to reconstruct the image file, again being careful with
> >> >> > the
> >> >> > implicit character conversions.  All of this is very inefficient.
> >> >> > Is there any appetite for adding a responseBlob getter on XHR?
> >> >>
> >> >> There has been talk about exposing a responseBody property which
> would
> >> >> contain the binary response. However ECMAScript is still lacking a
> >> >> binary type.
> >> >>
> >> >> Blob does fit the bill in that it represents binary data, however
> it's
> >> >> asynchronous nature is probably not ideal here, right?
> >> >>
> >> >> / Jonas
> >> >
> >> >
> >> > I think there are applications that do not require direct access to
> the
> >> > response data.
> >> > For example,
> >> > 1- Download a binary resource (e.g., an image) via XHR.
> >> > 2- Load the resource using Blob.URN (assuming URN moves from File to
> >> > Blob).
> >> > It may be the case that providing direct access to the response data
> may
> >> > be
> >> > more
> >> > expensive than just providing the application with a handle to the
> data.
> >> >  Consider
> >> > the case of large files.
> >>
> >> Ah, so you want the ability to have the XHR implementation stream to
> >> disk and then use a Blob to read from there? If so, you need more
> >> modifications as currently the XHR implementation is required to keep
> >> the whole response in memory anyway in order to be able to implement
> >> the .responseText property.
> >>
> >> So we'll need to add some way for the page to indicate to the
> >> implementation "I don't care about the .responseText or .responseXML
> >> properties, just responseBlob"
> >
> > I thought about this more, and I came to the same conclusion as you.  I
> > agree that we wouldn't want to support .responseText or .responseXML if
> we
> > were streaming directly to a file because the implied synchronous
> readback
> > from disk would suck.
> > I'm not sure how to add such a feature to XHR in a way that is not
> awkward.
> >  Perhaps if there was a way to bind a FileWriter to an XMLHttpRequest
> object
> > prior to calling send?
>
> Hmm.. what would that look like? Can you give an example of an API?
>
> I've been thinking for a while that we should add a 'streaming mode'
> for XHR anyway. Right now XHR is very inefficient when loading large
> amounts of data for two reasons. First of all the data needs to be
> appended to an existing buffer constantly, resulting in O(n^2)
> behavior. I.e. if you receive data in 1500 bytes packets, and download
> 1Mb of data, you end up first allocating 1500 bytes, then 3000, 4500,
> 6000, etc. All the way to the total file size. Second, as discussed
> here, you have to store the whole resulting file in memory.
>

A streaming mode would be excellent!



>
> It would be nice to be able to allow streaming such that every time a
> progress event is fired only the newly downloaded data is available.
> The UA is then free to throw away that data once the event is done
> firing. This would be useful in the cases when the page is able to do
> incremental parsing of the resulting document.
>
> If we add a 'load mode' flag on XMLHttpRequest, which can't be
> modified after send() is called, then streaming to a Blob c

Re: XMLHttpRequest.responseBlob

[Darin Fisher]

On Tue, Apr 27, 2010 at 1:33 PM, Jonas Sicking  wrote:

> On Tue, Apr 27, 2010 at 1:26 PM, Darin Fisher  wrote:
> >> It would be nice to be able to allow streaming such that every time a
> >> progress event is fired only the newly downloaded data is available.
> >> The UA is then free to throw away that data once the event is done
> >> firing. This would be useful in the cases when the page is able to do
> >> incremental parsing of the resulting document.
> >>
> >> If we add a 'load mode' flag on XMLHttpRequest, which can't be
> >> modified after send() is called, then streaming to a Blob could simply
> >> be another enum value for such a flag.
> >>
> >> There is still the problem of how the actual blob works. I.e. does
> >> .responseBlob return a new blob every time more data is returned? Or
> >> should the same Blob be constantly modifying? If modifying, what
> >> happens to any in-progress reads when the file is modified? Or do you
> >> just make the Blob available once the whole resource has been
> >> downloaded?
> >>
> >
> >
> > This is why I suggested using FileWriter.  FileWriter already has to
> > deal with
> > most of the problems you mentioned above,
>
> Actually, as far as I can tell FileWriter is write-only so it doesn't
> deal with any of the problems above.
>

When you use createWriter, you are creating a FileWriter to an existing
File.
The user could attempt to create a FileReader to the very same File while
a FileWriter is open to it.

It is true that for  there is no way to get at the
underlying
File object.  That is perhaps a good thing for the use case of downloading
to
a location specified by the user.



>
> > and you can get a FileWriter
> > either from  [1] or from a FileEntry object [2].
> > One possible API for engaging this mode might be:
> > var x = new XMLHttpRequest();
> > x.open("GET", url, true);
> > x.streamToFile(fileWriter);
> > x.send(null);
>
> And how would you get the actual data?
>
>
See above.

-Darin

Re: XMLHttpRequest.responseBlob

[Darin Fisher]

On Tue, Apr 27, 2010 at 2:04 PM, Jonas Sicking  wrote:

> On Tue, Apr 27, 2010 at 1:59 PM, Darin Fisher  wrote:
> > On Tue, Apr 27, 2010 at 1:33 PM, Jonas Sicking  wrote:
> >>
> >> On Tue, Apr 27, 2010 at 1:26 PM, Darin Fisher 
> wrote:
> >> >> It would be nice to be able to allow streaming such that every time a
> >> >> progress event is fired only the newly downloaded data is available.
> >> >> The UA is then free to throw away that data once the event is done
> >> >> firing. This would be useful in the cases when the page is able to do
> >> >> incremental parsing of the resulting document.
> >> >>
> >> >> If we add a 'load mode' flag on XMLHttpRequest, which can't be
> >> >> modified after send() is called, then streaming to a Blob could
> simply
> >> >> be another enum value for such a flag.
> >> >>
> >> >> There is still the problem of how the actual blob works. I.e. does
> >> >> .responseBlob return a new blob every time more data is returned? Or
> >> >> should the same Blob be constantly modifying? If modifying, what
> >> >> happens to any in-progress reads when the file is modified? Or do you
> >> >> just make the Blob available once the whole resource has been
> >> >> downloaded?
> >> >>
> >> >
> >> >
> >> > This is why I suggested using FileWriter.  FileWriter already has to
> >> > deal with
> >> > most of the problems you mentioned above,
> >>
> >> Actually, as far as I can tell FileWriter is write-only so it doesn't
> >> deal with any of the problems above.
> >
> > When you use createWriter, you are creating a FileWriter to an existing
> > File.
> > The user could attempt to create a FileReader to the very same File while
> > a FileWriter is open to it.
> > It is true that for  there is no way to get at the
> > underlying
> > File object.  That is perhaps a good thing for the use case of
> downloading
> > to
> > a location specified by the user.
>
> Ah. But as far as I can tell (and remember), it's still fairly
> undefined what happens when the OS file under a File/Blob object is
> mutated.
>
> / Jonas
>

Agreed.  I don't see it as a big problem.  Do you?  The application
developer is
in control.  They get to specify the output file (via FileWriter) that XHR
sends its
output to, and they get to know when XHR is done writing.  So, the
application
developer can avoid reading from the file until XHR is done writing.

-Darin