Re: [fileapi] Pull Request on GitHub

[Arun Ranganathan]

I won't be editing it either.

On Tue, Aug 16, 2016 at 4:44 AM, Marcos Caceres  wrote:

> On August 16, 2016 at 6:31:31 PM, Zhen Zhang (izgz...@gmail.com) wrote:
> > Hi,
> >
> > I have a PR on GitHub regarding some issues of wording in current File
> API spec: https://github.com/w3c/FileAPI/pull/42
> > , but nobody ever responded me there.
> > I wonder if I should discuss the patch somewhere else?
>
> It seems that no one has touched that API for about 8 months.
>
> Marijn, are you still editing that document? I guess Jonas won't be,
> but not sure about Arun.
>
>

Re: Directory Upload Proposal

[Arun Ranganathan]

On Fri, Apr 24, 2015 at 2:28 AM, Ali Alabbas a...@microsoft.com wrote:


 If there is sufficient interest, I would like to work on this within the
 scope of the WebApps working group.


And I'll help with the FileSystem API bit, ensuring that full spec.[3]
has bits about the Directory Upload proposal (outside of the sandbox).
There's an HTML WG bit, too, as Jonas and Anne v K point out. I'm not sure
what the best way to tackle that is, but I think a bug on HTML plus nudging
over IRC over attribute names will go a fair distance.

-- A*

flush() | was Re: FileSystem API Comments

[Arun Ranganathan]

Greetings Ali!

I’ve been thinking about the discussion of flush(), and would like to see if I 
can make my previous statement a bit more nuanced. It turns out that flush() 
(in the vein of fsync/sync) is pretty useful, and after discussion with a few 
folks within Mozilla, I realize that it isn’t as simple as tacking it on to the 
“write-family” of Promises — as you point out, it is a potentially expensive 
operation.

Something like a flush feature might help the following use cases:

1. Creating a database technology on top of the filesystem technology. This 
might include IndexedDB, but also WebSQL (as a hypothetical example). Most 
transactional operations like this need the ability to do something like flush.

2. Then, there’s the use case of compiling C++ codebases to JS. Well-known 
examples of this are games, leveraging asm.js. In this genre of use case, 
sometimes a large database is brought over (e.g. sqlite). It could be memory 
backed, but it is a definite bonus if it could be filesystem backed. Something 
like flush helps make that a possibility.

Now the question is how to do this in a WebAPI, allowing for the power along 
with the mitigations that a web app might need, notably for performance? A few 
ideas below:

On Oct 21, 2014, at 4:36 PM, Ali Alabbas a...@microsoft.com wrote:

  * flush()
  - This is costly functionality to expose and is likely to be overused by 
 callers. It would be beneficial to automatically flush changes to disk by 
 allowing the default file write behavior by the OS. For example, on Windows, 
 we would leave it up to the filesystem cache to determine the best time to 
 flush to disk. This is non-deterministic from the app's point of view, but 
 the only time it is a potential problem is when there's a hard power-off. 
 Most apps should not be concerned with this; only apps that have very high 
 data reliability requirements would need the granular control of flushing to 
 disk. In those cases a developer should use IndexedDB. So we should consider 
 obscuring this functionality since it's not a common requirement and has a 
 performance impact if it's widely used.


I agree with the idea of obscuring the functionality a bit, especially given 
that it might not be necessary for a large class of operations. A few ways to 
do that:

1. Add this to a dictionary option when coining the FileHandleWritable from the 
Directory (e.g. add it to something like the OpeWriteOptions: 
http://w3c.github.io/filesystem-api/Overview.html#widl-Directory-openWrite-Promise-FileHandleWritable--DOMString-File-path-OpenWriteOptions-options).

This way, the developer has the ability to “coin” a “more expensive” promise, 
if that particular set of write operations needs this feature.

2. Add this to the set of options on the FileHandleWritable.

This could be by dictionary, again. Or, it could be a boolean on the 
FileHandleWritable’s write(). This latter might not be specific enough. Like 
other implementations, ours is not going to buffer anything, but rely on the 
underlying operating system’s buffer for writes and reads.

3. Stick with the idea of a method, like flush(). In this case, we might have 
to caveat the use of this, since the possibility of inexperienced developer 
misuse is high :-) It might help to see if we can determine some boundaries on 
this.

Any feedback on some of these options would be valuable. I am thinking of 1. 
and 2.

— A*

FileAPI | Programmatic Triggering of Downloads

[Arun Ranganathan]

WebAppsWG,

At the recent F2F, I received an ACTION to strike one of the use cases from the 
File API, since the File API itself did not deliver on that use case.

This is ACTION 750: https://www.w3.org/2008/webapps/track/actions/750 
which asks me to remove the use case, since there isn’t a mechanism to trigger 
programmatic downloads to the underlying OS filesystem.

But upon reflection, although FileAPI itself hasn’t provided any API surface 
for this, the recently finished HTML specification has given us something 
useful, but not implemented widely yet:
 
http://www.w3.org/TR/html5/links.html#attr-hyperlink-download

The use case itself calls for things such as storing assets for later use, 
including for use with third-party apps such as Calendar apps, which I’ve 
demonstrated using the Blob constructor and the “download” feature, since not 
everyone present at the meeting knew you could do this:

http://aruner.net/resources/BlobURL/datePickerForCal.html

In Firefox and Chrome (desktop) it works by triggering a download and invoking 
the Calendar application on your particular OS (Outlook 2013 on Windows or 
Calendar on Mac OS X). Safari shows the iCalendar entry “in the raw” and 
although Safari on iOS allows images that are Blob URLs to be downloaded, it 
doesn’t allow much to be done with other data types that are blob: URLs, such 
as iCalendar entries.

I don’t think I should remove the use case, but instead said that the actual 
requirement was handled in HTML, even if implementation status isn’t at 100%.

The File API provides additional guidance for File objects:

http://dev.w3.org/2006/webapi/FileAPI/#processing-media-types


— A*

Re: [FileAPI] Seeking status and plans [Was: [TPAC2014] Creating focused agenda]

[Arun Ranganathan]

On Oct 22, 2014, at 8:05 AM, Arthur Barstow art.bars...@gmail.com wrote:

 * File API: Arun and Jonas; which v1 bugs are blocking a new LC; what are 
 next steps; timeline for LC.
 
 Arun, Jonas,
 
 Please see the above and respond accordingly. I am especially interested in 
 the File API status but please also include a summary of your plans for the 
 the Filesystem API.



1. File API: happy to move towards LC. Only active bug is a platform bug:

https://www.w3.org/Bugs/Public/show_bug.cgi?id=26603

which applies to FileAPI’s use of the incumbent settings object for origin 
determination of Blob URLs.

2. FileSystem API: http://w3c.github.io/filesystem-api/Overview.html

Plan: get feedback from implementors, fix the spec and resolve the spec.’s 
issues. Thanks to Ali Alabbas from the IE Platform Team for some early feedback.

— A*

Re: FileSystem API Comments

[Arun Ranganathan]

Ali,

First, thanks for your timely comments :) I’m in the process of editing the 
FileSystem API.

Responses inline:

On Oct 21, 2014, at 4:36 PM, Ali Alabbas a...@microsoft.com wrote:

  
 1.1 Use cases (3. Audio/Photo editor with offline access or local cache for 
 speed)
  
   * Edited files should be accessible by other client-side applications
  - Having the sandboxed file system share its contents between all apps 
 would allow apps to tamper with the files of another app. 


snip /

Admitedly, these use cases have been borrowed from the “File API: Directories 
and System” specification (which is now a W3C Note), at least for the purpose 
of providing equivalent functionality. In practice, everything you’ve pointed 
out makes it a hard problem to solve. 

The per-origin sandbox model also raises file lock issues on multiple access, 
but they are probably easier to solve, and not as prevalent.

Also, we’re going to forego the “temporary” and “persistent” distinctions that 
are in the draft I think. And while there’s a technical dependency on Quota 
Manager, I don’t think there’s a spec. dependency in terms of API. Of course, 
certain Directory operations may reject a promise with a quota error.


  
  
 3. The Directory Interface
  
   * Change events
  - I would like to revisit the discussion on apps getting notifications 
 of changes to files/directories. 


This is a good point; right now there’s no way to do this. I’m open to 
suggestions. An early scratch pad version of my spec. changes proposed 
Directory as an EventTarget also, but… this won’t work for a variety of reasons.



  
   * removeDeep()  move()
  - Do these support links or junctions? If not, what is the expected 
 behavior?


No; but the entire API doesn’t support these right now.


  
   * enumerate()
  - It would be useful to have pre-filtering support for the following: 
 file/directory, ranges, wildcard search. Currently we would be forced to 
 enumerate the entire set and manually filter out the items we want from the 
 result set.


I completely agree this would be useful, but there’s a problem to solve even 
before we get there! Right now, we say that we’ll fullfill the enumerate 
promise with something called “EventStream” which was initially a Tab Atkins 
proposal, and which would be really useful to get right. It’s underspecified 
right now, but I’m a fan of it ;-)

We’ll have to think about how to return wildcard searches, etc., and how to 
annotate results in the result set.


  
 4. The FileHandle Interface
  
   * FileHandles
  - Is this basically going to be the first to get the handle gets to use 
 it and all subsequent calls need to wait for the file handle to become 
 available again? Are there more details about the locking model used here?


Yes; essentially the “first invocation” uses then releases it.

A version of this problem was encountered when specifying FileReader 
(http://dev.w3.org/2006/webapi/FileAPI/#dfn-filereader) which used the internal 
state (but also accessible to the developer) “LOADING” to prevent multiple 
concurrent reads.


  
   * Auto-closing of FileHandles
  - This may cause confusion as it does not match the common developer 
 mental model of a file handle which is “opened” and then available for use 
 until it's “closed”. Perhaps it would be advantageous to have an explicit 
 close function as part of the FileHandle interface? 


There are pros and cons either way. I’d be interested in solving this for the 
lion’s share of use cases. I’m not strongly opinionated on the matter of an 
explicit close function (we have one on Blob, for example), but it seems even 
this has drawbacks.


   * AbortableProgressPromise
  - It is not clear how a developer would define the abort callback of an 
 AbortableProgressPromise. It seems that the user agent would be responsible 
 for setting the abort callback since it instantiates and returns the 
 AbortableProgressPromise.


We’re going to not use an AbortableProgressPromise, but we will probably have a 
new beast called CancelablePromise.


  
  
 5. The FileHandleWritable Interface
  
   * write()  flush()
  - It might be useful to have support for “transacted” streams where the 
 caller can write to a copy of the file and then have it atomically replaced: 
 swap the old file with the new one and then delete the old file. 


Agreed.


  
   * flush()
  - This is costly functionality to expose and is likely to be overused by 
 callers. 


Agreed — let’s flush flush().


 6. FileSystem Configuration Parameters
  
   * Dictionary DestinationDict
  - The DestinationDict seems to exist to facilitate the renaming of a 
 directory in conjunction with a move. However, the same operation is done 
 differently for files which makes the functionality non-uniform. Perhaps we 
 can add a rename() function to make it more intuitive?
  


I’ll commit to sample code and more “spec text” to make this clearer in my next 

Re: File API: reading a Blob

[Arun Ranganathan]

On Aug 11, 2014, at 7:24 AM, Anne van Kesteren ann...@annevk.nl wrote:

 
 Other than “chunks of bytes” which needs some normative backbone, is the 
 basic abstract model what you had in mind? If so, that might be worth 
 getting into File APi and calling it done, because that’s a reusable 
 abstract model for Fetch and for FileSystem.
 
 Yeah that looks good. https://whatwg.github.io/streams/ defines chunks
 and such, but is not quite there yet. But it is what we want to build
 this upon eventually.


I’ve fixed the model according to these discussions:

1. We’ve separated the read operation and the “annotated task read operation.” 
The former uses Fetch’s notion of a body, which is a byte stream, and so is 
convenient to work with. Hopefully, it should allow plugging right back into 
Fetch.

2. The methods now invoke an annotated task read operation.

http://dev.w3.org/2006/webapi/FileAPI/#reading-data-section

— A*

Re: File API: reading a Blob

[Arun Ranganathan]

On Sep 3, 2014, at 6:02 PM, Aymeric Vitte vitteayme...@gmail.com wrote:

 
 The fact is that most of the W3C groups impacted by streams (File, indexedDB, 
 MSE, WebRTC, WebCrypto, Workers, XHR, WebSockets, Media Stream, etc, I must 
 forget a lot here) seem not to care a lot about it and maybe just expect 
 streams to land in the right place in the APIs when they are available, by 
 some unknown magic.


I care about it. Till the API is totally baked, I’m amenable to getting the 
model right. File API now refers to chunks read, which is more correct. But I 
understand that your use cases aren’t catered to just yet; 
FileReader/FileReaderSync don’t do easily extractable partials.

I’d like to see if there’s interest in the earlier proposal, to extract a 
stream straight from Blob.


 
 I still think that the effort should start from now for all the APIs (as well 
 as the implementation inside browsers, which apparently has started for 
 Chrome, but Chrome was supposed to have started some implementation of the 
 previous Streams APIs, so it's not very clear), and that it should be very 
 clearly synchronized, disregarding vague assumptions from the groups about 
 low/high level and Vx releases, eluding the issue.


What issue is being eluded? Seems like another of your main use cases is to 
have URL.createObjectURL or URL.createFor return a streamable resource. I agree 
that’s a good problem to solve. 

— A*

Re: File API: reading a Blob

[Arun Ranganathan]

Welcome back - we missed you :-)


On Aug 5, 2014, at 9:43 AM, Anne van Kesteren ann...@annevk.nl wrote:

 On Thu, Jul 17, 2014 at 2:58 PM, Arun Ranganathan a...@mozilla.com wrote:
 There are two questions:
 
 1. How should FileReaderSync behave, to solve the majority of use cases?
 2. What is a useful underlying abstraction for spec. authors that can be 
 reused in present APIs like Fetch and future APIs?
 
 I'm not sure.


I strongly think we should leave FileReaderSync and FileReader alone. Also note 
that FileReaderSync and XHR (sync) are not different, in that both don’t do 
partial data. But we should have a stream api that evolves to read, and it 
might be something off Blob itself.

That leaves us the next problem, and what I think is the last problem in File 
API:



 Yeah, I now think that we want something even lower-level and build
 the task queuing primitive on top of that. (Basically by observing the
 stream that is being read and queuing tasks as data comes in, similar
 to Fetch. The synchronous case would just wait for the stream to
 complete.
 
 If I understand you correctly, you mean something that might be two-part 
 (some hand waving below, but …):
 
 To read a Blob object /blob/, run these steps:
 
 1. Let /s/ be a new buffer.
 
 2. Return /s/, but continue running these steps asynchronously.
 
 3. While /blob/'s underlying data stream is not closed, run these
substeps:
 
1. Let /bytes/ be the result of reading a chunk from /blob/'s
   underlying data.
 
2. If /bytes/ is not failure, push /bytes/ to /s/ and set
   /s/'s transmitted to /bytes/'s length.
 
3. Otherwise, signal some kind of error to /s/ and terminate
   these steps.
 
 AND
 
 To read a Blob object with tasks:
 
 1. Run the read a Blob algorithm above.
 2. When reading the first /bytes/ queue a task called process read.
 3. When pushing /bytes/ to /s/, queue a task called process read data.
 4. When all /bytes/ are pushed to /s/ queue a task called process read EOF.
 5. If an error condition is signaled queue a task called process error with 
 a failure reason.
 
 Is “chunk” implementation defined? Right now we assume 1 byte or 50ms. 
 “Chunk” seems a bit hand-wavy and hard to enforce, but… it might be the 
 right approach.
 
 Would have to discuss with Domenic, but something like chunks seems to
 be much closer to how these things actually work.


Other than “chunks of bytes” which needs some normative backbone, is the basic 
abstract model what you had in mind? If so, that might be worth getting into 
File APi and calling it done, because that’s a reusable abstract model for 
Fetch and for FileSystem.

— A*

Re: File API: reading a Blob

[Arun Ranganathan]

Aymeric,

(Sorry for the delay; I was traveling for the last part of July.) 

On Jul 17, 2014, at 11:25 AM, Aymeric Vitte vitteayme...@gmail.com wrote:

 I don't get this, most reads inside browsers are about fetching, and fetching 
 does increment a resource while it is loaded/displayed (image, video, crypto 
 operations, etc)
 
 And it's question of the fetch API in this thread.
 
 XHR has the same issue, fortunately you can use XHR with Range, now I think 
 this issue should stop propagating


What I meant is that while I don’t think FileReader *or* FileReaderSync should 
be modified *at this stage* to include partial data, there is a workaround:

1. After obtaining the Blob or File, use .slice() to generate something like a 
mySlices[ ] array of e.g. 2K each slice.

2. Then use fileReader.readAsArrayBuffer(mySlices[0]…). 

This should be relatively efficient, and in fact can be cued to work at the 
completion of the last read, based on the final event being fired (or something 
similar — you get the idea). This might be similar to XHR range.

This at least allows chunked reads. I agree that we need a stream-based API, 
but I don’t agree that it should be FileReader or FileReaderSync. That’s simply 
doing too much with too little. However, I think the next step would be to 
generate stream-based reads directly off the Blob, e.g. have something like 
myBlob.read(…) which returns a stream, which can be cued with API calls.


 1. Decoding was an issue with *readAsText*. I suppose we could make that 
 method alone be all or nothing.
 I don't see where the issue can be, it should behave like textdecoder and 
 reinject the undecoded bytes to the next chunk, personnaly I find really 
 useless to offer the 'text' methods in files/fetching APIs.


The issue is that it isn’t possible to know the total number of decoded bytes 
is till the whole file is decoded, but of course we could only represent the 
bytes read thus far in progress events. Again, I don’t think FileReader or 
FileReaderSync are the right APIs for that.  

— A*

Re: File API: reading a Blob

[Arun Ranganathan]

Aymeric,

(Sorry for the delay; I was traveling for the last part of July.) 

On Jul 17, 2014, at 11:25 AM, Aymeric Vitte vitteayme...@gmail.com wrote:

 I don't get this, most reads inside browsers are about fetching, and fetching 
 does increment a resource while it is loaded/displayed (image, video, crypto 
 operations, etc)
 
 And it's question of the fetch API in this thread.
 
 XHR has the same issue, fortunately you can use XHR with Range, now I think 
 this issue should stop propagating


What I meant is that while I don’t think FileReader *or* FileReaderSync should 
be modified *at this stage* to include partial data, there is a workaround:

1. After obtaining the Blob or File, use .slice() to generate something like a 
mySlices[ ] array of e.g. 2K each slice.

2. Then use fileReader.readAsArrayBuffer(mySlices[0]…). 

This should be relatively efficient, and in fact can be cued to work at the 
completion of the last read, based on the final event being fired (or something 
similar — you get the idea). This might be similar to XHR range.

This at least allows chunked reads. I agree that we need a stream-based API, 
but I don’t agree that it should be FileReader or FileReaderSync. That’s simply 
doing too much with too little. However, I think the next step would be to 
generate stream-based reads directly off the Blob, e.g. have something like 
myBlob.read(…) which returns a stream, which can be cued with API calls.


 1. Decoding was an issue with *readAsText*. I suppose we could make that 
 method alone be all or nothing.
 I don't see where the issue can be, it should behave like textdecoder and 
 reinject the undecoded bytes to the next chunk, personnaly I find really 
 useless to offer the 'text' methods in files/fetching APIs.


The issue is that it isn’t possible to know the total number of decoded bytes 
is till the whole file is decoded, but of course we could only represent the 
bytes read thus far in progress events. Again, I don’t think FileReader or 
FileReaderSync are the right APIs for that.  

— A*

Re: File API: reading a Blob

[Arun Ranganathan]

On Jul 14, 2014, at 3:47 AM, Anne van Kesteren ann...@annevk.nl wrote:

 On Thu, Jul 10, 2014 at 7:05 PM, Arun Ranganathan a...@mozilla.com wrote:
 On Jul 3, 2014, at 10:50 AM, Anne van Kesteren ann...@annevk.nl wrote:
 That would mean you would get different results between using
 FileReaderSync and XMLHttpRequest. That does not seem ideal
 
 The implementation train has already left the station on this. The 
 movitation of an “ideal match-up with XMLHttpRequest doesn’t seem strong 
 enough to revisit this by filing browser bugs across implementations (but 
 Cc’ing K. Huey also).
 
 Well, surely if we support both, we'd like them to work in the same
 way so they can share the same underlying abstraction.


There are two questions:

1. How should FileReaderSync behave, to solve the majority of use cases?
2. What is a useful underlying abstraction for spec. authors that can be reused 
in present APIs like Fetch and future APIs?

I don’t think it is necessary to mix the two questions for APIs that are 
already shipping. Do you think that FileReaderSync AND FileReader should 
support partial Blob data in read results? Or that stream-based reads should do 
this with a different API?


 
 We agreed some time ago to not have partial data.
 
 Pointer? I also don't really see how that makes sense given how
 asynchronous read would perform.


Well, the bug that removed them is: 
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23158 and dates to last year.

Problems really include decoding strings according to the encoding 
determination for incomplete Blobs:

http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/0063.html

Another thread covered deltas in progress events:

http://lists.w3.org/Archives/Public/public-webapps/2013JanMar/0069.html

I don’t have pointers to IRC conversations, but:

1. Decoding was an issue with *readAsText*. I suppose we could make that method 
alone be all or nothing.

2. Use cases. MOST reads are useful with all data, after which you could use 
Blob manipulation (e.g. *slice*). New result objects each time didn’t seem 
optimal. And, it was something punted to Streams since that seemed like a 
longer term direction. There was also the idea of a Promise-based File API that 
could be consumed by the FileSystem API.

But it might be useful to have an abstract read that helps Fetch and other 
things like Streams also:


 Yeah, I now think that we want something even lower-level and build
 the task queuing primitive on top of that. (Basically by observing the
 stream that is being read and queuing tasks as data comes in, similar
 to Fetch. The synchronous case would just wait for the stream to
 complete.



If I understand you correctly, you mean something that might be two-part (some 
hand waving below, but …):

 To read a Blob object /blob/, run these steps:

 1. Let /s/ be a new buffer. 

 2. Return /s/, but continue running these steps asynchronously.

 3. While /blob/'s underlying data stream is not closed, run these
substeps:

1. Let /bytes/ be the result of reading a chunk from /blob/'s
   underlying data.

2. If /bytes/ is not failure, push /bytes/ to /s/ and set
   /s/'s transmitted to /bytes/'s length.

3. Otherwise, signal some kind of error to /s/ and terminate
   these steps.

AND

To read a Blob object with tasks:

1. Run the read a Blob algorithm above.
2. When reading the first /bytes/ queue a task called process read.
3. When pushing /bytes/ to /s/, queue a task called process read data.
4. When all /bytes/ are pushed to /s/ queue a task called process read EOF.
5. If an error condition is signaled queue a task called process error with a 
failure reason.

Is “chunk” implementation defined? Right now we assume 1 byte or 50ms. “Chunk” 
seems a bit hand-wavy and hard to enforce, but… it might be the right approach.

— A*

Re: File API: reading a Blob

[Arun Ranganathan]

Aymeric,

On Jul 16, 2014, at 8:20 AM, Aymeric Vitte vitteayme...@gmail.com wrote:

 Example:
 var myfile=new Blob();
 
 //chunks are coming in
 myfile=new Blob([myfile,chunk],...)
 
 //mylink A tag
 mylink.href=URL.createObjectURL(myfile)
 
 click on mylink -- does not work
 
 Expected behavior: the file (a video for example) should play as it is 
 incremented.
 
 This is inconsistent with the standard files behavior (see [1] for example), 
 this example should work without having to use the Media Source Extensions 
 API.



This is a great use case, but breaks File API currently (and the abstract 
model, too).  

Options to consider might be:

1. Including partial Blob data with the existing FIle API. But we already ship 
as is, so we could forego decoding to get this right, but that doesn’t seem 
optimal either. 

2. Use a new API for this, such as Streams. Since we’ll use streams for 
non-file cases, it could be useful here (and might be a better fit for the use 
case, which features heavy video use).

— A*

Re: File API: Blob URL origin

[Arun Ranganathan]

On Jun 30, 2014, at 7:13 PM, Glenn Maynard gl...@zewt.org wrote:

 Why would the identifier not just be the blob URL itself?  


Done.

 
 Also, both Chrome and Firefox treat the entire URL as case-sensitive, eg. 
 Blob:... won't revoke the URL, or uppercasing the hostname portion in 
 Chrome.  Using the whole URL as the identifier makes this easy to do.
 
 Subsequent attempts to dereference url must return a network error should 
 be removed.  That should already be the consequence of unregistering the URL, 
 so this is a redundant requiremen


Done: http://dev.w3.org/2006/webapi/FileAPI/

— A*

Re: File API: reading a Blob

[Arun Ranganathan]

On Jul 3, 2014, at 10:50 AM, Anne van Kesteren ann...@annevk.nl wrote:

 On Thu, Jul 3, 2014 at 4:29 PM, Arun Ranganathan a...@mozilla.com wrote:
 OK, this is fixable. I’ll ensure that the read operation’s synchronous
 component does return the results thus far, but that FIleReaderSync itself
 ignores them in case of a midstream error, unless we collectively agree that
 it SHOULD return partial instead of “all or nothing” as an API. My
 understanding of the FileReaderSync requirement is all or nothing, but I’m
 open to being wrong via bug filing.
 
 That would mean you would get different results between using
 FileReaderSync and XMLHttpRequest. That does not seem ideal



The implementation train has already left the station on this. The movitation 
of an “ideal match-up with XMLHttpRequest doesn’t seem strong enough to 
revisit this by filing browser bugs across implementations (but Cc’ing K. Huey 
also).

We agreed some time ago to not have partial data.


 
 The tasks are still a bit of a concern as a normal implementation
 would not queue tasks. E.g. it's not even clear what task loop Fetch
 would retrieve this from as fetch is itself a different process.


Then let’s have a different read operation that doesn’t use the FileReader Task 
Source. The primary API in File API uses the event loop (FileReader), so for 
that purpose, the existing read operation is important. Early discussions about 
this made me feel that a task queue based system to asynchronously read blobs 
could be reusable by Fetch, and by future Promise-based APIs. 

Since it’s not reusable for Fetch, let’s not try and force it. We’ll do a new 
one along the lines you described at the start of this email.

— A*

Re: File API: reading a Blob

[Arun Ranganathan]

On Jul 3, 2014, at 4:14 AM, Anne van Kesteren ann...@annevk.nl wrote:

 It's unclear to me why we'd want to use the event loop for this,
 basically.


The FileReader object uses the event loop; your initial request for Fetch was 
to have a reusable “abstract” read operation which used tasks. You’ve since 
changed your mind, which is totally fine: we could have a different read 
operation that doesn’t use the event loop that’s put in place for Fetch, but 
FileReader needs the event loop. 

We’re talking about an abstract model in specification land on which to pin 
higher level concepts that culminate eventually in JS objects. It’s useful (I 
agreed long ago), but overhauling the current read operation for a change of 
mind/model is a lot of pain without elegant gain. 

Also, since there isn’t an observable stream or an object, but merely things 
like load progression (JPEG progression), tasks does seem useful for that. The 
one thing I suggested which we could do better is the “one byte vs. 50ms” 
model, and use the “chunk” concept of bytes that the streams folks use. The one 
thing I’m not clear on here is how to get to a pull-chunk size for files, but I 
think we could do this better.


 Also, the current set of synchronous steps (which I could
 opt not to use, granted, but other APIs already might, and I'd like us
 to be consistent) simply return failure when something bad happens
 rather than returning the bytes read so far. It seems like that is a
 problem, perhaps following from not having this lower-level
 description.



OK, this could be a problem. But this is really immediately usable by the 
FileReaderSync object on threads, for which a use case for partial data didn’t 
materialize (and in general, the spec. shunned partial data — references to 
those threads date way back in time, but they’re there). It seems that for a 
synchronous block of file i/o, all or nothing catered to most use cases.

But if it IS a problem — that is, if you think synchronous I/O has implications 
outside of FileReaderSync, OR that FileReaderSync’s return itself should be 
partial if a failure occurs, then let’s file bugs and solve them.

I’d REALLY like to have solid abstract models in place for Fetch, since I buy 
into the cross-purposability of it. But I’d also like shipping implementations 
to be defined (99% done in File API), with the small delta remaining — Blob URL 
autorevoking and Blob closing — to be nailed down.

— A*

Re: File API: reading a Blob

[Arun Ranganathan]

On Jul 3, 2014, at 10:17 AM, Anne van Kesteren ann...@annevk.nl wrote:

 So most of Fetch is asynchronous. If it's invoked with the synchronous
 flag set it's just that it waits for the entire response before
 returning. That's why I'd like to use the asynchronous path of reading
 a blob. But I'd like that not to be observably different from using
 the synchronous path of reading a blob. That seems wrong.



OK, this is fixable. I’ll ensure that the read operation’s synchronous 
component does return the results thus far, but that FIleReaderSync itself 
ignores them in case of a midstream error, unless we collectively agree that it 
SHOULD return partial instead of “all or nothing” as an API. My understanding 
of the FileReaderSync requirement is all or nothing, but I’m open to being 
wrong via bug filing.

Are you agreed (as far as it is possible for you to agree with me on anything) 
that the event loop async read might allow us to address cases like JPEG 
progression? It seems imminently usable here.

— A*

Re: File API: reading a Blob

[Arun Ranganathan]

On Jul 2, 2014, at 10:28 AM, Anne van Kesteren ann...@annevk.nl wrote:

 So what I need is something like this:
 
  To read a Blob object /blob/, run these steps:
 
  1. Let /s/ be a new body. [FETCH]
 
  2. Return /s/, but continue running these steps asynchronously.
 
  3. While /blob/'s underlying data stream is not closed, run these
 substeps:
 
 1. Let /bytes/ be the result of reading a chunk from /blob/'s
underlying data.
 
 2. If /bytes/ is not failure, push /bytes/ to /s/ and set
/s/'s transmitted to /bytes/'s length.
 
 3. Otherwise, signal some kind of error to /s/ and terminate
these steps.




Are you sure this simply cannot be done with the existing read operation, which 
uses annotated tasks for asynchronous use? The existing read operation serves 
FileReader and FileReaderSync (and maybe other APIs in FileSystem) and 
hopefully Fetch too.

For instance, I thought the idea was that within Fetch to read /blob/ we’d do 
something like: 

1. Let /s/ be a new body.  Return /s/ and perform the rest of the steps async.
2. Perform a read operation [File API] on /blob/.
3. To process read…
4. To process read data, transfer each byte read to /s/ and set /s/’s 
transmitted to the number of bytes read.

// Chunked byte transfer is possible within the 50ms delta for process read 
data. We could specify that here better.//

5. To process read EOF ...
6. Otherwise, to process read error with a failure reason on /s/ ….

Why is something like that unworkable and why do we need another variant of the 
read operation exactly?

— A*

Re: File API: Blob URL origin

[Arun Ranganathan]

On Jun 30, 2014, at 7:13 PM, Glenn Maynard gl...@zewt.org wrote:

 Why would the identifier not just be the blob URL itself?  The spec currently 
 makes the identifier just the scheme data, which seems much more complex than 
 it needs to be.  revokeObjectURL should simply be Remove the entry from the 
 Blob URL Store for URL.  If we want to allow revoking URLs that have a 
 fragment attached it'd still need to strip it off; Firefox does this, but 
 Chrome doesn’t.


There’s no good reason not to do this. The scheme data identifier is a 
hold-over from the “only UUID” model that Fx has right now (the original intent 
was to have a table full of UUIDs and blob references). I’m happy to change to 
this model.


 Also, both Chrome and Firefox treat the entire URL as case-sensitive, eg. 
 Blob:... won't revoke the URL, or uppercasing the hostname portion in 
 Chrome.  Using the whole URL as the identifier makes this easy to do.
 
 Subsequent attempts to dereference url must return a network error should 
 be removed.  That should already be the consequence of unregistering the URL, 
 so this is a redundant requirement.
 



OK, this is a good nit too.

— A*

Re: File API: Blob URL origin

[Arun Ranganathan]

On Jul 1, 2014, at 2:32 AM, Anne van Kesteren ann...@annevk.nl wrote:

 That works for me. That way we can make this a more generic store if
 god forbid we get more of these schemes.


While I think mediastream URLs may have died on the vine, using the same store 
for filesystem URLs would be good.

— A*

Re: File API: Blob URL origin

[Arun Ranganathan]

On Jun 28, 2014, at 4:42 AM, Anne van Kesteren ann...@annevk.nl wrote:

 I now defined the origin for blob URLs:
 http://url.spec.whatwg.org/#concept-url-origin Sorry for the delay.
 
 Still need to work out the correct Fetch integration.
 


Thanks :)

Removed origin extraction from FIle API, but added identifier extraction (based 
on the same model — that is, running the basic URL parser). This makes adding 
entires to the Blob URL Store clearer.

I think the Blob pieces for Fetch are in place now.

http://dev.w3.org/2006/webapi/FileAPI

— A* 

Re: File API: Blob URL origin

[Arun Ranganathan]

On Jun 30, 2014, at 4:20 PM, Anne van Kesteren ann...@annevk.nl wrote:

 I don't really understand this. Entries should be added when a blob
 URL is created.


They are! That is, at the time the method URL.createObjectURL(blob) is called 
on blob, that method adds an entry to the Blob URL Store: 
http://dev.w3.org/2006/webapi/FileAPI/#add-an-entry

I’ve only defined identifier extraction for use with adding an entry. Is that 
wrong?


 And should be checked/removed when a blob URL is used
 (through parsing it in the URL parser, as defined by the URL parser).


Yes, this is when URL parser *checks* the Blob URL Store, *after* entries have 
either been added or are not present (which generates a network error).


 There should be no need for such a definition in the File API specification.


Since adding scheme data as an identifer to the Blob URL Store is the 
requirement, we only use the basic URL parser to get the scheme data from a 
generated URL to add to the Blob URL Store.

— A*

Re: File API: Blob URL origin

[Arun Ranganathan]

On Jun 30, 2014, at 4:57 PM, Anne van Kesteren ann...@annevk.nl wrote:

 On Mon, Jun 30, 2014 at 10:48 PM, Arun Ranganathan a...@mozilla.com wrote:
 They are! That is, at the time the method URL.createObjectURL(blob) is
 called on blob, that method adds an entry to the Blob URL Store:
 http://dev.w3.org/2006/webapi/FileAPI/#add-an-entry
 
 I’ve only defined identifier extraction for use with adding an entry. Is
 that wrong?
 
 It seems like you could define identifier creation, use that, use the
 return value to add an entry, and then return blob: + the return
 value. Creating a URL first and then parsing it again to extract
 something seems needlessly complicated.



Well, the best way to define URL.revokeObjectURL(blobURL) seemed to be in terms 
of parsing to extract identifier (scheme data) and then delete the entry 
corresponding to identifier.

But you’re absolutely right that URL.create* methods shouldn’t have a 
dependency on the basic URL parser, and so I’ve redefined those methods along 
the lines you say above (namely, defining identifier creation, and then Blob 
URL creation).

That’s http://dev.w3.org/2006/webapi/FileAPI/#creating-revoking in today’s 
editor’s draft.

(So *now* maybe the path is clear for Fetch with Blobs.)

— A*

Re: Blob URL Origin

[Arun Ranganathan]

On Jun 10, 2014, at 2:57 AM, Anne van Kesteren ann...@annevk.nl wrote:

 On Tue, Jun 10, 2014 at 12:16 AM, Arun Ranganathan a...@mozilla.com wrote:
 Right now, the Blob URL Store is defined in terms of units of similar-origin 
 browsing contexts; each unit is required to have a Blob URL Store. As you 
 point out, that allows all origins within document.domain access to a given 
 Blob URL Store.
 
 Yeah, so unlike what the discussion claimed thus far, we did not in
 fact allow that much cross-origin blob URL usage. Only origins within
 the document.domain reach.
 
 
 1. Require that entries in the Blob URL Store also store origin
 
 I thought this was the idea. The identifier would be
 http://someorigin:70/uuid”.


Yes; there was some discussion about tuples vs. strings on IRC, but I think one 
leads to the other, and we can define how to extract the origin from a parsed 
Blob URL in terms of another use of the URL Parser instead of string parsing.


 
 2. Define it strictly as a same-origin store. I’m a bit fuzzy on how exactly 
 to define this; for instance, strictly the origin and not the effective 
 script origin of a Document?
 
 We could say that the store is bound to a global object. And then both
 URL.createObjectURL() and places that parse URLs hook into the entry
 setting object's global object's blob URL store.
 
 At that point the only benefit of putting the origin into the URL is
 so that new URL(blob).origin works.


This seems right; I think it would be rare that a developer would need to check 
origin, but it’s been pointed out that there some use cases for that. It seems 
better to introduce a method that doesn’t require creating a new object, but I 
don’t feel strongly about it.


 Something that is still unclear to me is what happens when you
 navigate to a blob URL. I guess that still technically works as the
 URL parsing would happen within the correct global.


If URL parsing doesn’t occur within the correct global, a network error will be 
the result, since there won’t be a corresponding entry in the Blob URL store 
that matches the identifier. So I think this sounds workable.

— A*

Re: Blob URL Origin

[Arun Ranganathan]

On Jun 9, 2014, at 3:23 AM, Anne van Kesteren ann...@annevk.nl wrote:

 On Thu, May 29, 2014 at 11:42 AM, Anne van Kesteren ann...@annevk.nl wrote:
 However, I wonder if this at a standards level should come into play
 in the URL parser. After all that creates a structured clone of the
 blob in question. The lookup for the blob ID should probably fail at
 that point meaning it does not really matter when you then try to
 fetch that URL as it will simply not have an associated blob.
 
 I filed a bug https://www.w3.org/Bugs/Public/show_bug.cgi?id=25987 for
 this, but it seems worth discussing here.
 
 A blob URL store is already limited to all the origins that can reach
 each other through document.domain. So cross-origin blob usage was
 already limited per the specification. It seems like what we should do
 is instead make this a same-origin store. And then when URLs are
 parsed you'd only have access to the same-origin (and *not* effective
 origin) blob URL store. In turn that means it does not matter much
 whether you put origins in the blob URLs, but I suppose we cold do it
 for clarity. It would also make new URL(blobURL).origin work.


Right now, the Blob URL Store is defined in terms of units of similar-origin 
browsing contexts; each unit is required to have a Blob URL Store. As you point 
out, that allows all origins within document.domain access to a given Blob URL 
Store.

We could:

1. Require that entries in the Blob URL Store also store origin (namely the 
effective script origin of the settings object when the URL was created), in 
addition to identifier and reference to the Blob object. Requests without same 
origin as that of the entry in the Blob URL store for that identifier must 
fail. This keeps the Blob URL Store requirement as is, but adds origin as 
something in the store.

2. Define it strictly as a same-origin store. I’m a bit fuzzy on how exactly to 
define this; for instance, strictly the origin and not the effective script 
origin of a Document?

— A*

File API For Review | was Re: Blob URL Origin

[Arun Ranganathan]

The 2 June 2014 Editor’s Draft of the File API solves some bugs and technical 
issues with Blob URLs. Review is encouraged, with a view towards a LCWD 
publication:

http://dev.w3.org/2006/webapi/FileAPI/

In particular:

1. It nails down syntax differences between user agents on Blob URLs. 
2. It specifies origin and origin extraction from Blob URL strings.
3. It provides other pieces of plumbing that specifications like URL and Fetch 
can use.

A few issues remain, but all of these have dependencies:

https://www.w3.org/Bugs/Public/buglist.cgi?bug_status=NEWbug_status=ASSIGNEDbug_status=REOPENEDemail1=arun%40mozilla.comemailassigned_to1=1emailreporter1=1emailtype1=exactlist_id=38706

Notably, dependencies on Fetch (for Fetch of Blob URLs), and dependencies on 
WebIDL (e.g. removing DOMError altogether requires something done for error 
handling; supplanting FileList with array requires this part nailed down). I’m 
happy to have provisional specification text in place before WebIDL adopts 
fixes, but with guidance from the editor(s) of WebIDL.

I expect a CfC to follow about the specification.

— A*


On May 29, 2014, at 5:42 AM, Anne van Kesteren ann...@annevk.nl wrote:

 On Thu, May 29, 2014 at 8:38 AM, Jonas Sicking jo...@sicking.cc wrote:
 On Thu, May 22, 2014 at 1:29 AM, Anne van Kesteren ann...@annevk.nl wrote:
 For fetching blob URLs (and prolly filesystem and indexeddb) we
 effectively act as if the request's mode was same-origin. Allowing
 tainted cross-origin requests would complicate UUID (for the UA) and
 memory (for the page) management in a multiprocess environment.
 
 Hmm.. I think that is effectively it yes. I.e. even though img says
 that it wants to permit cross-origin loads, we'd override that if the
 fetch is for a blob: URL and only permit same-origin loads. Is that
 what you mean?
 
 Yes.
 
 However, I wonder if this at a standards level should come into play
 in the URL parser. After all that creates a structured clone of the
 blob in question. The lookup for the blob ID should probably fail at
 that point meaning it does not really matter when you then try to
 fetch that URL as it will simply not have an associated blob.
 
 
 -- 
 http://annevankesteren.nl/
 

Re: File API - Writer suspension

[Arun Ranganathan]

On Jun 1, 2014, at 1:22 PM, Julian Ladbury 
julian.ladb...@berrick-computing.co.uk wrote:

 I fail to understand why work on this API has been suspended.
 


Just to be clear, by “this API” I think you mean: 
http://dev.w3.org/2009/dap/file-system/file-writer.html



 HTML5, JavaScript and CSS together are becoming a natural platform of choice 
 on which to write portable applications. Indeed, I have just started work on 
 just such a project, welcoming the chance it gives to break away from 
 proprietary solutions.
 
 An essential part of any mature application is the ability to share data with 
 other applications. As an example, JSON provides an ideal way to do this: but 
 to be useful, it has to be possible to save a JSON file on a local system 
 (internet access cannot, and should not, be taken for granted) which can be 
 transmitted by email or other simple means.
 



I think this is the primary family of use cases around the FileSystem API:

http://w3c.github.io/filesystem-api/Overview.html

which is a successor specification.

A few things:

1. You can already create Blobs without using the BlobBuilder API (which has 
been deprecated), and you can already save them, but through a user prompt (the 
“File Save As” dialog).

This Specifiction discourse thread is useful for other workarounds, and a 
discussion of their shortcomings: 
http://discourse.specifiction.org/t/saving-files-api/63

Yes, this area of the platform is currently underspecified. Hopefully not for 
long though :)

2. In the interim, you can probably use IndexedDB to address the immediate use 
case. 

— A*

Re: Blob URL Origin

[Arun Ranganathan]

On May 22, 2014, at 4:29 AM, Anne van Kesteren ann...@annevk.nl wrote:

 Thanks, I'm convinced.
 
 So now I'd like to know what policy we want so we can carefully define it.


The lastest editor’s draft of the File API specifies what we discussed in this 
email thread as syntax for Blob URLs:

http://dev.w3.org/2006/webapi/FileAPI/#DefinitionOfScheme

and origin, including how to serialize the Blob URL.



 For blob URLs (and prolly filesystem and indexeddb) we put the origin
 in the URL and define a way to extract it again so new
 URL(blob).origin does the right thing.


I wonder if .origin should be static?



 For fetching blob URLs (and prolly filesystem and indexeddb) we
 effectively act as if the request's mode was same-origin. Allowing
 tainted cross-origin requests would complicate UUID (for the UA) and
 memory (for the page) management in a multiprocess environment.


We’re not allowing them.

— A*

Re: Blob URL Origin

[Arun Ranganathan]

On May 19, 2014, at 5:33 AM, Frederik Braun fbr...@mozilla.com wrote:

 On 15.05.2014 22:46, Glenn Maynard wrote:
 On Thu, May 15, 2014 at 12:07 PM, Jonas Sicking jo...@sicking.cc
 mailto:jo...@sicking.cc wrote:
 
On Thu, May 15, 2014 at 6:52 AM, Anne van Kesteren ann...@annevk.nl
mailto:ann...@annevk.nl wrote:
 I was thinking about the latter and that would not work if the URL was
 revoked. Unless we store origin at parse time.
 
Good point. Without using the explicit syntax we couldn't return a
consistent result for the origin.
 
 
 I'm not against the explicit model, but I think it would be very
 desirable if there was a way to get the origin out of the blob URL with
 some sort of API.
 It's not unlikely that developers will implement their own (flawed) URL
 parsing to determine the origin of a blob URL, if it's explicitly in the
 blob URL.


I think this could be done with the existing URL API’s origin attribute so that 
it does the right thing with a Blob URL: 
http://url.spec.whatwg.org/#dom-url-origin 

Of course, right now that won’t work in Fx. I wonder if we could fix that? Or 
add another static function to URL.

—A*

Re: Blob URL Origin

[Arun Ranganathan]

On May 12, 2014, at 8:28 AM, Anne Van Kesteren ann...@annevk.nl wrote:

 It still seems a bit sad though to tie these URLs to origins in this
 fashion. Jonas is correct that there are inconsistencies in how data
 URLs and origins behave across browsers, but it seems like we should
 sort those out first then if we want a consistent story.




Since Blobs can be passed around in a number of well-known ways, it seems that 
the most legitimate origin of a Blob URL is the origin of the script that 
coined it. I’m not entirely sure how to take action on “it still seems a bit 
sad” though. Sad because of security considerations? After drying my tears, I 
can’t construct a meaningful attack, but I’d welcome more information about 
what benefits are gained by encoding certain “HTTP-reserved” components of URL 
nomenclature (and here, Chrome is inconsistent between blob: and filesystem:). 
Sad because of aesthetics? It’s pretty enough for Safari.

And really, all user agents seem to agree that the origin is that of the 
settings object today. That model seems to work. The remaining question is the 
pro and con of denoting this in the URL’s syntax. abarth’s advice is to put the 
syntax horse in front of the origin cart: 
http://krijnhoetmer.nl/irc-logs/whatwg/20140508#l-913

Also, if it’s “sad” because it doesn’t match data: URL’s way of reckoning 
origin, that doesn’t seem sad to me. 

— A*

Re: Blob URL Origin

[Arun Ranganathan]

On May 12, 2014, at 10:31 AM, Boris Zbarsky bzbar...@mit.edu wrote:

 On 5/12/14, 5:28 AM, Anne van Kesteren wrote:
 so blob:https://origin:42/uuid would be fine.
 
 I'd really rather we didn't make web pages parse these strings to get the 
 origin.  A static method on Blob that takes a valid blob: URI and returns its 
 origin seems like it should be pretty easy for UAs to implement, though.


We actually aren’t obliging web pages parse these strings to get the origin. In 
fact, blob: URL strings shouldn’t even be of interest to web pages. They aren’t 
today, and I don’t envision them being of interest even with “origin tagging.” 
That is, I can’t think of why exactly a web developer would want to look into 
the blob: URL strings. UA’s should just “do the right thing” once a Blob URL is 
coined.

The question is really whether origin should be implicit or explicit. Fx’s 
implementation makes it implicit. so that there’s no way to deduce origins from 
the Blob URL itself, but it just “does the right thing” in terms of origin 
strictures. That hasn’t been a problem, but it’s hard to spec. it that way. 
Also, it makes Blob URLs only usable within APIs that are aware of them, which 
honestly is the case today.

So what if we tag origin into the strings? Would that be so bad? It’s not doing 
anything other than denoting the incumbent script setting object’s origin, no? 
Even in the Chrome/Safari cases, I can’t think of web developers using that 
information.

— A*

Re: data:, was: Blob URL Origin

[Arun Ranganathan]

On May 12, 2014, at 6:26 AM, Julian Reschke julian.resc...@gmx.de wrote:

 Could you please clarify what spec you are referring to, and in which way 
 it's not implemented correctly?


Well, I think http://tools.ietf.org/html/rfc6454#section-4 is is supposed to be 
normative for data: URL origin. But, implementations don’t behave this way; the 
problem is inheriting origins. Here’s the pertinent thread:

http://lists.w3.org/Archives/Public/public-webapps/2014JanMar/0682.html

Re: [April2014Meeting] Building an Issue and Bug focused agenda

[Arun Ranganathan]

I took the liberty of adding File + FileSystem to Thursday's agenda, taking 
place at 11.30 AM PT. I am happy to dial in on either day, so this isn’t set in 
stone if someone strongly objects.

— A*

On Apr 9, 2014, at 11:13 AM, Arthur Barstow art.bars...@nokia.com wrote:

 On 4/9/14 12:03 AM, ext Maciej Stachowiak wrote:
 Would the WG be willing to put Web Components related topics on Friday when 
 building the agenda? (Note: there are at least some Apple folks who will be 
 there both days in any case.)
 
 On Apr 8, 2014, at 8:40 AM, Dimitri Glazkov dglaz...@chromium.org 
 mailto:dglaz...@chromium.org wrote:
 
 Actually, Friday sounds better for me too!
 
 For a relatively large subject area (3 specs and the various discussion 
 topics Dimitri identified [1]) and broad interest, it could be better to 
 start discussions on Thursday and try to narrow the more contentious topics 
 (and potentially benefit from discussions during that evening's gatherings 
 and/or a good night's sleep). On the other hand, having a long-ish 
 dedicated time slot on Friday also has benefits.
 
 Having seen no objections to putting Web Components on Friday's agenda, I 
 just added it to Friday's 09:00 - 12:30 time slots [2].
 
 -AB
 
 [1] http://lists.w3.org/Archives/Public/public-webapps/2014AprJun/0074.html
 [2] https://www.w3.org/wiki/Webapps/April2014Meeting#Agenda_Friday_April_11
 
 

Re: [April2014Meeting] Building an Issue and Bug focused agenda

[Arun Ranganathan]

On Apr 2, 2014, at 6:50 AM, Arthur Barstow art.bars...@nokia.com wrote:

 File API; led by Jonas (will Arun join remotely?); which high priority bugs 
 need discussion; status of LC processing; next steps


If a dial-in is provided, I’d be delighted to join remotely.

— A*

Re: On starting WebWorkers with blob: URLs...

[Arun Ranganathan]

On Mar 12, 2014, at 6:54 PM, Ian Hickson wrote:

 For blob: URLs we agreed to make this pretty explicit: 
 http://dev.w3.org/2006/webapi/FileAPI/#originOfBlobURL
 
 Unfortunately, scripts don't have origins these days, so this definition 
 doesn't really work.



It didn't work since it wasn't formally specific about effective script origin 
according to the script setting object, but I've fixed that so hopefully this 
definition should work:

http://dev.w3.org/2006/webapi/FileAPI/#originOfBlobURL

-- A*

Re: On starting WebWorkers with blob: URLs...

[Arun Ranganathan]

On Feb 19, 2014, at 7:09 PM, Jonas Sicking wrote:

 On Wed, Feb 19, 2014 at 3:51 PM, Travis Leithead
 travis.leith...@microsoft.com wrote:
 Seems like our specs are getting a little behind implementations. In IE11 we 
 are finding that several new sites, notably those using WebGL content, have 
 a dependency on starting web workers with a Blob URL. As I understand it:
 
 The W3C Web Workers spec (CR stage) forbids use of the data and [implicitly] 
 blob protocols
 The WHATWG HTML spec (Living stage) allows data protocol, but not blob.
 Implementations of Firefox and Chrome support blob protocols
 
 1. Seems like a spec should say this somewhere...
 
 Agreed! It's a bit tricky since the concept of origins and thus same
 origin for data: and blob: is a bit unclear still. I.e. browsers
 don't behave consistently. Definitely not between each other, and
 sometimes not internally within a browser IIRC.


Well, for data: URLs what's normative might be: 
http://tools.ietf.org/html/rfc6454#section-4 but sadly that's not really clear 
(what's host here?).

For blob: URLs we agreed to make this pretty explicit: 
http://dev.w3.org/2006/webapi/FileAPI/#originOfBlobURL

Also, Blob URL Stores apply to each unit of similar origin browing contexts, 
which means that URL.createObjectURL and URL.createFor essentially give URL 
fetch and URL parse license to operate only within the origin of the script 
those methods apply to; URL.revokeObjectURL only applies to that origin also.

So now, we've made it pretty explicit for Blob URLs.


 2. In the W3C where would we spec this? (Workers V2?)
 
 I care less strongly about this. There's also the synchronous message
 passing API which I'd still like to see added to the workers spec.


IMHO it makes sense in Workers V2.

-- A*

Re: [File System APIs] If one is good, then two must be better?

[Arun Ranganathan]

On Feb 5, 2014, at 7:03 AM, Charles McCathie Nevile wrote:
 Similarly the issue here is not whether we can make a specification for
 one or the other approach that *could* be a standard, since it seems we
 can, but whether one or the other is a clear candidate to be a real
 standard - i.e. what people *will* actually do...
 
 I think one of the mistakes with IndexedDB (and appcache for that matter)
 was that the participants in the discussion were too heavily biased toward
 browser implementors, without enough input or involvement from working
 developers. Which meant that we standardised something that didn't meet
 people's expectations as well as we and they hoped.


Of course I agree that the path to standardization should involve input from 
developers, and should be based on what people will actually do.  There's 
been some support for a Promises-based FileSystem API in lieu of a 
callback-based one, both on this listserv and on public-script-coord, which is 
where the proposal started: 
http://lists.w3.org/Archives/Public/public-script-coord/2013JulSep/0379.html

To borrow an expression from that thread, we're not blank-slateists here: 
there are 3 separate APIs for JavaScript file system APIs, including the one in 
node.js (implemented as a posix-style API, familiar from other programming 
environments), from Google (implemented in Chrome as a callbacks-based 
asynchronous API), and from Mozilla (implementing the proposal we're discussing 
here [1], but sandboxed as of now). This is a call for feedback, from web 
developers and implementers.  I only made the point that when considering 
*implementations* we should consider similar beasts.

The thread about having a FileSystem API is a long one, and goes back a long 
time.  The motivation is to provide something asynchronous (but not callbacks 
stapled to posix shells), in keeping with the flavor of APIs currently deployed 
for the web (Promises, not events or callbacks), that is easy to use and with a 
low probability of doing bad things.  Feedback from web developers about that 
endeavor is invaluable.


 I hope that when we make a choice it's one that not only matches the
 reality of what gets implemented, but helps us provide what the market
 really wants. I presume everyone here hopes for that. I think a key part
 of how to get there is to listen carefully to the feedback we get, and
 look around at what people are doing, rather than just relying on
 formulaic rules of thumb or bureaucratic tallying of test results.


I agree with the spirit behind this comment, and am optimistic that you don't 
actually think that the call for opinions on a proposal, or the call for 
implementor support, is a sign of a formulaic rule of thumb or a 
bureaucratic tallying of test results.

-- A*
[1] Interfaces only: http://w3c.github.io/filesystem-api/Overview.html

Re: [File System APIs] If one is good, then two must be better?

[Arun Ranganathan]

On Feb 5, 2014, at 11:12 AM, Ian Clelland wrote:

 The great thing about Cordova is that it doesn't have to be a single platform 
 -- every developer has the power to choose the APIs that they want to use, 
 and every published app is essentially running on its own custom web platform.
 
 I wasn't presenting the Cordova data point as we're doing this already, and 
 it's going to be too much work to change now. Far from it; Cordova is a 
 flexible platform, and someone could write a plugin providing the 
 Mozilla-backed Filesystem API tomorrow, and it wouldn't cause any hardship to 
 developers who didn't choose to adopt it.




My point was merely to say that  every published app running on its own custom 
web platform is a different kind of implementation than a web browser, that's 
all :-)

I think it's *really great* that Cordova provides a flexible platform for web 
APIs to land, and that plugins can be deployed rapidly. I'd *gladly* welcome an 
implementation of the FileSystem proposal in Cordova, or at least more feedback 
on the proposal as it stands currently.

-- A*

Re: [File System APIs] If one is good, then two must be better?

[Arun Ranganathan]

On Feb 5, 2014, at 6:51 AM, Sebastian Kippe wrote:

 For what it’s worth, we’re currently building a mobile app for use with both 
 Cordova and Web browsers.
 
 The main use case of our app is caching large video files and playing them 
 back directly from the filesystem. So filesystem URLs are a critical feature 
 there.
 
 Other than that, the callback-based approach of the Google proposal let us 
 end up in callback hell right from the start, so we wrapped the whole API in 
 promises as best as we could, just to make it usable for our app developers. 
 In that department, Mozilla’s promises-based proposal might be much cleaner 
 and easier to work with.


This is really useful feedback :-)  Specifics about the API as it develops 
would be really helpful as well.

-- A* 

Re: [File System APIs] If one is good, then two must be better?

[Arun Ranganathan]

On Feb 4, 2014, at 6:15 AM, Charles McCathie Nevile wrote:

 On Mon, 03 Feb 2014 19:09:53 +0400, Arthur Barstow art.bars...@nokia.com 
 wrote:
 
 On 1/31/14 10:44 AM, ext Ian Clelland wrote:
 Hi Art,
 
 For what it's worth, theFile API: Directories and System is also 
 implemented (and supported) by Apache Cordova[1]. The implementation is 
 essentially complete for mobile applications on Android, iOS and FireOS, 
 with nearly-complete support on Blackberry and Windows Phone.
 
 While our plugin registry was counting downloads, it was the 
 most-downloaded plugin for the platform by a wide margin, so I believe it 
 is being used actively.
 
 Thanks for this information Ian!
 
 I don't know if Cordova should count as a browser implementation for the 
 purposes of this WG, but we are implementing the APIs and making them 
 available to (hybrid) web application developers.
 
 The group has some flexibility regarding the specifics of the 
 interoperability criteria used to advance a spec along the Recommendation 
 track, but we haven't talked about the criteria for these specs since they 
 are still working drafts.
 
 And the particular question here isn't about CR criteria, but about whether 
 one or other approach is more likely to achieve the consensus of 
 interoperable implementation.
 
 Which essentially means whether implementations are likely to switch, or 
 credible future implementors have a strong preference for one over the other.
 
 In which case, what Cordova does (and more to the point what developers do 
 with it) seems relevant information to consider as we try to find a consensus.



Two interoperable implementations of a specification should determine the way 
forward. While I think distributions like PhoneGap are extremely useful as 
web-like abstractions on top of disparate mobile platforms, it is not 
straightforward to make a clear apples to apples comparision for API 
interoperability between PhoneGap and a web browser, or conduct common test 
cases. Naturally, the distinctions blur, but I still think they exist.

A web page using the FileSystem API in JavaScript and working in two separate 
browser implementations seems like a good measure of interoperability, and I 
think this should be what helps us make a determination.

-- A*

Re: [File System APIs] If one is good, then two must be better?

[Arun Ranganathan]

On Jan 31, 2014, at 10:21 AM, Arthur Barstow wrote:

 Hi Eric, Arun, Jonas, All,
 
 
 We also have this spec from Arun and Jonas:
 
 * FileSystem API http://w3c.github.io/filesystem-api/Overview.html. The 
 last update of the ED was 2-Oct-2013 and this spec has not been published as 
 a TR.
 
 My understanding is the only implementation of Eric's APIs is Chrome. I do 
 not know the implementation status of Mozilla's spec. If anyone has 
 additional information about the implementation status or plans of either 
 effort, please let us know.
 


The specification is being implemented in Fx with the interfaces as in the 
document you link to, but the current implemention in Fx is sandboxed and not 
exposed to web content.  We'd like to change that at some point.


 The last discussion about the relationship between these different efforts 
 was August 2013 [Aug-2013] and prior to that, there was some discussion 
 during the April 2013 f2f meeting [April-2013].
 
 Ultimately, I think there is broad agreement a single API that is broadly 
 implemented and deployed would be `best` (f.ex. reduces FUD, lightens 
 implementation costs, lightens deployment costs, etc.). Although I would 
 (still) like to be optimistic we can agree to converge on a single API, 
 previous discussions about this do make me skeptical ...
 
 Eric, Arun, Jonas - can you agree and commit to converge your efforts, f.ex. 
 just have a single API?
 


We can certainly commit to editing the proposal we put forward, and would like 
to hear from other browser vendors about their opinions.  That will inform what 
we should do collectively.

-- A*

Re: CfC: publish Candidate Recommendation of File API; deadline November 28

[Arun Ranganathan]

On Dec 18, 2013, at 6:38 PM, Arthur Barstow wrote:
 
 Are there any other open bugs, issues, comments, etc. for File API?
 


https://www.w3.org/Bugs/Public/buglist.cgi?bug_status=NEWbug_status=ASSIGNEDbug_status=REOPENEDemail1=arun%40mozilla.comemailassigned_to1=1emailreporter1=1emailtype1=exactlist_id=30639

is the current list.  Some came up on 12/15/2013 and are minor.


 Unless I hear otherwise, I will assume the group wants to address the above 
 before a new Technical Report is published and depending on the outcome of 
 these issues, the next publication could be a Last Call Working Draft or a 
 Candidate Recommendation.
 


I agree with this.

-- A*

Re: File API: closed Blob objects

[Arun Ranganathan]

On Dec 12, 2013, at 10:07 AM, Anne van Kesteren wrote:

 See https://www.w3.org/Bugs/Public/show_bug.cgi?id=24072 for context.
 
 What's the use case for close()?


Essentially, the use case is for web developers to be able to flush a Blob 
reference that isn't used, allowing the user agent to manage resources.

This was expressed in: https://www.w3.org/Bugs/Public/show_bug.cgi?id=16952#c2



 
 If we are to have close(), let's define it in such a way that once you
 invoke it the Blob represents the empty byte sequence. This way other
 code that takes and operates on Blob objects does not have to special
 case closed Blob objects, but can just treat them as Blob objects that
 happen to be empty.


Currently, we say that :

Calling the close method on a Blob must disable it such that it cannot be used 
again; dereferencing a blob: URL bound to a Blob object on which close() has 
been called results in a network error. A Blob on which close() has been called 
must have a size of 0. This must be an irreversible and non-idempotent 
operation; once close() has been called on a Blob, that Blob cannot be used 
again.

Is size of 0 a sufficient constraint? I'm inclined to think so, and that we 
already support your suggestion.

Also, note that neutering a transferable object and calling close on Blob are 
*different things.*  We can make Blob's size 0 on close().  But, neutering a 
transferable object should throw on subsequent access.  You'll recall that we 
forced them to be different things based on bugs such as:

1. https://www.w3.org/Bugs/Public/show_bug.cgi?id=18611 and
2. https://www.w3.org/Bugs/Public/show_bug.cgi?id=23416

-- A*



-- A*

Re: File API: closed Blob objects

[Arun Ranganathan]

On Dec 12, 2013, at 2:01 PM, Jonas Sicking wrote:

 
 On Dec 12, 2013 9:55 AM, Arun Ranganathan a...@mozilla.com wrote:Reading 
 from a Blob that has been closed() is a developer error with high degree of 
 certainty. Nothing useful can come out of that.
 
 So I think we do the developer a benefit by treating it as such and signaling 
 an error through whatever mechanism is appropriate depending on how the 
 reading was done.
 
 We can certainly debate if throwing an exception or if asynchronously 
 reporting a network error is the right thing to do for things like 
 XHR.send(closedBlob) or FileReader.readAsX(closedBlob).
 

So right now (in spec.) any Blob URL operations on a closed Blob must result in 
a network error. FileReader.readAsX(closedBlob) operations are equivalent to 
reading a Blob of size 0.  

I can understand the argument that the above might be a bit inconsistent; after 
all, we could do a Blob URL operation on a Blob of size 0 (where nothing 
happens, since no bytes are fetched), just as we allow FileReader reads on a 
resource of size 0 (e.g. result is a 0 bytes ArrayBuffer).   We should pick one 
thing: treat it as 0, and proceed accordingly (often, this is silent failure), 
or treat it as something that should be reported back as an error.

If we go the throw or error route, since we've already spec'd a network error 
for fetched Blob URLs when the underlying Blob has been closed with .close(), 
the choice of what to do for FileReader.readAsX(closedBlob) is whether to throw 
or to asynchronously use an error event with a DOMError (possibly with a 
NotReadableError).   

On December 12, 2013 at 2:06 PM, bzbarsky wrote:
 Or form submission after fileInput.files[0].close() ?

This example makes me think that on balance, for consistency, maybe we should 
throw, and so should XHR.send(closedBlob).

-- A*

Re: File API: closed Blob objects

[Arun Ranganathan]

On Dec 12, 2013, at 3:14 PM, Jonas Sicking wrote:

 This does seem like an argument to create a asynchronous error rather than 
 throwing an exception though, since for form submission there would be no 
 place to throw an exception from.


Just to be clear, are you proposing an asynchronous error reported through an 
error event firing off of FileReader with a DOMError?

Also, it isn't clear what fileListObject.files[0].close() would do for an 
asynchronous error, so perhaps both are necessary: treating it as 0 bytes 
(empty byte sequence) *and* (where possible) using an asynchronous error 
reporting mechanism.

-- A*

Re: CfC: publish Candidate Recommendation of File API; deadline November 28

[Arun Ranganathan]

On Dec 1, 2013, at 10:24 AM, Arthur Barstow wrote:

 Since this CfC was started, Arun reopened [23853] on November 28 and [23946] 
 was filed on November 30.
 
 Arun - what's the plan here vis-à-vis this CfC?
 
 -Thanks, ArtB
 
 [23853] https://www.w3.org/Bugs/Public/show_bug.cgi?id=23853


Timely implementor feedback will help close this one out.  I will initiate a 
separate email thread about this issue.

 [23946] https://www.w3.org/Bugs/Public/show_bug.cgi?id=23946


I don't believe this issue is accompanied with a strong enough use case, and 
think it likely that it will be marked WONTFIX during the timeframe of the CfC.

-- A*

Re: [FileAPI] LC Comment Tracking

[Arun Ranganathan]

Art,

All LC commentary (http://www.w3.org/wiki/Webapps/LCWD-FileAPI-20130912) has 
been addressed and I think the draft is ready to be published as CR status.

The draft is: http://dev.w3.org/2006/webapi/FileAPI/

-- A*

On Nov 8, 2013, at 10:15 AM, Arun Ranganathan wrote:

 Hi Art,
 
 On Nov 7, 2013, at 9:40 AM, Arthur Barstow wrote:
 
 
 Since it appears you will not be at WebApps' f2f meeting next week, I would 
 appreciate it if you would please summarize the status of the comment 
 processing, your next steps, etc. I am especially interested in whether or 
 not you consider any of the bug fixes you applied as substantive and/or 
 add a new feature (which would require a new LC).
 
 
 Most LC commentary that was substantive became a spec bug; I've fixed most 
 such spec. bugs, and the contributor/commentor has been notified.
 
 In my opinion, the biggest change is to the File constructor.  This is 
 https://www.w3.org/Bugs/Public/show_bug.cgi?id=23479.  I don't think this is 
 a new feature, since the previous document pushed to /TR had a constructor, 
 athough a different signature.
 
 Other changes include moving Blob URL to be redefined in terms of terminology 
 in the WHATWG URL spec, in lieu of ABNFs.
 
 If you provide a dial-in on the day that you discuss File + FileSystem, I can 
 try and dial in, but this depends on time.  There will be others present from 
 Mozilla  :)  The LC commentary is tracked at 
 http://www.w3.org/wiki/Webapps/LCWD-FileAPI-20130912
 
 -- A*
 
 
 
 -Thanks, ArtB
 
 [1] http://www.w3.org/wiki/Webapps/LCWD-FileAPI-20130912
 
 
 On 9/12/13 10:39 AM, ext Arthur Barstow wrote:
 [ Bcc public-sysapps ; comments from SysApps are welcome ]
 
 This is a Request for Comments for the 12 September 2013 Last Call
 Working Draft of File API:
 
 http://www.w3.org/TR/2013/WD-FileAPI-20130912/
 
 The comment deadline is October 24 and all comments should be sent to
 the public-webapps@w3.org list with a subject: prefix of [FileAPI].
 
 The spec's bug list is [Bugs] and the few `approved` tests we have can
 be run in a browser at [Tests].
 
 -Thanks, ArtB
 
 [Bugs] http://tinyurl.com/Bugs-FileAPI
 [Tests] http://w3c-test.org/web-platform-tests/master/FileAPI/
 
 
 
 
 
 

Re: Why no TypeError in FileReaderSync.readAsArrayBuffer

[Arun Ranganathan]

Norman,
On Nov 7, 2013, at 5:00 PM, Norman Rzepka | scalable minds wrote:

 because I didn't know a FileEntrySync was actually not a File, I passed an 
 instance of FileEntrySync to a FileReaderSync. Instead I got a NotFoundError 
 which I found very confusing as I know the file was present. So I was 
 wondering why there is no TypeError for such situations?
 


The API you're testing -- namely code which uses FileEntrySync primitives -- is 
likely to work in Chrome only, and isn't a standard with other browser vendors, 
which do implement the File API.

Currently, the FileSystem API (which, going forward is this draft; 
http://w3c.github.io/filesystem-api/Overview.html) doesn't have a FileEntrySync 
primitive.  And the File API is 
http://dev.w3.org/2006/webapi/FileAPI/#FileReaderSync .

In this particular case, I think it's the right behavior: FileReaderSync 
expects a valid File or Blob reference, and isn't able to identify one, so says 
one isn't found.

-- A*



 Best, 
 Norman
 
 --
 Norman Rzepka
 
 norman.rze...@scalableminds.com
 http://scalableminds.com
 
 scalable minds
 UG (haftungsbeschränkt)  Co. KG
 Amtsgericht Potsdam, HRA 5753
 Geschäftsführer: Dominic Bräunlein, Tom Bocklisch, Tom Herold, Norman Rzepka, 
 Thomas Werkmeister

Re: Why no TypeError in FileReaderSync.readAsArrayBuffer

[Arun Ranganathan]

On Nov 11, 2013, at 11:08 AM, Tab Atkins Jr. wrote:

 On Mon, Nov 11, 2013 at 5:57 AM, Arun Ranganathan a...@mozilla.com wrote:
 In this particular case, I think it's the right behavior: FileReaderSync
 expects a valid File or Blob reference, and isn't able to identify one, so
 says one isn't found.
 
 No, NotFoundError is for when some search is unsuccessful but
 necessary.  Looking at the type of an argument doesn't seem like a
 search without a *lot* of stretching.  It's a TypeError, if you're
 going to throw anything.
 

You're right, a TypeError is less of a stretch.

This should be a browser bug, though.

-- A*



 ~TJ
 

Re: [FileAPI] LC Comment Tracking

[Arun Ranganathan]

Hi Art,

On Nov 7, 2013, at 9:40 AM, Arthur Barstow wrote:

 
 Since it appears you will not be at WebApps' f2f meeting next week, I would 
 appreciate it if you would please summarize the status of the comment 
 processing, your next steps, etc. I am especially interested in whether or 
 not you consider any of the bug fixes you applied as substantive and/or add 
 a new feature (which would require a new LC).


Most LC commentary that was substantive became a spec bug; I've fixed most such 
spec. bugs, and the contributor/commentor has been notified.

In my opinion, the biggest change is to the File constructor.  This is 
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23479.  I don't think this is a 
new feature, since the previous document pushed to /TR had a constructor, 
athough a different signature.

Other changes include moving Blob URL to be redefined in terms of terminology 
in the WHATWG URL spec, in lieu of ABNFs.

If you provide a dial-in on the day that you discuss File + FileSystem, I can 
try and dial in, but this depends on time.  There will be others present from 
Mozilla  :)  The LC commentary is tracked at 
http://www.w3.org/wiki/Webapps/LCWD-FileAPI-20130912

-- A*



 -Thanks, ArtB
 
 [1] http://www.w3.org/wiki/Webapps/LCWD-FileAPI-20130912
 
 
 On 9/12/13 10:39 AM, ext Arthur Barstow wrote:
 [ Bcc public-sysapps ; comments from SysApps are welcome ]
 
 This is a Request for Comments for the 12 September 2013 Last Call
 Working Draft of File API:
 
 http://www.w3.org/TR/2013/WD-FileAPI-20130912/
 
 The comment deadline is October 24 and all comments should be sent to
 the public-webapps@w3.org list with a subject: prefix of [FileAPI].
 
 The spec's bug list is [Bugs] and the few `approved` tests we have can
 be run in a browser at [Tests].
 
 -Thanks, ArtB
 
 [Bugs] http://tinyurl.com/Bugs-FileAPI
 [Tests] http://w3c-test.org/web-platform-tests/master/FileAPI/
 
 
 
 

Re: [FileAPI]

[Arun Ranganathan]

Note that all LC Commentary, including that sent on this listserv, is tracked 
here: 
http://www.w3.org/wiki/Webapps/LCWD-FileAPI-20130912

-- A*
On Oct 31, 2013, at 9:12 AM, Arthur Barstow wrote:

 Hi,
 
 For the purposes of tracking your comments for the September 12 File API Last 
 Call Working Draft, please let us know if Arun's reply is satisfactory or 
 not. In the absence of a reply from you by November 7, we will assume Arun's 
 reply is OK with you.
 
 -Thanks, ArtB
 
 On 10/23/13 6:04 PM, ext Arun Ranganathan wrote:
 Hi there!
 
 
 On Oct 23, 2013, at 12:32 PM, psweatte wrote:
 
 7.2 Interface File:
 -add creationDate property
 
 Thanks for your feedback.  *Most* filesystems don't really have a creation 
 time.  While Windows does, Unix-style OS return the *change time or last 
 modified time.  Since we want fidelity with OS filesystems wherever 
 possible, I'm not sure this is a viable property to add.
 
 
 -add size property
 This already exists via the inheritance relationship with Blob.
 
 
 -If the last modification date and time are not known, the attribute must 
 return an empty string
 
 Currently the spec says to return the current date and time.  It's a Date 
 not a String.
 
 
 8.3. Event Handler Attributes
 
 I think the current set of event handler attributes is sufficient, 
 especially given the backdrop of event models in general not being the best 
 way to asynchronously access large data sets such as File and Blob.
 
 
 -add onNotfounderror event handler
 -add onReaderror event handler
 -add onSecurityerror event handler
 -add onHTTPerror event handler
 -add onSelectfrombrowse event handler
 
 
 
 
 
 

Re: [FileAPI]

[Arun Ranganathan]

Hi there!


On Oct 23, 2013, at 12:32 PM, psweatte wrote:

 7.2 Interface File:
 -add creationDate property


Thanks for your feedback.  *Most* filesystems don't really have a creation 
time.  While Windows does, Unix-style OS return the *change time or last 
modified time.  Since we want fidelity with OS filesystems wherever possible, 
I'm not sure this is a viable property to add.


 -add size property

This already exists via the inheritance relationship with Blob.


 -If the last modification date and time are not known, the attribute must 
 return an empty string


Currently the spec says to return the current date and time.  It's a Date not a 
String.


 8.3. Event Handler Attributes


I think the current set of event handler attributes is sufficient, especially 
given the backdrop of event models in general not being the best way to 
asynchronously access large data sets such as File and Blob.


 -add onNotfounderror event handler
 -add onReaderror event handler
 -add onSecurityerror event handler
 -add onHTTPerror event handler
 -add onSelectfrombrowse event handler
 
 
 

Re: Problem in the Entry interface description (filesystem)

[Arun Ranganathan]

Greetings Alexis,

The specification actively edited now is:

http://w3c.github.io/filesystem-api/Overview.html

-- A*

On Oct 10, 2013, at 10:00 AM, alexis dufrenoy wrote:

 Hello,
 
 In the description of the Entry interface located here:
 
 http://dev.w3.org/2009/dap/file-system/pub/FileSystem/#the-entry-interface
 
 I noticed the following sentence:
 
 An abstract interface representing entries in a file system, each of which 
 may be a File or DirectoryEntry., with a link leading to the File interface. 
 Obviously, the File interface is not a descendant of the Entry interface, but 
 FileEntry is, as can been easily checked on both descriptions:
 
 http://dev.w3.org/2006/webapi/FileAPI/#dfn-file
 http://dev.w3.org/2009/dap/file-system/pub/FileSystem/#the-fileentry-interface
 
 File is a Blob.
 
 I hope it helps, but I'm sure I'm not the first to notice.
 
 Regards,
 
 
 Alexis Dufrenoy

Re: [FileAPI] Questions on using Blobs for img src and a href

[Arun Ranganathan]

On Oct 3, 2013, at 6:35 PM, Brian Matthews (brmatthe) wrote:


 First is the status bar display for anchors while hovering over them. As 
 expected, it's the blob URL. While this is completely correct and exactly 
 what I'd expect, I'm not sure how useful it is. For an anchor with a normal 
 URL, the user is told something about where the resource is (the domain name 
 and path, http://example.com/order;), and what it is (the last element of 
 the path, invoice.pdf). With a blob URL, they're told where it is (at least 
 for those who know what a blob URL is, or accept that blob:... is something 
 magic they don't need to know about :-) ), but nothing about what it is.


This is an implementation issue that goes beyond the spec. (as others have 
remarked, this kind of thing falls under the purview of UI/UX, which web 
specifications rarely touch on meaningfully), but I think it is probably a 
useful informative comment that I'll add as part of LC Commentary changes.

I think you should file bugs on all browser projects that you'd like to see 
this kind of thing implemented in.  It's a useful feature for end-users.


 Second, and related, is what happens when someone saves an image or target of 
 a link. In both cases with normal URLs, there's a name component and the 
 user agent can use that as the name of the resulting file, or suggest it if 
 doing a Save As With blob URLs, there is no name, so the user agent has 
 to make up a name. So with Firefox one gets fun names like index.gif (when 
 saving an img), or bf_UK+0O.gif and y+f+wR9a.pdf (when saving the target of 
 an anchor), and Chrome uses the opaqueString part of the URL, resulting in 
 names like 49c122d8-0958-4dfd-ac9c-0a6245c5f91f..gif. None of those exactly 
 brim with semantic content.


OK, I've filed https://www.w3.org/Bugs/Public/show_bug.cgi?id=23450

I'm not fully convinced by other commentors on this listserv that we should use 
Content-Disposition as how to do this.  I think how Save As is implemented is 
a detail left to browsers, but I think the part of using file name is useful 
during Save As.


 Also, 8.5.6 step 1 in the spec starts Fire a progress event called error. 
 Set the error attribute;. Doesn't firing an event call the event handlers 
 immediately? If so, this seems to be saying to call the error handler 
 *before* setting the error attribute,  which seems backwards.


Yes, I agree!  https://www.w3.org/Bugs/Public/show_bug.cgi?id=23451

Thanks for your thoughtful LC commentary.

-- A*

Re: [file-{writer,system}-api] Seeking status and plans

[Arun Ranganathan]

On Oct 2, 2013, at 12:29 PM, Arthur Barstow wrote:

 Hi Eric,
 
 If any of the data for your two File APIs in [PubStatus] is not accurate, 
 please provide corrections.
 
 Also, if you have any new information re your plans for these specs (last 
 published 17-Apr-2012), or the specs' status with respect to being feature 
 complete, please let us know.
 
 All - besides Chrome, if there are other implementations of these specs, 
 please let us know.
 
 -Thanks, ArtB


I've created a draft of the FileSystem API in GitHub, based on discussions with 
w3-staff:

https://github.com/w3c/filesystem-api

This API is different than the Google FileSystem API proposal, athough it will 
refine the concept of the filesystem: scheme, initially proposed by Google.

-- A*

Re: CfC: publish LCWD of File API; deadline August 23

[Arun Ranganathan]

The LCWD of File API is http://www.w3.org/TR/FileAPI/

LC comment period is till October 24 2013 -- comments strongly encouraged!

-- A*

On Aug 16, 2013, at 8:35 AM, Arthur Barstow wrote:

 Arun proposed (see below) WebApps publish a Last Call Working Draft of File 
 API and this is a Call for Consensus to do so, using the following ED as the 
 basis:
 
 http://dev.w3.org/2006/webapi/FileAPI/
 
 This CfC satisfies the group's requirement to record the group's decision to 
 request advancement for this LCWD. Note the Process Document states the 
 following regarding the significance/meaning of a LCWD:
 
 [[
 http://www.w3.org/2005/10/Process-20051014/tr.html#last-call
 
 Purpose: A Working Group's Last Call announcement is a signal that:
 
 * the Working Group believes that it has satisfied its relevant technical 
 requirements (e.g., of the charter or requirements document) in the Working 
 Draft;
 
 * the Working Group believes that it has satisfied significant dependencies 
 with other groups;
 
 * other groups SHOULD review the document to confirm that these dependencies 
 have been satisfied. In general, a Last Call announcement is also a signal 
 that the Working Group is planning to advance the technical report to later 
 maturity levels.
 ]]
 
 The proposed review period for this LC is 4 weeks.
 
 There are two open [Bugs] for this spec.
 
 If you have any comments or concerns about this CfC, please send them to 
 public-webapps@w3.org by August 23 at the latest. Positive response is 
 preferred and encouraged and silence will be considered as agreement with the 
 proposal.
 
 -Thanks, AB
 
 [Bugs] http://tinyurl.com/Bugs-FileAPI
 
 
  Original Message 
 Subject:  File* Specifications | Status and some Promises
 Resent-Date:  Thu, 15 Aug 2013 17:50:25 +
 Resent-From:  public-webapps@w3.org
 Date: Thu, 15 Aug 2013 13:49:38 -0400
 From: ext Arun Ranganathan a...@mozilla.com
 To:   Web Applications Working Group WG public-webapps@w3.org
 
 
 
 Greetings WG,
 
 There are three moving proposals in the File* arena, and I thought I'd 
 catpure what these are and what the status of these are.
 
 1. The File API, currently updated as editor's draft at 
 http://dev.w3.org/2006/webapi/FileAPI/
 
 Review strongly encouraged :)  Notable fixes are:
 
 i. A File constructor has been added, which has been a longstanding request.
 ii. A new static method on URL called URL.createAndRevoke has been added that 
 gets defaults right, and does away with autoRevoke, which nobody implemented 
 for URL.createObjectURL.
 iii. A Blob URL Store and a Revocation List, useful for URL Fetch (see 
 https://www.w3.org/Bugs/Public/show_bug.cgi?id=17765).
 
 This draft should be ready for LC and should be considered the File API v1.
 
 2. Re-writing File APi to be Promises based, which should be considered a v2. 
  The biggest departure from the current state of technology here would really 
 be that we can have Promises right off the Blob itself, which is zeitgeist 
 thinking about these things (and admittedly makes life easier than an 
 XHR-inspired FileReader, which will be made legacy).  The best stab at a 
 strawperson was this one by Jonas:
 
 http://lists.w3.org/Archives/Public/public-webapps/2013AprJun/0727.html
 
 But Stream itself temporarily lives here without too much implementation 
 backbone: https://dvcs.w3.org/hg/streams-api/raw-file/tip/Overview.htm
 
 I think we *should* have a Promises-based File API off of Blob, but I'm not 
 totally sure about the Stream API as it is currently written; I'm amenable to 
 editing the File API v2 portion, modulo our proposal for Stream.
 
 3. Then there's the FileSystem API, currently discussed here: 
 http://lists.w3.org/Archives/Public/public-script-coord/2013JulSep/0379.html
 
 This was the upshot of discussions on this listserv, as well as internally at 
 Mozilla.  I'm amenable to editing this, which is separate but related to 1. 
 and 2.
 
 -- A*
 
 
 
 
 
 
 
 

Re: File API: File's name property

[Arun Ranganathan]

On Sep 6, 2013, at 11:42 AM, Anne van Kesteren wrote:

 On Wed, Sep 4, 2013 at 11:45 PM, Glenn Maynard gl...@zewt.org wrote:
 On Tue, Sep 3, 2013 at 12:04 PM, Anne van Kesteren ann...@annevk.nl wrote:
 The problem is that once you put it through the URL parser it'll
 become /. And I suspect given directory APIs and such it'll go
 through that layer at some point.
 
 I don't follow.  Backslashes in filenames are escaped in URLs
 (http://zewt.org/~glenn/test%5Cfile), like all the other things that require
 escaping.
 
 If the raw input to the URL parser includes a backslash, it'll be
 treated as a forward slash. I am not really expecting people to use
 encodeURI or such utilities.


I think it may be ok to restrict / and \.  I don't think we lose too much 
here by not allowing historically directory delimiting characters in file 
names.

The question is what to do with a /  or a \.   I'm inclined to say UAs 
should treat those as U+FFFD.

 
 Well, my suggestion was rawName and name (which would have loss of
 information), per the current zip archive API design.
 
 Having a separate field is fine.  This is specific to ZIPs, so it feels like
 it belongs in a ZipFile subclass, not File itself.
 
 Is it? There's no other file systems where the file names are
 effectively byte sequences? If that's the case, maybe that's fine.


Well…. 

Some file systems don't store names as unrestricted byte sequences (older 
Windows), but GNU systems usually do.  Some byte sequences are not valid names. 
Conversely, names of existing files may not be representable as byte sequences 
(and sometimes there are two representations -- e.g. Amèlie.txt will either use 
00e9 or 0065 0031 for the è  -- both are Unicode equivalents, but are different 
byte sequences). Some file systems perform Unicode canonicalization on file 
names, which is more or less what I think the Web should do.

I think we run only a small risk of information loss, but I DO think that File 
name should be an [EnforceUTF16] DOMString.  That way, we have the best shot at 
byte sequences based on the underlying characterization.

Summary: I'll punt on File.rawName till a rainier day than today, but I will 
restrict / and \ since they are historically directory separators.  I know 
that there are OTHER characters that we can also restrict, but these two are 
the big ones and get us some 80-20 sanitization :)

Glenn said:

 It might be better to wait until we have a filesystem API, then piggyback on
 that...

+1.

-- A*

Re: File API: File's name property

[Arun Ranganathan]

Well, https://www.w3.org/Bugs/Public/show_bug.cgi?id=23138 is to make the 
'type' attribute a ByteString.  Is that your request here for the name 
attribute as well?

It wouldn't be wise to restrict '/' or '\' or try to delve too deep into 
platform land BUT the FileSystem API introduces directory syntax which might 
make being lax a fly in the ointment for later.


On Aug 29, 2013, at 10:48 AM, Anne van Kesteren wrote:

 As currently specified File's name property seems to be a code unit
 sequence. In zip archives the resource's path is a byte sequence. I
 don't really know what popular file systems do. Given that a File has
 to be transmitted over the wire now and then, including it's name
 property value, a code unit sequence seems like the wrong type. It
 would at least lead to information loss which I'm not sure is
 acceptable if we can prevent it (or at least make it more obvious that
 it is going on, by doing a transformation early on).
 
 We may also want to restrict \ and / to leave room for using these
 objects in path-based contexts later.
 
 
 -- 
 http://annevankesteren.nl/
 

Re: File API: lacks model

[Arun Ranganathan]

Anne, 

This feedback is a bit vague. 


 I think you should describe the underlying model for Blob/File so
 other specifications can more easily hook into it. E.g. if I want to
 represent a File object with a name /x/ and type /y/ there's not a
 clear way to do that right now. This also leads to issues such as [value 
 space of properties]


Why isn't there a clear way to do that now?  
https://www.w3.org/Bugs/Public/show_bug.cgi?id=23138 notwithstanding, you can 
either obtain a File from the underlying file system with a best guess type 
by the UA, OR create one with a constructor.  What's missing exactly?


 E.g. A |blob| provides asynchronous access to a byte sequence. A blob
 has an associated |type| which is a MIME type. (Which could be a link
 to the variant of MIME type you use.) And then you build the API on
 top of this model.
 

I treat http://mimesniff.spec.whatwg.org/ as relatively normative here.  What 
am I missing?

-- A*

Re: File API: File's name property

[Arun Ranganathan]

On Sep 3, 2013, at 12:28 PM, Anne van Kesteren wrote:

 On Tue, Sep 3, 2013 at 5:14 PM, Glenn Maynard gl...@zewt.org wrote:
 On Tue, Sep 3, 2013 at 10:17 AM, Anne van Kesteren ann...@annevk.nl wrote:
 I don't think you want those conversion semantics for name. I do think
 we want the value space for names across different systems to be
 equivalent, which if we support zip basically means bytes.
 
 I don't really understand the suggestion of using a ByteString for
 File.name.  Can you explain how that wouldn't break
 https://zewt.org/~glenn/picker.html, if the user picks a file named
 漢字.txt?
 
 ByteString doesn't work. A byte sequence might. If the platform does
 file names in Unicode it would be converted to bytes using utf-8.


Which in fact is how I think we should do File.name.  We'll stick to DOMString, 
but think it should specify a conversion to a byte sequence using utf-8.  And, 
restrict separators such as / and \.

-- A*

Re: Extending createObjectUrl to MediaStream?

[Arun Ranganathan]

On Aug 28, 2013, at 9:48 AM, Dominique Hazael-Massieux wrote:

 Hi WebApps WG,
 
 The Media Capture Task Force (responsible for specs that deal with
 MediaStream objects [1]) has been considering whether one should be able
 to assign a MediaStream to video and audio elements via an URL
 obtained via createObjectURL or not, and is seeking feedback on the
 question.
 
 More precisely:
 A. we define a new srcObject attribute on HTMLMediaElement objects that
 can take directly a MediaStream object and make it playable
  video.srcObject = mediastream;
 B. the spec also supports assigning a MediaStream via the src attribute,
 via an URL obtained through createObjectURL:
 video.src = URL.createObjectURL(mediastream);
 
 While there are ongoing discussions on how to spec B properly [2] (which
 will require coordination with the WebApps Working Group), we are first
 and foremost wondering if that option is needed at all.
 
 We are thus looking for input on the use cases for createObjectURL as
 used for the File API, and whether these use cases would also apply to
 our MediaStream case. In general, is there a need for any object
 readable by media elements to support an URL-based approach for
 consistency with the rest of the platform?



The fact that URL.createObjectURL exists at all is hitched to the ongoing use 
of strings as URLs in places like Image (img src=) and CSS.  This also 
created the discussion about autoRevoke (namely the automatic revocation of 
these references), for which we added URL.createFor.  As it stands, developers 
should bear the onus of calling URL.revokeObjectURL coupled to 
URL.createObjectURL, which may be a tall request.

MediaStream may not be as ubiquitous as Blob, and you may be able to avoid URL 
strings altogether.  If that's a fair assumption, option A seems like a good 
plan.  If you go the option B route, then you'll have to also think about 
revocation and URL.createFor, since you'll have the same problem.

On balance, I favor Option A, unless you make the case for strings :)  If you 
choose Option B, I'm happy to work with your WG to make sure File API has the 
right hook for MediaStream objects, per 
https://www.w3.org/Bugs/Public/show_bug.cgi?id=19594

-- A*

-- A*

Re: FileSystem API

[Arun Ranganathan]

On Aug 26, 2013, at 8:01 AM, Arthur Barstow wrote:

 Hi Eric, Arun, Jonas, All,
 
 
 2) Is any vendor other than Mozilla actually interested in this
 proposal?  When it was brought up on public-webapps, and at the
 WebApps F2F, it dropped with a resounding thud.
 
 Given the standardization failure of the Chrome FileSystem API, this
 could be a massive waste of time.  Or it could just be a way for
 Mozilla to document its filesystem API, since we've already got
 documentation of the Chrome API, but then you don't need to drag
 public-script-coord into that.
 
 These are good questions and points. I don't feel real strongly here re our 
 next steps other than I think we should try to get consensus on a high-level 
 plan to help set expectations accordingly.
 
 It seems we have a few options, some are not necessarily mutually exclusive 
 ...
 
 1. Leave Eric's specs in WD state
 
 2. Move Eric's specs to LC-CR (feature set should probably be 
 restricted/limited to what is already implemented in Chrome); block in CR 
 until there are two or more implementations
 
 3. Move Eric's specs to WG Notes and stop work (as was done with Web SQL 
 Database)
 
 4. Merge the two proposals
 
 5. Formally start work on Mozilla proposal knowing there is some overlap with 
 Eric's specs


I don't actually think it's possible to merge the two proposals.  I'm in favor 
of option 5.

-- A*

Re: File API - Progress - Question about Partial Blob data

[Arun Ranganathan]

On Aug 22, 2013, at 12:07 PM, Jonas Sicking wrote:

 I think you might have misunderstood my initial comment.
 
 I agree that the current partial data solution is not good. I think we
 should remove it.
 


I'd really like other implementors to weigh in before we remove Partial Blob 
Data.  Cc'ing folks who helped with it.


 I think we should instead create a better solution in v2 of the API
 which is based on something other than FileReader and which has the
 ability to deliver data on the form of here's the data that was
 loaded since last notification.


I agree that we should do a better way.

-- A*

Re: File API - Progress - Question about Partial Blob data

[Arun Ranganathan]

On Aug 20, 2013, at 7:13 PM, Aymeric Vitte wrote:

 The specs says :
 
  It can also return partial Blob data. Partial Blob data is the part of the 
 File or Blob that has been read into memory currently; when processing the 
 read method readAsText, partial Blob data is a DOMString that is incremented 
 as more bytes are loaded (a portion of the total) [ProgressEvents], and 
 when processing readAsArrayBuffer partial Blob data is an ArrayBuffer 
 [TypedArrays] object consisting of the bytes loaded so far (a portion of the 
 total)[ProgressEvents]. The list below is normative for the result attribute 
 and is the conformance criteria for this attribute
 
 What is the rationale for that? The result attribute should better contain 
 for progress events the latest data read and not the data read from the 
 begining that you could easily reconstitute while the contrary requires more 
 work.
 
 Use case: calculate the hash of a file while you are reading it.


The ask for deltas to be sent with progress notifications came up on this 
listserv before -- see the thread starting at 
http://lists.w3.org/Archives/Public/public-webapps/2013JanMar/0069.html

I agree that the deltas model is also useful, but you can see there's some 
implementation history with XHR here as well.  The use case is to receive the 
file bits as they are constituted from the read (just as with an HTTP request, 
where you get the bits so far till the rest are constituted).

A good way to solve the use case of meaningful deltas might be with the Streams 
API, still TBD.


 PS: I did not test it in all browsers, but unless I am using it wrongly, the 
 result attribute is always null for progress events.


But not null for the onload ?  In many cases, a progress might not fire, 
depending on file size.

-- A*

Re: File* Specifications | Status and some Promises

[Arun Ranganathan]

On Aug 20, 2013, at 8:58 AM, Anne van Kesteren wrote:

 On Thu, Aug 15, 2013 at 6:49 PM, Arun Ranganathan a...@mozilla.com wrote:
 ii. A new static method on URL called URL.createAndRevoke has been added 
 that gets defaults right, and does away with autoRevoke, which nobody 
 implemented for URL.createObjectURL.
 
 My problem with these names is that they suggest you get a URL object
 whereas you actually get a string. They also seem longish. I suggest
 we name the revoking variant just create or a synonym that has less
 of a suggestion of creating a URL object.


I agree, and got the same feedback in off-list correspondence.

The ship has already sailed on URL.createObjectURL unfortunately, but we can 
get the newer function's name right.  Jonas proposes URL.createFor in lieu of 
URL.createAndRevoke.

-- A*

 
 
 -- 
 http://annevankesteren.nl/

FileSystem API

[Arun Ranganathan]

Greetings Eric and WG,

The Chair and I were discussing setting up repositories for the specifications 
discussed here 
(http://lists.w3.org/Archives/Public/public-webapps/2013JulSep/0307.html), 
notably the FileSystem API and File API v2.  Before creating a repository to 
edit the FileSystem API, we thought we'd check with you about the first 
proposal, which Chrome implements, and get the Google perspective.

You've edited the first FileSystem API proposal, which currently lives here 
(http://www.w3.org/TR/file-system-api/).  Can I create a repository and edit 
the other proposal for FileSystem API, which currently exists as an email 
thread 
(http://lists.w3.org/Archives/Public/public-script-coord/2013JulSep/0379.html) ?

Just checking to see if there are any objections or concerns that would stop a 
draft or future WG activity.  Of course, technical nits should be heard as 
well, and can proceed concurrently with a draft :)

-- A*

File* Specifications | Status and some Promises

[Arun Ranganathan]

Greetings WG,

There are three moving proposals in the File* arena, and I thought I'd catpure 
what these are and what the status of these are.

1. The File API, currently updated as editor's draft at 
http://dev.w3.org/2006/webapi/FileAPI/

Review strongly encouraged :)  Notable fixes are:

i. A File constructor has been added, which has been a longstanding request.
ii. A new static method on URL called URL.createAndRevoke has been added that 
gets defaults right, and does away with autoRevoke, which nobody implemented 
for URL.createObjectURL.
iii. A Blob URL Store and a Revocation List, useful for URL Fetch (see 
https://www.w3.org/Bugs/Public/show_bug.cgi?id=17765).

This draft should be ready for LC and should be considered the File API v1.  

2. Re-writing File APi to be Promises based, which should be considered a v2.  
The biggest departure from the current state of technology here would really be 
that we can have Promises right off the Blob itself, which is zeitgeist 
thinking about these things (and admittedly makes life easier than an 
XHR-inspired FileReader, which will be made legacy).  The best stab at a 
strawperson was this one by Jonas:

http://lists.w3.org/Archives/Public/public-webapps/2013AprJun/0727.html

But Stream itself temporarily lives here without too much implementation 
backbone: https://dvcs.w3.org/hg/streams-api/raw-file/tip/Overview.htm

I think we *should* have a Promises-based File API off of Blob, but I'm not 
totally sure about the Stream API as it is currently written; I'm amenable to 
editing the File API v2 portion, modulo our proposal for Stream. 

3. Then there's the FileSystem API, currently discussed here: 
http://lists.w3.org/Archives/Public/public-script-coord/2013JulSep/0379.html

This was the upshot of discussions on this listserv, as well as internally at 
Mozilla.  I'm amenable to editing this, which is separate but related to 1. and 
2.

-- A*

Re: File constructor

[Arun Ranganathan]

On Jul 16, 2013, at 8:40 AM, Anne van Kesteren wrote:

 On Mon, Jul 15, 2013 at 11:37 PM, Arun Ranganathan a...@mozilla.com wrote:
 On Jul 15, 2013, at 8:01 PM, Anne van Kesteren wrote:
 So exposing FormData's contents came up again.
 
 Can you send me a pointer to that discussion, so I can plug into it?
 
 My bad: https://www.w3.org/Bugs/Public/show_bug.cgi?id=22680
 
 
 https://www.w3.org/Bugs/Public/show_bug.cgi?id=20887
 
 Sounds like mapping FormData to a File hasn't come up yet.
 
 That's the third thread referenced in that bug. Seems like there's not
 much actual objections. I thought implementers had concerns.


I don't think there's actual technical objections (though at one point, I did 
argue for leaving File as something that held real file objects from the 
underlying file system, and exposing a name on Blob).  But I think a File 
constructor can be added.

-- A*

Re: File constructor

[Arun Ranganathan]

On Jul 15, 2013, at 8:01 PM, Anne van Kesteren wrote:

 So exposing FormData's contents came up again.


Can you send me a pointer to that discussion, so I can plug into it?  


 The natural model is to
 have FormData be map of string to either a string or File object. This
 is very much equivalent to exposing a constructor for File to create a
 Blob with a filename. I was told people objected to that in the past.
 What were the objections and are they still relevant?
 


Here's a bug with references to the objections( which were really wondering 
what the right use case is): 
https://www.w3.org/Bugs/Public/show_bug.cgi?id=20887

Sounds like mapping FormData to a File hasn't come up yet.

-- A*

Re: Blob URLs | autoRevoke, defaults, and resolutions

[Arun Ranganathan]

On May 7, 2013, at 10:45 AM, Anne van Kesteren wrote:

 On Mon, May 6, 2013 at 11:11 PM, Jonas Sicking jo...@sicking.cc wrote:
 The only thing that's different about XHR is that the first step in my
 list lives in one function, and the other steps live in another
 function. Doesn't seem to have any effect on the discussions here
 other than that we'd need to define which of the two functions does
 the step which grabs a reference to the Blob.
 
 Fair enough. So I guess we can indeed fix this by changing
 http://fetch.spec.whatwg.org/#concept-fetch to get a reference to the
 Blob/MediaStream/... before returning early as Arun suggested.


\o/ :-)

Filed https://www.w3.org/Bugs/Public/show_bug.cgi?id=21955 

-- A*

Blob URLs | autoRevoke, defaults, and resolutions

[Arun Ranganathan]

At the recent TPAC for Working Groups held in San Jose, Adrian Bateman, Jonas 
Sicking and I spent some time taking a look at how to remedy what the spec. 
says today about Blob URLs, both from the perspective of default behavior and 
in terms of what correct autoRevoke behavior should be.  This email is to 
summarize those discussions.

Blob URLs are used in different parts of the platform today, and are expected 
to work on the platform wherever URLs do.  This includes CSS, MediaStream and 
MediaSource use cases [1], along with use of 'src='.   

(Separate discussions about a v2 of the File API spec, including use of a 
Futures-based model in lieu of the event model, took place, but submitting a 
LCWD with major interoperability amongst all browsers is a good goal for this 
draft.)

Here's a summary of the Blob URL issues:

1. There's the relatively easy question of defaults.  While the spec says that 
URL.createObjectURL should create a Blob URL which has autoRevoke: true by 
default [2], there isn't any implementation that supports this, whether that's 
IE's oneTimeOnly behavior (which is related but different), or Firefox's 
autoRevoke implementation.  Chrome doesn't touch this yet :)

The spec. will roll back the default from true to false.  At least this 
matches what implementations do; there's been resistance to changing the 
default due to shipping applications relying on autoRevoke being false by 
default, or at least implementor reluctance [1].

Switching the default to false would enable IE, Chrome, andFirefox to have 
interoperability with URL.createObjectURL(blobArg), though such a default 
places burdens on web developers to couple create* calls with revoke* calls to 
not leak Blobs.  Jonas proposes a separate method, 
URL.createAutoRevokeObjectURL, which creates an autoRevoke URL.  I'm lukewarm 
on that :-\

2. Regardless of the default, there's the hard question of what to do with Blob 
URL revocation.  Glenn / zewt points out that this applies, though perhaps less 
dramatically, to *manually* revoked Blob URLs, and provides some test cases 
[3].  

Options are:

2a. To meticulously special-case Blob URLs, per Bug 17765 [4].  This calls for 
a synchronous step attached to wherever URLs are used to peg Blob URL data at 
fetch, so that the chance of a concurrent revocation doesn't cause things to 
behave unpredictably.  Firefox does a variation of this with keeping channels 
open, but solving this bug interoperably is going to be very hard, and has to 
be done in different places across the platform.  And even within CSS.  This is 
hard to move forward with.

2b.To adopt an 80-20 rule, and only specify what happens for some cases that 
seem common, but expressly disallow other cases.  This might be a more muted 
version of Bug 17765, especially if it can't be done within fetch [5].  

This could mean that the blob clause for basic fetch[5] only defines some 
cases where a synchronous fetch can be run (TBD) but expressly disallows others 
where synchronous fetching is not feasible.  This would limit the use of Blob 
URLs pretty drastically, but might be the only solution.  For instance, 
asynchronous calls accompanying embed, defer etc. might have to be 
expressly disallowed.  It would be great if we do this in fetch [5] :-)

Essentially, this might be to do what Firefox does but document what 
dereference means [6], and be clear about what might break.  Most 
implementors acknowledge that use of Blob URLs simply won't work in some cases 
(e.g. CSS cases, etc.).  We should formalize that; it would involve listing 
what works explicitly.  Anne?

2c. Re-use oneTimeOnly as in IE's behavior for autoRevoke (but call it 
autoRevoke).  But we jettisoned this for race conditions e.g.

// This is in IE only
 
img2.src = URL.createObjectURL(fileBlob, {oneTimeOnly: true});

// race now! then fail in IE only
img1.src = img2.src;

will fail in IE with oneTimeOnly.  It appears to fail reliably, but again, 
dereference URL may not be interoperable here.  This is probably not what we 
should do, but it was worth listing, since it carries the brute force of a 
shipping implementation, and shows how some % of the market has actively solved 
this problem :)

3. We can lift origin restrictions in v2 on Blob URL; currently, one shipping 
implementation (IE) actively relies on origin restrictions, but expressed 
willingness to phase this out.  Most use cases needing Blob data across origins 
can be met without needing Blob URLs to not be origin restricted.  Blob URLs 
must be unguessable for that to happen, and today, they aren't unguessable in 
some implementations.

-- A*


[1] https://www.w3.org/Bugs/Public/show_bug.cgi?id=19594
[2] http://dev.w3.org/2006/webapi/FileAPI/#creating-revoking
[3] http://lists.w3.org/Archives/Public/public-webapps/2013AprJun/0294.html
[4] https://www.w3.org/Bugs/Public/show_bug.cgi?id=17765
[5] http://fetch.spec.whatwg.org/#concept-fetch
[6] 

Re: File API: Blob.type

[Arun Ranganathan]

On Apr 5, 2013, at 3:17 PM, Alexey Proskuryakov wrote:

 
 03 апр. 2013 г., в 13:11, Arun Ranganathan a...@mozilla.com написал(а):
 
 My only concern is that blob.type should never contain parameters.  
 Comparing it to text/plain or image/jpeg should work, and not 
 mysteriously fail a year later when somebody eventually throws a MIME type 
 parameter into the mix.  Today, all browsers expose text files at 
 text/plain.  If a browser a year from now decides to call text files with a 
 UTF-8 BOM text/plain; charset=UTF-8, it'll break interop.
 
 What specifies how a File gets its type? The only requirement I can find is 
 that User agents must not attempt heuristic determination of type, which I 
 think implies that something like inputElement.files[0].type is always  for 
 a file chosen by a user via input type=file.


The spec. now overreaches a bit :-( 

Not allowing heuristic mechanisms was merely to restrict encoding determination 
as per at lease one implementation's experience with it being substandard: 
https://bugzilla.mozilla.org/show_bug.cgi?id=848842

But now maybe we're going a bit far.  Should we standardize how UAs do 
auto-detect of file type, including something about extensions and some BOM 
methods?  This seems to be complicated and may be unnecessary -- most UAs do 
this just about right in the absence of a standard.


 Guessing MIME type from file name or metadata is always a heuristic, as not 
 all platforms will know that archive.sit means application/x-stuffit.
 
 At the same time, browsers do autodetect types for many files. We'll need to 
 autodetect when serializing a form for submission anyway, so exposing this 
 information a little earlier only makes sense.
 
 I think that these concerns can be resolved by specifying what File.type is 
 more explicitly. The spec can just say that parameters are not allowed in the 
 browser chosen type.


That seems sensible!  By *not* allowing charset parameters in types determined 
by UAs, these are now set by web applications only, which may mitigate Glenn's 
concerns.

Maybe the way forward is to leave this to UAs, and:

1. Say UAs should return file type, if known.
2. UAs must not use heuristics or statistical methods to determine encoding and
3. UAs must not set the charset parameter in the returned type for text/plain.  
This will then defer to the encoding spec. and attempt fallback decoding.  
Where a web application sets a charset parameter, this will do the right thing 
for readAsText with fallback decoding.

 
 Additionally, determining a blob's file type seems like the most obvious 
 use of this property, and making people say if(blob.type.split(;)[0] == 
 'text/plain') is simply not a good interface.
 
 
 OK -- you're strongly opinionated on the matter of NOT allowing a charset 
 parameter.  I'd like to see if implementers who had an opinion on its 
 usefulness can weigh in -- Darin?  Alexey?
 
 
 I do not have a very strong opinion. I like the simpler API of passing 
 parameters through the type attribute, as it's specified currently. This also 
 matches XMLHttpRequest API better. And of course, keeping existing behavior 
 means that we won't break the web.

I like it too.  We keep charset, but don't let user agents set it for 
auto-detected files; it can only be set with a Blob constructor or a slice 
call.  Blob.type is a string that can be set by developers and has normative 
requirements that are not strict tokenization requirements, so I think we're 
fine here.

-- A*

Re: File API: Blob.type

[Arun Ranganathan]

On Mar 19, 2013, at 8:52 PM, Glenn Maynard wrote:

 On Tue, Mar 19, 2013 at 1:41 PM, Arun Ranganathan a...@mozilla.com wrote: 
 
  2.Convert every character in relativeContentType to lower case.
 
 I recommend referencing Converting a string to ASCII lowercase in HTML.  
 http://www.whatwg.org/specs/web-apps/current-work/#converted-to-ascii-lowercase


Done.

 
  1.If relativeContentType contains any non-ASCII characters, then set 
  relativeContentType to the empty string and return from these substeps.
  3.If relativeContentType contains any line break characters like CR 
  or LF or any CTLs or separators, then set relativeContentType to the 
  empty string and return from these substeps.
 
 #3 is too vague.  I recommend combining #1 and #3, saying: If any character 
 in relativeContentType outside of the range U+0020 to U+007E.  That's the 
 printable ASCII range, and excludes all control characters.
 


Done (+ thanks).


  4.Parse relativeContentType as an RFC2616 media-type, tokenizing it 
  according to the ABNF for media-type [RFC2616] with the ASCII / character 
  separating tokens representing the type and subtype productions. If 
  relativeContentType cannot be tokenized according to the ABNF for 
  media-type [RFC2616], then set relativeContentType to the empty string and 
  return from these substeps.
 
 I'm not sure we should be this strict.  I'd lean towards keeping it simple, 
 allowing any string at all as long as it contains only lowercase, printable 
 ASCII.


Done -- we restrict it now, but don't mandate tokenization along the lines of 
RFC2616.


 You don't need to say The following requirements are normative for this 
 parameter.  That's what the normative language that follows (must) means.


Done.


 My only concern is that blob.type should never contain parameters.  Comparing 
 it to text/plain or image/jpeg should work, and not mysteriously fail a 
 year later when somebody eventually throws a MIME type parameter into the 
 mix.  Today, all browsers expose text files at text/plain.  If a browser a 
 year from now decides to call text files with a UTF-8 BOM text/plain; 
 charset=UTF-8, it'll break interop.
 
 Additionally, determining a blob's file type seems like the most obvious use 
 of this property, and making people say if(blob.type.split(;)[0] == 
 'text/plain') is simply not a good interface.


OK -- you're strongly opinionated on the matter of NOT allowing a charset 
parameter.  I'd like to see if implementers who had an opinion on its 
usefulness can weigh in -- Darin?  Alexey?

http://dev.w3.org/2006/webapi/FileAPI/

-- A*

Re: File API: why is there same-origin restriction on blob URLs?

[Arun Ranganathan]

On Mar 28, 2013, at 1:55 AM, Anne van Kesteren wrote:

 On Wed, Mar 27, 2013 at 8:16 PM, Arun Ranganathan a...@mozilla.com wrote:
 They're very different than data URLs.  What's a good use case for making 
 them cross-origin, that isn't addressed by use of postMessage?
 
 Well, the question we started with was why they're restricted to same
 origin. That does not appear to be answered yet.


The restriction was due to the (perhaps misguided?) safety assumption that it 
was prudent to restrict file references via this scheme to the origin invoking 
URL.createObjectURL.  The initial proposal was scoped to origin, and dates to 
some antiquity in email from Hixie, which said: 
 
URL in this scheme would have an intrinsic origin that is equal to the 
origin of the script context (the first script in HTML5 terms) that 
invoked the API to get the URL. This URL could then be passed around as a 
string and would be treated as a resource from the same origin as that 
script, and could thus be used with img and canvas and so on.

 http://lists.w3.org/Archives/Public/public-webapps/2009AprJun/1110.html

We can relax this requirement, since it has been pointed out that it is a bit 
restrictive, but I'd like to ensure that Bad Things won't happen. Implementor 
feedback is welcome.


 As for a use case, you could have a widget that does something given a
 URL. E.g. put Googly eyes on the image resource. This widget could be
 embedded anywhere and all you'd need to do is postMessage() it a URL.
 Requiring a special code path for URLs generated from a Blob seems
 annoying for both the consumer of the widget and the producer of it.


Glenn adds that there may need to be some special casing anyway 
(http://lists.w3.org/Archives/Public/public-webapps/2013JanMar/1038.html), with 
respect to revocation, but I'm open to relaxing the origin restriction if 
assumptions about this being prudent prove to be invalid.

-- A*

Re: File API: why is there same-origin restriction on blob URLs?

[Arun Ranganathan]

On Mar 26, 2013, at 8:30 PM, Jonas Sicking wrote:

 On Tue, Mar 26, 2013 at 2:17 PM, Anne van Kesteren ann...@annevk.nl wrote:
 Hi,
 
 Is there any particular reason why we restrict blob URLs to the same
 origin as the script that created them? In effect they are pretty much
 like capability URLs (containing an unguessable token). So if someone
 decides to share one, that should be okay I think. This would be
 useful in the context of sandboxed code (iframe sandbox) and
 presumably elsewhere too.
 
 I think the original concern was that implementations might not be
 able to reliably generate unguessable URLs. Potentially that's
 something that we could require though.


We already require this -- opaque strings should be globally unique.  


 
 However we'd still need to nail down what the new behavior should be.
 Should it behave like data: URLs? The main advantage of those is that
 implementations still don't agree on how those should behave.


They're very different than data URLs.  What's a good use case for making them 
cross-origin, that isn't addressed by use of postMessage?

-- A*

Re: File API for Review

[Arun Ranganathan]

On Mar 25, 2013, at 1:52 PM, Anne van Kesteren wrote:

 On Mon, Mar 25, 2013 at 5:45 PM, Arun Ranganathan
 aranganat...@mozilla.com wrote:
 So just to be clear, do you think we should remove 500-style responses 
 altogether, and *completely defer* to network error, which essentially 
 involves throwing on expired / revoked / invalid Blob URLs?  I'm ok with 
 that if so.
 
 Yeah, I would prefer if we used network error consistently
 throughout the platform for something went wrong with this URL
 rather than try to re-imagine a non-HTTP situation in terms of HTTP.
 Because it gives nicer to work with events and such out of the box and
 because it makes exposing more detailed information in the future
 easier.


Done.

http://dev.w3.org/2006/webapi/FileAPI/#NetworkError

-- A*

Re: File API for Review

[Arun Ranganathan]

- Anne vK said:  -
 On Tue, Mar 19, 2013 at 5:57 PM, Arun Ranganathan a...@mozilla.com
 wrote:
  On Feb 13, 2013, at 11:37 AM, Anne van Kesteren wrote:
  We're not actually leaving what exactly to return open to
  implementors.  They *must* return a 500.  They *may* additionally
  provide a message, which is akin to console messages.
 
 No, an HTTP status message is an actual thing and exposed via
 XMLHttpRequest.


You're right; this should be defined better in the File API.


 
   Also, networkerror isn't really strongly defined; XMLHttpRequest
   uses this to throw on network errors (NetworkError), and there
   isn't currently a Fetch in HTML that leverages networkerrors.
This is not obviously reusable here in the blob: URL context.
 
 I think it is. Network error is for whenever something goes wrong
 with
 the request. Otherwise this would give a load event rather than an
 error event. data: URLs use network errors too now:
 http://xhr.spec.whatwg.org/#data:-urls-and-http


So just to be clear, do you think we should remove 500-style responses 
altogether, and *completely defer* to network error, which essentially involves 
throwing on expired / revoked / invalid Blob URLs?  I'm ok with that if so.

-- A*

Re: File API for Review

[Arun Ranganathan]

On Feb 13, 2013, at 11:37 AM, Anne van Kesteren wrote:

 
 Specification bugs still exist
 
 I think we should rename URI to URL. That's what everyone is converging on.
 


Done.


 I'm also not convinced that leaving what exactly to return in the HTTP
 scenario open to implementors is a good thing. 


(There isn't an HTTP scenario here).


 So what I actually think we should do here is treat this as a networkerror. 
 XMLHttpRequest already knows about that concept and every other end point 
 also deals with network errors in a predictable and standardized way. 
 Phrasing such as Act as if a network error occurred seems sufficient for 
 now (until Fetch provides hooks).


We're not actually leaving what exactly to return open to implementors.  They 
*must* return a 500.  They *may* additionally provide a message, which is akin 
to console messages.  Also, networkerror isn't really strongly defined; 
XMLHttpRequest uses this to throw on network errors (NetworkError), and there 
isn't currently a Fetch in HTML that leverages networkerrors.  This is not 
obviously reusable here in the blob: URL context.

The spec. doesn't need to mention anything about throwing -- we're returning a 
500 if a blob: URL cannot be dereferenced, not throwing an exception.  This is 
specific, and not left to implementations to determine.


 Just like HTML, CSS, etc. this specification should defer to
 http://encoding.spec.whatwg.org/ for its encoding related
 requirements.


Done (and uses most of your proposed changes): 
http://dev.w3.org/2006/webapi/FileAPI/#encoding-determination


 
 I don't think we should throw for limitations on URL length. We always
 leave undefined lengths unaddressed in specifications, including with
 regards to how to handle them.
 


Done.

-- A*

Re: File API: Blob.type

[Arun Ranganathan]

On Mar 7, 2013, at 7:19 PM, Glenn Maynard wrote:


 Chrome, at least, throws on new Blob([], {type: 漢字}), as well as 
 lowercasing the string.
 


Stricter rules are in place for type both while constructing Blob and for 
slice calls:

http://dev.w3.org/2006/webapi/FileAPI/#constructorBlob

and 

http://dev.w3.org/2006/webapi/FileAPI/#slide-method-algo

I agree with previous comments you've made about ByteString not solving any 
problems that Anne vK brings up; instead, I think using DOMString is probably 
ok, with tighter rules on what is valid and what should be ignored.  Throwing a 
SyntaxError might be overkill to developers and a bit too punitive; instead, I 
advocate sticking with the original spirit of the opaque string idea and 
ignoring bad use of type.


 A couple points:
 
 - I disagree that we should discourage comparing against Blob.type, but 
 ultimately it's such an obvious use of the property, people will do it 
 whether it's encouraged or not.  I'd never give it a second thought, since 
 that appears to be its very purpose.  Web APIs should be designed defensively 
 around how people will actually use the API, not how we wish they would.  
 Unless lots of Blob.type parameters actually include parameters, code will 
 break unexpectedly when it ends up encountering one.
 - The RFC defines a protocol (Content-Type), not a JavaScript API, and a 
 good protocols are rarely good APIs.  Having Blob.type be the literal value 
 of a Content-Type header isn't an elegant API.  You shouldn't need to do 
 parsing of a string value to extract text/plain, and you shouldn't have to 
 do serialization to get text/plain; charset=UTF-8.
 

So the type attribute of a Blob object isn't the *literal* value of the 
header; it's the type of the Blob, expressed as a MIME type.  When 
dereferencing Blob URLs, you get this type back with the Content-Type header, 
as you do normally in HTTP scenarios.  This is a well-understood behavior, and 
I agree with points you've made about not being beholden to the RFC when 
designing an API.  

I think the question here is whether or not to include *separate attributes* on 
the Blob interface for the rarely used Charset Parameter, namely anything after 
the semicolon in MIME types of the sort: text/plain;charset=UTF-8.  I've 
considered all your arguments by way of developer advocacy, and actually think 
we'll do developers a disservice by adding to the Blob interface:

1. The Charset Parameter consideration applies only to text/plain.  There are 
numerous other MIME types that don't use it: application/*, audio/*, image/*, 
video/*, etc.  Complicating the interface on the off-chance that a stray use of 
the Charset parameter breaks a direct equality comparison is too much API for 
too little.

2. The Charset Parameter even in the context of text/plain isn't common enough 
to warrant a special case for text/plain within the API.

3. In general, it's a pretty stable assumption to conclude that developers will 
expect type to be surfaced later along with Content-Type when dereferencing 
a Blob URI.  I don't think we've made an assumption that's terribly galling.

-- A*

Re: File API: Blob.type

[Arun Ranganathan]

Alexey,


On Mar 7, 2013, at 3:02 PM, Alexey Proskuryakov wrote:

 
 The current File API spec seems to have a mismatch between type in 
 BlobPropertyBag, and type as Blob attribute. The latter declaratively states 
 that the type is an ASCII lower case string. As mentioned by Glenn before, 
 WebKit interpreted this by raising an exception in constructor for non-ASCII 
 input, and lowercasing the string. I think that this is a reasonable reading 
 of the spec. I'd be fine with raising exceptions for invalid types more 
 eagerly.
 
 This is the text in question:
 
 (1)
 type, a DOMString which corresponds to the Blob object's type attribute. If 
 not the empty string, user agents must treat it as an RFC2616 media-type 
 [RFC2616], and as an opaque string that can be ignored if it is an invalid 
 media-type. This value must be used as the Content-Type header when 
 dereferencing a Blob URI.
 
 
 
 (2)
 type
 The ASCII-encoded string in lower case representing the media type of the 
 Blob, expressed as an RFC2046 MIME type [RFC2046]. On getting, conforming 
 user agents must return the MIME type of the Blob, if it is known. If 
 conforming user agents cannot determine the media type of the Blob, they 
 must return the empty string. A string is a valid MIME type if it matches 
 the media-type token defined in section 3.7 Media Types of RFC 2616 
 [RFC2616]. If not the empty string, user agents must treat it as an RFC2616 
 media-type [RFC2616], and as an opaque string that can be ignored if it is 
 an invalid media-type. This value must be used as the Content-Type header 
 when dereferencing a Blob URI.


This is now clarified; the mismatch is a spec. bug.  Thanks for pointing this 
out.


 It would be helpful to have the terminology corrected, and to have this 
 generally clarified - for example, validity is mentioned here, but seems to 
 be unused.
 


Conditions for validity have been clarified; this doesn't warrant throwing a 
SyntaxError, but it does specify when implementations should ignore poor use of 
MIME type strings, e.g. here's additional clarification in the slice call:

http://dev.w3.org/2006/webapi/FileAPI/#slide-method-algo


 It seems pretty clear from normative text that charset parameter is supposed 
 to work. A non-normative example supports that too. I agree with Arun that 
 this seems best to keep as is.

+1.


 However, https://bugs.webkit.org/show_bug.cgi?id=111380 is about a 
 different case - it's about posting multipart form data that has Blob 
 elements with invalid media-types. I'm not even sure which spec is in charge 
 of this behavior - I don't think that anything anywhere says that Blob.type 
 affects media-type of posted multipart data, even though that's obviously the 
 intention. XMLHttpRequest spec defers to HTML, which defers to RFC2388, which 
 mentions files returned via filling out a form, but not Blobs (which is no 
 surprise given its age).


In fact, I'm not sure if Blob.type should influence the type of multipart form 
data.  Consider the concatenation of several Blobs into a new Blob, as the Blob 
constructor allows.  What should the type of a newly constructed Blob be,  if 
it consists of several differently typed Blobs?  The spec. suggests 
disregarding the type of each Blob, but encourages the right use of type within 
the Blob constructor.  

I'm also not sure multipart form data falls under the aegis of the File API, 
but at least Blobs with invalid types is the same us having no type now (empty 
string).


 Making Blobs only hold valid media-types would solve practical issues, but it 
 would be helpful to know what formally defines multipart data serialization 
 with blobs.
 
 We also previously had 
 https://bugs.webkit.org/attachment.cgi?id=177736action=review for sending 
 non-multipart data. Back then, we determined that Content-Type:  should be 
 sent when the value is invalid. I'm no longer sure if that's right. For this 
 case, XMLHttpRequest authoritatively defines the behavior, although heavily 
 leaning on File API to decide when the type attribute is empty:
 
 If the object's type attribute is not the empty string let mime type be its 
 value.
 
 
 Note that mime type is then directly used as default media-type for 
 Content-Type header, but it's not parsed to set encoding variable. The 
 encoding could be needed to update a charset in author provided Content-Type 
 header field in later steps of the algorithm. This is probably not right, as 
 Blob should know its encoding better than code that sets header fields on an 
 XMLHttpRequest object.
 


Yes, but implementations can't heuristically determine a Blob's type now.  Type 
has to be specified correctly or ignored.   What Blob should know is now as 
good as what it is constructed to have as its type, though at read time, thanks 
to the Encoding Spec, we can determine a fallback encoding.

-- A*

Re: File API for Review

[Arun Ranganathan]

On Mar 8, 2013, at 8:28 AM, Henri Sivonen wrote:

 Additionally, I think http://www.w3.org/TR/FileAPI/#dfn-type should
 clarify that the browser must not use statistical methods to guess the
 charset parameter part of the type as part of determining the type.
 Firefox currently asks magic 8-ball, but the magic 8-ball is broken.
 AFAICT, WebKit does not guess, so I hope it's possible to remove the
 guessing from Firefox.
 
 (The guessing in Firefox relies on a big chunk of legacy code that's
 broken and shows no signs of ever getting fixed properly. The File API
 is currently the only thing in Firefox that exposes the mysterious
 behavior of said legacy code to the Web using the default settings of
 Firefox, so I'm hoping to remove the big chunk of legacy code instead
 of fixing it properly.)


I think we can jettison Fx's use of magic 8-ball code with this update:

http://dev.w3.org/2006/webapi/FileAPI/#dfn-type

Additional spec'ing on when to ignore the string if provided incorrectly will 
follow.

-- A*

Re: File API: Blob.type

[Arun Ranganathan]

On Mar 6, 2013, at 7:42 PM, Glenn Maynard wrote: 

On Wed, Mar 6, 2013 at 8:29 AM, Anne van Kesteren ann...@annevk.nl wrote: 
On Wed, Mar 6, 2013 at 2:21 PM, Glenn Maynard gl...@zewt.org wrote: 
 Blob.type is a MIME type, not a Content-Type header. It's a string of 
 codepoints, not a series of bytes. XHR is a protocol-level API, so maybe it 
 makes sense there, but it doesn't make sense for Blob. 

 It's a Content-Type header value and should have those restrictions. 

 It's not a Content-Type header, it's a MIME type. That's part of a 
 Content-Type header, 
 but they're not the same thing. 

In fact, the intent is that the value of Blob.type is reflected in the 
Content-Type, and that setting Blob.type means that when fetching that Blob as 
a blob: you'll get the value of Blob.type in the Content-Type header. This 
model *did* allow for charset params -- it always has (perhaps not advertised, 
but it always has). 

At some point there was a draft that specified *strict* parsing for compliance 
with RFC2046, including tokenization (/) and eliminating non-ASCII cruft. But 
we scrapped that because bugs in all major browser projects showed that this 
spec. text was callously ignored. And I didn't want to spec. fiction, so we 
went with the current model for Blob.type, which is, as Anne points out, pretty 
lax. 

That doesn't make sense. Blob.type isn't a string of bytes, it's a string of 
Unicode codepoints that happens 
 to be restricted to the ASCII range. Applying WebKit's validity checks 
 (with the addition of disallowing nonprintable characters) will make it have 
 the restrictions you want; 
 ByteString has nothing to do with this. 

I'm in favor of introducing stricter rules for Blob.type, and I'm also in favor 
of allowing charset params; Glenn's example of 'if(blob.type == text/plain)' 
will break, but I don't think we should be encouraging strict equality 
comparisons on blob.type (and in fact, should *discourage* it as a practice). 

But I'm not sure about why we'd choose ByteString in lieu of being strict with 
what characters are allowed within DOMString. Anne, can you shed some light on 
this? And of course we should eliminate CR + LF as a possibility at constructor 
invocation time, possibly by throwing. 

Glenn: I think that introducing a separate interface for other parameters 
actually takes away from the elegance of a simple Blob.type. The RFC doesn't 
separate them, and I'm not sure we should either. My reading of the RFC is that 
parameters *are an intrinsic part of* the MIME type. 

-- A* 

- Original Message -

 On Wed, Mar 6, 2013 at 8:29 AM, Anne van Kesteren  ann...@annevk.nl
  wrote:

  On Wed, Mar 6, 2013 at 2:21 PM, Glenn Maynard  gl...@zewt.org 
  wrote:
 
   Blob.type is a MIME type, not a Content-Type header. It's a
   string
   of
 
   codepoints, not a series of bytes. XHR is a protocol-level API,
   so
   maybe it
 
   makes sense there, but it doesn't make sense for Blob.
 

  It's a Content-Type header value and should have those
  restrictions.
 

 It's not a Content-Type header, it's a MIME type. That's part of a
 Content-Type header, but they're not the same thing.

 But String vs. ByteString has nothing to do with the restrictions
 applied to it.

  Making it a ByteString plus additional restrictions will make it do
  as
 
  required.
 

 That doesn't make sense. Blob.type isn't a string of bytes, it's a
 string of Unicode codepoints that happens to be restricted to the
 ASCII range. Applying WebKit's validity checks (with the addition of
 disallowing nonprintable characters) will make it have the
 restrictions you want; ByteString has nothing to do with this.

 On Wed, Mar 6, 2013 at 11:47 AM, Darin Fisher  da...@chromium.org 
 wrote:

  So the intent is to allow specifying attributes like charset?
  That
  sounds useful.
 
 I don't think so. This isn't very well-defined by RFC2046 (it seems
 vague about the relationship of parameters to MIME types), but I'm
 pretty sure Blob.type is meant to be only a MIME type, not a MIME
 type plus content-type parameters. Also, it would lead to a poor
 API: you could no longer simply say 'if(blob.type == text/plain)';
 you'd have to parse it out yourself (which I expect nobody is
 actually doing).

 Other parameters should have a separate interface, eg.
 blob.typeParameters.charset = UTF-8, if we want that.

 --
 Glenn Maynard

Re: File API for Review

[Arun Ranganathan]

Anne,


 On Wed, Feb 6, 2013 at 7:58 PM, Arun Ranganathan a...@mozilla.com
 wrote:
  3. Progress events have been clarified.
 
 You're still using the old IDL syntax for event handlers.
 


Fixed.



 I think we should rename URI to URL. That's what everyone is
 converging on.


Fixed.


 I'm also not convinced that leaving what exactly to return in the
 HTTP
 scenario open to implementors is a good thing. We've been through
 such
 things before and learned that handwaving is bad. Lets just pick
 something.


Just to be clear, are you referring to the 500 Error Condition for Blob URLs?  
If so, the only handwaving is about the text of the error message.  I'm happy 
to tighten even this.



 Just like HTML, CSS, etc. this specification should defer to
 http://encoding.spec.whatwg.org/ for its encoding related
 requirements.


I fully agree that what we've currently got, which favors a heuristic guessing 
model for encoding, and forces UTF-8 in a void, isn't sufficient.  AND I agree 
that the encoding spec. is much more detailed.  But what exactly does deferring 
to it entail?

Right now, the specification encourages user agents to get encoding from:

1. The encoding parameter supplied with the readAsText.
2. A byte order detection heuristic, if 1. is missing.
3. The charset component of Blob.type, if provided and if 1. and 2. yield no 
result.
4. Just use utf-8 if 1, 2, and 3 yield no result.

Under the encoding spec., it returns failure if encoding isn't valid, and it 
returns failure if the BOM check fails.  So should the spec. say something 
about throwing?


 
 I don't think we should throw for limitations on URL length. We
 always
 leave undefined lengths unaddressed in specifications, including with
 regards to how to handle them.


OK.

-- A*

Re: Please add constructors for File and FileList

[Arun Ranganathan]

Greetings Victor!


On Dec 10, 2012, at 12:02 PM, Victor Costan wrote:

 Dear Web Applications WG,
 
 1) Please add a File constructor.


This has cropped up a few times :)  I've logged a spec bug for this feature: 
https://www.w3.org/Bugs/Public/show_bug.cgi?id=20887

Could you flesh out your original use case a bit more?  As currently expressed, 
it sums up to I could write better unit tests which doesn't constitute what 
I'm looking for in a use case.

It strikes me that the chief ask is to bind a Blob to name.  This would make 
life simpler with FormData and with existing server applications.  It's been 
pointed out that the barrier between Blob and File is pretty thin, and I'm open 
to jettisoning the thought pattern that we should think of File objects as 
strictly filesystem backed (on disk).  So, what if we allowed the Blob 
constructor to take a name also?  This might allow Blobs to fall into the 80 
of the 80-20 rule :)  

Could you update the bug (or this listserv) with better use cases?  

I'm a bit less upbeat about:


 2) Please add a FileList constructor.
 
 What I really want is some way to add files to an input
 type=file listed in
 http://wiki.whatwg.org/wiki/New_Features_Awaiting_Implementation_Interest
 
 I think that one reasonable way to get there is to have a FileList
 constructor that takes an array of File instances, and have the
 ability to assign that FileList to the files attribute of the input.
 This avoids making FileList mutable.
 
 This would also help me write better tests for my code. Currently,
 input type=file is the only form field whose value can't be set
 inside JavaScript, so I can't write automated tests for input
 type=file-related code.
 
 Asides from improving testing, this would allow me to implement the
 following _easily_:
 
 * filters for uploaded content (e.g. resize a big image before uploading)
 * saving the file a user selected in an IndexedDB, and loading it
 back into the input type=file if the page is accidentally
 refreshed
 
 These features can be implemented without FileList support, but they
 impact the application's design. For example, filters can be
 implemented using FormData from XMLHttpRequest level 2 to upload
 Blobs, but switching from plain form submission to XHR-based
 submission can impact other parts of the application code, whereas
 changing the files behind the input would be a localized change.


As it stands currently, a FileList object typically stems from user action.  
For instance, interacting with the file picker or with drag and drop.  In 
general, this model is desirable for security reasons -- the legacy HTML form 
and file picker ensures that web applications aren't doing things they 
shouldn't be doing with the underlying file system without user participation. 
I don't actually think it is desirable to modify input type = file… from 
within JavaScript.

I'd like a better understanding of the broader implications of a FileList 
constructor, just out of abundant caution.  I have similar concerns about 
https://www.w3.org/Bugs/Public/show_bug.cgi?id=17125 which is also filed on the 
FileList object, but these seem easier to manage.

Hope to hear back,

-- A*

File API for Review

[Arun Ranganathan]

Greetings WebApps WG!

Review on the File API is encouraged:

http://dev.w3.org/2006/webapi/FileAPI/

A few substantial changes that might need particular attention before we 
initiate a call for LCWD or something comparably official:

1. autoRevoke behavior of Blob URIs has changed.  Previous drafts made the 
autoRevoke behavior on by default (by consensus), but didn't harness revocation 
to a suitable and unambiguous scope.  Thanks to a fix to HTML[1] we can 
leverage the global script clean-up jobs list, to which we add revocation of 
Blob URIs.  Blob URIs are thus either scoped to the next time global script 
clean-up jobs are processed (by default), OR when document unloading steps are 
processed if the developer opts out of the default but never pairs it with a 
call to URL.revokeObjectURL, OR when URL.revokeObjectURL is called.

In particular, this behavior defers from shipping implementations such as IE10. 
 This is possibly the biggest change:

http://dev.w3.org/2006/webapi/FileAPI/#lifeTime

http://dev.w3.org/2006/webapi/FileAPI/#creating-revoking

2. An utility to smooth line ending variations (Unix vs. Windows) has been 
added, but is an orphan interface currently.  Nobody's fussed over this, and it 
might not be a problem at all, but I'd like to draw your attention to it :)  
While currently only relevant for DOMString arguments to the Blob constructor, 
we might work with the utility to add other arguments (including 
ArrayBufferViews, etc.).

http://dev.w3.org/2006/webapi/FileAPI/#convenienceAPI

3. Progress events have been clarified.

4. readAsDataURL currently makes progress notifications throughout the read, 
but only returns a non-null return value at the end of the read.

http://dev.w3.org/2006/webapi/FileAPI/#readAsDataURL

Specification bugs still exist, and this draft doesn't address all open bugs 
[2]; further bugs and issues, if any, should be logged or discussed on this 
listserv.  The remaining bugs present use cases that aren't major enough to 
warrant feature extensions.

-- A*

[1] https://www.w3.org/Bugs/Public/show_bug.cgi?id=19554

[2] 
https://www.w3.org/Bugs/Public/buglist.cgi?product=WebAppsWGcomponent=File%20APIresolution=---list_id=4913

Re: [File API] About Partial Blob Data, XHR and Streams API

[Arun Ranganathan]

On Jan 22, 2013, at 9:42 AM, Cyril Concolato wrote:

 Hi Arun,
 
 Le 22/01/2013 15:04, Arun Ranganathan a écrit :
 Hi Cyril,
 
 
 1) I'm wondering why in progressive mode, does the spec say:  partial Blob 
 data is an ArrayBuffer [TypedArrays] object consisting of the bytes loaded 
 so far. Why isn't it the bytes loaded since the previous progress event? 
 
 AR: It is always a new ArrayBuffer.  The phraseology so far could be 
 replaced by bytes loaded since the previous progress event though I'm not 
 always sure that will be the case. 
 I understood from Jonas' answer that it was a new ArrayBuffer. What remained 
 unclear was the content of the ArrayBuffer: all the data from the beginning 
 of the read operation (which was problematic), or only the data read since 
 the previous progress event (which I prefer). If, as you say, this is latter 
 option, then please fix the spec. as so far is very ambiguous.


Hi Cyril: the model you propose, of deltas since the last read, is actually 
*not* what the spec's intent is.  Your model does have advantages, but released 
software follows the former model, namely *all* the data since the beginning of 
the read operation.  The partial results consist of all the bytes, starting 
from the beginning, but perhaps not the last byte, depending on progress event 
and file size.

I'll have to clarify the prose, since from your question, it is clear that this 
isn't well described.

-- A*

Re: [File API] About Partial Blob Data, XHR and Streams API

[Arun Ranganathan]

Hi Cyril, 

1) I'm wondering why in progressive mode, does the spec say:  partial Blob 
data is an ArrayBuffer [ TypedArrays ] object consisting of the bytes loaded so 
far . Why isn't it the bytes loaded since the previous progress event? 

AR: It is always a new ArrayBuffer. The phraseology so far could be replaced 
by bytes loaded since the previous progress event though I'm not always sure 
that will be the case. 

 In my use case, the binary data resource might have an infinite size,
 in which case the result objects will grow forever.
 I looked at the Streams API [1] to see if there was any difference
 for that but I couldn't see any. Reading with the FileReader
 interface a Stream (dynamic length) or a Blob (fixed length) seems
 to always return the whole content.
AR: Here, do you mean, you never get a progressevent other than load and 
loadend in your tests? Certainly, if you had binary data of infinite size, 
you'll get  several result objects. The file API, particularly 
FileReader, shouldn't be used in streaming scenarios. 

-- A* 

Re: [admin] Publishing specs before TPAC: CfC start deadline is Oct 15

[Arun Ranganathan]

- Original Message -
 On 10/9/12 4:13 PM, ext Arun Ranganathan wrote:
  On Sep 26, 2012, at 10:27 AM, Arthur Barstow wrote:
 
  * File API - Arun can you get this spec ready for LC by October
  15?
 
  Yes.
 
 ATM, File API has 8 open bugs [1]. 


I've rationalized these down to 2 bugs.  

1. Bug 17125[1] is a feature that we should mark v.next; it calls for the 
ability to retroactively unselect an erroneous selection that is present in 
FileList.  I don't think this is a pressing feature.

2. Bug 19554[2] is a spec. request feature in WHATWG/HTML5, especially useful 
for autoRevoke semantics for Blob URLs.  I'm not sure whether autoRevoke is at 
risk because of this bug, since implementations have shown that stable state 
(what the spec. uses now) is problematic for autoRevoke.  But I'll let this be 
discussed in LC commentary or in upcoming TPAC discussions.

P.S. Dom's WebIDL checker reports a bug in the Blob constructor 

Thanks for catching that :)  I think that particular bug is fixed.

And I'm sorry this is a day late (e.g. not ready by Oct. 15).  I've gotten it 
PubReady, using a push out date of Oct. 18.  Hope that works: 
http://dev.w3.org/2006/webapi/FileAPI/


-- A*

[1] https://www.w3.org/Bugs/Public/show_bug.cgi?id=17125
[2] https://www.w3.org/Bugs/Public/show_bug.cgi?id=19554

Re: [admin] Publishing specs before TPAC: CfC start deadline is Oct 15

[Arun Ranganathan]

On Sep 26, 2012, at 10:27 AM, Arthur Barstow wrote:

 * File API - Arun can you get this spec ready for LC by October 15?

Yes.

-- A*

Re: [FileAPI] Blob constructor should probably take a sequence, not an IDL array object

[Arun Ranganathan]

On Sep 11, 2012, at 1:07 AM, Cameron McCormack wrote:

 Arun Ranganathan:
 I've pinged heycam to see if this is a proper use of the sequence type.  I'm 
 not sure it allows for such a variation in parameters.
 
 I agree with Boris, it makes sense to use sequence here.  Whenever you just 
 want to take a list of values in an operation argument, and you don't want to 
 keep a reference to a platform array object, you should use a sequence.


Done. http://dev.w3.org/2006/webapi/FileAPI/#dfn-Blob


-- A*

Re: [FileAPI] blob: protocol need a content-length header

[Arun Ranganathan]

Benjamin,

I filed the following:

https://www.w3.org/Bugs/Public/show_bug.cgi?id=18548

I think we should add Content-Length.

-- A*

On Aug 12, 2012, at 5:41 PM, Benjamin BERNARD wrote:

 I build de demo script (for firefox) here : 
 http://experiments.benvii.com/blob_content_length/
 You will also notice that the player's load event isn't called.
 
 Content-Length should be added to firefox (maybe open a ticket on 
 https://bugzilla.mozilla.org/) but it should also be recommended in the 
 section 11.7.3. Request and Response Headers of the spec.
 
 Thanks for responding.
 
 Benjamin BERNARD
 
 
 Le 12/08/2012 21:23, Jonas Sicking a écrit :
 On Sun, Aug 12, 2012 at 2:56 AM, Benjamin BERNARD
 benjamin.bern...@benvii.com  wrote:
 Hi,
 I was developing an offline music web App when I discover that is no
 Content-length header specified here :
 http://www.w3.org/TR/FileAPI/#ProtocolExamples
 So when you play an audio/video file stored as a blob (under a blob URI)
 it's considered by the player as streaming content which means you can't get
 the duration of a song for instance (it has an infinite duration). I think
 it might be the consequence of not providing a content-length header.
 
 I experienced it using Firefox I heard Internet Explorer already provide
 this header.
 
 Moreover, I don't understand why there is no content-length header
 recommended in the spec because when you use URL.createObjectURL(blob), blob
 has a finished size (correct me if I'm wrong). So a content-length header
 should also be provided and recommended in the spec.
 Yes, I agree, we should have a content-length header similar to the
 content-type header.
 
 In Gecko things are a bit complicated because we don't have headers on
 anything but http channels. But we do have the concept of a length of
 a response for all channels so that should take care of it. Not sure
 off the top of my head why it doesn't. Filing a bug with an example
 would be great.
 
 / Jonas
 

Re: [File API] File behavior under modification

[Arun Ranganathan]

On Jul 11, 2012, at 10:02 PM, Glenn Maynard wrote:

 
 What's the main problem with it being nullable?  A fabricated date seems 
 strange, but instead of being nullable we could spec what the fabricated date 
 is.  I'm just not totally sure what the pros and cons are here.
 
 If you call d.getYear() and d is null, you get an exception, which has a 
 very high chance of breaking the app if it's not checked for.  Giving an 
 arbitrary (but well-defined) default is much less likely to break things that 
 badly.  This is much more important when the null case is rare, because 
 nobody is going to remember to test for it.
  
 Can you log a bug so that I can provide guidance for this in spec?
 
 Sorry, but to avoid rereading the thread, a bug for what exactly?  Do you 
 need anything beyond https://www.w3.org/Bugs/Public/show_bug.cgi?id=17746?

Bug 17746 is for better definitions of snapshots.

I minted Bug 17762 (https://www.w3.org/Bugs/Public/show_bug.cgi?id=17762) to 
replace a nullable Date with something better.  Strawperson suggestions welcome 
for what this arbitrary date should be, or else I may arbitrarily choose an 
unlikely Towel Day in the past (http://towelday.org/ -- don't panic).

 
-- A*

Re: [File API] File behavior under modification

[Arun Ranganathan]

Glenn:


On May 21, 2012, at 9:44 PM, Glenn Maynard wrote:

 On Mon, May 21, 2012 at 6:05 PM, Eric U er...@google.com wrote:
 According to the latest editor's draft [1], a File object must always
 return an accurate lastModifiedDate if at all possible.
 On getting, if user agents can make this information available, this
 MUST return a new Date[HTML] object initialized to the last modified
 date of the file; otherwise, this MUST return null.
 
 However, if the underlying file has been modified since the creation
 of the File, reads processed on the File must throw exceptions or fire
 error events.
 ...if the file has been modified on disk since the File object
 reference is created, user agents MUST throw a NotReadableError...
 
 (I wish this spec would remove the screaming MUSTs; HTML doesn't do this 
 anymore, and it's so much easier to read.)

WHAT?

Heh :)  Point well taken.  I've muted the musts.


  
 These seem somewhat contradictory...you can always look at the
 modification time and see that it's changed, but if you try to read it
 after a change, it blows up.
 The non-normative text about security concerns makes me think that
 perhaps both types of operations should fail if the file has changed
 [... guarding against modifications of files on disk after a
 selection has taken place].  That may not be necessary, but if it's
 not, I think we should call it out in non-normative text that explains
 why you can read the mod time and not the data.
 
 I think lastModifiedDate should never change.  It should be the mtime of the 
 version of the file that the File represents a snapshot of.
 
 This avoids synchronicity issues: reading the value twice in a row in the 
 same script should never give a different value or throw the second time but 
 not the first, because that exposes the multithreading nature of the 
 filesystem.  
 
 It also avoids implying that this attribute needs to perform synchronous I/O 
 in the middle of script execution to check the file's current timestamp, 
 which of course should never happen.
 
 If we want to allow querying whether the mtime of a file has changed, it 
 should be done with a new asynchronous API.  I'm not sure that's needed, 
 though, since you can always just read one byte from the file; if it fails, 
 the file has changed.
 

I agree that making snapshotting clearer might be a good idea. 

It is true that reading size and lastModifiedDate are synchronous, but this 
seemed a small trade-off compared to data reads.  

My instinct is that an asynchronous API for mtime is overkill.


 (While I'm thining about it, does lastModifiedDate really need to be 
 nullable?  Systems without file timestamps are so rare that it's probably 
 better to require them to fabricate a date, so we don't introduce bugs into 
 people's code for rare cases.)

What's the main problem with it being nullable?  A fabricated date seems 
strange, but instead of being nullable we could spec what the fabricated date 
is.  I'm just not totally sure what the pros and cons are here.


 
 This came up in https://bugs.webkit.org/show_bug.cgi?id=86811; I
 believe WebKit is currently noncompliant with this part of the spec,
 and we were debating the correct behavior.  Currently WebKit delays
 grabbing the modification time of the file until it's been referenced
 by a read or slice(), so it won't notice modifications that happen
 between selection and read.
 
 This sounds very wrong to me.  If I open a File in a page (eg. select it with 
 an input), the model is that I'm giving it access to the file as it was at 
 the time I dragged it in.  If the snapshot is delayed until the first read, 
 the page will be able to see changes made later, as long as it doesn't touch 
 the file immediately.  That breaks the whole security model.
  

Strong +1.  


 Also, slice() is a synchronous API, so it should never cause blocking file 
 I/O.  That's fundamental to the API.
 
 That was done because the slice creates a
 File object reference, but in my reading creating the File referring
 to the file should be the time of the snapshot, not creating a Blob
 referring to a File.
 
 FWIW, agreed.
 

Can you log a bug so that I can provide guidance for this in spec?

-- A*

Re: [File API] File behavior under modification

[Arun Ranganathan]

Glenn,


On May 22, 2012, at 11:48 AM, Glenn Maynard wrote:

 On Tue, May 22, 2012 at 1:41 AM, Kinuko Yasuda kin...@google.com wrote:
 In my understanding WebKit's behavior is querying the metadata / reading the 
 content as lazy as possible, partly because the spec was/is ambiguous 
 (especially about when the file metadata should be captured) and also we 
 didn't want to break existing Web applications.
 
 But the synchronicity of the current .size, .modificationTime and .slice() 
 implementation has been a big headache and I'm all happy if we can make the 
 behavior clearer.
 
 The whole point of the Blob API is to never, ever do file I/O synchronously 
 in the UI thread.  (I think the read into memory at the time a read 
 operation is initiated bit is what's confusing things.  I don't think it was 
 intended to mean take the snapshot at the first read operation, but that's 
 what it sounds like.)  Some other issues:
 


snip /


 - The access restrictions applied to File also need to be clearly applied to 
 Blobs created with slice().
 - modified on disk since the File object reference is created seems 
 incorrect.  If you structured clone a File (eg. postMessage), you create a 
 new File object.  That shouldn't refresh the snapshot; you should still 
 only be able to access the file as it was when you received the *original* 
 File.
 
 I'd suggest defining this more precisely, and at the level of Blob.  Here's 
 an attempt:
 
 - In section 6, add: Each Blob has a snapshot state, which is initially set 
 to the state of the underlying storage.  Note: this value is preserved 
 through structured clone.
 - Also in section 6, add: If, at the time of any read operation on the Blob, 
 the state of the underlying storage containing the Blob is not equal to 
 snapshot state, the read must fail with a NotReadableError.
 - In 6.3.2. The slice method, add: Let the snapshot state of A be the 
 snapshot state of O.  (This means the snapshot state follows through slices.)
 - In 7.1, remove For synchronous reads   The requirement to fail is now 
 defined by section 6, and the way it fails (passing on the NotReadableError 
 or an error event) should be defined by the associated read operation.
 
 This stores a conceptual snapshot of the underlying storage at the time the 
 Blob (or File) is created.  The snapshot state is purely conceptual, 
 representing the logical underlying snapshot that the Blob represents.  If it 
 represents a file on disk, and the file changes (or is deleted), it has a new 
 snapshot state.
 
 This approach puts the requirement at the Blob level.  For Blobs whose 
 underlying storage can never be modified (regular, old-fashioned blobs), it 
 has no effect, since the snapshot state never changes.  The snapshot is made 
 when the a Blob (or File) is constructed, except for slice() and structured 
 clone, which copy the source object's state.
 


These are really solid proposals, and I've filed Bug 17746 to keep track of 
this.

https://www.w3.org/Bugs/Public/show_bug.cgi?id=17746

-- A*

 -- 
 Glenn Maynard
 

Re: [File API] File behavior under modification

[Arun Ranganathan]

On May 23, 2012, at 9:58 AM, Glenn Maynard wrote:

 On Wed, May 23, 2012 at 3:03 AM, Kinuko Yasuda kin...@chromium.org wrote:
 Just to make sure, I assume 'the underlying storage' includes memory.
 
 Right.  For simple Blobs without a mutable backing store, all of this 
 essentially optimizes away.
 
 We should also make it clear whether .size and .lastModifiedDate should 
 return live state or should just returning the same constant values.  (I 
 assume the latter)
 
 It would be the values at the time of the snapshot state.  (I doubt it was 
 ever actually intended that lastModifiedDate always return the file's latest 
 mtime.  We'll find out when one of the editors gets around to this thread...)
 

I think the ideal behavior is that it reflects values at snapshot state, but 
that reads if snapshot state has modified fail.

-- A*

Re: [File API] Blob URI creation

[Arun Ranganathan]

On May 30, 2012, at 6:48 PM, Glenn Maynard wrote:

 On your main question, I've had the same thought in the past--a url 
 property on Blob which simply creates a new auto-revoking blob URL.  I didn't 
 bring it up since I'm not sure if creating a URL for a blob is actually 
 something you do so often that it's worth having a shortcut.  If so, a 
 function is probably better than a property--more future-proof, and it'd be 
 unusual on the platform to have a property that returns a different value 
 every time you read it.
 
 On Wed, May 30, 2012 at 1:50 PM, Rich Tibbett ri...@opera.com wrote:
 Yes, this might be a better solution. I was working on what was available in 
 the editor's draft and looking for a way to remove the need to ever call 
 revokeObjectUrl.
 
 This is part of what's wrong with oneTimeOnly--it *doesn't* actually 
 completely remove the need to call revokeObjectUrl.  For example:
 
 function f(blob) {
 var url = URL.createObjectURL(blob, {oneTimeOnly: true});
 if(showImage)
 img.src = url;
 else
 URL.revokeObjectURL(url);
 }
 
 Without the revoke case, the URL (and so the whole blob) is leaked as it's 
 never actually used.  autoRevoke doesn't have this problem.
 
 Arun/Jonas: Can we hide this feature in the spec before more people implement 
 it, or at least tag it with not ready for implementations or something?


I'll do one better, and introduce autoRevoke semantics:

http://www.w3.org/TR/2012/WD-FileAPI-20120712/#creating-revoking

By default, this does not need a corresponding revokeObjectURL() call.  In 
order for Blob URLs to persist past a stable state (for that unit of script) 
createObjectURL has to be invoked with autoRevoke set to false.


 That is, you shouldn't ever have to pass a Blob URI obtained via Blob.getURL 
 through revokeObjectUrl because it assumes some auto-revocation behavior. 
 Using microtasks to release at the next stable state does seem ok as long as 
 developers have a very good understanding of when a Blob URI will be 
 implicitly revoked. Saying that you can use a Blob URI exactly once, as per 
 onetimeonly could still end up being easier to understand though.
 
 (s/microtasks/stable states/; they're not quite the same)
 
 It's actually a bit hard to understand (or easy to misunderstand), since 
 there's no clear concept of using a URL.  For example, if you start two 
 XHR's on the same URL one after the other in the same script, the order in 
 which the fetches actually begin is undefined (they happen in different task 
 queues), so which would succeed is undefined.  (Some work would also be 
 needed here for the autoRevoke approach, but it's much simpler.)
 
 autoRevoke is pretty simple from the user's perspective: the URL is revoked 
 when your script returns to the browser.
 

In fact, I think this addresses the lion's share of use cases, and if a 
developer wants to explicitly create longer lasting Blob URLs, they have that 
option (just not by default).

-- A*

Re: BlobBuilder.append() should take ArrayBufferView in addition to ArrayBuffer

[Arun Ranganathan]

On Apr 27, 2012, at 1:28 AM, Anne van Kesteren wrote:

 On Fri, 27 Apr 2012 00:13:42 +0200, Arun Ranganathan 
 aranganat...@mozilla.com wrote:
 The constructor will switch to use ArrayBufferView in lieu of ArrayBuffer, 
 but the read method exposed on FileReader and FileReaderSync will read files 
 into memory as ArrayBuffers.
 
 Since the constructor is not widely deployed yet would it not be better to 
 stick to ArrayBufferView exclusively, just like XMLHttpRequest.send()?


The Blob constructor will already use ArrayBufferView.  Above, do you mean, 
have a method of the sort FileReader.readAsArrayBufferView or something like 
that?  

*.readAsArrayBuffer is already deployed, and it might be weird to force a view 
on an ArrayBuffer when read into memory.

-- A*


 
 
 -- 
 Anne van Kesteren
 http://annevankesteren.nl/
 

Re: BlobBuilder.append() should take ArrayBufferView in addition to ArrayBuffer

[Arun Ranganathan]

On Apr 24, 2012, at 7:00 PM, David Herman wrote:

 On Apr 24, 2012, at 3:53 PM, David Herman wrote:
 
 On Apr 12, 2012, at 2:48 PM, Arun Ranganathan wrote:
 
 I intend to add ArrayBufferView as a parameter to the Blob constructor .
 
 Would it be possible also to allow passing an ArrayBuffer with an offset and 
 length as an additional alternative? This would eliminate the need to create 
 a view when you just want a slice of the buffer but don't particularly 
 want/need to think of it as a Uint8/Uint32/whatever array.
 
 Oh, I just saw in the bug that the idea is to *eliminate* the ArrayBuffer 
 argument option entirely. If that's the case then disregard my question.


The constructor will switch to use ArrayBufferView in lieu of ArrayBuffer, but 
the read method exposed on FileReader and FileReaderSync will read files into 
memory as ArrayBuffers.  

-- A*

Re: Delay in File * spec publications in /TR/ [Was: CfC: publish LCWD of File API; deadline March 3]

[Arun Ranganathan]

 On Mar 30, 2012, at 2:25 PM, ext Eric U wrote:
 
  On Fri, Mar 30, 2012 at 5:39 AM, Arthur Barstow
  art.bars...@nokia.com wrote:
  Hi All - the publication of the File API LC was delayed because of
  some logistical issues for Arun as well as some additional edits
  he intends to make.
  
  This delay also resulted in Eric's two File * specs not being
  published since they have a dependency on the latest File API
  spec.
  
  Arun - can you please give us at least a rough idea when you
  expect the spec to be ready for LC publication?
  
  Jonas - as co-Editor of File API, can you help get the File API LC
  published?
  
  Eric - your File * docs were last published in April 2011 so I
  think it would be good to get new versions published in /TR/
  soon-ish. OTOH, if they have dependencies on the latest File API,
  it may be better to postpone their publication until File API is
  published. WDYT?
  
  If it's going to be more than a month to get Arun+Jonas's spec up,
  we
  might as well go ahead and publish mine; they've had quite a bit of
  change.  If it's less than that, let's just do them all together.
 
 Eric - I haven't received any responses from Arun or Jonas so I will
 contact you off list about getting your two File specs ready for
 publication on /TR/.


Apologies for the delay in getting back to the listserv about changes to File 
API.  Internal logistics have been resolved, and I look forward to making edits 
on the File API specification.  

As Eric points out, it is probably going to be more than 2 weeks till I can 
account for all the necessary edits.  Eric's drafts don't need to be held up 
this timeline, though we should probably keep some things closely in sync., 
especially events, errors, etc.

-- A* 

Re: [FileAPI] Deterministic release of Blob proposal

[Arun Ranganathan]

Feras, 

In practice, I think this is important enough and manageable enough to include 
in the spec., and I'm willing to slow the train down if necessary, but I'd like 
to understand a few things first. Below: 

- Original Message -

 At TPAC we discussed the ability to deterministically close blobs
 with a few
 others.

 As we’ve discussed in the createObjectURL thread[1], a Blob may
 represent
 an expensive resource (eg. expensive in terms of memory, battery, or
 disk
 space). At present there is no way for an application to
 deterministically
 release the resource backing the Blob. Instead, an application must
 rely on
 the resource being cleaned up through a non-deterministic garbage
 collector
 once all references have been released. We have found that not having
 a way
 to deterministically release the resource causes a performance impact
 for a
 certain class of applications, and is especially important for mobile
 applications
 or devices with more limited resources.

 In particular, we’ve seen this become a problem for media intensive
 applications
 which interact with a large number of expensive blobs. For example, a
 gallery
 application may want to cycle through displaying many large images
 downloaded
 through websockets, and without a deterministic way to immediately
 release
 the reference to each image Blob, can easily begin to consume vast
 amounts of
 resources before the garbage collector is executed.

 To address this issue, we propose that a close method be added to the
 Blob
 interface.
 When called, the close method should release the underlying resource
 of the
 Blob, and future operations on the Blob will return a new erro r , a
 ClosedError.
 This allows an application to signal when it's finished using the
 Blob.

Do you agree that Transferable 
(http://dev.w3.org/html5/spec/Overview.html#transferable-objects) seems to be 
what we're looking for, and that Blob should implement Transferable? 

Transferable addresses the use case of copying across threads, and neuters 
the source object (though honestly, the word neuter makes me wince -- naming 
is a problem on the web). We can have a more generic method on Transferable 
that serves our purpose here, rather than *.close(), and Blob can avail of 
that. This is something we can work out with HTML, and might be the right thing 
to do for the platform (although this creates something to think about for 
MessagePort and for ArrayBuffer, which also implement Transferable). 

I agree with your changes, but am confused by some edge cases: 

 To support this change, the following changes in the File API spec
 are needed:

 * In section 6 (The Blob Interface)
 - Addition of a close method. When called, the close method releases
 the
 underlying resource of the Blob. Close renders the blob invalid, and
 further
 operations such as URL.createObjectURL or the FileReader read methods
 on
 the closed blob will fail and return a ClosedError. If there are any
 non-revoked
 URLs to the Blob, these URLs will continue to resolve until they have
 been
 revoked.
 - For the slice method, state that the returned Blob is a new Blob
 with its own
 lifetime semantics – calling close on the new Blob is independent of
 calling close
 on the original Blob.

 *In section 8 (The FIleReader Interface)
 - State the FileReader reads directly over the given Blob, and not a
 copy with
 an independent lifetime.

 * In section 10 (Errors and Exceptions)
 - Addition of a ClosedError. If the File or Blob has had the close
 method called,
 then for asynchronous read methods the error attribute MUST return a
 “ClosedError” DOMError and synchronous read methods MUST throw a
 ClosedError exception.

 * In section 11.8 (Creating and Revoking a Blob URI)
 - For createObjectURL – If this method is called with a closed Blob
 argument,
 then user agents must throw a ClosedError exception.

 Similarly to how slice() clones the initial Blob to return one with
 its own
 independent lifetime, the same notion will be needed in other APIs
 which
 conceptually clone the data – namely FormData, any place the
 Structured Clone
 Algorithm is used, and BlobBuilder.
 Similarly to how FileReader must act directly on the Blob’s data, the
 same notion
 will be needed in other APIs which must act on the data - namely
 XHR.send and
 WebSocket. These APIs will need to throw an error if called on a Blob
 that was
 closed and the resources are released.
So Blob.slice() already presumes a new Blob, but I can certainly make this 
clearer. And I agree with the changes above, including the addition of 
something liked ClosedError (though I suppose this is an important enough error 
+ exception to hash out with HTML and DOM4, and once again, the name is TBD). 

In your implementation, what happens exactly to Eric's edge cases, namely: 

xhr.send(blob); 
blob.close(); // method name TBD 

// AND 

frameRef.src = URL.createObjectURL(blob); 
blob.close() // method name TBD 

In my opinion, the