Re: Feedback on Quota Management API

[Eric Uhrhane]

On Wed, May 30, 2012 at 9:54 AM, Tobie Langel to...@fb.com wrote:


 On 5/30/12 6:30 PM, Kinuko Yasuda kin...@chromium.org wrote:

Thanks for the feedback!

On Tue, May 29, 2012 at 11:07 PM, Tobie Langel to...@fb.com wrote:

On 5/17/12 11:02 AM, Kinuko Yasuda kin...@chromium.org wrote:

For context for others, I assume they are comments for the draft pushed
at:
https://dvcs.w3.org/hg/quota/Overview.html


I'm super excited to see an API for this is in the works. It's been a much
wanted feature by developers.

Couple of thoughts/questions (sorry for the late feedback, I was on
parental leave):

1. My experience with measuring maximum storage quota in existing
implementations shows that while some implementations share a common quota
across different data stores (e.g. 10 MB for both localStorage and
AppCache on iOS last time I looked), not all do. What's the reasoning
behind enforcing this (is it easier for implementors? Better for
developers?) and is there agreement across implementors that this is the
way forward?

I believe this is for developers and users.  (I don't think this would
make
it easier for implementors)
Today we have multiple API options to store data locally, but most users
wouldn't care which API an app is using.  They might be interested in
how much data is stored by an app or why my disk is getting tighter,
but wouldn't be interested in I'm ok with API X storing B bytes, but
not for API Y.  Similarly I can imagine developers would care about
how much more they can store for their app, but they wouldn't want to
care about multiple quota values for each API.  Overtime they
may want to switch storage APIs, but if the switch requires quota
adjustment across storage that sounds very awkward.

This idea has been discussed several times on this list before, and
in my belief there's some consensus we want to have a single quota
across different storages (some pointers from past discussion: [1][2]).

[1]
http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/0400.html
[2]
http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/0357.html

 Makes sense and thanks for the pointers.

2. If the case is that we're going for a binary choice (i.e. persistent
vs. temp data stores) why not create specific methods for each rather than
pass a constant (or string) as first argument: e.g.:

    navigator.storageInfo.queryTemporaryUsageAndQuota


and

    navigator.storageInfo.queryPersistentUsageAndQuota


That'll avoid having to deal with typos in the first arg, error handling
in that case, etc.

I haven't thought about this before, nor have a strong opinion on this,
but I remember that in the past someone has commented that only two
storage types might not be enough.  I don't think we want more storage
types right now, but keeping it as enum or constant would make it easier
to add more storage types in a future.  What do you think?

 I can't think of a storage type (or anything else, for that matter) that
 would be neither persistent nor temporary.

How about session, which is guaranteed to go away when the browser
exits, document-local which isn't shared by other documents in the
same domain, window-local, which isn't shared by other windows...I'm
sure there are more.  Let's not box ourselves in.

 Also: http://robert.ocallahan.org/2012/05/canvas-getcontext-mistake.html

 --tobie

Re: Feedback on Quota Management API

[Eric Uhrhane]

On Thu, May 17, 2012 at 2:02 AM, Kinuko Yasuda kin...@chromium.org wrote:
 Thanks for the feedback!

 For context for others, I assume they are comments for the draft pushed at:
 https://dvcs.w3.org/hg/quota/Overview.html

 On Tue, May 15, 2012 at 8:17 PM, Anne van Kesteren ann...@annevk.nl wrote:

 1) Put storageInfo on window.navigator, not window directly.


 That sounds reasonable to me given that what window.navigator is supposed to
 represent.
 I'll change them if no one objects.


 2) Use the IDL callback syntax. The current IDL is outdated.


 Thanks, I will fix them.


 3) Use a string enum rather than a constant.


 They were introduced after the FileSystem API's [1] storage type, which also
 uses a constant.
 I have no strong opinion on this one and it seems like using string enum is
 more reasonable, but maybe we should change them together if we do so?
 (Eric, was there a reason we use constants rather than string / string enum
 in FileSystem API?)

I just wrote it that way because I'm a C++ coder.  We can migrate the
filesystem over to string constants.  If Chrome's TEMPORARY and
PERSISTENT get redefined to temporary and persistent from their
current integer values, folks who are using them won't even notice the
difference.

 [1] File API: Directories and System http://www.w3.org/TR/file-system-api/

 --
 Anne — Opera Software
 http://annevankesteren.nl/
 http://www.opera.com/

Re: FileSystem API: Adding file size field to Metadata?

[Eric Uhrhane]

On Tue, Feb 28, 2012 at 1:52 PM, Darin Fisher da...@chromium.org wrote:
 On Tue, Feb 28, 2012 at 10:47 AM, Kinuko Yasuda kin...@chromium.org wrote:

 Hi,

 While looking at the FileSystem API draft I noticed that we only expose
 'modificationTime' in 'Metadata' object.  Since FileEntry itself doesn't
 have 'size' field unlike File object maybe it's reasonable to add 'size'
 field to Metadata?

 http://www.w3.org/TR/file-system-api/#the-metadata-interface

 Without adding this we can indirectly get the file size by creating 'File'
 object via FileEntry.file() method and accessing File.size, but when an app
 wants to display the modificationTime and file size at once (and it sounds
 very plausible use case) the app needs to call two different async methods--
 which doesn't sound very nice.  WDYT?

 Thanks,
 Kinuko




 I think this is a nice improvement.  File size is very obviously something
 one might expect to be included in meta data for a file :-)

 -Darin

Yup.  I've had this recommended offline at Google as a way to speed up
directory listings.  Currently, for each Entry that you get from a
DirectoryReader, you have to call getMetadata to get mtime, then
getFile to get size.  This cuts the number of calls in half, which can
be a pretty big deal for a large directory.  I'll put it in.

Charles, I'd like to leave the may-be-nice ones out for now--it's easy
to add them later, and the current design also makes it easy for
implementers to add other fields experimentally.  If others think that
this rises to the same importance of size, we can certainly discuss
it.

Re: HTML5 Filesystem API feedback

[Eric Uhrhane]

On Fri, Apr 15, 2011 at 4:01 PM, Arun Ranganathan a...@mozilla.com wrote:
 On 4/15/11 6:29 PM, Aryeh Gregor wrote:

 On Wed, Apr 13, 2011 at 4:35 PM, Robert Gindargi...@chromium.org  wrote:

 * The FileError object is a bit awkward to work with.  I found that I
 frequently had every reason to expect my calls to succeed (because
 they were a follow-on to something that already succeeded), but I
 wanted to log the failure reason in the event they didn't.  The code
 online suggests a switch/case statement to turn error codes into
 mnemonic strings.  This requires a hardcoded list of all known errors,
 and a call out to this utility function every time you want to display
 the error reason in a readable way.  I suggest adding the mnemonic
 string as a property of FileError, and displaying it as part of the
 toString.  (See util.getFileErrorMnemonic() and
 util.installFileErrorToString() for example implementations.)

 I'd suggest solving this the same way as DOMException does, for
 consistency:


 http://dvcs.w3.org/hg/domcore/raw-file/tip/Overview.html#exception-domexception

 It has a name property that contains the name, e.g.,
 INDEX_SIZE_ERR or INVALID_STATE_ERR.  Of course, if you want to
 display a real error message you have to localize it, but an
 easily-recognizable, memorable, Google-able string is still extremely
 useful in many cases.  Plus, DOMException already does it and it's
 about two lines to spec, so why not?


 This sounds like a reasonable addition to both FileException and FileError;
 I'll make this change in File API.

That sounds great.

Re: HTML5 Filesystem API feedback

[Eric Uhrhane]

Robert:

First of all, thanks for the feedback.

On Wed, Apr 13, 2011 at 1:35 PM, Robert Ginda rgi...@chromium.org wrote:
 Hello public-webapps,

 I've spent a bit of time with the filesystem API recently, building out a
 set of common file dialogs (open, save-as, etc) for ChromeOS.  We have
 a private API call to get access to a special filesystem that contains the
 user's downloads folder and the mounted external storage, but beyond
 that the dialogs use the standard filesystem API calls.

 The bulk of the code is here:
 http://src.chromium.org/viewvc/chrome/trunk/src/chrome/browser/resources/file_manager/js/

 I've run into a few awkward bits that I'd like to bring up here to see what
 everyone else thinks.

 * The async API is really awkward when you want to resolve multiple
 entries.  I have a case where I'd like to resolve a list of
 directories and then do something when that completes.  It would be
 helpful if the DirectoryEntry class had a getDirectories() method.
 I've built my own in file_manager/js/util.js in the function called
 util.getDirectories.

In general, I'm pretty happy to see this sort of thing.  You may not
like to hear this, but any problem that can just be solved by layering
a small javascript library on top of this API is a problem we don't
need to fix in the first pass of the API.  We want it to start out
powerful and simple, and eventually, if that library is something that
absolutely everyone seems to need, we may want to incorporate it.  So
it is very helpful to see the kinds of libraries that get built on
top, and thanks for sending the pointer, but I suggest we wait and see
before making any changes.

BTW, you can make your getDirectories method faster if you don't wait
for a response before sending off the next query [assuming no memory
constraints].  Just fire off all the requests in one go, then keep a
countdown of responses in your callback-wrapper.  When your countdown
hits zero, fire the we're done signal.

 * That leads to a desire to get entries in the general sense.  In my
 case I knew ahead of time that I had a list of directories.  If I
 didn't know whether I had been handed directories or files, the code
 would get pretty hairy.  So in addition to getDirectories, a
 getEntries method (and getEntry, too) would be super useful.

This is a method that would look up, but not create, an Entry, given a
path?  Can you outline the circumstances in which you'd use this, but
where you wouldn't want to list the whole parent directory?

 * Another helper I found useful was util.forEachDirEntry from the same
 file previously mentioned.  This function invokes a callback once for
 each entry in a directory.  This saves the caller from having to
 create the reader and walk the results array during the callback.  It
 makes client code significantly more readable.  I'd love to see
 DirectoryEntry.forEach(...).

I can see the utility there pretty clearly.

 * The FileError object is a bit awkward to work with.  I found that I
 frequently had every reason to expect my calls to succeed (because
 they were a follow-on to something that already succeeded), but I
 wanted to log the failure reason in the event they didn't.  The code
 online suggests a switch/case statement to turn error codes into
 mnemonic strings.  This requires a hardcoded list of all known errors,
 and a call out to this utility function every time you want to display
 the error reason in a readable way.  I suggest adding the mnemonic
 string as a property of FileError, and displaying it as part of the
 toString.  (See util.getFileErrorMnemonic() and
 util.installFileErrorToString() for example implementations.)

Hmm...I'm not sure what other APIs usually do about this.  As soon as
you start adding strings that you intend to be human-readable, you
start getting into all sorts of internationalization issues.  Anyone?

Eric

Re: [FileAPI] Result of calling MultipleReads on FileReader

[Eric Uhrhane]

On Tue, Apr 12, 2011 at 11:44 AM, Arun Ranganathan a...@mozilla.com wrote:
 On 4/11/11 1:39 PM, Adrian Bateman wrote:

 On Monday, April 11, 2011 10:23 AM, Eric Uhrhane wrote:

 On Mon, Apr 11, 2011 at 9:33 AM, Arun Ranganathana...@mozilla.com
  wrote:

 In general, I'm averse to throwing, since I think it puts an additional
 burden on the developer (to catch, for example).

 I don't think so.  I think that calling two read methods is the kind
 of thing that will come up in initial testing, the exception won't be
 caught at all, and thus will be noticed and the bug removed.

  On the main thread, with
 your proposal, all reads will stop since an exception has been raised.
  Of
 course, when all reads are synchronous, throwing makes sense, since
 we're
 not generating events that the developer listens for.  Thus, in the case
 of
 FileReaderSync, I'll introduce spec. text (and more WebIDL formalism)
 that
 makes it clear that multiple reads will raise a FileException.

 I'd *much* rather have an exception than silently get some random
 result depending on which method I accidentally called last.

  The burden on the vigilant developer is to watch
 for errors, but the developer can at least obtain the result of the
 latest
 read call without an exception.

 I agree with Eric - I prefer to fail fast when the developer has misused
 an API.
 It makes it much easier to see that there is a mistake and debug why.

 So the key question to me is Is this misuse? I can't think of a good use
 case
 where a developer would want to call read again during the LOADING state
 and
 expect that the correct behaviour would be to implicitly call abort().
 That
 doesn't seem like a common scenario. In the EMPTY or DONE states makes
 perfect
 sense - that makes FileReader reusable - but during LOADING I think is a
 developer
 mistake.


 I felt it desirable to stick to what XHR does, which is an abort on previous
 calls.  Eric, Adrian: do you both feel we shouldn't be consistent with XHR?
  If so, an exception makes sense.  But I honestly think similarity with XHR
 is desirable in this case.

I'd rather have an exception, and no interruption to the read already
in progress.  I don't think similarity with XHR, which is already
somewhat different, is worth the cost.

Eric

Re: [FileAPI] BlobBuilder.getBlob should clear the BlobBuilder

[Eric Uhrhane]

On Tue, Apr 12, 2011 at 3:38 PM, Kyle Huey m...@kylehuey.com wrote:
 Hello All,

 In the current FileAPI Writer spec a BlobBuilder can be used to build a
 series of blobs like so:

   var bb = BlobBuilder();
   bb.append(foo);
   var foo = bb.getBlob();
   bb.append(bar);
   var bar = bb.getBlob();
   foo.size; // == 3
   bar.size; // == 6

 My concern with this pattern is that it seems that one of the primary use
 cases is to keep a BlobBuilder around for a while to build up a blob over
 time.  A BlobBuilder left around could potentially entrain large amounts of
 memory.  I propose that BlobBuilder.getBlob() clears the BlobBuilder,
 returning it to an empty state.  The current behavior also doesn't seem
 terribly useful to me (though I'm happy to be convinced otherwise) and be
 easily replicated on top of the proposed behavior (immediately reappending
 the Blob that was just retrieved.)

 Thoughts/comments?

 - Kyle

If you don't have a use for a current behavior, you can always just
drop the BlobBuilder as soon as you're done with it, and it'll get
collected.  I think that's simpler and more intuitive than having it
clear itself, which is a surprise in an operation that looks
read-only.  In the other case, where you actually want the append
behavior, it's faster and simpler not to have to re-append a blob
you've just pulled out of it.

Eric

Re: [FileAPI] Result of calling MultipleReads on FileReader

[Eric Uhrhane]

On Mon, Apr 11, 2011 at 9:05 AM, Adrian Bateman adria...@microsoft.com wrote:
 On Monday, April 11, 2011 8:28 AM, Arun Ranganathan wrote:
 On 3/31/11 6:12 PM, Eric Uhrhane wrote:
  I think it's cleaner and simpler just to throw.  FileReader and XHR
  are already different enough that a bit more, as long as it's a
  usability improvement, isn't a big deal.  The efficiency improvement
  is just a bonus.

 Eric: are you sure you mean throw or do you mean, fire error event or
 abort?  Note that only FileReaderSync (running on threads) actually
 throws in terms of an exception.

 I think throwing an exception would be appropriate here. The FileReader
 will know synchronously that it is already doing a read and that calling
 read again is an invalid operation. Firing the error event should be
 reserved for errors that come from the operation of the reader and not
 because the developer called it incorrectly.

My thoughts exactly.  That's the way I've structured things in
FileWriter as well.  I think it's much better to find out about bad
code as soon as possible.

On Mon, Apr 11, 2011 at 9:33 AM, Arun Ranganathan a...@mozilla.com wrote:
 In general, I'm averse to throwing, since I think it puts an additional
 burden on the developer (to catch, for example).

I don't think so.  I think that calling two read methods is the kind
of thing that will come up in initial testing, the exception won't be
caught at all, and thus will be noticed and the bug removed.

  On the main thread, with
 your proposal, all reads will stop since an exception has been raised.  Of
 course, when all reads are synchronous, throwing makes sense, since we're
 not generating events that the developer listens for.  Thus, in the case of
 FileReaderSync, I'll introduce spec. text (and more WebIDL formalism) that
 makes it clear that multiple reads will raise a FileException.

 On the main thread, what we have now (which, as you correctly point out,
 needs to be better defined) is that the previous read aborts, but the latest
 call goes through.  I like this since at least some result is obtained,
 rather than an exception.

I'd *much* rather have an exception than silently get some random
result depending on which method I accidentally called last.

  The burden on the vigilant developer is to watch
 for errors, but the developer can at least obtain the result of the latest
 read call without an exception.

Re: [FileAPI] FileWriter and read-only files

[Eric Uhrhane]

On Mon, Apr 11, 2011 at 12:16 PM, Glenn Maynard gl...@zewt.org wrote:
 File objects should have a readOnly property, indicating whether write
 permission is granted by the user.  Files returned from input elements
 should, by default, set it.  Constructing a FileWriter using a File with its
 readOnly property set should throw an exception.

Are you thinking of FileEntry?  File objects are always immutable, and
there's currently no way to get from a File to a FileWriter.

Having an immutable flag on a FileEntry would make sense, but as
there's currently no API to get a FileWriter outside of the FileSystem
[which has no immutable files], there's no rush to put it in--it can
wait for an API expansion that would let you get such a thing.

 Later, it would be good to also have an attribute on input to allow
 opening files for write.  For example, input type=file file=write to ask
 for write permission (setting readOnly if only read permission was granted,
 eg. with an open read-only checkbox), and input type=file file=create to
 also ask the browser to show a Save As dialog instead of an Open dialog.

We discussed this a while back, and people generally preferred
something like FileSaver instead of markup.

Eric

Re: ISSUE-173 (ericu): terminal FileWriter progress events should be queued [File API: Writer]

[Eric Uhrhane]

This is now resolved.

On Fri, Dec 10, 2010 at 3:30 PM, Eric Uhrhane er...@google.com wrote:
 On Fri, Dec 10, 2010 at 3:17 PM, James Robinson jam...@google.com wrote:
 On Fri, Dec 10, 2010 at 2:04 PM, Eric Uhrhane er...@google.com wrote:

 On Fri, Dec 10, 2010 at 2:39 AM, Anne van Kesteren ann...@opera.com
 wrote:
  On Fri, 10 Dec 2010 03:24:38 +0100, Web Applications Working Group Issue
  Tracker sysbot+trac...@w3.org wrote:
 
  ISSUE-173 (ericu): terminal FileWriter progress events should be queued
  [File API: Writer]
 
  http://www.w3.org/2008/webapps/track/issues/173
 
  Raised by: Eric Uhrhane
  On product: File API: Writer
 
  When a FileWriter successfully completes a write, currently it:
  * dispatches a write event
  * sets readyState to DONE
  * dispatches a writeend event
 
  If you want to start a new write, you can't do it in onwrite, since
  readyState is still WRITING.  Those events should be queued for
  asynchronous
  delivery, so that readyState is DONE by the time they get handled.  If
  you
  set up a new write in onwrite, you'll still run the risk of getting
  confused
  by the subsequent writeend from the previous write, but that's
  detectable.
 
  I'll have to look and see what other events should be marked as queued.
 
  Why not queue a task that changes readyState and then dispatches write
  followed by writeend, synchronously from the task. That is how a
  number of
  things work in XMLHttpRequest.

 That would work too.  Any reason that you don't want to set readyState
 before queueing the task?  This is already happening asynchronously,
 in response to the write finishing--the important thing is just to
 make sure the events are queued, and readyState is updated, before the
 first handler runs.

 I'm not familiar with this particular API, but in general I think it's
 important that state variables be set at the same time that the relevant
 event fires.  In other words, code that polls readyState or similar
 attributes should not be able to observe any change before the related event
 is fired.

 Yeah, that makes sense [+thanks to Jonas for pointing this out
 off-list].  I'll link the readyState change with onwrite in the
 success case and onerror in the failure case, then have onwriteend
 come after.

 - James

  --
  Anne van Kesteren
  http://annevankesteren.nl/
 
 

Re: [FileAPI] Result of calling MultipleReads on FileReader

[Eric Uhrhane]

On Thu, Mar 31, 2011 at 2:55 PM, Adrian Bateman adria...@microsoft.com wrote:
 On Thursday, March 31, 2011 10:19 AM, Arun Ranganathan wrote:
 On 3/30/11 2:01 PM, Eric Uhrhane wrote:
  On Mon, Mar 28, 2011 at 5:37 PM, Adrian Batemanadria...@microsoft.com
 wrote:
  Is there a reason for the current spec text?
  I don't know the original rationale, but in the absence of any strong
  technical constraint, I'd much prefer that subsequent read calls just
  throw an exception immediately.  They seem likely to be indicative of
  bad code anyway, and it's so much easier to debug when you find out
  right away.
 

 The original rationale was to do what XHR does w.r.t. open()!
 Essentially, the goal was:

 1. To abort previous reads in favor of the last one, like how XHR does.
 2. The last read goes through.

 So: Adrian/Eric -- do you object to keeping this like XHR in terms of
 aborting previous reads?  What I should *definitely* do is make spec.
 text more robust to reflect this; in general I want to make asynchronous
 parts of this spec. more like HTML5 here.

I think it's cleaner and simpler just to throw.  FileReader and XHR
are already different enough that a bit more, as long as it's a
usability improvement, isn't a big deal.  The efficiency improvement
is just a bonus.

 As long as the spec is clear I don't mind whether a subsequent read throws
 or includes an implicit abort() call. That's not the way I read the current
 spec text. We need to know things like whether we should queue any events
 for the abort or just silently move on to the new read, etc. It would be
 good if we can be explicit about when you're allowed to call readAs
 (this way it sounds like always) and if anything is different between the
 LOADING and DONE states.

Agreed on the need for this to be very explicit.  But I think we
should skip all the extra queued events by just making it illegal.

 Thanks,

 Adrian.

Re: [FileAPI] Result of calling MultipleReads on FileReader

[Eric Uhrhane]

On Mon, Mar 28, 2011 at 5:37 PM, Adrian Bateman adria...@microsoft.com wrote:
 As we continue to experiment with the File API, I'm trying to understand the
 rationale for the Multiple Reads section:
 http://dev.w3.org/2006/webapi/FileAPI/#MultipleReads

 The spec says:

   If multiple read methods are called on the same FileReader object, user
   agents MUST only process the last call to a read method, which is the
   call that occurs last in a script block that has the already started
   flag set [HTML5].

 I'm trying to understand the rationale for respecting the LAST call - is it
 common for people to call read lots of times and want the last one to be
 respected. Since the read happens asynchronously, we'd rather kick off the
 read operation as soon as the first read is called and give an error to
 subsequent read calls. I'm not sure what the use case is for wanting the last
 one (you can always call abort() and start again).

 Is there a reason for the current spec text?

I don't know the original rationale, but in the absence of any strong
technical constraint, I'd much prefer that subsequent read calls just
throw an exception immediately.  They seem likely to be indicative of
bad code anyway, and it's so much easier to debug when you find out
right away.

 Thanks,

 Adrian.

 --
 Adrian Bateman
 Program Manager - Internet Explorer - Microsoft Corporation
 Phone: +1 (425) 538 5111
 Email: mailto:adria...@microsoft.com

Re: File API and Directory API feedback

[Eric Uhrhane]

On Thu, Feb 10, 2011 at 4:55 PM, Eric Uhrhane er...@google.com wrote:
 On Thu, Feb 10, 2011 at 3:27 PM, Ian Hickson i...@hixie.ch wrote:

 A couple of points I noticed while briefly perusing the File API specs:

 * Blob.size has no conformance criteria (no musts). It could return a
 random number each time and that would be conforming. It seems like it
 should have at least some constraint related to how a FileReader
 interacts with the Blob.

 * Is Blob.size supposed to be synchronous? I'm looking into adding a way
 to get a Blob from canvas, but trying to avoid the mistake I made with
 toDataURL of having the data be synchronously available. That's fine with
 Blob and FileReader, the data can be read async, but the browser won't
 know the size of the file ahead of time, which means that I guess I should
 have a callback to get the Blob? It seems sad that we need to be async
 both in getting the Blob _and_ in getting the data from the Blob.

 Yes, Blob.size is synchronous.  This came up the last time people
 discussed getting one from a canvas, and nobody had a good solution
 that I can recall.

 * Why are createObjectURL() and revokeObjectURL() on the URL interface
 object rather than on Window or Navigator? This is highly unusual (it's
 the first case I can think of outside JS where we have done this).

 This came out of discussions at TPAC in Lyon; some people didn't want
 it cluttering up the Window namespace.

 * FileSystem is quite heavy-weight when all you want is a file.
 Discounting error handling, getting a file stored as /file in a
 filesytem requires the following code:

   window.requestFileSystem(1, 0, function (fs) {
     fs.root.getFile('/file', {}, function (fe) {
       fe.file(function (f) {
         processFile(f);
       })
     });
   });

 Two proposals to help reduce the confusion this causes:

  - I think we should rename 'getFile' to 'getFileEntry' since 'File' is
   something different than what 'getFile' provides.

 I'm open to that if folks want it.  It seems reasonable.

  - We should consider adding a getFile() method on FileSystem, and maybe
   even on DirectoryEntry, which gives you a File directly. That would
   allow the above to be rewritten as:

   window.requestFileSystem(1, 0, function (fs) {
     fs.getFile('/file', function (f) {
       processFile(f);
     });
   });

 ...which is at least a little improvement.

 If we were going to put it anywhere, DirectoryEntry makes more sense
 than FileSystem, I think.  [fs-fs.root in the above example]

 * What is the use case for FileSystem.name?

 It's just a human-readable identifier that might be useful for
 debugging AFAIK.  I'd have no objection to dropping it, actually.

 * What is FileEntry.toURI? If we have it at all, it should at least be
 called toURL to be consistent with the rest of the platform, but do we
 need it? Is it redundant with createObjectURL()? What are the cross-origin
 implications of such a URL getting leaked to another origin?

 Regarding createObjectURL--the URLs it creates for Blobs are tied to
 the lifetime of the page, and are generally undecipherable Blob URNs.
 The URLs returned by toURI are persistent, and the intent is that
 they'll be human-readable and -editable, and contain the obvious path
 information.  We can certainly change the name to toURL if that fits
 better.

Having heard no other comments on this, I've changed the name to toURL
and changed resolveLocalFileSystemURI to resolveLocalFileSystemURL, to
be more consistent with the rest of the platform.

 There's some discussion at [1] regarding cross-origin use and the URI format.

 Regarding cross-origin leaks, we still haven't settled whether the
 URLs can be used cross-origin or not.  We may just not want to make
 that work at all, given that there's no way to provide access controls
 once a URL leaks.

 * In the FileSystem API spec, Error Code Descriptions and The
 FileException exception seem to disagree on codes.

 I see that when I updated most references to the new values, I missed
 the table in 7.3.  Thanks for catching that.  Fixed.

 * The subsections of Dealing with errors and exceptions give the error
 codes in a different order each time.

 I see the definitions of FileException and FileError using the same
 numeric order on the constants, and I see the constants declared in
 7.1.2 and 7.2.2 in alphabetical order.  Is that what you see?  Is the
 only problem that 7.3's order doesn't match 7.2.2?  I'll fix that.

 * None of the methods really describe what they do. For example,
 createWriter is defined as Creates a new FileWriter associated with the
 file that this FileEntry represents. There are no conformance
 requirements there. Nothing says when the successCallback is called or
 when the errorCallback is called. Nothing says what the parameters to
 those callbacks are. etc.

 I'll flesh that out.  However, the callback parameters are defined in
 the callback definitions [e.g. 5.6.5].  I can certainly add some

Re: File API and Directory API feedback

[Eric Uhrhane]

On Wed, Mar 16, 2011 at 4:59 PM, Charles Pritchard ch...@jumis.com wrote:
 On 3/16/2011 4:34 PM, Eric Uhrhane wrote:

 On Thu, Feb 10, 2011 at 4:55 PM, Eric Uhrhaneer...@google.com  wrote:

 On Thu, Feb 10, 2011 at 3:27 PM, Ian Hicksoni...@hixie.ch  wrote:

 A couple of points I noticed while briefly perusing the File API specs:

 * Blob.size has no conformance criteria (no musts). It could return a
 random number each time and that would be conforming. It seems like it
 should have at least some constraint related to how a FileReader
 interacts with the Blob.

 * Is Blob.size supposed to be synchronous? I'm looking into adding a way
 to get a Blob fromcanvas, but trying to avoid the mistake I made with
 toDataURL of having the data be synchronously available. That's fine
 with
 Blob and FileReader, the data can be read async, but the browser won't
 know the size of the file ahead of time, which means that I guess I
 should
 have a callback to get the Blob? It seems sad that we need to be async
 both in getting the Blob _and_ in getting the data from the Blob.

 Yes, Blob.size is synchronous.  This came up the last time people
 discussed getting one from a canvas, and nobody had a good solution
 that I can recall.

 Seems like the qualities of the FileEntry interface. Returns a blob
 interface async, which can then be read async.
 canvas.toFileEntry().file( callback )

 toFileEntry is synchronous, returning a generic fileentry object,
 file method is async, and can return a typed array or blob.

 Was there discussion on using FileEntry semantics?

Not that I recall.  It's not a great match, really, because the data
returned from a Canvas doesn't have a name, a parent directory, etc.
If you just need an asynchronous way to get a File or Blob from a
Canvas, just make Canvas.toFile take a callback.

Re: [whatwg] Intent of the FileSystem API

[Eric Uhrhane]

On Mon, Feb 28, 2011 at 4:59 PM, Charles Pritchard ch...@jumis.com wrote:
 On 2/28/2011 4:10 PM, Eric Uhrhane wrote:

 On Mon, Feb 28, 2011 at 2:54 PM, Charles Pritchardch...@jumis.com
  wrote:

 I'd like some clarification on the intent of the FileSystem API:
 requestFileSystem permanent, getDirectory and getFile.

 ...

 They're not particularly easy to use, compared to the IndexedDB data
 store,

 I find files to be easier to use than databases, but I'm clearly
 biased ;'.  However, that's not really the point.  Files and
 databases serve different needs, and various use cases are better
 served by each than the other.  If you want transactions or indexing,
 you want IDB.  If you want fast, mutable binary blobs with persistent
 URLs, you want FileSystem.

 It's much easier to send a query via IndexedDB to list all files in a
 directory
 mask, than it is to build up the same thing using getDirectory.

 I'm quite happy with File, FileWriter, and so forth.

 It's much easier/quicker to post a binary blob at an arbitrary path using a
 data store, than it is going through full file system
 semantics; a bunch of create directory and file calls.

 For scalability, you've got Amazon S3 and Google Storage as data store
 models.

 I target the old WebSQL, localStorage, file system,
 IDB and remote services, to stash images and metadata.

 IDB has been the easiest to support, as the others generally require more
 work to stash metadata information,
 and have uncertain locking.


 but if they are OS level commands,
 then they do help with usability of a web app with the rest of the OS:
 with
 managing files directly from the OS file manager.

 Is the FileSystem API intended to expose a folder on the host operating
 system, or is it just an abstraction
 which can be bolted onto any data system (such as, using SQLite as a
 backend, as appCache and IndexedDB do)  ?

 While the current API could in theory be implemented either way, there
 have been many requests for expansions to the API that would allow for
 access to directories outside the sandbox, e.g. giving Flickr access
 to My Photos.  While that's not yet been specced, that would clearly
 require real pass-through access and real file names.  The rest of the
 API could of course be identical, and I hope that eventually it will
 serve both use cases.

 I'm supportive of a virtual FS for the temporary file system, but
 using a real sandbox for the permanent system.

I don't think it's likely that we'll want to have different APIs for
temporary vs. persistent storage, in the origin-specific sandbox.  But
if you mean a different API [different path limitations, anyway] for
the sandbox vs. other mount points such as My Photos, then yes, I
think that makes sense.

 I'm sure we'll get there eventually, serving both cases.

 Are there any plans of reviewing/visiting efficient File/Blob support in
 indexeddb?

 I can't speak for the indexedDB developers, but the last I heard
 on-list, they were at least hoping to support immutable Blobs, but not
 soon.

 What's the hold up? Working with base64 encoding is slow.
 createObjectURL is nice.

 More on those portability issues:

 We've been implementing the current FileSystem API spec in Chrome, and
 had planned that our implementation of the sandbox would be just a
 passthrough to a directory under the user's profile.  We've been
 testing that out, currently only allowing access to extensions and
 apps, but we've gotten feedback that long profile paths are causing a
 problem on Windows.

 Yes, I submitted that feedback :-)

 My understanding is that it's quite reasonable to get longer paths than the
 current 255 bytes.
 Keep in mind, that includes the whole path to the AppData profile directory.

 There are certainly issues in old copies of windows, but I think they've
 been addressed since.

 speed of access and ease of modification, but we'll be storing the
 paths in an internal database.

 I did the same on the file system as well, as I needed that flexibility
 to get around the various constraints.

 I kept original file names, which caused some problems with the 255 byte
 limit.

 stashed the file in
 sha1hash.substr(-2) + '/' + sha1Hash + '/' originalFilename
 with metadata in
 sha1hash.substr(-2) + '/' + sha1Hash + '/' metaDataHash + '/entry.txt'

 The metadata entry.txt files are in there as a precaution and for
 transparency
 if the user is accessing things outside of the app.

 ..

 Would your virtual FS be up to including a Dropbox style directory,
 where users could copy files into, and they'd be accessible to the FS api?

No.  That'd have to wait for the proposed expanded API that would
allow for other mount points.

 That's also how I modeled mine, including a Dropbox directory underneath
 it all, for users to easily copy files into.

 In some cases, it's much easier to drag/drop a hundred files into the
 directory,
 through the system file manager. Also allows for the OS to treat the dropbox
 as a queue

Re: [whatwg] Intent of the FileSystem API

[Eric Uhrhane]

On Mon, Feb 28, 2011 at 6:21 PM, Glenn Maynard gl...@zewt.org wrote:
 On Mon, Feb 28, 2011 at 7:41 PM, Eric Uhrhane er...@google.com wrote:

 Sorry--I meant to push this over to public-webapps, as Ian suggested.
 [+cc public-webapps, whatwg-BCC]

 Currently (reviewing for context), the spec tries to reach filename
 consistency (only allowing filenames to be used which can be used
 consistently on all browsers) by restricting filenames to the lowest-common
 denominator.  With this path length issue, I don't think that approach is
 correct, since it seems like everyone's going to have to implement path
 virtualization anyway.

Indeed.

 Based on that, I'd like to suggest a model change:

 - Only place basic restrictions on filenames, giving the broadest set of
 filenames that are generally supported (eg. no slashes, nulls, and some
 reasonable length limits).

What would you suggest for limitations?  If we're requiring
virtualization, it seems to me that we could be quite liberal.

 - For sandboxed directories, it's guaranteed that all legal filenames are
 permitted, whether or not the local filesystem supports them.  If the native
 filesystem isn't capable of supporting all legal filenames, virtualization
 is required.
 - For mounted directories, no guarantee of filename consistency is made:
 you can create filenames on some systems that aren't allowed on others.
 Filenames invalid on a particular system raise an exception.

 This gives filename consistency for sandboxed directories without applying
 quirks of all operating systems to users of all other operating systems.

 This expressly gives no guarantee of filename consistency for mounted
 directories (which don't exist yet, of course).  As we talked about before,
 I think this is critical when dealing with directories shared with native
 applications.  If the user creates hello?.txt, then a web app version of
 gzip must be able to create hello?.txt.gz if the system can do so.
 Similarly, if the user creates family photos: Bill.jpg, a web app image
 resizer must be able to create family photos: Bill (thumbnail).jpg.

 (I'm not trying to get into the particulars of mounted directories now,
 but I needed to mention them to explain how this suggestion fits together,
 since directories interacting with native applications can't be
 virtualized.)

Yes, and I'm finding myself agreeing with you, although I've argued
the other side in the past.  Sometimes you just have to try
implementing it to figure out what's not going to work.

 On Mon, Feb 28, 2011 at 7:59 PM, Charles Pritchard ch...@jumis.com wrote:
 There are certainly issues in old copies of windows, but I think they've
 been addressed since.

 It still happens in Win7, even in Windows Explorer.  I'm pretty sure this is
 just PATH_MAX, which hasn't changed.

 --
 Glenn Maynard

Re: [whatwg] Intent of the FileSystem API

[Eric Uhrhane]

On Tue, Mar 1, 2011 at 11:37 AM, Glenn Maynard gl...@zewt.org wrote:
 On Tue, Mar 1, 2011 at 1:13 PM, Eric Uhrhane er...@google.com wrote:
 What would you suggest for limitations?  If we're requiring
 virtualization, it seems to me that we could be quite liberal.

 I'd suggest only the restrictions that are required for the API: no
 , ., .., and no filenames containing forward slashes.

That makes sense.

 Maybe some fairly high filename component limit (NAME_MAX) for sanity;
 maybe 4k (in UTF-16).

 Maybe disallow nulls; I'm not sure if this is special enough to
 actually need to be in this set.

I like it, though, as allowing nulls in strings is likely to lead to user error.

 I think the 5000 entries per directory limitation should also go
 away.  Virtualization hides any local directory
 limitations--presumably files on disk wouldn't mirror the virtualized
 directory structure.

Right.

 (By the way, I'm not sure from the spec which error code is used for
 invalid filenames.  Also--though this section may be irrelevant if you
 go with this approach--section 8 says This section is non-normative.
 and then contains normative requirements.)

The non-normative flag is only supposed to be for the next paragraph,
not the whole section.  Thanks--I'll fix that when I rewrite that
section.

 Yes, and I'm finding myself agreeing with you, although I've argued
 the other side in the past.  Sometimes you just have to try
 implementing it to figure out what's not going to work.

 Sure, I don't think anyone predicted OS-related problems that were
 quite that difficult to mask with filename restrictions.  (It feels
 like I should have thought of it, too, since I've hit the this issue
 several times natively when extracting archives with long
 filenames...)

 --
 Glenn Maynard

Re: [whatwg] Intent of the FileSystem API

[Eric Uhrhane]

Sorry--I meant to push this over to public-webapps, as Ian suggested.
[+cc public-webapps, whatwg-BCC]

On Mon, Feb 28, 2011 at 4:10 PM, Eric Uhrhane er...@google.com wrote:
 On Mon, Feb 28, 2011 at 2:54 PM, Charles Pritchard ch...@jumis.com wrote:
 I'd like some clarification on the intent of the FileSystem API:
 requestFileSystem permanent, getDirectory and getFile.

 The FileSystem API has several goals.  Some of them are targeted by
 the current spec, and some have been discussed, but put off until
 later.  We didn't want to take on too much in a single pass [hence the
 three layers of file-related specs so far].

 Are they intended to directly pass through to the host operating system, or
 are they just programming constructs?

 The intent for the local sandbox described in the current FileSystem
 spec was that that would be up to the implementer.  If a UA implements
 it as a passthrough, the files could then be accessible to client-side
 apps as well as the web.  That's reason for the restrictions in the
 uniformity of interface section--if you're going to expose the true
 local filesystem, you run into different restrictions on names and
 paths on each platform, and we want code that uses this API to run
 everywhere.  However, we've been running into some issues in this area
 with our implementation--more on that below.

 They're not particularly easy to use, compared to the IndexedDB data store,

 I find files to be easier to use than databases, but I'm clearly
 biased ;'.  However, that's not really the point.  Files and
 databases serve different needs, and various use cases are better
 served by each than the other.  If you want transactions or indexing,
 you want IDB.  If you want fast, mutable binary blobs with persistent
 URLs, you want FileSystem.

 but if they are OS level commands,
 then they do help with usability of a web app with the rest of the OS: with
 managing files directly from the OS file manager.

 Is the FileSystem API intended to expose a folder on the host operating
 system, or is it just an abstraction
 which can be bolted onto any data system (such as, using SQLite as a
 backend, as appCache and IndexedDB do)  ?

 While the current API could in theory be implemented either way, there
 have been many requests for expansions to the API that would allow for
 access to directories outside the sandbox, e.g. giving Flickr access
 to My Photos.  While that's not yet been specced, that would clearly
 require real pass-through access and real file names.  The rest of the
 API could of course be identical, and I hope that eventually it will
 serve both use cases.

 Are there any plans of reviewing/visiting efficient File/Blob support in
 indexeddb?

 I can't speak for the indexedDB developers, but the last I heard
 on-list, they were at least hoping to support immutable Blobs, but not
 soon.

 More on those portability issues:

 We've been implementing the current FileSystem API spec in Chrome, and
 had planned that our implementation of the sandbox would be just a
 passthrough to a directory under the user's profile.  We've been
 testing that out, currently only allowing access to extensions and
 apps, but we've gotten feedback that long profile paths are causing a
 problem on Windows.

 Our current implementation on Windows uses the standard API calls that
 limit paths to 256 characters long.  If your profile directory is
 deeply nested [e.g. because you have a really long username], code
 that works for your friend might not work for you.  We thought we'd
 fix that by using the long-path APIs that allow creation of paths up
 to ~32k chars long.  However, it seems that, even though Windows has
 these APIs, most programs don't use them.  In particular, Windows
 Explorer doesn't, so it chokes on them and can't delete files in
 directories with long names.  We think it would be really awkward if
 Chrome were creating directories under its profile directory that
 couldn't easily be deleted, so we're going to start virtualizing our
 filesystem implementation.  We'll still have real native files for
 speed of access and ease of modification, but we'll be storing the
 paths in an internal database.

 This then brings up the question of whether one can specify a uniform,
 useful interface for the API that *doesn't* require virtualization of
 the filesystem, but I'll be bringing that up in another thread.

     Eric

Re: [File API: Directories and System] callbacks or events

[Eric Uhrhane]

On Tue, Feb 15, 2011 at 3:46 AM, Olli Pettay olli.pet...@helsinki.fi wrote:
 On 02/12/2011 01:08 AM, Eric Uhrhane wrote:

 On Fri, Feb 11, 2011 at 7:43 AM, Olli Pettayolli.pet...@helsinki.fi
  wrote:

 Hi all,

 the current File API: Directories and System seems to use
 callbacks and not events, yet other
 File APIs (the ones for read and write) use events.
 That is quite major inconsistency in the APIs.
 IIRC there was already some discussion about which approach to use
 when the API for read was designed and it was decided that events should
 be
 used.

 Using events would make it rather easy to track moves, copies etc, of a
 file. Just set the event listeners when the entry is first time
 accessed, and then you get notified whenever the file is moved etc.

 FileReader and FileWriter are fundamentally different than FileSystem.
  Reading or writing a file is an ongoing process, hence progress
 events make a lot of sense.  Getting a file handle, deleting a file,
 creating a directory, etc., are all very binary.  They've happened or
 they haven't, and there's no progress to report.  Thus callbacks make
 sense for those operations.

 It sounds like you're looking for some sort of a FileSystemWatcher
 object that would let you keep track of everything that's happening in
 a filesystem.

 No I'm not, although using events would give that kind of
 functionality for free.

There is no free.

 I'm looking for API consistency.

Different APIs have different requirements and fulfill different use
cases.  There will inevitably be differences in interface.

  That's not a request I've heard before; if you've got

 specific use cases in mind, please post them.

       Eric

Re: [File API: Directories and System] callbacks or events

[Eric Uhrhane]

On Fri, Feb 11, 2011 at 7:43 AM, Olli Pettay olli.pet...@helsinki.fi wrote:
 Hi all,

 the current File API: Directories and System seems to use
 callbacks and not events, yet other
 File APIs (the ones for read and write) use events.
 That is quite major inconsistency in the APIs.
 IIRC there was already some discussion about which approach to use
 when the API for read was designed and it was decided that events should be
 used.

 Using events would make it rather easy to track moves, copies etc, of a
 file. Just set the event listeners when the entry is first time
 accessed, and then you get notified whenever the file is moved etc.

FileReader and FileWriter are fundamentally different than FileSystem.
 Reading or writing a file is an ongoing process, hence progress
events make a lot of sense.  Getting a file handle, deleting a file,
creating a directory, etc., are all very binary.  They've happened or
they haven't, and there's no progress to report.  Thus callbacks make
sense for those operations.

It sounds like you're looking for some sort of a FileSystemWatcher
object that would let you keep track of everything that's happening in
a filesystem.  That's not a request I've heard before; if you've got
specific use cases in mind, please post them.

  Eric

Re: File API and Directory API feedback

[Eric Uhrhane]

On Fri, Feb 11, 2011 at 4:33 AM, Michael[tm] Smith m...@w3.org wrote:
 Eric Uhrhane er...@google.com, 2011-02-10 16:55 -0800:

 On Thu, Feb 10, 2011 at 3:27 PM, Ian Hickson i...@hixie.ch wrote:
  Is there somewhere that such issues should be filed?

 I'm not sure about the File API--it used to use [2], but I'm not sure
 if it still does.
 I've got [3] for FileWriter and [4] for FileSystem.  I'd appreciate
 any issues you'd care to log.

 If you care to use bugzilla, I've created components for FileWriter and
 FileSystem in the WebApps product in the W3C bugzilla:

  http://www.w3.org/Bugs/Public/describecomponents.cgi?product=WebAppsWG

Michael, thanks.  However, I chose to use the tracker originally since
that's what the File API was using.  I'd rather not have 2 different
places to log issues.  If there's a strong reason I should move to
Bugzilla, let me know, but otherwise I think we should remove those
components.

 Eric, if you create a W3C bugzilla account (and give me a heads-up after),
 I'll set you as the default assignee for those components.

  --Mike

 --
 Michael[tm] Smith
 http://people.w3.org/mike

Re: File API and Directory API feedback

[Eric Uhrhane]

On Thu, Feb 10, 2011 at 3:27 PM, Ian Hickson i...@hixie.ch wrote:

 A couple of points I noticed while briefly perusing the File API specs:

 * Blob.size has no conformance criteria (no musts). It could return a
 random number each time and that would be conforming. It seems like it
 should have at least some constraint related to how a FileReader
 interacts with the Blob.

 * Is Blob.size supposed to be synchronous? I'm looking into adding a way
 to get a Blob from canvas, but trying to avoid the mistake I made with
 toDataURL of having the data be synchronously available. That's fine with
 Blob and FileReader, the data can be read async, but the browser won't
 know the size of the file ahead of time, which means that I guess I should
 have a callback to get the Blob? It seems sad that we need to be async
 both in getting the Blob _and_ in getting the data from the Blob.

Yes, Blob.size is synchronous.  This came up the last time people
discussed getting one from a canvas, and nobody had a good solution
that I can recall.

 * Why are createObjectURL() and revokeObjectURL() on the URL interface
 object rather than on Window or Navigator? This is highly unusual (it's
 the first case I can think of outside JS where we have done this).

This came out of discussions at TPAC in Lyon; some people didn't want
it cluttering up the Window namespace.

 * FileSystem is quite heavy-weight when all you want is a file.
 Discounting error handling, getting a file stored as /file in a
 filesytem requires the following code:

   window.requestFileSystem(1, 0, function (fs) {
     fs.root.getFile('/file', {}, function (fe) {
       fe.file(function (f) {
         processFile(f);
       })
     });
   });

 Two proposals to help reduce the confusion this causes:

  - I think we should rename 'getFile' to 'getFileEntry' since 'File' is
   something different than what 'getFile' provides.

I'm open to that if folks want it.  It seems reasonable.

  - We should consider adding a getFile() method on FileSystem, and maybe
   even on DirectoryEntry, which gives you a File directly. That would
   allow the above to be rewritten as:

   window.requestFileSystem(1, 0, function (fs) {
     fs.getFile('/file', function (f) {
       processFile(f);
     });
   });

 ...which is at least a little improvement.

If we were going to put it anywhere, DirectoryEntry makes more sense
than FileSystem, I think.  [fs-fs.root in the above example]

 * What is the use case for FileSystem.name?

It's just a human-readable identifier that might be useful for
debugging AFAIK.  I'd have no objection to dropping it, actually.

 * What is FileEntry.toURI? If we have it at all, it should at least be
 called toURL to be consistent with the rest of the platform, but do we
 need it? Is it redundant with createObjectURL()? What are the cross-origin
 implications of such a URL getting leaked to another origin?

Regarding createObjectURL--the URLs it creates for Blobs are tied to
the lifetime of the page, and are generally undecipherable Blob URNs.
The URLs returned by toURI are persistent, and the intent is that
they'll be human-readable and -editable, and contain the obvious path
information.  We can certainly change the name to toURL if that fits
better.

There's some discussion at [1] regarding cross-origin use and the URI format.

Regarding cross-origin leaks, we still haven't settled whether the
URLs can be used cross-origin or not.  We may just not want to make
that work at all, given that there's no way to provide access controls
once a URL leaks.

 * In the FileSystem API spec, Error Code Descriptions and The
 FileException exception seem to disagree on codes.

I see that when I updated most references to the new values, I missed
the table in 7.3.  Thanks for catching that.  Fixed.

 * The subsections of Dealing with errors and exceptions give the error
 codes in a different order each time.

I see the definitions of FileException and FileError using the same
numeric order on the constants, and I see the constants declared in
7.1.2 and 7.2.2 in alphabetical order.  Is that what you see?  Is the
only problem that 7.3's order doesn't match 7.2.2?  I'll fix that.

 * None of the methods really describe what they do. For example,
 createWriter is defined as Creates a new FileWriter associated with the
 file that this FileEntry represents. There are no conformance
 requirements there. Nothing says when the successCallback is called or
 when the errorCallback is called. Nothing says what the parameters to
 those callbacks are. etc.

I'll flesh that out.  However, the callback parameters are defined in
the callback definitions [e.g. 5.6.5].  I can certainly add some
explanations, though.

 * Some of the conformance requirements are very unclear about what they
 mean. For example, File and directory names must not end in period or
 space: is that a requirement on authors or implementations? What happens
 if they do?

I'll clarify that.  It's a requirement on 

Re: Quota API to query/request quota for offline storages (e.g. IndexedDB, FileSystem)

[Eric Uhrhane]

2011/2/7 Kinuko Yasuda kin...@chromium.org:
 On Sat, Feb 5, 2011 at 7:29 AM, Glenn Maynard gl...@zewt.org wrote:
 On Fri, Feb 4, 2011 at 12:07 AM, Kinuko Yasuda kin...@chromium.org wrote:

 If we want to make the quota API treat each API differently this would
 make a lot sense, but I'm not fully convinced by the idea.
 Putting aside the localStorage for now, do you still see significant
 issues in having a sshared single quota?  Also let me note that
 this API does not and should not guarantee that the app can actually
 *write* that amount of data into the storage, even after the quota is
 granted, and the UA could still stop an API to write further even if
 it's within the quota.

 I suppose that even the 2-3x difference--requesting 256 MB and actually
 getting 512 MB over different APIs--is acceptable, since to users,
 requesting storage is an order-of-magnitude question more than a precise
 number.  As long as implementations are still allowed to implement separate
 quotas if they want, it's probably acceptable for this API to not reflect
 them precisely and to be biased towards a shared quota.

 If we think that most of users/developers wouldn't be interested in
 specifying 'giving X bytes to storage A' and 'giving Y bytes to
 storage B' or such that while both storage A and B eats the user's
 disk, then probably UA should evolve in that direction.  That's my
 assumption and why the proposed API look like that (i.e. biased
 towards a shared quota).

Agreed.  Users won't care about the distinction, so let's make it so
developers don't have to care.

 2011/2/4 Ian Fette (イアンフェッティ) ife...@google.com
 For instance, if a user has been using a site for months, uses it
 frequently, and the site hits its 5GB limit but there's still 300GB free on
 the drive, perhaps we just give the site another 5GB and give the user a
 passive indication that we've done so, and let them do something if they
 actually care.

 That's interesting; reducing the amount users are nagged about things that
 they probably don't care about is important.  It would also need to suppress
 prompting from calls to requestQuota if the quota increase would have been
 allowed automatically.

 The proposing API itself doesn't specify the frequency of
 notifications or the behavior of out-of-quota scenarios, but probably
 it might be worth adding some note about calling 'requestQuota()' does
 not (and should not) always need to result in the UA prompting, and it
 must be generally prohibited prompting the user too frequently.

Possibly in non-normative text, but I think it's implied if you just
leave it out that implementers can do whatever they want with the UI.

 The bottom line of whether we should prompt or not is, I suppose, if
 UA ask for the user's permission to store some data in the storage,
 the UA shouldn't delete the data without the user's permission.

Yup.

Re: Quota API to query/request quota for offline storages (e.g. IndexedDB, FileSystem)

[Eric Uhrhane]

On Thu, Feb 3, 2011 at 4:32 AM, João Eiras joao.ei...@gmail.com wrote:
 Howdy.

   interface StorageInfo {

 Should probably be QuotaInfo or even QuotaManager ? Storage can and
 will be confused with Web Storage.

    // storage type
     const unsigned short TEMPORARY = 0;
     const unsigned short PERSISTENT = 1;


 Only two values seem not enough for me and I disagre with the
 nomenclature. Would be betteer to have
 LOCAL_STORAGE, SESSION_STORAGE, WEB_SQL_DBS, INDEXED_DB and so on. But
 that would create artificial dependencies with the other specs, so it
 would be better if type passed to the query and request functions to
 just be an opaque string which would allow any offline storage spec to
 refer to this one instead and specify a type of their own like
 localStorage, sessionStorage, indexedDb.

The point is that apps and users should be able to communicate about
how much disk space should be used.  Why would users care which API a
web app is using for its storage?  If google.com needs 10MB to store
my email, why should I care whether it's in indexedDB, localStorage,
or the FileSystem?

     // To query how much storage is available and currently in use.
     void queryUsage(unsigned short storageType,
                     UsageCallback usageCallback);


 Given that quota values are usually just preferences, hence
 lightweight enough, this function should be sync for the sake of
 simplicity.

This doesn't request just the preference; it returns how much space is
actually used.  That might be an expensive operation, might require
disk IO to fetch a value from an internal database, etc.  By default,
we try to make all new APIs async so as not to constrain
implementation.

 General note: using bytes as measure is probably overkill. While I
 doubt localStorage would even exceed a couple megabytes, making an
 entire e-mail account available offline with indexedDb or web sql db
 could easily climb to a couple GBs. Perhaps kilobytes is better ?

Possibly, although there's no harm in allowing extra precision, and
nothing forcing UAs not to round values anyway.

Re: Quota API to query/request quota for offline storages (e.g. IndexedDB, FileSystem)

[Eric Uhrhane]

On Thu, Feb 3, 2011 at 9:23 AM, João Eiras joao.ei...@gmail.com wrote:
 Should probably be QuotaInfo or even QuotaManager ? Storage can and
 will be confused with Web Storage.

    // storage type
     const unsigned short TEMPORARY = 0;
     const unsigned short PERSISTENT = 1;


 Only two values seem not enough for me and I disagre with the
 nomenclature. Would be betteer to have
 LOCAL_STORAGE, SESSION_STORAGE, WEB_SQL_DBS, INDEXED_DB and so on. But
 that would create artificial dependencies with the other specs, so it
 would be better if type passed to the query and request functions to
 just be an opaque string which would allow any offline storage spec to
 refer to this one instead and specify a type of their own like
 localStorage, sessionStorage, indexedDb.

 The point is that apps and users should be able to communicate about
 how much disk space should be used.  Why would users care which API a
 web app is using for its storage?  If google.com needs 10MB to store
 my email, why should I care whether it's in indexedDB, localStorage,
 or the FileSystem?


 Before storing X amount of data with an API, the code requests enough
 space for the storage area that API covers. That's how I understand
 the use case. Then it's up to the user agent to communicate that
 appropriately to the user, if applicable.

Why would we need to have the app specify I want to store 5MB in
indexedDB and 3MB in the FileSystem?  The user won't care; at most,
the user needs to know This app wants to use up 8MB of space.  If
the user doesn't care, why make the app make two requests?  With two
separate requests, you might prompt the user for the first 5MB, then
immediately have to prompt them again for the next 3MB.  Better just
to do it all in one go: the app says I want to store up to 8MB, then
uses it in whichever APIs it likes.

 APIs are web dev scope, not user scope, so storage areas need to be
 differentiated. I doubt that any browser ATM shares a single quota
 between all different storage types. Opera doesn't for instance.

I believe that's what this proposal is trying to address.

     // To query how much storage is available and currently in use.
     void queryUsage(unsigned short storageType,
                     UsageCallback usageCallback);


 Given that quota values are usually just preferences, hence
 lightweight enough, this function should be sync for the sake of
 simplicity.

 This doesn't request just the preference; it returns how much space is
 actually used.  That might be an expensive operation, might require
 disk IO to fetch a value from an internal database, etc.

 Hum, right. But then my comment does apply to requestQuota no ? No
 need to ping data files to store a new value for the quota, which is
 usually just a pref.

That's an implementation detail.  Plus, storing a preference to a
persistent database involves file IO, which can take an arbitrarily
long time and/or fail.
An async API leaves a lot of freedom for developers, so in general we
try to use them for everything new outside the worker context.

Re: [FileAPI: Directories and System] Some comments

[Eric Uhrhane]

Thanks for the feedback, Peter--comments inline below.

On Sun, Jan 30, 2011 at 9:10 AM, Peta Byte 256petab...@googlemail.com wrote:
 Hello everybody,
 it's the first time I participate in a working group's mailinglist like
 this, so when
 my concerns rather belong to the implementation area (like the mailinglists
 of browser developers) please correct me.

 # Foreword:
 I studied Eric Uhrhane's working draft of File API: Directories and System
 [1] and wrote
 some demo apps (including a multi-view File Manager that can extract tar
 archives) using
 the implementation that landed in recent releases of Chromium's Dev Channel.
 Doing so I
 noticed some aspects of the drafted API that could be improved.

 # Suggestions
 ## Callback methods used in asynchronous filesystem interfaces
 Nearly all registerable callbacks [2] are called being bound to the global
 scope object,
 instead to the object they were registered on (File-/Directory-/Entry,
 DirectoryReader, asf.).
 In my opinion this is unfavorable because in practice the async operation
 one starts
 (like querying metadata of an Entry or the contents of a directory) almost
 always will
 be handled in context of the entity the received information belongs to.
 Only the extensive use of closures and/or additional logic that keeps track
 of running
 async operations (in order to map the response and trigger further events)
 allows to
 work around this behaviour. Sure, that might be the rationale for 3rd party
 libaries, but
 as a webdeveloper that strives for performant code (like most of us do)
 I'd suggest to
 to avoid this overhead and just spec out to which objects those callbacks
 should
 be bound to.

Having the callbacks be global functions, rather than methods on the
object where the method was invoked, is pretty standard across all the
client-side storage APIs.  While I see the benefit of your suggestion,
I think the uniformity of the interface is more important.  We don't
want to surprise people who are using both the FileSystem and, say,
IndexedDatabase, by having inconsistent behavior.

 ## toURI-method of Entry Interface
 I'd recommend to spec out a common URI scheme for resources within a local
 filesystem, too.

Agreed.  What do you think of the URI scheme I proposed?
See http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/0218.html

 Some reasons for this:
     * Content of a permant filesystem should (theoretically) exist for a
 longer period of time (read
       for more than one sessions)
     * Avoids that user agents mistakenly interpret the value of
 href-attributes as local File URLs
       (though the file path begins with a slash)
     * When multiple filesystem instances are used within a session, a user
 agent is able to identify the
       specific filesystem where the resource identified by a URL resides in.
 In my demo file browser I introduced a pseudo URI scheme that looks like
 this:
     `lfs://file__0:Persistent/a/b/c.txt`
 Where the scheme lfs stands for Local FileSystem (as far as I know not
 officially reserved;
 only used internally by Cascading, a Java library to work with Hadoop FSs) .
 As authority part
 I simply used the value of the name attribute of the FileSystem interface.
 Then I appended
 the absolute filepath that every Entry has (fullPath attribute).
 Bubbling Click events of HTMLAnchorElements are catched either by the code
 of a custom
 widget or -- finally -- at the HTMLBodyElement that asks the user how to
 handle it or simply
 suppresses it.
 To dereference the URLs I created a wrapper go the LocalFileSystem/-Sync
 interface that is
 used to request filesystem instances. The name of every requested filesystem
 is saved
 together with a reference to the according FIleSystem in a lookup table and
 used to resolve
 the filesystem based on the authority part of a LFS-URI. Up to now, this is
 a wacky solution
 because there is no way to request a FileSystem by it's internal name. As
 long as I only use
 one persistent FileSystem this might work over more than one sessions. Of
 course, no serious
 solution.
 This leads to my next suggestion:

 ## Extend LocalFileSystem/-Sync interface by method to request FileSystem by
 its internal name
 As outlined in the section above, a precondition for persistent URLs of
 Local FileSystem resources
 that can be dereferenced by 3rd party applications is a method that tries to
 request a FileSystem
 by its internal name/id.

I don't understand.  What internal name do you need?

 Well, that's it. :-)
 Would like to head what you think about my suggestions.

 Best regards,
 Peter





 [1] http://www.w3.org/TR/2010/WD-file-system-api-20101026/#idl-def-LocalFileSystem
 [2] http://www.w3.org/TR/2010/WD-file-system-api-20101026/#callbacks

[FileSystem]: URI format, uses

[Eric Uhrhane]

The Entry.toURI method specified in the FileSystem spec [1] currently
has an open issue to define its format.  I believe we also need to
describe the ways in which it can and cannot be used, as some
potential uses may have security implications.

I propose the following format:

filesystem:{protocol}://{domain}[:port]/{storage type}/{path}

e.g. filesystem:https://www.google.com/persistent/images/logo.png

I think that, for the domain that owns the asset referred to by the
URI, pretty much any reasonable use should be allowed:
video/audio/img/iframe/script sources, XHR [GET only], etc.  I'm
iffier on allowing any access to other origins, even for e.g. img
sources, even though they're normally allowed cross-origin.  I'd love
to hear security arguments against and use cases for cross-origin
access.  Of course, it's always easiest/safest to start out not
allowing such a thing and relax the rules later.

Thanks in advance for any comments.

 Eric Uhrhane
 er...@google.com

[1] http://dev.w3.org/2009/dap/file-system/file-dir-sys.html#widl-Entry-toURI

Re: file-system-api: filename restrictions

[Eric Uhrhane]

Glenn:

Sorry about the slow response; I was on vacation, and am only now catching up.

We've discussed these issues before, see
http://lists.w3.org/Archives/Public/public-device-apis/2010Jan/0229.html
for much of the initial discussion.  However, you've brought up a new
point that I think is worth addressing.

On Sun, Dec 19, 2010 at 11:26 AM, Glenn Maynard gl...@zewt.org wrote:
 Section 8 Uniformity of interface will cause headaches for some use
 cases.  For example, an application may want to allow the user to fill
 a directory with images, then output a thumbnail of each image x.jpg
 into a subdirectory with the same filename, thumbs/x.jpg.

 However, we're forbidden from creating a new file with invalid
 filenames, even if they exist elsewhere.  The operation will fail, and
 we'll have to tell our Linux users with images named at the beach:
 moon rock?.jpg that they have to obey Windows filename
 conventions--which will probably be upsetting.  It'd also be a
 difficult rule for users to follow; while it's easy in Windows since
 it's globally enforced in all applications, Linux users would have to
 memorize the rules themselves.

Actually, it's not just that Linux users now have to worry about
Windows rules; Windows users also have to worry about Linux rules, in
particular the path length limitation, which is 255 bytes on Linux but
255 UTF-16 code points on Windows.

 It's also a pain for backing up files, eg. copying moon rock?.jpg to
 moon rock?.jpg~, and for safe writes, writing to moon
 rock?.jpg.new and then renaming the finished file over the original.

 These seem like bigger problems than the one it's trying to solve.  Is
 it really insufficient for these rules to define what filenames must
 be supported, that any others may not be, and to suggest a UA log if
 nonportable filenames are created?  (Of all filename issues, the only
 one that I've ever found to be a serious real-world portability issue
 is case-insensitivity.)

Yes, I believe that's insufficient.  We've discussed this before, and
1) We really do want a fully-portable subset to be the standard; code
should work everywhere if it works anywhere.  You shouldn't have to
code to OSX any more than you should have to code to Opera--just code
to the web platform.
2) Developers often don't read UA logs.  We should fail early on the
dev box, rather than failing later on the user's machine.

 I guess there are other issues with reading data created outside of the API:

 - filenames that can't be decoded to a DOMString, eg. undecodable
 bytes in a UTF-8 filesystem.  This is common in Linux after eg.
 unzipping a ZIP containing SJIS filenames.  Should these simply be
 ignored with a log?

I'm looking into the encoding problems now, and will respond later.
In general we should be able to read any such file already, at the
very least by enumerating the directory to get the FileEntry, but
creating files with valid names may be tricky.

 - existing filenames that differ only by case.  Similarly, should the
 UA just ignore all but one of them and make a log to the console?

There's no problem accessing those through directory enumeration, or
via a supplied path.  You just can't create this situation using the
API.

 Should whitespace in section 8.3 simply indicate space, U+0020?

It looks like it should; my mistake.  Thanks!

 Windows does allow creating filenames ending with NBSP and other
 Unicode whitespace characters, and it's not clear whether this should
 be allowed.  Other whitespace (\r, \n, \t) is covered by the control
 character rule.

 Sorry if this is a rehash of past topics.

The API is designed as it is to support a couple of different
situations, only one of which is currently specced, but both of which
have been discussed.  What's specced so far is a per-origin sandbox
that web apps can use for client-side storage.  Depending on the UA's
implementation of it, it's possible that the files stored there will
be exposed to the host machine and potentially shared with apps
outside of the browser, but we generally expect the browser to create
most or all of them.  Thus it makes sense to take a
least-common-denominator [LCD] approach, so that code that works on
any platform works on all platforms.  If other apps create files there
we should be able to access them no matter what, but things will go
more smoothly if said apps respect our restrictions.

However, an obvious expansion of this API which we've talked a lot
about is the ability to expose other mount points to the browser.
For example, a trusted app might be granted access to My Photos or
another similar directory.  There the majority of the files are
expected to be created by apps outside of the browser, and you run
into the thumbnail problem you describe above, where a
read-modify-write of a path or even a copy operation can inadvertently
create a file path that's banned by the API, but is legal on the host
system.

In a perfect world, I think we'd want all paths that came from 

Re: FileAPI use case: making an image downloading app

[Eric Uhrhane]

On Fri, Dec 17, 2010 at 3:34 PM, Gregg Tavares (wrk) g...@google.com wrote:
 Sorry if this has been covered before.
 I've been wanting to write an app to download images from photo sites and
 I'm wondering if this use case has been considered for the FileAPI wrt
 Directories and System.
 If I understand the current spec it seems like their are some possible
 issues.
 #1) The spec says there is a 5000 file limit or directory.

That's a conservative number I chose for performance reasons, not
because of hard limits.  While most modern filesystems officially
support far larger directories than that, I've been warned that many
bog down quite a bit by the time you hit 1 entries.  If anyone can
point me to actual data on the major filesystems, I'm happy to bump
that number up, but I feel that we should have some limit that
generally works with decent performance on all the major platforms.
If that means you have to do some work to work around the limit, well,
you'd probably have to have done that to work around the slowness
anyway, if you wanted to write a portable app.

I see a test at
http://www.frank4dd.com/howto/various/maxfiles-per-dir.htm with much
larger numbers; you can see the curve that come out.  I'd love to see
something like that for more filesystems and fewer files.  I know NTFS
used to do N^2 work to list directories, but I presume that's been
fixed by now.

 #2) The spec requires an app to specify how much storage it needs
 I understand the desire for the limits.  What about an app being able to
 request unlimited storage and unlimited files? The UA can decide how to
 present something to the user to grant permission if they want.

Chrome has implemented an unlimited storage permission for its apps.
 This is really beyond the scope of this spec; I'd like to leave it
open for UAs to experiment with.  While you do have to give a hint as
to how much storage you plan to use, we really don't restrict what
actions UAs might take in response.

 Arguments against leaving it as is:
 The 5000 file limit seems arbitrary. Any app that hits that limit will
 probably require serious re-engineering to work around it. It will not only
 have to some how describe a mapping between files on a server that may not
 have that limit, it also has the issue the user might have something
 organized that way and will require the user to re-organize. I realize that
 5000 is a large number. I'm sure the author of csh though 1700 entires in a
 glob was a reasonable limit as well. We all know how well that turned out
 :-(   It's easy to imagine a video editing app that edits
 and composites still images. If there are a few layers and 1 image per layer
 it could easily add up to more than 5000 files in a single folder.
 The size limit also has issues. For many apps the size limit will be no
 problem but for others  Example: You make ray tracing program, it traces
 each frame and saves the files to disc. When it runs out of room, what is
 its options? (1) fail. (2) request a file system of a larger size. The
 problem is (2) will require user input. Imagine you start your render before
 you leave work expecting it to finish by the time you get in only to find
 that 2 hours after you left the UA popped up a confirmation this app
 requests more space

Yes, that's exactly what I'd anticipate; I personally wouldn't want an
app to be able to fill up my hard drive without getting my permission.
 However, this isn't required by the spec, and UAs are free to
experiment with UI and quota policies as they see fit.

 The same would be true for an image downloading app. You start the app to
 download a 1000 images, not knowing their sizes before hand. You start it
 and leave. Half way through the UA puts up a conformation asking for more
 space.
 Have these use cases already come up and been rejected?

Re: FileAPI use case: making an image downloading app

[Eric Uhrhane]

On Fri, Dec 17, 2010 at 5:03 PM, Gregg Tavares (wrk) g...@google.com wrote:


 On Fri, Dec 17, 2010 at 4:16 PM, Charles Pritchard ch...@jumis.com wrote:

 We're actively developing such functionality.

 The limit per directory is for the sake of the os file system. If you want
 to create a data store, use indexedDB or use standard file system practices
 (create a subdirectory tree).

 I think you're missing the point. If I have a folder with 6000 files on some
 server and I want to mirror that through the FileAPI to the user's local
 machine I'm screwed. I can't **mirror** it if it's not a mirror.


 Disk space availability (quota) is an issue no matter what happens. When
 downloading 1000 images, you'll still only be doing so x at a time.

 I don't see the point you're trying to make here. I don't know the size of
 the images before hand. Many internet APIs make getting the sizes
 prohibitively expensive. One REST call per file. So before I can download a
 single file I'd have to issue 1000 REST XHRs to find the sizes of the files
 (waiting the several minutes for that to complete) before I can ask the user
 for the space needed.  That's not a good user experience. If on the other
 hand the user can give me permission for unlimited space then I can just
 start downloading the files without having to find out their ultimate size.
 I suppose I can just request 1 terabyte up front. Or query how much space is
 free and ask for all of it.

I think that in this particular case, you'd want a better server API.
Hey, I want to download this whole directory.  How much space will I
need?.  I realize that that's not always an option, but it's an
alternative to what I just posted elsewhere in this thread: let the UA
come up with a better quota system+UI.

 This issues are inherent in the design of any application of scale. At
 this point, the file system API does work for the use case you're
 describing.

 It'd be nice to see Blob storage better integrated with Web Storage Apis.
 Ian has already spoken to this, but no followers yet (afaik).

 -Charles



 On Dec 17, 2010, at 3:34 PM, Gregg Tavares (wrk) g...@google.com
 wrote:

  Sorry if this has been covered before.
 
  I've been wanting to write an app to download images from photo sites
  and I'm wondering if this use case has been considered for the FileAPI wrt
  Directories and System.
 
  If I understand the current spec it seems like their are some possible
  issues.
 
  #1) The spec says there is a 5000 file limit or directory.
 
  #2) The spec requires an app to specify how much storage it needs
 
  I understand the desire for the limits.  What about an app being able to
  request unlimited storage and unlimited files? The UA can decide how to
  present something to the user to grant permission if they want.
 
  Arguments against leaving it as is:
 
  The 5000 file limit seems arbitrary. Any app that hits that limit will
  probably require serious re-engineering to work around it. It will not only
  have to some how describe a mapping between files on a server that may not
  have that limit, it also has the issue the user might have something
  organized that way and will require the user to re-organize. I realize that
  5000 is a large number. I'm sure the author of csh though 1700 entires in a
  glob was a reasonable limit as well. We all know how well that turned out
  :-(   It's easy to imagine a video editing app that edits and composites
  still images. If there are a few layers and 1 image per layer it could
  easily add up to more than 5000 files in a single folder.
 
  The size limit also has issues. For many apps the size limit will be no
  problem but for others  Example: You make ray tracing program, it 
  traces
  each frame and saves the files to disc. When it runs out of room, what is
  its options? (1) fail. (2) request a file system of a larger size. The
  problem is (2) will require user input. Imagine you start your render 
  before
  you leave work expecting it to finish by the time you get in only to find
  that 2 hours after you left the UA popped up a confirmation this app
  requests more space
 
  The same would be true for an image downloading app. You start the app
  to download a 1000 images, not knowing their sizes before hand. You start 
  it
  and leave. Half way through the UA puts up a conformation asking for more
  space.
 
  Have these use cases already come up and been rejected?
 
 
 

Re: ISSUE-173 (ericu): terminal FileWriter progress events should be queued [File API: Writer]

[Eric Uhrhane]

On Fri, Dec 10, 2010 at 3:17 PM, James Robinson jam...@google.com wrote:
 On Fri, Dec 10, 2010 at 2:04 PM, Eric Uhrhane er...@google.com wrote:

 On Fri, Dec 10, 2010 at 2:39 AM, Anne van Kesteren ann...@opera.com
 wrote:
  On Fri, 10 Dec 2010 03:24:38 +0100, Web Applications Working Group Issue
  Tracker sysbot+trac...@w3.org wrote:
 
  ISSUE-173 (ericu): terminal FileWriter progress events should be queued
  [File API: Writer]
 
  http://www.w3.org/2008/webapps/track/issues/173
 
  Raised by: Eric Uhrhane
  On product: File API: Writer
 
  When a FileWriter successfully completes a write, currently it:
  * dispatches a write event
  * sets readyState to DONE
  * dispatches a writeend event
 
  If you want to start a new write, you can't do it in onwrite, since
  readyState is still WRITING.  Those events should be queued for
  asynchronous
  delivery, so that readyState is DONE by the time they get handled.  If
  you
  set up a new write in onwrite, you'll still run the risk of getting
  confused
  by the subsequent writeend from the previous write, but that's
  detectable.
 
  I'll have to look and see what other events should be marked as queued.
 
  Why not queue a task that changes readyState and then dispatches write
  followed by writeend, synchronously from the task. That is how a
  number of
  things work in XMLHttpRequest.

 That would work too.  Any reason that you don't want to set readyState
 before queueing the task?  This is already happening asynchronously,
 in response to the write finishing--the important thing is just to
 make sure the events are queued, and readyState is updated, before the
 first handler runs.

 I'm not familiar with this particular API, but in general I think it's
 important that state variables be set at the same time that the relevant
 event fires.  In other words, code that polls readyState or similar
 attributes should not be able to observe any change before the related event
 is fired.

Yeah, that makes sense [+thanks to Jonas for pointing this out
off-list].  I'll link the readyState change with onwrite in the
success case and onerror in the failure case, then have onwriteend
come after.

 - James

  --
  Anne van Kesteren
  http://annevankesteren.nl/
 
 

Re: FileSystem API: Avoiding Upload Forms And Temporary Downloads

[Eric Uhrhane]

On Mon, Nov 29, 2010 at 11:45 AM, Joran Greef jo...@ronomon.com wrote:
 I have some questions regarding the FileSystem API:

 1. It would be great to be able to let the user choose where they want their 
 sandboxed directory located for the web app, i.e. on the desktop for quick 
 access. That way they can drag files directly to the directory, which could 
 be used as a dropbox for synching to a server. Would this be possible (or at 
 least a mechanism to link to the directory wherever the browser may choose to 
 place it)? Otherwise, apps like Dropbox would not be possible in a browser.

Yes, we've certainly talked about this as a logical next step for the
API.  We want to make sure the basic security/usability/quota issues
with the current spec are dealt with before we expand to such use
cases, but I think there's a lot of great stuff that could be built on
this idea.  For example, you might want to give flickr.com access to
your photos directory.

 2. It seems that dragging a file out of a web app is currently copy-on-write? 
 So you drag a file out the web app into Excel but subsequent changes in Excel 
 would be lost to the web app (it seems like it's already possible for the web 
 app to poll the sandboxed directory for file changes)? If so, it means that 
 the FileSystem API would force the following work flow: the user saves a temp 
 file somewhere (probably on the desktop) then re-uploads using a web form, 
 and then minimizes the browser and deletes the temp file, and then maximizes 
 the browser again? That would be a bad case of Fitts' Law and quickly become 
 a show-stopper if the user needs to frequently edit files using native 
 applications.

I don't see how Fitts' Law is relevant, but you raise an interesting
point about drag-and-drop.  When you drag a link out of an HTML
document, that's usually a drag of a link, not a file.  GMail
implements attachment drag-out, but that's a copy, not a path.  The
FileSystem API will give you URIs that you'd build a page around just
like any other; if you want different behavior than web URIs, we'd
have to change something.  I think you'd really want to implement this
by overriding drag+drop behavior rather than changing the file API,
but you may also need to give scripts access to the true underlying
paths in order to make that work, and there are security issues with
that.  I'll keep it in mind; if you have a specific proposal for an
API, please post it.

 3. It must be possible to link to files within the sandboxed directory and 
 have them open in the default native application. I can understand that 
 .exe's need to be neutered, but content files such as .doc and .xls must have 
 a method for opening in the default application. Would this be possible? 
 Otherwise the only solution would be to trigger the Download window, creating 
 a temp file in the Downloads folder for a file that already exists on the 
 filesystem?

This runs into the same issues as #2--local URIs would get treated
differently than online URIs, we'd have to tell the applications the
true locations of the files, and we'd lose any protection provided by
the download manager.  I'm not going to say that this is impossible,
but it might require elevated privileges beyond those available to the
average web page, and not everyone would want it.  I'd like to see an
extension do this and see how it went before I'd expect the web
platform to allow it.

 4. In my mind, the FileSystem Api has a shot at improving user experience by 
 helping to avoid file upload forms, and temp file downloads. I'm not sure 
 these goals are possible with the current spec?

I'm not sure what you're asking here.

 These use-cases may prove to be vital building blocks for the next wave of 
 networked applications and it would be great to see them in the new 
 FileSystem API.

 Regards

 Joran Greef

progress events in sync XHR not entirely ruled out

[Eric Uhrhane]

In the XHR2 spec, most references to progress events make it clear
that they're only sent for asynchronous XHRs.  However, in three
cases, the events aren't written as being exclusive to the async case.
 I'm guessing these are just oversights:

1) 
http://dev.w3.org/2006/webapi/XMLHttpRequest-2/#cross-origin-request-event-rules:

The cross-origin request event rules are as follows:

If the cross-origin request status is preflight complete
Make upload progress notifications.

2) http://dev.w3.org/2006/webapi/XMLHttpRequest-2/#switch-done:

When it is said to switch to the DONE state run these steps:
1. If the asynchronous flag is false update the response entity body.
2. Switch the state to DONE.
3. Dispatch a readystatechange event.
4. Dispatch a progress event called load.
5. Dispatch a progress event called loadend.

3) http://dev.w3.org/2006/webapi/XMLHttpRequest-2/#the-abort-method

However, if #1 and #2 are fixed, then there's no way to call the abort
method on a sync XHR anyway.

Eric

Re: Updates to FileAPI

[Eric Uhrhane]

SGTM

On Fri, Nov 12, 2010 at 11:07 AM, Arun Ranganathan
aranganat...@mozilla.com wrote:
 - Original Message -
 On 11/12/10 11:53 AM, Boris Zbarsky wrote:
  OK, then we need to define rules for instantiating a Date object
  here
  (in the face of strings that may or may not be valid date format
  strings, or may be ambiguous, note). Is the proposal to just call
  new
  Date(str)?

 Er, nevermind. We're talking about file modification dates, not web
 resource modification dates, so my concern is unfounded.


 Right -- to be totally clear, we're talking about 
 http://www.w3.org/TR/FileAPI/#dfn-file having a readonly Date attribute that 
 returns the last modified on disk data.

 -- A*

Re: Discussion of File API at TPAC in Lyon

[Eric Uhrhane]

On Thu, Nov 11, 2010 at 8:52 AM, Arun Ranganathan
aranganat...@mozilla.com wrote:
 At the recent Technical Plenary and All WG Meetings in Lyon, File API[1] was 
 discussed, and there are some take away action items that I minuted for 
 myself for File API, but I'm not sure they are reflected in ACTION items, 
 etc.  From my own notes:

 Essentially, strong opinions were voiced against having top-level methods 
 createObjectURL and revokeObjectURL. So the biggest change was to introduce a 
 new top-level object (ObjectURL) which would have methods to obtain a string 
 Blob URI.  This removes the need for a revocation mechanism, since now the 
 ObjectURL object (which would take as a constructor the Blob object) would 
 oversee lifetime issues.  This is a big change, but potentially one that 
 allows us to work with the emerging URL API (which hopefully is going 
 somewhere).

While I agree that we came up with the new top-level object [called
the dummy object in the minutes] to hold createObjectURL and
revokeObjectURL, I don't think we actually threw away the second
method.  It would still be useful to be able to throw away Blob URLs
explicitly, so as to avoid keeping the Blobs around forever in
long-lived windows.

Also, I believe we decided that this should be disjoint from the URL
object that abarth is speccing:

  arun: is it worth make global dummy object the same thing being
  specced by adam barth

  no

  jonas: abarth's thing is to solve parsing urls. this isn't want we
  need to do with blob urls

  anne: not so sure

  jonas: there's a vague resemblance given that they both revolve
  around URLs

  sam: agrees
  ... especially since adam's thing doens't exist yet

Checking again, my interpretation of the minutes is the same as my
memory, so I can't possibly be mistaken ;'.

 There were additional discussions about Content-Disposition and further 
 headers introduced to Blob URIs, but we agreed that this should go to the 
 listserv for further discussion.  The question of *further* HTTP-like 
 behaviors on Blob URIs is still open for discussion.  Notably, 
 Content-Disposition is desired for download management, but using a header to 
 toggle browser behavior seems a bit arbitrary, and there may be better ways 
 to approach the issue.

Yeah, I think we made some good progress there, but no conclusions.
I'll start another thread about the headers.

 While I look forward to the minutes from the WebApps meeting, does anyone in 
 attendance agree or disagree that these are the main points to take away, or 
 wish to add something else?  Note that at least two implementations are 
 around the corner with window.createObjectURL and window.revokeObjectURL.  
 Vendor prefixing is a viable option in the mean time.

 -- A*
 [1] http://www.w3.org/TR/FileAPI/

Re: Discussion of File API at TPAC in Lyon

[Eric Uhrhane]

On Thu, Nov 11, 2010 at 10:02 AM, Jonas Sicking jo...@sicking.cc wrote:
 On Thu, Nov 11, 2010 at 8:52 AM, Arun Ranganathan
 aranganat...@mozilla.com wrote:
 At the recent Technical Plenary and All WG Meetings in Lyon, File API[1] was 
 discussed, and there are some take away action items that I minuted for 
 myself for File API, but I'm not sure they are reflected in ACTION items, 
 etc.  From my own notes:

 Essentially, strong opinions were voiced against having top-level methods 
 createObjectURL and revokeObjectURL. So the biggest change was to introduce 
 a new top-level object (ObjectURL) which would have methods to obtain a 
 string Blob URI.  This removes the need for a revocation mechanism, since 
 now the ObjectURL object (which would take as a constructor the Blob object) 
 would oversee lifetime issues.  This is a big change, but potentially one 
 that allows us to work with the emerging URL API (which hopefully is going 
 somewhere).

 Actually, this was a brain-fart on my part. What was suggested was
 that we simply allow:

 img.src = myFile;
 img.src = myBlob;
 img.src = myFutureStream;
 img.src = http://www.sweden.se/ABBA.jpg;;

 These things could be implemented without lifetime worries.

 What we might need is a IDL construct so that a specification can just say

 interface HTMLImageElement {
  ...
  attribute URLThingamajig src;
  ...
 };

 Which would automatically define that it accepts files/blobs/strings.
 And gives us a central place to update when we want to add streams and
 other things.

While this is a clean API, it doesn't work for passing URLs to
plugins, and it doesn't work when folks construct a bunch of DOM via
innerHTML.  And if you add a way to get a string from one of these
objects, you're back with the lifetime problem again.

Re: XHR responseArrayBuffer attribute: suggestion to replace asBlob with responseType

[Eric Uhrhane]

On Mon, Oct 25, 2010 at 3:21 PM, Chris Rogers crog...@google.com wrote:

 There's been some recent discussion in the webkit-dev mailing list about
 the efficiency of implementation of the responseArrayBuffer attribute.
  People are concerned that it would require keeping two copies of the data
 around (raw bytes, and unicode text version) since it's unknown up-front
 whether responseText, or responseArrayBuffer will be accessed.  I have a
 few possible solutions to this problem:

 1.  Defer decoding the text, and instead buffer the raw data as it comes
 in.  If there's any access to responseText (or responseXML), then the
 buffered data can be decoded into text at that time, and the buffered raw
 data discarded.  If that case happens, then from that point on no raw data
 buffering would happen and the text would be accumulated as it is right now.
  Otherwise, if responseText is never accessed then the raw data continues to
 buffer until it's completely loaded.  Then an access to responseArrayBuffer
 can easily convert the raw bytes to an ArrayBuffer.

 The idea is that once responseText or responseXML is accessed, then it
 would no longer be possible to access responseArrayBuffer (an exception
 would be thrown).
 Conversely, once responseArrayBuffer is accessed, then it would no longer
 be possible to use responseText or responseXML (an exception would be
 thrown).
 This approach does seem a little strange because of the mutually exclusive
 nature of the access.  However, it seems that it would be hard to come up
 for a reasonable use case where both the raw bytes *and* the text would be
 needed for the same XHR.

 2. Darin Fisher has suggested this approach: Add an asArrayBuffer
 attribute (similar to asBlob) which *must* be set before sending the
 request if there will be a subsequent access of the responseArrayBuffer
 attribute.

 3. Get rid of the asBlob attribute and add a new responseType attribute
 which could be:
 Text   --- this is the default
 XML
  Bytes
 ... other types yet to be defined

 Approach (2) can lead to illegal combinations (both asBlob and
 asArrayBuffer set to true), but approach (3) is not as discoverable at
 runtime as (2), because (2) allows for the simple existence check of either
 attribute asBlob or asArrayBuffer.

 It's certainly easy to check for illegal combinations on send, or for that
matter disallow setting more than one of them in the first place.

Also, 3 allows for a simple existence check as well.

I can accept any of the proposed solutions and would like to hear what
 others think about these or other approaches.

 Chris

Re: using BlobBuilder append method with generated binary data

[Eric Uhrhane]

Toni:

BlobBuilder now has an append() method that takes an ArrayBuffer,
and FileReader has readAsArrayBuffer.  Do these together satisfy your
needs?

Eric

On Wed, Oct 20, 2010 at 3:39 AM, Toni Ruottu toni.ruo...@iki.fi wrote:
 I have discussed the topic before on some Chromium bug threads. I
 searched the archives of this mailing list for blobs and generated
 content, but was not too successful. There may be multiple different
 reasons why one would need to build blobs from generated data. The one
 that concerns me, is being able to do binary POSTs with XmlHTTPRequest
 in a standard manner. To send binary data over XHR one is required to
 send out a blob, but first the blob needs to be constructed with
 BlobBuilder. Let me fill you in on how I see the current situation.
 Maybe someone can spot an error in my line of thought.

 A new (empty) BlobBuilder is created by stating...
 var bb = new BlobBuilder();

 After creating a BlobBuilder, the user appends some data into the blob
 being generated by using the append function. The append function is
 overloaded and can be used to append multiple different types of data
 into the blob. Currently only existing blobs, and utf-8 text are
 supported.

 One can add an existing blob into the new one by doing...
 bb.append(existingBlob);

 One can add a text string in utf-8 format by doing...
 bb.append(aTextString);

 Once the data is in there, it is possible to construct the new blob with...
 var blob = bb.getBlob()

 A blob has a slice method which makes it possible to turn a large blob
 into smaller ones. It is thus possible to cut out single bytes from a
 blob and use them construct new blobs by appending copies of these
 sample bytes together with the BlobBuilder.

 As sample bytes can only be generated by writing text into the blob as
 utf-8, it is not possible to generate all 8-bit patterns. More
 specifically, the bit patterns that are illegal in any given utf-8
 string, are not possible to generate using the current API. This leads
 developers into using various hacks to overcome the issue.

 For example one could ask the user to provide a file with sample
 bytes. Methods for reading a file into a blob are available, so there
 is no need to generate the sample bytes. There is however no way to
 make sure that the user provided a file with correct bit-patterns as
 the blob interface is to a large degree a read-only interface. A
 demonstration of using a file to overcome the issue is available at
 http://www.cs.helsinki.fi/u/twruottu/testi/bpost.html

 To solve the problem a third append function which accepts a list of
 integers, and adds the corresponding bytes into the blob is needed.
 Once such method has been introduced it should be possible to write
 the 7-bit ascii string foo into a BlobBuilder by doing...
 bb.append([102,111,111]);

 It should also be possible to add bytes 0x00 and 0xff into the
 BlobBuilder by doing...
 bb.append([0,255]);

 Would such append method be a possible addition as I defined it? Is
 there some alternative way for achieving the same result. Which
 specification should such feature go into?


  thank you for your time, --Toni Ruottu

Re: CfC: WD of File API; deadline October 22

[Eric Uhrhane]

I support this.

On Mon, Oct 18, 2010 at 7:03 AM, Arthur Barstow art.bars...@nokia.com wrote:

  Arun and Jonas would like to publish a new Working Draft of the File API
 spec and this is Call for Consensus to do so:

  http://dev.w3.org/2006/webapi/FileAPI/

 As with all of our CfCs, positive response is preferred and encouraged and
 silence will be assumed to be assent.

 The deadline for comments is October 22. This comment period is a bit
 shorter than our typical 1-week review period but this will allow us to sync
 the publication of this spec with the File API: {Writer, Directories and
 System} specs. If this short comment period causes any issues or concerns,
 please let us know.

 -Art Barstow

Re: Updates to FileAPI

[Eric Uhrhane]

On Mon, Oct 18, 2010 at 12:37 PM, Arun Ranganathan a...@mozilla.com wrote:
  On 10/12/10 2:24 PM, Eric Uhrhane wrote:

 On Tue, Oct 12, 2010 at 11:18 AM, Arun Ranganathana...@mozilla.com
  wrote:

  On 10/12/10 2:12 PM, Eric Uhrhane wrote:

 Arun:

 On Tue, Oct 12, 2010 at 10:46 AM, Arun Ranganathana...@mozilla.com
  wrote:

  WebApps WG,

 There have been some updates to the File API.

 http://dev.w3.org/2006/webapi/FileAPI/

 Notable changes are:

 1. Exception codes are no longer harnessed to DOMException's exception
 codes, per discussion on this listserv.

 2. Metadata attributes creationDate and lastModifiedDate have been
 added
 to
 the File object, per discussion on the WHATWG listserv.

 My apologies for not bringing this up earlier; I just realized that
 the discussions I've been involved with weren't on the public lists,
 and I didn't share their conclusions on the relevant threads.

 The creationDate is going to be problematic.  It works fine on
 Windows, but not on Unix-style systems.

 On Unix, ctime is *not* creation time.  It's the last time the inode
 was modified, so e.g. changing permission bits will alter it.
 http://en.wikipedia.org/wiki/MAC_times has a good writeup on ctime.

 I see that your spec allows for an empty string on systems where
 creationDate isn't available, but my gut response is not to add a
 Windows-specific feature.

 (Yes, this discussion also occurred on WHATWG).

 Well, there was discussion in May about how modificationDate might be
 earlier than creationDate on Windows, due to odd copy behavior, but no
 mention IIRC that this couldn't work at all on Unix systems, which I
 think is significant.

 Is your suggestion to *only* allow lastModifiedDate, and not to support
 creationDate at all, even though implementations can return the empty
 string
 on [Unix-style] systems where creationDate is unreliable?

 Yes.  If we publish APIs that only work on Windows, Windows-based
 developers will write code that works on their test machines, but
 fails elsewhere.  Web code is supposed to be more portable than that.

 Eric: I agree with this, so I'll strike creationDate as an attribute on File
 objects.

 (FWIW, though, the fact that we return ArrayBuffer objects on file reads,
 which allows for endian considerations, means that we already allow a back
 door for platform specific code).

Yeah, I just don't see any efficient way of getting around that.  And
at least you *can* write code that'll work on any platform--you just
have to be aware of the issues.  And libraries can be written that'll
wrap ArrayBuffers and take care of it for you, if that turns out to be
useful.

Thanks for the update.

Eric

Re: Updates to FileAPI

[Eric Uhrhane]

On Tue, Oct 12, 2010 at 10:46 AM, Arun Ranganathan a...@mozilla.com wrote:

  WebApps WG,

 There have been some updates to the File API.

 http://dev.w3.org/2006/webapi/FileAPI/

 Notable changes are:

 1. Exception codes are no longer harnessed to DOMException's exception
 codes, per discussion on this listserv.

 I've updated the exception codes in the FileWriter and FileSystem specs to
match, and extended appropriately.  Now that we're not tied to the
DOMException codes, I was able to add codes that had been requested for some
very file-system-specific errors.

2. Metadata attributes creationDate and lastModifiedDate have been added to
 the File object, per discussion on the WHATWG listserv.

 3. Blob no longer has a .url attribute.  Instead, Window and
 WorkerGlobalScope have been extended with methods createBlobURL and
 revokeBlobURL, per discussion on this listserv.

 4. The Blob URI section has undergone changes, per discussion on this
 listserv.  Notably, Blob supports the addition of an HTTP Content-Type,
 which implementations must return as a response header when Blob URIs are
 dereferenced.

 5. There are (ongoing) fixes to bugs, including incorrect uses of long long
 (in lieu of unsigned long long), per (ongoing) discussion on the listserv.

 I've updated the FileWriter and FileSystem specs to use unsigned long long
where appropriate as well.  Thanks for pointing that out.

In off-list discussion, a few points have come up, which I'll summarize
 below:

 1. The emerging HTML5 Device specification [1] includes a section on
 streams and an affiliated Stream API, which relies normatively on Blob URIs
 [2] defined in the File API.  Since we've eliminated the .url property on
 Blob, we should also eliminate the .url property on the Stream object.
  There has been some discussion on renaming the methods createBlobURL and
 revokeBlobURL to be more generic, so that use cases such as Stream can be
 accommodated.  This is an ongoing discussion.  In general, there's consensus
 around eliminating the .url attribute from Stream.

 2. There is ongoing discussion about the addition of Content-Disposition as
 well (as an attribute on Blob, as a parameter to Blob.slice, and as a
 response header when dereferencing blob: URIs), to facilitate and optimize
 downloads.  The use of a Content-Disposition header allows URLs (both
 http:// and blob:) to dereference content straight for download, with the
 benefit of header-determined processing (e.g. the download doesn't occur
 first).  Another suggestion to address this use case is that instead of
 supporting Content-Disposition, to allow for an additional URL property on
 the FileSaver constructor, modulo domain restrictions.  This discussion is
 ongoing.

 3. In general, there's ongoing discussion on allowing *even more* HTTP
 response behavior when dereferencing blob: URIs.  I strongly favor a strict
 subset of HTTP, and only a use-case driven addition of further response
 headers and response codes.  Arguments cited in favor of including all of
 http:// are that blob: URIs should be completely indistinguishable from
 HTTP URIs, thus allowing maximum reuse with XHR and other aspects of the
 platform.  Currently, I believe we allow for a great deal of intermingling
 without reproducing HTTP in its entirety within Blob URIs.

 -- A*

 [1] http://dev.w3.org/html5/html-device/
 [2] http://dev.w3.org/html5/html-device/#stream-api

Re: Updates to FileAPI

[Eric Uhrhane]

Arun:

On Tue, Oct 12, 2010 at 10:46 AM, Arun Ranganathan a...@mozilla.com wrote:
  WebApps WG,

 There have been some updates to the File API.

 http://dev.w3.org/2006/webapi/FileAPI/

 Notable changes are:

 1. Exception codes are no longer harnessed to DOMException's exception
 codes, per discussion on this listserv.

 2. Metadata attributes creationDate and lastModifiedDate have been added to
 the File object, per discussion on the WHATWG listserv.

My apologies for not bringing this up earlier; I just realized that
the discussions I've been involved with weren't on the public lists,
and I didn't share their conclusions on the relevant threads.

The creationDate is going to be problematic.  It works fine on
Windows, but not on Unix-style systems.

On Unix, ctime is *not* creation time.  It's the last time the inode
was modified, so e.g. changing permission bits will alter it.
http://en.wikipedia.org/wiki/MAC_times has a good writeup on ctime.

I see that your spec allows for an empty string on systems where
creationDate isn't available, but my gut response is not to add a
Windows-specific feature.

 3. Blob no longer has a .url attribute.  Instead, Window and
 WorkerGlobalScope have been extended with methods createBlobURL and
 revokeBlobURL, per discussion on this listserv.

 4. The Blob URI section has undergone changes, per discussion on this
 listserv.  Notably, Blob supports the addition of an HTTP Content-Type,
 which implementations must return as a response header when Blob URIs are
 dereferenced.

 5. There are (ongoing) fixes to bugs, including incorrect uses of long long
 (in lieu of unsigned long long), per (ongoing) discussion on the listserv.

I'll update my specs to use unsigned long long where appropriate as well.
I'll also be updating my extensions to FileError/FileException.  Shall
I leave a gap, or just start numbering where you left off?

 In off-list discussion, a few points have come up, which I'll summarize
 below:

 1. The emerging HTML5 Device specification [1] includes a section on
 streams and an affiliated Stream API, which relies normatively on Blob URIs
 [2] defined in the File API.  Since we've eliminated the .url property on
 Blob, we should also eliminate the .url property on the Stream object.
  There has been some discussion on renaming the methods createBlobURL and
 revokeBlobURL to be more generic, so that use cases such as Stream can be
 accommodated.  This is an ongoing discussion.  In general, there's consensus
 around eliminating the .url attribute from Stream.

 2. There is ongoing discussion about the addition of Content-Disposition as
 well (as an attribute on Blob, as a parameter to Blob.slice, and as a
 response header when dereferencing blob: URIs), to facilitate and optimize
 downloads.  The use of a Content-Disposition header allows URLs (both
 http:// and blob:) to dereference content straight for download, with the
 benefit of header-determined processing (e.g. the download doesn't occur
 first).  Another suggestion to address this use case is that instead of
 supporting Content-Disposition, to allow for an additional URL property on
 the FileSaver constructor, modulo domain restrictions.  This discussion is
 ongoing.

 3. In general, there's ongoing discussion on allowing *even more* HTTP
 response behavior when dereferencing blob: URIs.  I strongly favor a strict
 subset of HTTP, and only a use-case driven addition of further response
 headers and response codes.  Arguments cited in favor of including all of
 http:// are that blob: URIs should be completely indistinguishable from HTTP
 URIs, thus allowing maximum reuse with XHR and other aspects of the
 platform.  Currently, I believe we allow for a great deal of intermingling
 without reproducing HTTP in its entirety within Blob URIs.

 -- A*

 [1] http://dev.w3.org/html5/html-device/
 [2] http://dev.w3.org/html5/html-device/#stream-api

Re: Updates to FileAPI

[Eric Uhrhane]

On Tue, Oct 12, 2010 at 11:18 AM, Arun Ranganathan a...@mozilla.com wrote:
  On 10/12/10 2:12 PM, Eric Uhrhane wrote:

 Arun:

 On Tue, Oct 12, 2010 at 10:46 AM, Arun Ranganathana...@mozilla.com
  wrote:

  WebApps WG,

 There have been some updates to the File API.

 http://dev.w3.org/2006/webapi/FileAPI/

 Notable changes are:

 1. Exception codes are no longer harnessed to DOMException's exception
 codes, per discussion on this listserv.

 2. Metadata attributes creationDate and lastModifiedDate have been added
 to
 the File object, per discussion on the WHATWG listserv.

 My apologies for not bringing this up earlier; I just realized that
 the discussions I've been involved with weren't on the public lists,
 and I didn't share their conclusions on the relevant threads.

 The creationDate is going to be problematic.  It works fine on
 Windows, but not on Unix-style systems.

 On Unix, ctime is *not* creation time.  It's the last time the inode
 was modified, so e.g. changing permission bits will alter it.
 http://en.wikipedia.org/wiki/MAC_times has a good writeup on ctime.

 I see that your spec allows for an empty string on systems where
 creationDate isn't available, but my gut response is not to add a
 Windows-specific feature.

 (Yes, this discussion also occurred on WHATWG).

Well, there was discussion in May about how modificationDate might be
earlier than creationDate on Windows, due to odd copy behavior, but no
mention IIRC that this couldn't work at all on Unix systems, which I
think is significant.

 Is your suggestion to *only* allow lastModifiedDate, and not to support
 creationDate at all, even though implementations can return the empty string
 on [Unix-style] systems where creationDate is unreliable?

Yes.  If we publish APIs that only work on Windows, Windows-based
developers will write code that works on their test machines, but
fails elsewhere.  Web code is supposed to be more portable than that.

 3. Blob no longer has a .url attribute.  Instead, Window and
 WorkerGlobalScope have been extended with methods createBlobURL and
 revokeBlobURL, per discussion on this listserv.

 4. The Blob URI section has undergone changes, per discussion on this
 listserv.  Notably, Blob supports the addition of an HTTP Content-Type,
 which implementations must return as a response header when Blob URIs are
 dereferenced.

 5. There are (ongoing) fixes to bugs, including incorrect uses of long
 long
 (in lieu of unsigned long long), per (ongoing) discussion on the
 listserv.

 I'll update my specs to use unsigned long long where appropriate as well.
 I'll also be updating my extensions to FileError/FileException.  Shall
 I leave a gap, or just start numbering where you left off?

 I think you can probably start where what's in File API leaves off.  We
 should keep tightly in sync :-)

Sounds good; thanks.

Re: File descriptors

[Eric Uhrhane]

Luka:

 Have you looked at the filesystem spec [1]?  It might have some
of the features you're looking for.  It won't let you read arbitrary
local files directly, for security reasons, but if you generate or
download the files yourself, or if the user selects them for you, you
have full access to them.  Incidentally, you don't have to read an
entire File or Blob into memory at once if you don't want to; you can
use slice() to pull out just a part of it at a time.

 Eric

[1] http://dev.w3.org/2009/dap/file-system/file-dir-sys.html

2010/10/4  lmarce...@foi.hr:
 Dear people of W3C,
 My name is Luka Marčetić, I am a student at Faculty of Organization and
 Informatics in Croatia.
 This, I guess, would qualify as a feature request. Your FileAPI draft is a
 step in the direction I want to take, but it's not quite there yet. I need
 something to allow me to:

  * read local files,
  * show them in web documents, and
  * manipulate them

 FileAPI does the first thing, and partly the third one. The problem is
 that it, as I understand it, loads the complete file into memory
 (readAsBinaryString) which is inconvenient for big files, such as video.
 Expanding on the video example, DataURI scheme fails for the same reason.
 Being designed for addressing the second issue, it falls short to meet
 (my) real world needs. Finally, the file:// URI also needs mentioning
 here. It would be able to solve the second issue entirely, but is of
 nearly no use, being blocked by default by most or all browsers. I can't
 expect my future users to fiddle with browser configuration just to get my
 website working.

 To solve these issues, I would like to propose C-like file descriptors to
 be used coupled with a new file descriptor URI scheme. It would allow me
 to, for example, let the user choose a video file he wants to edit, and
 then show it in a video tag, like this:

 video src=fd://file controls=controls/video

 Lastly, a file manipulation mechanism would be useful for fully addressing
 the last of the three issues. File API mechanism is the only one of the
 three mentioned tools that partially addresses it. But it does so in a
 manner that simply necessitates its (imo) flaw. In my opinion, the correct
 way to deal with this would be a fread function, resembling more the PHP's
 fread function than C's counterpart (tellp/tellg, and fwrite implied).
 This modification would, I suppose, be done to the FileReader object:

 I hope you will see the merit of this. Thank you for your attention,
 --Luka Marčetić

Re: File descriptors

[Eric Uhrhane]

2010/10/4  lmarce...@foi.hr:
 Hello again,
 thanks for your replies, let me address them one by one:

 Eric: Does not slicing a Blob require for it to be read into memory first
 with functions such as getTextAsBinary?

No.  If you use FileReader.readAsBinaryString, you're reading it into
memory, and what you've got is a string, not a blob.  Just use slice()
on the original blob, which need not be in memory [in fact I think
most large blobs will reside on disk in most implementations].

 Boris: Might be, will see how other things play out.

 Jonas: I keep getting undefined for myFile.url.

 Thanks again.
 --Luka Marčetić

Re: Seeking agenda items for WebApps' Nov 1-2 f2f meeting

[Eric Uhrhane]

Works fine for me.  I'll be there all of Monday and Tuesday.  Due to
jetlag morning vs. afternoon's probably irrelevant to me, as I won't
have any idea what time it is ;'.

On Tue, Sep 28, 2010 at 2:30 PM, Jonas Sicking jo...@sicking.cc wrote:
 The later the better for me. If we can make it after noon I'll be
 there for sure.

 / Jonas

 On Tue, Sep 28, 2010 at 1:37 PM, Jeremy Orlow jor...@google.com wrote:
 I'm OK with any time slot.

 On Tue, Sep 28, 2010 at 8:57 PM, Arthur Barstow art.bars...@nokia.com
 wrote:

  Hi All,

 Currently, no one has requested a specific day + time slot for any of the
 proposed topics:

  http://www.w3.org/2008/webapps/wiki/TPAC2010

 When our IndexedDB participants agree on a time slot on Tuesday the 2nd,
 I'll add it to the agenda. Pablo, Jonas, Jeremy - please propose a time.

 Day + time slot proposals for the agenda topics already proposed are also
 welcome (as are proposals for additional topics).

 -Art Barstow

 On 9/28/10 3:28 PM, ext Pablo Castro wrote:

 It looks like there will be good critical mass for IndexedDB discussions,
 so I'll try to make it as well. Tuesday would be best for me as well for an
 IndexedDB meeting so I can travel on Sunday/Monday.

 -pablo

 -Original Message-
 From: Jonas Sicking [mailto:jo...@sicking.cc]
 Sent: Tuesday, September 28, 2010 10:53 AM
 To: Jeremy Orlow
 Cc: Pablo Castro; art.bars...@nokia.com; public-webapps
 Subject: Re: Seeking agenda items for WebApps' Nov 1-2 f2f meeting

 I'm not 100% sure that I'll make TPAC this year, but if I do, I likely
 won't make monday. So a tuesday schedule would fit me better too.

 / Jonas

 On Tue, Sep 28, 2010 at 8:36 AM, Jeremy Orlowjor...@google.com  wrote:

 Is it possible to schedule IndexedDB for Tuesday?  I'm pretty sure that
 I
 can be there then, but Monday is more up in the air at this moment.
 Thanks!
 Jeremy
 On Thu, Sep 2, 2010 at 3:28 AM, Jonas Sickingjo...@sicking.cc  wrote:

 I'm hoping to be there yes. Especially if we'll get a critical mass of
 IndexedDB contributors.

 / Jonas

 On Wed, Sep 1, 2010 at 7:18 PM, Pablo
 Castropablo.cas...@microsoft.com
 wrote:

 -Original Message-
 From: public-webapps-requ...@w3.org
 [mailto:public-webapps-requ...@w3.org] On Behalf Of Arthur Barstow
 Sent: Tuesday, August 31, 2010 4:32 AM

 The WebApps WG will meet face-to-face November 1-2 as part of the
 W3C's
 2010 TPAC meeting week [TPAC].

 I created a stub agenda item page and seek input to flesh out
 agenda:

 http://www.w3.org/2008/webapps/wiki/TPAC2010

 [TPAC] includes a link to the Registration page, a detailed schedule
 of
 the group meetings, and other useful information.

 The registration fee is 40€ per day and will increase to 120€ per
 day
 after October 22.

 -Art Barstow

 [TPAC] http://www.w3.org/2010/11/TPAC/

 For folks working on IndexedDB, are you guys planning on attending the
 TPAC? Given the timing of the event it may be a great opportunity to
 get
 together and iron out a whole bunch of issues at once. It would be
 good to
 know ahead of time so we can all make plans if we have critical mass.

 Thanks
 -pablo

Re: Small tweak to FileWriter's event sequences

[Eric Uhrhane]

I've now made this change.

On Fri, Sep 24, 2010 at 1:15 PM, Eric Uhrhane er...@google.com wrote:
 The abort sequence in FileWriter looks like this:

 If readyState is DONE, throw a FileException with error code
 INVALID_STATE_ERR and terminate this overall series of steps.
 Set readyState to DONE.
 Terminate any steps having to do with writing a file.
 Dispatch a progress event called error. Set the error attribute to a
 FileError object with the appropriate code (in this case, ABORT_ERR;
 see error conditions).
 Dispatch a progress event called abort
 Dispatch a progress event called writeend
 Stop dispatching any further progress events.

 In this sequence, readyState gets set to DONE before any of the
 progress events get sent out.  This leaves open a window of
 opportunity in which one could start another write/truncate [e.g. in
 onError] before the abort and writeend events get dispatched.  At that
 point you'd get a writeend error that didn't refer to the write in
 progress, which is at best nonintuitive.

 I'm planning to move the setting of readyState to just before the
 writeEnd event gets dispatched.
 That way readyState==DONE always means it's safe to start another
 operation, and an abort or writeend event is always  unambiguously
 tied to a particular write.

 Similarly, in the FileWriter's write and truncate sequences,
 readyState gets set to DONE before the error and writeend progress
 events are dispatched.  I'm going to make the same change there, for
 the same reasons, and I'll alter the FileSaver's event sequence to
 match.

 Let me know if you see any problem with this.

 Thanks,

     Eric

Re: FileSystem API - overwrite flag for copy/move?

[Eric Uhrhane]

2010/9/16 Eric Uhrhane er...@google.com:
 How about this?

 For a move/copy of a file on top of existing file, or a directory on
 top of an existing empty directory, you get an automatic overwrite.
 A move/copy of a file on top of an existing directory, or of a
 directory on top of an existing file, will always fail.
 A move/copy of a file or directory on top of an existing non-empty
 directory will always fail.

 That matches Posix[1] rename behavior, and should cover most or all of
 the normal use cases.
 If necessary, we can consider adding a don't overwrite flag, but
 that may be difficult to implement without race conditions on all
 platforms.

I've added comments to this effect to the moveTo definitions.

 Regarding recursive deletion of directories:

 One option is to add a flag to remove(); that flag will be ignored if
 the Entry is a file, so it's not as clean as it might be, but it keeps
 the interface small.
 Another is to add a removeRecursively() method to DirectoryEntry; this
 makes it really clear what's going on, and might prevent some
 accidental deletions.

I've added removeRecursively to DirectoryEntry and DirectoryEntrySync.

 Which do you prefer?

 [1] http://www.opengroup.org/onlinepubs/009695399/functions/rename.html

 2010/9/9 Kinuko Yasuda kin...@chromium.org:
 On Thu, Sep 9, 2010 at 12:37 AM, Ian Fette (イアンフェッティ) ife...@google.com
 wrote:

 I think recursive copy/remove is a very valid use case. As for overwrite,
 is a flag necessary? On most OSes you already get overwrite as the default
 behaviour (at least from APIs, many interactive UAs such as Explorer on
 windows will prompt), is there a compelling argument why it should be
 different for a web api?

 Making overwriting mode default for copy/move sounds reasonable to me too.
  Especially if we allow recursive remove (and I think there would be more
 need for this one) it'd look more consistent.
 As for providing options, I was a bit concerned about the complexity in
 programming with nested callbacks, but in this case it's not a big deal
 (we'll need only two) and wouldn't be a problem.
 I'm more concerned with recursive remove though.  The js code to do that
 isn't very pretty.

 On Thu, Sep 9, 2010 at 12:22 AM, Kinuko Yasuda kin...@chromium.org
 wrote:

 On Thu, Sep 9, 2010 at 12:12 AM, Kinuko Yasuda kin...@chromium.org
 wrote:

 On Tue, Sep 7, 2010 at 6:12 PM, Eric Uhrhane er...@google.com wrote:

 On Mon, Aug 30, 2010 at 9:27 PM, Kinuko Yasuda kin...@chromium.org
 wrote:
  Hi,
  I have a question about Entry.moveTo/copyTo behavior defined in
  the File API: Directories and System [1].
  Currently the API doesn't specify how Entry.moveTo() and copyTo()
  should
  behave
  when a source entry is a file and there's *already* a file at the
  destination path.
  Should UAs overwrite the existing file at the destination path or
  not?
  Or maybe we should add an 'overwrite' flag to indicate if the script
  wants
  to overwrite an existing file or not?

 I'm open to a flag.  We're already up to 4 parameters to each of those
 methods, though...5 is a bit ungainly.  I'm concerned that we might
 find another flag to add at some point, and we'd then be up to 6.
 What about adding an flags object, as in getFile, to allow for
 expansion?

 Adding a flag or flags object (suppose the other thread about Flags will
 be settled) sounds good to me.
 Or I think it's also ok to explicitly disallow overwriting copy/move,
 i.e. specify that 'it is an error to copy or move a file or directory if
 there's already an entry'.  In this case it might be better to have 
 another
 error code like ENTRY_EXISTS_ERR so that the user script can act
 differently.  (But in general having a handy option would make programming
 much easier in async context where every operation requires one or two
 callbacks.)
 If we're going to add 'overwrite' flag, there'll be a few more things to
 be made clear.
 For example I wonder how the overwriting copy/move should behave when
 there's already a **directory** at the destination path/name.
 Should the UA remove the existing directory and create a new entry at
 the same path?
 This sounds reasonable but it'll also provide a handy alternative way to
 remove a directory recursively.

 By the way how do you think about recursive remove?
 Is there a reason (or past discussion) not to have recursive option in
 remove?  (I mean, other than the fact that adding more and more options to
 every method doesn't look very clean.)
 I found that it's not very easy to remove a directory when there're
 children in it -- it requires multiple DirectoryReader.readEntries and
 Entry.remove in a nested way.
 Thanks,

 Or should the UA create a new entry *under* the directory?
 This behavior doesn't sound like 'overwriting'.  The resulting path will
 be like 'destParentPath/name/name' which doesn't sound quite consistent 
 with
 the spec either.

  Similarly I wondered if we'd want to have a 'recursive' flag for
  moveTo/copyTo.
  I think

Small tweak to FileWriter's event sequences

[Eric Uhrhane]

The abort sequence in FileWriter looks like this:

If readyState is DONE, throw a FileException with error code
INVALID_STATE_ERR and terminate this overall series of steps.
Set readyState to DONE.
Terminate any steps having to do with writing a file.
Dispatch a progress event called error. Set the error attribute to a
FileError object with the appropriate code (in this case, ABORT_ERR;
see error conditions).
Dispatch a progress event called abort
Dispatch a progress event called writeend
Stop dispatching any further progress events.

In this sequence, readyState gets set to DONE before any of the
progress events get sent out.  This leaves open a window of
opportunity in which one could start another write/truncate [e.g. in
onError] before the abort and writeend events get dispatched.  At that
point you'd get a writeend error that didn't refer to the write in
progress, which is at best nonintuitive.

I'm planning to move the setting of readyState to just before the
writeEnd event gets dispatched.
That way readyState==DONE always means it's safe to start another
operation, and an abort or writeend event is always  unambiguously
tied to a particular write.

Similarly, in the FileWriter's write and truncate sequences,
readyState gets set to DONE before the error and writeend progress
events are dispatched.  I'm going to make the same change there, for
the same reasons, and I'll alter the FileSaver's event sequence to
match.

Let me know if you see any problem with this.

Thanks,

 Eric

Re: FileSystem API - overwrite flag for copy/move?

[Eric Uhrhane]

How about this?

For a move/copy of a file on top of existing file, or a directory on
top of an existing empty directory, you get an automatic overwrite.
A move/copy of a file on top of an existing directory, or of a
directory on top of an existing file, will always fail.
A move/copy of a file or directory on top of an existing non-empty
directory will always fail.

That matches Posix[1] rename behavior, and should cover most or all of
the normal use cases.
If necessary, we can consider adding a don't overwrite flag, but
that may be difficult to implement without race conditions on all
platforms.

Regarding recursive deletion of directories:

One option is to add a flag to remove(); that flag will be ignored if
the Entry is a file, so it's not as clean as it might be, but it keeps
the interface small.
Another is to add a removeRecursively() method to DirectoryEntry; this
makes it really clear what's going on, and might prevent some
accidental deletions.

Which do you prefer?

[1] http://www.opengroup.org/onlinepubs/009695399/functions/rename.html

2010/9/9 Kinuko Yasuda kin...@chromium.org:
 On Thu, Sep 9, 2010 at 12:37 AM, Ian Fette (イアンフェッティ) ife...@google.com
 wrote:

 I think recursive copy/remove is a very valid use case. As for overwrite,
 is a flag necessary? On most OSes you already get overwrite as the default
 behaviour (at least from APIs, many interactive UAs such as Explorer on
 windows will prompt), is there a compelling argument why it should be
 different for a web api?

 Making overwriting mode default for copy/move sounds reasonable to me too.
  Especially if we allow recursive remove (and I think there would be more
 need for this one) it'd look more consistent.
 As for providing options, I was a bit concerned about the complexity in
 programming with nested callbacks, but in this case it's not a big deal
 (we'll need only two) and wouldn't be a problem.
 I'm more concerned with recursive remove though.  The js code to do that
 isn't very pretty.

 On Thu, Sep 9, 2010 at 12:22 AM, Kinuko Yasuda kin...@chromium.org
 wrote:

 On Thu, Sep 9, 2010 at 12:12 AM, Kinuko Yasuda kin...@chromium.org
 wrote:

 On Tue, Sep 7, 2010 at 6:12 PM, Eric Uhrhane er...@google.com wrote:

 On Mon, Aug 30, 2010 at 9:27 PM, Kinuko Yasuda kin...@chromium.org
 wrote:
  Hi,
  I have a question about Entry.moveTo/copyTo behavior defined in
  the File API: Directories and System [1].
  Currently the API doesn't specify how Entry.moveTo() and copyTo()
  should
  behave
  when a source entry is a file and there's *already* a file at the
  destination path.
  Should UAs overwrite the existing file at the destination path or
  not?
  Or maybe we should add an 'overwrite' flag to indicate if the script
  wants
  to overwrite an existing file or not?

 I'm open to a flag.  We're already up to 4 parameters to each of those
 methods, though...5 is a bit ungainly.  I'm concerned that we might
 find another flag to add at some point, and we'd then be up to 6.
 What about adding an flags object, as in getFile, to allow for
 expansion?

 Adding a flag or flags object (suppose the other thread about Flags will
 be settled) sounds good to me.
 Or I think it's also ok to explicitly disallow overwriting copy/move,
 i.e. specify that 'it is an error to copy or move a file or directory if
 there's already an entry'.  In this case it might be better to have another
 error code like ENTRY_EXISTS_ERR so that the user script can act
 differently.  (But in general having a handy option would make programming
 much easier in async context where every operation requires one or two
 callbacks.)
 If we're going to add 'overwrite' flag, there'll be a few more things to
 be made clear.
 For example I wonder how the overwriting copy/move should behave when
 there's already a **directory** at the destination path/name.
 Should the UA remove the existing directory and create a new entry at
 the same path?
 This sounds reasonable but it'll also provide a handy alternative way to
 remove a directory recursively.

 By the way how do you think about recursive remove?
 Is there a reason (or past discussion) not to have recursive option in
 remove?  (I mean, other than the fact that adding more and more options to
 every method doesn't look very clean.)
 I found that it's not very easy to remove a directory when there're
 children in it -- it requires multiple DirectoryReader.readEntries and
 Entry.remove in a nested way.
 Thanks,

 Or should the UA create a new entry *under* the directory?
 This behavior doesn't sound like 'overwriting'.  The resulting path will
 be like 'destParentPath/name/name' which doesn't sound quite consistent 
 with
 the spec either.

  Similarly I wondered if we'd want to have a 'recursive' flag for
  moveTo/copyTo.
  I think for directories we can simply assume that the user wants to
  move/copy
  them recursively, but it might be good to add some notion about that
  in the
  spec.

 How about I add a note indicating

Re: FileSystem API - overwrite flag for copy/move?

[Eric Uhrhane]

2010/9/16 Kinuko Yasuda kin...@chromium.org:
 On Thu, Sep 16, 2010 at 2:50 PM, Eric Uhrhane er...@google.com wrote:
 How about this?

 For a move/copy of a file on top of existing file, or a directory on
 top of an existing empty directory, you get an automatic overwrite.
 A move/copy of a file on top of an existing directory, or of a
 directory on top of an existing file, will always fail.
 A move/copy of a file or directory on top of an existing non-empty
 directory will always fail.

 This sounds good to me.
 I think sticking to the compatibility with Posix would make sense to
 developers too.

 That matches Posix[1] rename behavior, and should cover most or all of
 the normal use cases.
 If necessary, we can consider adding a don't overwrite flag, but
 that may be difficult to implement without race conditions on all
 platforms.

 We can achieve this in a safer way by combining getFile/getDirectory
 with exclusive flag and move/copy, so I think it's reasonable not to
 have don't overwrite flag.

 Regarding recursive deletion of directories:

 One option is to add a flag to remove(); that flag will be ignored if
 the Entry is a file, so it's not as clean as it might be, but it keeps
 the interface small.
 Another is to add a removeRecursively() method to DirectoryEntry; this
 makes it really clear what's going on, and might prevent some
 accidental deletions.

 I have no strong opinion on this but removeRecursively might be error prone?
 (Recursive remove can be very destructive and it should be triggered
 with an explicit intention.)

Which do you think is more error-prone, a method called
removeRecursively, or a boolean flag to remove?

 Which do you prefer?

 [1] http://www.opengroup.org/onlinepubs/009695399/functions/rename.html

 2010/9/9 Kinuko Yasuda kin...@chromium.org:
 On Thu, Sep 9, 2010 at 12:37 AM, Ian Fette (イアンフェッティ) ife...@google.com
 wrote:

 I think recursive copy/remove is a very valid use case. As for overwrite,
 is a flag necessary? On most OSes you already get overwrite as the default
 behaviour (at least from APIs, many interactive UAs such as Explorer on
 windows will prompt), is there a compelling argument why it should be
 different for a web api?

 Making overwriting mode default for copy/move sounds reasonable to me too.
  Especially if we allow recursive remove (and I think there would be more
 need for this one) it'd look more consistent.
 As for providing options, I was a bit concerned about the complexity in
 programming with nested callbacks, but in this case it's not a big deal
 (we'll need only two) and wouldn't be a problem.
 I'm more concerned with recursive remove though.  The js code to do that
 isn't very pretty.

 On Thu, Sep 9, 2010 at 12:22 AM, Kinuko Yasuda kin...@chromium.org
 wrote:

 On Thu, Sep 9, 2010 at 12:12 AM, Kinuko Yasuda kin...@chromium.org
 wrote:

 On Tue, Sep 7, 2010 at 6:12 PM, Eric Uhrhane er...@google.com wrote:

 On Mon, Aug 30, 2010 at 9:27 PM, Kinuko Yasuda kin...@chromium.org
 wrote:
  Hi,
  I have a question about Entry.moveTo/copyTo behavior defined in
  the File API: Directories and System [1].
  Currently the API doesn't specify how Entry.moveTo() and copyTo()
  should
  behave
  when a source entry is a file and there's *already* a file at the
  destination path.
  Should UAs overwrite the existing file at the destination path or
  not?
  Or maybe we should add an 'overwrite' flag to indicate if the script
  wants
  to overwrite an existing file or not?

 I'm open to a flag.  We're already up to 4 parameters to each of those
 methods, though...5 is a bit ungainly.  I'm concerned that we might
 find another flag to add at some point, and we'd then be up to 6.
 What about adding an flags object, as in getFile, to allow for
 expansion?

 Adding a flag or flags object (suppose the other thread about Flags will
 be settled) sounds good to me.
 Or I think it's also ok to explicitly disallow overwriting copy/move,
 i.e. specify that 'it is an error to copy or move a file or directory if
 there's already an entry'.  In this case it might be better to have 
 another
 error code like ENTRY_EXISTS_ERR so that the user script can act
 differently.  (But in general having a handy option would make 
 programming
 much easier in async context where every operation requires one or two
 callbacks.)
 If we're going to add 'overwrite' flag, there'll be a few more things to
 be made clear.
 For example I wonder how the overwriting copy/move should behave when
 there's already a **directory** at the destination path/name.
 Should the UA remove the existing directory and create a new entry at
 the same path?
 This sounds reasonable but it'll also provide a handy alternative way to
 remove a directory recursively.

 By the way how do you think about recursive remove?
 Is there a reason (or past discussion) not to have recursive option in
 remove?  (I mean, other than the fact that adding more and more options to
 every method doesn't look very clean.)
 I

Re: ArrayBuffer and ByteArray questions

[Eric Uhrhane]

On Wed, Sep 15, 2010 at 6:53 PM, Arun Ranganathan a...@mozilla.com wrote:
 On 9/7/10 10:08 PM, Darin Fisher wrote:

 On Tue, Sep 7, 2010 at 5:23 PM, Kenneth Russell k...@google.com wrote:

 On Tue, Sep 7, 2010 at 4:19 PM, Nathan nat...@webr3.org wrote:
  Jian Li wrote:
 
  Hi,
 
  Several specs, like File API and WebGL, use ArrayBuffer, while other
  spec,
  like XMLHttpRequest Level 2, use ByteArray. Should we change to use the
  same
  name all across our specs? Since we define ArrayBuffer in the Typed
  Arrays
  spec (
 
 
  https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/doc/spec/TypedArray-spec.html),
  should we favor ArrayBuffer?
 
  In addition, can we consider adding ArrayBuffer support to BlobBuilder,
  FormData, and XMLHttpRequest.send()?
 
  which reminds me, I meant to ask if the aforementioned TypedArray spec
  should be brought in to webapps / w3c land? seems to complement the
  other
  base types used in webidl etc rather well + my gut reaction was why
  isn't
  this standardized within w3c?

 There's no particular reason why the Typed Array spec is being
 standardized in the Khronos group, aside from the fact that these
 array-like types originated in the WebGL spec and have evolved to meet
 use cases specified by WebGL. We have been hoping that they would have
 uses outside of WebGL, and some discussions have occurred with working
 groups such as TC39 to see how they might be better specified and
 standardized. We'd be open to hosting the spec development elsewhere.

 Vlad mentioned to me off-list that Mozilla has implemented an
 experimental mozResponseArrayBuffer on XHR objects, and will likely do
 the same on the File API to add a readAsArrayBuffer alongside
 readAsBinaryString etc.

 -Ken



 It sounds like ArrayBuffer is the name that is gaining traction (to circle
 back to Jian's original question about naming).

 In fact, readAsArrayBuffer / ArrayBuffer is used with FileReader, and will
 be the names going forward.  ArrayBuffer is gaining traction as the used
 name :)

 -- A*

I've added BlobBuilder.append(ArrayBuffer).

Eric

Re: File API exception codes

[Eric Uhrhane]

On Thu, Sep 9, 2010 at 6:39 AM, Arun Ranganathan
aranganat...@mozilla.com wrote:
 - Original Message -
 On Wed, Sep 8, 2010 at 12:43 AM, Anne van Kesteren ann...@opera.com
 wrote:

 
  That works for me too, but then please use internally consistent
  numbering
  rather than some codes matching DOMException and the new codes not
  matching
  DOMException as that is just too confusing, especially going
  forward. I.e.
  DOMException might gain a similar exception but it will have a
  different
  number, so only for the older numbers it will match, etc. It just
  does not
  make much sense.

 OK, so we stick with the current interfaces, but try to keep the
 numbers all matching/nonconflicting. Works for me.

 OK, we'll use in-order numbering and not have any kind of dependency on 
 DOMException.

Sorry, I misread the conclusion.  In-order numbering does make more
sense.  Let me know when you're about to update your numbering, and
I'll fix mine to go with them.

Re: Seeking agenda items for WebApps' Nov 1-2 f2f meeting

[Eric Uhrhane]

On Sat, Sep 11, 2010 at 5:43 AM, Arthur Barstow art.bars...@nokia.com wrote:
  On 9/11/10 6:18 AM, ext Anne van Kesteren wrote:

 On Tue, 31 Aug 2010 13:31:50 +0200, Arthur Barstowart.bars...@nokia.com
 wrote:

 I created a stub agenda item page and seek input to flesh out agenda:

 http://www.w3.org/2008/webapps/wiki/TPAC2010

 I would like to discuss specifics of Web DOM Core with various
 implementors. I added Web DOM Core to the page for now. I will probably
 flush it out some more over the next couple of weeks.


 Thanks Anne. So the list of agenda topics now is: Web DOM Core, Indexed
 Database API and DOM 3 Events.

 What is the level of interest, by Editors and others, in including
 additional WebApps' specs on the Nov 1-2 agenda:

  http://www.w3.org/2008/webapps/wiki/PubStatus#API_Specifications

 In particular:

 * File * specs - Eric, Arun are you going to attend this meeting?

I'll be there, and am happy to talk about File stuff.  We're moving
right along on a first implementation of all 3 File specs.

 * Web IDL - Cameron - will you attend this meeting?

 * CORS, UMP - Anne will attend but what about MarkM and Tyler? Jeff, Thomas
 - are you planning some type of Web Application Security meeting/BoF?

 * HTML5 APIs (server-events, Web *) - Hixie - will you attend this meeting?

 * Others?

 Of course we should continue to use our mail lists for technical discussions
 even if a spec is on the f2f agenda.

 FYI, the widgets group is not planning to meet those days.

 -Art Barstow

Re: ArrayBuffer and ByteArray questions

[Eric Uhrhane]

On Tue, Sep 7, 2010 at 4:09 PM, Jian Li jia...@chromium.org wrote:
 Hi,
 Several specs, like File API and WebGL, use ArrayBuffer, while other spec,
 like XMLHttpRequest Level 2, use ByteArray. Should we change to use the same
 name all across our specs? Since we define ArrayBuffer in the Typed Arrays
 spec
 (https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/doc/spec/TypedArray-spec.html),
 should we favor ArrayBuffer?
 In addition, can we consider adding ArrayBuffer support to BlobBuilder,
 FormData, and XMLHttpRequest.send()?

It seems like an obvious addition for BlobBuilder or XHR.send, but do
we need it in both, or is one sufficient?

 Thanks,
 Jian

Re: File API exception codes

[Eric Uhrhane]

On Wed, Sep 8, 2010 at 12:43 AM, Anne van Kesteren ann...@opera.com wrote:
 On Wed, 08 Sep 2010 06:12:36 +0200, Arun Ranganathan
 aranganat...@mozilla.com wrote:

 So I shall do as you did in the erstwhile Web SQL Database API.  Perhaps I
 can keep the codes that I'm reusing from DOMException as is, so that they
 are consistent with DOMException (for greater developer familiarity).  But
 in the future, additional exceptions that are File (or Blob) centric needn't
 keep pace with DOMException's exception codes.

 I would prefer if you either used consistent numbering or otherwise used
 DOMException. We can design a new DOMError that follows the Web SQL Database
 API conventions. After all, we are creating a new version of DOM Core.


 Which brings us back to having a dedicated exception and error interface
 for the File* APIs scenario, despite how attractive it is to have
 DOMException reused here.  This allows specific codes that are File/Blob
 centric anyway.  Cool?

 That works for me too, but then please use internally consistent numbering
 rather than some codes matching DOMException and the new codes not matching
 DOMException as that is just too confusing, especially going forward. I.e.
 DOMException might gain a similar exception but it will have a different
 number, so only for the older numbers it will match, etc. It just does not
 make much sense.

OK, so we stick with the current interfaces, but try to keep the
numbers all matching/nonconflicting.  Works for me.

Re: Flags for getFile/getDirectory (Re: New draft of FileSystem API posted)

[Eric Uhrhane]

On Tue, Sep 7, 2010 at 11:13 PM, Ian Hickson i...@hixie.ch wrote:
 On Tue, 7 Sep 2010, Darin Fisher wrote:

 Based on the ValidityState example, it seems that the members of Flags
 should be camelCase then instead of UPPERCASE?

 The platform convention, insofar as there is a convention, is that
 constants are uppercase, members are camelCase, and interfaces and
 constructors are TitleCase.

I was thinking of those as constants, but really they're members, so
yes, they should be camelCase.  I'll fix them.

 --
 Ian Hickson               U+1047E                )\._.,--,'``.    fL
 http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
 Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: File API exception codes

[Eric Uhrhane]

On Mon, Sep 6, 2010 at 12:27 PM, Ian Hickson i...@hixie.ch wrote:
 On Mon, 6 Sep 2010, Nathan wrote:

 Just noticed that File API specifies NOT_READABLE_ERR as code 24,
 whereas 24 is already used for DATA_CLONE_ERR
 http://dev.w3.org/html5/spec/common-dom-interfaces.html#data_clone_err

 Not sure if this is an issue or not, but a heads up regardless.

 I've updated HTML5 to use 25 for DATA_CLONE_ERR. We're keeping an informal
 registry of codes here:

   http://wiki.whatwg.org/wiki/Exception_Codes

 I've added NOT_READABLE_ERR and ENCODING_ERR to the wiki.

 Why does File API use FileError/FileException and not DOMException? If it
 doesn't use DOMException, these don't actually clash, and it seems weird
 that we would use the same numbers. Can this be fixed?

I'd be happy to use DOMException in FileWriter and FileSystem if Arun
switched his API over.

 Same question for the File System and File Writer APIs.

 To add exception codes, just do something like what HTML5 does:

  [Supplemental] exception DOMException {
    const unsigned short URL_MISMATCH_ERR = 21;
    const unsigned short QUOTA_EXCEEDED_ERR = 22;
    const unsigned short DATA_CLONE_ERR = 25;
  };

 --
 Ian Hickson               U+1047E                )\._.,--,'``.    fL
 http://ln.hixie.ch/       U+263A                /,   _.. \   _\  ;`._ ,.
 Things that are impossible just take longer.   `._.-(,_..'--(,_..'`-.;.'

Re: Flags for getFile/getDirectory (Re: New draft of FileSystem API posted)

[Eric Uhrhane]

On Tue, Sep 7, 2010 at 12:41 AM, Kinuko Yasuda kin...@chromium.org wrote:
 Hi Eric,
 (re-sending from the correct address)

 I've been re-reading the spec and started wondering if we really want to
 have a new interface / javascript object for Flags.
 The Flags interface is used to specify two behavioral options (CREATE and
 EXCLUSIVE) for DirectoryEntry.getFile and getDirectory and defined as
 following:

 4.2. The Flags Interface
 http://dev.w3.org/2009/dap/file-system/file-dir-sys.html#the-flags-interface

 interface Metadata {

 attribute boolean CREATE;

 attribute boolean EXCLUSIVE;

 };

 (by the way I assumed you meant Flags but not Metadata.)

 As the example code shows (4.2.2 Examples), each Flags object will likely be
 used as a 'one-time' object just to specify the behavior per each
 getFile/getDirectory.  If that's the case, isn't it simpler/easier to use a
 bitfield and use single unsigned int parameter for the flags [proposal 1]?

That was my original proposal; see the very brief discussion at
http://lists.w3.org/Archives/Public/public-device-apis/2010Apr/0036.html.
 Robin [+CC] suggested we switch to an interface object.  I think the
current form is nicely explicit and easy to understand and use.  The
bitflags are traditional in POSIX, but not very JavaScript-ish.

  Or if  we're not going to  add more options it might be also good to simply
 use two separate boolean parameters [proposal 2].

I don't like this.  The more parameters you have, the harder the
methods get to use.  This mechanism would make expanding the API quite
messy.  Also, having two boolean parameters right next to each other
is error-prone.

I think that ORing together bits is better than adding more
parameters, and I think the current proposal is cleaner than that.
But I'm open to debate, of course.

 [Proposal 1]

 interface DirectoryEntry {

      const unsigned short CREATE = 0x01;

      const unsigned short EXCLUSIVE = 0x02;
      void  getFile(in DOMString path, optional unsigned short options, ...);

      void getDirectory(in DOMString path, optional unsigned short options,
 ...);

       ...

 };

 [Proposal 2]

 interface DirectoryEntry {

      void  getFile(in DOMString path, bool create, bool exclusive, ...);

      void getDirectory(in DOMString path, bool create, bool exclusive, ...);

        ...

 };

 What do you think?

 Thanks,

 Kinuko

 On Wed, Jul 7, 2010 at 6:50 PM, Eric Uhrhane er...@google.com wrote:

 I've posted a new draft of File API: Directories and System [1].  In
 this draft I've rolled in quite a bit of feedback that I received
 since first posting it on DAP--many apologies for the delay.  This is
 the first draft produced since we agreed to move this spec from DAP to
 WebApps; I hope those of you who have time will give it a look and let
 me know what you think.

 In general I've tried to address any comment I was sent and had not
 already addressed via email.  The few that didn't make it in, I've
 responded to below.

 My thanks to Robin Berjon and Mike Clement for all their feedback.

 Robin:
  - data stored there by the application should not be deleted by the
 UA without user intervention, UA should require permission from the
 user, The application may of course delete it at will - these
 sound like real conformance statements, therefore SHOULD, SHOULD NOT,
 and MAY.

 Those are in a non-normative section; is that language still appropriate
 there?

 Robin:
 [discussion about speccing the URI format]

 Left as an open issue.

 Mike:
 [discussion about multiple sandboxes per origin]

 I think that would be very easy and clean to add later if desired, and
 in the mean time, one can use subdirectories.

 Mike:
 getFile/getDirectory are a bit overloaded.  How about including
 methods like exists(), createFile() and createDirectory()?  Though
 these methods are easily implemented in terms of getFile/getDirectory,
 I always prefer more direct API methods that help make the code easier
 to understand.  I expect, though, that you are attempting to be a low
 level as possible here.

 As Robin pointed out, adding extra round-trips will slow things down.
 Also, it can encourage race conditions.  These are easy for libraries
 to implement via wrappers.

 Mike:
 [request for creation time in getMetadata]

 It may be hard to support reliably cross-platform [2].

 Robin:
 [specifying a single locale everywhere]

 I don't think that'll make folks very happy if it's not their locale.
 If I e.g. try to force my locale on Turkish Windows users, they're
 going to see some interesting errors trying to share files with apps
 outside the browser, or for that matter even saving certain groups of
 files from inside the browser.

    Eric

 [1] http://dev.w3.org/2009/dap/file-system/file-dir-sys.html
 [2] http://en.wikipedia.org/wiki/MAC_times

Re: FileSystem API - overwrite flag for copy/move?

[Eric Uhrhane]

On Mon, Aug 30, 2010 at 9:27 PM, Kinuko Yasuda kin...@chromium.org wrote:
 Hi,
 I have a question about Entry.moveTo/copyTo behavior defined in
 the File API: Directories and System [1].
 Currently the API doesn't specify how Entry.moveTo() and copyTo() should
 behave
 when a source entry is a file and there's *already* a file at the
 destination path.
 Should UAs overwrite the existing file at the destination path or not?
 Or maybe we should add an 'overwrite' flag to indicate if the script wants
 to overwrite an existing file or not?

I'm open to a flag.  We're already up to 4 parameters to each of those
methods, though...5 is a bit ungainly.  I'm concerned that we might
find another flag to add at some point, and we'd then be up to 6.
What about adding an flags object, as in getFile, to allow for
expansion?

 Similarly I wondered if we'd want to have a 'recursive' flag for
 moveTo/copyTo.
 I think for directories we can simply assume that the user wants to
 move/copy
 them recursively, but it might be good to add some notion about that in the
 spec.

How about I add a note indicating that directory copies are always recursive?
I don't think we need anything for move.

 Thanks,
 [1] http://dev.w3.org/2009/dap/file-system/file-dir-sys.html

Re: Using FileWriter.truncate() to extend the length of a file?

[Eric Uhrhane]

On Thu, Aug 26, 2010 at 2:04 PM, Darin Fisher da...@chromium.org wrote:
 I noticed that FileWriter.truncate() can only be used to shorten a file, and
 there does not seem to be a good way to grow a file using FileWriter without
 appending data to it.  By contrast, the POSIX ftruncate function can be used
 to grow a file (zero padding it):
 From ftruncate(2):
   If the file previously was larger than this size, the extra data is lost.
  If  the  file  previously
    was shorter, it is extended, and the extended part reads as null bytes
 ('\0').

I see no reason we can't add this; I just left it off for simplicity,
but your use case makes sense.

 I think there are use cases for wanting to grow a file in advance of writing
 data to it.  For example, you might implement a system that downloads chunks
 of a file in parallel using multiple XMLHttpRequest objects.  The chunks may
 arrive out-of-order.
 A possible alternative API would be to support seeking beyond the end of the
 file or writing to a starting offset that is beyond the length fo the file.
  It may also be reasonable to support all of those in addition to truncating
 to an offset greater than the length of the file.

I think this may be a little messier, and there's no reason to make up
a new paradigm when the POSIX one is well-known and sufficient.

If nobody objects, I'll just add it.

Eric

[File API: FileSystem] Small change: createWriter

[Eric Uhrhane]

Darin and I just noticed that FileEntry.createWriter was specced as
synchronously returning a FileWriter.  That's a bug--given that a
FileWriter has a length member, it would imply a required synchronous
stat on the underlying file.  I've changed createWriter to be
asynchronous to fix this.

The method file() had the same issue, so it got the same fix.

 Eric

Re: Lifetime of Blob URL

[Eric Uhrhane]

I agree with Dmitry: window.createBlobUrl() makes it clearer.
Querying blob.url shouldn't have side effects.
As Jonas points out, we should keep the creation and destruction
methods near each other, so window.destroyBlobUrl() would be the
opposite function.

As for getBlobUrl vs. createBlobUrl: the latter sounds like it returns
a new URL each time.  The former is less explicit.  If we're returning
a unique URL per call, then create is clearly better.  Are we
requiring that each call to xxxBlobUrl have a matched destroyBlobUrl,
even if we're returning the same URL?

I think BlobError and BlobException make a bit more sense, but I'm not
too adamant about it.

On Sat, Aug 21, 2010 at 1:00 PM, Jian Li jia...@google.com wrote:
 I do not see any more discussions on blob URL API in recent days. Any more
 thoughts or conclusion?
 In addition, do we want to rename FileError and File Exception to BlobError
 and BlobException to match with BlobReader naming, or rather keep them
 intact?

 On Mon, Aug 2, 2010 at 3:22 PM, Dmitry Titov dim...@chromium.org wrote:

 It feels developers will make less errors with window.getBlobUrl(blob)
 kind of API, because, unlike blob.url, it doesn't violate pretty common
 programming assumptions (like querying a property of the same object should
 return the same value if nothing was done to the object, or that value of
 property should not depend on what is a global object in the  context of the
 query if the blob is the same). The spec language describing why and when
 blob.url returns different values with different lifetimes would be a bit
 hairy...
 Agree though that functionally they are the same.

 On Mon, Aug 2, 2010 at 3:05 PM, Jonas Sicking jo...@sicking.cc wrote:

 On Mon, Aug 2, 2010 at 2:19 PM, Michael Nordman micha...@google.com
 wrote:
  On Mon, Aug 2, 2010 at 1:39 PM, Jonas Sicking jo...@sicking.cc wrote:
 
  On Fri, Jul 30, 2010 at 12:01 PM, Michael Nordman
  micha...@google.com
  wrote:
  
  
   On Thu, Jul 29, 2010 at 4:33 PM, Jonas Sicking jo...@sicking.cc
   wrote:
  
   Sorry about the slow response. I'm currently at blackhat, so my
   internet connectivity is somewhat... unreliable, so generally
   having
   to try to stay off the webs :)
  
   On Tue, Jul 27, 2010 at 1:16 PM, Dmitry Titov dim...@chromium.org
   wrote:
Thanks Jonas,
Just to clarify some details we had while discussing this, could
you
confirm
if this matches with your thinking (or not):
1. If blob was created in window1, blob.url was queried, then
passed
(as
JS
object) to window2, and window1 was closed - then the url gets
invalidated
when window1 is closed, but immediately re-validated if window2
queries
blob.url. The url string is going to be the same, only there will
be
a
time
interval between closing window1 and querying blob.url in
window2,
during
which loading from the url returns 404.
  
   Actually, it might make sense to make blob.url, when queried by
   window2, return a different string. This makes things somewhat more
   consistent as to when a URL is working an when not.
  
   Now suppose window2 queries the .url attribute before window1 is
   closed?
   I
   think most people would expect the same value as returned in window1
   (yes?).
   Having the same or different value depending on whether the
   attribute
   was
   queried before or after another window was closed seems confusing. I
   think
   having the .url remain consistent from frame to frame/window to
   window
   could
   help with debugging.
 
  The idea would be that we *always* return different urls depending on
  which window queries a url. This gives the most consistent behavior in
  that every url given is always limited to the lifetime of the current
  window. No matter what windows around it does.
 
  If that's the idea, then I would vote for a non-instance method
  somewhere to
  provide the context specific URL. Having a simple attribute accessor
  return
  different values depending on which context its being accessed in is
  very
  unusual behavior.
  Can't say that its ideal, but window.getBlobUrl(blob) and
  window.revokeBlobUrl(...) would be an improvement.

 I can't say that I'm a big fan of this syntax given that I think the
 current syntax works fine in most cases. I'm definitely curious to
 hear what others think though.

 / Jonas

Re: Possible addition to file API

[Eric Uhrhane]

Take a look at this discussion of XHR.responseBlob:
http://lists.w3.org/Archives/Public/public-webapps/2010JulSep/0536.html

Eric

On Sat, Aug 21, 2010 at 12:39 PM, J David Eisenberg
jdavid.eisenb...@gmail.com wrote:
 At present, the only way a web page can read a file's binary data via
 the File API (http://dev.w3.org/2006/webapi/FileAPI) is if the person
 reading the page enters the file name in an input type=file/. The
 API applies to only local files, if I read it correctly. Is it
 possible to extend the spec to allow a script to read any file that
 can been loaded as part of the HTML page, such as an img/? This
 extension would let web pages read the EXIF data of an image that is
 part of an img element; a very useful capability for web apps.
 Similarly, you could access metadata from a file destined for an
 audio or video element. The spec seems to suggest that this is
 possible: http://dev.w3.org/2006/webapi/FileAPI/#url , but is it
 implemented in any browsers?

 I understand there are good security reasons for restricting file
 reads; you don't want someone else's web page to be able to go
 willy-nilly through your local hard disk and send all the information
 back to the Bad Guys. But this scenario is different; if you have
 loaded an img/, you must already have read permission for it, and
 its data is there in the browser. I'm just asking to be able to access
 its bits.

 Precedent: think of link rel=stylesheet href=blah.css/; you have
 access to all the elements in that file via the DOM, and there's no
 security issue involved, even if it's a cross-domain href. That's
 because it is *not* an arbitrary file on the user's local file system;
 it's one that the author has explicitly specified and is allowed to
 read.

 I think my suggestion here meets the same criteria. Am I missing
 something obvious on the security front?

 The back story for this idea (you may safely skip this portion if you
 are not interested): My brother and I have just spent the past few
 days looking over several large boxes of photos that my mom had stored
 away. Most of them had nothing written on the back, so we had no idea
 of where or when the pictures were taken, nor who the people in them
 were, nor what the picture meant to Mom. Obviously, metadata
 matters. This told me that I need to start using the comments
 feature in GIMP to add descriptions to my photos.

 A few days ago, some of our family got together and I told my cousin
 George that I'm going to start doing this. He said that was nice, but
 the only way to get the data out is to use a program similar to the
 one that you used when editing the image, and most people won't have
 that. So I thought, You know, it would be great if you could get
 access to an img/'s EXIF data from the browser.

FileWriter behavior between writes

[Eric Uhrhane]

This just came up during implementation discussions, so I wanted to
share it with the list.

The behavior of FileWriter in between writes isn't currently
specified.  We haven't said anything about when the underlying file is
opened or closed.  Does FileWriter.write() always do an
open-write-close, with the JS object effectively holding only a path
and offset, or could the implementation choose to keep the file open,
and have the effective offset come from the underlying file pointer?

My take: we should act as if we're always doing an open-write-close,
for maximum portability.  I believe that different operating systems
and file systems are likely to have varying behavior when there are
multiple simultaneous callers opening files at the same time.  For
example, what if script A has a FileWriter for /foo.txt and script B
[using the FileSystem api] moves it elsewhere?  If the file is closed,
the next write from A may act as if the file was never there.  If the
file stayed open, on some systems the write would succeed, but the
data would land at the file's new location.

Similar issues come up when files are opened for reading, then written
from another script, written from multiple scripts, etc.

Specifying that each FileWriter.write() should be indistinguishable
from an open-write-close wouldn't remove all platform-specific
behavior, but it would certainly reduce the number of problem cases.

Any thoughts?

Thanks,

  Eric Uhrhane
  er...@google.com

Re: New draft of FileSystem API posted

[Eric Uhrhane]

Fixed.

While I was in there, I deleted an unused class [FileSystemsCallback]
and changed the boolean persistent parameter [from requestFileSystem
and requestFileSystemSync] to be a more-general flag, to allow for
future expansion.

   Eric

On Fri, Aug 13, 2010 at 11:23 AM, Kinuko Yasuda kin...@chromium.org wrote:
 On Fri, Aug 13, 2010 at 10:58 AM, Eric Uhrhane er...@google.com wrote:

 On Thu, Aug 12, 2010 at 11:08 PM, Kinuko Yasuda kin...@chromium.org
 wrote:
  Hi Eric,
  Thanks for your reply.
  Actually after sending that email I had started to think that caching
  isFile
  / isDirectory information in memory would be ok if user could get an
  informative error code when an entry becomes stale -- and seems like
  that's
  the case.  So I'm almost convinced :)
  One thing I'm not fully convinced yet about FileEntry / DirectoryEntry
  is
  if we really want to make FileEntry a subclass of File.
  To me it looks like they have slightly different semantics - File is an
  immutable snapshot of file content, while Entry is more like a handle
  for
  meta-level operations that actually modify the underlying disk image.
  File's size attribute and slice() method assume its snapshot doesn't
  change
  after it was captured, but FileEntry has createWriter method and it will
  likely change the file content.  It brings several questions: if
  FileEntry
  == File in what timing are UAs supposed to capture a snapshot?  Do we
  want
  to have a synchronous size attribute on a mutable Entry?  If we move an
  Entry to another name, does its File.type need to change according to
  its
  new name?

 You're quite right.  A mutable FileEntry shouldn't have a synchronous
 size member, and slice() and type are likewise problematic.

 How about if we add a getFile() method that returns a File object,
 snapshotted at the time of the call?  Hmm...that name's going to be a
 bit confusing, though, since there's a getFile on DirectoryEntry that
 does something completely different.  Perhaps just file()?  It should
 definitely be a method, not a member, to make clear that it's making
 calls to the underlying filesystem.

 file() sounds good to me.
 (Oh yes we have getFile on DirectoryEntry...)


  On Wed, Aug 11, 2010 at 1:25 PM, Eric Uhrhane er...@google.com wrote:
 
  My apologies for the slow response; I'm now back from my vacation.
 
  On Tue, Jul 20, 2010 at 6:27 PM, Kinuko Yasuda kin...@google.com
  wrote:
   Hi Eric,
   Thanks for brushing up the draft.
   We had some internal discussion about the API details and came up
   with a
   question regarding is{File,Directory} attributes of Entry interface.
   It seems like user agent needs to be able to tell if a given entry is
   file
   or directory synchronously (or from its cache), but we wonder what
   should
   happen if the underlying file object is changed later.  For example,
   we
   may
   have a situation like this:
   1. a user code gets (or creates) a file entry.
   2. another flow of the code (or another code in the same
   origin) removes
   the
   same entry and creates a *directory* at the same path.
   3. the original code refers isFile attribute of the entry. -- should
   it
   be
   true or false?
 
  My intention was that the UA would look up that information when first
  creating the Entry and keep it cached.  It's not the kind of thing
  that changes very often, so I didn't think we needed it to be a live
  query every time.  If the user tries to write to a file that's become
  a directory, or vice-versa, that's what INVALID_STATE_ERROR is for.
 
  This is an unusual error, thus it's appropriate that it be handled via
  exception.
 
   If an Entry is just a reference (i.e. a pathname) of a system file
   entity,
   it would look natural that it is resolved at run-time thus returns
   false
   in
   the above case.   But if so we'll have two problems: 1) we'll need to
   make
   synchronous stat calls to get the attribute values, and 2) as we have
   different interfaces for file and directory, we may end up with
   having
   invalid FileEntry objects for directories - or vice versa.
   Would it be possible to have a single unified interface for file and
   directory and let scripts figure out the info at runtime (e.g. in
   each
   asynchronous filesystem operation)?
 
  Do you mean, allow all file operations and directory operation, and
  fail if you use the wrong one?  UAs already have to fail on operations
  that don't make sense [or fail in the underlying implementation],
  whether or not we keep a unified interface, so I think that would just
  clutter up each subtype with the others' methods.
 
 
 
  If I'm misunderstanding you, please give an example of your proposed
  interface and a situation which it would improve.
 
  You're not misunderstanding me.   I meant if UAs didn't cache the
  information that an entry is file or directory it would be natural to
  make
  wrong operations fail at run-time, but otherwise I fully agree that
  cluttering

Re: New draft of FileSystem API posted

[Eric Uhrhane]

My apologies for the slow response; I'm now back from my vacation.

On Tue, Jul 20, 2010 at 6:27 PM, Kinuko Yasuda kin...@google.com wrote:
 Hi Eric,
 Thanks for brushing up the draft.
 We had some internal discussion about the API details and came up with a
 question regarding is{File,Directory} attributes of Entry interface.
 It seems like user agent needs to be able to tell if a given entry is file
 or directory synchronously (or from its cache), but we wonder what should
 happen if the underlying file object is changed later.  For example, we may
 have a situation like this:
 1. a user code gets (or creates) a file entry.
 2. another flow of the code (or another code in the same origin) removes the
 same entry and creates a *directory* at the same path.
 3. the original code refers isFile attribute of the entry. -- should it be
 true or false?

My intention was that the UA would look up that information when first
creating the Entry and keep it cached.  It's not the kind of thing
that changes very often, so I didn't think we needed it to be a live
query every time.  If the user tries to write to a file that's become
a directory, or vice-versa, that's what INVALID_STATE_ERROR is for.
This is an unusual error, thus it's appropriate that it be handled via
exception.

 If an Entry is just a reference (i.e. a pathname) of a system file entity,
 it would look natural that it is resolved at run-time thus returns false in
 the above case.   But if so we'll have two problems: 1) we'll need to make
 synchronous stat calls to get the attribute values, and 2) as we have
 different interfaces for file and directory, we may end up with having
 invalid FileEntry objects for directories - or vice versa.
 Would it be possible to have a single unified interface for file and
 directory and let scripts figure out the info at runtime (e.g. in each
 asynchronous filesystem operation)?

Do you mean, allow all file operations and directory operation, and
fail if you use the wrong one?  UAs already have to fail on operations
that don't make sense [or fail in the underlying implementation],
whether or not we keep a unified interface, so I think that would just
clutter up each subtype with the others' methods.

If I'm misunderstanding you, please give an example of your proposed
interface and a situation which it would improve.

 On Wed, Jul 7, 2010 at 6:50 PM, Eric Uhrhane er...@google.com wrote:

 I've posted a new draft of File API: Directories and System [1].  In
 this draft I've rolled in quite a bit of feedback that I received
 since first posting it on DAP--many apologies for the delay.  This is
 the first draft produced since we agreed to move this spec from DAP to
 WebApps; I hope those of you who have time will give it a look and let
 me know what you think.

 In general I've tried to address any comment I was sent and had not
 already addressed via email.  The few that didn't make it in, I've
 responded to below.

 My thanks to Robin Berjon and Mike Clement for all their feedback.

 Robin:
  - data stored there by the application should not be deleted by the
 UA without user intervention, UA should require permission from the
 user, The application may of course delete it at will - these
 sound like real conformance statements, therefore SHOULD, SHOULD NOT,
 and MAY.

 Those are in a non-normative section; is that language still appropriate
 there?

 Robin:
 [discussion about speccing the URI format]

 Left as an open issue.

 Mike:
 [discussion about multiple sandboxes per origin]

 I think that would be very easy and clean to add later if desired, and
 in the mean time, one can use subdirectories.

 Mike:
 getFile/getDirectory are a bit overloaded.  How about including
 methods like exists(), createFile() and createDirectory()?  Though
 these methods are easily implemented in terms of getFile/getDirectory,
 I always prefer more direct API methods that help make the code easier
 to understand.  I expect, though, that you are attempting to be a low
 level as possible here.

 As Robin pointed out, adding extra round-trips will slow things down.
 Also, it can encourage race conditions.  These are easy for libraries
 to implement via wrappers.

 Mike:
 [request for creation time in getMetadata]

 It may be hard to support reliably cross-platform [2].

 Robin:
 [specifying a single locale everywhere]

 I don't think that'll make folks very happy if it's not their locale.
 If I e.g. try to force my locale on Turkish Windows users, they're
 going to see some interesting errors trying to share files with apps
 outside the browser, or for that matter even saving certain groups of
 files from inside the browser.

    Eric

 [1] http://dev.w3.org/2009/dap/file-system/file-dir-sys.html
 [2] http://en.wikipedia.org/wiki/MAC_times

[File Writer] new draft of FileWriter up

[Eric Uhrhane]

I've posted a new draft of the File Writer API [1].  I've made the
split between a save-one-blob interface and the richer interface
that'll be needed for the filesystem [and possibly other ways of
saving files] that we recently discussed [2].  I've switched to the
terminology FileSaver for the former and FileWriter for the
latter, and moved to a constructor for the former [3].  I've also done
quite a bit of miscellaneous cleanup.

I look forward to your feedback.  There will be a bit of a delay in my
responses, as I'm about to go on vacation for a few weeks, but I'll
respond to all comments when I come back.

 Eric

[1] http://dev.w3.org/2009/dap/file-system/file-writer.html
[2] http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/1223.html
[3] http://lists.w3.org/Archives/Public/public-webapps/2010JulSep/0156.html

Re: How to get a FileWriter/BlobWriter/BlobSaver

[Eric Uhrhane]

On Fri, Jul 9, 2010 at 4:42 PM, Maciej Stachowiak m...@apple.com wrote:

 On Jul 8, 2010, at 3:47 PM, Eric Uhrhane wrote:

 On Fri, Jul 2, 2010 at 8:19 PM, Jonas Sicking jo...@sicking.cc wrote:
 On Thu, Jul 1, 2010 at 3:31 PM, Eric Uhrhane er...@google.com wrote:
 The biggest unknown in the current BlobWriter spec [1] is how you
 obtain one in the first place.
 There are two current proposals, which I've summarized below.  I've
 heard only a few voices on this topic, and would love to get more
 opinions.  New proposals would be welcome as well.

 If you favor one of the below, and don't think I've done it justice,
 please do weigh in with further details.


 Proposal A: a new input control.

 You'd add markup such as input type=saveas or button type=saveas.
 It could be styled however you please; it would pop up a standard
 File-SaveAs dialog, so there's no security problem there.  You'd
 handle the event when the user selects a save location in order to
 grab the BlobWriter.

 Pluses:
  It parallels input type=file.
  It's user-initiated, so the default flow doesn't ever surprise the
 user with a popup.


 Proposal B: a new method on Window.

 window.saveAs(function(blobWriter) { /* grab the writer and use it
 */}, function(error) {...});

 Pluses:
  It's a simple async API like many others we provide.
  App developers are free to provide any UI or control flow they like.


 In either case, we could add a parameter for the Blob, so that the
 system knows what it's saving, and/or a suggested mime-type and base
 name.  Adding the Blob itself would let us eliminate the
 writeFile()/save() call mentioned in [2], so that you'd only grab the
 returned BlobWriter if you wanted to handle the progress events.

 I'm not a fan of the input type=saveas solution. In part because no
 actual input is occurring. If we really want something like that then
 I think reviving the old bb element is the way to go.

 The user is inputting a location at which to save the file.

 However what I really think we should do is to provide some API which
 given a blob and an optional filename, asynchronously returns an
 object at which progress events are fired and which has an abort()
 function. I don't care much if this function lives on a new object,
 a'la SimpleFileWriter, or if it's added to something like the
 Navigator object.

 OK, I'm not hearing any support whatsoever for option A, so let's go
 with something like SimpleFileWriter/BlobSaver.

 How about this?

    var blobSaver = window.saveAs(blob);
    blobSaver.onerror = onError;
    blobSaver.onwriteend = onWriteEnd;

 It would be great if we could avoid adding methods to Window with simple and 
 common names like saveAs(), since they pollute the global namespace. saveAs 
 sounds like a name that may well conflict with existing code. FileSaver or 
 BlobSaver would be less likely to conflict, so perhaps the entry point can be 
 a constructor, rather than a function.

More like this?

   var writer = new SimpleFileWriter(blob, optionalName);
   writer.onerror = onError;
   writer.onwriteend = onWriteEnd;

 Other thoughts:
 - I still think the names should be prefixed with File, not Blob.

That's fine for the writers; I like your logic that a FooWriter writes
to a Foo, not from it.
I think the converse is true for a reader, so BlobReader makes the most sense.

 - What would the behavior be? Would this pop up a dialog where the user picks 
 a file? If so, should it be subject to a user gesture limitation?

Should we spec that, or leave it loose?
One option I was just discussing over here was that if it's in
response to a user gesture, it pops up a SaveAs dialog; if it's not,
it goes to an infobar.  I don't want to spec this so closely that UAs
can't experiment a bit.

 - Part of the objection to input type=saveas is because there is no input 
 occuring. But HTML5 also has an output element.  Perhaps output type=file 
 is an option worth considering?


 Regards,
 Maciej

Re: How to get a FileWriter/BlobWriter/BlobSaver

[Eric Uhrhane]

On Fri, Jul 2, 2010 at 8:19 PM, Jonas Sicking jo...@sicking.cc wrote:
 On Thu, Jul 1, 2010 at 3:31 PM, Eric Uhrhane er...@google.com wrote:
 The biggest unknown in the current BlobWriter spec [1] is how you
 obtain one in the first place.
 There are two current proposals, which I've summarized below.  I've
 heard only a few voices on this topic, and would love to get more
 opinions.  New proposals would be welcome as well.

 If you favor one of the below, and don't think I've done it justice,
 please do weigh in with further details.


 Proposal A: a new input control.

 You'd add markup such as input type=saveas or button type=saveas.
 It could be styled however you please; it would pop up a standard
 File-SaveAs dialog, so there's no security problem there.  You'd
 handle the event when the user selects a save location in order to
 grab the BlobWriter.

 Pluses:
  It parallels input type=file.
  It's user-initiated, so the default flow doesn't ever surprise the
 user with a popup.


 Proposal B: a new method on Window.

 window.saveAs(function(blobWriter) { /* grab the writer and use it
 */}, function(error) {...});

 Pluses:
  It's a simple async API like many others we provide.
  App developers are free to provide any UI or control flow they like.


 In either case, we could add a parameter for the Blob, so that the
 system knows what it's saving, and/or a suggested mime-type and base
 name.  Adding the Blob itself would let us eliminate the
 writeFile()/save() call mentioned in [2], so that you'd only grab the
 returned BlobWriter if you wanted to handle the progress events.

 I'm not a fan of the input type=saveas solution. In part because no
 actual input is occurring. If we really want something like that then
 I think reviving the old bb element is the way to go.

The user is inputting a location at which to save the file.

 However what I really think we should do is to provide some API which
 given a blob and an optional filename, asynchronously returns an
 object at which progress events are fired and which has an abort()
 function. I don't care much if this function lives on a new object,
 a'la SimpleFileWriter, or if it's added to something like the
 Navigator object.

OK, I'm not hearing any support whatsoever for option A, so let's go
with something like SimpleFileWriter/BlobSaver.

How about this?

var blobSaver = window.saveAs(blob);
blobSaver.onerror = onError;
blobSaver.onwriteend = onWriteEnd;
...

[where nothing actually happens until execution exits]

Re: BlobWriter simplification/split

[Eric Uhrhane]

On Fri, Jul 2, 2010 at 6:22 PM, Jonas Sicking jo...@sicking.cc wrote:
 On Fri, Jul 2, 2010 at 2:52 PM, Maciej Stachowiak m...@apple.com wrote:

 On Jun 30, 2010, at 10:29 AM, Eric Uhrhane wrote:

 On Wed, Jun 30, 2010 at 10:18 AM, Jonas Sicking jo...@sicking.cc wrote:

 May I propose FileWriter in place of BlobWriter? ;-)
 You are actually always writing to files, so it would make a lot of sense 
 IMO.

 We renamed BlobReader based on the perspective that it took data from
 a Blob and read it into memory.  Likewise BlobWriter takes data from a
 Blob and writes it to a file.  I think the symmetry makes sense.
 Calling it FileWriter also works, but then you're naming by
 destination instead of source, so I don't think it complements
 BlobReader as well.

 I think it makes sense to name writer objects by the destination rather than 
 the source. It's normal to speak of reading from X, and writing to Y. You 
 rarely say you are writing from Y. If you saw a random class called 
 StreamWriter, would you expect it writes *to* a Stream, or *from* a Stream?

 Put another way, the essence of this class is that it writes something to a 
 file, not that it writes a Blob to some unspecified location. If we wanted 
 to add functionality to write something other than a Blob to a File (such 
 as, say, writing a typed array, or writing a string), then it would 
 plausibly make sense as part of the same interface used for writing a Blob
 to a file. If we made an API to write a Blob somewhere else (such as to a 
 memory location or somewhere on the network), it really would not make sense 
 as part of the same interface.

 Thus FileWriter.

 I also think that the name Blob is pretty unpleasant and I would rather not 
 increase its prominence in the API.

 Agreed. On both accounts. I have yet to hear anyone like the name
 'Blob' for anything.

I don't see what's wrong with the name Blob; I like it just fine.  And
I think it's a bit odd to have a BlobReader but a FileWriter, but if
that's what everybody's happy with, I'm OK with it.

Eric

New draft of FileSystem API posted

[Eric Uhrhane]

I've posted a new draft of File API: Directories and System [1].  In
this draft I've rolled in quite a bit of feedback that I received
since first posting it on DAP--many apologies for the delay.  This is
the first draft produced since we agreed to move this spec from DAP to
WebApps; I hope those of you who have time will give it a look and let
me know what you think.

In general I've tried to address any comment I was sent and had not
already addressed via email.  The few that didn't make it in, I've
responded to below.

My thanks to Robin Berjon and Mike Clement for all their feedback.

Robin:
 - data stored there by the application should not be deleted by the
UA without user intervention, UA should require permission from the
user, The application may of course delete it at will - these
sound like real conformance statements, therefore SHOULD, SHOULD NOT,
and MAY.

Those are in a non-normative section; is that language still appropriate there?

Robin:
[discussion about speccing the URI format]

Left as an open issue.

Mike:
[discussion about multiple sandboxes per origin]

I think that would be very easy and clean to add later if desired, and
in the mean time, one can use subdirectories.

Mike:
getFile/getDirectory are a bit overloaded.  How about including
methods like exists(), createFile() and createDirectory()?  Though
these methods are easily implemented in terms of getFile/getDirectory,
I always prefer more direct API methods that help make the code easier
to understand.  I expect, though, that you are attempting to be a low
level as possible here.

As Robin pointed out, adding extra round-trips will slow things down.
Also, it can encourage race conditions.  These are easy for libraries
to implement via wrappers.

Mike:
[request for creation time in getMetadata]

It may be hard to support reliably cross-platform [2].

Robin:
[specifying a single locale everywhere]

I don't think that'll make folks very happy if it's not their locale.
If I e.g. try to force my locale on Turkish Windows users, they're
going to see some interesting errors trying to share files with apps
outside the browser, or for that matter even saving certain groups of
files from inside the browser.

Eric

[1] http://dev.w3.org/2009/dap/file-system/file-dir-sys.html
[2] http://en.wikipedia.org/wiki/MAC_times

How to get a FileWriter/BlobWriter/BlobSaver

[Eric Uhrhane]

The biggest unknown in the current BlobWriter spec [1] is how you
obtain one in the first place.
There are two current proposals, which I've summarized below.  I've
heard only a few voices on this topic, and would love to get more
opinions.  New proposals would be welcome as well.

If you favor one of the below, and don't think I've done it justice,
please do weigh in with further details.


Proposal A: a new input control.

You'd add markup such as input type=saveas or button type=saveas.
It could be styled however you please; it would pop up a standard
File-SaveAs dialog, so there's no security problem there.  You'd
handle the event when the user selects a save location in order to
grab the BlobWriter.

Pluses:
  It parallels input type=file.
  It's user-initiated, so the default flow doesn't ever surprise the
user with a popup.


Proposal B: a new method on Window.

window.saveAs(function(blobWriter) { /* grab the writer and use it
*/}, function(error) {...});

Pluses:
  It's a simple async API like many others we provide.
  App developers are free to provide any UI or control flow they like.


In either case, we could add a parameter for the Blob, so that the
system knows what it's saving, and/or a suggested mime-type and base
name.  Adding the Blob itself would let us eliminate the
writeFile()/save() call mentioned in [2], so that you'd only grab the
returned BlobWriter if you wanted to handle the progress events.

  Eric

[1] http://dev.w3.org/2009/dap/file-system/file-writer.html
[2] http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/1223.html

Re: BlobWriter simplification/split

[Eric Uhrhane]

On Thu, Jul 1, 2010 at 4:34 PM, Jonas Sicking jo...@sicking.cc wrote:
 On Wed, Jun 30, 2010 at 10:18 AM, Jonas Sicking jo...@sicking.cc wrote:
 On Tue, Jun 29, 2010 at 5:17 PM, Eric Uhrhane er...@google.com wrote:
 Following up on discussions mainly at [1] and use cases at [2], I'd
 like to propose splitting the BlobWriter [née FileWriter] class, with
 an eye to solving some UI problems and simplifying implementation.

 When saving a Blob to a location outside the FileSystem API sandbox,
 we want to prompt the user exactly once, and want to be able to
 indicate that a write is taking place, e.g. by throbbing the download
 indicator.  We want the throbbing to go away as soon as the write is
 done, and we don't want the app to have continued privileges to write
 outside the sandbox.  [That's been debated, but I think it's beyond
 the scope of what we're working on so far, so let's leave that case
 for later expansion.]

 When writing inside the sandbox, we probably don't need the throbber
 at all, and we definitely don't want to prompt the user on each write.
  Leaving aside the question of how one obtains a BlobWriter without
 using the sandbox and when exactly prompts happen [I'll open another
 thread for that], I think that:

 *  We don't want to have the same API call cause prompts to pop up on
 some instances of BlobWriter and not others.
 *  We don't want to have the same API call be reusable on some
 instances of BlobWriter and not others.

 I propose the following split:

 Define a parent class, SimpleBlobWriter, that's used for one-shot Blob
 export.  This is what you use when you don't have the FileSystem API,
 when the user selects the save location.  It contains a writeFile()
 method, which writes a Blob to a file in a single operation.  It is an
 error to call writeFile if readyState is not INIT, so SimpleBlobWriter
 is single-use.  Its other members are abort(), readyState, the ready
 state constants, error, length, and the progress event handlers, as
 defined in the current spec.

 Derive from SimpleBlobWriter the class BlobWriter, which adds
 position, truncate(), seek(), and write(), as defined in the current
 spec.  This is what the FileSystem API's createWriter() call would
 return.

 This lets you have different APIs for different behaviors, and removes
 fields from SimpleBlobWriter that aren't actually useful without the
 FileSystem API.

 How does that sound?

 Sounds great!

 Actually, i realized that I have second thoughts on some details here.

 It would be great if the save as functionality provided by the
 SimpleBlobWriter worked such that at the time the where do you want
 to save a file dialog pops up, the UA already knows what Blob is
 going to be saved. This has several advantages over the UA first
 receiving a request to save something, and then receiving the data
 that is to be saved. For example:

 * If the Blob is a File, default the filename to File.name
 * Allows the UA to display the size of the file, thus allowing the
 user to save to different locations depending on the expected size of
 the file
 * Allows the UA to determine the type of the Blob and adjust the
 filename based on that. Firefox does this for content-disposition
 save-as dialogs.

Yeah,take a look at [1] as well; this could eliminate the need for the
save/writeFile call on the simpler interface.

 As a separate question, at what URL are the latest drafts available?
 Is http://dev.w3.org/2009/dap/file-system/file-writer.html still
 authoritative?

Yes, I haven't moved or renamed it, though the path and name are now
both obsolete.
That's probably worth doing as of WebApps publishing an official draft of it.

 Eric

[1] http://lists.w3.org/Archives/Public/public-webapps/2010JulSep/0023.html

Re: BlobWriter simplification/split

[Eric Uhrhane]

On Wed, Jun 30, 2010 at 10:18 AM, Jonas Sicking jo...@sicking.cc wrote:
 On Tue, Jun 29, 2010 at 5:17 PM, Eric Uhrhane er...@google.com wrote:
 Following up on discussions mainly at [1] and use cases at [2], I'd
 like to propose splitting the BlobWriter [née FileWriter] class, with
 an eye to solving some UI problems and simplifying implementation.

 When saving a Blob to a location outside the FileSystem API sandbox,
 we want to prompt the user exactly once, and want to be able to
 indicate that a write is taking place, e.g. by throbbing the download
 indicator.  We want the throbbing to go away as soon as the write is
 done, and we don't want the app to have continued privileges to write
 outside the sandbox.  [That's been debated, but I think it's beyond
 the scope of what we're working on so far, so let's leave that case
 for later expansion.]

 When writing inside the sandbox, we probably don't need the throbber
 at all, and we definitely don't want to prompt the user on each write.
  Leaving aside the question of how one obtains a BlobWriter without
 using the sandbox and when exactly prompts happen [I'll open another
 thread for that], I think that:

 *  We don't want to have the same API call cause prompts to pop up on
 some instances of BlobWriter and not others.
 *  We don't want to have the same API call be reusable on some
 instances of BlobWriter and not others.

 I propose the following split:

 Define a parent class, SimpleBlobWriter, that's used for one-shot Blob
 export.  This is what you use when you don't have the FileSystem API,
 when the user selects the save location.  It contains a writeFile()
 method, which writes a Blob to a file in a single operation.  It is an
 error to call writeFile if readyState is not INIT, so SimpleBlobWriter
 is single-use.  Its other members are abort(), readyState, the ready
 state constants, error, length, and the progress event handlers, as
 defined in the current spec.

 Derive from SimpleBlobWriter the class BlobWriter, which adds
 position, truncate(), seek(), and write(), as defined in the current
 spec.  This is what the FileSystem API's createWriter() call would
 return.

 This lets you have different APIs for different behaviors, and removes
 fields from SimpleBlobWriter that aren't actually useful without the
 FileSystem API.

 How does that sound?

 Sounds great!

 [Better names for SimpleBlobWriter and
 BlobWriter would be quite welcome, BTW.]

 May I propose FileWriter in place of BlobWriter? ;-)
 You are actually always writing to files, so it would make a lot of sense IMO.

We renamed BlobReader based on the perspective that it took data from
a Blob and read it into memory.  Likewise BlobWriter takes data from a
Blob and writes it to a file.  I think the symmetry makes sense.
Calling it FileWriter also works, but then you're naming by
destination instead of source, so I don't think it complements
BlobReader as well.

 An alternative would be to rename SimpleBlobWriter to FileSaver and
 have the two be completely separate interfaces. It doesn't seem like
 the inheritance is adding much anyway?

I do like *Saver, though, since it clearly just saves an entire Blob.
How about BlobSaver [with method save() instead of writeFile] and
BlobWriter?

The inheritance gets you length, abort(), error, the state constants,
and all the progress events, which may have identical handlers in many
cases.  I think that's probably enough to be worthwhile.  Anyone else
want to chime in?

 Eric

Re: XHR.responseBlob and BlobWriter [nee FileWriter]

[Eric Uhrhane]

On Tue, Jun 29, 2010 at 11:28 AM, Jonas Sicking jo...@sicking.cc wrote:
 On Mon, Jun 28, 2010 at 6:58 PM, Eric Uhrhane er...@google.com wrote:
 The discussion at [1] got tangled up with the debate of .URL vs. .url,
 so I'm starting a new thread to pick back up the original topic: how
 do we save binary data from XMLHttpRequest?  Here's my proposal [built
 mainly from the good ideas other folks posted in the original thread].

 Use case 1: the developer wants to download a Blob of data, use it for
 a while [e.g. slicing out sub-Blobs and displaying them as images],
 then have it go away automatically.
 Use case 2: the developer wants to download a Blob of data, saving it
 in a location of the user's choice outside the sandbox.
 Use case 3: the developer wants to download a Blob of data, save it in
 the sandboxed FileSystem, and access it again later.

 XHR will have a responseBlob property.
 In order to signal the XHR that it should spool to disk and supply
 responseBlob, a flag must be set before send() is called.  Call this
 wantBlob for now, although a better name would be appreciated.
 If wantBlob is set, responseBlob will be valid when the XHR completes,
 and responseText and responseXML will be null.
 If wantBlob is not set, responseBlob will be null, and the XHR will
 function as it does now.

 When wantBlob is set, on all progress events before the XHR is
 complete, responseBlob will be null.  As of completion, it will return
 a Blob containing the full contents of the download.  [We could later
 add incremental Blobs in progress events, but let's keep this simple
 to start with.]

 This satisfies use case 1 as-is.
 With the BlobWriter spec [2], as currently written [assuming we agree
 on how to get our hands on a BlobWriter], it takes care of use case 2.
 With the FileSystem spec [3], as currently written, it takes care of use 
 case 3.

 I think this takes care of the major use cases, doesn't force anyone
 to implement FileSystem to solve the cases that don't really require
 it, removes any need for synchronous IO, and is pretty simple for
 developers to understand and use.  What do you think?

 Sounds great to me! Also note that this will allow

 Use case 3': the developer wants to download a Blob of data, save it in
 IndexedDB, and access it again later.

I don't see Blob mentioned in the structured clone algorithm, although
File is there.  I doubt it would be much additional work to support all Blobs.

 Eric

BlobWriter simplification/split

[Eric Uhrhane]

Following up on discussions mainly at [1] and use cases at [2], I'd
like to propose splitting the BlobWriter [née FileWriter] class, with
an eye to solving some UI problems and simplifying implementation.

When saving a Blob to a location outside the FileSystem API sandbox,
we want to prompt the user exactly once, and want to be able to
indicate that a write is taking place, e.g. by throbbing the download
indicator.  We want the throbbing to go away as soon as the write is
done, and we don't want the app to have continued privileges to write
outside the sandbox.  [That's been debated, but I think it's beyond
the scope of what we're working on so far, so let's leave that case
for later expansion.]

When writing inside the sandbox, we probably don't need the throbber
at all, and we definitely don't want to prompt the user on each write.
 Leaving aside the question of how one obtains a BlobWriter without
using the sandbox and when exactly prompts happen [I'll open another
thread for that], I think that:

*  We don't want to have the same API call cause prompts to pop up on
some instances of BlobWriter and not others.
*  We don't want to have the same API call be reusable on some
instances of BlobWriter and not others.

I propose the following split:

Define a parent class, SimpleBlobWriter, that's used for one-shot Blob
export.  This is what you use when you don't have the FileSystem API,
when the user selects the save location.  It contains a writeFile()
method, which writes a Blob to a file in a single operation.  It is an
error to call writeFile if readyState is not INIT, so SimpleBlobWriter
is single-use.  Its other members are abort(), readyState, the ready
state constants, error, length, and the progress event handlers, as
defined in the current spec.

Derive from SimpleBlobWriter the class BlobWriter, which adds
position, truncate(), seek(), and write(), as defined in the current
spec.  This is what the FileSystem API's createWriter() call would
return.

This lets you have different APIs for different behaviors, and removes
fields from SimpleBlobWriter that aren't actually useful without the
FileSystem API.

How does that sound?  [Better names for SimpleBlobWriter and
BlobWriter would be quite welcome, BTW.]

 Eric

[1] http://lists.w3.org/Archives/Public/public-webapps/2010JanMar/0910.html
[buried rather deep in the thread]
[2] http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/1206.html

Re: [File API] Recent Updates To Specification + Co-Editor

[Eric Uhrhane]

On Mon, Jun 28, 2010 at 2:20 PM, Arun Ranganathan a...@mozilla.com wrote:
 Greetings WebApps WG,

 I have made edits to the File API specification [1].  There are a few things
 of note that I'd like to call the WG's attention to.

 1. There is a name change in effect.  FileReader has been re-named
 BlobReader, upon request from Chrome team folks[2][3].  The name
 BlobReader won't win awards in a beauty pageant, but it tersely describes
 an object to read Blobs (which could originate from the underlying file
 system *or* be generated *within* a Web App).  My present understanding is
 that FileWriter will also undergo a name change.  Naming is really hard.
  Firefox already ships with FileReader, but I see the point of having an
 object named for what it does, which in this case is certainly more than
 file reading from the underlying file system.  I also abhor bike shedding,
 especially over naming, but this is something that's exposed to the authors.
  I have not renamed FileError or FileException.  In the case of errors and
 exceptions, I think *most* scenarios will occur as a result of issues with
 the underlying file system.  These names should remain.

I've just made the corresponding changes to FileWriter [-BlobWriter]
in both the FileWriter and FileSystem specs.  I've not changed the URL
of the FileWriter spec, though, for simplicity.

 2. I've updated the URL scheme for Blobs using an ABNF that calls for an
 opaque string which is a term I define in the specification.  There was
 much discussion about this aspect of the File API specification, and I think
 the existing scheme does allow for user agents to tack on origin information
 in the URL (this is not something the spec. says you should do).  The actual
 choice of opaque string is left to implementations, though the specification
 suggests UUID in its canonical form (and provides an ABNF for this).  I
 think this is the most any specification has said on the subject of URLs.

 3. There is an additional asynchronous read method on BlobReader, and an
 additional synchronous read method on BlobReaderSync, namely
 readAsArrayBuffer.  These use the TypedArrays definition initially defined
 by the WebGL WG [4].

 4. I am moving on from my full-time role at Mozilla to a part-time
 consulting role.  I'll continue to be an editor of the File API, but I am
 stepping down as Chair of the WebGL WG.  I'll continue to be active in
 standards communities, though :-)

 5. I spoke to Jonas Sicking, who expressed willingness to be a co-editor of
 the File API specification.  Most people who work on HTML5 and WebApps know
 Jonas' contributions to both WGs; with everyone's consent, I'd like to
 nominate him as co-editor.  His model for an asynchronous event-driven API
 is what prompted the initial rewrite, and he also works on both File API and
 IndexedDB implementation (amongst other things).

Sounds good.  Thanks Jonas!

 -- A*

 [1] http://dev.w3.org/2006/webapi/FileAPI/
 [2] http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/0755.html
 [3] http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/0716.html
 [4]
 https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/doc/spec/TypedArray-spec.html

XHR.responseBlob and BlobWriter [nee FileWriter]

[Eric Uhrhane]

The discussion at [1] got tangled up with the debate of .URL vs. .url,
so I'm starting a new thread to pick back up the original topic: how
do we save binary data from XMLHttpRequest?  Here's my proposal [built
mainly from the good ideas other folks posted in the original thread].

Use case 1: the developer wants to download a Blob of data, use it for
a while [e.g. slicing out sub-Blobs and displaying them as images],
then have it go away automatically.
Use case 2: the developer wants to download a Blob of data, saving it
in a location of the user's choice outside the sandbox.
Use case 3: the developer wants to download a Blob of data, save it in
the sandboxed FileSystem, and access it again later.

XHR will have a responseBlob property.
In order to signal the XHR that it should spool to disk and supply
responseBlob, a flag must be set before send() is called.  Call this
wantBlob for now, although a better name would be appreciated.
If wantBlob is set, responseBlob will be valid when the XHR completes,
and responseText and responseXML will be null.
If wantBlob is not set, responseBlob will be null, and the XHR will
function as it does now.

When wantBlob is set, on all progress events before the XHR is
complete, responseBlob will be null.  As of completion, it will return
a Blob containing the full contents of the download.  [We could later
add incremental Blobs in progress events, but let's keep this simple
to start with.]

This satisfies use case 1 as-is.
With the BlobWriter spec [2], as currently written [assuming we agree
on how to get our hands on a BlobWriter], it takes care of use case 2.
With the FileSystem spec [3], as currently written, it takes care of use case 3.

I think this takes care of the major use cases, doesn't force anyone
to implement FileSystem to solve the cases that don't really require
it, removes any need for synchronous IO, and is pretty simple for
developers to understand and use.  What do you think?

   Eric

[1] http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/0313.html
[2] http://dev.w3.org/2009/dap/file-system/file-writer.html
[3] http://dev.w3.org/2009/dap/file-system/file-dir-sys.html

Re: Transferring File* to WebApps - redux

[Eric Uhrhane]

On Fri, Jun 18, 2010 at 10:53 AM, SULLIVAN, BRYAN L (ATTCINW)
bs3...@att.com wrote:
 Eric,
 Based upon your responses there are a couple of additional things we
 need to check on, but I'll look deeper into the current specs for the
 things that should already be supported.

 The additional things are:

 - how to safely grant access outside the sandbox, which depends upon
 ability to discover available file systems in persistent storage and
 select arbitrary locations within them. We would prefer that this
 functional capability is addressed within the File* APIs directly rather
 than through an extension API in DAP, thus we may propose two additional
 methods for the LocalFilesystem interface:
   getDefaultLocation: as from
 http://bondi.omtp.org/1.1/apis/filesystem.html#getDefaultLocationN101E4
   getRootLocations: as from
 http://bondi.omtp.org/1.1/apis/filesystem.html#getRootLocationsN10253
 These methods can be covered under the general one-time prompt approach
 or under the policy approach (once we have worked out how the File* APIs
 are covered under that, e.g. by association of widget feature URIs).

Given that these methods would have security implications well beyond
the current spec, I'd prefer to have them added in a later pass,
either in an extension spec or in a later version of the filesystem
spec.  We've got enough to argue about in there already, and if we try
to do too much in one pass, we'll bog down.  These certainly look like
logical extensions to what we've been discussing, though.

 - clarify the difference between memory and persistent storage: In
 memory usually means the content is in the working RAM of a device. The
 File* specs talk about working with files in memory which I take to
 mean that the file has been read from persistent storage into the
 working memory of the browser (upon selection by the user), and is being
 read or updated (with presumed writing back to persistent storage
 later). The different may be subtle but is important. If what's meant
 instead of a file copy in memory is that the webapp can *directly*
 access/update a file in persistent storage, then I suggest the spec
 terminology be updated to make that clear.

Arun: in 5.4 and 5.5.1, I think you want to change in memory to
into memory to match the other 15-or-so instances of that phrase in
the spec.  Bryan, is that sufficient?

 Thanks,
 Bryan Sullivan | ATT


 -Original Message-
 From: Eric Uhrhane [mailto:er...@google.com]
 Sent: Wednesday, June 16, 2010 4:42 PM
 To: SULLIVAN, BRYAN L (ATTCINW)
 Cc: a...@mozilla.com; Robin Berjon; public-device-a...@w3.org; Ian
 Fette; Web Applications Working Group WG
 Subject: Re: Transferring File* to WebApps - redux

 Sorry about the delay in response; I've been out of the office for the
 past 10 days.  [Also, sorry Bryan--I forgot to reply-all.]

 On Tue, Jun 15, 2010 at 3:24 PM, SULLIVAN, BRYAN L (ATTCINW)
 bs3...@att.com wrote:
 I am not meaning to be unfair, perhaps the message is not coming
 through
 clearly enough.

 There are specific technical requirements that we need these APIs to
 fulfill, that I indicated to Thomas in another email:
 1) access filesystems on the host device

 FileSystem/FileWriter/FileReader do this.

 2) traverse directories

 FileSystem does this.  Currently it's only specced to do so within a
 per-origin sandbox, but the API could be used outside the sandbox if
 another spec defined a mechanism to grant such access safely.

 3) read files as they exist on the filesystem (not just a local
 representation in memory, as currently defined in the File API to my
 understanding), in bytes and lines

 FileReader does this [not sure what you mean about a local
 representation--if you can read an on-disk file, you're doing so via
 memory].

 4) write files (similar requirement to write directly to the
 filesystem), in bytes and lines, with overwrite and append options

 FileSystem/FileWriter do this [details of appending still being hammered
 out].

 5) do the above programmatically in Javascript (not dependent just
 upon
 user selection of an input element and interaction with a file
 selector)

 FileSystem does this.  And no, there's no need for the UA to prompt
 the user on each access; permissions should be more on the order of
 can access temporary filesystem storage and can access persistent
 filesystem storage, and need only be granted once.

 6) provide security for this using the policy-framework approach as
 being defined for DAP APIs

 This remains for DAP to work out.  It should be fairly straightforward
 to add a policy-based mechanism to grant access to FileSystem APIs
 [e.g. your example documents folder, via resolveLocalFilesystemURI,
 mentioned elsewhere in this thread].

 Thanks,

    Eric

Re: Transferring File* to WebApps - redux

[Eric Uhrhane]

Sorry about the delay in response; I've been out of the office for the
past 10 days.  [Also, sorry Bryan--I forgot to reply-all.]

On Tue, Jun 15, 2010 at 3:24 PM, SULLIVAN, BRYAN L (ATTCINW)
bs3...@att.com wrote:
 I am not meaning to be unfair, perhaps the message is not coming through
 clearly enough.

 There are specific technical requirements that we need these APIs to
 fulfill, that I indicated to Thomas in another email:
 1) access filesystems on the host device

FileSystem/FileWriter/FileReader do this.

 2) traverse directories

FileSystem does this.  Currently it's only specced to do so within a
per-origin sandbox, but the API could be used outside the sandbox if
another spec defined a mechanism to grant such access safely.

 3) read files as they exist on the filesystem (not just a local
 representation in memory, as currently defined in the File API to my
 understanding), in bytes and lines

FileReader does this [not sure what you mean about a local
representation--if you can read an on-disk file, you're doing so via
memory].

 4) write files (similar requirement to write directly to the
 filesystem), in bytes and lines, with overwrite and append options

FileSystem/FileWriter do this [details of appending still being hammered out].

 5) do the above programmatically in Javascript (not dependent just upon
 user selection of an input element and interaction with a file selector)

FileSystem does this.  And no, there's no need for the UA to prompt
the user on each access; permissions should be more on the order of
can access temporary filesystem storage and can access persistent
filesystem storage, and need only be granted once.

 6) provide security for this using the policy-framework approach as
 being defined for DAP APIs

This remains for DAP to work out.  It should be fairly straightforward
to add a policy-based mechanism to grant access to FileSystem APIs
[e.g. your example documents folder, via resolveLocalFilesystemURI,
mentioned elsewhere in this thread].

Thanks,

Eric

Re: HTML5 File

[Eric Uhrhane]

Hey all--I'm sorry it's taken me so long to respond to this thread.
I'm a little short on bandwidth right now, and that's likely going to
get worse for at least a couple of weeks.

First of all, I think this discussion should include DAP [+CC].  DAP
folks, this discussion started at
http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/0886.html.

My take is that I've gotten lots of useful feedback on FileWriter on
both webapps and DAP.  However, recently most of the FileWriter
discussion has been on webapps, mainly as asides on other topics, and
the last time I sent a request for comments to DAP it received no
responses.

I would like to see both specs be officially discussed on webapps.  I
think we really need input from all the browser companies in order to
make a solid API, and in order to produce something that everyone's
eager to implement.  If they won't come to DAP, then this part should
come to them.  I'd also appreciate it if DAP folks who've contributed
to the specs so far continued to be involved.  I don't know what the
logistics of that would be.  The specs are clearly within the charters
of both groups.  And of course several folks have been popping back
and forth between lists anyway.

I'm not really too bothered about the exact form of the discussion and
publication, as long as we get everyone involved.

Thanks,

 Eric

On Fri, Jun 4, 2010 at 1:36 PM, Arun Ranganathan a...@mozilla.com wrote:
 On 6/3/10 4:13 AM, Robin Berjon wrote:

 On Jun 2, 2010, at 23:02 , Jonas Sicking wrote:


 I don't know who makes these decisions, but I'd imagine the editor
 holds a certain amount of sway.


 Decisions of what is in scope for a WG are made by the members (i.e. you)
 when a WG is created. When DAP was created, people felt rather strongly
 (personally, I disagreed, I know that Arun had similar concerns) that adding
 deliverables to WebApps would be a bad idea as it already had many, and
 because there was already a lot of traffic on its list.

 To be clear, I was *very much in favor* of FileAPI-related items being added
 to WebApps, but was less enthusiastic about Widget-related items or the Web
 SQL Database item.  David Baron, Mozilla's Advisory Committee
 Representative, made this stance public in a blog post:

 http://blog.mozilla.com/standards/2010/04/30/mozilla-at-w3c-review-of-web-applications-wg-charter/

 I'll note that the current charter --
 http://www.w3.org/2010/webapps/charter/ --  uses the following language when
 discussing File API:

  An API for representing file objects in web applications, as well as
 programmatically selecting them and accessing their data. This may include
 file writing and file system APIs. This replaces the File Upload
 specification.

 This language in my opinion certainly includes FileWriter and anything
 FileSystem related, and moving from DAP -- WebApps should NOT warrant a
 charter review.  This language was approved by all members.  Moving to
 file-related APIs to WebApps (from DAP) has the following advantages:

 1. Wide implementor review.  My concern is that not ALL browser vendors are
 members of DAP; ALL browser vendors are members of WebApps.  Moreover, since
 a charter review/amendment doesn't seem necessary, given the inclusive
 language around file APIs, I think there is a strong case to be made for
 this work to proceed in WebApps.

 2. Family of specifications living together.  Changes to FileAPI impace XHR
 (at least with the introduction of ArrayBuffers); Blob is useful in other
 areas, and FileWriter proposes a BlobBuilder.


 This was discussed publicly in the months leading up to DAP being chartered
 (including with involvement from Mozilla participants) but the eventual
 balance became the one we have today. I think (though I do not know for
 sure) that one factor in this was the fact that the File API which is so
 nicely alive today had, while DAP was being chartered, not been updated
 since 2006 and was still called the File Upload API.


 This is true.  But, I see no impediment to changing this for the better,
 given the existing charter language on WebApps.  Do you, or does anyone that
 is a member of the DAP WG?  Likewise, does any member of the WebApps WG
 object strongly?

 -- A*

Re: Updates to File API

[Eric Uhrhane]

Arun:

In the latest version of the spec I see that readAsDataURL, alone
among the readAs* methods, still takes a File rather than a Blob.  Is
that just an oversight, or is that an intentional restriction?

 Eric

On Thu, May 13, 2010 at 5:27 AM, Arun Ranganathan a...@mozilla.com wrote:
 Greetings WebApps WG,

 I have updated the editor's draft of the File API to reflect changes that
 have been in discussion.

 http://dev.w3.org/2006/webapi/FileAPI

 Notably:

 1. Blobs now allow further binary data operations by exposing an ArrayBuffer
 property that represents the Blob.  ArrayBuffers, and affiliated Typed
 Array views of data, are specified in a working draft as a part of the
 WebGL work [1].  This work has been proposed to ECMA's TC-39 WG as well.  We
 intend to implement some of this in the Firefox 4 timeframe, and have reason
 to believe other browsers will as well.  I have thus cited the work as a
 normative reference [1].  Eventually, we ought to consider further read
 operations given ArrayBuffers, but for now, I believe exposing Blobs in this
 way is sufficient.

 2. url and type properties have been moved to to the underlying Blob
 interface.  Notably, the property is now called 'url' and not 'urn.'  Use
 cases for triggering 'save as' behavior with Content-Disposition have not
 been addressed[2], although I believe that with FileWriter and
 BlobBuilder[3] they may be addressed differently.  This change reflects
 lengthy discussion (e.g. start here[4])

 3. The renaming of the property to 'url' also suggests that we should cease
 to consider an urn:uuid scheme.  I solicited implementer feedback about URLs
 vs. URNs in general.  There was a general preference to URLs[5], though this
 wasn't a strong preference.   Moreover, Mozilla's implementation currently
 uses moz-filedata: .  The current draft has an editor's note about the use
 of HTTP semantics, and origin issues in the context of shared workers.  This
 is work in progress; I have removed the section specifying urn:uuid and hope
 to have an update with a section covering the filedata: scheme (with
 filedata:uuid as a suggestion).  I welcome discussion about this.  I'll
 point out that we are coining a new scheme, which we originally sought to
 avoid :-)

 4. I have changed event order; loadend now fires after an error event [6].

 -- A*

 [1]
 https://cvs.khronos.org/svn/repos/registry/trunk/public/webgl/doc/spec/TypedArray-spec.html
 [2] http://www.mail-archive.com/public-webapps@w3.org/msg06137.html
 [3] http://dev.w3.org/2009/dap/file-system/file-writer.html
 [4] http://lists.w3.org/Archives/Public/public-webapps/2010JanMar/0910.html
 [5] http://lists.w3.org/Archives/Public/public-webapps/2009OctDec/0462.html
 [6] http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/0062.html

Re: Updates to File API

[Eric Uhrhane]

On Wed, Jun 2, 2010 at 3:44 PM, Arun Ranganathan a...@mozilla.com wrote:
 On 6/2/10 3:42 PM, Eric Uhrhane wrote:

 Arun:

 In the latest version of the spec I see that readAsDataURL, alone
 among the readAs* methods, still takes a File rather than a Blob.  Is
 that just an oversight, or is that an intentional restriction?


 That's intentional; readAsDataURL was cited as useful only in the context of
 File objects.  Do you think it makes sense in the context of random Blob
 objects?  Does it make sense on slice calls on a Blob, for example?

Sure, why not?  Why would this be limited to File objects?

A File is supposed to refer to an actual file on the local hard drive.
 A Blob is a big bunch of data that you might want to do something
with.  There's nothing special about a File when it comes to what
you're doing with the data.

Just as we moved File.url up to Blob, I think File.readAsDataURL
belongs there too.

Re: Updates to File API

[Eric Uhrhane]

On Wed, Jun 2, 2010 at 3:57 PM, Arun Ranganathan a...@mozilla.com wrote:
 On 6/2/10 3:48 PM, Eric Uhrhane wrote:

 Sure, why not?  Why would this be limited to File objects?

 A File is supposed to refer to an actual file on the local hard drive.
  A Blob is a big bunch of data that you might want to do something
 with.  There's nothing special about a File when it comes to what
 you're doing with the data.

 Just as we moved File.url up to Blob, I think File.readAsDataURL
 belongs there too.


 Fair enough; I'm amenable to moving it.  So specifically, you're okay with a
 DataURL on a Blob?  It might not be anything useful; with a File, you at
 least have the possibility of a whole unsliced image file.  Could you give
 me a use case where this is really useful for Blob objects?

One that's come up for Blob.url is a packed file of image thumbnails:
you can do one big download, then slice and display the pieces.  If
you're doing any display by data URLs, that would work there too. To
be honest, I think a lot of the data URL use cases are better served
by Blob.url anyway, so I'm not sure how many will remain once this
spec is fully implemented, but can you think of a data URL use case
that really depends on the data coming from a File on disk instead of
a  Blob?

 Also, above you probably mean specifying that readAsDataURL (a method on
 FileReader) works on Blob objects, not File.readAsDataURL ;-)

Yeah.  Brain-o.

Re: Updates to File API

[Eric Uhrhane]

On Mon, May 17, 2010 at 3:37 PM, Dmitry Titov dim...@chromium.org wrote:
 I have couple of questions, mostly clarifications I think:
 1. FileReader takes Blob but there are multiple hints that the blob should
 be actually a 'file'. As we see Blob concept grows in popularity with such
 specs as FileWriter that defines BlobBuilder. Other proposals include Image
 Resizing that returns a Blob with a compressed image data. Can all types of
 Blobs be 'read' using FileReader? If not, then it would be logical to only
 accept File parameter. If any type of Blob can be read (as I think the
 spirit of the spec assumes) then would it be less confusing to cange the
 name to BlobReader?

I'd support that.  I think we always want to allow reading of any type
of Blob--it's the interchange container--so calling it BlobReader
makes sense.  Arun, how do you feel about that?

Would FileWriter ever be used to write anything other than a File?  I
think not, so it should probably stay as it is, despite the lack of
symmetry.

 2. The FileReader.result is a string. There could be useful cases where it
 could be useful to read the data as ArrayBuffer. For example, if a page
 tries to crack the JPG file to extract the EXIF metadata. Maybe returning a
 Blob that can later be asked for ArrayBuffer would be as good.

You're going to give a Blob to a FileReader, and get the same Blob back?

 Dmitry
 On Fri, May 14, 2010 at 11:52 AM, Arun Ranganathan a...@mozilla.com wrote:

 On 5/13/10 9:32 PM, Darin Fisher wrote:

 Glad to hear that you didn't intend sync access :-)


 I have thoughts on Blob and how it should behave (and about the
 inheritance relationship between Blob and File), which is why I left the
 unfortunate error in the editor's draft for now (commented out and
 caveated).  This is the subject of a separate email thread (but don't worry
 -- while my thoughts on Blob and ArrayBuffer may be in some flux, sync
 access to File objects is *always* going to be a no-no, I promise :-) ).

 Now aside from the Blob - ArrayBuffer relationship, which I introduced,
 the rest of the changes are in keeping with threads discussing the File API.

 Can you define the contentType parameter to slice better?  Is that
 intended
 to correspond to the value of a HTTP Content-Type response header?  For
 example, can the contentType value include a charset attribute?  It might
 be
 useful to indicate that a slice of a file should be treated as text/html
 with a specific encoding.



 I'm happy to define it better in terms of what it *should* be, but web
 developers are likely to use it in ways that we can't predict, which is why
 forcing Content-Types is useful, but weird.  Why exactly do you mean when
 you say that a slice of a file should be treated as text/html with a
 specific encoding?  Can you give me a use case that illustrates why this is
 a good way to define this?

 I'm also a fan of providing a way to specify optional
 Content-Disposition
 parameters in the slice call.

 So I'm really not a Content-Disposition fan, since all the use cases I've
 seen so far seem to be to force download behavior (or trigger Download
 Manager).  Is there something I'm missing -- e.g. is there something here
 that FileWriter or BlobBuilder do *not* address, that putting
 Content-Disposition on Blob URLs *does* address?  Sorry if I'm missing
 something obvious.

 -- A*

Re: Updates to File API

[Eric Uhrhane]

On Fri, May 14, 2010 at 11:52 AM, Arun Ranganathan a...@mozilla.com wrote:
 On 5/13/10 9:32 PM, Darin Fisher wrote:

 Glad to hear that you didn't intend sync access :-)


 I have thoughts on Blob and how it should behave (and about the inheritance
 relationship between Blob and File), which is why I left the unfortunate
 error in the editor's draft for now (commented out and caveated).  This is
 the subject of a separate email thread (but don't worry -- while my thoughts
 on Blob and ArrayBuffer may be in some flux, sync access to File objects is
 *always* going to be a no-no, I promise :-) ).

 Now aside from the Blob - ArrayBuffer relationship, which I introduced, the
 rest of the changes are in keeping with threads discussing the File API.

 Can you define the contentType parameter to slice better?  Is that
 intended
 to correspond to the value of a HTTP Content-Type response header?  For
 example, can the contentType value include a charset attribute?  It might
 be
 useful to indicate that a slice of a file should be treated as text/html
 with a specific encoding.



 I'm happy to define it better in terms of what it *should* be, but web
 developers are likely to use it in ways that we can't predict, which is why
 forcing Content-Types is useful, but weird.  Why exactly do you mean when
 you say that a slice of a file should be treated as text/html with a
 specific encoding?  Can you give me a use case that illustrates why this is
 a good way to define this?

I can't speak for Darin, but I'd think the same reasoning that applies
whenever a server adds those headers via HTTP should apply whenever a
client-side app wants to add them to a Blob.url.

 I'm also a fan of providing a way to specify optional
 Content-Disposition
 parameters in the slice call.

 So I'm really not a Content-Disposition fan, since all the use cases I've
 seen so far seem to be to force download behavior (or trigger Download
 Manager).  Is there something I'm missing -- e.g. is there something here
 that FileWriter or BlobBuilder do *not* address, that putting
 Content-Disposition on Blob URLs *does* address?  Sorry if I'm missing
 something obvious.

It is indeed generally intended to trigger Download Manager.  If you
take a look at my use case at [1], the idea is to give web developers
a facility that's just like the one they're already using, so that
anything they do with URLs for files online they can also do with URLs
for Blobs offline/client-side.

The FileWriter spec's a bit up in the air over the same issue; I
haven't yet specced a good way for FileWriter to solve this problem,
so it's hard to say it's going to handle it better.

 Eric

[1] http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/0412.html

Re: XMLHttpRequest.responseBlob

[Eric Uhrhane]

On Wed, Apr 28, 2010 at 12:45 PM, Darin Fisher da...@chromium.org wrote:
 On Wed, Apr 28, 2010 at 11:57 AM, Michael Nordman micha...@google.com
 wrote:


 On Wed, Apr 28, 2010 at 11:21 AM, Jonas Sicking jo...@sicking.cc wrote:

 Ugh, sent this originally to just Darin. Resending to the list.

 On Wed, Apr 28, 2010 at 10:11 AM, Darin Fisher da...@chromium.org
 wrote:
  On Tue, Apr 27, 2010 at 2:04 PM, Jonas Sicking jo...@sicking.cc
  wrote:
 
  On Tue, Apr 27, 2010 at 1:59 PM, Darin Fisher da...@chromium.org
  wrote:
   On Tue, Apr 27, 2010 at 1:33 PM, Jonas Sicking jo...@sicking.cc
   wrote:
  
   On Tue, Apr 27, 2010 at 1:26 PM, Darin Fisher da...@chromium.org
   wrote:
It would be nice to be able to allow streaming such that every
time
a
progress event is fired only the newly downloaded data is
available.
The UA is then free to throw away that data once the event is
done
firing. This would be useful in the cases when the page is able
to
do
incremental parsing of the resulting document.
   
If we add a 'load mode' flag on XMLHttpRequest, which can't be
modified after send() is called, then streaming to a Blob could
simply
be another enum value for such a flag.
   
There is still the problem of how the actual blob works. I.e.
does
.responseBlob return a new blob every time more data is
returned? Or
should the same Blob be constantly modifying? If modifying, what
happens to any in-progress reads when the file is modified? Or
do
you
just make the Blob available once the whole resource has been
downloaded?
   
   
   
This is why I suggested using FileWriter.  FileWriter already has
to
deal with
most of the problems you mentioned above,
  
   Actually, as far as I can tell FileWriter is write-only so it
   doesn't
   deal with any of the problems above.
  
   When you use createWriter, you are creating a FileWriter to an
   existing
   File.
   The user could attempt to create a FileReader to the very same File
   while
   a FileWriter is open to it.
   It is true that for input type=saveas there is no way to get at
   the
   underlying
   File object.  That is perhaps a good thing for the use case of
   downloading
   to
   a location specified by the user.
 
  Ah. But as far as I can tell (and remember), it's still fairly
  undefined what happens when the OS file under a File/Blob object is
  mutated.
 
  / Jonas
 
  Agreed.  I don't see it as a big problem.  Do you?  The application
  developer is
  in control.  They get to specify the output file (via FileWriter) that
  XHR
  sends its
  output to, and they get to know when XHR is done writing.  So, the
  application
  developer can avoid reading from the file until XHR is done writing.

 Well, it seems like a bigger deal here since the file is being
 constantly modified as we're downloading data into it, no? So for
 example if you grab a File object after the first progress event, what
 does that File object contain after the second? Does it contain the
 whole file, including the newly downloaded data? Or does it contain
 only the data after the first progress event? Or is the File object
 now invalid and can't be used?

 What gears did about that was to provide a 'snapshot' of the
 downloaded data each time responseBlob was called, with
 the 'snapshot' being consistent with the progress events
 having been seen by the caller. The 'snapshot' would remain
 valid until discarded by the caller. Each snapshot just provided
 a view onto the same data which maybe was in memory or
 maybe had spilled over to disk unbeknownst to the caller.


 I'm also still unsure that a FileWriter is what you want generally. If
 you're just downloading temporary data, but data that happens to be so
 large that you don't want to keep it in memory, you don't want to
 bother the user asking for a location for that temporary file. Nor do
 you want that file to be around once the user leaves the page.


 I think the point about not requiring the caller to manage the 'file' are
 important.


 Sure, if the use case is actually downloading and saving a file for
 the user to use, rather than for the page to use, then a FileWriter
 seems like it would work. I.e. if you want something like
 Content-Disposition: attachment, but where you can specify request
 headers. Is that the use case?

 Mods to xhr to access the response more opaquely is a fairly general
 feature request. One specific use case is to download a resource via xhr
 and then save the results in a sandboxed file system. So for the page to
 use.

 ^^^ That is the use case I'm primarily interested in.

I think there are a couple of important use cases here, and FileWriter
really only works for one of them.  It would work fine for a sandboxed
filesystem, as you say.  However, if you just want to get a chunk of
binary data from the server, and don't want to manage its lifetime [or
don't have permission to use the 

Re: [FileAPI] Blob.URN?

[Eric Uhrhane]

I've been going through this thread trying to figure out how to make
FileWriter [1] work cleanly for the various use cases presented, and I
think the reason I've been having so much trouble is that it's just a
bad idea to do so.  Its original design constraints rule out some of
the use cases.  I think we'd be better off with two different APIs
than with trying to mash everything into one object.

Here's my reasoning; please pardon the fragmentary recap:

Jonas wrote:
 So we could allow FileWriter to be created directly in addition to
 using input type=saveas. When instantiated directly any call to
 .write will bring up a save-as dialog. This is useful for the cases
 where Darin and others wanted to use content-disposition to bring up
 the save-as dialog.

This would make a FileWriter object behave differently based on where
it came from, which seems like it's going to lead to bugs.

Jonas wrote:
 On a separate note, I have no idea how to create UI that makes the
 user understand what they grant when selecting a file destination
 using input type=saveas. I.e. that they are granting the web page
 continuous writing (but not reading?) capabilities to the specific
 file location.

and

 The main problem I have with the currently suggested UI is that it
 still doesn't explain to the user what he is granting at the time when
 the user is asked to grant access. It's only after the user has only
 granted access that it's explained what is happening. That seems like
 a problem.

This is a critical point.  We went back and forth on it for a bit, on
just how different a FileWriter was from a save-as, and on how to use
the download manager to indicate an open [although perhaps not active]
FileWriter, but couldn't really solve it in general.

Jonas again:
 I've so far thought of the main use case for multiple writing
 something like a document editor, where the user chooses a file
 location once, and then the web application can continuously save
 every 5 minutes or so to that location. As well as any time the user
 explicitly presses some 'save' button. It seems like this work flow
 doesn't go well with the UI you are describing.

We currently have no good idea for how to do the UI for this, but
thinking back, it's an explicit non-goal of the FileWriter API.  In
the use cases I posted before making my FileWriter proposal [2],
that's what I called a Group 2 use case, meaning that it's got
security issues we don't yet know how to explain to the user, just as
Jonas points out.

If you want to edit and save to the same file over and over without
multiple prompts, you should be saving to a sandboxed area using the
FileSystem API [3].

Tab brings up other use cases:
 One that we thought of today was the case of playing a game over the
 web (perhaps using WebSockets?) and wanting to save a replay.  You
 don't want to have to build up a Blob until the very end, and then
 deal with all the possible failure conditions.  If the user's power
 goes out halfway, frex, it would be nice to at least have a partial
 file downloaded.

 Another is to save to disk a streaming video resource that you are
 viewing part of at the moment.  Same concerns apply - you'd like a
 usable file with as much data as possible in it despite any failure
 modes.

Both of these are simple and safe using the sandboxed filesystem, and
annoying or confusing using FileWriter and a save-as dialog.

 I would have much rather focused on starting simple and create an API
 for the already existing save as dialog. That is a problem that has
 already been solved for a long time, and so we wouldn't have to argue
 about if a good UI can be created. It is also the feature set that we
 had previously discussed on this list as being a good starting point.

 / Jonas

I agree.  Here's what I propose:

FileWriter is needed for the FileSystem API.  It's not a great match
for a one-time export of a data Blob from a web app.  I think I should
take out all the how do I get a FileWriter stuff that's in [1],
replacing it with a note that you just get one from the FileSystem
API.

For one-time export of a Blob to a user-selected location, we have
several attractive options.  We can add a separate API, which can be
quite simple.  On activation [either by control/button or
window.saveAs(blob), TBD], it pops up a file selector, saves, and
reports success/failure.  Alternatively, we can go the blob.url route
with content-disposition.

Michael said:
 When it comes to questions like these (when offline, how to do something
 that happens in the online case)... i tend to say why not do it exactly the
 same way as its done in the online case... so load a resource with a
 Content-Disposition:attachment header on it :)

To which Jonas replied:
 online doesn't have File objects, it has URLs, so the analogy is
 pretty poor IMHO.

I think the analogy works quite well.  Online you have a URL.  You
click on it, and due to sideband information in the headers, it
downloads instead of opening.  If 

Re: XMLHttpRequest.responseBlob

[Eric Uhrhane]

On Mon, Apr 26, 2010 at 11:03 PM, Darin Fisher da...@chromium.org wrote:
 On Mon, Apr 26, 2010 at 3:52 PM, Jonas Sicking jo...@sicking.cc wrote:

 On Mon, Apr 26, 2010 at 3:39 PM, Darin Fisher da...@chromium.org wrote:
  On Mon, Apr 26, 2010 at 3:29 PM, Jonas Sicking jo...@sicking.cc wrote:
 
  On Mon, Apr 26, 2010 at 3:21 PM, Darin Fisher da...@chromium.org
  wrote:
   There is some interest from application developers at Google in being
   able
   to get a Blob corresponding to the response body of a XMLHttpRequest.
   The use case is to improve the efficiency of getting a Blob from a
   binary
   resource downloaded via XHR.
   The alternative is to play games with character encodings so that
   responseText can be used to fetch an image as a string, and then use
   BlobBuilder to reconstruct the image file, again being careful with
   the
   implicit character conversions.  All of this is very inefficient.
   Is there any appetite for adding a responseBlob getter on XHR?
 
  There has been talk about exposing a responseBody property which would
  contain the binary response. However ECMAScript is still lacking a
  binary type.
 
  Blob does fit the bill in that it represents binary data, however it's
  asynchronous nature is probably not ideal here, right?
 
  / Jonas
 
 
  I think there are applications that do not require direct access to the
  response data.
  For example,
  1- Download a binary resource (e.g., an image) via XHR.
  2- Load the resource using Blob.URN (assuming URN moves from File to
  Blob).
  It may be the case that providing direct access to the response data may
  be
  more
  expensive than just providing the application with a handle to the data.
   Consider
  the case of large files.

 Ah, so you want the ability to have the XHR implementation stream to
 disk and then use a Blob to read from there? If so, you need more
 modifications as currently the XHR implementation is required to keep
 the whole response in memory anyway in order to be able to implement
 the .responseText property.

 So we'll need to add some way for the page to indicate to the
 implementation I don't care about the .responseText or .responseXML
 properties, just responseBlob

 / Jonas


 I thought about this more, and I came to the same conclusion as you.  I
 agree that we wouldn't want to support .responseText or .responseXML if we
 were streaming directly to a file because the implied synchronous readback
 from disk would suck.
 I'm not sure how to add such a feature to XHR in a way that is not awkward.
  Perhaps if there was a way to bind a FileWriter to an XMLHttpRequest object
 prior to calling send?

A FileWriter is quite a bit more limited than a File/Blob.  You can't
slice it, for instance, and you can't write it to multiple files
without writing, reading back, and writing again.

FileWriter updated

[Eric Uhrhane]

I've rolled in feedback from the Device APIs and Policy WG [1], the
WebApps WG [2], and some private communications [reporting typo-level
errors] and posted a new draft [3].  In addition to trivial fixes, I
also took care of the following issues:

From Anne van Kesteren:

 I think the exception on append() should move to setting the endings 
 attribute.
 I.e. when setting endings to an unsupported value it  throws. It should throw
 a SYNTAX_ERR exception. That would be most compatible with similar attributes
 in HTML5. (Attributes limited to a known set of values.)

I've changed to SYNTAX_ERR, and the exception is now thrown where the
bad value is supplied.

For Dmitry Titov:

I cut the ending options to just two: transparent and native.  I've
left in the issue that we may want to drop ending conversion entirely,
but I've yet to hear consensus for that.

For Dmitry Titov and Kinuko Yasuda:

I've converted the endings attribute to a parameter to append().

For Robin Berjon:

Fixed up or added normative references top DOM3 events and progress events.
Clarified the description about what happens when the user requests an
unsupported ending conversion type.

For Darin Fisher and Jonas Sicking:

Added a contentType parameter to BlobBuilder.

For Kinuko Yasuda:

Added notes about obtaining a FileWriterSync.

The big unresolved issue at present is how one obtains a FileWriter
for its basic use (that is, where one is not using the FileSystem API
[4]).  I have some thoughts about this that I'll be posting in a fresh
thread.

Eric

[1] http://lists.w3.org/Archives/Public/public-device-apis/2010Mar/0072.html
[whole thread]
[2] http://lists.w3.org/Archives/Public/public-webapps/2010JanMar/0910.html
[see later in the thread]
[3] http://dev.w3.org/2009/dap/file-system/file-writer.html
[4] http://dev.w3.org/2009/dap/file-system/file-dir-sys.html

Re: [IndexedDB] Granting storage quotas

[Eric Uhrhane]

On Wed, Apr 21, 2010 at 7:58 AM, Robin Berjon ro...@berjon.com wrote:
 On Tue, 20 Apr 2010 14:37:33 -0700 Michael Nordman micha...@google.com 
 wrote:
 I think ericu is baking in a distinction in between 'permanent' and
 'temporary' in the FileSystem API he's working on. Some harmony across all
 flavors of local storage could be good.

 He is:

  http://dev.w3.org/2009/dap/file-system/file-dir-sys.html#using-localfilesystemsync

 I haven't seen ericu pop up in this thread but I guess he's reading it. 
 Either way, it's certainly an approach that I'd be happy to see fine-tuned to 
 be more or less consistent across various local storage mechanisms.

Thanks Robin--I'm actually just now catching up on most of my email,
as I've been out of the office for a week.

I agree completely that we need consistency across storage mechanisms.
 Using Nikunj's terminology, I'd love to see an evictable/persistent
split on all storage types, and some kind of unified quota management
for all client-side storage.  I'm not sure that we need to spec out
all the quota management stuff--we should leave a lot of room for UAs
to experiment with how to interact with the user--but if we can add a
simple mechanism for declaring/requesting quota across all storage
types, I think that would be very helpful.

Dumi has proposed a programmatic API, and Tab has suggested a new
input type.  I think either of those would be OK as long as
interaction is completely asynchronous.

 Eric

Re: [FileAPI] Blob.URN?

[Eric Uhrhane]

Sorry about the slow response, Dmitry.

On Wed, Mar 24, 2010 at 11:29 AM, Dmitry Titov dim...@google.com wrote:
 Those seem to open up so many edge cases...
 If we have a File constructor like this, now we have a new category of File
 objects that do have names but are not related to actual files on a local
 file system, and perhaps have different lifetime expectations.
 Ability to specify a name and content disposition also does not fall in
 naturally. For example, is what happens when a blob with 'inline'
 disposition used in anchor's href or iframe's src? What happens if one
 specifies the creation/modification time parameters? What would it mean to
 specify a size parameter in content disposition when underlying Blob also
 has size property?

We could just disallow any parameter that doesn't make sense in this context.

 Even one step back, are we sure there is a use case for Blob.urn? If Blobs
 are sliced from the files on a local filesystem that user selected via a
 File Dialog, what would be a scenario for slice/load? There were requests to
 have a single network resource that can be slice/loaded on arrival, but was
 the slicing and loading of local files discussed? The most obvious use case
 for local files is to be uploaded, perhaps in slices.

Here's a previous discussion of some use cases:
http://lists.w3.org/Archives/Public/public-webapps/2009OctDec/0506.html

 On Wed, Mar 24, 2010 at 1:58 AM, Anne van Kesteren ann...@opera.com wrote:

 On Wed, 24 Mar 2010 03:40:36 +0100, Michael Nordman micha...@google.com
 wrote:

 This has been discussed before, not sure what the conclusion was (if any)
 http://www.mail-archive.com/public-webapps@w3.org/msg06137.html

 http://www.mail-archive.com/public-webapps@w3.org/msg06345.htmlsnip

 In order for the URN to be useful, it would have to have a mediaType
 associated with it, and then there's content-disposition to think
 about, which then wants a file name as well...boy, that's a lot of
 baggage.  However, since these aren't really inherent properties of
 the Blob, just of the way you want the browser to view the Blob, it
 would seem natural to me do to something like this:

    interface Blob {
      ...
      DOMString getURN(in DOMString mediaType,
           [Optional] in DOMString contentDisposition,
           [Optional] in DOMString name);
    };

 /snip

 Wouldn't it be better to have a constructor for File. I.e.

  File(Blob, name, type, contentdisposition)

 or some such. (Maybe some of the attributes on File should be made
 mutable, i.e. name and mime...)


 Also, didn't we decide to change URN to URL? As far as I can tell that is
 how Gecko is implementing it.


 --
 Anne van Kesteren
 http://annevankesteren.nl/

Re: [FileAPI] Blob.URN?

[Eric Uhrhane]

On Tue, Mar 30, 2010 at 2:57 PM, Darin Fisher da...@chromium.org wrote:
 On Tue, Mar 30, 2010 at 12:22 AM, Jonas Sicking jo...@sicking.cc wrote:

 On Tue, Mar 23, 2010 at 2:23 PM, Darin Fisher da...@chromium.org wrote:
  Apologies if this has been discussed before, but I'm curious why URN is
  not
  a property of Blob.  It seems like it would be useful to be able to load
  a
  slice of a File.  For example, this could be used by an application to
  fetch
  all of its subresources out of a single file.

 IIRC originally it was placed on File since Blobs do not have a
 content type. However I think there is general agreement that it
 should be moved to Blob.

 However it would be great to be able to assign a content type to a
 Blob. Possibly slice() could take a optional argument.


 Adding an optional parameter to slice() sounds attractive indeed.
 BlobBuilder [1] should probably also have such an optional argument.
 -Darin
 [1] http://dev.w3.org/2009/dap/file-system/file-writer.html#the-blobbuilder-interface

That seems reasonable.  But if we move the content type and
disposition to the Blob, and can get a URL from a Blob, then File is
left with...just a name?

Re: [FileAPI] Blob.URN?

[Eric Uhrhane]

On Thu, Apr 1, 2010 at 2:07 AM, Jonas Sicking jo...@sicking.cc wrote:
 On Thu, Apr 1, 2010 at 1:03 AM, Darin Fisher da...@chromium.org wrote:
 On Wed, Mar 31, 2010 at 3:01 PM, Eric Uhrhane er...@google.com wrote:

 On Wed, Mar 31, 2010 at 2:40 PM, Jonas Sicking jo...@sicking.cc wrote:
  On Wed, Mar 31, 2010 at 1:55 AM, Robin Berjon ro...@berjon.com wrote:
  On Mar 31, 2010, at 01:56 , Darin Fisher wrote:
  The only way to get a FileWriter at the moment is from input
  type=saveas.  What is desired is a way to simulate the load of a 
  resource
  with Content-Disposition: attachment that would trigger the browser's
  download manager.
 
  I don't think that input type=saveas is a good solution for this, for
  one it falls back to a text input control, which is less than ideal. I 
  think
  that the File Writer should trigger downloads on an API call since that
  doesn't introduce security issues that aren't already there. I'll make a
  proposal for that.
 
  Why not simply allow FileWriters be instantiated using:
 
  x = new FileWriter;
 
  And then make every call to .write open up the save as dialog.

 You wouldn't want to prompt on every write; developers might want to
 append a chunk at a time.
 You could prompt on creation, if you didn't mind that being a synchronous
 call.

 The reason I made an html element be the way of getting a FileWriter
 was to make the normal usage pattern not involve modal dialogs popping
 up in front of the user unprompted.  With an inpput control, they can
 choose when to interact with it, rather than having a speedbump shoved
 in front of them.


 Please no more JS functions that block on modal dialogs! :-)
 @sicking:  It seems odd to vary the behavior of .append based on how the
 FileWriter was created.  Perhaps we should provide some other asynchronous
 means of creating a FileWriter?

 I'm not suggesting any blocking JS functions. Calling
 FileWriter.write() is specified as an asynchronous function. This
 gives a great opportunity to display the save-as dialog and, if the
 user press 'cancel', fire an error event, otherwise start firing
 progress events.

Even without the JS API blocking, modal dialogs that come up without
user intervention are still annoying.  It's a much nicer experience
for the user to be able to call one up by clicking a SaveAs button or
the equivalent, don't you think?  It's also closer to the File Reader
API.

 Relying on asynchronous fetching of the FileWriter to deal with the
 save-as dialog seems to have two problems:
 1. The file type isn't known at this time, so the user won't know what
 he's saving.

The user never truly knows what he's saving, even with a download, but
he does at least pick the filename in either case.
We could require a file type in the html element, if that would help.

 2. This always seems to grant the web page the ability to write
 unlimited number of times and unlimited amount of data, to the given
 location. Today, for things like Content-Disposition, the user only
 grants the right to save a single, known, file once. This is enough to
 solve many use cases.

This is still just a single file, in a known location of the user's
choice.  It's just different that it's not closed right away, as with
a long-running download, but a bit more so.  How about this: we add a
close() call, after which no writes can be made.  From the time the
FileWriter's created until close() is called [or it's GCed, or the
document goes away], we consider the file open.  UAs are free e.g. to
display the download manager with an indication that the file's being
written, and the current byte count.  When you close the FileWriter,
it's like the download has completed.

Re: [FileAPI] Blob.URN?

[Eric Uhrhane]

On Wed, Mar 31, 2010 at 3:55 PM, Jonas Sicking jo...@sicking.cc wrote:
 On Wed, Mar 31, 2010 at 3:01 PM, Eric Uhrhane er...@google.com wrote:
 On Wed, Mar 31, 2010 at 2:40 PM, Jonas Sicking jo...@sicking.cc wrote:
 On Wed, Mar 31, 2010 at 1:55 AM, Robin Berjon ro...@berjon.com wrote:
 On Mar 31, 2010, at 01:56 , Darin Fisher wrote:
 The only way to get a FileWriter at the moment is from input 
 type=saveas.  What is desired is a way to simulate the load of a 
 resource with Content-Disposition: attachment that would trigger the 
 browser's download manager.

 I don't think that input type=saveas is a good solution for this, for 
 one it falls back to a text input control, which is less than ideal. I 
 think that the File Writer should trigger downloads on an API call since 
 that doesn't introduce security issues that aren't already there. I'll 
 make a proposal for that.

 Why not simply allow FileWriters be instantiated using:

 x = new FileWriter;

 And then make every call to .write open up the save as dialog.

 You wouldn't want to prompt on every write; developers might want to
 append a chunk at a time.
 You could prompt on creation, if you didn't mind that being a synchronous 
 call.

 The reason I made an html element be the way of getting a FileWriter
 was to make the normal usage pattern not involve modal dialogs popping
 up in front of the user unprompted.  With an inpput control, they can
 choose when to interact with it, rather than having a speedbump shoved
 in front of them.

 So we could allow FileWriter to be created directly in addition to
 using input type=saveas. When instantiated directly any call to
 .write will bring up a save-as dialog. This is useful for the cases
 where Darin and others wanted to use content-disposition to bring up
 the save-as dialog.

 On a separate note, I have no idea how to create UI that makes the
 user understand what they grant when selecting a file destination
 using input type=saveas. I.e. that they are granting the web page
 continuous writing (but not reading?) capabilities to the specific
 file location.

They're granting write access to a single location, as with a
download.  This explicitly doesn't grant read access, and the write
access doesn't survive the document, so it's not so different from a
slow download.  I just sent a proposal for UI to indicate the
difference in another response [1]; sorry, I probably should have
merged several of these emails.

Eric

[1] http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/0009.html

Re: [FileAPI] Blob.URN?

[Eric Uhrhane]

On Wed, Mar 31, 2010 at 2:40 PM, Jonas Sicking jo...@sicking.cc wrote:
 On Wed, Mar 31, 2010 at 1:55 AM, Robin Berjon ro...@berjon.com wrote:
 On Mar 31, 2010, at 01:56 , Darin Fisher wrote:
 The only way to get a FileWriter at the moment is from input 
 type=saveas.  What is desired is a way to simulate the load of a 
 resource with Content-Disposition: attachment that would trigger the 
 browser's download manager.

 I don't think that input type=saveas is a good solution for this, for one 
 it falls back to a text input control, which is less than ideal. I think 
 that the File Writer should trigger downloads on an API call since that 
 doesn't introduce security issues that aren't already there. I'll make a 
 proposal for that.

 Why not simply allow FileWriters be instantiated using:

 x = new FileWriter;

 And then make every call to .write open up the save as dialog.

You wouldn't want to prompt on every write; developers might want to
append a chunk at a time.
You could prompt on creation, if you didn't mind that being a synchronous call.

The reason I made an html element be the way of getting a FileWriter
was to make the normal usage pattern not involve modal dialogs popping
up in front of the user unprompted.  With an inpput control, they can
choose when to interact with it, rather than having a speedbump shoved
in front of them.

Re: The most basic File API use case

[Eric Uhrhane]

On Wed, Dec 16, 2009 at 11:58 AM, Jonas Sicking jo...@sicking.cc wrote:
 On Wed, Dec 16, 2009 at 9:55 AM, Robin Berjon ro...@berjon.com wrote:
 Hi Jonas,

 On Dec 10, 2009, at 19:42 , Jonas Sicking wrote:
 On Tue, Dec 8, 2009 at 9:03 AM, Robin Berjon ro...@berjon.com wrote:
 [Constructor(DOMString mediaType, DOMString fileName)]
 interface FileWriter {
    // saving operations
    void save (DOMString content, optional DOMString encoding, optional 
 DOMString endings);
    void save (Blob data);

 Hmm.. I'm not entirely convinced that it's worth having the first of
 these two methods. You're already up to two optional arguments, and
 more might be needed (such as if to include a BOM or not). It might be
 simpler to simply require people to create Blobs and then save that
 Blob.

 I could live with other options but there are things that are quite specific 
 to writing text, amongst them at the very least using the same encoding 
 throughout (which is clumsy if you have to append several times to a blob 
 and specify it every time. I thought of having a TextBlob that could take 
 encoding, line endings, BOM, etc. as parameters to its constructor and then 
 passing that to save() but I'm not entirely sure that what we get from the 
 change is really valuable.

 But if to need to append several times then you can't use the above
 interface anyway, can you?

 How about:

  void save (DOMString content, TextOptions options);

 where the second argument would be an object literal that could capture all 
 of this?

 Another option would be:

  Blob textToBlob (DOMString content, TextOptions options);

 I don't really see that this would be much more convenient than:

 bb = new BlobBuilder;
 bb.appendText(myString, ...);
 filewriter.save(bb.getBlob());



 possibly on another interface but again I'm not sure what that gains us. Do 
 we have use cases for textual blobs being used elsewhere? If yes, then I'm 
 thinking:

 interface TextBlobBuilder {
    attribute DOMString content;
    attribute DOMString encoding;
    attribute DOMString endings;
    attribute boolean BOM;
    Blob generateBlob ();
 };

 Thoughts?

 I don't think I understand this proposal. Can you elaborate, or show
 code that would use this?

    // abort the save
    void abort();

    // state codes
    const unsigned short INITIAL = 0;
    const unsigned short WRITING = 1;
    const unsigned short DONE    = 2;
    readonly attribute unsigned short readyState;

    // error, same as FileReader but with some additions
    readonly attribute FileError error;

    // event handler attributes
    attribute Function onloadstart;
    attribute Function onprogress;
    attribute Function onload;
    attribute Function onabort;
    attribute Function onerror;
    attribute Function onloadend;

 I think most of this is overkill. What is the use case for progress
 events here? I.e. why does the page need to know how much data has
 been written? And why would you want to abort writing the file?

 Well, if there are use cases for reading, the same apply for writing. If you 
 build a large file (e.g. for graphics editing) and save it to a slow storage 
 (e.g. network drive, SIM) then it could take a very measurable amount of 
 time (this happens in Photoshop even on powerful computers), and if it does 
 you probably want to inform the user and to provide her with a way to give 
 up.

 This is essentially a mirror of FileReader; I think it makes sense and not 
 just for consistency.

 Well.. I'm still not convinced that anyone will actually use progress
 events for reading files.

I think progress events are reasonable for anything that might take a
long time.  If you're loading/saving a large movie file from/to a
network filesystem, it's good to be able to tell the difference
between the action being hung and the write making slow progress.
This way you can show the user a progress bar.

 The idea of allowing the page to start a write and then aborting it
 scares me a little. And I'm not sure I see the use case. For example I
 could imagine implementations running a virus scan on the data before
 bringing up the save as dialog, or checking if the page is trying to
 write an executable. In these cases a call to abort() could then cause
 slightly different data to be written to disc than was originally
 checked.

If you're able to generate the data to be saved programmatically, you
can do write a byte at a time, and you can generate it after asking
for permission to save the file.  I don't think it makes sense to run
a scanner on it before it hits the disk, so I don't think there's any
security issue here that's not also there in any case where you can
write arbitrary files to the disk.

 I guess removing the abort() call would make me feel easier about
 this. Or specifying that a call to abort() causes all the data written
 so far to be deleted.

 My question still remains what the use case for abort() is though.

Abort() is useful if the user gets sick of 

Re: File API: Blob and underlying file changes.

[Eric Uhrhane]

On Wed, Jan 20, 2010 at 1:45 PM, Dmitry Titov dim...@chromium.org wrote:
 So it seems there is 2 ideas on how to handle the underlying file changes in
 case of File and Blob objects, nicely captured by Arun above:
 1. Keep all Blobs 'mutating', following the underlying file change. In
 particular, it means that Blob.size and similar properties may change from
 query to query, reflecting the current file state. In case the Blob was
 sliced and corresponding portion of the file does not exist anymore, it
 would be clamped, potentially to 0, as currently specified. Read operations
 would simply read the clamped portion. That would provide similar behavior
 of all Blobs regardless if they are the Files or obtained via slice(). It
 also has a slight disadvantage that every access to Blob.size or
 Blob.slice() will incur synchronous file I/O. Note that current
 File.fileSize is already implemented like that in FF and WebKit and uses
 sync file I/O.
 2. Treat Blobs that are Files and Blobs that are produced by slice() as
 different blobs, semantically. While former ones would 'mutate' with the
 file on the disk (to keep compat with form submission), the later would
 simply 'inherit' the file information and never do sync IO. Instead, they
 would fail later during async read operations. This has disadvantage of Blob
 behaving differently in some cases, making it hard for web developers to
 produce correct code. The synchronous file IO would be reduced but not
 completely eliminated, because the Blobs that are Files would continue to
 'sync' with the underlying file stats during sync JS calls. One benefit is
 that it allows detection of file content change, via checks of modification
 time captured when the first slice() operation is performed and verified
 during async read operations, which provides a way to implement reliable
 file operations in face of changing files, if the developer wants to spent
 an effort to do so.

 It seems folks on the thread do not like the duplicity of Blobs (hard to
 program and debug), and there is also a desire to avoid synchronous file IO.
 It seems the spec today leans more to the #1. The only problem with it is
 that it's hard to implement some scenarios, like a big file upload in chunks
 - in case the file changes, the result of upload may actually be a mix of
 new and old file contents and there is no way to check... Perhaps we can
 expose File.modificationTime? It still dos not get rid of sync I/O...

I think it could.  Here's a third option:

Make all blobs, file-based or not, just as async as the blobs in
option 2.  They never do sync IO, but could potentially fail future
read operations if their metadata is out of date [e.g. reading beyond
EOF].  However, expose the modification time on File via an async
method and allow the user to pass it in to a read call to enforce
fail if changed since this time.  This keeps all file accesses
async, but still allows for chunked uploads without mixing files
accidentally.  If we allow users to refresh the modification time
asynchronously, it also allows for adding a file to a form, changing
the file on disk, and then uploading the new file.  The user would
look up the mod time when starting the upload, rather than when the
file's selected.

Eric

 Dmitry
 On Fri, Jan 15, 2010 at 12:10 PM, Dmitry Titov dim...@chromium.org wrote:

 On Fri, Jan 15, 2010 at 11:50 AM, Jonas Sicking jo...@sicking.cc wrote:

 This doesn't address the problem that authors are unlikely to even
 attempt to deal with this situation, given how rare it is. And even
 less likely to deal with it successfully given how hard the situation
 is reproduce while testing.

 I don't know how rare the case is. It might become less rare if there is
 an uploader of big movie files and it's easy to overwrite the big movie file
 by hitting 'save' button in movie editor while it is still uploading...
 Perhaps such uploader can use other means to detect the file change
 though...
 It would be nice to spell out some behavior though, or we can end up with
 some incompatible implementations. Speaking about Blob.slice(), what is
 recommended behavior of resultant Blobs on the underlying file change?



 / Jonas

Re: File API: Blob and underlying file changes.

[Eric Uhrhane]

On Wed, Jan 20, 2010 at 3:23 PM, Dmitry Titov dim...@chromium.org wrote:
 On Wed, Jan 20, 2010 at 2:30 PM, Eric Uhrhane er...@google.com wrote:

 I think it could.  Here's a third option:
 Make all blobs, file-based or not, just as async as the blobs in
 option 2.  They never do sync IO, but could potentially fail future
 read operations if their metadata is out of date [e.g. reading beyond
 EOF].  However, expose the modification time on File via an async
 method and allow the user to pass it in to a read call to enforce
 fail if changed since this time.  This keeps all file accesses
 async, but still allows for chunked uploads without mixing files
 accidentally.  If we allow users to refresh the modification time
 asynchronously, it also allows for adding a file to a form, changing
 the file on disk, and then uploading the new file.  The user would
 look up the mod time when starting the upload, rather than when the
 file's selected.

 It would be great to avoid sync file I/O on calls like Blob.size. They would
 simply return cached value. Actual mismatch would be detected during actual
 read operation.
 However then I'm not sure how to keep File derived from Blob, since:
 1) Currently, in FF and WebKit File.fileSize is a sync I/O that returns
 current file size. The current spec says File is derived from Blob and Blob
 has Blob.size property that is likely going to co-exist with File.fileSize
 for a while, for compat reasons. It's weird for file.size and file.fileSize
 to return different things.

True, but we'd probably want to deprecate file.fileSize anyway and
then get rid of it, since it's synchronous.

 2) Currently, xhr.send(file) does not fail and sends the version of the file
 that exists somewhere around xhr.send(file) call was issued. Since File is
 also a Blob, xhr.send(blob) would behave the same which means if we want to
 preserve this behavior the Blob can not fail async read operation if file
 has changed.
 There is a contradiction here. One way to resolve it would be to break File
 is Blob and to be able to capture the File as Blob by having
 file.getAsBlob(). The latter would make a snapshot of the state of the file,
 to be able to fail subsequent async read operations if the file has been
 changed.
 I've asked a few people around in a non-scientific poll and it seems
 developers expect Blob to be a 'snapshot', reflecting the state of the file
 (or Canvas if we get Canvas.getBlob(...)) at the moment of Blob creation.
 Since it's obviously bad to actually copy data, it seems acceptable to
 capture enough information (like mod time) so the read operations later can
 fail if underlying storage has been changed. It feels really strange if
 reading the Blob can yield some data from one version of a file (or Canvas)
 mixed with some data from newer version, without any indication that this is
 happening.
 All that means there is an option 3:
 3. Treat all Blobs as 'snapshots' that refer to the range of underlying data
 at the moment of creation of the Blob. Blobs produced further by
 Blob.slice() operation inherit the captured state w/o actually verifying it
 against 'live' underlying objects like files. All Blobs can be 'read' (or
 'sent') via operations that can fail if the underlying content has changed.
 Optionally, expose snapshotTime property and perhaps read if not changed
 since parameter to read operations. Do not derive File from Blob, rather
 have File.getAsBlob() that produces a Blob which is a snapshot of the file
 at the moment of call. The advantage here is that it removes need for sync
 operations from Blob and provides mechanism to ensure the changing
 underlying storage is detectable. The disadvantage is a bit more complexity
 and bigger change to File spec.

That sounds good to me.  If we're treating blobs as snapshots, I
retract my suggestion of the read-if-not-changed-since parameter.  All
reads after the data has changed should fail.  If you want to do a
chunked upload, don't snapshot your file into a blob until you're
ready to start.  Once you've done that, just slice off parts of the
blob, not the file.