[FileAPI] Immutability and structured clones of File

[Glenn Maynard]

On Wed, Apr 13, 2011 at 1:01 AM, Darin Fisher da...@chromium.org wrote:

 Blobs are views on immutable data.  WebKit's implementation will reject
 reads on a Blob, which points to a file that has since been modified.  (This
 should be part of the spec if it is not.)


This suggests some answers to questions I've had:

http://lists.w3.org/Archives/Public/public-webapps/2011AprJun/0161.html(third
paragraph)
http://lists.whatwg.org/pipermail/whatwg-whatwg.org/2011-January/029821.html

What you describe reconciles File's immutability with the mutability of
actual files.  A File doesn't just represent a file; it represents a
particular version of a file, and if that version is no longer available
because the file was modified, then reading the File is an error, as if the
file had been deleted.

Is this described in the FileAPI spec?  The filesystem spec suggests it.
FileEntry.getFile() says Returns a File that *represents the current state
of the file*.  I don't see it described in File itself[2], though.

It still leaves the question of whether you can store a File in persistent
storage (localStorage, IndexedDB, saved History sessions) by structured
clone.  This allows sites to access files they were previously granted
access to.  I suspected security concerns: users don't expect that when they
open a file on a site and close the tab, when they return it still has
access to the file.  This seems less of an issue if File is restricted to
the particular version of the file at the time it was selected.  (The site
could have read the file and stored its contents with localStorage or the
Filesystem API the first time around, anyway.)

It would be particularly useful for History states, so restoring a browser
session can restore open files along with everything else.  Chrome allows
storing File in History states, but unfortunately they're lost on session
restore.


[1] http://dev.w3.org/2009/dap/file-system/file-dir-sys.html#methods-5
[2] http://dev.w3.org/2006/webapi/FileAPI/#file

-- 
Glenn Maynard

Re: [FileAPI] BlobBuilder.getBlob should clear the BlobBuilder

[Jonas Sicking]

On Tue, Apr 12, 2011 at 5:33 PM, Eric Uhrhane er...@google.com wrote:
 On Tue, Apr 12, 2011 at 3:38 PM, Kyle Huey m...@kylehuey.com wrote:
 Hello All,

 In the current FileAPI Writer spec a BlobBuilder can be used to build a
 series of blobs like so:

   var bb = BlobBuilder();
   bb.append(foo);
   var foo = bb.getBlob();
   bb.append(bar);
   var bar = bb.getBlob();
   foo.size; // == 3
   bar.size; // == 6

 My concern with this pattern is that it seems that one of the primary use
 cases is to keep a BlobBuilder around for a while to build up a blob over
 time.  A BlobBuilder left around could potentially entrain large amounts of
 memory.  I propose that BlobBuilder.getBlob() clears the BlobBuilder,
 returning it to an empty state.  The current behavior also doesn't seem
 terribly useful to me (though I'm happy to be convinced otherwise) and be
 easily replicated on top of the proposed behavior (immediately reappending
 the Blob that was just retrieved.)

 Thoughts/comments?

 - Kyle

 If you don't have a use for a current behavior, you can always just
 drop the BlobBuilder as soon as you're done with it, and it'll get
 collected.  I think that's simpler and more intuitive than having it
 clear itself, which is a surprise in an operation that looks
 read-only.  In the other case, where you actually want the append
 behavior, it's faster and simpler not to have to re-append a blob
 you've just pulled out of it.

The problem is that this optimizes for the rare case when you're
creating several blobs which are prefixes of each other.

It's not at all rare for pages to inadvertently hold on to objects
longer than they need. This bogs down both the users machine and
webpage. Yes, pages can fix this by dropping all the references to
an object and wait for GC, but it's all too common mistake not to do
this.

If we think that people will use BlobBuilder to create large blobs,
then it's better to have explicit API for dropping that rather than
relying on GC. Here we additionally have the advantage that we
wouldn't risk people forgetting to use the explicit API since that is
the same API as dropping the data.

Another advantage of dropping the memory automatically is that you
don't need to copy any data into the Blob. Instead you can just make
the Blob take ownership of whatever memory buffers you've built up
during the various calls to .append. You could technically implement
some sort of copy-on-write scheme, but that introduces complexity.

Flip it around, what is the argument for keeping the memory owned by
the BlobBuilder? If it's just that the name looks read-only, I'd be
fine with renaming the extraction-function to something else.

/ Jonas

Re: [FileAPI] File.slice spec bug

[Jonas Sicking]

On Tue, Apr 12, 2011 at 10:01 PM, Darin Fisher da...@chromium.org wrote:
 On Tue, Apr 12, 2011 at 5:59 PM, Brendan Eich bren...@mozilla.org wrote:

 Darin's position is leaning toward not breaking compatibility with what
 Chrome has shipped for a while. That's one consideration. It can't be the
 only consideration, or there's no point having a discussion and whatever
 Chrome ships first is an instant standard.

 Agreed.  I certainly don't assert that whatever Chrome ships first should be
 regarded as standard.  Our rapid release schedule depends on platform
 features beginning life with a vendor prefix.  I believe that we goofed in
 this case by not vendor-prefixing the slice method.  More on this below...


 When we talked to Kenneth Arnold of Google about a typed array method that
 was more subtly misnamed slice, he quickly worked with other WebGL folks
 to rename it to subarray. That in spite of some support for the typed
 array slice method having shipped in some beta releases (at least; not
 sure if any final releases supported slice).

 s/Kenneth Arnold/Kenneth Russell/
 WebGL was disabled by default (hidden behind a flag) up until Chrome 9.  It
 was trivial to change the API from the point of view of Chrome prior to it
 coming out from behind a flag.


 The subtler differences between typed array's subarray (formerly named
 slice) and JS's Array slice:

 (a) The subarray method returns a view of mutable ArrayBuffer data, its
 elements alias the elements of the typed array from which it came. This is
 the big reason to avoid using slice.

 (b)  While subarray takes (start, end) parameters and allows end to be
 omitted; and it handles negative indexes; it does not allow start to be
 omitted too, whereas JS (after Python) does.

 http://www.khronos.org/registry/typedarray/specs/latest/

 http://www.khronos.org/webgl/public-mailing-list/archives/1102/msg00028.html

 The DOM is full of crazy inconsistency from insta-standards in the
 mid-'90s, starting with my DOM level 0 work. That is not a good pattern to
 repeat now that we have more balanced market actors and the benefits of
 hindsight. We should try to agree on less inconsistent APIs and prototype
 them, but no single prototype implementation can create a standard. Only
 interoperable de-facto standards can do that.

 At this point we need to be completely concrete, no leaning one way or
 another:

 0. Do nothing.

 1. Should we change slice, possibly breaking Chrome-only content that
 detects it by name but does not test its functionality?

 2. Should we rename slice, allowing Chrome to keep the old name for a
 while if it must?

 Solution 0 just leaves a noxious inconsistency on two counts: length not
 end as the second parameter, the lack of negative index support. Is there
 another difference with Array slice (which comes from Python and is in Ruby
 too)? What about optionality of the arguments?

 Doing 1 is simpler for future users who don't have to deal with the legacy
 that Chrome-only content might represent.

 Doing 2 avoids breaking any such legacy, but may increase the cognitive
 load for users who see the legacy and the new renamed method.

 What about negative indexes? Is there time to spec and implement these per
 Python/JS/Ruby?

 Without negative index support, the case for 2 is stronger again.

 If Blob slice returns an aliasing view onto mutable data instead of a
 copy, just as typed arrays' subarray method does, then the case for 2 is
 extremely strong.

 Blobs are views on immutable data.  WebKit's implementation will reject
 reads on a Blob, which points to a file that has since been modified.  (This
 should be part of the spec if it is not.)



 I do not see any mutator methods in
 http://dev.w3.org/2006/webapi/FileAPI/, but with the typed array precedent I
 didn't want to assume.

 I favor 1 if negative index and optional argument semantics can be made to
 match precedent, and assuming no way to mutate shared Blob/sub-Blob data.

 If not, I suggest 2, with subBlob or a better name -- that one hurts due
 to the shifted-B, but subblob seems also bad for the run-together bb.

 /be

 One option we can consider is to rename Blob.slice to Blob.webkitSlice and
 adopt the newer arguments.  I think we would want to similarly vendor-prefix
 BlobBuilder and FileReader as those too are non-standard.  This change runs
 just as much risk of breaking Chrome-specific content; however, the failure
 mode (absence of a method) should be easier for apps to cope with.
 I'm not stating that we will do this yet.  I'm just proposing it as another
 approach to consider.  I'll discuss this idea with others on the team
 tomorrow.
 By the way, I'm not too concerned about having to fix Google web apps.  I'm
 much more concerned about third-party, Chrome-specific content available as
 either apps or extensions through the Chrome app store.  It is hard to grep
 for instances of Blob.slice!!

Removing Blob.slice and replacing it with Blob.prefixSlice for 

Re: Server Sent Events vs Web Sockets?

[Ilkka Huotari]

This is a reply to an old message
(http://lists.w3.org/Archives/Public/public-webapps/2010AprJun/0087.html)
and I don't know if this goes to the right thread.

Would Server Sent Events be more or less efficient that Web Sockets?
Does one of these use less resources than the other? Or, are the (more
or less) the same at the lower level?

Thanks,
Ilkka

Re: [FileAPI] BlobBuilder.getBlob should clear the BlobBuilder

[Glenn Maynard]

On Wed, Apr 13, 2011 at 2:46 AM, Jonas Sicking jo...@sicking.cc wrote:

 Another advantage of dropping the memory automatically is that you
 don't need to copy any data into the Blob. Instead you can just make
 the Blob take ownership of whatever memory buffers you've built up
 during the various calls to .append. You could technically implement
 some sort of copy-on-write scheme, but that introduces complexity.


You don't actually need copy-on-write for that, so long as Blobs are
immutable.  You only need to refcount the underlying chunks comprising the
Blob, so the chunks can be shared between Blobs.

I think a complete, mature Blob implementation would be fairly complex,
anyway.  For example, in order to support very large blobs, Blob and
BlobBuilder might scratch to disk transparently.  (This wouldn't happen
during BlobBuilder.append, since that's synchronous.  Rather, it would
happen when other async APIs create Blobs, pushing the amount of memory used
by blobs over some threshold; that way the swapping of blobs to disk could
be done asynchronously.)

Since it appears Blob is becoming a major API building block for storing
larger blocks of data, I don't think this is unreasonable complexity to
expect in the longer term.

 The problem is that this optimizes for the rare case when you're
 creating several blobs which are prefixes of each other.

The above having been said, it's not necessary for BlobBuilder to keep its
data around in order to satisfy the uncommon blobs which are prefixes of
each other case.  You can do the following:

  var bb = new BlobBuilder();
  bb.append(lots_of_data);
  var blob1 = bb.getBlobAndReset(); // does what it says
  bb.append(blob1);
  bb.append(some more_data);
  var blob2 = bb.getBlobAndReset(); // returns lots_of_data + some_more_data

This could be optimized by the browser as above: the second, larger blob
could share data with the first.

-- 
Glenn Maynard

Re: More questions about contextual reference nodes

[Lachlan Hunt]

On 2011-04-13 04:43, Cameron McCormack wrote:

Lachlan Hunt:

This seems to differ from the algorithm given for T[], which
requires that the object be either an array host object or a native
object, which would not handle the JQuery case.  The sequenceT
type seems more generic than that as the algorithm seems to be able
to support any object with a length and indexed properties.


Why wouldn’t it handle the JQuery case?  Isn’t it a native object?


I didn't think so, but then perhaps I misunderstand what the spec means 
by a native object.  ECMAScript defines a native object as


  object in an ECMAScript implementation whose semantics are fully
   defined by this specification rather than by the host environment.

So that would include things like Array, String, etc.  I don't think 
NodeList fits that definition, since it's provided by the host 
environment, not ECMAScript.


I also thought something like this would not be a native object either:

  function init() {
...
  }
  var x = new init();

JQuery more or less does that to create an object with custom 
properties, and then basically uses Array.prototype.push.apply(x, elms) 
to add the collection to that object.



The issue of whether you want to throw eagerly if passed an array with
elements of the wrong type to an operation taking a sequenceT  is an
interesting one, which probably still could bear some more discussion.
On the one hand, it helps ensure implementations don’t diverge based on
when during the course of running the method these elements from the
sequence are looked up.  I could mean a lot of useless checks, though.


It's nice so that cases where authors try to pass in an array of things 
that aren't nodes are handled well.  The other alternative is just to 
accept a collection of anything, and iterate through it to find all the 
Element nodes, ignoring everything else.  I don't see any reason to 
prefer the latter approach.


--
Lachlan Hunt - Opera Software
http://lachy.id.au/
http://www.opera.com/

Re: More questions about contextual reference nodes

[Lachlan Hunt]

On 2011-04-13 06:32, Cameron McCormack wrote:

Lachlan Hunt:

OK. Then I'm not sure what the practical difference between the
Element[] or sequenceElement  would be then, nor which one to use.


... the only difference is that with Element[] you can distinguish
between null and an array of [length 0], while with sequenceElement  you
can’t.


I don't think it's necessary to distinguish between null and length 0 
here, since they both mean that there are no contextual reference elements.


However, with the way in which the IDL is overloaded, it's not clear to 
me which of the two overloaded methods gets invoked when the parameter 
is null.


The IDL says:
  querySelector(in DOMString selectors, in optional Element refElement);
  querySelector(in DOMString selectors, in sequenceNode refNodes);

When the author invokes:

  x.querySelector(a, null);

Does it invoke the method with refElement or refNodes?

(Although, in this case, it shouldn't matter, since the algorithm I 
wrote to determine contextual reference nodes ignores null in either case)



While i  n:

Let x be the result of calling [[Get]] on V with property name ToString(i).
Set Ei to be the result of converting x to an IDL value of type T.


This step can throw (as part of the conversion process defined for
converting to type T); seems like that would involve throwing from
the whole algorithm.


Yep.


Which conversion algorithm applies here?  Is it 4.1.15. object, or 
4.1.16. Interface types?


http://dev.w3.org/2006/webapi/WebIDL/#es-object

The former never throws, the latter will throw a TypeError. So am I 
right in assuming it's the latter?


--
Lachlan Hunt - Opera Software
http://lachy.id.au/
http://www.opera.com/

Re: [FileAPI] File.slice spec bug

[Brendan Eich]

On Apr 13, 2011, at 7:01 AM, Darin Fisher wrote:

 Agreed.  I certainly don't assert that whatever Chrome ships first should be 
 regarded as standard.  Our rapid release schedule depends on platform 
 features beginning life with a vendor prefix.  I believe that we goofed in 
 this case by not vendor-prefixing the slice method.  More on this below...

Vendor prefix may just add to the names developers have to juggle, and kick the 
can down the road a bit. If we can go from WD to REC with a single name and 
signature/semantics, even better.

Here is a case where hedging with a vendor prefix arguably would have helped, 
in hindsight. OTOH other HTML5 prototype implementations in the past many years 
did well to avoid vendor prefixes.


 s/Kenneth Arnold/Kenneth Russell/

Jetlag dredged up an old Unix name -- apols to Ken R.


 Blobs are views on immutable data.  WebKit's implementation will reject reads 
 on a Blob, which points to a file that has since been modified.  (This should 
 be part of the spec if it is not.)

This does seem slightly underspecified :-P.


 One option we can consider is to rename Blob.slice to Blob.webkitSlice and 
 adopt the newer arguments.  I think we would want to similarly vendor-prefix 
 BlobBuilder and FileReader as those too are non-standard.  This change runs 
 just as much risk of breaking Chrome-specific content; however, the failure 
 mode (absence of a method) should be easier for apps to cope with.
 
 I'm not stating that we will do this yet.  I'm just proposing it as another 
 approach to consider.  I'll discuss this idea with others on the team 
 tomorrow.

This sounds like a good way forward for the various parties.


 By the way, I'm not too concerned about having to fix Google web apps.  I'm 
 much more concerned about third-party, Chrome-specific content available as 
 either apps or extensions through the Chrome app store.  It is hard to grep 
 for instances of Blob.slice!!

Indeed. http://codesearch.google.com's regexps are not quite up to this -- need 
to mix in some http://doctorjs.org/ and make a semantic grep.

/be

CfC: WebApps testing process; deadline April 20

[Arthur Barstow]

I have updated WebApps' testing process documents to reflect comments 
submitted to the initial draft process [1]. As such, this is a Call for 
Consensus to agree to the testing process as described in:


http://www.w3.org/2008/webapps/wiki/Testing
http://www.w3.org/2008/webapps/wiki/Submission
http://www.w3.org/2008/webapps/wiki/Approval
http://www.w3.org/2008/webapps/wiki/Harness

Agreeing with this CfC, means you agree the WG should use this process 
going forward. However, since this is a new process, I think we also 
must recognize that: changes and tweaks may need to be made as we 
implement the process, common sense will ultimately prevail and we will 
use a consensus-based approach to resolving any issues that may arise.


If you have any comments or concerns about this proposal, please send 
them to public-webapps by April 20 at the latest.


As with all of our CfCs, positive response is preferred and encouraged 
and silence will be assumed to be agreement with the proposal.


-Art Barstow

[1]  http://lists.w3.org/Archives/Public/public-webapps/2011JanMar/1086.html

Please use public-webapps-testsuite for testing discussions

[Arthur Barstow]

The public-webapps-testsu...@w3.org list is now operational:

  http://lists.w3.org/Archives/Public/public-webapps-testsuite/

Please use this list for WebApps' testing discussions.

To subscribe to this list, send a subscribe e-mail to:

  public-webapps-testsuite-requ...@w3.org

For discussions about testharness.js, please use the 
public-test-in...@w3.org list:


  http://lists.w3.org/Archives/Public/public-test-infra/

-AB

Re: [FileAPI] File.slice spec bug

[Darin Fisher]

On Tue, Apr 12, 2011 at 11:50 PM, Jonas Sicking jo...@sicking.cc wrote:

 On Tue, Apr 12, 2011 at 10:01 PM, Darin Fisher da...@chromium.org wrote:
  On Tue, Apr 12, 2011 at 5:59 PM, Brendan Eich bren...@mozilla.org
 wrote:
 
  Darin's position is leaning toward not breaking compatibility with
 what
  Chrome has shipped for a while. That's one consideration. It can't be
 the
  only consideration, or there's no point having a discussion and whatever
  Chrome ships first is an instant standard.
 
  Agreed.  I certainly don't assert that whatever Chrome ships first should
 be
  regarded as standard.  Our rapid release schedule depends on platform
  features beginning life with a vendor prefix.  I believe that we goofed
 in
  this case by not vendor-prefixing the slice method.  More on this
 below...
 
 
  When we talked to Kenneth Arnold of Google about a typed array method
 that
  was more subtly misnamed slice, he quickly worked with other WebGL
 folks
  to rename it to subarray. That in spite of some support for the typed
  array slice method having shipped in some beta releases (at least; not
  sure if any final releases supported slice).
 
  s/Kenneth Arnold/Kenneth Russell/
  WebGL was disabled by default (hidden behind a flag) up until Chrome 9.
  It
  was trivial to change the API from the point of view of Chrome prior to
 it
  coming out from behind a flag.
 
 
  The subtler differences between typed array's subarray (formerly named
  slice) and JS's Array slice:
 
  (a) The subarray method returns a view of mutable ArrayBuffer data, its
  elements alias the elements of the typed array from which it came. This
 is
  the big reason to avoid using slice.
 
  (b)  While subarray takes (start, end) parameters and allows end to be
  omitted; and it handles negative indexes; it does not allow start to be
  omitted too, whereas JS (after Python) does.
 
  http://www.khronos.org/registry/typedarray/specs/latest/
 
 
 http://www.khronos.org/webgl/public-mailing-list/archives/1102/msg00028.html
 
  The DOM is full of crazy inconsistency from insta-standards in the
  mid-'90s, starting with my DOM level 0 work. That is not a good pattern
 to
  repeat now that we have more balanced market actors and the benefits of
  hindsight. We should try to agree on less inconsistent APIs and
 prototype
  them, but no single prototype implementation can create a standard. Only
  interoperable de-facto standards can do that.
 
  At this point we need to be completely concrete, no leaning one way or
  another:
 
  0. Do nothing.
 
  1. Should we change slice, possibly breaking Chrome-only content that
  detects it by name but does not test its functionality?
 
  2. Should we rename slice, allowing Chrome to keep the old name for a
  while if it must?
 
  Solution 0 just leaves a noxious inconsistency on two counts: length not
  end as the second parameter, the lack of negative index support. Is
 there
  another difference with Array slice (which comes from Python and is in
 Ruby
  too)? What about optionality of the arguments?
 
  Doing 1 is simpler for future users who don't have to deal with the
 legacy
  that Chrome-only content might represent.
 
  Doing 2 avoids breaking any such legacy, but may increase the cognitive
  load for users who see the legacy and the new renamed method.
 
  What about negative indexes? Is there time to spec and implement these
 per
  Python/JS/Ruby?
 
  Without negative index support, the case for 2 is stronger again.
 
  If Blob slice returns an aliasing view onto mutable data instead of a
  copy, just as typed arrays' subarray method does, then the case for 2 is
  extremely strong.
 
  Blobs are views on immutable data.  WebKit's implementation will reject
  reads on a Blob, which points to a file that has since been modified.
  (This
  should be part of the spec if it is not.)
 
 
 
  I do not see any mutator methods in
  http://dev.w3.org/2006/webapi/FileAPI/, but with the typed array
 precedent I
  didn't want to assume.
 
  I favor 1 if negative index and optional argument semantics can be made
 to
  match precedent, and assuming no way to mutate shared Blob/sub-Blob
 data.
 
  If not, I suggest 2, with subBlob or a better name -- that one hurts
 due
  to the shifted-B, but subblob seems also bad for the run-together
 bb.
 
  /be
 
  One option we can consider is to rename Blob.slice to Blob.webkitSlice
 and
  adopt the newer arguments.  I think we would want to similarly
 vendor-prefix
  BlobBuilder and FileReader as those too are non-standard.  This change
 runs
  just as much risk of breaking Chrome-specific content; however, the
 failure
  mode (absence of a method) should be easier for apps to cope with.
  I'm not stating that we will do this yet.  I'm just proposing it as
 another
  approach to consider.  I'll discuss this idea with others on the team
  tomorrow.
  By the way, I'm not too concerned about having to fix Google web apps.
  I'm
  much more concerned about third-party,