RE: CFC: Publish a FPWD of IndexedDB 2.0

[Ali Alabbas]

+1

> On Tue, Aug 09, 2016 at 08:07:10, Alexander Schmitz wrote:
> 
> +1
> 
> Alexander Schmitz
> jQuery Foundation
> 
> On Tue, Aug 9, 2016 at 8:45 AM, Dylan Barrell   > wrote:
> 
> 
>   +1
> 
>   On Tue, Aug 9, 2016 at 5:06 AM, Léonie Watson   > wrote:
> 
> 
>   Quick reminder that this CFC closes at the end of day
> tomorrow (Wednesday 10th August). Thanks.
> 
>   Léonie.
> 
>   --
>   @LeonieWatson tink.uk   Carpe diem
> 
> 
>   On 03/08/2016 15:46, Léonie Watson wrote:
> 
> 
>   Hello WP,
> 
>   This is a Call For Consensus (CFC) to publish a First
> Public Working
>   Draft (FPWD) of IndexedDB 2.0 [1].
> 
>   We are still exploring different ways of responding to
> a CFC. Please
>   choose one of the following methods:
> 
>   1. Reply by email to this thread (on
>   public-webapps@w3.org
>  ).
> 
>   2. Reply or +1 on Github:
>   https://github.com/w3c/IndexedDB/issues/84
> 
> 
>   There is no need to use more than one method.
> The WP chairs will collate
>   the results across all channels.
> 
>   Please respond by end of day on Wednesday 10th
> August. Positive
>   responses are encouraged, but silence will be taken
> as consent with the
>   proposal.
> 
>   Thanks
>   Léonie on behalf of the WP chairs and team
>   [1]
>   http://w3c.github.io/IndexedDB/
> 
> 
> 
> 
> 
> 
> 
>   --
> 
>   Download the aXe browser extension for free:
> 
> 
>   Firefox:
> https://addons.mozilla.org/en-US/firefox/addon/axe-devtools
> 
>   Chrome:
> https://chrome.google.com/webstore/detail/axe/lhdoppojpmngadmnindne
> jefpo
> kejbdd?hl=en-US
>  ejefp
> okejbdd?hl=en-US>
> 
> 
>   Life is ten percent what happens to you and ninety percent how you
> respond to it. - Lou Holtz
> 
> 
> 

Re: [service worker] f2f meeting notes, next meeting details

[Ali Alabbas]

Hello, here are the day 2 meeting notes from the service worker F2F in 
Redmond,. The attendees are CC'd if there are any corrections that need to be 
made or if anyone has any questions or concerns that they would like to raise.

=

Service Worker F2F
Redmond, WA
April 12, 2016

Day 2 Notes

Attendees:
-
 * Microsoft - Ali Alabbas, Jatinder Mann, Adrian Bateman, Travis Leithead, 
Todd Reifsteck
 * Google - Joshua Bell, Matt Falkenhagen, Jake Archibald, Alex Russell, Marijn 
Kruisselbrink
 * Mozilla - Ehsan Akhgari, Ben Kelly, Andrew Sutherland (remote)
 * Samsung - Jungkee Song
 * Apple - Ted O'Connor


Agenda & Notes:
-
 * Foreign fetch architecture (#878: Foreign fetch vs non-credentialed requests)
  * Reduce state on the serial object
  * Is there a way to fetch something locally that will always be visible?
   * Yes, if there's no origin, it will be an opaque response
  * New CORS header that the destination needs to support?
   * alice.com (A) foreign fetch to bob.com (B), response from B to A: 
that's a cross-origin request
  * Want to reuse CORS infrastructure as much as possible
  * Must create a new Response with the CORS headers
  * SW is like B's server
  * Response always creates a copy of the response and then modifies it
  * Mechanisms should be the same in that the service worker
  * Should we loop the request back through CORS or should we just make it 
visible for the origin?
  * Adding a new step with what the service worker responds with (adding 
extra step in the network layer)
  * Current security model is that A can do whatever it wants to A
   * A has a security boundary (SOP) since it's in a different origin
  * Constraining foreign fetch would defeat the purpose
  * A can only see what credentials that are exposed through scripts (which 
are none today)
  * We shouldn't allow transparent smuggling of data
   * A requesting data from C is not allowed
   * A requests data from B which then requests from C and passes that 
data on will have the same headers as C unless B creates a new response with 
different headers
  * CORS does not fit model
  * Why make foreign fetch more complicated than postMessage in terms of 
how origin B lets origin A have access to bytes?
   * postMessage doesn't allow you to send opaque bytes
  * Response time is when the CORS check occurs
   * After the network request goes out is when the check occurs
  * B can do whatever it wants to B and expose it to A
  * We should minimize the number of policy security mechanisms so that we 
do not have to service different policies
  * A can make a request that initiates a foreign fetch even and 
intercepted by B's service worker without having known and potentially with 
no-credentials
  * We don't need preflight because it requires that you are going to the 
network and SWs don't necessarily require going to the network
  * Fonts and analytics are the biggest use case for foreign fetch at the 
moment
  * Opaque requests can allow no-credentialed request
   * It would create some issues with maintainability
   * Would prefer to have some sort of partitioning instead
  * New issue created: #878: Foreign fetch vs non-credentialed requests
   * Summary from issue: As currently specified foreign fetch doesn't 
play together well with non-credentialed requests. In a world without foreign 
fetch, a website on origin A can fetch something from origin B without 
credentials, and there is no way (modulo fingerprinting) for B to associate 
that request with its cookies. If B can intercept the request with foreign 
fetch this is no longer the case as B itself was fetched with credentials, and 
can make credentialed requests and access data that was cached using 
credentials. There are a few options here:
   * Live with this: non-credentialed requests aren't an important 
security/privacy boundary since B can use fingerprinting etc. anyway to track 
things and associate requests with credentials.
   * Completely disable foreign fetch interception of non-credentialed 
requests. This would be very unfortunate as things like fonts are always 
fetched without credentials, so this would disable a major use case of foreign 
fetch.
   * Require origin A to opt in to having its non-credentialed requests 
be intercepted by a foreign fetch service worker. This would be rather 
unfortunate as well, as it would largely get rid of the transparent nature of 
foreign fetch, making it much less powerful.
   * Somehow make it possible for origin B to say "this service worker 
does not need credentials". In the case of something like the fonts use case 
this would be done by adding an extra flag to the Link: header that installs 
the

Re: [service worker] f2f meeting notes, next meeting details

[Ali Alabbas]

Hello, here are the day 1 meeting notes from the service worker F2F. The 
attendees are CC'd if there are any corrections that need to be made or if 
anyone has any questions or concerns that they would like to raise.

=

Service Worker F2F
April 11, 2016


Attendees:
-
   * Microsoft - Ali Alabbas, Jatinder Mann, Adrian Bateman, Travis Leithead
   * Google - Joshua Bell, Matt Falkenhagen, Jake Archibald, Alex Russell, 
Marijn Kruisselbrink
   * Mozilla - Ehsan Akhgari, Ben Kelly
   * Samsung - Jungkee Song
   * Apple - Ted O'Connor


Agenda:
-
   * Vendor comments on general direction
   * v1 issues
 

Vendor comments on general direction (e.g. Tim’s comment)
-
   * Service workers are overly complicated for simple use cases
   * Ted O’Connor: don't worry about Tim's comment
   * Matt Falkenhagen: Average time for starting up SW when a fetch is 
registered is 150 ms
 

v1 Issues:
-
   * #861: be more explicit that jobs always run asynchronously
  * Already implemented this way in Gecko, but would be nice to have it be 
part of the spec
  * Spec originally allowed register to run sync, but need to be consistent
  * Don't want to spawn new jobs and have jobs running in parallel (we want 
to have one job queue)
  * Resolution: update spec language to ensure that all jobs are scheduled 
asynchronously
   * #850: FetchEvent.respondWith does something weird with the body of a 
response
  * Streams need to be figured out (pipeTo algorithm)
  * In the meantime, some steps have been put in place until everything is 
spec'd in Fetch
  * Consuming body input to the API so that scripts can't read after that
  * Comment from Yushino from Google who was working on Streams: pipeTo 
transfers the object via teeing
  * We probably don't want to tee because we want to consume data
  * Outcome: wait and see then change the spec with what is required to 
accommodate changes from Fetch API
   * #848: "Wait for all the tasks queued by Update State" language is 
problematic
   * #851: Install algorithm step 14 should clear waiting worker before 
updating state to redundant
   * #860: spec should queue tasks to expose attribute changes on ServiceWorker 
and ServiceWorkerRegistration
  * Queuing task per execution environment
  * Expectation is that the attributes are in a certain that state
  * There may always be a race between getters
  * Fully deterministic way for getters in this fashion is via event 
listeners
  * Bug in Gecko: registration.installing and activating currently returns 
null
  * Outcome: queue a task to update these values per context, when the 
state changes.
 * 
https://slightlyoff.github.io/ServiceWorker/spec/service_worker/#update-state-algorithm
 - this should queue a task to update the serviceWorker.state objects, and the 
registration.waiting (etc) objects.
 * In the updatefound listener, reg.installing should always be the new 
worker, even if there's no install event.
 * In the statechange listener, the registration should be updated.
   * #816: ExtendableMessageEvent.source cannot be SameObject
  * Outcome: update spec reflecting IDL changes
  * Related issue filed in IDL - no update now, will ping
   * #810: MessagePort[] no longer valid in WebIDL
  * Resolution: spec needs to be updated
   * #870: Inconsistencies due to when clients are created
  * When opening a new tab, Chrome (in initial fetch event) there will be a 
client there and in Mozilla there isn't
  * Potential client ID can be used to provide information about what the 
client is if it's not created yet
  * Want to have a rich Fetch event that can include if it was a soft/hard 
reload
  * Once it's in the history and doesn't show up in client lists, the 
document shouldn't exist anymore
  * No client is created for content-disposition downloads
 * This is a reason for just using potential client ID rather than 
re-using the client ID
  * What do you want to do with a client before it's created?
 * Developer may want to cater the content to fill in the client based 
on the window size
  * Do we create a client before the request goes through even if the 
client does not end up being used?
  * Don't update service worker if there is a reload of a page since there 
is no moment when the client is disengaged from being controlled by the service 
worker
  * For window.open, about:blank inherits the creator (i.e., it is 
controlled)
  * We are already committed to having a potential client if we have a 
potential client ID, so we should be able to introspect that client in the 
clients list
  * about:blank is the url of the reserved (potential) client
  * Resolution: have client ID and potential client ID, clients.matchAll 
will not 

RE: [service worker] f2f meeting notes, next meeting details

[Ali Alabbas]

Here are notes I captured during the meeting:


Service Workers Face-to-Face Meeting Notes

Date: January 26, 2016
Place: Mozilla's San Francisco office

Attendees
--
 * Microsoft - Ali Alabbas, Jatinder Mann
 * Google - Jake Archibald, Alex Russell, Ilya Grigorik, Marijn Kruisselbrink
 * Mozilla - Anne van Kesteren, Benjamin Kelly
 * Apple - Edward O'Connor
 * Samsung - Jungkee Song
 * Conrad Irwin


Agenda
--
 * Discuss outstanding version 1 spec issues from GitHub 
 * Cache API enhancements, especially transactions and making addAll more useful
 * Foreign fetch
 * Navigation handling


Spec Issues
---
 #814: should subresource FetchEvents trigger an update if 24-hours since last 
check?
 * Resolved: FetchEvents should be treated like other ExtendableEvents with 
the 24-hour check update
 #813: Should mixed content always be blocked?
 * Resolved: allow passthrough of mixed content and adding it to the cache. 
We should also look at persisting mixed content warnings until storage is 
cleared.
 #808: What is FetchEvent.clientId for navigation requests
 * Resolved: pages are explicitly closed by user action so it makes sense 
to have it be null. Also, on navigation or close the client should not be 
gettable.
 #800: should update() during top level script evaluation be ignored?
 * Resolved: single job queue for register(), update() and soft update (as 
they are classified as equivalent jobs). So, update() from the installing 
worker of a new registration is rejected with InvalidStateError
 * Benjamin Kelly will review to make sure this is no longer an issue
 #788: should update on navigation or the subsequent updatefound event be 
delayed until document DOM is loaded?
 * Resolved: add a note to the spec to delay triggering of an update until 
after DOMContentLoaded of the associated page. If it’s a worker, don’t update 
until after the worker script has run.
 * Specifically, Chrome's implementation tries to schedule the updates 
after the worker stops
  * Schedule an update 1 second after page load for a navigation in a 
SW's scope.
  * Whenever the active worker stops, if it hasn't attempted an update 
in 24 hours, and no update is scheduled, schedule an update.
  * If the active worker has been running without stopping for 5 
minutes, if hasn't attempted an update in 24 hours, and no update is scheduled, 
schedule an update.
 #787: what should the document base URI for an intercepted navigation
 * Requires further investigation. Jake to write a blog post to ask 
developers if we can safely make this change – showing both CSS and the 
document and asking where links and the background images would go.
 #783: Race when registering multiple service workers for the same scope
 * Resolved: having a single job queue where register/update/unregister 
jobs are scheduled in FIFO order. Same job is aggregated when scheduled. Allow 
the installing worker finish installing or eventually timeout, allowing the 
next SW in queue
 * Timeouts in Blink: 30 seconds for executing sync JS (e.g. while(true)), 
five minutes for any event - e.g. event.waitUntil(new Promise())
 #771: Allow waitUntil to be called multiple times, async
 * Resolved: it would be useful to support this especially when a promise 
doesn’t resolve before returning to the event loop since waitUntil needs to be 
called before returning to the event loop - requires specifying
 #757: passthrough service worker can break sites using redirected stylesheets
 * Synthetic responses from service workers - 
e.respondWith(fetch(e.request)) - will assume the request url (i.e., when 
there's no response url). Thus, navigation will not be affected since it 
doesn't look at the response url, but stylesheets will be fixed and other 
workers will get the same behavior.
 #737: Make the Cache API deal with request's redirect mode not being "follow"
 * Requires further investigation
  * We should be able to store opaque redirects when redirect mode is 
not “follow” (not return an error) so that the original site doesn’t need to go 
to the network and can instead pass on to another service worker
  * E.g. cache.add(redirectingNavUrl, { mode: 'same-origin', redirect: 
'manual' }) stores an opaque redirect
 #728: When does the openWindow promise resolve?
 * Resolved with the following:
  * We should buffer messages until startMessages is invoked
   * Page onload invokes startMessages
   * Assignment to navigator.serviceWorker.onmessage invokes 
startMessages
   * Invocation of navigator.serviceWorker.startMessages() invokes 
startMessages
  * No change on when openWindow resolves
 #718: Proposal: Optimized No-Fetch Service Workers
 * Resolved: check which event listeners have been registered via 
addEventListener and optimize based on that - additionally, addEventListener 
should thro

Directory Upload

[Ali Alabbas]

Hello WebApps WG and Incubator CG members,

As you may know, we (Microsoft) have been collaborating with Mozilla on 
evolving the new directory upload proposal [1]. It has recently been added to 
the Incubator Community Group and we are looking forward to have everyone get 
involved with providing feedback on this initial proposal. If you haven't 
already made a first-pass read of the spec, I invite you to take some time to 
do that as it is a relatively short document that we are trying to get some 
more eyes on.

As we wait for the spec to stabilize, and to solve the existing interop gap 
with Chrome with regards to directory uploads, we are implementing the 
webkitRelativePath property for the File interface and webkitdirectory 
attribute for the input tag [2]. This allows sites to show a directory picker 
and to identify the relative file structure of the directory a user selects.

Supporting webkit-prefixed properties is not an endorsement of the old way of 
doing it - it is an interop realization. For this reason, we will consider the 
webkit-prefixed API as deprecated in Microsoft Edge (as we do with other 
webkit-prefixed APIs we support for compatibility). The old API is synchronous 
and doesn't provide a natural way of traversing directories. That is why we are 
working closely with Mozilla and encouraging everyone in the community to look 
into the directory upload proposal and to provide feedback.

Thank you,
Ali 

[1] https://wicg.github.io/directory-upload/proposal.html
[2] https://dev.modern.ie/platform/status/webkitdirectoryandwebkitrelativepath
[3] https://dev.modern.ie/platform/status/directoryupload/

RE: Directory Upload Proposal

[Ali Alabbas]

Thank you for the feedback Jonas. After speaking to the OneDrive team at 
Microsoft, they let me know that their use case for this would involve hiding 
the file input and just spoofing a click to invoke the file/folder picker. This 
makes me believe that perhaps there should be less of an emphasis on UI and 
more on providing the functionality that developers need. On that note, there 
is actually a 5th option that we can entertain. We could have three different 
kinds of file inputs: one type for files, another for directories, and yet 
another for handling both files and directories. This means that if a developer 
has a use case for only wanting a user to pick a directory (even on Mac OS X), 
they would have that option. This would also future-proof for Windows/Linux if 
they ever introduce a file and folder picker in the future. This kind of 
solution would be more additive in nature which would mitigate any heavy 
changes that could run the risk of breaking sites.

What do you think about this option?

Thanks,
Ali

On Friday, May 8, 2015 at 3:29 PM, Jonas Sicking jo...@sicking.cc wrote:

On Tue, May 5, 2015 at 10:50 AM, Ali Alabbas a...@microsoft.com wrote:
 I recommend that we change the dir attribute to directories and keep 
 directory the same as it is now to avoid clashing with the existing dir 
 attribute on the HTMLInputElement. All in favor?

There's no current directory attribute, and the current dir
attribute stands for direction and not directory, so I'm not sure which 
clash you are worried about?

But I'm definitely fine with directories. I've used that in the examples 
below.

 As for the behavior of setting the directories attribute on a file input, 
 we have the following options:

 1) Expose two buttons in a file input (choose files and choose 
 directory) (see Mozilla's proposal [1])
 - To activate the choose directory behavior of invoking the 
 directory picker there would need to be a method on the 
 HTMLInputElement e.g. chooseDirectory()
 - To activate the choose files behavior of invoking the files 
 picker, we continue to use click() on the file input

 2) Expose two buttons in file input for Windows/Linux (choose files 
 and choose directory) and one button for Mac OS X (choose files and 
 directories)
 - Allows users of Mac OS X to use its unified File/Directory picker
 - Allows users of Windows/Linux to specify if they want to pick files 
 or a directory
 - However, there would still need to be a method to activate the 
 choose directory button's behavior of invoking the directory picker 
 (e.g. chooseDirectory() on the HTMLInputElement)
 - This results in two different experiences depending on the OS

 3) Expose only one button; Windows/Linux (choose directory) and Mac 
 OS X (choose files and directories)
 - Allows users of Mac OS X to use its unified File/Directory picker
 - Windows/Linux users are only able to select a directory
 - click() is used to activate these default behaviors (no need for an 
 extra method such as chooseDirectory() on the HTMLInputElement 
 interface)
 - For Windows/Linux, in order to have access to a file picker, 
 app/site developers would need to create another file input without 
 setting the directories attribute
 - Can have something like isFileAndDirectorySupported so that 
 developers can feature detect and decide if they need to have two 
 different file inputs for their app/site (on Windows/Linux) or if they 
 just need one (on Mac OS X) that can allow both files and directories

 4) Expose only one button (choose directory)
 - User must select a directory regardless of OS or browser (this 
 normalizes the user experience and the developer design paradigm)
 - To make users pick files rather than directories, the developer 
 simply does not set the directories attribute which would show the 
 default file input
 - Developers that want to allow users the option to select directory 
 or files need to provide two different inputs regardless of OS or 
 browser

Hi Ali,

I think the only really strong requirement that I have is that I'd like to 
enable the platform widget on OSX which allows picking a file or a directory. 
This might also be useful in the future on other platforms if they grow such a 
widget.

I understand that this will mean extra work on the part of the developer, 
especially in the case when the developer render their own UI. However authors 
generally tend to prefer to optimize a good UI experience over saving a few 
lines of code. This seems especially true in this case given that the author 
has chosen to provide their own UI rather than use the browser-provided one.

So that leaves us with options 2 and 3.

I think both of these are options that I can live with. And for authors that 
render their own UI the difference is only one of syntax.
And most likely authors will wrap our API in a library since none of the APIs 
here are particularly nice.

However I do think that 3 has some disadvantages for authors that *do

RE: Directory Upload Proposal

[Ali Alabbas]

I recommend that we change the dir attribute to directories and keep 
directory the same as it is now to avoid clashing with the existing dir 
attribute on the HTMLInputElement. All in favor?

As for the behavior of setting the directories attribute on a file input, we 
have the following options:

1) Expose two buttons in a file input (choose files and choose directory) 
(see Mozilla's proposal [1])
- To activate the choose directory behavior of invoking the directory picker 
there would need to be a method on the HTMLInputElement e.g. chooseDirectory()
- To activate the choose files behavior of invoking the files picker, we 
continue to use click() on the file input

2) Expose two buttons in file input for Windows/Linux (choose files and 
choose directory) and one button for Mac OS X (choose files and directories)
- Allows users of Mac OS X to use its unified File/Directory picker
- Allows users of Windows/Linux to specify if they want to pick files or a 
directory
- However, there would still need to be a method to activate the choose 
directory button's behavior of invoking the directory picker (e.g. 
chooseDirectory() on the HTMLInputElement)
- This results in two different experiences depending on the OS

3) Expose only one button; Windows/Linux (choose directory) and Mac OS X 
(choose files and directories)
- Allows users of Mac OS X to use its unified File/Directory picker
- Windows/Linux users are only able to select a directory
- click() is used to activate these default behaviors (no need for an extra 
method such as chooseDirectory() on the HTMLInputElement interface)
- For Windows/Linux, in order to have access to a file picker, app/site 
developers would need to create another file input without setting the 
directories attribute 
- Can have something like isFileAndDirectorySupported so that developers can 
feature detect and decide if they need to have two different file inputs for 
their app/site (on Windows/Linux) or if they just need one (on Mac OS X) that 
can allow both files and directories

4) Expose only one button (choose directory)
- User must select a directory regardless of OS or browser (this normalizes the 
user experience and the developer design paradigm)
- To make users pick files rather than directories, the developer simply does 
not set the directories attribute which would show the default file input
- Developers that want to allow users the option to select directory or files 
need to provide two different inputs regardless of OS or browser

I am leaning towards option 3 or 4 since it allows us to avoid needing an extra 
method such as chooseDirectory() but also allows the developer to cater the 
experience based on the capabilities of the OS via feature detection as is the 
case with option 3. It also allows us to keep the current look and feel of the 
file inputs meaning we don't break current sites nor do we create an experience 
that puts backwards compatibility at risk by outright changing the behavior of 
the file input.

Thanks,
Ali

[1] https://bugzilla.mozilla.org/show_bug.cgi?id=846931

RE: Directory Upload Proposal

[Ali Alabbas]

On Tue, Apr 28, 2015 at 4:15 PM, Tab Atkins Jr. jackalm...@gmail.com wrote:

On Tue, Apr 28, 2015 at 3:53 PM, Ryan Seddon seddon.r...@gmail.com wrote:
 To enable developers to build future interoperable solutions, we've 
 drafted a proposal [4], with the helpful feedback of Mozilla and 
 Google, that focuses strictly on providing the mechanisms necessary 
 to enable directory uploads.

 The use of the dir attribute seems odd since I can already apply dir=rtl
 to an input to change the text direction.

Good catch; that's a fatal naming clash, and needs to be corrected.
The obvious one is to just expand out the name to directory.

~TJ

Aaron opened an issue for this on GitHub [1] and I agree that it is a problem 
and we should definitely rename it to something else! One option might be to 
change dir to directory, but we would need a different name for directory (the 
attribute that gets back the virtual root holding the selected files and 
folders).

[1] https://github.com/InternetExplorer/directory-upload/issues/1

Directory Upload Proposal

[Ali Alabbas]

Hello WebApps Group,

For the past few years sites have been enabling directory uploads by using the 
old FileSystem API [1] in Chrome and Opera. In the April 2014 face to face 
WebApps group meeting [2], there was a CfC to move the old specs into a WG Note 
and to continue working on the new FileSystem API [3] instead, effectively 
discontinuing work on the old FileSystem API.

The new FileSystem API tries to satisfy a large number of use cases with an 
approach that is better than the old API. However, its broad scope and its 
dependencies on future capabilities (e.g. Observables) has slowed down the pace 
at which it is moving forward. Due to this, there is currently no interoperable 
way to enable directory uploads on the web. We have received lots of feedback 
from developers - including some working on Microsoft services such as OneDrive 
- that supporting directory upload is a high priority for them. 

To enable developers to build future interoperable solutions, we've drafted a 
proposal [4], with the helpful feedback of Mozilla and Google, that focuses 
strictly on providing the mechanisms necessary to enable directory uploads. The 
proposal uses a subset of the new FileSystem API so that it can be expanded in 
the future to enable more scenarios when the new FileSystem API is further 
along. To help understand the proposal (especially the APIs involved) I have 
written a polyfill [5] and demo page [6] that is built on top of Chrome's old 
FileSystem API.

If there is sufficient interest, I would like to work on this within the scope 
of the WebApps working group.

Thank you,
Ali

[1] Old FileSystem API: http://www.w3.org/TR/file-system-api/
[2] WebApps April '14 Meeting: 
http://www.w3.org/2014/04/10-webapps-minutes.html#item02
[3] New FileSystem API: http://w3c.github.io/filesystem-api/
[4] Proposal: http://internetexplorer.github.io/directory-upload/proposal.html
[5] Polyfill: 
https://github.com/InternetExplorer/directory-upload/blob/gh-pages/polyfill.js
[6] Demo: http://internetexplorer.github.io/directory-upload/index.html

RE: flush() | was Re: FileSystem API Comments

[Ali Alabbas]

Hi Arun,

We believe that the flush property should be specified when getting the file 
handle as in option 1. One benefit of this is that it will enable the buffer of 
both reads and writes for the same handle. On the other hand, if we specify it 
on every write operation (as in option 2) we could run into inconsistencies 
when invoking the write method with and without the flag.

Based on our interpretation of option 1, it seems as though the flush() 
function would not be available. This is how we perceived its usage would look 
like:

navigator.getFileSystem().then(function(root) {
return root.openWrite(path/to/file.txt, {autoFlushing: true});
}).then(function(fileHandle) {
fileHandle.write(blob);  // data is written to the system cache and is 
flushed to disk without delay
});

Thank you,
Ali


From: Arun Ranganathan [mailto:a...@mozilla.com] 
Sent: Friday, October 31, 2014 11:19 AM
To: Ali Alabbas
Cc: Web Applications Working Group WG
Subject: flush() | was Re: FileSystem API Comments

Greetings Ali!

I've been thinking about the discussion of flush(), and would like to see if I 
can make my previous statement a bit more nuanced. It turns out that flush() 
(in the vein of fsync/sync) is pretty useful, and after discussion with a few 
folks within Mozilla, I realize that it isn't as simple as tacking it on to the 
write-family of Promises - as you point out, it is a potentially expensive 
operation.

Something like a flush feature might help the following use cases:

1. Creating a database technology on top of the filesystem technology. This 
might include IndexedDB, but also WebSQL (as a hypothetical example). Most 
transactional operations like this need the ability to do something like flush.

2. Then, there's the use case of compiling C++ codebases to JS. Well-known 
examples of this are games, leveraging asm.js. In this genre of use case, 
sometimes a large database is brought over (e.g. sqlite). It could be memory 
backed, but it is a definite bonus if it could be filesystem backed. Something 
like flush helps make that a possibility.

Now the question is how to do this in a WebAPI, allowing for the power along 
with the mitigations that a web app might need, notably for performance? A few 
ideas below:

On Oct 21, 2014, at 4:36 PM, Ali Alabbas a...@microsoft.com wrote:


 * flush()
 - This is costly functionality to expose and is likely to be overused by 
callers. It would be beneficial to automatically flush changes to disk by 
allowing the default file write behavior by the OS. For example, on Windows, we 
would leave it up to the filesystem cache to determine the best time to flush 
to disk. This is non-deterministic from the app's point of view, but the only 
time it is a potential problem is when there's a hard power-off. Most apps 
should not be concerned with this; only apps that have very high data 
reliability requirements would need the granular control of flushing to disk. 
In those cases a developer should use IndexedDB. So we should consider 
obscuring this functionality since it's not a common requirement and has a 
performance impact if it's widely used.


I agree with the idea of obscuring the functionality a bit, especially given 
that it might not be necessary for a large class of operations. A few ways to 
do that:

1. Add this to a dictionary option when coining the FileHandleWritable from the 
Directory (e.g. add it to something like the OpeWriteOptions: 
http://w3c.github.io/filesystem-api/Overview.html#widl-Directory-openWrite-Promise-FileHandleWritable--DOMString-File-path-OpenWriteOptions-options).

This way, the developer has the ability to coin a more expensive promise, 
if that particular set of write operations needs this feature.

2. Add this to the set of options on the FileHandleWritable.

This could be by dictionary, again. Or, it could be a boolean on the 
FileHandleWritable's write(). This latter might not be specific enough. Like 
other implementations, ours is not going to buffer anything, but rely on the 
underlying operating system's buffer for writes and reads.

3. Stick with the idea of a method, like flush(). In this case, we might have 
to caveat the use of this, since the possibility of inexperienced developer 
misuse is high :-) It might help to see if we can determine some boundaries on 
this.

Any feedback on some of these options would be valuable. I am thinking of 1. 
and 2.

- A*

FileSystem API Comments

[Ali Alabbas]

Hello,

I'm with the IE Platform team at Microsoft. We have a few comments on the 
latest editor's draft of the newly proposed FileSystem API [1].

1.1 Use cases (3. Audio/Photo editor with offline access or local cache for 
speed)

  * Edited files should be accessible by other client-side applications
 - Having the sandboxed file system share its contents between all apps 
would allow apps to tamper with the files of another app. This could result in 
corrupted files and perhaps an invalid state for some apps that expect certain 
contents to exist in a file. This makes us wonder: should we warn users about 
files that are being opened and written to? If an app is just doing a read, can 
it open a file or directory without the user's permission, or could this pose a 
possible issue as well? Also, is the Quota Management API going to be a 
dependency? It's unclear what we would do with regards to requesting permission 
to access files. Will this spec be responsible for defining what 
questions/permission inquiries are presented and when they are presented to the 
user? For example, what happens when one file is locked for use by a different 
application? Is the user notified and given the option to open a read-only copy 
of that file?


3. The Directory Interface

  * Change events
 - I would like to revisit the discussion on apps getting notifications of 
changes to files/directories. There are many scenarios where an application 
would want to react to renames/moves of a file/directory. There would also be 
value in being notified of a change to a directory's structure. If an app has a 
file browser that allows a user to select files and/or directories and another 
app makes changes to the sandboxed filesystem, then it would be expected that 
the first app should be notified and would be able to refresh its directory 
tree. Otherwise it would require the user to somehow force a refresh which 
would not be a good user experience since the user would expect the file 
browser to update on its own.

  * removeDeep()  move()
 - Do these support links or junctions? If not, what is the expected 
behavior?

  * enumerate()
 - It would be useful to have pre-filtering support for the following: 
file/directory, ranges, wildcard search. Currently we would be forced to 
enumerate the entire set and manually filter out the items we want from the 
result set.
 - Callers often know exactly whether or not they want to enumerate 
files or folders. For example, an image upload service may only be interested 
in the files present in a directory rather than all of its directories. Perhaps 
it would be useful to have enumerateFiles() and enumerateDirectories() for this 
purpose? Or we could have another argument for enumerate() that is an enum 
(directory, file).
 - Supporting optimized pagination of large directories. We could have 
arguments for a starting index and length we would be able to specify a range 
of items to retrieve within the result set.
 - Supporting the wildcard character to pre-filter a list of 
files/directories (e.g. *.jpg).

4. The FileHandle Interface

  * FileHandles
 - Is this basically going to be the first to get the handle gets to use it 
and all subsequent calls need to wait for the file handle to become available 
again? Are there more details about the locking model used here?

  * Auto-closing of FileHandles
 - This may cause confusion as it does not match the common developer 
mental model of a file handle which is opened and then available for use 
until it's closed. Perhaps it would be advantageous to have an explicit close 
function as part of the FileHandle interface? With the current behavior there 
can be overhead with the unintended closure of the FileHandle that would 
require a developer to continuously open/close a FileHandle. The currently 
defined behavior assumes that a developer is done with all their file 
manipulations when they have completed a promise chain. However, a developer 
may want to keep the FileHandle open to be used elsewhere at some other point 
in time that is not related to the current promise chain. An example of the 
usefulness of having an explicit close function is if you were to implement a 
word processor and wanted to lock down the file that it currently has open for 
the period of its editing. This way you are free to continue operating on that 
file for the duration that it is open, protecting the file from other 
processes, and not having to undergo the costly setup and teardown of a file 
handle.

  * AbortableProgressPromise
 - It is not clear how a developer would define the abort callback of an 
AbortableProgressPromise. It seems that the user agent would be responsible for 
setting the abort callback since it instantiates and returns the 
AbortableProgressPromise.


5. The FileHandleWritable Interface

  * write()  flush()
 - It might be useful to have support for transacted streams where