Re: IndexedDB: Syntax for specifying persistent/temporary storage
Hi Kinuko and Eric, Thanks for providing feedback from a Google perspective! Definitely still feels like there's a lot of possibilities and options here. No combination of solutions seem obviously best to me, so please do take the below as me thinking out loud rather than having strong opinions. On Wed, Dec 11, 2013 at 10:53 PM, Kinuko Yasuda kin...@chromium.org wrote: Fyi, Chrome applies similar policy to (A) for installed apps, but only when a special permission is requested in its manifest file. For regular apps their 'temporary' storage is still under eviction control. Interesting! I had hoped that we would be able to align on storage policies for installed/bookmarked hosted apps (not packaged apps). Maybe that was too naive :-) Though sounds like we still are very close. Close enough that it we can probably align on API. The problem with this solution is that it doesn't give bookmarked apps the ability to create truly temporary data. Even data that a bookmarked app puts in the temporary storage is effectively treated as persistent and so not deleted if we start to run low on disk space. Temporary storage for apps is a feature that Android has, and that to some extent *nix OSs has had through use of /tmp. It definite is something that seems nice for constrained mobile devices. As a slightly conservative variation, would it work if we just 'unlock' persistent storage for webapps when they're 'bookmarked' or got an installed app-like permission? The app could check if it could use persistent storage without requesting (by issuing quota query API, or maybe via some event), and convert some data into persistent if they like, but UA doesn't do so automatically. The UA continues to evict 'temporary' data even for 'bookmarked' webapps (UA could prioritize temporary data for bookmarked ones if it wants). I think this would work, but it has downsides. See my answer below to Eric. B) We could create a temporary/default/persistent split. I.e. we create three different storage categories. The default is what's used if no storage category is explicitly specified a IDB database is created. For normal webpages default is treated like temporary. I.e. it is counted towards the same quotas as temporary storage and it's deleted automatically if we run low on space. However once a page transitions to being an installed app data in default is treated as persistent. temporary storage would always be treated as truly temporary. I.e. the implementation is always free to delete data there if the user is running low on storage. If data stored by bookmarked apps count towards the same quotas as temporary data stored by web pages is an implementation decision. persistent storage would behave as in A. I.e. webpages can't use it without there being a prompt involved at some point. Bookmarked webapps can use it without prompt. This sounds kinda reasonable to me, but I'd imagine some webapps (who are serious about their storage) would probably want to know if the default is either temporary or persistent in this case, and we may need a tiny API for that. Definitely, if we go with this solution we should provide a way to know what policy is used for default. D) Alex Russell proposed adding an event which asks a page I'm low on storage, please clear out any data that is not important. This could be combined with either A or C to enable installed apps to have really temporary data. It has the advantage that data that isn't critical can be stored in the same IDB database as critical data which is really nice. I.e. you no longer have to separate critical and non-critical data into separate databases which prevent transactions that touch both. However it has the disadvantage that it makes clearing out temporary data a *really* heavy operation. It means having to launch all apps in the background and have them run code. This could mean spawning lots of processes and doing lots of complicated queries and other IO. I'm actually a fan of this idea, as it gives most flexible option (but with some cost). I wouldn't say it doesn't make the eviction operation heavy, but UA could probably do smarter than launching all apps at once. Most apps may not be interested in such an event at all, and UA could only check the apps that have shown interest on adjusting their data on their own, probably via some registration mechanism. UA could also use some order like LRU to launch apps one by one until it can get enough available space. One of the situations when we need to clear out temporary storage is when we're in the middle of a IDB (or presumably WebSQL) transaction and run low on disk space. The page might already have been granted permanent storage so we don't want to simply abort the transaction and blame quota. Would a notification such as the one described here help in that situation? I.e. would you really want to pause the current transaction and start firing up
Re: Request for feedback: Streams API
Hi, Implementation of back pressure is important for handling large data stably. The Streams need to ask the consumer code to notify it when it can consume more data. Triggering this signal by method invocation is one of possible options. Promises fit this well. To address smaug___'s concern, we could choose not to 1-to-1 correspond delivery of data and pull operation. Deliver data using callback invocation, while allowing the consuming code to tell amount it want to pull via pullAmount. Like WritableByteStream allows write() ignoring back pressure, we allow ReadableByteStream to push available data to consuming code ignoring back pressure. As far as the producer inside WritableByteStream does work respecting pullAmount, back pressure mechanism should still work well. One concern Domenic showed in the chat will remain un-addressed. That is an issue with callback setting timing. If we take Promise read() approach, arrived data never vanishes into the void. I understand this error-proneness. Takeshi On Mon, Dec 16, 2013 at 9:21 PM, Olli Pettay olli.pet...@helsinki.fiwrote: On 12/04/2013 06:27 PM, Feras Moussa wrote: The editors of the Streams API have reached a milestone where we feel many of the major issues that have been identified thus far are now resolved and incorporated in the editors draft. The editors draft [1] has been heavily updated and reviewed the past few weeks to address all concerns raised, including: 1. Separation into two distinct types -ReadableByteStream and WritableByteStream 2. Explicit support for back pressure management 3. Improvements to help with pipe( ) and flow-control management 4. Updated spec text and diagrams for further clarifications There are still a set of bugs being tracked in bugzilla. We would like others to please review the updated proposal, and provide any feedback they may have (or file bugs). Thanks. -Feras [1] https://dvcs.w3.org/hg/streams-api/raw-file/tip/Overview.htm So per https://www.w3.org/Bugs/Public/show_bug.cgi?id=24054 it is not clear to me why the API is heavily Promise based. Event listeners tend to work better with stream like APIs. (The fact the Promises are hip atm is not a reason to use them for everything ;) ) -Olli
Re: inline declarative manifest, was Re: New manifest spec - ready for FPWD?
On Mon, Dec 9, 2013 at 1:33 AM, Yoav Weiss y...@yoav.ws wrote:
or
something else. Like you said, I think it's a conversation we need to
have with the HTML people.
I’ll investigate a bit more. I’ve added a bug here:
https://github.com/w3c/manifest/issues/91
I’ll just note that having link rel=manifest and script type
manifest would kinda suck… if we can just have:
* script type=“application/manifest+json” src=“manifest.json”/script
* script type=“application/manifest+json”{}/script that would be
great.
Implementations would probably have to be careful and exclude this kind of
scripts from the usual script handling (e.g. blocking of the HTML parser,
triggering the preloader, etc), to reduce the impact on the critical path.
This is a good point. Would this have performance implications for
down-level browsers? I don't know if prescanners etc in contemporary
browsers are smart enough to ignore script tags that use a non-JS
type attribute.
IMO, it might be better not to define an explicit way to inline the
manifest, and let authors simply use data URIs to do that, if they see such
a need.
e.g. link rel=manifest href=data:application/manifest+json,{ ... }
If this becomes a common authoring pattern, an explicit mechanism might be a
good fit. Otherwise, there may not be a need.
This sounds mostly ok.
The only problem is that we need to figure out how updates works. Once
the user has bookmarked a website, it's important that the website can
update the data that's stored in the manifest. For example that they
can tweak their icon as they slightly change their branding. Or that
they can add that they now need a back-button rendered.
For normal markup like link rel=manifest href=/manifest.json, the
way the implementation would do this is by remembering the URL of the
manifest, and at regular intervals reload that URL to see if anything
needs updating.
For a data: URL that obviously doesn't work as the URL will always
return exactly the same manifest.
For an inline manifest, the way to do this would be to use the URL of
the page that contains the manifest. I.e. the UA would at regular
intervals reload the URL that contains the inline manifest, and
extract the updated manifest from that.
That does bring up a important question though. How does a developer
migrate from an inline manifest to an external manifest if the
application grows. I don't think there are clean solutions for that.
You could at that point honor a link to an external manifest. Not
great, but should work pretty intuitively in most cases.
So if we were to use data-URLs, then I think we still would have to
write language into the spec for how to treat those to allow them to
be updated.
All in all though, this is a pretty good argument against internal
manifests in any form.
/ Jonas
Re: inline declarative manifest, was Re: New manifest spec - ready for FPWD?
On 12/17/13 3:29 AM, Jonas Sicking wrote: This is a good point. Would this have performance implications for down-level browsers? I don't know if prescanners etc in contemporary browsers are smart enough to ignore script tags that use a non-JS type attribute. Gecko's is not. Not least because as far as I can tell scripts with unknown type are in fact always loaded, just not executed, and block the parser while they're loading, so you do in fact want to preload them! I'm not sure whether that behavior is Gecko-specific or not, but I suspect not: I recall people using unknown script types to do preloading in general. -Boris
Re: Passsword managers and autocomplete='off'
Thanks for the feedback, everyone. A few people at this point have suggested emailing the wha...@whatwg.org list since this is really an HTML feature; I'll do that in a few. In response to Ian's question, I'm referring to the W3 WebForms standard: http://www.w3.org/Submission/web-forms2/#the-autocomplete I've also inlined some responses to Maciej below. On Fri, Dec 13, 2013 at 12:47 AM, Maciej Stachowiak m...@apple.com wrote: On Dec 12, 2013, at 11:20 AM, Joel Weinberger j...@chromium.org wrote: Hi all. For a while now, we have wanted on Chrome to ignore autocomplete='off' for password fields for the password manager. We believe that the current respect for autocomplete='off' for passwords is, in fact, harming the security of users by making browser password managers significantly less useful than they should be, thus discouraging their adoption, making it difficult for users to generate, store, and use more complex or (preferably) random passwords. Additionally, the added benefit of autocomplete='off' for security is questionable at best. We believe that our implementation of this ignore functionality actually falls within the letter of the web-forms standard. A user's password save for an autocomplete='off' field requires a user interaction to save (we do not do it automatically), which ultimately is not different than a copy/paste approach from the user. Additionally, we have taken precautions against password harvesting via XSS. We do not autofill into the DOM until the user has made a gesture (click, keypress, etc.) within the page, and we never autofill into iframe forms (we wait for a user to explicitly select their username from a dropdown). Part of the issue here is that autocomplete='off' is overloaded. It is simultaneously meant to denote a secure or sensitive field *or* that a field's completion will be handled by the application itself. Thus, we are not proposing to ignore autocomplete='off' for our form fill as there are many places where the application itself creates a suggestion box, and we have no desire to override that functionality. Rather, we care about the sensitive use, which in the case of password fields, is already denoted by the input type='password'. In the latest version of Chrome (currently in our Canary build), we have already implemented this feature. However, we will putting in behind a flag shortly so that it is not the default, but to still allow users to opt into this. We hope to make this the default for users in the not very distant future. What are this group's thoughts on this? Any particular concerns with this approach? While we believe that we are within the letter of the standards in our approach, we would love to see this made explicitly clear in the standards and hopefully see other browsers adopt this in the future, as we believe it is in the security interests of all users. Safari has similar behavior available as anon-default user preference: Allow AutoFill even for websites that request passwords not be saved. Even with this setting enabled, some user interaction is required both to save and to fill. We agree that on net, refusing to autofill passwords harms security. As of yesterday, in Chrome Canary, it is now a flag that users can turn on as well. Not quite the menu option it is in Safari, but an option nonetheless. We did not make it the default, because website owners have objections to bypassing autofill=off as a default behavior. The primary types of objections are: (1) Public computer scenarios. Accidentally saving a password on a shared or public computer may endanger the user's account. The browser saving passwords seems like the least of concerns with public computers :-P From our perspective, given that key loggers etc are a big threat on a public computer, we don't consider the benefit here to outweigh the overall user benefit of having managed passwords. (2) Casual walk-up attacks, for example temporary access to a friend's or relative's computer. AKA friendly fraud. Similar to the public computer case, from our perspective, this benefit doesn't seem to outweigh the benefit of having more complex, managed passwords. I should also mention that, in general, Chrome explicitly does not consider a physically local attacker in our threat model. At least some website operators (often for financial or shopping sites) are more worried about these threats than about the risk of users using lower quality or shared passwords passwords. This factor is magnified in Safari where we suggest autogenerated per-site random passwords, but only if we can autofill them. This is precisely our concern. We believe the priorities of website operators are badly misplaced. There is a huge amount of evidence of the problems with non-complex passwords (see various password database leaks of late), and extremely low evidence of the prevalence of these various local attacker threats. We
Re: Passsword managers and autocomplete='off'
On Tue, 17 Dec 2013, Joel Weinberger wrote: Thanks for the feedback, everyone. A few people at this point have suggested emailing the wha...@whatwg.org list since this is really an HTML feature; I'll do that in a few. In response to Ian's question, I'm referring to the W3 WebForms standard: http://www.w3.org/Submission/web-forms2/#the-autocomplete That is actually the WHATWG spec from 2005. It's absurdly out of date. :-) This evolved into what is now this: http://whatwg.org/html#attr-fe-autocomplete Please, as an implementor, never look at a document that is more than a week or so old. If the date isn't really recent, then you're almost certainly looking at something obsolete, and implementing it will just mean you're implementing known-buggy text. (Or, you're looking at something that's fallen into decay with no active maintenance, which is almost as bad.) -- Ian Hickson U+1047E)\._.,--,'``.fL http://ln.hixie.ch/ U+263A/, _.. \ _\ ;`._ ,. Things that are impossible just take longer. `._.-(,_..'--(,_..'`-.;.'
Re: Passsword managers and autocomplete='off'
On Dec 17, 2013, at 11:21 AM, Joel Weinberger j...@chromium.org wrote: Thanks for the feedback, everyone. A few people at this point have suggested emailing the wha...@whatwg.org list since this is really an HTML feature; I'll do that in a few. In response to Ian's question, I'm referring to the W3 WebForms standard: http://www.w3.org/Submission/web-forms2/#the-autocomplete That is pretty out of date and not a standards track document. Safari has similar behavior available as anon-default user preference: Allow AutoFill even for websites that request passwords not be saved. Even with this setting enabled, some user interaction is required both to save and to fill. We agree that on net, refusing to autofill passwords harms security. As of yesterday, in Chrome Canary, it is now a flag that users can turn on as well. Not quite the menu option it is in Safari, but an option nonetheless. We did not make it the default, because website owners have objections to bypassing autofill=off as a default behavior. The primary types of objections are: (1) Public computer scenarios. Accidentally saving a password on a shared or public computer may endanger the user's account. The browser saving passwords seems like the least of concerns with public computers :-P From our perspective, given that key loggers etc are a big threat on a public computer, we don't consider the benefit here to outweigh the overall user benefit of having managed passwords. (2) Casual walk-up attacks, for example temporary access to a friend's or relative's computer. AKA friendly fraud. Similar to the public computer case, from our perspective, this benefit doesn't seem to outweigh the benefit of having more complex, managed passwords. I should also mention that, in general, Chrome explicitly does not consider a physically local attacker in our threat model. To be clear, I'm not saying I agree with these reasons. But these are the reasons some major financial or shopping sites state when asked why they use autocomplete=off. We have mentioned counter-arguments nearly identical to what you say, and in most cases, the site operators were not persuaded. (We did not say we explicitly don't consider physically local attackers, as we do give it some consideration; though obviously we cannot protect against a sufficiently determined attacker with physical access.) I also forgot to mention: (3) At least some sites believe that consumer finance regulations require them to use autocomplete=off. They believe their requirement to protect the user's authentication information and prevent it from being accessed by third parties extends to preventing passwords from being automatically stored on the user's computer. At least some website operators (often for financial or shopping sites) are more worried about these threats than about the risk of users using lower quality or shared passwords. This factor is magnified in Safari where we suggest autogenerated per-site random passwords, but only if we can autofill them. This is precisely our concern. We believe the priorities of website operators are badly misplaced. There is a huge amount of evidence of the problems with non-complex passwords (see various password database leaks of late), and extremely low evidence of the prevalence of these various local attacker threats. We applaud Safari's generated password management and believe it is the best interest of our users to make their passwords more complex for all of the web. It's also the case that by prioritizing the request of the website over the desire of users to manage their passwords, we are violating the Priority of Constituencies. I suppose that by offering a flag we are somewhat skirting around that, but I don't consider it a real option. I largely agree with this. I do not know if we can collectively assuage the worries of sites that use autocomplete=off. So far, though, I'm not aware of any complaints about our non-default setting to bypass it. I'm not sure we can convince website operators of anything like this without making the change happen and demonstrating the world won't collapse. I suspect that a lot of this is a fear of the unknown. The sites that are concerned about password autocomplete have at least two possible actions they could take that made us hesitant to flip the default: (A) They could blacklist browsers that ignore autocomplete=off - this has been explicitly threatened in the past. It's also been claimed that, even if the sites did not want to do this, regulatory bodies would require them to do so because of point #3 above. (B) They could reimplement password input from scratch (using script and not using a real password field) in a way that evades browser autocomplete heuristics. This would be bad because it would prevent even off-by-default autocomplete=off bypass from working. For these reasons, we
Re: IndexedDB: Syntax for specifying persistent/temporary storage
Option C) using numeric priority are good enough for most web application. It has advantage of easy to implement. Option C) can be combined with option B) by defining that 0 means temporary, 1 means persistent and undefined means default. Any other values should throw error. Since we can query allocated quota and knowing estimated app data size, persistent will be grantee. Kyaw
Re: Passsword managers and autocomplete='off'
On Tue, Dec 17, 2013 at 2:12 PM, Maciej Stachowiak m...@apple.com wrote: On Dec 17, 2013, at 11:21 AM, Joel Weinberger j...@chromium.org wrote: Thanks for the feedback, everyone. A few people at this point have suggested emailing the wha...@whatwg.org list since this is really an HTML feature; I'll do that in a few. In response to Ian's question, I'm referring to the W3 WebForms standard: http://www.w3.org/Submission/web-forms2/#the-autocomplete That is pretty out of date and not a standards track document. My apologies; originally I looked at the document Ian referenced ( http://www.whatwg.org/specs/web-apps/current-work/multipage/association-of-controls-and-forms.html#attr-fe-autocomplete), but when asked, I did a quick search and pulled up the first thing I saw :-) Safari has similar behavior available as anon-default user preference: Allow AutoFill even for websites that request passwords not be saved. Even with this setting enabled, some user interaction is required both to save and to fill. We agree that on net, refusing to autofill passwords harms security. As of yesterday, in Chrome Canary, it is now a flag that users can turn on as well. Not quite the menu option it is in Safari, but an option nonetheless. We did not make it the default, because website owners have objections to bypassing autofill=off as a default behavior. The primary types of objections are: (1) Public computer scenarios. Accidentally saving a password on a shared or public computer may endanger the user's account. The browser saving passwords seems like the least of concerns with public computers :-P From our perspective, given that key loggers etc are a big threat on a public computer, we don't consider the benefit here to outweigh the overall user benefit of having managed passwords. (2) Casual walk-up attacks, for example temporary access to a friend's or relative's computer. AKA friendly fraud. Similar to the public computer case, from our perspective, this benefit doesn't seem to outweigh the benefit of having more complex, managed passwords. I should also mention that, in general, Chrome explicitly does not consider a physically local attacker in our threat model. To be clear, I'm not saying I agree with these reasons. But these are the reasons some major financial or shopping sites state when asked why they use autocomplete=off. We have mentioned counter-arguments nearly identical to what you say, and in most cases, the site operators were not persuaded. (We did not say we explicitly don't consider physically local attackers, as we do give it some consideration; though obviously we cannot protect against a sufficiently determined attacker with physical access.) I also forgot to mention: (3) At least some sites believe that consumer finance regulations require them to use autocomplete=off. They believe their requirement to protect the user's authentication information and prevent it from being accessed by third parties extends to preventing passwords from being automatically stored on the user's computer. At least some website operators (often for financial or shopping sites) are more worried about these threats than about the risk of users using lower quality or shared passwords. This factor is magnified in Safari where we suggest autogenerated per-site random passwords, but only if we can autofill them. This is precisely our concern. We believe the priorities of website operators are badly misplaced. There is a huge amount of evidence of the problems with non-complex passwords (see various password database leaks of late), and extremely low evidence of the prevalence of these various local attacker threats. We applaud Safari's generated password management and believe it is the best interest of our users to make their passwords more complex for all of the web. It's also the case that by prioritizing the request of the website over the desire of users to manage their passwords, we are violating the Priority of Constituencieshttp://www.schemehostport.com/2011/10/priority-of-constituencies.html. I suppose that by offering a flag we are somewhat skirting around that, but I don't consider it a real option. I largely agree with this. I do not know if we can collectively assuage the worries of sites that use autocomplete=off. So far, though, I'm not aware of any complaints about our non-default setting to bypass it. I'm not sure we can convince website operators of anything like this without making the change happen and demonstrating the world won't collapse. I suspect that a lot of this is a fear of the unknown. The sites that are concerned about password autocomplete have at least two possible actions they could take that made us hesitant to flip the default: (A) They could blacklist browsers that ignore autocomplete=off - this has been explicitly threatened in the past. It's also been claimed that, even if the
