Re: CORS performance
Anne van Kesteren ann...@annevk.nl wrote: Concerns raised by Monsur https://lists.w3.org/Archives/Public/public-webapps/2012AprJun/0260.html and others before him are still valid. When you have an HTTP API on another origin you effectively get a huge performance penalty. Even with caching of preflights, as each fetch is likely to go to a distinct URL. Definitely there is a huge performance penalty per-request when the preflight isn't cached. But: 1. Preflight is only necessary for a subset of CORS requests. Preflight is never done for GET or HEAD, and you can avoid preflight for POST requests by making your API accept data in a format that matches what HTML forms post. Therefore, we're only talking about PUT, DELETE, less common forms of POST, and other less commonly-used methods. 2. It seems very wasteful to design an API that requires multiple PUT/DELETE/POST requests to complete a transaction (I'm using the word transaction loosely here; I don't mean ACID, necessarily). A lot of people think that REST means that you should only change a resource by PUT/DELETE/POSTing to its URL, however that's a misunderstanding of what REST is really about. Regardless of CORS, it is a good idea to design APIs in such a way that every modification transaction can be done in one or as few requests as possible, and this can be done a way that is in line with RESTful design. This type of design is more-or-less required when you need ACID transactions anyway. 3. Often you can predict which resources need preflight. With HTTP/2 and SPDY, the server can use the server push mechanism to push preflight responses to the client before the client even sends them. Given #1, #2, and #3, I'm a little bit unsure how bad the performance problem really is, and how bad it will be going forward. It would be good to see a concrete example to get a better understanding of the issue. Cheers, Brian
Re: CORS performance
On Thu, Feb 19, 2015 at 2:45 AM, Anne van Kesteren ann...@annevk.nl wrote: On Thu, Feb 19, 2015 at 11:43 AM, Brian Smith br...@briansmith.org wrote: 1. Preflight is only necessary for a subset of CORS requests. Preflight is never done for GET or HEAD, and you can avoid preflight for POST requests by making your API accept data in a format that matches what HTML forms post. Therefore, we're only talking about PUT, DELETE, less common forms of POST, and other less commonly-used methods. Euh, if you completely ignore headers, sure. But most HTTP APIs will use some amount of custom headers, meaning *all* methods require a preflight. Is it really true that most HTTP APIs will sue some amount of custom headers? And, is is it necessary for these APIs to be designed such that the custom headers are required? Cheers, Brian
Re: CORS performance
Dale Harvey d...@arandomurl.com wrote: The REST api pretty much by design means a unique url per request CouchDB has http://wiki.apache.org/couchdb/HTTP_Bulk_Document_API, which allows you to fetch or edit and create multiple documents at once, with one HTTP request. CouchDB's documentation says you're supposed to POST a JSON document for editing, but the example doesn't set the Content-Type on the request so presumably it is OK to set the Content-Type to text/plain. This means that you'd have ONE request and ZERO preflights to edit N documents. in this case a lot of the requests look like GET origin/_change?since=0 GET origin/_change?since=the last id A GET like this won't require preflight unless you set custom header fields on the request. Are you setting custom headers? If so, which ones and why? I looked at the CouchDB documentation and it doesn't mention any custom header fields. Thus, it seems to me like none of the GET requests should require preflight. Also, if your server is SPDY or HTTP/2, you should be able to configure it so that when the server receives a request GET /whatever/123, it replies with the response for that request AND pushes the response for the not-even-yet-sent OPTIONS /whatever/123 request. In that case, even if you don't use the preflight-less bulk document API and insist on using PUT, there's zero added latency from the preflight. Cheers, Brian
Re: CORS performance
On Thu, Feb 19, 2015 at 4:49 AM, Dale Harvey d...@arandomurl.com wrote: so presumably it is OK to set the Content-Type to text/plain Thats not ok, but may explain my confusion, is Content-Type considered a Custom Header that will always trigger a preflight? To be clear, my comment was about POST requests to the bulk document API, not about other requests. I ran your demo and observed the network traffic using Wireshark. Indeed, OPTIONS requests are being sent for every GET. But, that is because you are setting the Content-Type header field on your GET requests. Since GET requests don't have a request body, you shouldn't set the Content-Type header field on them. And, if you do, then browsers will treat it as a custom header field. That is what forces the preflight for those requests. Compare the network traffic for these two scripts: script xhr=new XMLHttpRequest(); xhr.open(GET, http://skimdb.iriscouch.com/registry/_changes?timeout=25000style=all_docssince=209limit=100_nonce=xhGtdb3XqOaYCWh4;, true); xhr.setRequestHeader(Accept,application/json); xhr.setRequestHeader(Content-Type,application/json); xhr.send(); /script script xhr=new XMLHttpRequest(); xhr.open(GET, http://skimdb.iriscouch.com/registry/_changes?timeout=25000style=all_docssince=209limit=100_nonce=xhGtdb3XqOaYCWh4;, true); xhr.setRequestHeader(Accept,application/json); xhr.send(); /script They are the same, except the second one doesn't set the Content-Type header, and thus it doesn't cause the preflight to be sent. if so then none of the caching will apply, CouchDB requires sending the appropriate content-type CouchDB may require sending Accept: application/json, but that isn't considered a custom header field, so it doesn't trigger preflight. The /_changes requests are only part of the problem, once we receive the changes information we then have to request information about individual documents which all have a unique id GET /registry/mypackagename We do one of those per document (70,000 npm docs), all trigger a preflight (whether or not custom headers are involved) I believe none of these require preflight unless a mistake is being made (probably setting Content-Type on GET requests). Also, regardless, you can use the CouchDB bulk document API to fetch all these documents in one request, instead of 70,000 requests. Also performance details aside every week somebody has a library or proxy that sends some custom header or they just missed a step when configuring CORS, its a constant source of confusion for our users. We try to get around it by providing helper scripts but Anne's proposal mirroring flashes cross domain.xml sounds vastly superior to the current implementation from the developers perspective. I agree that things can be improved here. I think the solution may be better developer tools. In particular, devtools should tell you exactly why a request triggered preflight. Cheers, Brian
Re: CORS performance
Dale Harvey d...@arandomurl.com wrote: I believe none of these require preflight unless a mistake is being made (probably setting Content-Type on GET requests). http://www.w3.org/TR/cors/#preflight-result-cache-0 If the cache is against the url, and we are sending requests to different urls, wont requests to different urls always trigger a preflight? In general, if your GET requests don't set custom headers, preflight isn't necessary, because CORS has an optimization for GET (and POST) that avoids preflight, for exactly the cases like yours.. Also, regardless, you can use the CouchDB bulk document API to fetch all these documents in one request, instead of 70,000 requests. CouchDB has no bulk document fetch api Sorry. I was reading http://docs.couchdb.org/en/latest/api/database/bulk-api.html#db-bulk-docs and assumed it had been implemented already. But I see that maybe you are trying to do something slightly different anyway with PouchDB. Regardless, no preflight should be necessary for this and so Anne's proposal won't help with it. I agree that things can be improved here. I think the solution may be better developer tools. In particular, devtools should tell you exactly why a request triggered preflight. Whats wrong with 'This origin is part of the public internet and doesnt need any complications or restrictions due to CORS' ie Anne proposal? I didn't say anything was wrong with Anne's proposal. What I said is that it would help to have somebody present a concrete example of where it would be useful. Cheers, Brian
Re: CORS performance proposal
On Thu, Feb 19, 2015 at 5:29 AM, Anne van Kesteren ann...@annevk.nl wrote: When the user agent is about to make its first preflight to an origin (timeout up to the user agent), it first makes a preflight that looks like: OPTIONS * Access-Control-Request-Origin-Wide-Cache: [origin] Access-Control-Request-Method: * Access-Control-Request-Headers: * This would make CORS preflight even slower for every server that doesn't implement the new proposal (i.e. every currently-deployed server and probably most servers deployed in the future). Perhaps the OPTIONS * request could be made in parallel with the initial preflight requests. But, then, what happens when the information in the OPTIONS * response conflicts with the information in the normal preflight request? I think this has a reasonable tradeoff between security and opening up all the power of the HTTP APIs on the server without the performance hit. It still makes the developer very conscious about the various features involved. I think developer consciousness is exactly the issue here: 1. Let's say you want to add OPTIONS * preflight support to an existing web application. How do you go about finding all the things that need to change to make that safe to do? It seems very difficult to successfully find every place the app assumes it is protected by the fact that it doesn't do CORS. 2. Similar to #1, let's say that two teams develop two parts of a website. One of the teams follows normal CORS rules and the other depends on the proposed OPTIONS * mechanism. This would be a disaster if/when both apps are deployed on the same origin. 3. Because of these issues, an organization forces its developers to develop every app as though every resource is CORS-enabled, to future-proof against the scenerio where OPTIONS * is deployed in the future. This makes the development of the web app more difficult and slower. 4. In the discussion of Entry Point Regulation (EPR) on WebAppSec, the main argument in favor of it is that it is impossible for developers to do things like #3 correctly and it is unreasonable for us to expect them to. I'm don't buy the EPR argument completely, but I do see some merit in the underlying secure by default argument behind EPR. Because of these concerns, I think that it would be worthwhile to study a concrete example of the problem, to make sure we correctly understand the use case we're trying to solve. As we saw yesterday with the PouchDB/CouchDB example, it is easy to accidentally and unnecessarily force a preflight. It may also be the case that we can find other, safer, ways to avoid preflights and/or optimize how they are done, such as by optimizing CORS for use with HTTP/2 server push mechanisms. But, we need to see real instances of the problem first. Cheers, Brian
Re: Write-only form fields (was Re: Proposal for a credential management API.)
On Fri, Aug 1, 2014 at 5:37 AM, Mike West mk...@google.com wrote: On Thu, Jul 31, 2014 at 6:37 PM, Brian Smith br...@briansmith.org wrote: particular, if we are worried about XSS stealing passwords then we have to consider the possibility that XSS has inserted a form without any httponly attributes being used, right? Correct. I think we'd also want a new CSP directive which toggles write-only status for all password fields on a given page: how about http://projects.mikewest.org/credentialmanagement/writeonly/#credentials-directive? There is some tension here between making things password-specific and simple vs. making them general and harder to understand. Defining this as a mechanism to protect only passwords keeps it simple. But, it seems wrong to have a way to protect passwords but not credit card numbers and social security numbers and other very sensitive input fields that don't use input type=password. This would work with (C) too, would it not? It may be a good idea to add an attribute to XHR to trigger such replacement, so that the browser doesn't have to attempt substitution for every HTTP request. I think we'd be able to get away with relying on magical UA behavior: if the browser process hands a nonce to a renderer, it can set a flag, and then look at POSTs generated by the page. As soon as one POST contains the nonce, clear the flag. My suspicion is that most login pages don't do much POSTing, so the overhead would be trivial. I am not sure that looking only at POSTs is sufficient. Also, some websites put login forms on every page (whether they should or not). But, I agree that it would be better to avoid the need for the attribute if we can. I'd prefer that approach, because I don't think we want to expose the actual mechanics to the web. The website shouldn't need to care about whether or not the password it's received is the real password or not. I suspect some websites will want to disable some aspects of their form validation code if they are dealing with placeholders instead of the real values, especially if the mechanism is extended to things such as social security numbers and credit card numbers. Based on a quick read of Mike's proposal, this would require Mike's proposed API to change to pass around tokens that represent passwords, instead of the password values themselves. This would add complication, but it would be useful. This approach adds complication to the UA's implementation, but shouldn't add complexity to the site consuming the API. This would probably not interact well with use of the WebCrypto API to encrypt the contents of input fields (passwords, credit card numbers, etc.) before submission. I'm pretty happy to break that use case, given that the credential API I've proposed is locked to secure origins. There's no advantage to using WebCrypto to doubly encrypt the password in this context, and I don't think it's something we should encourage. I think it is fine to say that this would be mutually-exclusive with WebCrypto-based approaches to encrypting passwords in the short term. However, I think it is too early in the history of WebCrypto to say that there's advantage to encrypting passwords (or other sensitive information like credit card numbers) in a way that protects them from the from the web server. I think it is likely that some way of composing WebCrypto and this mechanism will be necessary, eventually. Cheers, Brian
Re: Proposal for a credential management API.
On Thu, Jul 31, 2014 at 8:19 AM, Jacob S Hoffman-Andrews j...@eff.org wrote: I'd say there are approximately three styles for login form submission: A) No JS. A form with some input type=text's that gets submitted when you click an input type=submit. B) Some JS. A form that gets submitted by JS calling form.submit(). C) All JS. A set of inputs whose contents are extracted by JS and POST'ed via XHR to the server. Clearly we can't make C safe against XSS. But I think a lot of web sites currently use A or B, or would be willing to use them in exchange for better security. I think we can make C work too. Here's a rough idea: Define a new attribute 'httponly' for input elements. And/or the password form could be annotated with an attribute that indicates for which domain an XHR should be allowed to submit the password to. And/or, you could have a submit-password CSP directive to indicate which domains passwords are allowed to be submitted to. In particular, if we are worried about XSS stealing passwords then we have to consider the possibility that XSS has inserted a form without any httponly attributes being used, right? When a text input has httponly=true and the password manager saves the input value as a PendingCredential or a Credential, the password manager also stores an attribute httpOnly=true on the Credential. When the password manager autofills forms using a Credential with httpOnly=true, it should fill a placeholder value (possibly a numeric identifier for the Credential). I was thinking the placeholder would be a base64url-encoded cryptographically-random nonce of sufficient length, so that the browser can replace the placeholders within arbitrary HTTP requests, regardless of (most) use of JS to mangle forms before submitting them, and without worrying about replacing the wrong part. When a form is submitted, the password manager should intercept the HTTP request and replace the placeholder value with the contents of the Credential. This would work with (C) too, would it not? It may be a good idea to add an attribute to XHR to trigger such replacement, so that the browser doesn't have to attempt substitution for every HTTP request. Web browsers with sandboxed child processes have the networking logic in the more-privileged parent process. The purpose of sandboxing is to protect against exploits in the child process. It would be useful for the process/privilege separation of sandboxing to be able to protect the values of passwords--even if it can't always protect the *use* of the passwords--even in the event of a compromised child process. The placeholder technique described by Jacob would facilitate such protection by giving the browser the ability to withhold passwords from the child (renderer) processes. Based on a quick read of Mike's proposal, this would require Mike's proposed API to change to pass around tokens that represent passwords, instead of the password values themselves. This would add complication, but it would be useful. This would probably not interact well with use of the WebCrypto API to encrypt the contents of input fields (passwords, credit card numbers, etc.) before submission. However, it seems reasonable to think that we could provide some way to integrate both things. One way would be to define a new API for declarative crypto operations, that allow the browser to do the substitution and then the crypto without the application's JS logic ever seeing it. Another way would be to provide a mechanism for isolating JS code from the DOM (possible reusing the worker infrastructure) so that some small part of the page's JS code can do the necessary transformations given the cleartext passwords, without leaking them; this seems hard though. Also note how this is pretty at odds with the idea (as I vaguely understand it) that ServiceWorkers should be able to do anything that the browser could do, unless the placeholder replacement was done for outgoing requests made by ServiceWorkers too. But, I think the protection of passwords and similar secrets is worthwhile enough to make exceptions and/or do extra work to resolve this conflict. BTW, Jacob's placeholder idea is similar to the ideas in: https://bugzilla.mozilla.org/show_bug.cgi?id=653132 and https://bugzilla.mozilla.org/show_bug.cgi?id=759860#c2 AFAICT, many security people at Mozilla thought it was a good idea, but nobody has tried to implement it in Firefox yet. I also think it is a good idea for some browser to try it out. Cheers, Brian
Re: Proposal for a credential management API.
On Thu, Jul 31, 2014 at 9:37 AM, Brian Smith br...@briansmith.org wrote: Web browsers with sandboxed child processes have the networking logic in the more-privileged parent process. The purpose of sandboxing is to protect against exploits in the child process. It would be useful for the process/privilege separation of sandboxing to be able to protect the values of passwords--even if it can't always protect the *use* of the passwords--even in the event of a compromised child process. By the way, I don't know if any browsers do this, but AFAICT HttpOnly cookies can be protected by such process separation in the same way, and we should ensure that ServiceWorkers is defined and implemented in a way that allows for such protection to (continue to) work. Cheers, Brian
