[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
On Thu, Apr 30, 2009 at 11:40 AM, Darin Fisher da...@chromium.org wrote: On Tue, Apr 28, 2009 at 6:22 PM, Aaron Boodman a...@chromium.org wrote: On Tue, Apr 28, 2009 at 6:18 PM, Michael Nordman micha...@chromium.org wrote: + chromium-dev Can you please explain what you think has changed since such decisions were made (or why it's time to revisit such decisions)? I don't think there was code in webcore suitable for this purpose before... html parsing, javascript,sql interpretting... all dangerous from a security point of view (acting in very complex ways on untrusted web content). The backend logic for these new features aren't like that. Its not so much that its webcore code is untrusted, as much as the data it operates on is untrusted. I think this gets at the core of my question: is it OK to run webcore code in the browser process if it is similar in nature to chromium code we would run in the browser process? Or is there some deeper structural reason we don't want to do that? I fear that this is really a question for Darin, who is on vacation. can talk more when i get back, but in a nut shell: 1- we already use webcore indirectly (albeit in a very limited way) from the browser process 2- the challenge with doing so is threading: what is the webcore main thread? what things depend on this and what don't is not well defined. -darin I've been thinking thru how to share the backend appcache code and have some thoughts on the threading issue. When built into a single process (so not the chrome situation) * the 'backend' appache code is NOT used on the 'main' webkit thread * for the sake of discussion, lets call the thread it is used on the 'backend' webkit thread * the single process 'frontend' classes post tasks to the backend thread * sync apis are handled by waiting on a WaitableEvent in the frontend classes, which the will get signaled by the backend thread when done * async api completion is handled by posting tasks to the 'main' thread (probably easier to setup than having the main thead wait for multple waitable event handles in the main message loop) * generally, backend classes running on the 'backend' thread don't block that thread for file io. Any blocking file io work gets pushed off to a seperate file io handling thread(s). (This is similar to what we require of code that executes on chrome's IPC thread in the main browser process) When built into chrome * the 'backend' code is NOT used in the renderer at all * it is used on the IPC thread in the main browser process (and it doesn't block that thread) * the multiprocess 'frontend' classes send IPC message instead of posting thread tasks (sync and async) * the 'frontend' code is NOT used in the browser process all all I have always felt like running the WebCore backend in the browser was elegant Yea, but we need a webcore backend to run :) Well last time I looked at this there was already a Database backend :) - a --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
The story is even simpler for localStorage. Everything is fairly self contained and the only way it cares about the main thread is in asserts to verify SQLite is not used on the main thread. My guess is that the story for what's considered the main thread will change for each API much like the split between frontend and backend will change. For localStorage, I'll do the refactoring in such a way that the asserts call a virtual function which Chromium can override. Instead of returning whether or not it's the main thread, it'll return whether disk IO is allowed on the thread or not. J On Thu, Apr 30, 2009 at 12:39 PM, Michael Nordman micha...@google.comwrote: On Thu, Apr 30, 2009 at 12:26 PM, Michael Nordman micha...@google.com wrote: On Thu, Apr 30, 2009 at 11:40 AM, Darin Fisher da...@chromium.org wrote: On Tue, Apr 28, 2009 at 6:22 PM, Aaron Boodman a...@chromium.org wrote: On Tue, Apr 28, 2009 at 6:18 PM, Michael Nordman micha...@chromium.org wrote: + chromium-dev Can you please explain what you think has changed since such decisions were made (or why it's time to revisit such decisions)? I don't think there was code in webcore suitable for this purpose before... html parsing, javascript,sql interpretting... all dangerous from a security point of view (acting in very complex ways on untrusted web content). The backend logic for these new features aren't like that. Its not so much that its webcore code is untrusted, as much as the data it operates on is untrusted. I think this gets at the core of my question: is it OK to run webcore code in the browser process if it is similar in nature to chromium code we would run in the browser process? Or is there some deeper structural reason we don't want to do that? I fear that this is really a question for Darin, who is on vacation. can talk more when i get back, but in a nut shell: 1- we already use webcore indirectly (albeit in a very limited way) from the browser process 2- the challenge with doing so is threading: what is the webcore main thread? what things depend on this and what don't is not well defined. -darin I've been thinking thru how to share the backend appcache code and have some thoughts on the threading issue. When built into a single process (so not the chrome situation) * the 'backend' appache code is NOT used on the 'main' webkit thread * for the sake of discussion, lets call the thread it is used on the 'backend' webkit thread * the single process 'frontend' classes post tasks to the backend thread * sync apis are handled by waiting on a WaitableEvent in the frontend classes, which the will get signaled by the backend thread when done * async api completion is handled by posting tasks to the 'main' thread (probably easier to setup than having the main thead wait for multple waitable event handles in the main message loop) * generally, backend classes running on the 'backend' thread don't block that thread for file io. Any blocking file io work gets pushed off to a seperate file io handling thread(s). (This is similar to what we require of code that executes on chrome's IPC thread in the main browser process) When built into chrome * the 'backend' code is NOT used in the renderer at all * it is used on the IPC thread in the main browser process (and it doesn't block that thread) * the multiprocess 'frontend' classes send IPC message instead of posting thread tasks (sync and async) * the 'frontend' code is NOT used in the browser process all all In this view... * The 'main' webkit thread is free to be the UI thead in the browser process. I don't think we have a need for this for the appcache or localstorage systems yet... but perhaps when it comes time to prompt users for things or alert them to quota excesses and such this could become relevant for those system. * the 'backend' webkit thread == the IPC thread in the browser process --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
On Wed, Apr 29, 2009 at 7:02 AM, Peter Kasting pkast...@chromium.orgwrote: On Tue, Apr 28, 2009 at 6:38 PM, Jeremy Orlow jor...@chromium.org wrote: Darin was there on that lunch and was actually the one who first suggested running parts of WebCore in the browser to me during a 1:1. :-) Indeed, we're already planning to do it in other ways -- using a text field as a directly-instantiable class in the UI, for example. I don't really have much of an opinion on this discussion except that I'm not so negative about the original plan of writing our own backend as Jeremy is. Here are his objections: From a technical standpoint, it's unclear how testing would work since our test_shell would be testing a different backend from what's in Chromium. It also means we have more code to maintain, and that code is completely off of WebKit's radar. It also makes Apple mad and Dimitri sad. So really, this doesn't seem like a good solution. I'm not sure how test_shell matters w.r.t. testing this backend, since the primary test of the backend (and the frontend?) would presumably be through unittests (w/mocks where necessary), which test the code directly and don't care what backend it's in. test_shell already doesn't have the same code as the main browser for 100 other things. Having code to maintain off WebKit's radar is again not a new situation. I'm not talking about forked stuff, just anything that isn't in WebKit. Having the code in WebKit is a two-edged sword, since when it's there it's gatekeepered on WebKit code review (not a big problem anymore since we have a couple reviewers now) and the WebKit community doesn't seem to try to keep their tree green. So IMO having code in WebKit is still just as much of a maintenance burden on us except with the additional hurdle that it's not in our direct tree. As far as making people mad, feh. Let's decide the right implementation first. making people mad is a simplification of a complex set of things. Ideally, there would be one implementation in webkit for html features which would allow multiple things: 1. Better compatibility across webkit impls. This makes it easier for web devs. 2. Bugs fixes in one implementation help for both. 3. It helps move the web forward better because we may implement a feature and our webkit impls will get it. 4. The above means that we can make better use of layout tests both in knowing that the ones there should probably pass and contributing new ones too. 5. Giving back to the webkit community is a good thing. etc. But even if the original plan is IMO not so bad, I'm not opposed to the new plans either. People like Darin are much more qualified to comment on what would be *best* and I will be OK with whatever gets decided. PK --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
On Wed, Apr 29, 2009 at 7:43 AM, David Levin le...@google.com wrote: On Wed, Apr 29, 2009 at 7:02 AM, Peter Kasting pkast...@chromium.orgwrote: On Tue, Apr 28, 2009 at 6:38 PM, Jeremy Orlow jor...@chromium.orgwrote: Darin was there on that lunch and was actually the one who first suggested running parts of WebCore in the browser to me during a 1:1. :-) Indeed, we're already planning to do it in other ways -- using a text field as a directly-instantiable class in the UI, for example. I don't really have much of an opinion on this discussion except that I'm not so negative about the original plan of writing our own backend as Jeremy is. Here are his objections: From a technical standpoint, it's unclear how testing would work since our test_shell would be testing a different backend from what's in Chromium. It also means we have more code to maintain, and that code is completely off of WebKit's radar. It also makes Apple mad and Dimitri sad. So really, this doesn't seem like a good solution. I'm not sure how test_shell matters w.r.t. testing this backend, since the primary test of the backend (and the frontend?) would presumably be through unittests (w/mocks where necessary), which test the code directly and don't care what backend it's in. test_shell already doesn't have the same code as the main browser for 100 other things. Having code to maintain off WebKit's radar is again not a new situation. I'm not talking about forked stuff, just anything that isn't in WebKit. Having the code in WebKit is a two-edged sword, since when it's there it's gatekeepered on WebKit code review (not a big problem anymore since we have a couple reviewers now) and the WebKit community doesn't seem to try to keep their tree green. So IMO having code in WebKit is still just as much of a maintenance burden on us except with the additional hurdle that it's not in our direct tree. As far as making people mad, feh. Let's decide the right implementation first. making people mad is a simplification of a complex set of things. Ideally, there would be one implementation in webkit for html features which would allow multiple things: 1. Better compatibility across webkit impls. This makes it easier for web devs. 2. Bugs fixes in one implementation help for both. 3. It helps move the web forward better because we may implement a feature and our webkit impls will get it. 4. The above means that we can make better use of layout tests both in knowing that the ones there should probably pass and contributing new ones too. 5. Giving back to the webkit community is a good thing. etc. David hit on most of the points I planned to make. making people mad was obviously a gross oversimplification of the situation. :-) One thing that I forgot to mention was that the WebKit guys are now starting to think about making WebKit more attractive to others that want to use a multi-process model. To me, this is a very noble goal, and fits in with the original goals of Chromium, so it's something we should try to support--even if it means a bit more work for us. But even if the original plan is IMO not so bad, I'm not opposed to the new plans either. People like Darin are much more qualified to comment on what would be *best* and I will be OK with whatever gets decided. I hate speaking for Darin because it's possible that there's some details here he did not agree with, but I'm pretty sure that most of this is exactly what we talked about, and that he was supportive of it. I agree that having our own backend still _could_ be the way to go, but I guess I'd like to give this a shot first. Note also that localStorage is actually a pretty simple example. It'll be interesting to see what things look like for other features, like AppCache. There, having our own backend probably has more impact, though I'm not sure if it's a positive or negative impact. J --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
I think it's a great idea and the only drawback I can see is the WTF dependency and the security implications, which shouldn't be anything we couldn't overcome. The biggest challenges IMHO would be: 1) clearly identifying what backend and frontend mean and where the separation occurs. I worry (perhaps without substance) that the definition may vary from port to port. 2) general logistics of actually lopping off huge parts of existing WebKit code and refactoring them in-flight. But I totally think we (you) are up to the challenge :) :DG On Tue, Apr 28, 2009 at 5:05 PM, Jeremy Orlow jor...@chromium.org wrote: Yes, yes, I know this is a horrible idea, but please hear me out :-) Last week, a couple of us (Darin F, Michael N, Jeremy M, and I) had lunch at Apple to talk to talk about sharing more code. HTML 5 brings with it a lot of APIs that reach outside of the top level browsing context boundary (i.e. the render process boundary in Chromium). We talked specifically about localStorage and appCache. Although I believe the following generalizes well for any such API, I recognize that there are some unique constraints for stuff like databases...so I'm not even going to talk about them right now. Anyhow... For a while now, I've looked at a bunch of ways to make localStorage multi-process aware, but really none of them have any hope except one: splitting localStorage into a frontend and backend. The frontend would be the portion in each renderer process that connects into the JS bindings. A single backend would store all the data and be shared by the frontends. Originally, my plan was to do this split and then write my own back end in the browser process, but there are several problems with this. From a technical standpoint, it's unclear how testing would work since our test_shell would be testing a different backend from what's in Chromium. It also means we have more code to maintain, and that code is completely off of WebKit's radar. It also makes Apple mad and Dimitri sad. So really, this doesn't seem like a good solution. Assuming the only viable solution is having several frontends talking to one backend (I'm confident this is true) and assuming having our own backend is not viable (this also seems true), then the only choice is for us to use the WebCore backend. We can't run this in any renderer process since the response times for browser-renderer communication are unbounded. So that leaves either the browser process or some browser helper process. Creating a helper process for the browser seems like a pretty interesting idea to me since there's already a lot of somewhat dangerous stuff running in the browser process. (The only thing I can remember right now is v8 for parsing .pac files, but I know there's more.) Basically, the helper process would be a sandboxed process where anything dangerous but not bound to a single renderer process would run. Ideally it would store little to no state so that the browser could easily restart it and resend any pending messages if it crashed. For localStorage, the backend (which is part of WebCore) would run there and all localStorage messages would be proxied through the browser process. The VFS could be used to retrieve file handles. The other option is to simply run part of WebCore's localStorage within the browser process. LocalStorage only depends on WTF, so this really isn't _that_ terrible of an idea. Thoughts? Anyhow, the WebKit guys we talked to like the idea of a split frontend/backend, especially if it means we'll continue sharing code. I believe Michael is going to be doing something similar for AppCache. J --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
Yup, I'm looking to work thru things in a similar fashion with anders (original appcache author). On Wed, Apr 29, 2009 at 11:53 AM, David Levin le...@google.com wrote: I think how to split/refactor depends on the feature, understanding the current code, and working with the appropriate webkit dev. For workers, we thought about how/where it made sense to split the impl for chrome and talked with a...@webkit about it. There was some iteration as we figured out things better and as he made suggestions and it has worked out fine so far. On Wed, Apr 29, 2009 at 11:49 AM, Michael Nordman micha...@chromium.org wrote: +chromium-dev Designing things in this front/back fashion and re-using the code entirely in chrome removes at least one high-coefficient of friction surface rubbing between the chromium and webkit teams. We will likely pay a higher price up front, but it should pay dividends down the road. Add an interesting feature and iphone, andriod, chrome, and safari all pick it up. I'm worried about the logistics of pulling this off for the appcache given the existing impl is live and in use in iphone and safari and soon andriod. We're talking about 'refactoring' or 'replacing' existing impls with new ones that support a remote backend. How can we reduce the upfront cost? * maybe build these new impls out in webcore w/o disrupting the existing impl * use the new impl for chrome (and any other webkit consumer that wishes to compile the new impls in would be free to do so) * somewhere down the road, deprecate the existing impl in favor of the new impl in webcore We haven't talked to the webkit guys about logistics like this, no data on where their head is? --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
On Tue, Apr 28, 2009 at 5:27 PM, Michael Nordman micha...@google.comwrote: In some sense we do have separate process in which to run sandboxed 'backend' code relevant to multiple renders if the need arises... the worker process. The way you stated this is a bit odd, but on the surface I guess this could be solved via special shared workers that ran WebKit code instead of javascript. That said, this means a lot of extra processes (for now, shared workers are each in their own process), this blocks localStorage on that, and actually makes the design more complicated. This might be worth exploring at some point, but not now. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
I have always felt like running the WebCore backend in the browser process with an IPC channel between it and the frontend was a really elegant design for this problem. And I never really understood why we weren't allowed to depend on WebCore (even indirectly) in the browser process. Is the idea that someday the browser and renderer processes might be separate binaries? - a On Tue, Apr 28, 2009 at 5:05 PM, Jeremy Orlow jor...@chromium.org wrote: Yes, yes, I know this is a horrible idea, but please hear me out :-) Last week, a couple of us (Darin F, Michael N, Jeremy M, and I) had lunch at Apple to talk to talk about sharing more code. HTML 5 brings with it a lot of APIs that reach outside of the top level browsing context boundary (i.e. the render process boundary in Chromium). We talked specifically about localStorage and appCache. Although I believe the following generalizes well for any such API, I recognize that there are some unique constraints for stuff like databases...so I'm not even going to talk about them right now. Anyhow... For a while now, I've looked at a bunch of ways to make localStorage multi-process aware, but really none of them have any hope except one: splitting localStorage into a frontend and backend. The frontend would be the portion in each renderer process that connects into the JS bindings. A single backend would store all the data and be shared by the frontends. Originally, my plan was to do this split and then write my own back end in the browser process, but there are several problems with this. From a technical standpoint, it's unclear how testing would work since our test_shell would be testing a different backend from what's in Chromium. It also means we have more code to maintain, and that code is completely off of WebKit's radar. It also makes Apple mad and Dimitri sad. So really, this doesn't seem like a good solution. Assuming the only viable solution is having several frontends talking to one backend (I'm confident this is true) and assuming having our own backend is not viable (this also seems true), then the only choice is for us to use the WebCore backend. We can't run this in any renderer process since the response times for browser-renderer communication are unbounded. So that leaves either the browser process or some browser helper process. Creating a helper process for the browser seems like a pretty interesting idea to me since there's already a lot of somewhat dangerous stuff running in the browser process. (The only thing I can remember right now is v8 for parsing .pac files, but I know there's more.) Basically, the helper process would be a sandboxed process where anything dangerous but not bound to a single renderer process would run. Ideally it would store little to no state so that the browser could easily restart it and resend any pending messages if it crashed. For localStorage, the backend (which is part of WebCore) would run there and all localStorage messages would be proxied through the browser process. The VFS could be used to retrieve file handles. The other option is to simply run part of WebCore's localStorage within the browser process. LocalStorage only depends on WTF, so this really isn't _that_ terrible of an idea. Thoughts? Anyhow, the WebKit guys we talked to like the idea of a split frontend/backend, especially if it means we'll continue sharing code. I believe Michael is going to be doing something similar for AppCache. J --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
Well, the question comes down to this: Can we trust running small amounts of WebCore in the browser process. Historically, the answer to running WebCore in the browser has been a resounding no. Can you please explain what you think has changed since such decisions were made (or why it's time to revisit such decisions)? My _feeling_ is that it would be OK with a security review, and my feeling is that a security review is possible since the code is fairly isolated, but that's just my opinion. I don't have anything terribly solid to back it up. J On Tue, Apr 28, 2009 at 5:46 PM, Michael Nordman micha...@chromium.orgwrote: On Tue, Apr 28, 2009 at 5:38 PM, Jeremy Orlow jor...@chromium.org wrote: On Tue, Apr 28, 2009 at 5:27 PM, Michael Nordman micha...@google.com wrote: In some sense we do have separate process in which to run sandboxed 'backend' code relevant to multiple renders if the need arises... the worker process. The way you stated this is a bit odd, but on the surface I guess this could be solved via special shared workers that ran WebKit code instead of javascript. That said, this means a lot of extra processes (for now, shared workers are each in their own process), this blocks localStorage on that, and actually makes the design more complicated. This might be worth exploring at some point, but not now. Let me re-iterate my main point... i dont think we need to sandbox the localstorage or appcache backend code, so we should be able to run that directly in the browser process. --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
+ chromium-dev Can you please explain what you think has changed since such decisions were made (or why it's time to revisit such decisions)? I don't think there was code in webcore suitable for this purpose before... html parsing, javascript,sql interpretting... all dangerous from a security point of view (acting in very complex ways on untrusted web content). The backend logic for these new features aren't like that. Its not so much that its webcore code is untrusted, as much as the data it operates on is untrusted. I have always felt like running the WebCore backend in the browser was elegant Yea, but we need a webcore backend to run :) --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
[chromium-dev] Re: Need to run parts of WebCore in either the browser process or some browser helper process
On Tue, Apr 28, 2009 at 6:26 PM, Evan Martin e...@chromium.org wrote: On Tue, Apr 28, 2009 at 5:59 PM, Aaron Boodman a...@chromium.org wrote: Is the idea that someday the browser and renderer processes might be separate binaries? Though this shouldn't drive your decision, about 50% of our code weight (at least by one metric) is WebKit. You could imagine that the browser process doesn't need a copy of that, and that the renderer process doesn't need a copy of all the browser process stuff (network stack, UI goop). On Linux, startup time is heavily affected by the cost of the dynamic linker pulling in GTK, which we only would need from the browser process if we had separate binaries. However, OS-level page sharing may mean having separate binaries doesn't actually help too much here. This is definitely something to keep in mind, but shouldn't we be able to just compile in the bits of webkit we care about? Yeah, we'll have 2 copies of WTF, but the rest should be pretty separate. On Tue, Apr 28, 2009 at 6:22 PM, Aaron Boodman a...@chromium.org wrote: I fear that this is really a question for Darin, who is on vacation. Darin was there on that lunch and was actually the one who first suggested running parts of WebCore in the browser to me during a 1:1. :-) --~--~-~--~~~---~--~~ Chromium Developers mailing list: chromium-dev@googlegroups.com View archives, change email options, or unsubscribe: http://groups.google.com/group/chromium-dev -~--~~~~--~~--~--~---
