> On May 22, 2016, at 3:39 PM, Daniel Olegovich Lazarenko <dani...@opera.com> > wrote: > > What if I make a bug report in bugzilla about making a design spec of this > feature. I could then write down implementation details options and summarize > all points we've discussed here. Maybe in a form of a google document. This > spec could then be reviewed and approved for action by you and any other > interested people you want to be involved. Do you think it could work better?
Before getting to the design of things, we usually spend quite a bit of time figuring out what the use cases are, and from the those use cases, what the different possible solutions are that address those use cases. I don’t feel we have gotten past the use case part yet. - Sam > > On Sun, May 22, 2016 at 11:59 PM, Brady Eidson <beid...@apple.com > <mailto:beid...@apple.com>> wrote: > >> On May 22, 2016, at 3:14 AM, Daniel Olegovich Lazarenko <dani...@opera.com >> <mailto:dani...@opera.com>> wrote: >> >> > It’s not yet clear what the ideal architecture is, which is one of the >> > reasons why the mentioned issued remains unresolved. >> >> What are the other reasons? > > Perhaps I misrepresented - AFAIK that is the only important reason. > >> Are there any reasons that block us from discussing the right architecture? >> I'd like to start working on a patch, but I need directions from you. > > I replied to this thread to describe significant issues with the two > approaches you suggested. > > But I am not able to conclude the thread by unilaterally giving directions to > a lone contributor on how to properly implement this feature. > > That’s a much broader conversation than just you and I. > > Thanks, > ~Brady > >> >> I'd like to come up with some sort of a plan for this as well. Since the >> desired approach sounds complicated, it would be nice to split it as a >> series of patches where each patch is committed separately and improves the >> feature towards the goal. >> >> On Sun, May 22, 2016 at 6:16 AM, Brady Eidson <beid...@apple.com >> <mailto:beid...@apple.com>> wrote: >> >>> On May 21, 2016, at 2:05 PM, Daniel Olegovich Lazarenko <dani...@opera.com >>> <mailto:dani...@opera.com>> wrote: >>> >>> > We are exploring ways to restore that full functionality - >>> > https://bugs.webkit.org/show_bug.cgi?id=138169 >>> > <https://bugs.webkit.org/show_bug.cgi?id=138169> >>> >>> Having custom scheme protocols is important to me too. I didn't know that >>> it is broken with WKWebView. Do you know what exactly is broken? >> >> From most developer’s perspective, what is broken is that their >> NSURLProtocol they can register in their “UI Process” that used to work in >> WK1 views no longer has any effect in WK2. >> >>> >>> I thought that if you call [WKBrowsingContextController >>> registerSchemeForCustomProtocol:] with your scheme, then it works. When I >>> checked last (a year ago) it was implemented in a way that the >>> WebProcess/NetworkingProcess would pass a message to UIProcess, and handle >>> the network request in the UIProcess. Did it change? >> >> That did not change. >> >> But that mechanism was never API, and even as SPI it is formally deprecated. >> >>> Assuming that registerSchemeForCustomProtocol still works the same way, you >>> basically state that you dislike the current solution (that does the work >>> in the UIProcess), and want to have a different architecture. >>> >>> For custom networking or proxying you have to run the app-provided code. >>> The basic strategy I proposed was to run it in the app process (i.e. >>> UIProcess). Since you don't want any networking in UIProcess, it means that >>> the app needs to spawn a dedicated process to do custom networking. This >>> process would run app-specific code (including NSURLProtocol-s), and >>> communicate by IPC with the NetworkingProcess. Is this a kind of >>> architecture you would like to have? >> >> It’s not yet clear what the ideal architecture is, which is one of the >> reasons why the mentioned issued remains unresolved. >> >> Thanks, >> ~Brady >> >> >>> >>> >>> On Fri, May 20, 2016 at 5:58 PM, Brady Eidson <beid...@apple.com >>> <mailto:beid...@apple.com>> wrote: >>> >>>> On May 20, 2016, at 2:30 AM, Daniel Olegovich Lazarenko <dani...@opera.com >>>> <mailto:dani...@opera.com>> wrote: >>>> >>>> Thank you for such a fast reply. That is amazing! :) >>>> Back to questions... >>>> >>>> > Are you primarily focused on a custom networking layer (e.g. your own >>>> > HTTP implementation?), >>>> > or with custom protocol handling for non-http protocols? >>>> >>>> I'm primarily concerned about HTTP and friends (HTTPS, SPDY, HTTP2), but >>>> if there are any other widely used protocols that's also interesting. FTP >>>> support is not interesting for me. Do you have any other specific things >>>> in mind? >>>> >>>> If there's a custom proprietary protocol that people handle in the app >>>> with their own code, for example, acme://acme.com:1234 >>>> <http://acme.com:1234/>, then proxying this thing is not very interesting >>>> to me, because it's very easy to proxy my own protocol handled by my own >>>> code. There's a case when "acme" is provided by some 3rd party, and the >>>> app author doesn't have the processing code. In such case it might be >>>> interesting to proxy it as well, but then again, I'm asking for a concrete >>>> example of such protocol (in WebKit context). >>> >>> In a WebKit1 app (WebView on Mac, UIWebView on iOS), app authors were able >>> to use NSURLProtocol to override any scheme they wished. >>> >>> While some such as yourself might’ve used it to override http from the >>> default handling, *many more* used it to implement custom protocols. e.g. >>> “acme://my-app-specific-url <>” >>> >>> We are exploring ways to restore that full functionality - >>> https://bugs.webkit.org/show_bug.cgi?id=138169 >>> <https://bugs.webkit.org/show_bug.cgi?id=138169> >>> >>>> > You seem to dismiss the Networking process’ crash recovery aspect. >>>> > "because in practice we know that most of the crashes happen in the >>>> > WebProcess parts”. >>>> > I’m curious what data you’re using to make that claim? >>>> >>>> Well, I'm not dismissing it, it's definitely a trade off that an app >>>> author will make by choosing to enable this option. >>>> The data comes from our web browser apps. We certainly see networking >>>> faults, but in total it was usually a minor part of all the WebKit >>>> crashes. To not sound subjective, I've looked through our current app >>>> version, which already has enough data to judge, and in the top WebKit >>>> crashes there are none in the network code. Most are crashes in >>>> JavaScriptCore, DOM and graphics subsystems. This is the experience we >>>> have over many versions and years of service. I might be able to show you >>>> the data in private if you want, although I'm sure that you have your own >>>> crash analysis system with much more data. >>> >>> Without getting in to specifics, the NetworkingProcess does crash. >>> >>> And while the WebContent process does crash way more, it usually only >>> effects that one web page. >>> >>> If networking code was back inside the UI Process, and it crashed, that >>> takes down the whole browser. >>> >>> Doing so would be reverting towards the single process architecture of >>> yesteryear, not progressing away from it. >>> >>>> Let's discuss the sandboxing a little bit. First of all, and correct me if >>>> I'm wrong: I thought that there's no 3rd party code execution in the >>>> networking process, >>> >>> Currently, there is no 3rd party code execution in the networking process. >>> >>>> and all the JS execution happens inside the WebProcess. >>> >>> That’s correct, but I’m not necessarily talking about JS. >>> >>>> The code that we want to run is our own code that we control that we >>>> implement in a safe and secure manner. >>> >>> The 1st party is “the Modern WebKit framework.” >>> The 3rd party is “the application’s code, as well as stuff downloaded from >>> the Internet. >>> >>>> In this sense it's no less secure. >>> >>> A single process app that does whatever it wants with the networking stack >>> is less secure to the user than a multi-process app that has well defined >>> behavior inside certain sandboxed parts of the app. >>> >>>> In the end we are always protected by the iOS/Mac sandbox. >>> >>> For a WKWebView app on iOS, while the app itself is sandboxed, the >>> Networking process is *much more* sandboxed. >>> >>>> Maybe I'm wrong about JS here, or do you have some other use case in mind? >>> >>> I think you misunderstood that I was talking about the native app code that >>> is a 3rd party client to the WebKit framework. >>> >>> But, there is an important JS aspect to this; Since there is messaging >>> between the WebContent process and the Networking process, a vulnerability >>> in the WebContent process can turn into a “remote exploit” of the >>> Networking process. >>> Therefore it is beneficial to have the Networking process tightened down as >>> much as possible. >>> >>> --- >>> >>> There’s another very important reason why offering a >>> “networking-in-UI-process mode” is not appealing. From a project >>> maintenance standpoint, the more configurations we have the harder it is to >>> test and develop within the project. >>> >>> Adding this “optional” networking mode would double certain testing >>> matrices and make development in this area (which there has been a lot of >>> lately) more difficult. >>> >>>> > Debugging the multi process architecture of WebKit2 >>>> > has not gotten any harder in years, active developers have all adapted, >>>> > and new developers tend to pick it up pretty quickly. This is not a >>>> > useful point. >>>> >>>> I'm sorry that you are rejecting this. Of course you can adapt to that, >>>> but it inevitably has a steeper learning curve, and takes longer time. >>>> Many app developers come from a single-process background and find >>>> multi-process debugging much harder. Often it's a real challenge to >>>> understand what's going on. I'm sure that you in your team have multiple >>>> stories that show how non-trivial it is, and tricks about dealing with it. >>>> Nevertheless, I agree, it's not a decisive point. >>> >>> AFAIK, we haven’t had a potential contributor to the WebKit Open Source >>> project decide to not contribute because of the multi-process architecture. >>> >>> But, regardless, the MP architecture is primarily how well WebKit works as >>> a browser engine for the user, and not about how easy it is for >>> single-process-only developers to contribute. >>> >>> Such developers can still actively contribute to the cross platform code of >>> the project (WebCore) in single process mode using MiniBrowser or >>> DumpRenderTree. >>> >>> Thanks, >>> ~Brady >>> >>>> >>>> >>>> >>>> On Thu, May 19, 2016 at 6:43 PM, Brady Eidson <beid...@apple.com >>>> <mailto:beid...@apple.com>> wrote: >>>> >>>>> On May 19, 2016, at 8:41 AM, Daniel Olegovich Lazarenko >>>>> <dani...@opera.com <mailto:dani...@opera.com>> wrote: >>>>> >>>>> I'd like to ask your for advice about implementation of a custom >>>>> networking layer >>>> >>>> Are you primarily focused on a custom networking layer (e.g. your own HTTP >>>> implementation?), or with custom protocol handling for non-http protocols? >>>> >>>>> ...with WKWebView on iOS. >>>> >>>> WKWebView is an API that ships on both OS X and iOS. When a design aspect >>>> of it affects both platforms (such as the networking behavior), we must >>>> consider both platforms. >>>> >>>>> Our current solution is based on NSURLProtocol, and the issues we had >>>>> with it in 2014 are unresolved: >>>>> https://bugs.webkit.org/show_bug.cgi?id=137302 >>>>> <https://bugs.webkit.org/show_bug.cgi?id=137302> >>>>> https://bugs.webkit.org/show_bug.cgi?id=138131 >>>>> <https://bugs.webkit.org/show_bug.cgi?id=138131> >>>>> >>>>> It was kind of a shoehorn hack, and so it was rejected by Benjamin >>>>> Poulain and Alexey Proskuryakov among other reviewers. >>>> >>>> I’m not sure it’s useful for WebKit to spend energy testing and >>>> maintaining a mechanism that *only* allows for HTTP-handling replacement >>>> and doesn’t also allow for the oft-requested feature of custom protocol >>>> handling. >>>> >>>>> Now I'm again looking for a better solution. >>>>> I'd really like to discuss it with somebody responsible, >>>> >>>> There is no single person responsible; the project works largely on >>>> consensus. When dealing with platform specific concerns such as this, it >>>> works on consensus of the platform owners. >>>> >>>> That said, I have been the primary caretaker of the Networking process >>>> since it’s inception, as well one of the primary caretakers of Mac/iOS >>>> networking in general for many years, so I’ll share my thoughts below. >>>> >>>>> There's currently 2 solutions I'm weighting: >>>>> Pass and use NetworkProcessCreationParameters.httpProxy to >>>>> NSURLSessionConfiguration (in NetworkSession and maybe other places). The >>>>> httpProxy solution is easy to implement and would look clean design-wise. >>>>> It would let us spawn an HTTP proxy on localhost and filter the traffic >>>>> there. There might be some complications, because it's not fully >>>>> transparent to the client side. For example HTTPS will have issues. All >>>>> in all this could be a fine short-term solution. >>>> While ToT WebKit contains an NSURLSession-based networking implementation >>>> for Mac/iOS, it also still contains an NSURLConnection implementation, >>>> which is unaffected by NSURLSession considerations. >>>> >>>> That a solution doesn’t work on all supported platforms is not a deal >>>> breaker, but it certainly makes it less interesting than one that does. >>>> >>>> HTTPS losing reliability is likely an unacceptable red flag. >>>> >>>> I’m not sure it’s useful for WebKit to spend energy testing and >>>> maintaining a mechanism that *only* allows for HTTP-handling replacement >>>> and doesn’t also allow for the oft-requested feature of custom protocol >>>> handling. >>>>> Add a new mode to the NetworkProcess, which would do all networking in >>>>> UIProcess (instead of spawning a new process). A mode would be optional >>>>> and controlled with some configuration setting (or NSUserDefaults). >>>>> The UIProcess solution is harder to implement, and it will affect more >>>>> code. It is somewhat controversial. One of the reasons of splitting out a >>>>> NetworkProcess was to have it respawn after crashes. Nevertheless we can >>>>> take this risk, because in practice we know that most of the crashes >>>>> happen in the WebProcess parts. >>>> >>>> You seem to dismiss the Networking process’ crash recovery aspect. >>>> "because in practice we know that most of the crashes happen in the >>>> WebProcess parts”. I’m curious what data you’re using to make that claim? >>>> >>>>> I don't see any other significant downsides of having the UIProcess >>>>> handling networking. >>>> >>>> The Networking process provides significant benefit unrelated to crash >>>> recovery that should not be abandoned for convenience sake. e.g. >>>> Sandboxing. >>>> >>>> Especially when moving the networking to the UI process would also end up >>>> moving 3rd party code execution into the UI process, this seems like an >>>> unacceptable regression. >>>> >>>>> In addition it can simplify the NetworkProcess debugging. >>>> >>>> Debugging the multi process architecture of WebKit2 has not gotten any >>>> harder in years, active developers have all adapted, and new developers >>>> tend to pick it up pretty quickly. This is not a useful point. >>>> >>>> Thanks, >>>> ~Brady >>>> >>>> >>> >>> >> >> > > > _______________________________________________ > webkit-dev mailing list > webkit-dev@lists.webkit.org > https://lists.webkit.org/mailman/listinfo/webkit-dev
_______________________________________________ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
