Re: [qutebrowser] Multi web engine support in the future
Hey, On Wed, Mar 06, 2019 at 02:07:31PM -0800, Jay Kamat wrote: > > Florian Bruhin writes: > > > - Chromium embedded framework (CEF) [1][2] - has Python bindings[3] with > > Qt integration[4], so the only one which would be reasonably possible > > to integrate with qutebrowser. However, I don't see much benefit over > > QtWebEngine, seeing that both are based on Chromium, and I'm not aware > > of any killer features (like Chromium extension support) which would > > be a good reason to look into it. > > One killer feature would be avoiding qtwebengine bugs (if cef is any better). I can't say I've tried, but I have my doubts that it is. Certainly not to the point (IMHO) that it'd be worth the effort. > I don't think cef is in any repos though, so it's probably not > practical for most people. Yeah, that was one big reason that I've never looked at it in detail. I think the idea is that you bundle it with your application (like e.g. Steam does) and compile it from source. There aren't even any official builds (though Spotify provides[1] unofficial ones). [1] http://opensource.spotify.com/cefbuilds/index.html > >> In many cases, I found bugs in patches I submitted by running them > >> against QtWebKit (that affected QtWebEngine in subtler ways) and vice > >> versa, > >> so to me, even if no one uses QtWebKit it remains useful. > > > > Do you have examples for that? > > I find that qtwebkit is more aggressive on cleaning up resources on the C > side. Running a trivial manual/automated test in qtwebkit for a behavior often > exposes use-after-free bugs. > > The most recent one is this commit: > > https://github.com/qutebrowser/qutebrowser/pull/4498/commits/128d17fe91db7fb1819d741c0aab072934abb3bb > > Of course, there are other cases as well, I've seen qtwebkit tests catch other > random bugs (which were bugs on the webengine side too but untested or > flakey). Fair point - that indeed sounds useful to have, but probably still not useful enough for me to want to continue maintaining QtWebKit support (even more so with Qt/PyQt 6 on the horizon, where I bet things will get real painful) :-/ > >> The multiple backend approach works because there are multiple backends > >> supported through similar (and in some cases, the same) PyQt5/Qt apis. > >> Adding > >> support for another engine would be pretty difficult as you would need to > >> integrate it into both qt (and PyQt5). > >> > >> I think it would be extremely difficult to add support for a renderer not > >> integrated into qt, as qutebrowser is heavily dependent on qt for many > >> things. > > > > If you check the CEF example[4] it's actually not that bad - in brief: > > > > - Integrate the renderer's main loop with Qt (or configure it to run it > > its own thread, like I think done there). > > > > - Get the renderer to render into your window, by giving it a low-level > > window handle like in that example, or by providing it with an OpenGL > > thingy (surface?) to render on (which is what QtWebEngine does with > > Chromium, I think). > > > > - Dispatch some events like the window being moved/resized to the > > renderer. > > > > - Depending how things are implemented, dispatch input events as well, > > if Qt is getting them instead of the renderer (doesn't seem to be > > necessary in the CEF example). > > I don't think that's necessarily the hard part. The hard part is syncing up > the entire API so everything works seamlessly. Since the QtWebEngine/WebKit > apis are large, this isn't trivial. That looks like a bare minimum > implementation, but for proper integration into qutebrowser there are many > more APIs that need to be exposed for everything to work as expected. Whatever renderer we're talking about would already expose those APIs, right? So there are "only" two things to consider: - Making that API usable from Python via bindings (those already exist for CEF, but no idea how tricky it'd be to bind Rust for Servo). - Using the Python API inside qutebrowser - I don't see a big difference between a Qt renderer and one independent from Qt here. Sure, you might need to convert e.g. URLs from whatever URL type it uses to QUrl, but that shouldn't be too big, compared to the work adding a new renderer takes anyways. > >> It would probably be easier to just update QtWebKit to use a newer version > >> of > >> WebKit. > > > > I doubt it. Doing that is not something a single developer is able to > > do, as we saw with QtWebKit. Annulen isn't the only one who seems to > > have given up either: https://github.com/annulen/webkit/issues/775 > > Like said above, I don't have any hopes of it happening without a > > company picking it up (which doesn't seem to be happening either). > > Updating QtWebKit must be easier because it's the exact same task (porting > webkitgtk or plain webkit to qt) with a huge advantage - it already has a > working example on an older version. (non chromium) browser engines don't have
Re: [qutebrowser] Multi web engine support in the future
Florian Bruhin writes: > - Chromium embedded framework (CEF) [1][2] - has Python bindings[3] with > Qt integration[4], so the only one which would be reasonably possible > to integrate with qutebrowser. However, I don't see much benefit over > QtWebEngine, seeing that both are based on Chromium, and I'm not aware > of any killer features (like Chromium extension support) which would > be a good reason to look into it. One killer feature would be avoiding qtwebengine bugs (if cef is any better). Currently, the qtwebengine backend is about as painful to use as qtwebkit for me (and trending worse, ever since qt 5.7). I don't think cef is in any repos though, so it's probably not practical for most people. >> In many cases, I found bugs in patches I submitted by running them >> against QtWebKit (that affected QtWebEngine in subtler ways) and vice versa, >> so to me, even if no one uses QtWebKit it remains useful. > > Do you have examples for that? I find that qtwebkit is more aggressive on cleaning up resources on the C side. Running a trivial manual/automated test in qtwebkit for a behavior often exposes use-after-free bugs. The most recent one is this commit: https://github.com/qutebrowser/qutebrowser/pull/4498/commits/128d17fe91db7fb1819d741c0aab072934abb3bb Of course, there are other cases as well, I've seen qtwebkit tests catch other random bugs (which were bugs on the webengine side too but untested or flakey). >> The multiple backend approach works because there are multiple backends >> supported through similar (and in some cases, the same) PyQt5/Qt apis. Adding >> support for another engine would be pretty difficult as you would need to >> integrate it into both qt (and PyQt5). >> >> I think it would be extremely difficult to add support for a renderer not >> integrated into qt, as qutebrowser is heavily dependent on qt for many >> things. > > If you check the CEF example[4] it's actually not that bad - in brief: > > - Integrate the renderer's main loop with Qt (or configure it to run it > its own thread, like I think done there). > > - Get the renderer to render into your window, by giving it a low-level > window handle like in that example, or by providing it with an OpenGL > thingy (surface?) to render on (which is what QtWebEngine does with > Chromium, I think). > > - Dispatch some events like the window being moved/resized to the > renderer. > > - Depending how things are implemented, dispatch input events as well, > if Qt is getting them instead of the renderer (doesn't seem to be > necessary in the CEF example). I don't think that's necessarily the hard part. The hard part is syncing up the entire API so everything works seamlessly. Since the QtWebEngine/WebKit apis are large, this isn't trivial. That looks like a bare minimum implementation, but for proper integration into qutebrowser there are many more APIs that need to be exposed for everything to work as expected. > >> It would probably be easier to just update QtWebKit to use a newer version of >> WebKit. > > I doubt it. Doing that is not something a single developer is able to > do, as we saw with QtWebKit. Annulen isn't the only one who seems to > have given up either: https://github.com/annulen/webkit/issues/775 > Like said above, I don't have any hopes of it happening without a > company picking it up (which doesn't seem to be happening either). Updating QtWebKit must be easier because it's the exact same task (porting webkitgtk or plain webkit to qt) with a huge advantage - it already has a working example on an older version. (non chromium) browser engines don't have that advantage. There are several browser wrappers floating out there that are maintained by small groups - see this project for an example. The problem is that no one cares enough to maintain qtwebkit at the moment, not that it's a monumental/impossible task. https://github.com/zserge/webview In some ways, qtwebengine is maintained even worse than qtwebkit, as it has multiple bugs which have never been present in qtwebkit, and it is struggling to add features which have been present in webkit for a long time. https://bugreports.qt.io/browse/QTBUG-65223 https://bugreports.qt.io/browse/QTBUG-67434 https://bugreports.qt.io/browse/QTBUG-51192 https://bugreports.qt.io/browse/QTBUG-50995 (although this is hopefully done) https://bugreports.qt.io/browse/QTBUG-62957 https://bugreports.qt.io/browse/QTBUG-51772 https://bugreports.qt.io/browse/QTBUG-68224 (fixed, but really!) - Lack of a 'DomContentLoaded' signal on the cpp side - White flashes on page load (seems to be getting worse every release)
Re: [qutebrowser] Multi web engine support in the future
Hi, On Wed, Mar 06, 2019 at 01:29:03PM -0300, Javier Ayres wrote: > I would love to see servo working with qutebrowser (once it is stable, > which I think is far from) or even gecko Gecko is pretty much unusable outside of Firefox AFAIK - I'm not aware of any projects using it as a standalone engine, and even projects which kind-of did (Conkeror[1] using Firefox' xulrunner) died when Mozilla deprecated that. [1] http://conkeror.org/ [2] https://developer.mozilla.org/en-US/docs/Archive/Mozilla/XULRunner > Also thanks for the links to related projects. I've been toying with the > idea of writing something like that but failed to find enough documentation > to even get started. FWIW there's also a project which tried to write a vim-like browser using servo, but it seems to have died quickly: https://github.com/meh/miserve Florian -- https://www.qutebrowser.org | m...@the-compiler.org (Mail/XMPP) GPG: 916E B0C8 FD55 A072 | https://the-compiler.org/pubkey.asc I love long mails! | https://email.is-not-s.ms/ signature.asc Description: PGP signature
Re: [qutebrowser] Multi web engine support in the future
Thanks for the responses guys. It makes a lot of sense that you don't have a definitive answer (although your expectations seem to go off in opposite directions :D) given that this highly depends on the availability of external projects. I would love to see servo working with qutebrowser (once it is stable, which I think is far from) or even gecko but I agree that maintaining an abstraction layer just in case that ever happens is wasted effort. > It would probably be easier to just update QtWebKit to use a newer version of > WebKit. Agreed, but as Florian says, it is still a monumental task. I do think that is still easier than coming up with a whole new qt integration with another engine from scratch. Also thanks for the links to related projects. I've been toying with the idea of writing something like that but failed to find enough documentation to even get started. On Wed, Mar 6, 2019, 09:25 Florian Bruhin wrote: > Hi, > > On Tue, Mar 05, 2019 at 10:58:46PM -0300, Javier Ayres wrote: > > With the ongoing work happening as part of the research project and > > the idea of dropping support for QtWebKit, I'm wondering what is the > > plan for the future of qutebrowser regarding the support of multiple > > web engines. Are you planning to make qutebrowser a > > QtWebEngine-exclusive browser or are you going to maintain/improve the > > existing abstraction layer that exists between qutebrowser and the > > engine? Any thoughts on adding support for another specific web > > engine? > > I don't really have a definitive answer on this. I do want to simplify > things where possible - in places where such a change would increase > maintainability and reduce complexity, I will (over time) merge > "abstract" code with QtWebEngine-specific code, or be less strict about > introducing QtWebEngine-specific things. > > As long as there isn't another possible backend on the horizon, I think > that's the right thing to do - if there's ever another backend > introduced, the abstractions used for QtWebKit/QtWebEngine might need > some larger adjustments anyways. > > In short, there's no point in things being more complex than they have > to be, and I do think complexity/maintainability is a rather big issue > at the moment. I'd rather have things a bit simpler, and removing > QtWebKit support will be a big step in that direction (moving things > from the core to extensions is another big one). > > As for specific web engines: There are only three active engines I can > think of: > > - WebKitGTK - obviously not integrated with Qt. QtWebKit tries to do > so, but I don't see that going anywhere in the future. Some company > with enough resources would need to pick it up, and seeing that hasn't > happened since Qt removed it three years ago, I doubt it ever will. > > - Chromium embedded framework (CEF) [1][2] - has Python bindings[3] with > Qt integration[4], so the only one which would be reasonably possible > to integrate with qutebrowser. However, I don't see much benefit over > QtWebEngine, seeing that both are based on Chromium, and I'm not aware > of any killer features (like Chromium extension support) which would > be a good reason to look into it. > > - Servo[5] - support for it would be amazing (and it'd be a real > alternative to QtWebEngine), but the whole ecosystem isn't there yet. > Embedding it into an existing project isn't really supported well > (though there's some community effort like [6] and [7]). Also, > someone would need to write Rust <-> Python bindings for it, and find > out how to integrate those into Qt. That someone hasn't appeared so > far, and it likely won't be me. ;-) > > [1] https://bitbucket.org/chromiumembedded/cef > [2] https://en.wikipedia.org/wiki/Chromium_Embedded_Framework > [3] https://github.com/cztomczak/cefpython > [4] https://github.com/cztomczak/cefpython/blob/master/examples/qt.py > [5] https://github.com/servo/servo > [6] https://github.com/paulrouget/servo-embedding-example > [7] https://github.com/paulrouget/servo-embedding-api > > On Tue, Mar 05, 2019 at 06:41:41PM -0800, Jay Kamat wrote: > > > > Javier Ayres writes: > > > > > With the ongoing work happening as part of the research project and > > > the idea of dropping support for QtWebKit, I'm wondering what is the > > > plan for the future of qutebrowser regarding the support of multiple > > > web engines. > > > > Most likely, the abstract base classes will continue to exist even if one > > engine's support is dropped, but there will be less incentive to maintain > > them. It would probably be more effort to merge everything back together > (with > > little gain). I also think that the maintenance burden of QtWebKit is > not that > > high. > > I agree it's not a high burden, but there are a lot of smaller things: > > - It *is* additional work for major features or refactorings. The last > things I remember were Greasemonkey support (though some of that was > also needed for
