Re: [webkit-dev] WebKit GPU rendering possibility
Hi, There is quite likely ways around using boost. It's main use is to provide mutex, threading and atomic, which is quite easy to do another way. The harder to replace of boost is in how FastUIDraw handles loading front via FreeType I want to change in a big way and killing the boost dependency is one of those goals. -Kevin From: Olmstead, Don [don.olmst...@sony.com] Sent: Tuesday, November 22, 2016 9:04 PM To: Rogovin, Kevin; webkit-dev@lists.webkit.org Subject: RE: WebKit GPU rendering possibility We actually mentioned fastuidraw while we were at the contributors meeting as a potential replacement for our Cairo rendering for PlayStation. When we got to a point with our port where we can start evaluating we’d be interested in taking a look. The only problem I had with the setup was a reliance on boost, but maybe there are ways around it. From: webkit-dev-boun...@lists.webkit.org [mailto:webkit-dev-boun...@lists.webkit.org] On Behalf Of Rogovin, Kevin Sent: Wednesday, November 2, 2016 9:36 AM To: webkit-dev@lists.webkit.org Subject: [webkit-dev] WebKit GPU rendering possibility Hi, I was directed here by some colleagues as this is the place to post the following to get started on the following proposal. I have been working on an experimental 2D renderer that requires a GPU, the project is open sourced on github at https://github.com/01org/fastuidraw. I gave a talk at the X Developers Conference this year which can be seen from https://www.x.org/wiki/Events/XDC2016/Program/rogovin_fast_ui_draw/ . I made a benchmark which makes heavy use of rotations and clipping and ported to SKIA, Qt’s QPainter and Cairo. The benchmark and its ports are in the git repo linked above under the branch with_ports_of_painter-cells. It's performance advantage of FastUIDraw against the other renderers was quite severe (against Cairo and Qt's QPainter over 9 times and against SKIA about 5 times faster). I would like to explore the option of using FastUIDraw to implement a WebCore::GraphicsContext backend for the purpose of making drawing faster and more efficient on Intel devices that are equipped with a GPU. I also think that some minor modifications to WebKit’s use of GraphicsContext will also give some benefits. I have worked on WebKit a few years ago and knew/know my way around the rendering code very well (atleast at that time). Looking forward to collaboration, -Kevin Rogovin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] WebKit GPU rendering possibility
Hi, >I should mention, though, that we require support for hardware that only >supports OpenGL ES 2.0. >If FastUIDraw can't handle this, then we would need to keep a fallback >codepath that uses Cairo, which would be unfortunate. FastUIDraw requires features beyond what OpenGL ES 2.0 offers. With that in mind, the fall back is needed. I cannot stress how unfortunate it is to have the burden of needing to support hardware that only satisfies a specification that is nearly 10 years old and whose feature set corresponds to far more ancient times (essentially first generation DX9 cards, over 11 years ago). The jump in flexibility in handling data between ES2.0 and ES3.0/3.1 is massive. One question, what happens with WebGL 2.0 support on WebKit? I ask because WebGL 2.0 is essentially OpenGL ES 3.x for JavaScript. Best Regards, -Kevin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
Re: [webkit-dev] WebKit GPU rendering possibility
Hi, What are some good 2D UI graphics benchmarks that are cross-platform-ish? I'd think I need to port them to Fast UI Draw, but that is possible. I am very confident that Fast UI Draw will perform at the top by a large margin. The more complicated and heavier the load, the better it will do. -Kevin -Original Message- From: mmaxfi...@apple.com [mailto:mmaxfi...@apple.com] Sent: Thursday, November 3, 2016 9:07 PM To: Rogovin, Kevin Cc: Carlos Garcia Campos ; webkit-dev@lists.webkit.org Subject: Re: [webkit-dev] WebKit GPU rendering possibility It sounds like the primary focus of your work is improving performance. It also sounds like the only benchmark you’ve run is an artificial one that you constructed yourself. Given these two things, I would strongly hesitate to call our interest "significant community enthusiasm.” Why don’t you start by running some of the many existing graphics benchmarks with your library? Please correct me if my assumptions are mistaken. Thanks, Myles > On Nov 3, 2016, at 12:50 AM, Rogovin, Kevin wrote: > > Adding a new GraphicsContext is what I want to do as it seems the path of > least pain and suffering. However, all the other things of a backend I do not > need to do. I do not know how to add a GraphicsContext backend in terms of > makefile magicks and configuration. I also do not know the plumbing for > making it active. In theory, FastUIDraw's GraphicsContext will work on any > platform that does OpenGL 3.3 or OpenGL ES 3.0. What is the plumbing to do > this? Years ago I remember that the build configuration is what governed what > backend was built... and I usually just piggy packed onto another... years > ago I remember there was like an SDL style backend that did not require a > large toolkit, just SDL.. is that still alive? where is it? I could piggy > back the work there if it still is alive... > > Also, to get permission to do this work, I need significant community > enthusiasm otherwise I will not be able to justify the large amount of work > needed. This is another area where I need a great deal of help. > > Best Regards, > -Kevin Rogovin > > -Original Message- > From: Carlos Garcia Campos [mailto:carlo...@webkit.org] > Sent: Thursday, November 3, 2016 9:43 AM > To: Rogovin, Kevin ; Myles C. Maxfield > > Cc: webkit-dev@lists.webkit.org > Subject: Re: [webkit-dev] WebKit GPU rendering possibility > > El jue, 03-11-2016 a las 07:35 +, Rogovin, Kevin escribió: >> Hi, >> >> The main issue of making a Cairo backend to FastUIDraw is clipping. >> Cairo tracks the clipping region in CPU and does things that are fine >> for CPU-based rendering (i.e. span based rendering) but are >> absolutely awful for GPU rendering (from my slides, one sees that GL >> backed QPainter and Cairo do much worse than CPU backed). FastUIDraw >> only supports clipIn and clipOut and pushes all the clipping work to >> the GPU with almost no CPU work. It does NOT track the clipping >> region at all. I can give more technical details how it works (and >> those details are why FastUIDraw cannot be used a backend for Cairo). >> For those interested in where the code is located for clipping in >> FastUIDraw, it is located at src/fastuidraw/painter/painter.cpp, >> methods clipInRect, clipOutPath and clipInPath. Their implementations >> are very short and simple and are quite cheap on CPU. > > I see. Then I guess adding a new GraphicsContext for FastUIDraw is the > easiest and best way to try this out in WebKit. Would it be possible to just > add a new GraphicsContext implementation? or would you also need to change > other parts of the graphics implementation or the GraphicsContext API itself? > >> Best Regards, >> -Kevin >> >> -Original Message- >> From: Carlos Garcia Campos [mailto:carlo...@webkit.org] >> Sent: Thursday, November 3, 2016 9:27 AM >> To: Rogovin, Kevin ; Myles C. Maxfield > fi...@apple.com> >> Cc: webkit-dev@lists.webkit.org >> Subject: Re: [webkit-dev] WebKit GPU rendering possibility >> >> El jue, 03-11-2016 a las 06:58 +, Rogovin, Kevin escribió: >>> Hi! >>> >>> Question answers: >>> 1. Currently FastUIDraw has a backend to OpenGL 3.3 and OpenGL >>> ES 3.0. One of its design goals is to make it not terribly awful to >>> write a backend to different 3D API’s. >>> 2. I think I was unclear in my video. I have NOT migrated ANY >>> UI rendering library to use Fast UI Draw. What I have done is made a >>> demo >>> (painter-cells) and ported that demo to Fast UI Draw, Cairo, Qt’s >>> QPainter and SKIA. Th
Re: [webkit-dev] WebKit GPU rendering possibility
Adding a new GraphicsContext is what I want to do as it seems the path of least pain and suffering. However, all the other things of a backend I do not need to do. I do not know how to add a GraphicsContext backend in terms of makefile magicks and configuration. I also do not know the plumbing for making it active. In theory, FastUIDraw's GraphicsContext will work on any platform that does OpenGL 3.3 or OpenGL ES 3.0. What is the plumbing to do this? Years ago I remember that the build configuration is what governed what backend was built... and I usually just piggy packed onto another... years ago I remember there was like an SDL style backend that did not require a large toolkit, just SDL.. is that still alive? where is it? I could piggy back the work there if it still is alive... Also, to get permission to do this work, I need significant community enthusiasm otherwise I will not be able to justify the large amount of work needed. This is another area where I need a great deal of help. Best Regards, -Kevin Rogovin -Original Message- From: Carlos Garcia Campos [mailto:carlo...@webkit.org] Sent: Thursday, November 3, 2016 9:43 AM To: Rogovin, Kevin ; Myles C. Maxfield Cc: webkit-dev@lists.webkit.org Subject: Re: [webkit-dev] WebKit GPU rendering possibility El jue, 03-11-2016 a las 07:35 +0000, Rogovin, Kevin escribió: > Hi, > > The main issue of making a Cairo backend to FastUIDraw is clipping. > Cairo tracks the clipping region in CPU and does things that are fine > for CPU-based rendering (i.e. span based rendering) but are absolutely > awful for GPU rendering (from my slides, one sees that GL backed > QPainter and Cairo do much worse than CPU backed). FastUIDraw only > supports clipIn and clipOut and pushes all the clipping work to the > GPU with almost no CPU work. It does NOT track the clipping region at > all. I can give more technical details how it works (and those details > are why FastUIDraw cannot be used a backend for Cairo). > For those interested in where the code is located for clipping in > FastUIDraw, it is located at src/fastuidraw/painter/painter.cpp, > methods clipInRect, clipOutPath and clipInPath. Their implementations > are very short and simple and are quite cheap on CPU. I see. Then I guess adding a new GraphicsContext for FastUIDraw is the easiest and best way to try this out in WebKit. Would it be possible to just add a new GraphicsContext implementation? or would you also need to change other parts of the graphics implementation or the GraphicsContext API itself? > Best Regards, > -Kevin > > -Original Message- > From: Carlos Garcia Campos [mailto:carlo...@webkit.org] > Sent: Thursday, November 3, 2016 9:27 AM > To: Rogovin, Kevin ; Myles C. Maxfield fi...@apple.com> > Cc: webkit-dev@lists.webkit.org > Subject: Re: [webkit-dev] WebKit GPU rendering possibility > > El jue, 03-11-2016 a las 06:58 +, Rogovin, Kevin escribió: > > Hi! > > > > Question answers: > > 1. Currently FastUIDraw has a backend to OpenGL 3.3 and OpenGL > > ES 3.0. One of its design goals is to make it not terribly awful to > > write a backend to different 3D API’s. > > 2. I think I was unclear in my video. I have NOT migrated ANY > > UI rendering library to use Fast UI Draw. What I have done is made a > > demo > > (painter-cells) and ported that demo to Fast UI Draw, Cairo, Qt’s > > QPainter and SKIA. The diffs between the ports is almost trivial (it > > really is just using those different rendering API’s). > > That makes me wonder, would it be possible to add a new cairo backend > based on FastUIDraw? That would make very easy to try it out with the > current GraphicsContext cairo backend. > > > 3. There are a few areas: > > a. Reduce some render to offscreen buffers. When I worked with > > WebKit YEARS ago, I saw a few instances of rendering to texture that > > are unnecessary and even harm performance for GPU rendering. The > > first example was where a brush pattern with an image and/or > > gradient applied is to be drawn tiled across an area. WebKit (at > > that time) first drew a single instance of that pattern to an image, > > then drew that image tiled. For GPU renderers we can (very easily) > > just do the repeat pattern (of both original image and gradient) > > from a shader. > > Another instance happens at RenderLayer where a new GraphicsContext > > “layer” is started on a transformation that has rotation or > > perspective. For FastUIDraw, this is not necessary, though if a > > layer is transparent, then it is. > > b. In addition, FastUIDraw has an interface so that if “what” > > is > > drawn is unchanged but the “how”
Re: [webkit-dev] WebKit GPU rendering possibility
Hi, The main issue of making a Cairo backend to FastUIDraw is clipping. Cairo tracks the clipping region in CPU and does things that are fine for CPU-based rendering (i.e. span based rendering) but are absolutely awful for GPU rendering (from my slides, one sees that GL backed QPainter and Cairo do much worse than CPU backed). FastUIDraw only supports clipIn and clipOut and pushes all the clipping work to the GPU with almost no CPU work. It does NOT track the clipping region at all. I can give more technical details how it works (and those details are why FastUIDraw cannot be used a backend for Cairo). For those interested in where the code is located for clipping in FastUIDraw, it is located at src/fastuidraw/painter/painter.cpp, methods clipInRect, clipOutPath and clipInPath. Their implementations are very short and simple and are quite cheap on CPU. Best Regards, -Kevin -Original Message- From: Carlos Garcia Campos [mailto:carlo...@webkit.org] Sent: Thursday, November 3, 2016 9:27 AM To: Rogovin, Kevin ; Myles C. Maxfield Cc: webkit-dev@lists.webkit.org Subject: Re: [webkit-dev] WebKit GPU rendering possibility El jue, 03-11-2016 a las 06:58 +, Rogovin, Kevin escribió: > Hi! > > Question answers: > 1. Currently FastUIDraw has a backend to OpenGL 3.3 and OpenGL ES > 3.0. One of its design goals is to make it not terribly awful to write > a backend to different 3D API’s. > 2. I think I was unclear in my video. I have NOT migrated ANY UI > rendering library to use Fast UI Draw. What I have done is made a demo > (painter-cells) and ported that demo to Fast UI Draw, Cairo, Qt’s > QPainter and SKIA. The diffs between the ports is almost trivial (it > really is just using those different rendering API’s). That makes me wonder, would it be possible to add a new cairo backend based on FastUIDraw? That would make very easy to try it out with the current GraphicsContext cairo backend. > 3. There are a few areas: > a. Reduce some render to offscreen buffers. When I worked with > WebKit YEARS ago, I saw a few instances of rendering to texture that > are unnecessary and even harm performance for GPU rendering. The first > example was where a brush pattern with an image and/or gradient > applied is to be drawn tiled across an area. WebKit (at that time) > first drew a single instance of that pattern to an image, then drew > that image tiled. For GPU renderers we can (very easily) just do the > repeat pattern (of both original image and gradient) from a shader. > Another instance happens at RenderLayer where a new GraphicsContext > “layer” is started on a transformation that has rotation or > perspective. For FastUIDraw, this is not necessary, though if a layer > is transparent, then it is. > b. In addition, FastUIDraw has an interface so that if “what” is > drawn is unchanged but the “how” changes, then a caller can cache the > “what” to send to the GPU. To be explicit, “what” to draw is > essentially attributes and indices and those values do NOT depend on > the state of “how” to draw. Examples of “how” to draw: current > transformation, brush to apply, clipping applied, stroking parameters > (including dash pattern) and blending mode. I admit that I am quite > proud of being able to use the same attributes an indices even if > stroking parameters (stroking width, miter limit and dash pattern) > change. Text rendering “what” to draw does depend on what glyphs one > wants to use. Specifically, if drawing coverage font glyphs, then > attributes and indices values change if one wants to draw the glyph > biffer, but for the GPU rendered glyphs they do not. > 4. The renderer implements full 3x3 transformations. However, the > renderer does NOT implement out-of-order transparency. For a GPU, a > 3x3 transformation is cheap (naturally!). The renderer does handle, > with a very little additional overhead changing clipping even between > nasty rotations or perspective changes. The demo painter-cells > deliberately pushes and does lots of nasty clipping and the > performance impact of it on FastUIDraw is very small. > 5. Drawing text is a right pain in the rear. Currently, > FastUIDraw has 3 methods to draw text: coverage, distance field and an > original GPU algorithm that I devised for another open sourced project > years ago. Coverage is needed when glyphs are drawn small and hinting > becomes important. The original GPU algorithm keeps corners sharps and > does a computation in the fragment shader to compute a coverage value. > Distance field is a fall back which has render quality issues (namely > corners are rounded) but is very, very cheap. > I want to write an additional glyph renderer that is much faster than > the original GPU method and ke
Re: [webkit-dev] WebKit GPU rendering possibility
get the better my chances for continuing the work. Best Regards, -Kevin Rogovin From: Myles C. Maxfield [mailto:mmaxfi...@apple.com] Sent: Thursday, November 3, 2016 1:30 AM To: Rogovin, Kevin Cc: webkit-dev@lists.webkit.org Subject: Re: [webkit-dev] WebKit GPU rendering possibility Hello, This is certainly interesting work! I have a few questions about the approach of this renderer. 1. What API is this on top of? OpenGL? Metal? Vulkan? Raw GPU commands[1]? 2. You mention in your video that you have already migrated Cairo on top of your new tech. Traditionally, a web engine is divorced from a 2D rendering engine such as Cairo. Why can’t the ports of WebKit which use Cairo get this new tech without any change? 3. What sort of API changes do you have in mind to make GraphicsContext adopt? 4. Out of curiosity, does the renderer implement 3D transforms? Did you have to implement 3-D triangle subdivision along intersections (perhaps for order-independent transparency)? 5. Which algorithm did you choose to draw text? Historically, the WebKit team has hesitated to allow experiments in the OpenSource repository. Traditionally, this sort of exploratory work is done in a branch, and only after it has proved to be an improvement, the work is adopted on trunk. Thanks, Myles [1] https://01.org/linuxgraphics/documentation/hardware-specification-prms On Nov 2, 2016, at 9:35 AM, Rogovin, Kevin mailto:kevin.rogo...@intel.com>> wrote: Hi, I was directed here by some colleagues as this is the place to post the following to get started on the following proposal. I have been working on an experimental 2D renderer that requires a GPU, the project is open sourced on github at https://github.com/01org/fastuidraw. I gave a talk at the X Developers Conference this year which can be seen from https://www.x.org/wiki/Events/XDC2016/Program/rogovin_fast_ui_draw/ . I made a benchmark which makes heavy use of rotations and clipping and ported to SKIA, Qt’s QPainter and Cairo. The benchmark and its ports are in the git repo linked above under the branch with_ports_of_painter-cells. It's performance advantage of FastUIDraw against the other renderers was quite severe (against Cairo and Qt's QPainter over 9 times and against SKIA about 5 times faster). I would like to explore the option of using FastUIDraw to implement a WebCore::GraphicsContext backend for the purpose of making drawing faster and more efficient on Intel devices that are equipped with a GPU. I also think that some minor modifications to WebKit’s use of GraphicsContext will also give some benefits. I have worked on WebKit a few years ago and knew/know my way around the rendering code very well (atleast at that time). Looking forward to collaboration, -Kevin Rogovin ___ webkit-dev mailing list webkit-dev@lists.webkit.org<mailto:webkit-dev@lists.webkit.org> https://lists.webkit.org/mailman/listinfo/webkit-dev ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
[webkit-dev] WebKit GPU rendering possibility
Hi, I was directed here by some colleagues as this is the place to post the following to get started on the following proposal. I have been working on an experimental 2D renderer that requires a GPU, the project is open sourced on github at https://github.com/01org/fastuidraw. I gave a talk at the X Developers Conference this year which can be seen from https://www.x.org/wiki/Events/XDC2016/Program/rogovin_fast_ui_draw/ . I made a benchmark which makes heavy use of rotations and clipping and ported to SKIA, Qt's QPainter and Cairo. The benchmark and its ports are in the git repo linked above under the branch with_ports_of_painter-cells. It's performance advantage of FastUIDraw against the other renderers was quite severe (against Cairo and Qt's QPainter over 9 times and against SKIA about 5 times faster). I would like to explore the option of using FastUIDraw to implement a WebCore::GraphicsContext backend for the purpose of making drawing faster and more efficient on Intel devices that are equipped with a GPU. I also think that some minor modifications to WebKit's use of GraphicsContext will also give some benefits. I have worked on WebKit a few years ago and knew/know my way around the rendering code very well (atleast at that time). Looking forward to collaboration, -Kevin Rogovin ___ webkit-dev mailing list webkit-dev@lists.webkit.org https://lists.webkit.org/mailman/listinfo/webkit-dev
