On Mon, Jan 10, 2022 at 6:51 AM Ramin Halavati <[email protected]> wrote:
> Thank you Philip. > > Since this accessibility support function is called for all renderer types > (in ChromeOS including Arc++ and Ash), there would be a better consistency > if we have the snapshotting code in the renderer. Therefore I am trying to > avoid using the browser side ui::GrabViewSnapshot (as in DevTools), unless > the alternative approach proves to be too complicated or unnecessarily > heavy. > > I tried using DataTransfer::NodeImage in this draft CL > <http://crrev.com/c/3370381>, but my understanding is that since it > triggers a repaint, we should make sure it is not in the middle of a > lifecycle update to avoid conflicts. Is there a notification of a > signal that tells when it's safe to request it? > It is important that we do not introduce lifecycle violations but I think that may already be guaranteed from when the accessibility code runs. For example, AxObject::GetRelativeBounds cannot be called from within a lifecycle update either. You can probably just synchronously call the paint function you need. > I assume it's the same for the SpoolSinglePage as it also redraws the > image. And performance wise, can we say that as this approach triggers a > repaint, it's heavier than the browser based snapshotting (using > ui::GrabViewSnapshot), as the latter is using the already painted output > and does not require a repaint? > The paint code in NodeImage is really just a walk of layout datastructures and isn't more expensive than things like layout, style, etc (i.e., it's O(|layout|)). Doing this for each object would be wildly inefficient though. Expanding on the thing about this being an ambiguous problem: data:text/html,<body style="background: rebeccapurple; filter: blur(1px)"><div style="height: 100px; width: 100px; border: 1px solid white; color:white;" draggable=true>hello world</div> The drag image for the div will only contain white and transparent pixels, and will not be blurred. If you want the purple background too, you'll want to do the full-page-paint+crop approach. > > Best, > Ramin > > On Fri, Jan 7, 2022 at 11:06 PM Philip Rogers <[email protected]> wrote: > >> This is a deceptively difficult problem in general. Our paint system uses >> the CSS definition of paint order which starts painting at stacking >> contexts and it is challenging to exclude non-stacked siblings of a >> non-stacked target element. Some other complications are whether ancestor >> effects/clips should be applied, and whether content behind the target >> should be included. >> >> DataTransfer::NodeImage is an attempt at this. Another option is to get >> the entire page painted (examples: devtools uses >> PageHandler::CaptureScreenshot and printing uses SpoolSinglePage), then >> crop it to the target element. >> >> On Fri, Jan 7, 2022 at 7:01 AM 'Ramin Halavati' via blink-dev < >> [email protected]> wrote: >> >>> Hi, >>> >>> I am trying to find a way to get pixels for any general node in >>> *third_party/blink/renderer/modules/accessibility. *We already have >>> AXNodeObject::ImageDataUrl >>> <https://source.chromium.org/chromium/chromium/src/+/main:third_party/blink/renderer/modules/accessibility/ax_node_object.cc;drc=a3ae56ec162fcbf9ee400e9ff80f1ae572957fd3;bpv=1;bpt=1;l=2355?gsn=ImageDataUrl&gs=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dsrc%2Fthird_party%2Fblink%2Frenderer%2Fmodules%2Faccessibility%2Fax_node_object.cc%23rau1_dUeeJHt__0C5nqtyH5ouOMnhdEN55IRAHfVRMk&gs=kythe%3A%2F%2Fchromium.googlesource.com%2Fchromium%2Fsrc%3Flang%3Dc%252B%252B%3Fpath%3Dsrc%2Fthird_party%2Fblink%2Frenderer%2Fmodules%2Faccessibility%2Fax_node_object.h%23z0USMj0KE_TnmxgTOdb4D-K1NmRUt1i1gyvtbSdYs5w> >>> function >>> to return the bitmap of a certain node, but it is only supporting HTML >>> element types that already include a picture (namely image, canvas, video) >>> and I am trying to expand to to any general element type. >>> >>> So far I've been looking for a way to get access to a painter or >>> compositor, but I did not find any promising solution yet. Can you point me >>> in the right direction? Is it even expected at this abstraction level? >>> >>> Thanks, >>> Ramin >>> >>> P.S., Sorry for naive questions and please point me to the right email >>> thread if this is not. >>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "blink-dev" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to [email protected]. >>> To view this discussion on the web visit >>> https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAORodQi_LSUFXWxpfAKQKKVhRzK1UvGVz%2B4unQUN2qskFSSafw%40mail.gmail.com >>> <https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAORodQi_LSUFXWxpfAKQKKVhRzK1UvGVz%2B4unQUN2qskFSSafw%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- You received this message because you are subscribed to the Google Groups "blink-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/a/chromium.org/d/msgid/blink-dev/CAJgFLLu_0_g5EkUTyQMTYjWW6T3bRZeRZLhv0anLQqXC98yC4g%40mail.gmail.com.
