On Mon, 15 Jun 2020 at 17:40, BALATON Zoltan <bala...@eik.bme.hu> wrote: > > On Mon, 15 Jun 2020, Peter Maydell wrote: > > The calculations for sb/se/db/de all have a term which > > multiplies by (width + pitch), which makes me suspect > > they also need a similar fix ? > > Maybe. I'll have to check again. Actually is there a simpler way to check > if two rectangles overlap when they are given with base, x, y, w, h, pitch > where base is the first byte of the screen, pitch is length of one line > and x,y is coordinates of top left corner and w,h is dimensions of the > rect. Now that I think about it we also need to take into accounf the > bytes per pixel value (1 << format) because base is given in bytes while > others are in pixels so these formulae likely need some fixes. Pixman has > some functions for these but those assume common base so to use those we > would need to bring the two rectangles to common base which I could not > find out how to do. Probably this is really simple for someone who already > did a lot of these before.
I think the thing that makes it particularly awkward is that the source and dest can have different pitches. That means it's not a simple "do two rectangles overlap" test because the dest area might not be a rectangle at all when looked at from the POV of the source. Do guests usually set src and dst pitch identical? If so it might be worth having a more accurate rectangle-overlap test for the common case and a looser check for the hard-to-handle case. I might have a think about this and draw some diagrams tomorrow :-) thanks -- PMM
