Check all four box corners in SP-GiST quad tree <@ search

When the centroid lies outside the query box, spg_quad_inner_consistent()
keeps only the quadrants that contain a corner of the box.  It meant to
test all four corners, but the last step set x back to low.x instead of
y to low.y, so it tested (low.x, high.y) twice and never (high.x, low.y).

With exact comparisons the missing corner would not matter: a box that
does not contain the centroid lies on one side of it on at least one
axis, so (high.x, low.y) shares a quadrant with another corner, but
getQuadrant() compares with FPeq(), which tiebreaks to the lowest
numbered quadrant.

When a box edge lies within EPSILON of the centroid, the corner on that
edge counts as being on the centroid's line, and (high.x, low.y) may be
the only corner in its quadrant.  That quadrant was then skipped.  Fix
the missing corner test.

Discussion: 
https://postgr.es/m/CANWCAZZZpkwrX0kC4Jo6KM27J6dyBdCDir-PbjEYAS06r%3D4-ag%40mail.gmail.com
Backpatch-through: 14

Branch
------
REL_17_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/9ad421a9b970b672136fc681f2001876dec01072

Modified Files
--------------
src/backend/access/spgist/spgquadtreeproc.c | 2 +-
src/test/regress/expected/spgist.out        | 9 +++++++++
src/test/regress/sql/spgist.sql             | 5 +++++
3 files changed, 15 insertions(+), 1 deletion(-)

Reply via email to