Check all four box corners in SP-GiST quad tree <@ search

When the centroid lies outside the query box, spg_quad_inner_consistent()
keeps only the quadrants that contain a corner of the box.  It meant to
test all four corners, but the last step set x back to low.x instead of
y to low.y, so it tested (low.x, high.y) twice and never (high.x, low.y).

With exact comparisons the missing corner would not matter: a box that
does not contain the centroid lies on one side of it on at least one
axis, so (high.x, low.y) shares a quadrant with another corner, but
getQuadrant() compares with FPeq(), which tiebreaks to the lowest
numbered quadrant.

When a box edge lies within EPSILON of the centroid, the corner on that
edge counts as being on the centroid's line, and (high.x, low.y) may be
the only corner in its quadrant.  That quadrant was then skipped.  Fix
the missing corner test.

Discussion: 
https://postgr.es/m/CANWCAZZZpkwrX0kC4Jo6KM27J6dyBdCDir-PbjEYAS06r%3D4-ag%40mail.gmail.com
Backpatch-through: 14

Branch
------
REL_16_STABLE

Details
-------
https://git.postgresql.org/pg/commitdiff/9cba87fe947fc3e713bb4ba3be25e34fdb64266d

Modified Files
--------------
src/backend/access/spgist/spgquadtreeproc.c | 2 +-
src/test/regress/expected/spgist.out        | 9 +++++++++
src/test/regress/sql/spgist.sql             | 5 +++++
3 files changed, 15 insertions(+), 1 deletion(-)

Reply via email to