Hi, 7cb9060dcde taught pg_restore_relation_stats() to reject a non-finite reltuples, on the grounds that Infinity and NaN pass the existing range check and then get stored and used verbatim. pg_restore_attribute_stats() has the same gap for its float arguments, which that thread did not cover.
CREATE TABLE t (a int);
INSERT INTO t SELECT g FROM generate_series(1, 1000) g;
ANALYZE t;
SELECT pg_restore_attribute_stats('schemaname', 'public', 'relname', 't',
'attname', 'a', 'inherited', false,
'null_frac', 'NaN'::real, 'n_distinct', 'Infinity'::real);
t
SELECT stanullfrac, stadistinct FROM pg_statistic
WHERE starelid = 't'::regclass;
NaN | Infinity
The values are stored, and the planner does not defend against them.
CLAMP_PROBABILITY() is two comparisons, both false for NaN, so it does not
neutralise a non-finite value the way it clamps an out-of-range finite one.
The effect is visible immediately:
-- with the NaN null_frac above:
EXPLAIN SELECT * FROM t WHERE a = 5;
Seq Scan on t (cost=0.00..17.50 rows=10000000000...000 width=4)
A non-finite n_distinct or correlation is worse than a bad row estimate: a
NaN correlation puts a literal "cost=0.29..NaN" on an index scan, which
then takes part in path cost comparisons.
This only comes in through the restore path -- ANALYZE never produces a
non-finite value, even for a column that itself contains Infinity/NaN,
since the stats are frequencies and ratios rather than the data. So the
realistic trigger is a corrupt or cross-version dump fed through
pg_restore_attribute_stats(), and once stored the value survives until the
next ANALYZE.
Patch attached. It rejects non-finite values for the scalar arguments
null_frac, n_distinct, correlation and range_empty_frac, and for the
float4[] arguments most_common_freqs, most_common_elem_freqs and
elem_count_histogram, dropping the bad value with a WARNING as the other
non-fatal checks do and letting the rest of the import proceed. The two
new checks live in stat_utils.c alongside the existing ones.
Two things I decided deliberately, happy to be overruled:
- A negative n_distinct encodes a distinct-value ratio rather than a
count, so it is still accepted, matching the -1.0 special case kept for
reltuples.
- These functions do only superficial validation by design (per
ce207d2a790), so I did not add range checks for finite-but-bogus values;
the planner does clamp those. This only closes the non-finite hole,
which the planner cannot.
--
Regards,
Ewan Young
v1-0001-Reject-non-finite-values-when-restoring-attribute-st.patch
Description: Binary data
