Hello Tomas,

I've been looking at the checkpoint patches (sorting, flush and FPW compensation) and realized we got gaussian/exponential distributions in pgbench which is useful for simulating simple non-uniform workloads.


But I think the current docs is a bit too difficult to understand for people without deep insight into statistics and shapes of probability distributions.

I think the idea is that (1) if you do not know anything distributions, probably you do not want expo/gauss, and (2) pg documentation should not try to be an introductory course in probability theory.

AFAICR I suggested to point to relevant wikipedia pages but this has been more or less rejected, so it ended up as it is, which is indeed pretty unconvincing.

Firstly, it'd be nice if we could add some figures illustrating the distributions - much better than explaining the shapes in text. I don't know if we include figures in the existing docs (probably not), but generating the figures is rather simple.

There is basically no figures in the documentation. Too bad, but it is understandable: what should be the format (svg, jpg, png, ...), should it be generated (gnuplot, others), what is the impact on the documentation build (html, epub, pdf, ...), how portable should it be, what about compressed formats vs git diffs?

Once you start asking these questions you understand why there are no figures:-)

A few more comments:

By default, or when uniform is specified, all values in the range are
drawn with equal probability. Specifying gaussian or exponential
options modifies this behavior; each requires a mandatory threshold
which determines the precise shape of the distribution.

I find the 'threshold' name to be rather unfortunate, as none of the probability distribution functions that I know use this term.

I think that it was proposed for gaussian, not sure why.

And even if there's one probability function that uses 'threshold' it has very little meaning in the others. For example the exponential distribution uses 'rate' (lambda). I'd prefer a neutral name (e.g. 'parameter').

Why not. Many places to fix, though (documentation & source code).

For a Gaussian distribution, the interval is mapped onto a standard
normal distribution (the classical bell-shaped Gaussian curve)
truncated at -threshold on the left and +threshold on the right.

Probably nitpicking, but left/right of what? I assume the normal distribution is placed at 0, so it's left/right of zero.

No, it is around the middle of the interval.

To be precise, if PHI(x) is the cumulative distribution function of
the standard normal distribution, with mean mu defined as (max + min)
/ 2.0, then value i between min and max inclusive is drawn with
probability: (PHI(2.0 * threshold * (i - min - mu + 0.5) / (max -
min + 1)) - PHI(2.0 * threshold * (i - min - mu - 0.5) / (max - min +
1))) / (2.0 * PHI(threshold) - 1.0). Intuitively, the larger the
threshold, the more frequently values close to the middle of the
interval are drawn, and the less frequently values close to the min
and max bounds.

Could we simplify the equation a bit? It's needlessly difficult to realize it's actually just CDF(i+0.5) - CDF(i-0.5). I think it'd be good to first define the CDF and then just use that.

ISTM that PHI is *the* normal CDF, which is more or less available as such in various environment (matlab, python, excel...). Well, why not defined the particular CDF and use it. Not sure the text would be that much lighter, though.

About 67% of values are drawn from the middle 1.0 / threshold and 95%
in the middle 2.0 / threshold; for instance, if threshold is 4.0, 67%
of values are drawn from the middle quarter and 95% from the middle
half of the interval.

This seems broken - too many sentences about the 67% and 95%.

The point is to provide rules of thumb to describe how the distribution is shaped. Any better sentence is welcome.

The minimum threshold is 2.0 for performance of the Box-Muller

Does it make sense to explicitly mention the implementation detail (Box-Muller transform) here?

It is too complex, I would avoid it. I would point to the wikipedia page if that could be allowed.



Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:

Reply via email to