Alex Herbert created STATISTICS-100:
---------------------------------------
Summary: Zipf distribution can use the Hurwitz zeta function to
avoid summation over all PDF values
Key: STATISTICS-100
URL: https://issues.apache.org/jira/browse/STATISTICS-100
Project: Commons Statistics
Issue Type: Improvement
Components: distribution
Affects Versions: 1.3
Reporter: Alex Herbert
Assignee: Alex Herbert
The [Zipf distribution
(Wikipedia)|https://en.wikipedia.org/wiki/Zipf%27s_law#Formal_definition] uses
a probability mass function (PMF) of:
{noformat}
1 1
pmf(k; N, s) = -- . ----
s H
k N,s
{noformat}
The normalising constant H_N,s is the N-th generalised harmonic number of order
N of s.
The cumulative probability for range [a, b] is computed by summation of the
power term k^-s for k in [a, b]. This can be millions of power terms for one
cumulative probability when the support [1, N] is large.
When s is above 1 the cumulative probability can use the [Hurwitz zeta function
(Wikipedia)|https://en.wikipedia.org/wiki/Hurwitz_zeta_function]:
{noformat}
oo 1
zeta(s, a) = sum ------
k=0 s
(k+a)
cdf(k; N, s) = (zeta(s, 1) - zeta(s, k+1)) / H_N,s
sf(k; N, s) = (zeta(s, k+1) - zeta(s, N+1)) / H_N,s
H_N,s = zeta(s, 1) - zeta(s, N+1)
{noformat}
The Hurwitz zeta function for s > 1 is absolutely convergent. This function can
be used to significantly improve the performance of the Zipf distribution.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
