Hello, Thanks for the summary of the PR Sebastian.
About the default value of python `quantile` being "exclusive". They give some explanation about why it is their default, but only in a commented bloc above the code of `quantile` and not in the actual documentation. You can see it here: https://github.com/python/cpython/blob/main/Lib/statistics.py#L615 For reference the documentation is here: https://docs.python.org/3/library/statistics.html#statistics.quantiles By the way, it is method 6 which is named "exclusive" and method 7 named "inclusive" by python. And "inclusive" and "exclusive" names seem to come from excel. Sincerely, Abel ----- Mail original ----- De: "Sebastian Berg" <sebast...@sipsolutions.net> À: "Discussion of Numerical Python" <numpy-discussion@python.org> Envoyé: Mercredi 13 Octobre 2021 17:25:19 Objet: [Numpy-discussion] Percentile/Quantile "interpolation" refactor Hi all, after a long time Abel has helped us and refactored the quantile and percentile functions' `interpolation` keyword. This was long overdue since NumPy implements three (the non-default) interpolation methods that appear to be very much non-standard. On the other hand, NumPy currently has no unbiased methods (i.e. population estimate). There are two main questions right now with respect to the API. First which names to use for the methods and second, how to deal with "outliers". The PR https://github.com/numpy/numpy/pull/19857#issuecomment-939852134 adds the methods and gives them (currently) the following names (sorted by the R methods) – the names will be used as string identifiers: 1. inverted cdf 2. averaged inverted cdf 3. closest observation 4. interpolated inverted cdf 5. hazen (name from wolfram) 6. weibull (name from wolfram) 7. linear (default! Better name deferred) 8. median unbiased 9. normal unbiased And additionally the four ones we currently have: * lower * higher * nearest * midpoint Number 5. and 6. are named "exclusive" and "inclusive" by Python in their `method` keyword argument. While I like the name `method=` and may want to move to it, I am not sure I like "inclusive" and "exclusive". The current plan was to defer the kwarg rename into a followup, although it should be discussed before the next release. The second main question is how to deal with outliers (this does not affect the default method 7, which finds the sample quantiles and not a population estimate). Wikipedia says this: Packages differ in how they estimate quantiles beyond the lowest and highest values in the sample, i.e. p < 1/N and p > (N − 1)/N. Choices include returning an error value, computing linear extrapolation, or assuming a constant value. The current choice is clipping (assuming a constant value), but this could be modified. Any feedback is appreciated! Otherwise, this will probably move forward in the current state for the next release. Cheers, Sebastian _______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: a...@cerfacs.fr _______________________________________________ NumPy-Discussion mailing list -- numpy-discussion@python.org To unsubscribe send an email to numpy-discussion-le...@python.org https://mail.python.org/mailman3/lists/numpy-discussion.python.org/ Member address: arch...@mail-archive.com
