[issue20479] Efficiently support weight/frequency mappings in the statistics module

Steven D'Aprano added the comment: Here is some further information on weights in statistics in general, and SAS and Stata specifically: https://blogs.sas.com/content/iml/2017/10/02/weight-variables-in-statistics-sas.html Quote: use the FREQ statement to specify integer frequencies for

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Oscar Benjamin added the comment: Sorry, sent too soon... > Matlab doesn't support even weighted mean as far as I can tell. There > is wmean on the matlab file exchange: https://stackoverflow.com/a/36464881/9450991 This is a separate function `wmean(data, weights)`. It has to be a separate

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Oscar Benjamin added the comment: > I would find it very helpful if somebody has time to do a survey of > other statistics libraries or languages (e.g. numpy, R, Octave, Matlab, > SAS etc) and see how they handle data with weights. Numpy has only sporadic support for this. The standard mean

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Steven D'Aprano added the comment: > Is this proposal still relevant? Yes. As Raymond says, deciding on a good API is the hard part. Its relatively simple to change a poor implementation for a better one, but backwards compatibility means that changing the API is very difficult. I would

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Raymond Hettinger added the comment: > Is this proposal still relevant? If so, I would > like to work on its implementation. The first question is the important one. Writing implementations is usually the easy part. Deciding on whether there is a real need and creating a usable,

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Rémi Lapeyre added the comment: Is this proposal still relevant? If so, I would like to work on its implementation. I think the third proposition to change the API to have a new `weights` parameter is the best has it does not blindly suppose that a tuple is a pair (value, weight) which

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Change by Rémi Lapeyre : -- versions: +Python 3.8 -Python 3.7 ___ Python tracker ___ ___ Python-bugs-list mailing list Unsubscribe:

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Nick Coghlan added the comment: Thinking back to my signal processing days, I have to agree that our weightings (filter definitions) were usually separate from our data (live signals). Similarly, systems engineering trade studies all maintained feature weights separately

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Raymond Hettinger added the comment: My recommendation is to have *weights* as an optional argument: statistics.mean(values, weights=None) While it is tempting to special case dicts and counters, I got feedback from Jake Vanderplas and Wes McKinney that in

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Change by Serhiy Storchaka : -- nosy: -serhiy.storchaka ___ Python tracker ___

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Wolfgang Maier added the comment: Well, I was thinking about frequencies (ints) when suggesting for x,m in data.items(): T = _coerce_types(T, type(x)) n, d = exact_ratio(x) partials[d] = partials_get(d, 0) + n*m in my previous message. To support weights (float or

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Oscar Benjamin added the comment: in my previous message. To support weights (float or Rational) this would have to be more sophisticated. I guess you'd do: for x,w in data.items(): T = _coerce_types(T, type(x)) xn, xd = exact_ratio(x) wn, wd = exact_ratio(w)

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Steven D'Aprano added the comment: Off the top of my head, I can think of three APIs: (1) separate functions, as Nick suggests: mean vs weighted_mean, stdev vs weighted_stdev (2) treat mappings as an implied (value, frequency) pairs (3) take an additional argument to switch between unweighted

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Oscar Benjamin added the comment: On 2 February 2014 11:55, Steven D'Aprano rep...@bugs.python.org wrote: (1) separate functions, as Nick suggests: mean vs weighted_mean, stdev vs weighted_stdev This would be my preferred approach. It makes it very clear which functions are available for

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Serhiy Storchaka added the comment: See also issue18844. -- nosy: +serhiy.storchaka ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20479 ___ ___

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Wolfgang Maier added the comment: -Ursprüngliche Nachricht- Von: Steven D'Aprano [mailto:rep...@bugs.python.org] Gesendet: Sonntag, 2. Februar 2014 12:55 An: wolfgang.ma...@biologie.uni-freiburg.de Betreff: [issue20479] Efficiently support weight/frequency mappings

[issue20479] Efficiently support weight/frequency mappings in the statistics module

Changes by Nick Coghlan ncogh...@gmail.com: -- dependencies: +Avoid inadvertently special casing Counter in statistics module versions: +Python 3.5 ___ Python tracker rep...@bugs.python.org http://bugs.python.org/issue20479

[issue20479] Efficiently support weight/frequency mappings in the statistics module

New submission from Nick Coghlan: Issue 20478 suggests ensuring that even weight/frequency mappings like collections.Counter are consistently handled as iterables in the current statistics module API. However, it likely makes sense to provide public APIs that support efficiently working with