[
https://issues.apache.org/jira/browse/NIFI-12389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess reassigned NIFI-12389:
-----------------------------------
Assignee: Matt Burgess
> Add variance and standard deviation to AttributeRollingWindow
> -------------------------------------------------------------
>
> Key: NIFI-12389
> URL: https://issues.apache.org/jira/browse/NIFI-12389
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Matt Burgess
> Assignee: Matt Burgess
> Priority: Major
>
> AttributeRollingWindow currently uses state to calculate metrics like sum,
> count, and mean (average) for a rolling time window (optionally using
> sub-windows to use smaller aggregates to estimate metrics for a larger total
> time window as it stores the window values in state).
> Variance and its positive square root (standard deviation) are other helpful
> univariate statistics. Using "online" algorithms such as Welford's method
> adds no extra memory requirement and runs in constant time. Apache Commons
> Math3 has such an implementation and could be used to add attributes to the
> FlowFile for variance and standard deviation. These metrics can be used for
> filtering out noise downstream, for example RouteOnAttribute and/or stateful
> UpdateAttribute where the current attribute value is more than 2 standard
> deviations away from the mean.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
