[
https://issues.apache.org/jira/browse/BEAM-2953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Reza ardeshir rokni updated BEAM-2953:
--------------------------------------
Description:
A general set of timeseries transforms that abstract the user from the process
of dealing with some of the common problems when dealing with timeseries using
BEAM (in stream or batch mode).
BEAM can be used to build out some very interesting pre-processing stages for
time series data. Some examples that will be useful:
- Downsampling time series based on simple MIN, MAX, COUNT, SUM, LAST, FIRST
- Creating a value for each downsampled window even if no value has been
emitted for the specific key.
- Loading the value of a downsample with the previous value (used in FX with
previous close being brought into current open value)
was:
As described in the phase 1 portion of this solution outline:
https://cloud.google.com/solutions/correlating-time-series-dataflow
BEAM can be used to build out some very interesting pre-processing stages for
time series data. Some examples that will be useful:
- Downsampling time series based on simple AVG, MIN, MAX
- Creating a value for each time window using generatesequence as a seed
- Loading the value of a downsample with the previous value (used in FX with
previous close being brought into current open value)
This will show some concrete examples of keyed state as well as the use of
combiners.
The samples can also be used to show how you can create a ordered list of
values per key from a unbounded topic which has multiple time series keys.
> Create more advanced Timeseries processing examples using state API
> -------------------------------------------------------------------
>
> Key: BEAM-2953
> URL: https://issues.apache.org/jira/browse/BEAM-2953
> Project: Beam
> Issue Type: Improvement
> Components: examples-java
> Affects Versions: 2.1.0
> Reporter: Reza ardeshir rokni
> Assignee: Reuven Lax
> Priority: Minor
> Time Spent: 3h 10m
> Remaining Estimate: 0h
>
> A general set of timeseries transforms that abstract the user from the
> process of dealing with some of the common problems when dealing with
> timeseries using BEAM (in stream or batch mode).
> BEAM can be used to build out some very interesting pre-processing stages for
> time series data. Some examples that will be useful:
> - Downsampling time series based on simple MIN, MAX, COUNT, SUM, LAST, FIRST
> - Creating a value for each downsampled window even if no value has been
> emitted for the specific key.
> - Loading the value of a downsample with the previous value (used in FX with
> previous close being brought into current open value)
>
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
