[
https://issues.apache.org/jira/browse/STREAMPIPES-292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17298712#comment-17298712
]
Janvi Talreja commented on STREAMPIPES-292:
-------------------------------------------
Hi,
I am Janvi, currently a senior at SKITM, Indore, India.
I have been working with Java for quite a long time now. I am interested in
working on this issue as a part of Google Summer of Code 2021.
> Introduce event windowing to the StreamPipes core/sdk
> -----------------------------------------------------
>
> Key: STREAMPIPES-292
> URL: https://issues.apache.org/jira/browse/STREAMPIPES-292
> Project: StreamPipes
> Issue Type: New Feature
> Components: Backend
> Reporter: Grainier Perera
> Priority: Major
> Labels: gsoc2021, mentor, streampipes
>
> h3. +*Apache StreamPipes*+
> Apache StreamPipes (incubating) is a self-service (Industrial) IoT toolbox to
> enable non-technical users to connect, analyze and explore IoT data streams.
> StreamPipes offers several modules including StreamPipes Connect to easily
> connect data from industrial IoT sources, the Pipeline Editor to quickly
> create processing pipelines and several visualization modules for live and
> historic data exploration. Under the hood, StreamPipes utilizes an
> event-driven microservice paradigm of standalone, so-called analytics
> microservices making the system easy to extend for individual needs.
> h3. +*Background*+
> Currently, window logic can be individually defined per pipeline element. The
> whole windowing logic needs to be declared in the controller and runtime
> logic needs to be individually added based on the selected runtime wrapper
> (Java, Siddhi, Flink, etc...).
> As many data processors benefit from using window-functions (i.e PEs such as
> Event Counter, Count Aggregation, Rate Limiter), windowing logic is often
> duplicated as it needs to be implemented for every new pipeline element. In
> addition, the feature set of supported window operators differs (and often
> depends on the developer) as it is unclear which windows and parameters
> should/can be offered.
> Therefore, adding support for explicit window semantics to the SDK/Core would
> make implementing data processors and sinks using windows much easier and
> less error-prone.
> h3. +*Tasks*+
> # Design and introduce new processor and controller classes for windowed
> event processors (e.g., WindowedDataProcessor) which handle the windowing
> logic internally and only expose the higher-level methods to users (i.e
> onCurrentEvent, onExpiredEvent, etc...).
> # Implement internal logic for few window functions (i.e TimeWindow,
> LengthWindow, TimeBatchWindow, LengthBatchWindow, etc...)
> # Write a few sample pipeline-elements using your new API!
> h3. +*Relevant Skills*+
> * Basic knowledge in StreamPipes core (cloning the repo, going through the
> codebase/documents would do).
> * Basic knowledge of stream analytics window functions (this is not a must,
> but it's awesome if you know your way around analytics window functions).
> * Some Java experience.
> h3. +*Learning Material*+
> +For StreamPipes:+
> * [https://streampipes.apache.org/docs/]
> * [https://streampipes.apache.org/media.html]
> * [https://github.com/apache/incubator-streampipes]
> * [https://github.com/apache/incubator-streampipes-extensions]
> +For Streaming Analytics:+
> *
> [https://docs.microsoft.com/en-us/azure/stream-analytics/stream-analytics-window-functions]
> *
> [https://www.mikulskibartosz.name/difference-between-tumbling-and-sliding-window/]
> *
> [https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/developing-storm-applications/content/understanding_sliding_and_tumbling_windows.html]
> +For the context for the issue:+
> * [https://www.mail-archive.com/[email protected]/msg00868.html]
> +*Mentor*+
> - Grainier Perera (grainier [at] apache.org).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
