[
https://issues.apache.org/jira/browse/STREAMPIPES-292?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17277994#comment-17277994
]
Grainier Perera commented on STREAMPIPES-292:
---------------------------------------------
Hi [~aditya.pratap],
Welcome to the opensource community. And it's great to hear that you are
interested in our project!
However, we'll have to stick to the official timeline to get started on this
GSoC project. You can refer to the timeline on [1] & [2].
Until then you can play around with the StreamPipes product, go through the
links provided in the issue description and make your self familiar with the
code base and windowing concepts.
If you come across any questions or issues, please feel free to ask here or on
the dev mailing list ([email protected]). We are more than happy to
help you.
[1] https://community.apache.org/gsoc.html
[2] https://summerofcode.withgoogle.com/how-it-works/#timeline
Grainier.
> Introduce event windowing to the StreamPipes core/sdk
> -----------------------------------------------------
>
> Key: STREAMPIPES-292
> URL: https://issues.apache.org/jira/browse/STREAMPIPES-292
> Project: StreamPipes
> Issue Type: New Feature
> Components: Backend
> Reporter: Grainier Perera
> Priority: Major
> Labels: gsoc2021, streampipes
>
> h3. +*Apache StreamPipes*+
> Apache StreamPipes (incubating) is a self-service (Industrial) IoT toolbox to
> enable non-technical users to connect, analyze and explore IoT data streams.
> StreamPipes offers several modules including StreamPipes Connect to easily
> connect data from industrial IoT sources, the Pipeline Editor to quickly
> create processing pipelines and several visualization modules for live and
> historic data exploration. Under the hood, StreamPipes utilizes an
> event-driven microservice paradigm of standalone, so-called analytics
> microservices making the system easy to extend for individual needs.
> h3. +*Background*+
> Currently, window logic can be individually defined per pipeline element. The
> whole windowing logic needs to be declared in the controller and runtime
> logic needs to be individually added based on the selected runtime wrapper
> (Java, Siddhi, Flink, etc...).
> As many data processors benefit from using window-functions (i.e PEs such as
> Event Counter, Count Aggregation, Rate Limiter), windowing logic is often
> duplicated as it needs to be implemented for every new pipeline element. In
> addition, the feature set of supported window operators differs (and often
> depends on the developer) as it is unclear which windows and parameters
> should/can be offered.
> Therefore, adding support for explicit window semantics to the SDK/Core would
> make implementing data processors and sinks using windows much easier and
> less error-prone.
> h3. +*Tasks*+
> # Design and introduce new processor and controller classes for windowed
> event processors (e.g., WindowedDataProcessor) which handle the windowing
> logic internally and only expose the higher-level methods to users (i.e
> onCurrentEvent, onExpiredEvent, etc...).
> # Implement internal logic for few window functions (i.e TimeWindow,
> LengthWindow, TimeBatchWindow, LengthBatchWindow, etc...)
> # Write a few sample pipeline-elements using your new API!
> h3. +*Relevant Skills*+
> * Basic knowledge in StreamPipes core (cloning the repo, going through the
> codebase/documents would do).
> * Basic knowledge of stream analytics window functions (this is not a must,
> but it's awesome if you know your way around analytics window functions).
> * Some Java experience.
> h3. +*Learning Material*+
> +For StreamPipes:+
> * [https://streampipes.apache.org/docs/]
> * [https://streampipes.apache.org/media.html]
> * [https://github.com/apache/incubator-streampipes]
> * [https://github.com/apache/incubator-streampipes-extensions]
> +For Streaming Analytics:+
> *
> [https://docs.microsoft.com/en-us/azure/stream-analytics/stream-analytics-window-functions]
> *
> [https://www.mikulskibartosz.name/difference-between-tumbling-and-sliding-window/]
> *
> [https://docs.cloudera.com/HDPDocuments/HDP3/HDP-3.1.5/developing-storm-applications/content/understanding_sliding_and_tumbling_windows.html]
> +For the context for the issue:+
> * [https://www.mail-archive.com/[email protected]/msg00868.html]
> +*Mentor*+
> - Grainier Perera (grainier [at] apache.org).
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
