[jira] [Updated] (FLINK-4520) Integrate Siddhi as a lightweight CEP Library
[
https://issues.apache.org/jira/browse/FLINK-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Till Rohrmann updated FLINK-4520:
-
Assignee: Hao Chen
> Integrate Siddhi as a lightweight CEP Library
> -
>
> Key: FLINK-4520
> URL: https://issues.apache.org/jira/browse/FLINK-4520
> Project: Flink
> Issue Type: New Feature
> Components: CEP
>Affects Versions: 1.2.0
>Reporter: Hao Chen
>Assignee: Hao Chen
> Labels: cep, library, patch-available
> Fix For: 1.2.0
>
>
> h1. flink-siddhi proposal
> h2. Abstraction
> Siddhi CEP is a lightweight and easy-to-use Open Source Complex Event
> Processing Engine (CEP) released as a Java Library under `Apache Software
> License v2.0`. Siddhi CEP processes events which are generated by various
> event sources, analyses them and notifies appropriate complex events
> according to the user specified queries.
> It would be very helpful for flink users (especially streaming application
> developer) to provide a library to run Siddhi CEP query directly in Flink
> streaming application.
> * http://wso2.com/products/complex-event-processor/
> * https://github.com/wso2/siddhi
> h2. Features
> * Integrate Siddhi CEP as an stream operator (i.e.
> `TupleStreamSiddhiOperator`), supporting rich CEP features like
> * Filter
> * Join
> * Aggregation
> * Group by
> * Having
> * Window
> * Conditions and Expressions
> * Pattern processing
> * Sequence processing
> * Event Tables
> ...
> * Provide easy-to-use Siddhi CEP API to integrate Flink DataStream API (See
> `SiddhiCEP` and `SiddhiStream`)
> * Register Flink DataStream associating native type information with
> Siddhi Stream Schema, supporting POJO,Tuple, Primitive Type, etc.
> * Connect with single or multiple Flink DataStreams with Siddhi CEP
> Execution Plan
> * Return output stream as DataStream with type intelligently inferred
> from Siddhi Stream Schema
> * Integrate siddhi runtime state management with Flink state (See
> `AbstractSiddhiOperator`)
> * Support siddhi plugin management to extend CEP functions. (See
> `SiddhiCEP#registerExtension`)
> h2. Test Cases
> * org.apache.flink.contrib.siddhi.SiddhiCEPITCase:
> https://github.com/haoch/flink/blob/FLINK-4520/flink-contrib/flink-siddhi/src/test/java/org/apache/flink/contrib/siddhi/SiddhiCEPITCase.java
> h2. Example
> {code}
> StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
> SiddhiCEP cep = SiddhiCEP.getSiddhiEnvironment(env);
> cep.registerExtension("custom:plus",CustomPlusFunctionExtension.class);
> cep.registerStream("inputStream1", input1, "id", "name",
> "price","timestamp");
> cep.registerStream("inputStream2", input2, "id", "name",
> "price","timestamp");
> DataStream> output = cep
> .from("inputStream1").union("inputStream2")
> .sql(
> "from every s1 = inputStream1[id == 2] "
> + " -> s2 = inputStream2[id == 3] "
> + "select s1.id as id_1, s1.name as name_1, s2.id as id_2, s2.name as
> name_2 , custom:plus(s1.price,s2.price) as price"
> + "insert into outputStream"
> )
> .returns("outputStream");
> env.execute();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FLINK-4520) Integrate Siddhi as a lightweight CEP Library
[
https://issues.apache.org/jira/browse/FLINK-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Chen updated FLINK-4520:
Labels: cep library patch-available (was: cep library)
Fix Version/s: 1.2.0
> Integrate Siddhi as a lightweight CEP Library
> -
>
> Key: FLINK-4520
> URL: https://issues.apache.org/jira/browse/FLINK-4520
> Project: Flink
> Issue Type: New Feature
> Components: CEP
>Affects Versions: 1.2.0
>Reporter: Hao Chen
> Labels: cep, library, patch-available
> Fix For: 1.2.0
>
>
> h1. flink-siddhi proposal
> h2. Abstraction
> Siddhi CEP is a lightweight and easy-to-use Open Source Complex Event
> Processing Engine (CEP) released as a Java Library under `Apache Software
> License v2.0`. Siddhi CEP processes events which are generated by various
> event sources, analyses them and notifies appropriate complex events
> according to the user specified queries.
> It would be very helpful for flink users (especially streaming application
> developer) to provide a library to run Siddhi CEP query directly in Flink
> streaming application.
> * http://wso2.com/products/complex-event-processor/
> * https://github.com/wso2/siddhi
> h2. Features
> * Integrate Siddhi CEP as an stream operator (i.e.
> `TupleStreamSiddhiOperator`), supporting rich CEP features like
> * Filter
> * Join
> * Aggregation
> * Group by
> * Having
> * Window
> * Conditions and Expressions
> * Pattern processing
> * Sequence processing
> * Event Tables
> ...
> * Provide easy-to-use Siddhi CEP API to integrate Flink DataStream API (See
> `SiddhiCEP` and `SiddhiStream`)
> * Register Flink DataStream associating native type information with
> Siddhi Stream Schema, supporting POJO,Tuple, Primitive Type, etc.
> * Connect with single or multiple Flink DataStreams with Siddhi CEP
> Execution Plan
> * Return output stream as DataStream with type intelligently inferred
> from Siddhi Stream Schema
> * Integrate siddhi runtime state management with Flink state (See
> `AbstractSiddhiOperator`)
> * Support siddhi plugin management to extend CEP functions. (See
> `SiddhiCEP#registerExtension`)
> h2. Test Cases
> * org.apache.flink.contrib.siddhi.SiddhiCEPITCase:
> https://github.com/haoch/flink/blob/FLINK-4520/flink-contrib/flink-siddhi/src/test/java/org/apache/flink/contrib/siddhi/SiddhiCEPITCase.java
> h2. Example
> {code}
> StreamExecutionEnvironment env =
> StreamExecutionEnvironment.getExecutionEnvironment();
> SiddhiCEP cep = SiddhiCEP.getSiddhiEnvironment(env);
> cep.registerExtension("custom:plus",CustomPlusFunctionExtension.class);
> cep.registerStream("inputStream1", input1, "id", "name",
> "price","timestamp");
> cep.registerStream("inputStream2", input2, "id", "name",
> "price","timestamp");
> DataStream> output = cep
> .from("inputStream1").union("inputStream2")
> .sql(
> "from every s1 = inputStream1[id == 2] "
> + " -> s2 = inputStream2[id == 3] "
> + "select s1.id as id_1, s1.name as name_1, s2.id as id_2, s2.name as
> name_2 , custom:plus(s1.price,s2.price) as price"
> + "insert into outputStream"
> )
> .returns("outputStream");
> env.execute();
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (FLINK-4520) Integrate Siddhi as a lightweight CEP Library
[
https://issues.apache.org/jira/browse/FLINK-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Chen updated FLINK-4520:
Description:
h1. flink-siddhi proposal
h2. Abstraction
Siddhi CEP is a lightweight and easy-to-use Open Source Complex Event
Processing Engine (CEP) released as a Java Library under `Apache Software
License v2.0`. Siddhi CEP processes events which are generated by various event
sources, analyses them and notifies appropriate complex events according to the
user specified queries.
It would be very helpful for flink users (especially streaming application
developer) to provide a library to run Siddhi CEP query directly in Flink
streaming application.
* http://wso2.com/products/complex-event-processor/
* https://github.com/wso2/siddhi
h2. Features
* Integrate Siddhi CEP as an stream operator (i.e.
`TupleStreamSiddhiOperator`), supporting rich CEP features like
* Filter
* Join
* Aggregation
* Group by
* Having
* Window
* Conditions and Expressions
* Pattern processing
* Sequence processing
* Event Tables
...
* Provide easy-to-use Siddhi CEP API to integrate Flink DataStream API (See
`SiddhiCEP` and `SiddhiStream`)
* Register Flink DataStream associating native type information with Siddhi
Stream Schema, supporting POJO,Tuple, Primitive Type, etc.
* Connect with single or multiple Flink DataStreams with Siddhi CEP
Execution Plan
* Return output stream as DataStream with type intelligently inferred from
Siddhi Stream Schema
* Integrate siddhi runtime state management with Flink state (See
`AbstractSiddhiOperator`)
* Support siddhi plugin management to extend CEP functions. (See
`SiddhiCEP#registerExtension`)
h2. Test Cases
* org.apache.flink.contrib.siddhi.SiddhiCEPITCase:
https://github.com/haoch/flink/blob/FLINK-4520/flink-contrib/flink-siddhi/src/test/java/org/apache/flink/contrib/siddhi/SiddhiCEPITCase.java
h2. Example
{code}
StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
SiddhiCEP cep = SiddhiCEP.getSiddhiEnvironment(env);
cep.registerExtension("custom:plus",CustomPlusFunctionExtension.class);
cep.registerStream("inputStream1", input1, "id", "name", "price","timestamp");
cep.registerStream("inputStream2", input2, "id", "name", "price","timestamp");
DataStream> output = cep
.from("inputStream1").union("inputStream2")
.sql(
"from every s1 = inputStream1[id == 2] "
+ " -> s2 = inputStream2[id == 3] "
+ "select s1.id as id_1, s1.name as name_1, s2.id as id_2, s2.name as
name_2 , custom:plus(s1.price,s2.price) as price"
+ "insert into outputStream"
)
.returns("outputStream");
env.execute();
{code}
was:
h1. flink-siddhi proposal
h2. Abstraction
Siddhi CEP is a lightweight and easy-to-use Open Source Complex Event
Processing Engine (CEP) released as a Java Library under `Apache Software
License v2.0`. Siddhi CEP processes events which are generated by various event
sources, analyses them and notifies appropriate complex events according to the
user specified queries.
It would be very helpful for flink users (especially streaming application
developer) to provide a library to run Siddhi CEP query directly in Flink
streaming application.
* http://wso2.com/products/complex-event-processor/
* https://github.com/wso2/siddhi
h2. Features
* Integrate Siddhi CEP as an stream operator (i.e.
`TupleStreamSiddhiOperator`), supporting rich CEP features like
* Filter
* Join
* Aggregation
* Group by
* Having
* Window
* Conditions and Expressions
* Pattern processing
* Sequence processing
* Event Tables
...
* Provide easy-to-use Siddhi CEP API to integrate Flink DataStream API (See
`SiddhiCEP` and `SiddhiStream`)
* Register Flink DataStream associating native type information with Siddhi
Stream Schema, supporting POJO,Tuple, Primitive Type, etc.
* Connect with single or multiple Flink DataStreams with Siddhi CEP Execution
Plan
* Return output stream as DataStream with type intelligently inferred from
Siddhi Stream Schema
* Integrate siddhi runtime state management with Flink state (See
`AbstractSiddhiOperator`)
* Support siddhi plugin management to extend CEP functions. (See
`SiddhiCEP#registerExtension`)
h2. Test Cases
* [`org.apache.flink.contrib.siddhi. SiddhiCEPITCase
`](https://github.com/haoch/flink/blob/FLINK-4520/flink-contrib/flink-siddhi/src/test/java/org/apache/flink/contrib/siddhi/SiddhiCEPITCase.java)
h2. Example
{code}
StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
SiddhiCEP cep = SiddhiCEP.getSiddhiEnvironment(env);
cep.registerExtension("custom:plus",CustomPlusFunctionExtension.class);
cep.registerStream("inputStream1", input1, "id", "name", "price","timestamp");
cep.registerStream("inputStream2", input2, "id",
[jira] [Updated] (FLINK-4520) Integrate Siddhi as a lightweight CEP Library
[
https://issues.apache.org/jira/browse/FLINK-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hao Chen updated FLINK-4520:
Description:
h1. flink-siddhi proposal
h2. Abstraction
Siddhi CEP is a lightweight and easy-to-use Open Source Complex Event
Processing Engine (CEP) released as a Java Library under `Apache Software
License v2.0`. Siddhi CEP processes events which are generated by various event
sources, analyses them and notifies appropriate complex events according to the
user specified queries.
It would be very helpful for flink users (especially streaming application
developer) to provide a library to run Siddhi CEP query directly in Flink
streaming application.
* http://wso2.com/products/complex-event-processor/
* https://github.com/wso2/siddhi
h2. Features
* Integrate Siddhi CEP as an stream operator (i.e.
`TupleStreamSiddhiOperator`), supporting rich CEP features like
* Filter
* Join
* Aggregation
* Group by
* Having
* Window
* Conditions and Expressions
* Pattern processing
* Sequence processing
* Event Tables
...
* Provide easy-to-use Siddhi CEP API to integrate Flink DataStream API (See
`SiddhiCEP` and `SiddhiStream`)
* Register Flink DataStream associating native type information with Siddhi
Stream Schema, supporting POJO,Tuple, Primitive Type, etc.
* Connect with single or multiple Flink DataStreams with Siddhi CEP Execution
Plan
* Return output stream as DataStream with type intelligently inferred from
Siddhi Stream Schema
* Integrate siddhi runtime state management with Flink state (See
`AbstractSiddhiOperator`)
* Support siddhi plugin management to extend CEP functions. (See
`SiddhiCEP#registerExtension`)
h2. Test Cases
* [`org.apache.flink.contrib.siddhi. SiddhiCEPITCase
`](https://github.com/haoch/flink/blob/FLINK-4520/flink-contrib/flink-siddhi/src/test/java/org/apache/flink/contrib/siddhi/SiddhiCEPITCase.java)
h2. Example
{code}
StreamExecutionEnvironment env =
StreamExecutionEnvironment.getExecutionEnvironment();
SiddhiCEP cep = SiddhiCEP.getSiddhiEnvironment(env);
cep.registerExtension("custom:plus",CustomPlusFunctionExtension.class);
cep.registerStream("inputStream1", input1, "id", "name", "price","timestamp");
cep.registerStream("inputStream2", input2, "id", "name", "price","timestamp");
DataStream> output = cep
.from("inputStream1").union("inputStream2")
.sql(
"from every s1 = inputStream1[id == 2] "
+ " -> s2 = inputStream2[id == 3] "
+ "select s1.id as id_1, s1.name as name_1, s2.id as id_2, s2.name as
name_2 , custom:plus(s1.price,s2.price) as price"
+ "insert into outputStream"
)
.returns("outputStream");
env.execute();
{code}
was:
h1. Flink Siddhi CEP Integration Proposal
h2. About Siddhi CEP
Siddhi CEP is a lightweight, easy-to-use Open Source Complex Event Processing
Engine (CEP) released as a Java Library under Apache Software License v2.0.
Siddhi CEP processes events which are generated by various event sources,
analyses them and notifies appropriate complex events according to the user
specified queries.
* http://wso2.com/products/complex-event-processor/
* https://github.com/wso2/siddhi
h2. Proposal
As known, siddhi is very lightweight and rich featured CEP library, supporting
most traditional CEP cases like:
* Filter
* Join
* Aggregation
* Group by
* Having
* Window
* Conditions and Expressions
* Pattern processing
* Sequence processing
* Event Tables
* Partitions
* Scripting:Support JavaScript & Scala Scripts within Siddhi Queries
* Query: SQL like query language
The proposal is to
* Embed siddhi CEP as an stream operator of Flink
* Support native siddhi query, extensions to run inside Flink
StreamExecutionEnvironment
* Integrate state management.
* Provide consistent DSL to integrate with Flink programing API.
> Integrate Siddhi as a lightweight CEP Library
> -
>
> Key: FLINK-4520
> URL: https://issues.apache.org/jira/browse/FLINK-4520
> Project: Flink
> Issue Type: New Feature
> Components: CEP
>Affects Versions: 1.2.0
>Reporter: Hao Chen
> Labels: cep, library
>
> h1. flink-siddhi proposal
> h2. Abstraction
> Siddhi CEP is a lightweight and easy-to-use Open Source Complex Event
> Processing Engine (CEP) released as a Java Library under `Apache Software
> License v2.0`. Siddhi CEP processes events which are generated by various
> event sources, analyses them and notifies appropriate complex events
> according to the user specified queries.
> It would be very helpful for flink users (especially streaming application
> developer) to provide a library to run Siddhi CEP query directly in Flink
> streaming application.
> * http://wso2.com/products/complex-event-processor/
> * https://github.com/wso2/siddhi
[jira] [Updated] (FLINK-4520) Integrate Siddhi as a lightweight CEP Library
[ https://issues.apache.org/jira/browse/FLINK-4520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hao Chen updated FLINK-4520: Description: h1. Flink Siddhi CEP Integration Proposal h2. About Siddhi CEP Siddhi CEP is a lightweight, easy-to-use Open Source Complex Event Processing Engine (CEP) released as a Java Library under Apache Software License v2.0. Siddhi CEP processes events which are generated by various event sources, analyses them and notifies appropriate complex events according to the user specified queries. * http://wso2.com/products/complex-event-processor/ * https://github.com/wso2/siddhi h2. Proposal As known, siddhi is very lightweight and rich featured CEP library, supporting most traditional CEP cases like: * Filter * Join * Aggregation * Group by * Having * Window * Conditions and Expressions * Pattern processing * Sequence processing * Event Tables * Partitions * Scripting:Support JavaScript & Scala Scripts within Siddhi Queries * Query: SQL like query language The proposal is to * Embed siddhi CEP as an stream operator of Flink * Support native siddhi query, extensions to run inside Flink StreamExecutionEnvironment * Integrate state management. * Provide consistent DSL to integrate with Flink programing API. was: Siddhi CEP is a lightweight, easy-to-use Open Source Complex Event Processing Engine (CEP) released as a Java Library under Apache Software License v2.0. Siddhi CEP processes events which are generated by various event sources, analyses them and notifies appropriate complex events according to the user specified queries. * http://wso2.com/products/complex-event-processor/ * https://github.com/wso2/siddhi > Integrate Siddhi as a lightweight CEP Library > - > > Key: FLINK-4520 > URL: https://issues.apache.org/jira/browse/FLINK-4520 > Project: Flink > Issue Type: New Feature > Components: CEP >Affects Versions: 1.2.0 >Reporter: Hao Chen > Labels: cep, library > > h1. Flink Siddhi CEP Integration Proposal > h2. About Siddhi CEP > Siddhi CEP is a lightweight, easy-to-use Open Source Complex Event Processing > Engine (CEP) released as a Java Library under Apache Software License v2.0. > Siddhi CEP processes events which are generated by various event sources, > analyses them and notifies appropriate complex events according to the user > specified queries. > * http://wso2.com/products/complex-event-processor/ > * https://github.com/wso2/siddhi > h2. Proposal > As known, siddhi is very lightweight and rich featured CEP library, > supporting most traditional CEP cases like: > * Filter > * Join > * Aggregation > * Group by > * Having > * Window > * Conditions and Expressions > * Pattern processing > * Sequence processing > * Event Tables > * Partitions > * Scripting:Support JavaScript & Scala Scripts within Siddhi Queries > * Query: SQL like query language > The proposal is to > * Embed siddhi CEP as an stream operator of Flink > * Support native siddhi query, extensions to run inside Flink > StreamExecutionEnvironment > * Integrate state management. > * Provide consistent DSL to integrate with Flink programing API. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
