Github user alpinegizmo commented on a diff in the pull request:
https://github.com/apache/flink/pull/4041#discussion_r119888607
--- Diff: docs/dev/libs/cep.md ---
@@ -246,63 +334,399 @@ pattern.where(event => ... /* some condition
*/).or(event => ... /* or condition
</div>
</div>
-Next, we can append further states to detect complex patterns.
-We can control the contiguity of two succeeding events to be accepted by
the pattern.
+##### Conditions on Contiguity
-Strict contiguity means that two matching events have to be directly the
one after the other.
-This means that no other events can occur in between.
-A strict contiguity pattern state can be created via the `next` method.
+FlinkCEP supports the following forms of contiguity between events:
+
+ 1. Strict Contiguity: which expects all matching events to appear
strictly the one after the other,
+ without any non-matching events in-between.
+
+ 2. Relaxed Contiguity: which simply ignores non-matching events appearing
in-between the matching ones.
+
+ 3. Non-Deterministic Relaxed Contiguity: which further relaxes contiguity
by also creating alternative
+ matches which ignore also matching events.
+
+To illustrate the above with an example, a pattern sequence `a+ b` (one or
more `a`s followed by a `b`) with
+input `a1, c, a2, b` will have the following results:
+
+ 1. Strict Contiguity: `a2 b` because there is `c` `a1` and `a2` so `a1`
is discarded.
+
+ 2. Relaxed Contiguity: `a1 b` and `a1 a2 b`, as `c` will get simply
ignored.
+
+ 3. Non-Deterministic Relaxed Contiguity: `a1 b`, `a2 b` and `a1 a2 b`.
+
+For looping states (e.g. `oneOrMore()` and `times()`) the default is
*relaxed contiguity*. If you want
+strict contiguity, you have to explicitly specify it by using the
`consecutive()` call, and if you want
+*non-deterministic relaxed contiguity* you can use the
`allowCombinations()` call.
<div class="codetabs" markdown="1">
<div data-lang="java" markdown="1">
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 25%">Pattern Operation</th>
+ <th class="text-center">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td><strong>where(condition)</strong></td>
+ <td>
+ <p>Defines a condition for the current state. Only if an
event satisifes the condition,
+ it can match the state. Multiple consecutive where()
clauses lead to their condtions being
+ ANDed:</p>
+{% highlight java %}
+patternState.where(new IterativeCondition<Event>() {
+ @Override
+ public boolean filter(Event value, Context ctx) throws Exception {
+ return ... // some condition
+ }
+});
+{% endhighlight %}
+ </td>
+ </tr>
+ <tr>
+ <td><strong>or(condition)</strong></td>
+ <td>
+ <p>Adds a new condition which is ORed with an existing
one. Only if an event passes one of the
+ conditions, it can match the state:</p>
+{% highlight java %}
+patternState.where(new IterativeCondition<Event>() {
+ @Override
+ public boolean filter(Event value, Context ctx) throws Exception {
+ return ... // some condition
+ }
+}).or(new IterativeCondition<Event>() {
+ @Override
+ public boolean filter(Event value, Context ctx) throws Exception {
+ return ... // alternative condition
+ }
+});
+{% endhighlight %}
+ </td>
+ </tr>
+ <tr>
+ <td><strong>subtype(subClass)</strong></td>
+ <td>
+ <p>Defines a subtype condition for the current pattern
state. Only if an event is of this subtype,
+ it can match the state:</p>
{% highlight java %}
-Pattern<Event, ?> strictNext = start.next("middle");
+patternState.subtype(SubEvent.class);
{% endhighlight %}
+ </td>
+ </tr>
+ <tr>
+ <td><strong>oneOrMore()</strong></td>
+ <td>
+ <p>Specifies that this state expects at least one occurrence
of a matching event.</p>
+ <p>By default a relaxed internal contiguity (between
subsequent events) is used. For more info on the
+ internal contiguity see <a
href="#consecutive_java">consecutive</a></p>
+ {% highlight java %}
+ patternState.oneOrMore();
+ {% endhighlight %}
+ </td>
+ </tr>
+ <tr>
+ <td><strong>times(#ofTimes)</strong></td>
+ <td>
+ <p>Specifies that this state expects an exact number of
occurrences of a matching event.</p>
+ <p>By default a relaxed internal contiguity (between
subsequent events) is used. For more info on the
+ internal contiguity see <a
href="#consecutive_java">consecutive</a></p>
+{% highlight java %}
+patternState.times(2);
+{% endhighlight %}
+ </td>
+ </tr>
+ <tr>
+ <td><strong>optional()</strong></td>
+ <td>
+ <p>Specifies that this pattern is optional, i.e. it may not
occur at all. This is applicable to all
+ aforementioned quantifiers.</p>
+ {% highlight java %}
+ patternState.oneOrMore().optional();
+ {% endhighlight %}
+ </td>
+ </tr>
+ <tr>
+ <td><strong>consecutive()</strong><a
name="consecutive_java"></a></td>
+ <td>
+ <p>Works in conjunction with oneOrMore() and times() and
imposes strict contiguity between the matching
+ events, i.e. any non-matching element breaks the match (as
in next()).</p>
+ <p>If not applied a relaxed contiguity (as in followedBy())
is used.</p>
+
+ <p>E.g. a pattern like:</p>
+ {% highlight java %}
+ Pattern.<Event>begin("start").where(new
SimpleCondition<Event>() {
+ @Override
+ public boolean filter(Event value) throws Exception {
+ return value.getName().equals("c");
+ }
+ })
+ .followedBy("middle").where(new SimpleCondition<Event>() {
+ @Override
+ public boolean filter(Event value) throws Exception {
+ return value.getName().equals("a");
+ }
+ }).oneOrMore().consecutive()
+ .followedBy("end1").where(new SimpleCondition<Event>() {
+ @Override
+ public boolean filter(Event value) throws Exception {
+ return value.getName().equals("b");
+ }
+ });
+ {% endhighlight %}
+ <p>Will generate the following matches for an input
sequence: C D A1 A2 A3 D A4 B</p>
+
+ <p>with consecutive applied: {C A1 B}, {C A1 A2 B}, {C A1 A2
A3 B}</p>
+ <p>without consecutive applied: {C A1 B}, {C A1 A2 B}, {C A1
A2 A3 B}, {C A1 A2 A3 A4 B}</p>
+ </td>
+ </tr>
+ <tr>
+ <td><strong>allowCombinations()</strong><a
name="allow_comb_java"></a></td>
+ <td>
+ <p>Works in conjunction with oneOrMore() and times() and
imposes non-deterministic relaxed contiguity
+ between the matching events (as in followedByAny()).</p>
+ <p>If not applied a relaxed contiguity (as in followedBy) is
used.</p>
+
+ <p>E.g. a pattern like:</p>
+ {% highlight java %}
+ Pattern.<Event>begin("start").where(new
SimpleCondition<Event>() {
+ @Override
+ public boolean filter(Event value) throws Exception {
+ return value.getName().equals("c");
+ }
+ })
+ .followedBy("middle").where(new SimpleCondition<Event>() {
+ @Override
+ public boolean filter(Event value) throws Exception {
+ return value.getName().equals("a");
+ }
+ }).oneOrMore().allowCombinations()
+ .followedBy("end1").where(new SimpleCondition<Event>() {
+ @Override
+ public boolean filter(Event value) throws Exception {
+ return value.getName().equals("b");
+ }
+ });
+ {% endhighlight %}
+ <p>Will generate the following matches for an input
sequence: C D A1 A2 A3 D A4 B</p>
+
+ <p>with combinations enabled: {C A1 B}, {C A1 A2 B}, {C A1
A3 B}, {C A1 A4 B}, {C A1 A2 A3 B}, {C A1 A2 A4 B}, {C A1 A3 A4 B}, {C A1 A2 A3
A4 B}</p>
+ <p>without combinations enabled: {C A1 B}, {C A1 A2 B}, {C
A1 A2 A3 B}, {C A1 A2 A3 A4 B}</p>
+ </td>
+ </tr>
+ </tbody>
+</table>
</div>
<div data-lang="scala" markdown="1">
+<table class="table table-bordered">
+ <thead>
+ <tr>
+ <th class="text-left" style="width: 25%">Pattern Operation</th>
+ <th class="text-center">Description</th>
+ </tr>
+ </thead>
+ <tbody>
+
+ <tr>
+ <td><strong>where(condition)</strong></td>
+ <td>
+ <p>Defines a condition for the current state. Only if an
event satisifes the condition,
+ it can match the state. Multiple consecutive where() clauses
lead to their condtions being
+ ANDed:</p>
{% highlight scala %}
-val strictNext: Pattern[Event, _] = start.next("middle")
+patternState.where(event => ... /* some condition */)
{% endhighlight %}
+ </td>
+ </tr>
+ <tr>
+ <td><strong>or(condition)</strong></td>
+ <td>
+ <p>Adds a new condition which is ORed with an existing
one. Only if an event passes one of the
+ conditions, it can match the state:</p>
+{% highlight scala %}
+patternState.where(event => ... /* some condition */)
+ .or(event => ... /* alternative condition */)
+{% endhighlight %}
+ </td>
+ </tr>
+ <tr>
+ <td><strong>subtype(subClass)</strong></td>
+ <td>
+ <p>Defines a subtype condition for the current pattern
state. Only if an event is of this subtype,
+ it can match the state:</p>
+{% highlight scala %}
+patternState.subtype(classOf[SubEvent])
+{% endhighlight %}
+ </td>
+ </tr>
+ <tr>
+ <td><strong>oneOrMore()</strong></td>
+ <td>
+ <p>Specifies that this state expects at least one
occurrence of a matching event.</p>
+ <p>By default a relaxed internal contiguity
(between subsequent events) is used. For more info on the
+ internal contiguity see <a
href="#consecutive_scala">consecutive</a></p>
+ {% highlight scala %}
+ patternState.oneOrMore()
+ {% endhighlight %}
+ </td>
+ </tr>
+ <tr>
+ <td><strong>times(#ofTimes)</strong></td>
+ <td>
+ <p>Specifies that this state expects an exact number
of occurrences of a matching event.</p>
+ <p>By default a relaxed internal
contiguity (between subsequent events) is used.
+ For more info on the internal
contiguity see <a href="#consecutive_scala">consecutive</a></p>
+ {% highlight scala %}
+ patternState.times(2)
+ {% endhighlight %}
+ </td>
+ </tr>
+ <tr>
+ <td><strong>optional()</strong></td>
+ <td>
+ <p>Specifies that this pattern is optional, i.e. it may not
occur at all. This is applicable to all
+ aforementioned quantifiers.</p>
+ {% highlight scala %}
+ patternState.oneOrMore().optional()
+ {% endhighlight %}
+ </td>
+ </tr>
+ <tr>
+ <td><strong>consecutive()</strong><a
name="consecutive_scala"></a></td>
+ <td>
+ <p>Works in conjunction with oneOrMore() and times() and
imposes strict contiguity between the matching
+ events, i.e. any non-matching element breaks the
match (as in next()).</p>
+ <p>If not applied a relaxed contiguity (as in
followedBy()) is used.</p>
+
+ <p>E.g. a pattern like:</p>
+ {% highlight scala %}
+ Pattern.begin("start").where(_.getName().equals("c"))
+ .followedBy("middle").where(_.getName().equals("a"))
+ .oneOrMore().consecutive()
+ .followedBy("end1").where(_.getName().equals("b"));
+ {% endhighlight %}
+
+ <p>Will generate the following matches for an input sequence:
C D A1 A2 A3 D A4 B</p>
+
+ <p>with consecutive applied: {C A1 B}, {C A1 A2
B}, {C A1 A2 A3 B}</p>
+ <p>without consecutive applied: {C A1 B}, {C A1
A2 B}, {C A1 A2 A3 B}, {C A1 A2 A3 A4 B}</p>
+ </td>
+ </tr>
+ <tr>
+ <td><strong>allowCombinations()</strong><a
name="allow_comb_java"></a></td>
+ <td>
+ <p>Works in conjunction with oneOrMore() and times() and
imposes non-deterministic relaxed contiguity
+ between the matching events (as in
followedByAny()).</p>
+ <p>If not applied a relaxed contiguity (as in
followedBy) is used.</p>
+
+ <p>E.g. a pattern like:</p>
+ {% highlight scala %}
+ Pattern.begin("start").where(_.getName().equals("c"))
+ .followedBy("middle").where(_.getName().equals("a"))
+ .oneOrMore().allowCombinations()
+ .followedBy("end1").where(_.getName().equals("b"));
+ {% endhighlight %}
+
+ <p>Will generate the following matches for an input
sequence: C D A1 A2 A3 D A4 B</p>
+
+ <p>with combinations enabled: {C A1 B}, {C A1 A2 B},
{C A1 A3 B}, {C A1 A4 B}, {C A1 A2 A3 B}, {C A1 A2 A4 B}, {C A1 A3 A4 B}, {C A1
A2 A3 A4 B}</p>
+ <p>without combinations enabled: {C A1 B}, {C A1 A2
B}, {C A1 A2 A3 B}, {C A1 A2 A3 A4 B}</p>
+ </td>
+ </tr>
+ </tbody>
+</table>
</div>
+
</div>
-Non-strict contiguity means that other events are allowed to occur
in-between two matching events.
-A non-strict contiguity pattern state can be created via the `followedBy`
or `followedByAny` method.
+### Combining States
+
+Now that we have seen how an individual state can look, it is time to see
how to combine them into a full pattern sequence.
+
+A pattern sequence has to start with an initial state, as shown below:
<div class="codetabs" markdown="1">
<div data-lang="java" markdown="1">
{% highlight java %}
-Pattern<Event, ?> nonStrictNext = start.followedBy("middle");
+Pattern<Event, ?> start = Pattern.<Event>begin("start");
{% endhighlight %}
</div>
<div data-lang="scala" markdown="1">
{% highlight scala %}
-val nonStrictNext : Pattern[Event, _] = start.followedBy("middle")
+val start : Pattern[Event, _] = Pattern.begin("start")
{% endhighlight %}
</div>
</div>
-For non-strict contiguity one can specify if only the first succeeding
matching event will be matched, or
-all. In the latter case multiple matches will be emitted for the same
beginning.
+Next, you can append more states to your pattern by specifying the desired
*contiguity conditions* between them.
+This can be done using:
+
+1. `next()`, for *strict*,
+2. `followedBy()`, for *relaxed*, and
+3. `followedByAny()`, for *non-deterministic relaxed* contiguity.
+
+or
+
+1. `notNext()`, if you do not want an event type to directly follow another
+2. `notFollowedBy()`, if you do not want an event type to be anywhere
between two other event types
+
+
+<span class="label label-danger">Attention</span> A pattern sequence
cannot end in `notFollowedBy()`.
+
+<span class="label label-danger">Attention</span> A `NOT` state cannot be
preceded by an optional one.
<div class="codetabs" markdown="1">
<div data-lang="java" markdown="1">
{% highlight java %}
-Pattern<Event, ?> nonStrictNext = start.followedByAny("middle");
+
+// strict contiguity
+Pattern<Event, ?> strict = start.next("middle").where(...);
+
+// relaxed contiguity
+Pattern<Event, ?> relaxed = start.followedBy("middle").where(...);
+
+// non-deterministic relaxed contiguity
+Pattern<Event, ?> nonDetermin = start.followedByAny("middle").where(...);
+
+// NOT pattern with strict contiguity
+Pattern<Event, ?> strictNot = start.notNext("not").where(...);
+
+// NOT pattern with relaxed contiguity
+Pattern<Event, ?> relaxedNot = start.notFollowedBy("not").where(...);
+
{% endhighlight %}
</div>
<div data-lang="scala" markdown="1">
{% highlight scala %}
-val nonStrictNext : Pattern[Event, _] = start.followedByAny("middle")
+
+// strict contiguity
+val strict: Pattern[Event, _] = start.next("middle").where(...)
+
+// relaxed contiguity
+val relaxed: Pattern[Event, _] = start.followedBy("middle").where(...)
+
+// non-deterministic relaxed contiguity
+val nonDetermin: Pattern[Event, _] =
start.followedByAny("middle").where(...)
+
+// NOT pattern with strict contiguity
+val strictNot: Pattern[Event, _] = start.notNext("not").where(...)
+
+// NOT pattern with relaxed contiguity
+val relaxedNot: Pattern[Event, _] = start.notFollowedBy("not").where(...)
+
{% endhighlight %}
</div>
-
</div>
-It is also possible to define a temporal constraint for the pattern to be
valid.
-For example, one can define that a pattern should occur within 10 seconds
via the `within` method.
+
+Bear in mind that relaxed contiguity means that only the first succeeding
matching event will be matched, while
+non-deterministic relaxed contiguity, multiple matches will be emitted for
the same beginning.
--- End diff --
... will be matched, while with non-deterministic ...
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
