[GitHub] flink issue #4296: [FLINK-7147] [cep] Support greedy quantifier in CEP
Github user dianfu commented on the issue: https://github.com/apache/flink/pull/4296 Sure. Have created FLINK-7496 to track this issue. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #4296: [FLINK-7147] [cep] Support greedy quantifier in CEP
Github user dawidwys commented on the issue: https://github.com/apache/flink/pull/4296 merging this. @dianfu could you create a JIRA for the `Optional` after `greedy` to track interest in that case. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #4296: [FLINK-7147] [cep] Support greedy quantifier in CEP
Github user dawidwys commented on the issue: https://github.com/apache/flink/pull/4296 +1, LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #4296: [FLINK-7147] [cep] Support greedy quantifier in CEP
Github user dianfu commented on the issue: https://github.com/apache/flink/pull/4296 @dawidwys Any comments? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #4296: [FLINK-7147] [cep] Support greedy quantifier in CEP
Github user dawidwys commented on the issue:
https://github.com/apache/flink/pull/4296
There is also one more problem. When we have optional after `greedy` it
does not work well. E.g. have a look at this test case:
@Test
public void testGreedyZeroOrMoreBeforeOptional2() {
List inputEvents = new ArrayList<>();
Event c = new Event(40, "c", 1.0);
Event a1 = new Event(41, "a", 2.0);
Event a2 = new Event(42, "a", 2.0);
Event d = new Event(43, "d", 3.0);
Event a3 = new Event(42, "a", 2.0);
Event e = new Event(44, "e", 3.0);
inputEvents.add(new StreamRecord<>(c, 1));
inputEvents.add(new StreamRecord<>(a1, 2));
inputEvents.add(new StreamRecord<>(a2, 3));
inputEvents.add(new StreamRecord<>(d, 4));
inputEvents.add(new StreamRecord<>(a3, 5));
inputEvents.add(new StreamRecord<>(e, 6));
// c a* d e
Pattern pattern =
Pattern.begin("start").where(new SimpleCondition() {
private static final long serialVersionUID =
5726188262756267490L;
@Override
public boolean filter(Event value) throws Exception {
return value.getName().equals("c");
}
}).followedBy("middle1").where(new SimpleCondition() {
private static final long serialVersionUID =
5726188262756267490L;
@Override
public boolean filter(Event value) throws Exception {
return value.getName().equals("a");
}
}).oneOrMore().optional().greedy().followedBy("middle2").where(new
SimpleCondition() {
private static final long serialVersionUID =
5726188262756267490L;
@Override
public boolean filter(Event value) throws Exception {
return value.getName().equals("d");
}
}).optional().followedBy("end").where(new
SimpleCondition() {
private static final long serialVersionUID =
5726188262756267490L;
@Override
public boolean filter(Event value) throws Exception {
return value.getName().equals("e");
}
});
NFA nfa = NFACompiler.compile(pattern,
Event.createTypeSerializer(), false);
final List resultingPatterns =
feedNFA(inputEvents, nfa);
compareMaps(resultingPatterns, Lists.newArrayList(
Lists.newArrayList(c, a1, a2, a3, e),
Lists.newArrayList(c, a1, a2, d, e)
));
}
Right know it also returns `c a1 a2 e`, which I think is not correct. I
don't think there is an easy way to fix it right now. I would suggest
restricting on the Pattern level that greedy must not be followed by an
`Optional` patten. I would like to hear opinions on that, @kl0u.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4296: [FLINK-7147] [cep] Support greedy quantifier in CEP
Github user dianfu commented on the issue: https://github.com/apache/flink/pull/4296 @dawidwys Regarding to the times().greedy(), the result is not expected and have fixed the issue in the latest PR. Also updated the doc. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #4296: [FLINK-7147] [cep] Support greedy quantifier in CEP
Github user dawidwys commented on the issue:
https://github.com/apache/flink/pull/4296
I think we are getting close to the final version. Good job! Still though
have two higher level comments:
- is current times().greedy() behaviour intended? For pattern `a{2, 5} b`
and sequence `a1 a2 a3 a4 b` shouldn't it return just one match: `a1 a2 a3 a4
b` Instead of three as in `testGreedyTimesRange`
- this feature should be documented in the docs. It should be also very
clearly stated there that it does not work for `GroupPatterns`
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4296: [FLINK-7147] [cep] Support greedy quantifier in CEP
Github user dianfu commented on the issue: https://github.com/apache/flink/pull/4296 @dawidwys Thanks a lot for the review. I have updated the patch. Currently, there is something wrong when the greedy state is followed by an optional state. This can be covered by test case GreedyITCase.testGreedyZeroOrMoreBeforeOptional2 (duplicate results will be got). Solutions from my mind are removing the duplicate results before returning the results in NFA or disabling this case for the time being. What's your thought? Do you have any suggestions to this problem? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #4296: [FLINK-7147] [cep] Support greedy quantifier in CEP
Github user dawidwys commented on the issue:
https://github.com/apache/flink/pull/4296
I've started reviewing it, but realised it is working as I expected only in
case where the inner consuming strategy is `STRICT`.
Let's have a look at test like this one:
@Test
public void testGreedyFollowedByInBetween() {
List inputEvents = new ArrayList<>();
Event c = new Event(40, "c", 1.0);
Event a1 = new Event(41, "a", 2.0);
Event a2 = new Event(42, "a", 2.0);
Event a3 = new Event(43, "a", 2.0);
Event d = new Event(44, "d", 3.0);
inputEvents.add(new StreamRecord<>(c, 1));
inputEvents.add(new StreamRecord<>(new Event(1, "dummy", ),
2));
inputEvents.add(new StreamRecord<>(a1, 3));
inputEvents.add(new StreamRecord<>(new Event(1, "dummy", ),
4));
inputEvents.add(new StreamRecord<>(a2, 5));
inputEvents.add(new StreamRecord<>(new Event(1, "dummy", ),
6));
inputEvents.add(new StreamRecord<>(a3, 7));
inputEvents.add(new StreamRecord<>(d, 8));
// c a* d
Pattern pattern =
Pattern.begin("start").where(new SimpleCondition() {
private static final long serialVersionUID =
5726188262756267490L;
@Override
public boolean filter(Event value) throws Exception {
return value.getName().equals("c");
}
}).followedBy("middle").where(new SimpleCondition() {
private static final long serialVersionUID =
5726188262756267490L;
@Override
public boolean filter(Event value) throws Exception {
return value.getName().equals("a");
}
}).oneOrMore().optional().greedy().followedBy("end").where(new
SimpleCondition() {
private static final long serialVersionUID =
5726188262756267490L;
@Override
public boolean filter(Event value) throws Exception {
return value.getName().equals("d");
}
});
NFA nfa = NFACompiler.compile(pattern,
Event.createTypeSerializer(), false);
final List resultingPatterns =
feedNFA(inputEvents, nfa);
compareMaps(resultingPatterns, Lists.newArrayList(
Lists.newArrayList(c, a1, a2, a3, d)
));
}
I would expect only that one result but I get:
Lists.newArrayList(c, a1, a2, a3, d),
Lists.newArrayList(c, a1, a2, d),
Lists.newArrayList(c, a1, d),
Lists.newArrayList(c, d)
Which is the same with or without the `greedy()` applied.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] flink issue #4296: [FLINK-7147] [cep] Support greedy quantifier in CEP
Github user dianfu commented on the issue: https://github.com/apache/flink/pull/4296 @dawidwys OK. Have a good time. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #4296: [FLINK-7147] [cep] Support greedy quantifier in CEP
Github user dawidwys commented on the issue: https://github.com/apache/flink/pull/4296 @dianfu Sorry for the delay, but unfortunately I will not have enough time for a proper review before my vacation. I will get back to it after 24.07. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] flink issue #4296: [FLINK-7147] [cep] Support greedy quantifier in CEP
Github user dianfu commented on the issue: https://github.com/apache/flink/pull/4296 @dawidwys Could you help to take a look at this PR? Thanks a lot in advance. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
