I wrote my question in the past, but now I want to ask about another strange
behavior I have encountered.
I will rewrite all relevant data.
I am trying to identify a port scan event:
Basic event is connection log. For each combination of source_ip and
destination_ip, detect a port scan event, if over 5 seconds there were more
than 2 connection logs with different ports .
The event will stay open for 10 seconds and an update will be sent for any new
port detected. Every update will contain the count of connection logs combining
it and their id ("marker").
This is my drl file:
----------------------------------------------------------------------------------------------------
package test;
import correlation.impl.drools.Log
import correlation.impl.drools.CorrelatedEvent
global correlation.server.EventsHandler externalEventsHandler;
declare Log
@role( event)
end
declare CorrelatedEvent
//@propertyReactive
@role( event)
@timestamp( getTimestamp())
@expires( 10s )
@duration( getDuration() )
end
// this rule will create a "Port Scan" event if none exist for this group-by
values rule "Create Port Scan Event"
dialect "java" //dialect can be either mvel or java for the RHS (only java
dialect support generic containers)
no-loop //this means - do not revaluate rules if the events were
altered in memory
when
$log : Log()
accumulate( Log( this after[0s,5s] $log, fieldsMap.get("src")
== $log.fieldsMap.get("src") , fieldsMap.get("dst") ==
$log.fieldsMap.get("dst"), $port : fieldsMap.get("port"));
$portSet : collectSet($port);
$portSet.size > 2 )
accumulate( Log( this after[0s,5s] $log, fieldsMap.get("src") ==
$log.fieldsMap.get("src") , fieldsMap.get("dst") == $log.fieldsMap.get("dst") ,
$portSet.contains(fieldsMap.get("port")) , $marker : fieldsMap.get("marker"));
$markerSet : collectSet($marker))
not CorrelatedEvent(getName() == "portScan" ,
fieldsMap.get("src") == $log.fieldsMap.get("src") , fieldsMap.get("dst") ==
$log.fieldsMap.get("dst"))
then
CorrelatedEvent $ce = new CorrelatedEvent();
$ce.setName("portScan");
$ce.setEventsHandler(externalEventsHandler);
$ce.setDurationInSec(10);
$ce.fieldsMap.put("src", $log.fieldsMap.get("src"));
$ce.fieldsMap.put("dst", $log.fieldsMap.get("dst"));
$ce.endUpdate($markerSet, $portSet);
insert($ce);
end
rule "Create Port Scan Event - update"
dialect "java" //dialect can be either mvel or java for the RHS (only java
dialect support generic containers)
no-loop
when
$ce: CorrelatedEvent(getName() == "portScan", $portSet:
getUniqueSet())
$log: Log(fieldsMap.get("src") == $ce.fieldsMap.get("src") ,
fieldsMap.get("dst") == $ce.fieldsMap.get("dst") ,
!$portSet.contains(fieldsMap.get("port")), this after $ce.getStartTime() , this
before $ce.getEndTime())
then
modify( $ce ) {endUpdate($log.fieldsMap.get("marker").toString(),
$log.fieldsMap.get("port"))} end
------------------------------------------------------------------------------------------------------------------------------------
I test it like this:
I insert a connection log and fire the rules every second. I have 25 logs with
the same "src" and "dst", but each has different (serial) "port" and "marker".
So after 12-13 logs, I expect to identify a new event with another consecutive
3 logs.
In each rule's RHS, I print the rule fired and the ports accumulated so far.
With existing implementation, I see the following output at 14th second:
rule fired: Create Port Scan Event - update
portSet: [1, 7, 11, 8, 5, 6, 3, 10, 4, 9, 12, 2]
rule fired: Create Port Scan Event
portSet: [13, 11, 12]
As we can see, the first rule processes logs already processed by the second
rule (11,12).
After examining the first rule, I thought I understood the reason for this
behavior.
I changed the order of conditions in the LHS of the first rule by moving "not
CorrelatedEvent..." to be the second condition.
But It still behaves the same.
Why is that? How can I prevent the first rule from processing logs already
processed by the second rule?
Thanks a lot!
---------------------------------------------------------------
Log class:
----------------------------------------------------------------
public class Log {
public HashMap<String, Object> fieldsMap = new HashMap<>(); }
-------------------------------------------------------------------------------------------
CorrelatedEvent class:
---------------------------------------------------------------------------------------------
public class CorrelatedEvent
{
public Map<String, Object> fieldsMap;
public int duration; //in ms
public long timestamp;
private String name;
private Set<String> markersSet;
private Set uniqueSet;
private long logsCount;
private Calendar startTime;
private Calendar endTime;
private EventsHandler eventsHandler;
public CorrelatedEvent()
{
startTime = Calendar.getInstance();
timestamp = startTime.getTime().getTime();
endTime = Calendar.getInstance();
endTime.setTime(startTime.getTime());
duration = 0;
fieldsMap = new HashMap<>();
markersSet = Collections.newSetFromMap(new ConcurrentHashMap<String,
Boolean>());
uniqueSet = Collections.newSetFromMap(new ConcurrentHashMap<Object,
Boolean>());
logsCount = 0;
}
public Set getUniqueSet()
{
return uniqueSet;
}
public void endUpdate(Set<String> markersSet, Set uniqueSet)
{
this.markersSet.addAll(markersSet);
this.uniqueSet.addAll(uniqueSet);
handleUpdate();
}
public void endUpdate(String marker, Object uniqueValue)
{
markersSet.add(marker);
uniqueSet.add(uniqueValue);
handleUpdate();
}
private void handleUpdate()
{
if (markersSet.size() != uniqueSet.size())
return;
if (markersSet.size() > logsCount) {
logsCount = this.markersSet.size();
if (eventsHandler == null)
return;
Map<String, Object> clonedFieldsMap = getClonedFieldsMap();
clonedFieldsMap.put("markers", markersSet.toString());
clonedFieldsMap.put("count", logsCount);
eventsHandler.handleEvent(clonedFieldsMap);
}
}
private Map<String, Object> getClonedFieldsMap()
{
Map<String, Object> clonedFieldsMap = new HashMap<>();
clonedFieldsMap.putAll(fieldsMap);
return clonedFieldsMap;
}
public Date getStartTime()
{
return startTime.getTime();
}
public void setDurationInSec(int duration)
{
this.duration = duration * 1000;
endTime.setTime(startTime.getTime());
endTime.add(Calendar.SECOND, duration);
}
public Date getEndTime()
{
return endTime.getTime();
}
public long getTimestamp()
{
return timestamp;
}
public int getDuration()
{
return duration;
}
public String getName()
{
return name;
}
public void setName(String name)
{
this.name = name;
fieldsMap.put("name", name);
}
public EventsHandler getEventsHandler()
{
return eventsHandler;
}
public void setEventsHandler(EventsHandler eventsHandler)
{
this.eventsHandler = eventsHandler;
}
}
------------------------------------------------------------------------------------------------
-----Original Message-----
From: Elran Dvir
Sent: Sunday, September 29, 2013 11:01 AM
To: Rules Users List
Subject: RE: [rules-users] Implementation of my use case - what am I doing
wrong?
Hi,
Thanks for the response.
I am not sure I fully understand your answer.
After all the rewriting of the rules, I decided to try "(this meets $ce || this
during $ce || this metby $ce)" again.
Now the rule is fired, but an exception is thrown (not all times the rule is
fired, just once between the third and fourth event):
Exception in thread "Thread-1" java.lang.RuntimeException:
java.lang.NoSuchMethodException:
com.checkpoint.correlation.impl.drools.Log.meets0()
at
org.drools.rule.constraint.ConditionAnalyzer.analyzeNode(ConditionAnalyzer.java:239)
at
org.drools.rule.constraint.ConditionAnalyzer.analyzeSingleCondition(ConditionAnalyzer.java:133)
at
org.drools.rule.constraint.ConditionAnalyzer.analyzeCondition(ConditionAnalyzer.java:99)
at
org.drools.rule.constraint.ConditionAnalyzer.analyzeCombinedCondition(ConditionAnalyzer.java:140)
at
org.drools.rule.constraint.ConditionAnalyzer.analyzeCondition(ConditionAnalyzer.java:96)
at
org.drools.rule.constraint.ConditionAnalyzer.analyzeCombinedCondition(ConditionAnalyzer.java:140)
at
org.drools.rule.constraint.ConditionAnalyzer.analyzeCondition(ConditionAnalyzer.java:96)
at
org.drools.rule.constraint.ConditionAnalyzer.analyzeCombinedCondition(ConditionAnalyzer.java:141)
at
org.drools.rule.constraint.ConditionAnalyzer.analyzeCondition(ConditionAnalyzer.java:96)
at
org.drools.rule.constraint.ConditionAnalyzer.analyzeCombinedCondition(ConditionAnalyzer.java:141)
at
org.drools.rule.constraint.ConditionAnalyzer.analyzeCondition(ConditionAnalyzer.java:96)
at
org.drools.rule.constraint.ConditionAnalyzer.analyzeCombinedCondition(ConditionAnalyzer.java:141)
at
org.drools.rule.constraint.ConditionAnalyzer.analyzeCondition(ConditionAnalyzer.java:96)
at
org.drools.rule.constraint.ConditionAnalyzer.analyzeCondition(ConditionAnalyzer.java:70)
at
org.drools.rule.constraint.MvelConditionEvaluator.getAnalyzedCondition(MvelConditionEvaluator.java:83)
at
org.drools.rule.constraint.MvelConstraint.executeJitting(MvelConstraint.java:270)
at
org.drools.rule.constraint.MvelConstraint.access$200(MvelConstraint.java:51)
at
org.drools.rule.constraint.MvelConstraint$ConditionJitter.run(MvelConstraint.java:250)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Caused by: java.lang.NoSuchMethodException:
com.checkpoint.correlation.impl.drools.Log.meets0()
at java.lang.Class.getMethod(Class.java:1622)
at
org.drools.rule.constraint.ConditionAnalyzer.analyzeNode(ConditionAnalyzer.java:230)
... 20 more
The complete latest drl file is followed.
Duration for Log is 0, duration for CorrelatedEvent is 10s.
Did you have a chance to look at my (original) fourth question? This is the
issue that bothers me the most.
If necessary, I'll write everything again.
Thanks a lot.
This is my drl file:
----------------------------------------------------------------------------------------------------
package test;
import correlation.impl.drools.Log
import correlation.impl.drools.CorrelatedEvent
global correlation.server.EventsHandler externalEventsHandler;
declare Log
@role( event)
end
declare CorrelatedEvent
//@propertyReactive
@role( event)
@timestamp( getTimestamp())
@expires( 10s )
@duration( getDuration() )
end
// this rule will create a "Port Scan" event if none exist for this group-by
values rule "Create Port Scan Event"
dialect "java" //dialect can be either mvel or java for the RHS (only java
dialect support generic containers)
no-loop //this means - do not revaluate rules if the events were
altered in memory
when
$log : Log()
accumulate( Log( this after[0s,5s] $log, fieldsMap.get("src")
== $log.fieldsMap.get("src") , fieldsMap.get("dst") ==
$log.fieldsMap.get("dst"), $port : fieldsMap.get("port"));
$portSet : collectSet($port);
$portSet.size > 2 )
accumulate( Log( this after[0s,5s] $log, fieldsMap.get("src") ==
$log.fieldsMap.get("src") , fieldsMap.get("dst") == $log.fieldsMap.get("dst") ,
$portSet.contains(fieldsMap.get("port")) , $marker : fieldsMap.get("marker"));
$markerSet : collectSet($marker))
not CorrelatedEvent(getName() == "portScan" ,
fieldsMap.get("src") == $log.fieldsMap.get("src") , fieldsMap.get("dst") ==
$log.fieldsMap.get("dst")) then
CorrelatedEvent $ce = new CorrelatedEvent();
$ce.setName("portScan");
$ce.setEventsHandler(externalEventsHandler);
$ce.setDurationInSec(10);
$ce.fieldsMap.put("src", $log.fieldsMap.get("src"));
$ce.fieldsMap.put("dst", $log.fieldsMap.get("dst"));
$ce.endUpdate($markerSet, $portSet);
insert($ce);
end
rule "Create Port Scan Event - update"
dialect "java" //dialect can be either mvel or java for the RHS (only java
dialect support generic containers)
no-loop
when
$ce: CorrelatedEvent(getName() == "portScan", $portSet:
getUniqueSet())
$log: Log(fieldsMap.get("src") == $ce.fieldsMap.get("src") ,
fieldsMap.get("dst") == $ce.fieldsMap.get("dst") ,
!$portSet.contains(fieldsMap.get("port")), (this meets $ce || this during $ce
|| this metby $ce)) then
modify( $ce ) {endUpdate($log.fieldsMap.get("marker").toString(),
$log.fieldsMap.get("port"))} end
------------------------------------------------------------------------------------------------------------------------------------
-----Original Message-----
From: [email protected]
[mailto:[email protected]] On Behalf Of Wolfgang Laun
Sent: Sunday, September 22, 2013 7:06 PM
To: Rules Users List
Subject: Re: [rules-users] Implementation of my use case - what am I doing
wrong?
On 22/09/2013, Elran Dvir <[email protected]> wrote:
> Thanks for the response.
>
>>Computationally, during the evaluation of the temporal operators.
>>Why would you want to do this? (My advice to replace the temporal
>>operators by the equivalent expressions was meant as a debugging aid,
>>to show you where the problem with this constraint is.)
> So if changing the condition to use startTimeStamp and endTimestamp
> caused a compilation error, How can I debug my problem? What is the
> equivalent of "(this meets $ce || this during $ce || this metby $ce)"
> without my own startTimeStamp and endTimestamp fields?
>
So I'm quoting my reply from an earlier mail <quote>
> Elran Dvir <[email protected]> wrote:
>
> 2) When I tested it with matching data, rule "Create Port Scan Event -
> update" was never fired. When I replaced "(this meets $ce || this
> during $ce
> || this metby $ce)" with "this after $ce.getStartTime() , this before
> $ce.getEndTime()" everything worked fine.
> Why?
Just take the constraints and replace the temporal operator by its definition
in the "Fusion" manual and use a little elementary math:
A meets || A during B || A metby B
becomes
abs( B.startTimestamp - A.endTimestamp ) == 0 ||
B.startTimestamp < A.startTimestamp && A.endTimestamp < B.endTimestamp ||
abs( A.startTimestamp - B.endTimestamp ) == 0 becomes
...
</quote>
Did you overlook the "use a little elementary math"? YOu'll have to provide the
values for start and, given the duration (zero,
presumably) the end time. If B has a duration of 0, there's no way the second
term can ever be true. And if A has a duration of 0 too, A and B must have the
same timestamp in order for the other terms to be true.
Note that "debugging" was meant to be done offline ;-)
> > - Why do you use this complex declare?
> > @timestamp( timestamp ) @duration( duration )
> > is sufficient.
>
> I want to protect the fields from being set outside their designated
> functions .I simplified it a bit in my correction.
I don't know of any danger of your field being set outside their designated
functions by simply writing the field name between the parentheses.
-W
_______________________________________________
rules-users mailing list
[email protected]
https://lists.jboss.org/mailman/listinfo/rules-users
Email secured by Check Point
_______________________________________________
rules-users mailing list
[email protected]
https://lists.jboss.org/mailman/listinfo/rules-users
