subject:"\[rules\-users\] Is there a faster way of doing this in Drools \?"

I've a use case where I need to create a rule dynamically every time the user
sends a request. 
My current understanding of Drools is that you need to create a
KnowledgeBuilder and then add the rules 

I'm creating the KnowledgeBase as follows. 

private static KnowledgeBase readKnowledgeBase() throws Exception {
long t1 = System.currentTimeMillis();
KnowledgeBuilder kbuilder =
KnowledgeBuilderFactory.newKnowledgeBuilder();  
long t11 = System.currentTimeMillis();


kbuilder.add(
org.drools.io.ResourceFactory.newByteArrayResource(getRule()),
ResourceType.DRL);

long t2 = System.currentTimeMillis();

KnowledgeBuilderErrors errors = kbuilder.getErrors();
if (errors.size()  0) {
for (KnowledgeBuilderError error: errors) {
System.err.println(error);
}
throw new IllegalArgumentException(Could not parse
knowledge.);
}
long t3 = System.currentTimeMillis();

KnowledgeBase kbase = KnowledgeBaseFactory.newKnowledgeBase();
long t4 = System.currentTimeMillis();

kbase.addKnowledgePackages(kbuilder.getKnowledgePackages());
long t5 = System.currentTimeMillis();


CollectionKnowledgePackage kpackages =
kbase.getKnowledgePackages();
for (KnowledgePackage knowledgePackage : kpackages) {
System.out.println(Package   + 
knowledgePackage.getName());
CollectionRule rules = knowledgePackage.getRules();
for (Rule rule : rules) {
System.out.println( + rule.getName());
}
}
long t6 = System.currentTimeMillis();

System.out.println( (t11-t1) +   + (t2-t11) +   + ( t3-t2) +   +
( t4-t3) +   + ( t5-t4) +   + ( t6-t5)+   );

return kbase;
}

Based on the timing logs the code take majority (more than 80%) of time in
only these two operations. Is there a way to make it faster ? 

KnowledgeBuilder kbuilder = KnowledgeBuilderFactory.newKnowledgeBuilder();  
kbuilder.add( org.drools.io.ResourceFactory.newByteArrayResource(getRule()),
ResourceType.DRL);

thanks. 



--
View this message in context: 
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

2012-05-09 Thread Mauricio Salatino

You can do that step once and then reload the compiled rules from the disk,
or use guvnor which will compile the rules for you.
Cheers

On Wed, May 9, 2012 at 12:55 PM, soumya_sd soumya...@yahoo.com wrote:

 I've a use case where I need to create a rule dynamically every time the
 user
 sends a request.
 My current understanding of Drools is that you need to create a
 KnowledgeBuilder and then add the rules

 I'm creating the KnowledgeBase as follows.

private static KnowledgeBase readKnowledgeBase() throws Exception {
long t1 = System.currentTimeMillis();
KnowledgeBuilder kbuilder =
 KnowledgeBuilderFactory.newKnowledgeBuilder();
long t11 = System.currentTimeMillis();


kbuilder.add(
 org.drools.io.ResourceFactory.newByteArrayResource(getRule()),
 ResourceType.DRL);

long t2 = System.currentTimeMillis();

KnowledgeBuilderErrors errors = kbuilder.getErrors();
if (errors.size()  0) {
for (KnowledgeBuilderError error: errors) {
System.err.println(error);
}
throw new IllegalArgumentException(Could not parse
 knowledge.);
}
long t3 = System.currentTimeMillis();

KnowledgeBase kbase = KnowledgeBaseFactory.newKnowledgeBase();
long t4 = System.currentTimeMillis();

kbase.addKnowledgePackages(kbuilder.getKnowledgePackages());
long t5 = System.currentTimeMillis();


CollectionKnowledgePackage kpackages =
 kbase.getKnowledgePackages();
for (KnowledgePackage knowledgePackage : kpackages) {
System.out.println(Package   +
 knowledgePackage.getName());
CollectionRule rules =
 knowledgePackage.getRules();
for (Rule rule : rules) {
System.out.println( + rule.getName());
}
}
long t6 = System.currentTimeMillis();

System.out.println( (t11-t1) +   + (t2-t11) +   + ( t3-t2) + 
  +
 ( t4-t3) +   + ( t5-t4) +   + ( t6-t5)+   );

return kbase;
}

 Based on the timing logs the code take majority (more than 80%) of time in
 only these two operations. Is there a way to make it faster ?

 KnowledgeBuilder kbuilder = KnowledgeBuilderFactory.newKnowledgeBuilder();
 kbuilder.add(
 org.drools.io.ResourceFactory.newByteArrayResource(getRule()),
 ResourceType.DRL);

 thanks.



 --
 View this message in context:
 http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888.html
 Sent from the Drools: User forum mailing list archive at Nabble.com.
 ___
 rules-users mailing list
 rules-users@lists.jboss.org
 https://lists.jboss.org/mailman/listinfo/rules-users




-- 
 - MyJourney @ http://salaboy.wordpress.com
 - Co-Founder @ http://www.jugargentina.org
 - Co-Founder @ http://www.jbug.com.ar

 - Salatino Salaboy Mauricio -
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?


salaboy wrote
 
 You can do that step once and then reload the compiled rules from the
 disk,
 or use guvnor which will compile the rules for you.
 Cheers
 
 
 

Thanks for responding so quickly. I tried using Guvnor. Do you think it will
be faster than this ? In the worse case, the rules that I need to apply can
change with every iteration i.e., they are completely dynamic and defined at
runtime. So, storing the rules is not an options. 

--
View this message in context: 
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3973915.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

2012-05-09 Thread Mauricio Salatino

Can you explain a little bit more about your context? having completely
dynamic rules is not an usual use case, rules tends to be static because
they define the business logic that you want to apply. So please elaborate
on the context.

On Wed, May 9, 2012 at 1:07 PM, soumya_sd soumya...@yahoo.com wrote:


 salaboy wrote
 
  You can do that step once and then reload the compiled rules from the
  disk,
  or use guvnor which will compile the rules for you.
  Cheers
 
 
 

 Thanks for responding so quickly. I tried using Guvnor. Do you think it
 will
 be faster than this ? In the worse case, the rules that I need to apply can
 change with every iteration i.e., they are completely dynamic and defined
 at
 runtime. So, storing the rules is not an options.

 --
 View this message in context:
 http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3973915.html
 Sent from the Drools: User forum mailing list archive at Nabble.com.
 ___
 rules-users mailing list
 rules-users@lists.jboss.org
 https://lists.jboss.org/mailman/listinfo/rules-users




-- 
 - MyJourney @ http://salaboy.wordpress.com
 - Co-Founder @ http://www.jugargentina.org
 - Co-Founder @ http://www.jbug.com.ar

 - Salatino Salaboy Mauricio -
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

Could you please provide a more detailed example?
The usecase you propose is really interesting and the optimization of
something very similar is on the todo-list (@salaboy: request-when rings a
bell?)
Now, depending on your exact requirements, there could be various options,
more or less clean ;)

First of all, we should determine whether a) you need a new rule or b) you
can just use a parametric rule and carry the parameters in a fact, e.g.:

when
ParamHolder( $x : threshold, $bid : beanId )
Bean( id == $bid, value  $x )
then
...
end

If you do need new rules, generating the drl on the fly is feasible but the
parsing and compilation is a bit expensive, so maybe one should try to build
the internal representation, to save at least the parsing process and jump
to the RETE construction. Btw, you mentioned that you'd have to generate a
rule for each transaction: would that rule lifecycle be limited to the
transaction too? i.e. would it be possible/necessary to remove the rule once
it has been used?

Davide

--
View this message in context: 
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3973949.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?


Davide Sottara wrote
 
 Could you please provide a more detailed example?
 The usecase you propose is really interesting and the optimization of
 something very similar is on the todo-list (@salaboy: request-when rings
 a bell?)
 Now, depending on your exact requirements, there could be various options,
 more or less clean ;)
 
 First of all, we should determine whether a) you need a new rule or b) you
 can just use a parametric rule and carry the parameters in a fact, e.g.:
 
 when
 ParamHolder( $x : threshold, $bid : beanId )
 Bean( id == $bid, value  $x )
 then
 ...
 end
 
 If you do need new rules, generating the drl on the fly is feasible but
 the parsing and compilation is a bit expensive, so maybe one should try to
 build the internal representation, to save at least the parsing process
 and jump to the RETE construction. Btw, you mentioned that you'd have to
 generate a rule for each transaction: would that rule lifecycle be
 limited to the transaction too? i.e. would it be possible/necessary to
 remove the rule once it has been used?
 
 Davide
 





Both of you are correct. This is not the typical use case Drools is designed
for. 

Here are more details about the the context and use case. 

Most of these rules will be used for filtering facts. 

For example, 

MyBean
{

var 1;
var 2; 
var n; 

}

We want to give the user the ability to define rules that can then be used
to filter a collection of face (CollectionMyBean) at runtime. So you can
see how this can change with every request. In one case, the user may only
want to see a subset that satisfies the condition 

var 1  2 and var 2.equals(something) 

and the same user may may want to use a rule 

var 3  10 and var 2.equals(something else). 

I hope you get the idea. So, even if my business model is fixed the rules
are not. Although the rules are not very complicated, you can imagine that
the user can defined pretty much anything at runtime.  


Can I use a parameteric rule for each of fact type in my business model?  




--
View this message in context: 
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3974296.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

Users changing rules IS the correct use-case of drools (I would even say that 
if rules never change, drools is likely to be worse than pure java).
What others said is changing the rules at each request by regenerating and 
recompiling is not.

I can't imagine that users may change rules at each request, so you must have, 
at least for some time (thus a lot of requests), a stable rule base.
What is costly is to compile rules, but you already know that.
So the idea is to keep in some kind of cache the actual rules, and only 
recompile them when they change.

The source of rules does not matter : you can generate the DRL according to 
your configuration, or make you users directly write rules (with guvnor, which 
will fits well if your rules are only basic filters).

The optimisation you have to do is to keep your compiled rules somewhere (by 
serializing your KnowlegdePackages or simply keep it in a kind of map), and 
reuse it until your source change : you have to find a way to detect that rules 
(or the config that is used to generate them) have changed, and then recompile 
a new up-to-date KnowledgePackage.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

Vincent Legendre wrote

Users changing rules IS the correct use-case of drools (I would even say
that if rules never change, drools is likely to be worse than pure java).
What others said is changing the rules at each request by regenerating
and recompiling is not.

I can't imagine that users may change rules at each request, so you must
have, at least for some time (thus a lot of requests), a stable rule base.
What is costly is to compile rules, but you already know that.
So the idea is to keep in some kind of cache the actual rules, and only
recompile them when they change.

The source of rules does not matter : you can generate the DRL according
to your configuration, or make you users directly write rules (with
guvnor, which will fits well if your rules are only basic filters).

The optimisation you have to do is to keep your compiled rules somewhere
(by serializing your KnowlegdePackages or simply keep it in a kind of
map), and reuse it until your source change : you have to find a way to
detect that rules (or the config that is used to generate them) have
changed, and then recompile a new up-to-date KnowledgePackage.

Thank you for your insights.

At this point I'm designing for the worst case i.e., I'm assuming the rules
with change with every request.

Moreover, this can happen on a regular basis in the system I'm trying to
design.

Product
{
price
ave_user_rating
num_ratings
category

}

User wants to see Product with with the following rules

at t=t_1 : If product.category in {A,B}.

at t=t_2 : If product.category in {A,B} AND price 100

at t=t_3 : If product.category in {A,B} AND price 100 AND num_ratings
100

You can imagine the user changing these rules in a span of less than a
minute.

I understand that Drools is designed for cases when rules changes often. But
can it support something like the one I described above where rules are
changing at a much faster rate?

Thanks.

--
View this message in context:
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3974483.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

Vincent Legendre wrote

Thank you for your insights.

At this point I'm designing for the worst case i.e., I'm assuming the rules
with change with every request.

Moreover, this can happen on a regular basis in the system I'm trying to
design.

Product
{
price
ave_user_rating
num_ratings
category

}

User wants to see Product with with the following rules

at t=t_1 : If product.category in {A,B}.

at t=t_2 : If product.category in {A,B} AND price 100

at t=t_3 : If product.category in {A,B} AND price 100 AND num_ratings
100

You can imagine the user changing these rules in a span of less than a
minute.

I understand that Drools is designed for cases when rules changes often. But
can it support something like the one I described above where rules are
changing at a much faster rate?

Thanks.

--
View this message in context:
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3974487.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

 At this point I'm designing for the worst case i.e., I'm assuming the rules
 with change with every request. 

Keeping a cache will work for the worst case, but will work faster for normal 
(see below) cases.


 You can imagine the user changing these rules in a span of less than a
 minute. 

Well, no ... it is really hard to imagine that kind of frequence as a normal 
use over a long time.
For all systems I made, rules are stable for much more than that. When changing 
rules, you may change them every minutes (to test the outcomes, just like 
debuging a java code), but the objective is to produce something stable that 
will be used in a production environment ...
Changing rules every minutes or less in a production environment is definitely 
not a usual use-case ...
Why your rules should change every minute ? What is the system your are 
building ? 


 I understand that Drools is designed for cases when rules changes often. But
 can it support something like the one I described above where rules are
 changing at a much faster rate? 

You won't be able to speed up the compile time.
You can only act on the volume and organization of what you are compiling.
So, you can use cache (as said before), and may be you can also cut your rules 
in smaller packages and only recompile the package where some modifications 
take place (if it take really some mesureable time, it is likely also because 
you have plenty of rules to compile).
If you have a single package with few rules, compile time should not be a 
problem..


And a last thing : In your first post, you said :
Based on the timing logs the code take majority (more than 80%) of time in
only these two operations. Is there a way to make it faster ?

KnowledgeBuilder kbuilder = KnowledgeBuilderFactory.newKnowledgeBuilder();  
kbuilder.add( org.drools.io.ResourceFactory.newByteArrayResource(getRule()),
ResourceType.DRL);

You first have to make the distinction between KnowledgeBuilder compilation 
time (up to drools code, and you can't change anything, at least easyly), 
and your getRule() method (guess one of your custom code), with may be speeded 
up.
For instance, if you can't globally cut your compiled rules into smaller 
compiled packages, may be you can cut your big resulting DRL in smaller pieces, 
regenerating only what have changed ...


___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

If the structure of the constraints/filters the user can define remains
static, and only the values are changing, then a single rule with
parametric joins will be enough. E.g.:

Params( $cat : categorySet, $minPrice : minPrice, $maxPrice : maxPrice,
)
Product( category in $cat, price = $minPrice = $maxPrice, num_ratings
... )

You'd then have to update the Params according to what the user e.g.
selects from a checkbox.
A null selection would then be converted e.g. categorySet = all
categories, minPrice = -infinity, maxPrice = +infinity etc...

If, on the other hand, the user is completely free to create their own
rules, this strategy may not be applicable. Wouldn't then the bottleneck be
the user writing/picking the rules (seconds) rather than their compilation
(a few milliseconds)? If so, you might have horizontal scalability issues
- allowing more and more users to do this - rather than vertical ones -
responding faster to a single user.
If, instead, the criteria are built automatically by a software agent of
some sort, then the direct creation might be taken into consideration. You
can skip the parsing step by creating the Abstract Syntax Tree directly -
the compiler APIs (Descrs) have a nice fluent structure.

Davide

btw, what are your rules supposed to do as a consequence? Do you have
chaining? or actions?
If all you need is selection and filtering, I wonder if a production rule
system is the best option
for your application, or at least this sub-module...

--
View this message in context:
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3974782.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

Vincent Legendre wrote

At this point I'm designing for the worst case i.e., I'm assuming the
rules
with change with every request.

Keeping a cache will work for the worst case, but will work faster for
normal (see below) cases.

I don't see how can a cache would work in the worse case. I mean if the rule
is not there in the cache then I've to figure out a way to load it. So in
the worse case, you will always have a cache miss. Is this not correct ?

Vincent Legendre wrote

You can imagine the user changing these rules in a span of less than a
minute.

Well, no ... it is really hard to imagine that kind of frequence as a
normal use over a long time.
For all systems I made, rules are stable for much more than that. When
changing rules, you may change them every minutes (to test the outcomes,
just like debuging a java code), but the objective is to produce something
stable that will be used in a production environment ...
Changing rules every minutes or less in a production environment is
definitely not a usual use-case ...
Why your rules should change every minute ? What is the system your are
building ?

I just gave you an example of the type of use case I want to implement -
realtime filtering of data. In this case, the data is product data and the
rules are filtering rules that the user has defined. Imagine if you want to
implement a website where the user can control the products filters. Again,
with all due respect, if you haven't seen a use case doesn't mean it doesn't
exist or is not relevant in other situations. I'm building an experimental
prototype, so for now if you can just assume that such a use case exists I
would appreciate that.

Vincent Legendre wrote

I understand that Drools is designed for cases when rules changes often.
But
can it support something like the one I described above where rules are
changing at a much faster rate?

You won't be able to speed up the compile time.
You can only act on the volume and organization of what you are compiling.
So, you can use cache (as said before), and may be you can also cut your
rules in smaller packages and only recompile the package where some
modifications take place (if it take really some mesureable time, it is
likely also because you have plenty of rules to compile).
If you have a single package with few rules, compile time should not be a
problem..

For now, I cannot imagine these rules to very complex. And I think I can
limit them to a single package and a few rules (less than 10.)

Vincent Legendre wrote

And a last thing : In your first post, you said :
Based on the timing logs the code take majority (more than 80%) of time in
only these two operations. Is there a way to make it faster ?

KnowledgeBuilder kbuilder = KnowledgeBuilderFactory.newKnowledgeBuilder();
kbuilder.add(
org.drools.io.ResourceFactory.newByteArrayResource(getRule()),
ResourceType.DRL);

You first have to make the distinction between KnowledgeBuilder
compilation time (up to drools code, and you can't change anything, at
least easyly),
and your getRule() method (guess one of your custom code), with may be
speeded up.
For instance, if you can't globally cut your compiled rules into smaller
compiled packages, may be you can cut your big resulting DRL in smaller
pieces, regenerating only what have changed ...

My getRule() just reads a drl file and returns it as a byte[]. Currently
it's taking around 0ms. So I don't think this is a bottleneck.

--
View this message in context:
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3974807.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

2012-05-09 Thread Wolfgang Laun

Merely filtering a subset of data items from a collection according to
user's changing needs and whims is NOT the primary use case of a production
rule system.  This kind of problem has been solved adequately and
efficiently by SQL queries or some similar DB query technique.

It is true that such filtering rules, if compiled and stored in a
KnowledgeBase, can achieve the same thing, but, since rules are targeted at
a more sophisticated use cases, this will never be as efficient as a simple
query system.

-W
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?


laune wrote
 
 Merely filtering a subset of data items from a collection according to
 user's changing needs and whims is NOT the primary use case of a
 production
 rule system.  This kind of problem has been solved adequately and
 efficiently by SQL queries or some similar DB query technique.
 
 It is true that such filtering rules, if compiled and stored in a
 KnowledgeBase, can achieve the same thing, but, since rules are targeted
 at
 a more sophisticated use cases, this will never be as efficient as a
 simple
 query system.
 
 -W
 
 

you bring up an excellent point. 

What if the data is streaming in nature and is not stored in a database and
outside of your control. 

 
Another approach is to cache the data locally (in a database) and then use
SQL to filter it. 

I decided not to use the database approach because of two reasons. 

1. The database write operation may be expensive. 
2. In the future I may decide apply more complex consequence to my rules
than just filtering. 

Thanks. 

--
View this message in context: 
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3974856.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

2012-05-09 Thread Wolfgang Laun

On 9 May 2012 19:52, soumya_sd soumya...@yahoo.com wrote:

laune wrote

Merely filtering a subset of data items from a collection according to
user's changing needs and whims is NOT the primary use case of a
production
rule system. This kind of problem has been solved adequately and
efficiently by SQL queries or some similar DB query technique.

It is true that such filtering rules, if compiled and stored in a
KnowledgeBase, can achieve the same thing, but, since rules are targeted
at
a more sophisticated use cases, this will never be as efficient as a
simple
query system.

-W

you bring up an excellent point.

What if the data is streaming in nature and is not stored in a database and
outside of your control.

It you can insert it into a WM, it can't be outside of your control. If
both data
and selection rules arrive at random, it's going to be chaotic. Some sort
of higher order control pattern will have to take care of that...

Another approach is to cache the data locally (in a database) and then use
SQL to filter it.

I decided not to use the database approach because of two reasons.

1. The database write operation may be expensive.

Storing a record in a DB is not really expensive.

2. In the future I may decide apply more complex consequence to my rules
than just filtering.

The effort for processing of selected entities is the same, irrespective of
their source, be it
a DB or a WM. Compiling some processing statements could be done by
invoking the
Java compiler, without the additional overhead of compiling DRL first.

-W

Thanks.

--
View this message in context:
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3974856.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

laune wrote

On 9 May 2012 19:52, soumya_sd lt;soumya_sd@gt; wrote:

laune wrote

It is true that such filtering rules, if compiled and stored in a
KnowledgeBase, can achieve the same thing, but, since rules are
targeted
at
a more sophisticated use cases, this will never be as efficient as a
simple
query system.

-W

you bring up an excellent point.

What if the data is streaming in nature and is not stored in a database
and
outside of your control.

Correct. Pardon my language. What I really meant by outside of my control
is that source databases are not controlled by me. I query external
databases at runtime to get the results I need and then filter a subset
based on the user defined needs.

Currently I'm using Apache Camel for getting my data from remote sources.

laune wrote

Another approach is to cache the data locally (in a database) and then
use
SQL to filter it.

I decided not to use the database approach because of two reasons.

1. The database write operation may be expensive.

Storing a record in a DB is not really expensive.

2. In the future I may decide apply more complex consequence to my rules
than just filtering.

The effort for processing of selected entities is the same, irrespective
of
their source, be it
a DB or a WM. Compiling some processing statements could be done by
invoking the
Java compiler, without the additional overhead of compiling DRL first.

I didn't understand what you mean by Compiling some processing statements
could be done by
invoking the Java compiler, without the additional overhead of compiling DRL
first..

--
View this message in context:
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3974949.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

@ laune - so are you suggesting that I don't use Drools ? 

--
View this message in context: 
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3974950.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

2012-05-09 Thread GPatel

so you are using drools to build on-the-fly queries based on user input. 
You could probably do this via static rules. Since you basically have 4 
variables (price, avg_user_rating, num_ratings, category), you could write 
4 rules, one each that filters based on one of the 4 variables and then 
chain the execution through salience:

rule price:
salience 1
when 
$p: Product(price in param)
then
add $p to result

rule avg_user_rating:
salience 2
when
$p: Product(avg_user_rating  param)
then
add $p to result


and so on




From:   soumya_sd soumya...@yahoo.com
To: rules-users@lists.jboss.org, 
Date:   05/09/2012 08:43 AM
Subject:Re: [rules-users] Is there a faster way of doing this in 
Drools ?
Sent by:rules-users-boun...@lists.jboss.org




Vincent Legendre wrote
 
 Users changing rules IS the correct use-case of drools (I would even say
 that if rules never change, drools is likely to be worse than pure 
java).
 What others said is changing the rules at each request by regenerating
 and recompiling is not.
 
 I can't imagine that users may change rules at each request, so you must
 have, at least for some time (thus a lot of requests), a stable rule 
base.
 What is costly is to compile rules, but you already know that.
 So the idea is to keep in some kind of cache the actual rules, and only
 recompile them when they change.
 
 The source of rules does not matter : you can generate the DRL according
 to your configuration, or make you users directly write rules (with
 guvnor, which will fits well if your rules are only basic filters).
 
 The optimisation you have to do is to keep your compiled rules somewhere
 (by serializing your KnowlegdePackages or simply keep it in a kind of
 map), and reuse it until your source change : you have to find a way to
 detect that rules (or the config that is used to generate them) have
 changed, and then recompile a new up-to-date KnowledgePackage.
 
 

Thank you for your insights. 

At this point I'm designing for the worst case i.e., I'm assuming the 
rules
with change with every request. 

Moreover, this can happen on a regular basis in the system I'm trying to
design. 

Product
{ 
  price
  ave_user_rating
  num_ratings
  category 
 
} 

User wants to see Product with with the following rules 

at t=t_1 : If product.category in {A,B}. 

at t=t_2 : If product.category in {A,B} AND price  100 

at t=t_3 :  If product.category in {A,B}  AND price  100 AND num_ratings 

100 

You can imagine the user changing these rules in a span of less than a
minute. 

I understand that Drools is designed for cases when rules changes often. 
But
can it support something like the one I described above where rules are
changing at a much faster rate? 


Thanks. 






--
View this message in context: 
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3974483.html

Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users



-
The information contained in this communication (including any
attachments hereto) is confidential and is intended solely for the
personal and confidential use of the individual or entity to whom
it is addressed. If the reader of this message is not the intended
recipient or an agent responsible for delivering it to the intended
recipient, you are hereby notified that you have received this
communication in error and that any review, dissemination, copying,
or unauthorized use of this information, or the taking of any
action in reliance on the contents of this information is strictly
prohibited. If you have received this communication in error,
please notify us immediately by e-mail, and delete the original
message. Thank you ___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

I did not assume that I've seen all use cases, nor that what I am a telling is 
the absolute truth. See no offense in my previous post.
Yes, I was supposing some use case of my own experience, but please note it is 
the first post when you describe what you are really trying to do, ie 
implementing a dymanic filter configured with a web form that generates rules 
to do the filtering job.
This is indeed highly dynamic, and for 10 rules, the cache will not be very 
relevant.
And you are perfectly correct when you say that if rules change at each 
request, using a cache won't speed the process. 

The cache is only usefull if the rules are stable for a time, or if only some 
few rules change from one time to the next one.
But if you have 10 rules, compile time should not be so long, so I guess that 
the guy writing filtering rules is expecting very quick results (so I guess 
that you have few data too, otherwise the compile time would be far much 
smaller that processing time, especially for 10 rules ...).


You want to test the adequation of drools for a use case where a user can set a 
set of filtering rules via a web form to filter some data somewhere ... 
I would say that :
   - If there is only simple filters you should not use drools (an in-memory DB 
is not so costly, at least not more than a drools' WM, and with hibernate you 
can write queries using a POJO-like syntax which is very close to rules' LHS)
   - You said that you are looking to drools to allow writing more complex 
rules than simple filter rules (like chaining and so on). In that case, SQL 
won't fit, and then you have to accept a performance overcome just because 
drools does much more than SQL. And to deal with that overcome, you have to 
think about partitionning/caching your rules (but for 10 rules, it will be 
difficult)


___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

And I did not see all pending posts ...

By reading them, there is another fact that lead to don't use drools.
You said that you query a distant DB. So, if you write rules to filter your 
data, you will have first to get ALL data from your distant DB, add ALL data in 
the drools WM, and only then fire rules on them to filter the data and get only 
a subset of them.

To me (my opinion), it is an anti-pattern. Filtering a large amount of data and 
return only a subset of them is the database job.

Oh, yes, you intend to do complex rules that cannot be set as SQL like syntax 
... 
You should think of cutting you system in two, by making each tool doing what 
it is designed for :
   - use SQL like for basic filtering, by sending the query to the distant DB 
and thus getting ONLY the data needed
   - make drools running of this subset of data to implement more complex things

___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

Vincent Legendre wrote

I did not assume that I've seen all use cases, nor that what I am a
telling is the absolute truth. See no offense in my previous post.

No offense taken at all. In fact, I would like to thank you for asking
important questions. These questions help me validate my design decisions.
Also, given that I'm new to Drools it also helps me understand it's
strengths and weaknesses.

Vincent Legendre wrote

Yes, I was supposing some use case of my own experience, but please note
it is the first post when you describe what you are really trying to do,
ie implementing a dymanic filter configured with a web form that generates
rules to do the filtering job.
This is indeed highly dynamic, and for 10 rules, the cache will not be
very relevant.
And you are perfectly correct when you say that if rules change at each
request, using a cache won't speed the process.

My example was to provide with you will enough context regarding a dynamic
filter where are highly dynamic. My user interface will be designed in such
a way to such that the user can change input values (which will then
generate the appropriate underlying rules.) So the user is not directly
writing rules.

Vincent Legendre wrote

You want to test the adequation of drools for a use case where a user can
set a set of filtering rules via a web form to filter some data somewhere
...
I would say that :
- If there is only simple filters you should not use drools (an
in-memory DB is not so costly, at least not more than a drools' WM, and
with hibernate you can write queries using a POJO-like syntax which is
very close to rules' LHS)

This looks like a good alternative. Let me explore this option. Thanks.

Vincent Legendre wrote

- You said that you are looking to drools to allow writing more complex
rules than simple filter rules (like chaining and so on). In that case,
SQL won't fit, and then you have to accept a performance overcome just
because drools does much more than SQL. And to deal with that overcome,
you have to think about partitionning/caching your rules (but for 10
rules, it will be difficult)

At this point I think I can accept the performance overhead. Being new to
Drools I was wondering if there is a way to make this stuff run faster.

--
View this message in context:
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3974999.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

Vincent Legendre wrote

And I did not see all pending posts ...

By reading them, there is another fact that lead to don't use drools.
You said that you query a distant DB. So, if you write rules to filter
your data, you will have first to get ALL data from your distant DB, add
ALL data in the drools WM, and only then fire rules on them to filter the
data and get only a subset of them.

To me (my opinion), it is an anti-pattern. Filtering a large amount of
data and return only a subset of them is the database job.

The filtering options provided by the remote databases (most of them are
exposed as REST APIs) are limited. I uses these options whenever I can to
filter data at the source. However, sometimes it's not possible to filter
everything at the source. In such cases I don't have any other option but to
get all the data and then apply my filters locally.

Vincent Legendre wrote

Oh, yes, you intend to do complex rules that cannot be set as SQL like
syntax ...
You should think of cutting you system in two, by making each tool doing
what it is designed for :
- use SQL like for basic filtering, by sending the query to the distant
DB and thus getting ONLY the data needed
- make drools running of this subset of data to implement more complex
things

Now as discussed below, I can apply simple filters by using a in-memory
database (or inserting the data into a SQL database and then using SQL
queries) OR load everything into a WM and then apply these dynamic rules.

I think both approaches can work, I just need to figure out the tradeoffs.

Thanks again !

--
View this message in context:
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3975015.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

 ... OR load everything into a WM and then apply these dynamic rules. 

Beware, I am not sure whether we can add/remove rules in an existing session (I 
don't think so but I may be wrong).
Basically, a WM is built from a kBase, not the inverse ...
So you will have to recompile rules, that re-add all data in the WM ... really 
not a good idea then ...

But, if you have your own GUI to set the filters, you should have some masks to 
speficify your filtering values, or at least some objects to store the 
filters/user's choices to display them in some screens, right ?

In that case, you can add these objects directly in the WM, and write generic 
rules that use these objects to filter the raw data (as 2 or 3 previous post 
suggest). This way, you build your KB once, create a 'deamon' session, add all 
raw data once, and then only insert/retract filtering objects, with the results 
updated dynamically too. And if you can only work on differences (ie a result 
entering or getting out the global result list by invoking a gui callback), it 
can even be faster than a DB ... because RETE is good to propagate small diffs 
over a large amount of data and deduce related diffs on the outcome (this is 
drools-planner raw idea too).




___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

Vincent Legendre wrote

... OR load everything into a WM and then apply these dynamic rules.

Beware, I am not sure whether we can add/remove rules in an existing
session (I don't think so but I may be wrong).
Basically, a WM is built from a kBase, not the inverse ...
So you will have to recompile rules, that re-add all data in the WM ...
really not a good idea then ...

But, if you have your own GUI to set the filters, you should have some
masks to speficify your filtering values, or at least some objects to
store the filters/user's choices to display them in some screens, right ?

In that case, you can add these objects directly in the WM, and write
generic rules that use these objects to filter the raw data (as 2 or 3
previous post suggest). This way, you build your KB once, create a
'deamon' session, add all raw data once, and then only insert/retract
filtering objects, with the results updated dynamically too. And if you
can only work on differences (ie a result entering or getting out the
global result list by invoking a gui callback), it can even be faster than
a DB ... because RETE is good to propagate small diffs over a large amount
of data and deduce related diffs on the outcome (this is drools-planner
raw idea too).

thanks again everyone ! I think I've enough information at this point to do
some more prototypes to actually get a better idea of all the important
things people have shared in this post.

I'm sure I'll have more implementation or drools specific questions once
I start exploring a little more.

Thanks.

--
View this message in context:
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3975132.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?

@Vincent: it should indeed be possible to hot-plug rules directly into a
knowledge BASE (meaning all sessions derived from that KB will be affected:
existing facts will be matched against the new rules). More typically, this
is done using a Knowledge Agent to manage the Knowledge Base.
Whether this is appropriate for this use case or not, is a totally different
matter, as thoroughly discussed :)

--
View this message in context: 
http://drools.46999.n3.nabble.com/Is-there-a-faster-way-of-doing-this-in-Drools-tp3973888p3975486.html
Sent from the Drools: User forum mailing list archive at Nabble.com.
___
rules-users mailing list
rules-users@lists.jboss.org
https://lists.jboss.org/mailman/listinfo/rules-users

Re: [rules-users] Is there a faster way of doing this in Drools ?