On 07/03/2012, gboro54 <[email protected]> wrote: > I have been reevaluating our use cases and have decided to make a couple > modifications:
> 4. With 3 in mind I am now trying to determine the best way to do this. My > thinking atm is to create x sessions(for this example let's say 5), each > associated with a thread via fireUntilHalt(). Since orders are streamed in > by another thread reading from the DB my thinking is this is the most > efficient way. However these leads me to a couple questions. How would I > determine that an order has been processes through the ruleset and is ready > for post processing(with fireAllRules I know that once the rules have fired > I can retract the Order and move it to post processing and perhaps this is > the answer that will be provided). With fireUntilHalt I thought about > raising an event which I would use CDI to listen for and then retract the > Order and pass it along for post processing. Is this a good approach? I am > open to any suggestion here. A low-salience rule ought to detect orders for which no other rules fire any more - that's the moment when it has to be kicked into post processing. > 5. After watching a video posted by salaboy(http://vimeo.com/27209589) I am > looking into some of the technologies he used. Particularly Kryo. The idea > would be that if an Order was eligible for certain post processing it would > be serialized to disc with a ranking(i.e key). Another part of the process > would then read one at a time and process them in order. Any thoughts around > this(I know this is not a drools question but I figured I would put it out > there anyway). For the session, provide an abstract service via a global that accepts the retracted order fact. Make postprocessing as independent as possible, could even RMI to another system if cycles are a bottleneck. Go to disk only if there's no other way, eg. when it is too slow and you need to spill to disk. -W > > Thanks Again > -Mike > > > > laune wrote >> >> A few random observations. >> >> A recent experiment has shown that insert takes 0.82 seconds for >> 100000 facts with one very simple rule with a CPU G620 @ 2.60GHz and: >> java version "1.7.0_02" >> Java(TM) SE Runtime Environment (build 1.7.0_02-b13) >> Java HotSpot(TM) 64-Bit Server VM (build 22.0-b10, mixed mode) >> >> There is, of course, no upper limit for insert as this depends on the >> number and complexity of the rules. And if you apply the jBPM Process >> paradigm, I'm not prepared to make any guesses. (It stands to reason >> that this will incur more overhead than straightforward processing in >> the Expert engine.) >> >> Estimating 1K per order, I'd say that i/o time for 150M orders is not >> to be neglected. >> >> I'd investigate batch (but not bulk) insertion of quantities, so that >> inserts and fire calls remain in good proportion. Also, balancing i/o >> threads and processing threads in relation to available CPUs shouldn't >> be different for an application using Drools. >> >> I've been re-reading some of this thread, and the requirement of >> ordered processing is going to be very difficult if the order this to >> be maintained over 150M facts. I mentioned dynamic property, but many >> different salience levels for a large number of activations is not >> efficient, since this requires a sequential search. >> >> -W >> >> >> On 3 March 2012 21:30, gboro54 <gboro54@> wrote: >>> >>> My previous statement was poorly written(that is what I get for trying to >>> respond via my cell phone ). I agree that the Order fact is simple enough >>> to >>> have a rule retract it once processing is completed. As far as the >>> threading, this is an ejb application being deployed on JBoss 7.1. >>> Because >>> of this I was hoping to use @Asyn and allow the container to really >>> manage >>> the threads. This method would simply request the cached session insert >>> the >>> Order fact, and start the jBPM process to process the Order. Perhaps this >>> part of the design is still flawed as I have only recently been thinking >>> about this and I would welcome all suggestions. >>> >>> Additionally is there any documentation on utilizing drools in a high >>> throughput environment(i.e white papers,etc.) This billing project is the >>> first of many potential projects we are looking to utilize drools for and >>> I >>> would love to share some more documentation on this(as well as read on >>> myself). >>> >>> >>> >>> gboro54 wrote >>> > >>> > Agreed that I can have a rule to retract the order....i was just >>> thinking >>> > if the overhead of a reevaluation but can live with that...As far as >>> the >>> > threading...this logic is going to be deployed on a jboss instance >>> using >>> > ejb so I was thinking about, using the jboss thread pool...so when I >>> say >>> > new thread it may not, be new put, pulled from a pool....does that >>> sound >>> > reasonable? >>> > >>> > >>> > >>> > laune wrote >>> >> >>> >> Starting a new thread (if that's what you mean by process) for reusing >>> >> an existing session to process another order is likely to create more >>> >> overhead >>> >> and it'll just make threads compete for this resources. >>> >> >>> >> Multiple threads each dedicated to a single session object might be a >>> >> better way to go. >>> >> >>> >> I don't see any benefit to make simple order facts into events just >>> for >>> >> the >>> >> sake >>> >> of making them expire automatically. There ought to be a well defined >>> >> state (or states) when processed orders are retracted by some rule. >>> >> >>> >> -W >>> >> >>> >> >>> >> On 3 March 2012 14:11, gboro54 <gboro54@> wrote: >>> >> >>> >>> Mark, >>> >>> >>> >>> I really appreciate the help and think I have come to a solution. >>> Let >>> >>> me >>> >>> know if this sounds reasonable. >>> >>> 1. We will continue to use jBPM to coordinate the rules and business >>> >>> logic >>> >>> that need to occur in calculating charges for orders. However we will >>> >>> work >>> >>> the process to only work on a per order level. >>> >>> 2. All orders are associated with accounts. We will keep one >>> >>> knowledgebase >>> >>> as the rule sets are the same and we will partition sessions by >>> >>> accounts. >>> >>> The flow will go as follows: >>> >>> a. If the session exists insert the order, start a new process >>> instance >>> >>> and >>> >>> fire all rules >>> >>> b. If the session has not been created: create the session, insert >>> all >>> >>> reference data that will be used by all orders in executions of the >>> rule >>> >>> set, insert the order, start a process, and fire all rules. This >>> session >>> >>> is >>> >>> then cached(via a hashmap more then likely) >>> >>> c. This process will be invoked asyn from the main thread, allowing >>> us >>> >>> to >>> >>> control the multithreading >>> >>> 3. Orders will be treated as an event and will be set to expire in a >>> >>> certain >>> >>> amount of time, allowing us to mange the memory footprint of each >>> >>> session. >>> >>> >>> >>> Does this sound reasonable based on what you know of our usecase? >>> >>> Additionally with expiring Orders does this cause a reevaluation of >>> the >>> >>> rules when the expiration occurs? The only other question I have is >>> does >>> >>> the >>> >>> expiration clock start when no more activation's get created for the >>> >>> given >>> >>> event? >>> >>> >>> >>> Thanks again. >>> >>> >>> >>> -- >>> >>> View this message in context: >>> >>> >>> http://drools.46999.n3.nabble.com/Drools-5-3-partitioned-rule-base-tp3793558p3795920.html >>> >>> Sent from the Drools: User forum mailing list archive at Nabble.com. >>> >>> _______________________________________________ >>> >>> rules-users mailing list >>> >>> [email protected] >>> >>> https://lists.jboss.org/mailman/listinfo/rules-users >>> >>> >>> >> >>> >> _______________________________________________ >>> >> rules-users mailing list >>> >> [email protected] >>> >> https://lists.jboss.org/mailman/listinfo/rules-users >>> >> >>> > >>> >>> >>> -- >>> View this message in context: >>> http://drools.46999.n3.nabble.com/Drools-5-3-partitioned-rule-base-tp3793558p3796689.html >>> Sent from the Drools: User forum mailing list archive at Nabble.com. >>> _______________________________________________ >>> rules-users mailing list >>> [email protected] >>> https://lists.jboss.org/mailman/listinfo/rules-users >> >> _______________________________________________ >> rules-users mailing list >> [email protected] >> https://lists.jboss.org/mailman/listinfo/rules-users >> > > -- > View this message in context: > http://drools.46999.n3.nabble.com/Drools-5-3-partitioned-rule-base-tp3793558p3806627.html > Sent from the Drools: User forum mailing list archive at Nabble.com. > > _______________________________________________ > rules-users mailing list > [email protected] > https://lists.jboss.org/mailman/listinfo/rules-users > _______________________________________________ rules-users mailing list [email protected] https://lists.jboss.org/mailman/listinfo/rules-users
