Re: [rules-users] Workaround: EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule
This issue has been resolved by https://issues.jboss.org/browse/DROOLS-516 (a discussion in the comments in https://issues.jboss.org/browse/DROOLS-498 is relevant, as this issue was originally reported as related to DROOLS-498). On Jul 14, 2014, at 6:11 PM, Davide Sottara dso...@gmail.com wrote: Thanks, we'll look at it tomorrow Davide On 07/15/2014 02:09 AM, Kent Anderson wrote: https://github.com/droolsjbpm/drools/pull/358 Per your advice, this pull request sets up the memory leak, but it cannot verify that it exists without diving into internal drools structures. Please let me know what else we can do to facilitate fixing this issue. On Jul 14, 2014, at 2:09 PM, Davide Sottara dso...@gmail.com wrote: I'd need to check, but if you can put together the basics of the test case - the combination of rules and facts that causes the leak, we'll add the checks to ensure it's solved. Mark should have given you enough pointers to the style we use for test cases. We really appreciate your collaboration, and since this is a critical bug we'll fix it asap Thanks!! Davide On 07/14/2014 08:33 PM, Kent Anderson wrote: Is there a reliable way to find an EventFactHandle instance buried within the TupleEntryQueue’s and the PhreakPropagationContext’s? The multiple containers of interface/impl’s makes it nearly impossible to write a “good” test that can peak under the hood where this memory leak is being held. If there is a recommended way to do it, I’d be happy to do so. Mail Attachment.png On Jul 13, 2014, at 7:31 PM, Mark Proctor mproc...@codehaus.org wrote: Could you submit a unit test as a pull request? http://docs.jboss.org/drools/release/5.5.0.Final/droolsjbpm-introduction-docs/html/gettingstarted.html Add it to here, and follow existing conventions: https://github.com/droolsjbpm/drools/blob/master/drools-compiler/src/test/java/org/drools/compiler/integrationtests/CepEspTest.java Mark On 11 Jul 2014, at 20:38, Kent Anderson kent.ander...@psware.com wrote: We have found a workaround that eliminates the leftover event (gone from Working Memory, but not from the JVM memory): The rule “forget it ever happened” (seen below) causes the problem. Re-writing it to remove the check for RAISE in the LHS eliminated the memory leak. Of course, our application requires the check for RAISE, so it can be accomplished by manually querying working memory from the RHS. It’s ugly, but it resolved the issue. query existsRaise($id) $raise : MyEvent( eventState == EventState.RAISE, eventId == $id ) end rule process clear no-loop when $clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId) then QueryResults results = kcontext.getKieRuntime().getQueryResults( existsRaise, $clearId ); if (results.size() == 0) { System.out.println( Forwarding CLEAR( + $clearId + ) ); } else { System.out.println(Forgetting RAISE/CLEAR( + $clearId + )); for (QueryResultsRow row : results){ MyEvent raise = (MyEvent) row.get ($raise); delete(raise); } } delete($clear); end This appears to be a similar situation to https://issues.jboss.org/browse/DROOLS-498. On Jul 10, 2014, at 3:54 PM, Kent Anderson kent.ander...@psware.com wrote: Correction: The original post did not include another rule that exists in the stream. The memory leak does not appear unless both rules are active in the stream. declare MyEvent @role(event) @timestamp(timestamp) end /* If a RAISE is buffered for N seconds, send it out */ rule forward raise no-loop duration (3s) when $raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId) then System.out.println(Forwarding RAISE( + $raiseId + )); delete($raise); end /* When CLEAR, and buffered, clear them both out */ rule forget it ever happened no-loop when $clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId) $raise : MyEvent(eventState == EventState.RAISE, eventId == $clearId) then System.out.println(Forgetting RAISE/CLEAR( + $clearId + )); delete($clear); delete($raise); end On Jul 10, 2014, at 2:50 PM, Kent Anderson kent.ander...@psware.com wrote: The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT: (Stream mode) declare MyEvent @role(event) @timestamp(timestamp) end /* If a RAISE is buffered for N seconds, send it out */ rule forward raise no-loop duration (3s) when $raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId) then
Re: [rules-users] Workaround: EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule
https://github.com/droolsjbpm/drools/pull/358 Per your advice, this pull request sets up the memory leak, but it cannot verify that it exists without diving into internal drools structures. Please let me know what else we can do to facilitate fixing this issue. On Jul 14, 2014, at 2:09 PM, Davide Sottara dso...@gmail.com wrote: I'd need to check, but if you can put together the basics of the test case - the combination of rules and facts that causes the leak, we'll add the checks to ensure it's solved. Mark should have given you enough pointers to the style we use for test cases. We really appreciate your collaboration, and since this is a critical bug we'll fix it asap Thanks!! Davide On 07/14/2014 08:33 PM, Kent Anderson wrote: Is there a reliable way to find an EventFactHandle instance buried within the TupleEntryQueue’s and the PhreakPropagationContext’s? The multiple containers of interface/impl’s makes it nearly impossible to write a “good” test that can peak under the hood where this memory leak is being held. If there is a recommended way to do it, I’d be happy to do so. Mail Attachment.png On Jul 13, 2014, at 7:31 PM, Mark Proctor mproc...@codehaus.org wrote: Could you submit a unit test as a pull request? http://docs.jboss.org/drools/release/5.5.0.Final/droolsjbpm-introduction-docs/html/gettingstarted.html Add it to here, and follow existing conventions: https://github.com/droolsjbpm/drools/blob/master/drools-compiler/src/test/java/org/drools/compiler/integrationtests/CepEspTest.java Mark On 11 Jul 2014, at 20:38, Kent Anderson kent.ander...@psware.com wrote: We have found a workaround that eliminates the leftover event (gone from Working Memory, but not from the JVM memory): The rule “forget it ever happened” (seen below) causes the problem. Re-writing it to remove the check for RAISE in the LHS eliminated the memory leak. Of course, our application requires the check for RAISE, so it can be accomplished by manually querying working memory from the RHS. It’s ugly, but it resolved the issue. query existsRaise($id) $raise : MyEvent( eventState == EventState.RAISE, eventId == $id ) end rule process clear no-loop when $clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId) then QueryResults results = kcontext.getKieRuntime().getQueryResults( existsRaise, $clearId ); if (results.size() == 0) { System.out.println( Forwarding CLEAR( + $clearId + ) ); } else { System.out.println(Forgetting RAISE/CLEAR( + $clearId + )); for (QueryResultsRow row : results){ MyEvent raise = (MyEvent) row.get ($raise); delete(raise); } } delete($clear); end This appears to be a similar situation to https://issues.jboss.org/browse/DROOLS-498. On Jul 10, 2014, at 3:54 PM, Kent Anderson kent.ander...@psware.com wrote: Correction: The original post did not include another rule that exists in the stream. The memory leak does not appear unless both rules are active in the stream. declare MyEvent @role(event) @timestamp(timestamp) end /* If a RAISE is buffered for N seconds, send it out */ rule forward raise no-loop duration (3s) when $raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId) then System.out.println(Forwarding RAISE( + $raiseId + )); delete($raise); end /* When CLEAR, and buffered, clear them both out */ rule forget it ever happened no-loop when $clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId) $raise : MyEvent(eventState == EventState.RAISE, eventId == $clearId) then System.out.println(Forgetting RAISE/CLEAR( + $clearId + )); delete($clear); delete($raise); end On Jul 10, 2014, at 2:50 PM, Kent Anderson kent.ander...@psware.com wrote: The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT: (Stream mode) declare MyEvent @role(event) @timestamp(timestamp) end /* If a RAISE is buffered for N seconds, send it out */ rule forward raise no-loop duration (3s) when $raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId) then System.out.println(Forwarding RAISE( + $raiseId + )); delete($raise); end I see the rule fire as expected, printing out the message 3 seconds after the event is added into the session. While the event is waiting, I see a FactCount of 1 in the session. After the rule fires, the fact count goes to 0. However, using JVisualVm, querying the heap dump shows 1 instance of MyEvent, referenced by an EventFactHandle and several other Drools objects. Is this a bug, or is there a better way to write this rule so Drools’ internals let go of the object after it is no longer a fact? PastedGraphic-1.png PastedGraphic-2.png ___ rules-users mailing list
Re: [rules-users] Workaround: EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule
Thanks, we'll look at it tomorrow Davide On 07/15/2014 02:09 AM, Kent Anderson wrote: https://github.com/droolsjbpm/drools/pull/358 Per your advice, this pull request sets up the memory leak, but it cannot verify that it exists without diving into internal drools structures. Please let me know what else we can do to facilitate fixing this issue. On Jul 14, 2014, at 2:09 PM, Davide Sottara dso...@gmail.com wrote: I'd need to check, but if you can put together the basics of the test case - the combination of rules and facts that causes the leak, we'll add the checks to ensure it's solved. Mark should have given you enough pointers to the style we use for test cases. We really appreciate your collaboration, and since this is a critical bug we'll fix it asap Thanks!! Davide On 07/14/2014 08:33 PM, Kent Anderson wrote: Is there a reliable way to find an EventFactHandle instance buried within the TupleEntryQueue's and the PhreakPropagationContext's? The multiple containers of interface/impl's makes it nearly impossible to write a good test that can peak under the hood where this memory leak is being held. If there is a recommended way to do it, I'd be happy to do so. Mail Attachment.png On Jul 13, 2014, at 7:31 PM, Mark Proctor mproc...@codehaus.org mailto:mproc...@codehaus.org wrote: Could you submit a unit test as a pull request? http://docs.jboss.org/drools/release/5.5.0.Final/droolsjbpm-introduction-docs/html/gettingstarted.html Add it to here, and follow existing conventions: https://github.com/droolsjbpm/drools/blob/master/drools-compiler/src/test/java/org/drools/compiler/integrationtests/CepEspTest.java Mark On 11 Jul 2014, at 20:38, Kent Anderson kent.ander...@psware.com mailto:kent.ander...@psware.com wrote: We have found a workaround that eliminates the leftover event (gone from Working Memory, but not from the JVM memory): The rule forget it ever happened (seen below) causes the problem. Re-writing it to remove the check for RAISE in the LHS eliminated the memory leak. Of course, our application requires the check for RAISE, so it can be accomplished by manually querying working memory from the RHS. It's ugly, but it resolved the issue. query existsRaise($id) $raise : MyEvent( eventState == EventState.RAISE, eventId == $id ) end rule process clear no-loop when $clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId) then QueryResults results = kcontext.getKieRuntime().getQueryResults( existsRaise, $clearId ); if (results.size() == 0) { System.out.println( Forwarding CLEAR( + $clearId + ) ); } else { System.out.println(Forgetting RAISE/CLEAR( + $clearId + )); for (QueryResultsRow row : results){ MyEvent raise = (MyEvent) row.get ($raise); delete(raise); } } delete($clear); end This appears to be a similar situation to https://issues.jboss.org/browse/DROOLS-498. On Jul 10, 2014, at 3:54 PM, Kent Anderson kent.ander...@psware.com mailto:kent.ander...@psware.com wrote: Correction: The original post did not include another rule that exists in the stream. The memory leak does not appear unless both rules are active in the stream. declare MyEvent @role(event) @timestamp(timestamp) end /* If a RAISE is buffered for N seconds, send it out */ rule forward raise no-loop duration (3s) when $raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId) then System.out.println(Forwarding RAISE( + $raiseId + )); delete($raise); end /* When CLEAR, and buffered, clear them both out */ rule forget it ever happened no-loop when $clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId) $raise : MyEvent(eventState == EventState.RAISE, eventId == $clearId) then System.out.println(Forgetting RAISE/CLEAR( + $clearId + )); delete($clear); delete($raise); end On Jul 10, 2014, at 2:50 PM, Kent Anderson kent.ander...@psware.com mailto:kent.ander...@psware.com wrote: The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT: (Stream mode) declare MyEvent @role(event) @timestamp(timestamp) end /* If a RAISE is buffered for N seconds, send it out */ rule forward raise no-loop duration (3s) when $raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId) then System.out.println(Forwarding RAISE( + $raiseId + )); delete($raise); end I see the rule fire as expected, printing out the message 3 seconds after the event is added into the session. While the event is waiting, I see a FactCount of 1 in the session. After the rule fires, the fact count goes to 0. However, using JVisualVm, querying the heap dump shows 1 instance of MyEvent, referenced by an EventFactHandle and several other Drools objects. Is this a bug, or is there a better way to write this rule so Drools' internals let go of the object after it is no longer a fact?
Re: [rules-users] Workaround: EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule
Could you submit a unit test as a pull request? http://docs.jboss.org/drools/release/5.5.0.Final/droolsjbpm-introduction-docs/html/gettingstarted.html Add it to here, and follow existing conventions: https://github.com/droolsjbpm/drools/blob/master/drools-compiler/src/test/java/org/drools/compiler/integrationtests/CepEspTest.java Mark On 11 Jul 2014, at 20:38, Kent Anderson kent.ander...@psware.com wrote: We have found a workaround that eliminates the leftover event (gone from Working Memory, but not from the JVM memory): The rule “forget it ever happened” (seen below) causes the problem. Re-writing it to remove the check for RAISE in the LHS eliminated the memory leak. Of course, our application requires the check for RAISE, so it can be accomplished by manually querying working memory from the RHS. It’s ugly, but it resolved the issue. query existsRaise($id) $raise : MyEvent( eventState == EventState.RAISE, eventId == $id ) end rule process clear no-loop when $clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId) then QueryResults results = kcontext.getKieRuntime().getQueryResults( existsRaise, $clearId ); if (results.size() == 0) { System.out.println( Forwarding CLEAR( + $clearId + ) ); } else { System.out.println(Forgetting RAISE/CLEAR( + $clearId + )); for (QueryResultsRow row : results){ MyEvent raise = (MyEvent) row.get ($raise); delete(raise); } } delete($clear); end This appears to be a similar situation to https://issues.jboss.org/browse/DROOLS-498. On Jul 10, 2014, at 3:54 PM, Kent Anderson kent.ander...@psware.com wrote: Correction: The original post did not include another rule that exists in the stream. The memory leak does not appear unless both rules are active in the stream. declare MyEvent @role(event) @timestamp(timestamp) end /* If a RAISE is buffered for N seconds, send it out */ rule forward raise no-loop duration (3s) when $raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId) then System.out.println(Forwarding RAISE( + $raiseId + )); delete($raise); end /* When CLEAR, and buffered, clear them both out */ rule forget it ever happened no-loop when $clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId) $raise : MyEvent(eventState == EventState.RAISE, eventId == $clearId) then System.out.println(Forgetting RAISE/CLEAR( + $clearId + )); delete($clear); delete($raise); end On Jul 10, 2014, at 2:50 PM, Kent Anderson kent.ander...@psware.com wrote: The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT: (Stream mode) declare MyEvent @role(event) @timestamp(timestamp) end /* If a RAISE is buffered for N seconds, send it out */ rule forward raise no-loop duration (3s) when $raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId) then System.out.println(Forwarding RAISE( + $raiseId + )); delete($raise); end I see the rule fire as expected, printing out the message 3 seconds after the event is added into the session. While the event is waiting, I see a FactCount of 1 in the session. After the rule fires, the fact count goes to 0. However, using JVisualVm, querying the heap dump shows 1 instance of MyEvent, referenced by an EventFactHandle and several other Drools objects. Is this a bug, or is there a better way to write this rule so Drools’ internals let go of the object after it is no longer a fact? PastedGraphic-1.png PastedGraphic-2.png ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Workaround: EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule
Thanks for reporting, we'll investigate both this and 498 Davide On 07/11/2014 08:38 PM, Kent Anderson wrote: We have found a workaround that eliminates the leftover event (gone from Working Memory, but not from the JVM memory): The rule forget it ever happened (seen below) causes the problem. Re-writing it to remove the check for RAISE in the LHS eliminated the memory leak. Of course, our application requires the check for RAISE, so it can be accomplished by manually querying working memory from the RHS. It's ugly, but it resolved the issue. query existsRaise($id) $raise : MyEvent( eventState == EventState.RAISE, eventId == $id ) end ruleprocess clear no-loop when $clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId) then QueryResults results = kcontext.getKieRuntime().getQueryResults( existsRaise, $clearId ); if (results.size() == 0) { System.out.println( Forwarding CLEAR( + $clearId + ) ); } else { System.out.println(Forgetting RAISE/CLEAR( + $clearId + )); for (QueryResultsRow row : results){ MyEvent raise = (MyEvent) row.get ($raise); delete(raise); } } delete($clear); end This appears to be a similar situation to https://issues.jboss.org/browse/DROOLS-498. On Jul 10, 2014, at 3:54 PM, Kent Anderson kent.ander...@psware.com mailto:kent.ander...@psware.com wrote: Correction: The original post did not include another rule that exists in the stream. The memory leak does not appear unless both rules are active in the stream. declare MyEvent @role(event) @timestamp(timestamp) end /* If a RAISE is buffered for N seconds, send it out */ ruleforward raise no-loop duration(3s) when $raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId) then System.out.println(Forwarding RAISE( + $raiseId + )); delete($raise); end /* When CLEAR, and buffered, clear them both out */ ruleforget it ever happened no-loop when $clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId) $raise : MyEvent(eventState == EventState.RAISE, eventId == $clearId) then System.out.println(Forgetting RAISE/CLEAR( + $clearId + )); delete($clear); delete($raise); end On Jul 10, 2014, at 2:50 PM, Kent Anderson kent.ander...@psware.com mailto:kent.ander...@psware.com wrote: The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT: (Stream mode) declare MyEvent @role(event) @timestamp(timestamp) end /* If a RAISE is buffered for N seconds, send it out */ ruleforward raise no-loop duration(3s) when $raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId) then System.out.println(Forwarding RAISE( + $raiseId + )); delete($raise); end I see the rule fire as expected, printing out the message 3 seconds after the event is added into the session. While the event is waiting, I see a FactCount of 1 in the session. After the rule fires, the fact count goes to 0. However, using JVisualVm, querying the heap dump shows 1 instance of MyEvent, referenced by an EventFactHandle and several other Drools objects. Is this a bug, or is there a better way to write this rule so Drools' internals let go of the object after it is no longer a fact? PastedGraphic-1.png PastedGraphic-2.png ___ rules-users mailing list rules-users@lists.jboss.org mailto:rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users ___ rules-users mailing list rules-users@lists.jboss.org mailto:rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users
Re: [rules-users] Workaround: EventFactHandle retained in memory after FactCount reaches 0 for temporal-based rule
We have found a workaround that eliminates the leftover event (gone from Working Memory, but not from the JVM memory): The rule “forget it ever happened” (seen below) causes the problem. Re-writing it to remove the check for RAISE in the LHS eliminated the memory leak. Of course, our application requires the check for RAISE, so it can be accomplished by manually querying working memory from the RHS. It’s ugly, but it resolved the issue. query existsRaise($id) $raise : MyEvent( eventState == EventState.RAISE, eventId == $id ) end rule process clear no-loop when $clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId) then QueryResults results = kcontext.getKieRuntime().getQueryResults( existsRaise, $clearId ); if (results.size() == 0) { System.out.println( Forwarding CLEAR( + $clearId + ) ); } else { System.out.println(Forgetting RAISE/CLEAR( + $clearId + )); for (QueryResultsRow row : results){ MyEvent raise = (MyEvent) row.get ($raise); delete(raise); } } delete($clear); end This appears to be a similar situation to https://issues.jboss.org/browse/DROOLS-498. On Jul 10, 2014, at 3:54 PM, Kent Anderson kent.ander...@psware.com wrote: Correction: The original post did not include another rule that exists in the stream. The memory leak does not appear unless both rules are active in the stream. declare MyEvent @role(event) @timestamp(timestamp) end /* If a RAISE is buffered for N seconds, send it out */ rule forward raise no-loop duration (3s) when $raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId) then System.out.println(Forwarding RAISE( + $raiseId + )); delete($raise); end /* When CLEAR, and buffered, clear them both out */ rule forget it ever happened no-loop when $clear : MyEvent(eventState == EventState.CLEAR, $clearId : eventId) $raise : MyEvent(eventState == EventState.RAISE, eventId == $clearId) then System.out.println(Forgetting RAISE/CLEAR( + $clearId + )); delete($clear); delete($raise); end On Jul 10, 2014, at 2:50 PM, Kent Anderson kent.ander...@psware.com wrote: The following rule produces a memory leak in Drools 6.1.0-SNAPSHOT: (Stream mode) declare MyEvent @role(event) @timestamp(timestamp) end /* If a RAISE is buffered for N seconds, send it out */ rule forward raise no-loop duration (3s) when $raise : MyEvent(eventState == EventState.RAISE, $raiseId : eventId) then System.out.println(Forwarding RAISE( + $raiseId + )); delete($raise); end I see the rule fire as expected, printing out the message 3 seconds after the event is added into the session. While the event is waiting, I see a FactCount of 1 in the session. After the rule fires, the fact count goes to 0. However, using JVisualVm, querying the heap dump shows 1 instance of MyEvent, referenced by an EventFactHandle and several other Drools objects. Is this a bug, or is there a better way to write this rule so Drools’ internals let go of the object after it is no longer a fact? PastedGraphic-1.png PastedGraphic-2.png ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users ___ rules-users mailing list rules-users@lists.jboss.org https://lists.jboss.org/mailman/listinfo/rules-users