[jira] [Commented] (OOZIE-3409) Oozie Server : Possible memory leak in EL evaluation
[
https://issues.apache.org/jira/browse/OOZIE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16744906#comment-16744906
]
Andras Salamon commented on OOZIE-3409:
---
DO NOT MERGE YET
This patch fixes the {{TestLocalOozieExample}} problem.
I wanted to move ELEvaluator.checkForExistence method to a separate class, but
I realized that we already have a {{StringUtils}} class in the package, so I
moved it to there (and renamed it).
Because this method no longer uses EL evaluation, but only relies on String
manipulation, I was able to simplify the code a bit.
Still need more testing (the most important one is to heck for memory leaks)
but I upload the patch as is, because the precommit build might catch more
problems ignored by my grind tests.
> Oozie Server : Possible memory leak in EL evaluation
>
>
> Key: OOZIE-3409
> URL: https://issues.apache.org/jira/browse/OOZIE-3409
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Praneeth
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3409-01.patch, OOZIE-3409-02.patch
>
>
> I have noticed that there is a potential memory leak in the Oozie server due
> to one of its dependencies - {{commons-el:commons-el:1.0}}. Our oozie servers
> have been running out of memory occasionally. It happens more often when we
> have more activity. Upon analysis of the heap dump, it seems that the culprit
> here is the {{org.apache.commons.el.ExpressionEvaluatorImpl}} from
> {{commons-el}} which is used in the Oozie server through
> {{org.apache.oozie.util.ELEvaluator}} [1] . {{ExpressionEvaluatorImpl}} uses
> a static {{Map}} [2] to cache the EL evaluations. This map never gets
> cleared, eventually taking up all the memory. So, in an environment with
> heavy usage and long running oozie servers, this would lead to an OOM. The
> configurations for workflow actions we have are possibly larger than normal
> which is worsening the situation as the Map has to maintain larger strings.
> #
> [https://github.com/apache/oozie/blob/release-5.1.0/core/src/main/java/org/apache/oozie/util/ELEvaluator.java#L143]
> #
> [http://svn.apache.org/viewvc/commons/dormant/el/trunk/src/java/org/apache/commons/el/ExpressionEvaluatorImpl.java?view=markup#l94]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3409) Oozie Server : Possible memory leak in EL evaluation
[
https://issues.apache.org/jira/browse/OOZIE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16742913#comment-16742913
]
Andras Salamon commented on OOZIE-3409:
---
{{TestLocalOozieExample}} failure is caused by this patch, I need to work on it:
{noformat}
20:51:02 [INFO] ---
20:51:02 [INFO] T E S T S
20:51:02 [INFO] ---
20:51:28 [INFO] Running org.apache.oozie.example.TestLocalOozieExample
20:51:28 [ERROR] Tests run: 2, Failures: 0, Errors: 2, Skipped: 0, Time
elapsed: 25.311 s <<< FAILURE! - in
org.apache.oozie.example.TestLocalOozieExample
20:51:28 [ERROR]
testLocalOozieExampleEnd(org.apache.oozie.example.TestLocalOozieExample) Time
elapsed: 24.524 s <<< ERROR!
20:51:28 java.lang.NoSuchFieldError: IS_SECURITY_ENABLED
20:51:28at
org.apache.jasper.el.ELContextImpl.(ELContextImpl.java:83)
20:51:28at
org.apache.jasper.el.ExpressionEvaluatorImpl.parseExpression(ExpressionEvaluatorImpl.java:42)
20:51:28at
org.apache.jasper.el.ExpressionEvaluatorImpl.evaluate(ExpressionEvaluatorImpl.java:59)
20:51:28at
org.apache.oozie.util.ELEvaluator.evaluate(ELEvaluator.java:210)
{noformat}
> Oozie Server : Possible memory leak in EL evaluation
>
>
> Key: OOZIE-3409
> URL: https://issues.apache.org/jira/browse/OOZIE-3409
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Praneeth
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3409-01.patch
>
>
> I have noticed that there is a potential memory leak in the Oozie server due
> to one of its dependencies - {{commons-el:commons-el:1.0}}. Our oozie servers
> have been running out of memory occasionally. It happens more often when we
> have more activity. Upon analysis of the heap dump, it seems that the culprit
> here is the {{org.apache.commons.el.ExpressionEvaluatorImpl}} from
> {{commons-el}} which is used in the Oozie server through
> {{org.apache.oozie.util.ELEvaluator}} [1] . {{ExpressionEvaluatorImpl}} uses
> a static {{Map}} [2] to cache the EL evaluations. This map never gets
> cleared, eventually taking up all the memory. So, in an environment with
> heavy usage and long running oozie servers, this would lead to an OOM. The
> configurations for workflow actions we have are possibly larger than normal
> which is worsening the situation as the Map has to maintain larger strings.
> #
> [https://github.com/apache/oozie/blob/release-5.1.0/core/src/main/java/org/apache/oozie/util/ELEvaluator.java#L143]
> #
> [http://svn.apache.org/viewvc/commons/dormant/el/trunk/src/java/org/apache/commons/el/ExpressionEvaluatorImpl.java?view=markup#l94]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3409) Oozie Server : Possible memory leak in EL evaluation
[
https://issues.apache.org/jira/browse/OOZIE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733463#comment-16733463
]
Hadoop QA commented on OOZIE-3409:
--
Testing JIRA OOZIE-3409
Cleaning local git workspace
{color:green}+1 PATCH_APPLIES{color}
{color:green}+1 CLEAN{color}
{color:red}-1 RAW_PATCH_ANALYSIS{color}
.{color:green}+1{color} the patch does not introduce any @author tags
.{color:green}+1{color} the patch does not introduce any tabs
.{color:green}+1{color} the patch does not introduce any trailing spaces
.{color:green}+1{color} the patch does not introduce any star imports
.{color:red}-1{color} the patch contains 1 line(s) longer than 132
characters
.{color:green}+1{color} the patch adds/modifies 2 testcase(s)
{color:green}+1 RAT{color}
.{color:green}+1{color} the patch does not seem to introduce new RAT
warnings
{color:green}+1 JAVADOC{color}
.{color:green}+1{color} Javadoc generation succeeded with the patch
.{color:green}+1{color} the patch does not seem to introduce new Javadoc
warning(s)
.{color:orange}WARNING{color}: the current HEAD has 100 Javadoc warning(s)
{color:green}+1 COMPILE{color}
.{color:green}+1{color} HEAD compiles
.{color:green}+1{color} patch compiles
.{color:green}+1{color} the patch does not seem to introduce new javac
warnings
{color:green}+1{color} There are no new bugs found in total.
. {color:green}+1{color} There are no new bugs found in [sharelib/hive2].
. {color:green}+1{color} There are no new bugs found in [sharelib/spark].
. {color:green}+1{color} There are no new bugs found in [sharelib/oozie].
. {color:green}+1{color} There are no new bugs found in [sharelib/pig].
. {color:green}+1{color} There are no new bugs found in [sharelib/streaming].
. {color:green}+1{color} There are no new bugs found in [sharelib/hive].
. {color:green}+1{color} There are no new bugs found in [sharelib/distcp].
. {color:green}+1{color} There are no new bugs found in [sharelib/hcatalog].
. {color:green}+1{color} There are no new bugs found in [sharelib/sqoop].
. {color:green}+1{color} There are no new bugs found in [sharelib/git].
. {color:green}+1{color} There are no new bugs found in [client].
. {color:green}+1{color} There are no new bugs found in [docs].
. {color:green}+1{color} There are no new bugs found in [tools].
. {color:green}+1{color} There are no new bugs found in
[fluent-job/fluent-job-api].
. {color:green}+1{color} There are no new bugs found in [server].
. {color:green}+1{color} There are no new bugs found in [webapp].
. {color:green}+1{color} There are no new bugs found in [examples].
. {color:green}+1{color} There are no new bugs found in [core].
{color:green}+1 BACKWARDS_COMPATIBILITY{color}
.{color:green}+1{color} the patch does not change any JPA
Entity/Colum/Basic/Lob/Transient annotations
.{color:green}+1{color} the patch does not modify JPA files
{color:red}-1 TESTS{color}
.Tests run: 3106
.Tests failed : 0
.Tests in error : 2
.Tests timed out : 0
{color:red}-1{color} [ERROR] There are [2] test errors in [examples]. Listing
only the first [5] ones
testLocalOozieExampleEnd:org.apache.oozie.example.TestLocalOozieExample
testLocalOozieExampleKill:org.apache.oozie.example.TestLocalOozieExample
Check console output for the full list of errors/failures
.{color:orange}Tests failed at first run:{color}
TestPurgeXCommand#testPurgeCoordWithWFChild1
.For the complete list of flaky tests, see TEST-SUMMARY-FULL files.
{color:green}+1 DISTRO{color}
.{color:green}+1{color} distro tarball builds with the patch
{color:red}*-1 Overall result, please check the reported -1(s)*{color}
{color:red}. There is at least one warning, please check{color}
The full output of the test-patch run is available at
. https://builds.apache.org/job/PreCommit-OOZIE-Build/961/
> Oozie Server : Possible memory leak in EL evaluation
>
>
> Key: OOZIE-3409
> URL: https://issues.apache.org/jira/browse/OOZIE-3409
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Praneeth
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3409-01.patch
>
>
> I have noticed that there is a potential memory leak in the Oozie server due
> to one of its dependencies - {{commons-el:commons-el:1.0}}. Our oozie servers
> have been running out of memory occasionally. It happens more often when we
> have more activity. Upon analysis of the heap dump, it seems that the culprit
> here is the {{org.apache.commons.el.ExpressionEvaluatorImpl}} from
> {{commons-el}} which is used in the Oozie server through
> {{org.apache.oozie.util.ELEvaluator}} [1] . {{ExpressionEvaluatorImpl}} uses
> a static
[jira] [Commented] (OOZIE-3409) Oozie Server : Possible memory leak in EL evaluation
[
https://issues.apache.org/jira/browse/OOZIE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733459#comment-16733459
]
Praneeth commented on OOZIE-3409:
-
Thanks [~asalamon74]. {{tomcat-jasper}} seems to be a clean approach.
> Oozie Server : Possible memory leak in EL evaluation
>
>
> Key: OOZIE-3409
> URL: https://issues.apache.org/jira/browse/OOZIE-3409
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Praneeth
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3409-01.patch
>
>
> I have noticed that there is a potential memory leak in the Oozie server due
> to one of its dependencies - {{commons-el:commons-el:1.0}}. Our oozie servers
> have been running out of memory occasionally. It happens more often when we
> have more activity. Upon analysis of the heap dump, it seems that the culprit
> here is the {{org.apache.commons.el.ExpressionEvaluatorImpl}} from
> {{commons-el}} which is used in the Oozie server through
> {{org.apache.oozie.util.ELEvaluator}} [1] . {{ExpressionEvaluatorImpl}} uses
> a static {{Map}} [2] to cache the EL evaluations. This map never gets
> cleared, eventually taking up all the memory. So, in an environment with
> heavy usage and long running oozie servers, this would lead to an OOM. The
> configurations for workflow actions we have are possibly larger than normal
> which is worsening the situation as the Map has to maintain larger strings.
> #
> [https://github.com/apache/oozie/blob/release-5.1.0/core/src/main/java/org/apache/oozie/util/ELEvaluator.java#L143]
> #
> [http://svn.apache.org/viewvc/commons/dormant/el/trunk/src/java/org/apache/commons/el/ExpressionEvaluatorImpl.java?view=markup#l94]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3409) Oozie Server : Possible memory leak in EL evaluation
[
https://issues.apache.org/jira/browse/OOZIE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16733314#comment-16733314
]
Hadoop QA commented on OOZIE-3409:
--
PreCommit-OOZIE-Build started
> Oozie Server : Possible memory leak in EL evaluation
>
>
> Key: OOZIE-3409
> URL: https://issues.apache.org/jira/browse/OOZIE-3409
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Praneeth
>Assignee: Andras Salamon
>Priority: Major
> Attachments: OOZIE-3409-01.patch
>
>
> I have noticed that there is a potential memory leak in the Oozie server due
> to one of its dependencies - {{commons-el:commons-el:1.0}}. Our oozie servers
> have been running out of memory occasionally. It happens more often when we
> have more activity. Upon analysis of the heap dump, it seems that the culprit
> here is the {{org.apache.commons.el.ExpressionEvaluatorImpl}} from
> {{commons-el}} which is used in the Oozie server through
> {{org.apache.oozie.util.ELEvaluator}} [1] . {{ExpressionEvaluatorImpl}} uses
> a static {{Map}} [2] to cache the EL evaluations. This map never gets
> cleared, eventually taking up all the memory. So, in an environment with
> heavy usage and long running oozie servers, this would lead to an OOM. The
> configurations for workflow actions we have are possibly larger than normal
> which is worsening the situation as the Map has to maintain larger strings.
> #
> [https://github.com/apache/oozie/blob/release-5.1.0/core/src/main/java/org/apache/oozie/util/ELEvaluator.java#L143]
> #
> [http://svn.apache.org/viewvc/commons/dormant/el/trunk/src/java/org/apache/commons/el/ExpressionEvaluatorImpl.java?view=markup#l94]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3409) Oozie Server : Possible memory leak in EL evaluation
[
https://issues.apache.org/jira/browse/OOZIE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16732856#comment-16732856
]
Andras Piros commented on OOZIE-3409:
-
Thanks [~asalamon74] for the investigation. Can you make a try w/
{{org.apache.tomcat:jasper-el}} latest version running w/ Oozie's Jetty 9?
> Oozie Server : Possible memory leak in EL evaluation
>
>
> Key: OOZIE-3409
> URL: https://issues.apache.org/jira/browse/OOZIE-3409
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Praneeth
>Assignee: Andras Salamon
>Priority: Major
>
> I have noticed that there is a potential memory leak in the Oozie server due
> to one of its dependencies - {{commons-el:commons-el:1.0}}. Our oozie servers
> have been running out of memory occasionally. It happens more often when we
> have more activity. Upon analysis of the heap dump, it seems that the culprit
> here is the {{org.apache.commons.el.ExpressionEvaluatorImpl}} from
> {{commons-el}} which is used in the Oozie server through
> {{org.apache.oozie.util.ELEvaluator}} [1] . {{ExpressionEvaluatorImpl}} uses
> a static {{Map}} [2] to cache the EL evaluations. This map never gets
> cleared, eventually taking up all the memory. So, in an environment with
> heavy usage and long running oozie servers, this would lead to an OOM. The
> configurations for workflow actions we have are possibly larger than normal
> which is worsening the situation as the Map has to maintain larger strings.
> #
> [https://github.com/apache/oozie/blob/release-5.1.0/core/src/main/java/org/apache/oozie/util/ELEvaluator.java#L143]
> #
> [http://svn.apache.org/viewvc/commons/dormant/el/trunk/src/java/org/apache/commons/el/ExpressionEvaluatorImpl.java?view=markup#l94]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3409) Oozie Server : Possible memory leak in EL evaluation
[
https://issues.apache.org/jira/browse/OOZIE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16732850#comment-16732850
]
Andras Salamon commented on OOZIE-3409:
---
In the EL-1 bugreport they suggest to use tomcat's jasper el instead of commons
el: [https://markmail.org/message/nm4c3zwzmxhuhplu]
> Oozie Server : Possible memory leak in EL evaluation
>
>
> Key: OOZIE-3409
> URL: https://issues.apache.org/jira/browse/OOZIE-3409
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Praneeth
>Assignee: Andras Salamon
>Priority: Major
>
> I have noticed that there is a potential memory leak in the Oozie server due
> to one of its dependencies - {{commons-el:commons-el:1.0}}. Our oozie servers
> have been running out of memory occasionally. It happens more often when we
> have more activity. Upon analysis of the heap dump, it seems that the culprit
> here is the {{org.apache.commons.el.ExpressionEvaluatorImpl}} from
> {{commons-el}} which is used in the Oozie server through
> {{org.apache.oozie.util.ELEvaluator}} [1] . {{ExpressionEvaluatorImpl}} uses
> a static {{Map}} [2] to cache the EL evaluations. This map never gets
> cleared, eventually taking up all the memory. So, in an environment with
> heavy usage and long running oozie servers, this would lead to an OOM. The
> configurations for workflow actions we have are possibly larger than normal
> which is worsening the situation as the Map has to maintain larger strings.
> #
> [https://github.com/apache/oozie/blob/release-5.1.0/core/src/main/java/org/apache/oozie/util/ELEvaluator.java#L143]
> #
> [http://svn.apache.org/viewvc/commons/dormant/el/trunk/src/java/org/apache/commons/el/ExpressionEvaluatorImpl.java?view=markup#l94]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3409) Oozie Server : Possible memory leak in EL evaluation
[
https://issues.apache.org/jira/browse/OOZIE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16732827#comment-16732827
]
Andras Salamon commented on OOZIE-3409:
---
The commons.el bug reported in 2005.
> Oozie Server : Possible memory leak in EL evaluation
>
>
> Key: OOZIE-3409
> URL: https://issues.apache.org/jira/browse/OOZIE-3409
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Praneeth
>Assignee: Andras Salamon
>Priority: Major
>
> I have noticed that there is a potential memory leak in the Oozie server due
> to one of its dependencies - {{commons-el:commons-el:1.0}}. Our oozie servers
> have been running out of memory occasionally. It happens more often when we
> have more activity. Upon analysis of the heap dump, it seems that the culprit
> here is the {{org.apache.commons.el.ExpressionEvaluatorImpl}} from
> {{commons-el}} which is used in the Oozie server through
> {{org.apache.oozie.util.ELEvaluator}} [1] . {{ExpressionEvaluatorImpl}} uses
> a static {{Map}} [2] to cache the EL evaluations. This map never gets
> cleared, eventually taking up all the memory. So, in an environment with
> heavy usage and long running oozie servers, this would lead to an OOM. The
> configurations for workflow actions we have are possibly larger than normal
> which is worsening the situation as the Map has to maintain larger strings.
> #
> [https://github.com/apache/oozie/blob/release-5.1.0/core/src/main/java/org/apache/oozie/util/ELEvaluator.java#L143]
> #
> [http://svn.apache.org/viewvc/commons/dormant/el/trunk/src/java/org/apache/commons/el/ExpressionEvaluatorImpl.java?view=markup#l94]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3409) Oozie Server : Possible memory leak in EL evaluation
[
https://issues.apache.org/jira/browse/OOZIE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16732123#comment-16732123
]
Andras Salamon commented on OOZIE-3409:
---
We could create a hack solution like:
# Extending {{ExpressionEvaluatorImpl}} and not using the cache
# Counting EL expression evaluations and drop {{ExpressionEvaluatorImpl}} and
start over after a while
The best solution is to replace the obsolete {{commons.el}}. I've checked our
code, it seems to be this is the only place we use {{commons.el}} in Oozie.
There are seveal EL evaluators listed here:
[https://stackoverflow.com/a/17051440/21348] We might check JEXL (
[http://commons.apache.org/proper/commons-jexl/] ) since we already use that
in other parts of Oozie. Of course we have to be sure it does not have the same
bug.
> Oozie Server : Possible memory leak in EL evaluation
>
>
> Key: OOZIE-3409
> URL: https://issues.apache.org/jira/browse/OOZIE-3409
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Praneeth
>Assignee: Andras Salamon
>Priority: Major
>
> I have noticed that there is a potential memory leak in the Oozie server due
> to one of its dependencies - {{commons-el:commons-el:1.0}}. Our oozie servers
> have been running out of memory occasionally. It happens more often when we
> have more activity. Upon analysis of the heap dump, it seems that the culprit
> here is the {{org.apache.commons.el.ExpressionEvaluatorImpl}} from
> {{commons-el}} which is used in the Oozie server through
> {{org.apache.oozie.util.ELEvaluator}} [1] . {{ExpressionEvaluatorImpl}} uses
> a static {{Map}} [2] to cache the EL evaluations. This map never gets
> cleared, eventually taking up all the memory. So, in an environment with
> heavy usage and long running oozie servers, this would lead to an OOM. The
> configurations for workflow actions we have are possibly larger than normal
> which is worsening the situation as the Map has to maintain larger strings.
> #
> [https://github.com/apache/oozie/blob/release-5.1.0/core/src/main/java/org/apache/oozie/util/ELEvaluator.java#L143]
> #
> [http://svn.apache.org/viewvc/commons/dormant/el/trunk/src/java/org/apache/commons/el/ExpressionEvaluatorImpl.java?view=markup#l94]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3409) Oozie Server : Possible memory leak in EL evaluation
[
https://issues.apache.org/jira/browse/OOZIE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726019#comment-16726019
]
Praneeth commented on OOZIE-3409:
-
I have just noticed that {{org.apache.commons.el.ExpressionEvaluatorImpl}} is
used only in one place in {{oozie-core}} . As a quick dirty fix, we could
extend the class and override the evaluation method to skip writing to the map
when the flag is disabled.
> Oozie Server : Possible memory leak in EL evaluation
>
>
> Key: OOZIE-3409
> URL: https://issues.apache.org/jira/browse/OOZIE-3409
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Praneeth
>Priority: Major
>
> I have noticed that there is a potential memory leak in the Oozie server due
> to one of its dependencies - {{commons-el:commons-el:1.0}}. Our oozie servers
> have been running out of memory occasionally. It happens more often when we
> have more activity. Upon analysis of the heap dump, it seems that the culprit
> here is the {{org.apache.commons.el.ExpressionEvaluatorImpl}} from
> {{commons-el}} which is used in the Oozie server through
> {{org.apache.oozie.util.ELEvaluator}} [1] . {{ExpressionEvaluatorImpl}} uses
> a static {{Map}} [2] to cache the EL evaluations. This map never gets
> cleared, eventually taking up all the memory. So, in an environment with
> heavy usage and long running oozie servers, this would lead to an OOM. The
> configurations for workflow actions we have are possibly larger than normal
> which is worsening the situation as the Map has to maintain larger strings.
> #
> [https://github.com/apache/oozie/blob/release-5.1.0/core/src/main/java/org/apache/oozie/util/ELEvaluator.java#L143]
> #
> [http://svn.apache.org/viewvc/commons/dormant/el/trunk/src/java/org/apache/commons/el/ExpressionEvaluatorImpl.java?view=markup#l94]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3409) Oozie Server : Possible memory leak in EL evaluation
[
https://issues.apache.org/jira/browse/OOZIE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16726008#comment-16726008
]
Praneeth commented on OOZIE-3409:
-
Actually, I don't think it is just the actions that have EL expressions that
sit in the cache, I think the parsing is taking place for every action (or some
part of workflow config), whether or not it has an EL (call to parsing takes
place irrespectively) [1]. If it doesn't have an EL, the cache would store the
same content as both key and value. I was able to verify this by looking at the
contents of the cache.
This is only to point out that it is not just for EL based content. I do not
have a good proposal for a solution given the dependency is dormant. What is
the general practice for fixes to dormant projects ? Is it to try to find an
alternative and if none is found, try to revive the project ?
I agree that it is probably a problem to few deployments like ours which are
long running and relatively workflows, but this crashes our instances once in
2-3 months.
# I was checking the usage of
[https://github.com/apache/oozie/blob/master/core/src/main/java/org/apache/oozie/util/ELEvaluator.java#L201.]
> Oozie Server : Possible memory leak in EL evaluation
>
>
> Key: OOZIE-3409
> URL: https://issues.apache.org/jira/browse/OOZIE-3409
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Praneeth
>Priority: Major
>
> I have noticed that there is a potential memory leak in the Oozie server due
> to one of its dependencies - {{commons-el:commons-el:1.0}}. Our oozie servers
> have been running out of memory occasionally. It happens more often when we
> have more activity. Upon analysis of the heap dump, it seems that the culprit
> here is the {{org.apache.commons.el.ExpressionEvaluatorImpl}} from
> {{commons-el}} which is used in the Oozie server through
> {{org.apache.oozie.util.ELEvaluator}} [1] . {{ExpressionEvaluatorImpl}} uses
> a static {{Map}} [2] to cache the EL evaluations. This map never gets
> cleared, eventually taking up all the memory. So, in an environment with
> heavy usage and long running oozie servers, this would lead to an OOM. The
> configurations for workflow actions we have are possibly larger than normal
> which is worsening the situation as the Map has to maintain larger strings.
> #
> [https://github.com/apache/oozie/blob/release-5.1.0/core/src/main/java/org/apache/oozie/util/ELEvaluator.java#L143]
> #
> [http://svn.apache.org/viewvc/commons/dormant/el/trunk/src/java/org/apache/commons/el/ExpressionEvaluatorImpl.java?view=markup#l94]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
[jira] [Commented] (OOZIE-3409) Oozie Server : Possible memory leak in EL evaluation
[
https://issues.apache.org/jira/browse/OOZIE-3409?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16725879#comment-16725879
]
Andras Salamon commented on OOZIE-3409:
---
Yes, {{org.apache.commons.el.ExpressionEvaluatorImpl}} has static maps which
never gets clear. There is a constructor in the class which takes a boolean
{{pBypassCache}} argument, but it only bypass the reading of the cache, not the
writing:
[http://svn.apache.org/viewvc/commons/dormant/el/trunk/src/java/org/apache/commons/el/ExpressionEvaluatorImpl.java?view=markup#l256]
So theoretically it's true that adding more and more different EL expression
string will result an OOM. I But it will take probably a really large number to
reach that point.
Fixing that would not be easy, since commons-el is alrady dormant (latest
release was in 2003).
> Oozie Server : Possible memory leak in EL evaluation
>
>
> Key: OOZIE-3409
> URL: https://issues.apache.org/jira/browse/OOZIE-3409
> Project: Oozie
> Issue Type: Bug
>Affects Versions: 5.1.0
>Reporter: Praneeth
>Priority: Major
>
> I have noticed that there is a potential memory leak in the Oozie server due
> to one of its dependencies - {{commons-el:commons-el:1.0}}. Our oozie servers
> have been running out of memory occasionally. It happens more often when we
> have more activity. Upon analysis of the heap dump, it seems that the culprit
> here is the {{org.apache.commons.el.ExpressionEvaluatorImpl}} from
> {{commons-el}} which is used in the Oozie server through
> {{org.apache.oozie.util.ELEvaluator}} [1] . {{ExpressionEvaluatorImpl}} uses
> a static {{Map}} [2] to cache the EL evaluations. This map never gets
> cleared, eventually taking up all the memory. So, in an environment with
> heavy usage and long running oozie servers, this would lead to an OOM. The
> configurations for workflow actions we have are possibly larger than normal
> which is worsening the situation as the Map has to maintain larger strings.
> #
> [https://github.com/apache/oozie/blob/release-5.1.0/core/src/main/java/org/apache/oozie/util/ELEvaluator.java#L143]
> #
> [http://svn.apache.org/viewvc/commons/dormant/el/trunk/src/java/org/apache/commons/el/ExpressionEvaluatorImpl.java?view=markup#l94]
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
