Issue #837 has been reported by Harald Strack.
----------------------------------------
Bug #837: scriptInclude causes LSC to run out of filedescriptors
http://tools.lsc-project.org/issues/837
Author: Harald Strack
Status: New
Priority: Normal
Assigned to: Clément OUDOT
Category: Core
Target version: 2.2
Problem in version: 2.1, 2.2
h2. Problem
* When running LSC highly threaded and with > 200000 Accounts, it runs out of
filedescriptors
<pre>
Jul 23 17:54:54 - ERROR - java.io.FileNotFoundException:
/etc/lscprovision/lib.js (Too many open files)
Jul 23 17:54:54 - DEBUG - java.io.FileNotFoundException:
/etc/lscprovision/lib.js (Too many open files)
java.io.FileNotFoundException: /etc/lscprovision/lib.js (Too many open files)
at java.io.FileInputStream.open(Native Method) ~[na:1.7.0_71]
at java.io.FileInputStream.<init>(FileInputStream.java:146)
~[na:1.7.0_71]
at java.io.FileReader.<init>(FileReader.java:72) ~[na:1.7.0_71]
at
org.lsc.utils.JScriptEvaluator.instanceEval(JScriptEvaluator.java:299)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
org.lsc.utils.JScriptEvaluator.evalToString(JScriptEvaluator.java:102)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
org.lsc.utils.ScriptingEvaluator.evalToString(ScriptingEvaluator.java:129)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.beans.BeanComparator.cloneSrcBean(BeanComparator.java:531)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
org.lsc.beans.BeanComparator.calculateModifications(BeanComparator.java:158)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:808)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:742)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
Jul 23 17:54:54 - ERROR - Error while synchronizing ID {uid=00872908X20}:
java.lang.RuntimeException: No DN set! Read it from the source or set
lsc.tasks.NAME.d
n
Jul 23 17:54:54 - DEBUG - java.lang.RuntimeException: No DN set! Read it from
the source or set lsc.tasks.NAME.dn
java.lang.RuntimeException: No DN set! Read it from the source or set
lsc.tasks.NAME.dn
at org.lsc.beans.BeanComparator.getDstDN(BeanComparator.java:206)
~[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
org.lsc.beans.BeanComparator.calculateModifications(BeanComparator.java:172)
~[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:808)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:742)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
Jul 23 17:54:54 - ERROR - java.io.FileNotFoundException:
/etc/lscprovision/lib.js (Too many open files)
Jul 23 17:54:54 - DEBUG - java.io.FileNotFoundException:
/etc/lscprovision/lib.js (Too many open files)
java.io.FileNotFoundException: /etc/lscprovision/lib.js (Too many open files)
at java.io.FileInputStream.open(Native Method) ~[na:1.7.0_71]
at java.io.FileInputStream.<init>(FileInputStream.java:146)
~[na:1.7.0_71]
at java.io.FileReader.<init>(FileReader.java:72) ~[na:1.7.0_71]
at
org.lsc.utils.JScriptEvaluator.instanceEval(JScriptEvaluator.java:299)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
org.lsc.utils.JScriptEvaluator.evalToString(JScriptEvaluator.java:102)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
org.lsc.utils.ScriptingEvaluator.evalToString(ScriptingEvaluator.java:129)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.beans.BeanComparator.cloneSrcBean(BeanComparator.java:531)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
org.lsc.beans.BeanComparator.calculateModifications(BeanComparator.java:158)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:808)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:742)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71]
</pre>
* LSC closes the open file handles after a while, but too slowly to avoid
running out of FDs
<pre>
java 7070 root 1614r REG 253,0 3655 526044
/etc/lscprovision/lib.js
java 7070 root 1615r REG 253,0 3655 526044
/etc/lscprovision/lib.js
java 7070 root 1616r REG 253,0 3655 526044
/etc/lscprovision/lib.js
java 7070 root 1617r REG 253,0 3655 526044
/etc/lscprovision/lib.js
java 7070 root 1618r REG 253,0 3655 526044
/etc/lscprovision/lib.js
java 7070 root 1619r REG 253,0 3655 526044
/etc/lscprovision/lib.js
java 7070 root 1620r REG 253,0 3655 526044
/etc/lscprovision/lib.js
java 7070 root 1686r REG 253,0 3655 526044
/etc/lscprovision/lib.js
[root@springfield ~]# lsof -p 7070 | wc -l
1635
[root@springfield ~]# lsof -p 7070 | wc -l
1666
[root@springfield ~]# lsof -p 7070 | wc -l
1709
[root@springfield ~]# lsof -p 7070 | wc -l
1678
[root@springfield ~]# lsof -p 7070 | wc -l
1766
[root@springfield ~]# lsof -p 7070 | wc -l
1838
[root@springfield ~]# lsof -p 7070 | wc -l
2206
[root@springfield ~]# lsof -p 7070 | wc -l
2254
</pre>
h2. Solution
* We have to close the @FileReader@ that reads the scripting libs directly
after evaluation
* According to
http://www.java2s.com/Code/JavaAPI/javax.script/ScriptEngineevalReaderreader.htm,
the @FileReader@ has to be closed outside the Scritping Engine
Patch proposal attached
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://tools.lsc-project.org/my/account
_______________________________________________________________
Ldap Synchronization Connector (LSC) - http://lsc-project.org
lsc-dev mailing list
[email protected]
http://lists.lsc-project.org/listinfo/lsc-dev
