Issue #837 has been updated by Clément OUDOT. Status changed from New to Assigned Assigned to changed from Clément OUDOT to Raphaël Ouazana
Thanks for the patch. ---------------------------------------- Bug #837: scriptInclude causes LSC to run out of filedescriptors http://tools.lsc-project.org/issues/837 Author: Harald Strack Status: Assigned Priority: Normal Assigned to: Raphaël Ouazana Category: Core Target version: 2.2 Problem in version: 2.1, 2.2 h2. Problem * When running LSC highly threaded and with > 200000 Accounts, it runs out of filedescriptors <pre> Jul 23 17:54:54 - ERROR - java.io.FileNotFoundException: /etc/lscprovision/lib.js (Too many open files) Jul 23 17:54:54 - DEBUG - java.io.FileNotFoundException: /etc/lscprovision/lib.js (Too many open files) java.io.FileNotFoundException: /etc/lscprovision/lib.js (Too many open files) at java.io.FileInputStream.open(Native Method) ~[na:1.7.0_71] at java.io.FileInputStream.<init>(FileInputStream.java:146) ~[na:1.7.0_71] at java.io.FileReader.<init>(FileReader.java:72) ~[na:1.7.0_71] at org.lsc.utils.JScriptEvaluator.instanceEval(JScriptEvaluator.java:299) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at org.lsc.utils.JScriptEvaluator.evalToString(JScriptEvaluator.java:102) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at org.lsc.utils.ScriptingEvaluator.evalToString(ScriptingEvaluator.java:129) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at org.lsc.beans.BeanComparator.cloneSrcBean(BeanComparator.java:531) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at org.lsc.beans.BeanComparator.calculateModifications(BeanComparator.java:158) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:808) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:742) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71] Jul 23 17:54:54 - ERROR - Error while synchronizing ID {uid=00872908X20}: java.lang.RuntimeException: No DN set! Read it from the source or set lsc.tasks.NAME.d n Jul 23 17:54:54 - DEBUG - java.lang.RuntimeException: No DN set! Read it from the source or set lsc.tasks.NAME.dn java.lang.RuntimeException: No DN set! Read it from the source or set lsc.tasks.NAME.dn at org.lsc.beans.BeanComparator.getDstDN(BeanComparator.java:206) ~[lsc-core-2.2trunk-SSYSTEMS.jar:na] at org.lsc.beans.BeanComparator.calculateModifications(BeanComparator.java:172) ~[lsc-core-2.2trunk-SSYSTEMS.jar:na] at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:808) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:742) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71] Jul 23 17:54:54 - ERROR - java.io.FileNotFoundException: /etc/lscprovision/lib.js (Too many open files) Jul 23 17:54:54 - DEBUG - java.io.FileNotFoundException: /etc/lscprovision/lib.js (Too many open files) java.io.FileNotFoundException: /etc/lscprovision/lib.js (Too many open files) at java.io.FileInputStream.open(Native Method) ~[na:1.7.0_71] at java.io.FileInputStream.<init>(FileInputStream.java:146) ~[na:1.7.0_71] at java.io.FileReader.<init>(FileReader.java:72) ~[na:1.7.0_71] at org.lsc.utils.JScriptEvaluator.instanceEval(JScriptEvaluator.java:299) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at org.lsc.utils.JScriptEvaluator.evalToString(JScriptEvaluator.java:102) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at org.lsc.utils.ScriptingEvaluator.evalToString(ScriptingEvaluator.java:129) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at org.lsc.beans.BeanComparator.cloneSrcBean(BeanComparator.java:531) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at org.lsc.beans.BeanComparator.calculateModifications(BeanComparator.java:158) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:808) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:742) [lsc-core-2.2trunk-SSYSTEMS.jar:na] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) [na:1.7.0_71] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) [na:1.7.0_71] at java.lang.Thread.run(Thread.java:745) [na:1.7.0_71] </pre> * LSC closes the open file handles after a while, but too slowly to avoid running out of FDs <pre> java 7070 root 1614r REG 253,0 3655 526044 /etc/lscprovision/lib.js java 7070 root 1615r REG 253,0 3655 526044 /etc/lscprovision/lib.js java 7070 root 1616r REG 253,0 3655 526044 /etc/lscprovision/lib.js java 7070 root 1617r REG 253,0 3655 526044 /etc/lscprovision/lib.js java 7070 root 1618r REG 253,0 3655 526044 /etc/lscprovision/lib.js java 7070 root 1619r REG 253,0 3655 526044 /etc/lscprovision/lib.js java 7070 root 1620r REG 253,0 3655 526044 /etc/lscprovision/lib.js java 7070 root 1686r REG 253,0 3655 526044 /etc/lscprovision/lib.js [root@springfield ~]# lsof -p 7070 | wc -l 1635 [root@springfield ~]# lsof -p 7070 | wc -l 1666 [root@springfield ~]# lsof -p 7070 | wc -l 1709 [root@springfield ~]# lsof -p 7070 | wc -l 1678 [root@springfield ~]# lsof -p 7070 | wc -l 1766 [root@springfield ~]# lsof -p 7070 | wc -l 1838 [root@springfield ~]# lsof -p 7070 | wc -l 2206 [root@springfield ~]# lsof -p 7070 | wc -l 2254 </pre> h2. Solution * We have to close the @FileReader@ that reads the scripting libs directly after evaluation * According to http://www.java2s.com/Code/JavaAPI/javax.script/ScriptEngineevalReaderreader.htm, the @FileReader@ has to be closed outside the Scritping Engine Patch proposal attached -- You have received this notification because you have either subscribed to it, or are involved in it. To change your notification preferences, please click here: http://tools.lsc-project.org/my/account
_______________________________________________________________ Ldap Synchronization Connector (LSC) - http://lsc-project.org lsc-dev mailing list [email protected] http://lists.lsc-project.org/listinfo/lsc-dev
