Hi Dinesh, Look at the LSC command line options.
LSC is successfully syncing directories with millions of entries but probably notre with the same configuration. Can you generate a threaddump (jstack -l) to help US to identify where the threads are leaking ? Thanks, Le 19 nov. 2014 17:02, "Dinesh Babu" <[email protected]> a écrit : > Thanks Sebastien, > > > > Quartz kick starts our LSC sync process. Did not understand where I can > set this command line argument "-t"? > > > > Also has any of LSC users ever brought to your attention about socket > connection error while syncing from an LDAP with more than 80,000 users? We > are consistently getting this issue when syncing 80,000+ users. > > > > Regards, > > Dinesh Babu. > > Pitney Bowes Software > > 6 Hercules Way, Leavesden Park, Watford, Herts WD25 7GS > > Ph: +441923 279123 www.pb.com/software > > [email protected] > > Every connection is a new opportunity™ > > > > [image: cid:[email protected]] > > > > > > Please consider the environment before printing or forwarding this email. > If you do print this email, please recycle the paper. > > > > This email message may contain confidential, proprietary and/or privileged > information. It is intended only for the use of the intended recipient(s). > If you have received it in error, please immediately advise the sender by > reply email and then delete this email message. Any disclosure, copying, > distribution or use of the information contained in this email message to > or by anyone other than the intended recipient is strictly prohibited. > > > > > > *From:* Sébastien Bahloul [mailto:[email protected]] > *Sent:* 19 November 2014 15:48 > *To:* Dinesh Babu > *Cc:* [email protected]; [email protected] > *Subject:* Re: [lsc-users] Socket error while syncing from LDAP > > > > Hi Dinesh, > > > > The Thread pool size is an internal maximum size. The effective maximum > size is provided by your -t command line parameter value > > > > I kindly suggest to read the following thread to understand the current > implementation : > http://stackoverflow.com/questions/1800317/impossible-to-make-a-cached-thread-pool-with-a-size-limit/1800583#1800583 > > > > Regards, > > > Sebastien BAHLOUL > IAM / Security specialist > Ldap Synchronization Connector : http://lsc-project.org > Blog : http://sbahloul.wordpress.com/ > > > > 2014-11-19 16:21 GMT+01:00 Dinesh Babu <[email protected]>: > > In LSC code, see below, the thread pool size is set integer.MAX_VALUE > which actually is billions. When we have so much threads while syncing > large amount of records (we are trying to sync with 120,000+ records), that > will cause socket error, won't it? Is there any particular reason you did > not want to limit the tread pool size? > > > > protected SynchronizeThreadPoolExecutor(int threads) { > > super(threads, Integer.MAX_VALUE, > keepAliveTime, TimeUnit.SECONDS, new > ArrayBlockingQueue<Runnable>(Configuration.MAX_CONCURRENT_SYNCHRONIZED)); > > queue = getQueue(); > > } > > > > > > > > Regards, > > Dinesh Babu. > > Pitney Bowes Software > > 6 Hercules Way, Leavesden Park, Watford, Herts WD25 7GS > > Ph: +441923 279123 www.pb.com/software > > [email protected] > > Every connection is a new opportunity™ > > > > [image: cid:[email protected]] > > > > > > Please consider the environment before printing or forwarding this email. > If you do print this email, please recycle the paper. > > > > This email message may contain confidential, proprietary and/or privileged > information. It is intended only for the use of the intended recipient(s). > If you have received it in error, please immediately advise the sender by > reply email and then delete this email message. Any disclosure, copying, > distribution or use of the information contained in this email message to > or by anyone other than the intended recipient is strictly prohibited. > > > > > > *From:* Dinesh Babu > *Sent:* 16 November 2014 23:08 > *To:* [email protected]; [email protected] > *Subject:* RE: Socket error while syncing from LDAP > > > > In LSC, is it possible to introduce a pause after fetching certain number > of records from LDAP and before proceeding any further? I think I am > getting socket error because the LDAP socket queue has not got enough space > to accommodate new requests from LSC when LSC requests data from LDAP very > frequently for prolonged period of time. > > > > Regards, > > Dinesh Babu. > > Pitney Bowes Software > > 6 Hercules Way, Leavesden Park, Watford, Herts WD25 7GS > > Ph: +441923 279123 www.pb.com/software > > [email protected] > > Every connection is a new opportunity™ > > > > [image: cid:[email protected]] > > > > > > Please consider the environment before printing or forwarding this email. > If you do print this email, please recycle the paper. > > > > This email message may contain confidential, proprietary and/or privileged > information. It is intended only for the use of the intended recipient(s). > If you have received it in error, please immediately advise the sender by > reply email and then delete this email message. Any disclosure, copying, > distribution or use of the information contained in this email message to > or by anyone other than the intended recipient is strictly prohibited. > > > > > > *From:* Dinesh Babu > *Sent:* 12 November 2014 18:10 > *To:* [email protected]; [email protected] > *Subject:* Socket error while syncing from LDAP > > > > Hi, > > > > Has anyone seen this type of error during LSC syncing from LDAP? When LSC > bombards LDAP with queries does the LDAP socket gets closed or becomes > unavailable? > > > > 08:23:45,525 ERROR [org.lsc.jndi.JndiServices] (all-users-380033) Error > while lo > > oking for (sAMAccountName=$DK5K00-SEVROOJC1QTD) in > OU=testOrgUnitSync,DC=eo,DC=l > > ocal: javax.naming.CommunicationException: *Software caused connection > abort: soc* > > *ket write error [Root exception is java.net.SocketException: Software > caused con* > > *nection abort: socket write error]; remaining name* '' > > > > 08:23:45,527 ERROR [org.lsc.AbstractSynchronize] (all-users-383873) Error > while > > synchronizing ID {samaccountname=$VS7K00-F4ISAC5NM0KE}: > org.lsc.exception.LscSer > > viceException: javax.naming.NamingException: JBAS011843*: Failed > instantiate Init* > > *ialContextFactory com.sun.jndi.ldap.LdapCtxFactory from classloader > ModuleClassL* > > *oader for Module "deployment.admin-services.war:main" from Service Module > Loader* > > *[Root exception is javax.naming.CommunicationException: simple bind > failed*:* [IP address of LDAP]:389* [Root exception is > java.net.SocketException: Connection reset b > > y peer: socket write error]] > > > > > > > > Regards, > > Dinesh Babu. > > Pitney Bowes Software > > 6 Hercules Way, Leavesden Park, Watford, Herts WD25 7GS > > Ph: +441923 279123 www.pb.com/software > > [email protected] > > Every connection is a new opportunity™ > > > > [image: cid:[email protected]] > > > > > > Please consider the environment before printing or forwarding this email. > If you do print this email, please recycle the paper. > > > > This email message may contain confidential, proprietary and/or privileged > information. It is intended only for the use of the intended recipient(s). > If you have received it in error, please immediately advise the sender by > reply email and then delete this email message. Any disclosure, copying, > distribution or use of the information contained in this email message to > or by anyone other than the intended recipient is strictly prohibited. > > > > > > > ------------------------------ > > > > > _______________________________________________________________ > Ldap Synchronization Connector (LSC) - http://lsc-project.org > > lsc-users mailing list > [email protected] > http://lists.lsc-project.org/listinfo/lsc-users > > > > ------------------------------ > >
_______________________________________________________________ Ldap Synchronization Connector (LSC) - http://lsc-project.org lsc-dev mailing list [email protected] http://lists.lsc-project.org/listinfo/lsc-dev
