Issue #862 has been updated by Harald Strack.
Hi,
we experience the same or similar problem, also with more than 10000 users and
8 Threads
<pre>
ERROR - Error while synchronizing ID {...}: java.lang.NullPointerException
</pre>
I added some logging
<pre>
diff --git a/src/main/java/org/lsc/AbstractSynchronize.java
b/src/main/java/org/lsc/AbstractSynchronize.java
index 854cafc..18fd381 100644
--- a/src/main/java/org/lsc/AbstractSynchronize.java
+++ b/src/main/java/org/lsc/AbstractSynchronize.java
@@ -440,6 +440,7 @@ public abstract class AbstractSynchronize {
LOGGER.error("Error while synchronizing ID {}: {}", (lm != null
? lm.getMainIdentifier() : data), except.toString());
LOGGER.debug(except.toString(), except);
+ except.printStackTrace();
if (lm != null) {
// TODO Fix LdifLogger to avoid this
</pre>
And it seems that the LDAP connection gets dropped somehow:
<pre>
org.lsc.AbstractSynchronize Jun 23 02:08:02 - ERROR - Error while synchronizing
ID {uid=peter.lustig}: java.lang.NullPointerException
org.lsc.AbstractSynchronize Jun 23 02:08:02 - DEBUG -
java.lang.NullPointerException
java.lang.NullPointerException: null
at org.lsc.jndi.JndiServices.doGetEntry(JndiServices.java:568)
~[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.jndi.JndiServices.getEntry(JndiServices.java:529)
~[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.jndi.JndiServices.getEntry(JndiServices.java:507)
~[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
org.lsc.jndi.AbstractSimpleJndiService.get(AbstractSimpleJndiService.java:263)
~[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
org.lsc.jndi.SimpleJndiDstService.getBean(SimpleJndiDstService.java:135)
~[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
org.lsc.plugins.connectors.executable.ExecutableLdapDestinationService.getBean(ExecutableLdapDestinationService.java:156)
~[executable-1.0lsc2.2.jar:na]
at org.lsc.AbstractSynchronize.getBean(AbstractSynchronize.java:549)
~[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:776)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:742)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_101]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_101]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_101]
org.lsc.AbstractSynchronize Jun 23 02:08:02 - ERROR - Error while synchronizing
ID {uid=peter.lustig2}: java.lang.NullPointerException
org.lsc.AbstractSynchronize Jun 23 02:08:02 - DEBUG -
java.lang.NullPointerException
java.lang.NullPointerException: null
at org.lsc.jndi.JndiServices.doGetEntry(JndiServices.java:568)
~[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.jndi.JndiServices.getEntry(JndiServices.java:529)
~[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.jndi.JndiServices.getEntry(JndiServices.java:507)
~[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
org.lsc.jndi.AbstractSimpleJndiService.get(AbstractSimpleJndiService.java:263)
~[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
org.lsc.jndi.SimpleJndiDstService.getBean(SimpleJndiDstService.java:135)
~[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
org.lsc.plugins.connectors.executable.ExecutableLdapDestinationService.getBean(ExecutableLdapDestinationService.java:156)
~[executable-1.0lsc2.2.jar:na]
at org.lsc.AbstractSynchronize.getBean(AbstractSynchronize.java:549)
~[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:776)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at org.lsc.SynchronizeTask.run(AbstractSynchronize.java:742)
[lsc-core-2.2trunk-SSYSTEMS.jar:na]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[na:1.7.0_101]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[na:1.7.0_101]
at java.lang.Thread.run(Thread.java:745) [na:1.7.0_101]
</pre>
Seems that @at org.lsc.jndi.JndiServices.doGetEntry(JndiServices.java:568@
only @ctx@ - the LDAP context may be NULL
<pre>
ne = ctx.search(rewrittenBase, searchFilter, sc);
</pre>
Maybe it's a LDAP Server limit (happens more or less after 2 hours) but it may
also be related to this issue. Is there a way use your patch with appropriate
parameters (what values?) to fix this?
br
Harald
----------------------------------------
Bug #862: java.lang.NullPointerException when all entries exceed 10,000 on
multi threaded LSC run
http://tools.lsc-project.org/issues/862
Author: Frédéric POISSON
Status: Assigned
Priority: Normal
Assigned to: Raphaël Ouazana
Category: Core
Target version: 2.2
Problem in version:
Hello,
I've detected on some tasks with more than 10,000 ids for a single task, with a
multi threaded LSC run, that i have some errors like this :
<pre>
ERROR - Error while synchronizing ID {...}: java.lang.NullPointerException
</pre>
I understand that AbstractSynchronize.java run only 10,000 entries at a time,
and wait for the threadPool to finish all thread executions with a timeout of
900 seconds and then continue
with the next 10,000 and so on.
But at first initialization of the destination (or when there are many changes
or heavy load on the servers...) it could take more than 11,11 ids per seconds
to fully finish the thread executions. So the threadPool could stop some
threads before their execution. That's why when i have the above ERROR i have
also less ids from AbstractSynchronize.java counter than from real number of
entries return by the get all filter of the source, and the above ERROR are
also not count :
<pre>
INFO - All entries: 23042, to modify entries: 0, successfully modified
entries: 0, errors: 0
</pre>
That's why the 10,000 and the 900 seconds could be adapted with two JVM system
properties (see patch).
--
You have received this notification because you have either subscribed to it,
or are involved in it.
To change your notification preferences, please click here:
http://tools.lsc-project.org/my/account
_______________________________________________________________
Ldap Synchronization Connector (LSC) - http://lsc-project.org
lsc-dev mailing list
[email protected]
http://lists.lsc-project.org/listinfo/lsc-dev
