LdapNetworkConnection not thread-safe ?
Hi ApacheDS developers, I'm working on an open source project ( https://github.com/soluvas/ldap-tools ) which uses LdapNetworkConnection using shared v1.0.0-M12. Several threads are running in parallel (using Akka), all using the same LdapNetworkConnection to delete entries. However in some cases it locks up (deadlock? race condition?) and the last logs I get is : ... 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-14] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setsuna_hinagiku,ou=users,dc=berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-dispatcher-24] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=rumah_amal_salman_itb,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-18] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setyo_rini,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-1] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=pipit_nugroho,ou=users,dc=berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-dispatcher-15] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=yuliana_riris_basaria,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-16] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setia_budi,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 I also experience similar issues doing concurrent add()s. Is LdapNetworkConnection meant to be thread-safe? Or should I just use separate LdapConnection for each thread? -- Hendy Irawan - on Twitter http://twitter.com/hendybippo - on LinkedInhttp://id.linkedin.com/in/hendyirawan Web Developer | Bippo Indonesia http://www.bippo.co.id/ | Akselerator Bisnis | Bandung
Re: LdapNetworkConnection not thread-safe ?
it is not thread safe a bit of history: earlier we made it thread safe, but later dropped the thread safety features it still supports the async feature though On Wed, Jun 27, 2012 at 8:53 PM, Hendy Irawan he...@soluvas.com wrote: Hi ApacheDS developers, I'm working on an open source project ( https://github.com/soluvas/ldap-tools ) which uses LdapNetworkConnection using shared v1.0.0-M12. Several threads are running in parallel (using Akka), all using the same LdapNetworkConnection to delete entries. However in some cases it locks up (deadlock? race condition?) and the last logs I get is : ... 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-14] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setsuna_hinagiku,ou=users,dc=berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-dispatcher-24] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=rumah_amal_salman_itb,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-18] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setyo_rini,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-1] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=pipit_nugroho,ou=users,dc=berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-dispatcher-15] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=yuliana_riris_basaria,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-16] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setia_budi,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 I also experience similar issues doing concurrent add()s. Is LdapNetworkConnection meant to be thread-safe? Or should I just use separate LdapConnection for each thread? -- Hendy Irawan - on Twitter http://twitter.com/hendybippo - on LinkedInhttp://id.linkedin.com/in/hendyirawan Web Developer | Bippo Indonesia http://www.bippo.co.id/ | Akselerator Bisnis | Bandung -- Kiran Ayyagari
Re: LdapNetworkConnection not thread-safe ?
Le 6/27/12 5:23 PM, Hendy Irawan a écrit : Hi ApacheDS developers, I'm working on an open source project ( https://github.com/soluvas/ldap-tools ) which uses LdapNetworkConnection using shared v1.0.0-M12. Several threads are running in parallel (using Akka), all using the same LdapNetworkConnection to delete entries. It's not necessarily a good idea to use a single connection in many threads. Think about it as if they where JDBC connections. Note that it should be supported, as the response are associated with a request ID, which is incremented everytime you send a new request (and the responses are associated with the requestID, so we should not have a problem here.) I suggest you fill a JIRA expliciting the problem, we will investigate. In the mean ime, I assume it's safer to use one connection per thread. However in some cases it locks up (deadlock? race condition?) and the last logs I get is : ... 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-14] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setsuna_hinagiku,ou=users,dc=berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-dispatcher-24] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=rumah_amal_salman_itb,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-18] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setyo_rini,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-1] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=pipit_nugroho,ou=users,dc=berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-dispatcher-15] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=yuliana_riris_basaria,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-16] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setia_budi,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 I also experience similar issues doing concurrent add()s. Is LdapNetworkConnection meant to be thread-safe? Or should I just use separate LdapConnection for each thread? -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Cannot delete entry - ldap_delete: Operation not allowed on non-leaf (66) - NOT_ALLOWED_ON_NON_LEAF: failed for MessageType : DEL_REQUEST -- Re: LdapNetworkConnection not thread-safe ?
Hi, Thank you for the advice. I've filed a JIRA for thread-safety issue here https://issues.apache.org/jira/browse/DIRSHARED-135 For the time being I'm happy with a workaround. However, I experienced a more serious issue, that I cannot delete an entry, also not able to rename it. Seems like I managed to make the partition inconsistent in some way. I filed the issue here : https://issues.apache.org/jira/browse/DIRSERVER-1733 On Wed, Jun 27, 2012 at 10:59 PM, Emmanuel Lécharny elecha...@gmail.comwrote: Le 6/27/12 5:23 PM, Hendy Irawan a écrit : Hi ApacheDS developers, I'm working on an open source project ( https://github.com/soluvas/**ldap-toolshttps://github.com/soluvas/ldap-tools) which uses LdapNetworkConnection using shared v1.0.0-M12. Several threads are running in parallel (using Akka), all using the same LdapNetworkConnection to delete entries. It's not necessarily a good idea to use a single connection in many threads. Think about it as if they where JDBC connections. Note that it should be supported, as the response are associated with a request ID, which is incremented everytime you send a new request (and the responses are associated with the requestID, so we should not have a problem here.) I suggest you fill a JIRA expliciting the problem, we will investigate. In the mean ime, I assume it's safer to use one connection per thread. However in some cases it locks up (deadlock? race condition?) and the last logs I get is : ... 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-**dispatcher-14] INFO o.soluvas.ldaptools.cli.**PersonClear - Deleting uid=setsuna_hinagiku,ou=users,**dc=berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-**dispatcher-24] INFO o.soluvas.ldaptools.cli.**PersonClear - Deleting uid=rumah_amal_salman_itb,ou=**users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-**dispatcher-18] INFO o.soluvas.ldaptools.cli.**PersonClear - Deleting uid=setyo_rini,ou=users,dc=**berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-**dispatcher-1] INFO o.soluvas.ldaptools.cli.**PersonClear - Deleting uid=pipit_nugroho,ou=users,dc=**berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-**dispatcher-15] INFO o.soluvas.ldaptools.cli.**PersonClear - Deleting uid=yuliana_riris_basaria,ou=**users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-**dispatcher-16] INFO o.soluvas.ldaptools.cli.**PersonClear - Deleting uid=setia_budi,ou=users,dc=**berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 I also experience similar issues doing concurrent add()s. Is LdapNetworkConnection meant to be thread-safe? Or should I just use separate LdapConnection for each thread? -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com -- Hendy Irawan - on Twitter http://twitter.com/hendybippo - on LinkedInhttp://id.linkedin.com/in/hendyirawan Web Developer | Bippo Indonesia http://www.bippo.co.id/ | Akselerator Bisnis | Bandung
