LdapNetworkConnection not thread-safe ?
Hi ApacheDS developers, I'm working on an open source project ( https://github.com/soluvas/ldap-tools ) which uses LdapNetworkConnection using shared v1.0.0-M12. Several threads are running in parallel (using Akka), all using the same LdapNetworkConnection to delete entries. However in some cases it locks up (deadlock? race condition?) and the last logs I get is : ... 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-14] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setsuna_hinagiku,ou=users,dc=berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-dispatcher-24] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=rumah_amal_salman_itb,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-18] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setyo_rini,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-1] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=pipit_nugroho,ou=users,dc=berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-dispatcher-15] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=yuliana_riris_basaria,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-16] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setia_budi,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 I also experience similar issues doing concurrent add()s. Is LdapNetworkConnection meant to be thread-safe? Or should I just use separate LdapConnection for each thread? -- Hendy Irawan - on Twitter http://twitter.com/hendybippo - on LinkedInhttp://id.linkedin.com/in/hendyirawan Web Developer | Bippo Indonesia http://www.bippo.co.id/ | Akselerator Bisnis | Bandung
Re: LdapNetworkConnection not thread-safe ?
it is not thread safe a bit of history: earlier we made it thread safe, but later dropped the thread safety features it still supports the async feature though On Wed, Jun 27, 2012 at 8:53 PM, Hendy Irawan he...@soluvas.com wrote: Hi ApacheDS developers, I'm working on an open source project ( https://github.com/soluvas/ldap-tools ) which uses LdapNetworkConnection using shared v1.0.0-M12. Several threads are running in parallel (using Akka), all using the same LdapNetworkConnection to delete entries. However in some cases it locks up (deadlock? race condition?) and the last logs I get is : ... 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-14] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setsuna_hinagiku,ou=users,dc=berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-dispatcher-24] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=rumah_amal_salman_itb,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-18] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setyo_rini,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-1] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=pipit_nugroho,ou=users,dc=berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-dispatcher-15] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=yuliana_riris_basaria,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-16] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setia_budi,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 I also experience similar issues doing concurrent add()s. Is LdapNetworkConnection meant to be thread-safe? Or should I just use separate LdapConnection for each thread? -- Hendy Irawan - on Twitter http://twitter.com/hendybippo - on LinkedInhttp://id.linkedin.com/in/hendyirawan Web Developer | Bippo Indonesia http://www.bippo.co.id/ | Akselerator Bisnis | Bandung -- Kiran Ayyagari
Re: LdapNetworkConnection not thread-safe ?
Le 6/27/12 5:23 PM, Hendy Irawan a écrit : Hi ApacheDS developers, I'm working on an open source project ( https://github.com/soluvas/ldap-tools ) which uses LdapNetworkConnection using shared v1.0.0-M12. Several threads are running in parallel (using Akka), all using the same LdapNetworkConnection to delete entries. It's not necessarily a good idea to use a single connection in many threads. Think about it as if they where JDBC connections. Note that it should be supported, as the response are associated with a request ID, which is incremented everytime you send a new request (and the responses are associated with the requestID, so we should not have a problem here.) I suggest you fill a JIRA expliciting the problem, we will investigate. In the mean ime, I assume it's safer to use one connection per thread. However in some cases it locks up (deadlock? race condition?) and the last logs I get is : ... 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-14] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setsuna_hinagiku,ou=users,dc=berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-dispatcher-24] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=rumah_amal_salman_itb,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-18] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setyo_rini,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-1] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=pipit_nugroho,ou=users,dc=berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-dispatcher-15] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=yuliana_riris_basaria,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-dispatcher-16] INFO o.soluvas.ldaptools.cli.PersonClear - Deleting uid=setia_budi,ou=users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 I also experience similar issues doing concurrent add()s. Is LdapNetworkConnection meant to be thread-safe? Or should I just use separate LdapConnection for each thread? -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com
Cannot delete entry - ldap_delete: Operation not allowed on non-leaf (66) - NOT_ALLOWED_ON_NON_LEAF: failed for MessageType : DEL_REQUEST -- Re: LdapNetworkConnection not thread-safe ?
Hi, Thank you for the advice. I've filed a JIRA for thread-safety issue here https://issues.apache.org/jira/browse/DIRSHARED-135 For the time being I'm happy with a workaround. However, I experienced a more serious issue, that I cannot delete an entry, also not able to rename it. Seems like I managed to make the partition inconsistent in some way. I filed the issue here : https://issues.apache.org/jira/browse/DIRSERVER-1733 On Wed, Jun 27, 2012 at 10:59 PM, Emmanuel Lécharny elecha...@gmail.comwrote: Le 6/27/12 5:23 PM, Hendy Irawan a écrit : Hi ApacheDS developers, I'm working on an open source project ( https://github.com/soluvas/**ldap-toolshttps://github.com/soluvas/ldap-tools) which uses LdapNetworkConnection using shared v1.0.0-M12. Several threads are running in parallel (using Akka), all using the same LdapNetworkConnection to delete entries. It's not necessarily a good idea to use a single connection in many threads. Think about it as if they where JDBC connections. Note that it should be supported, as the response are associated with a request ID, which is incremented everytime you send a new request (and the responses are associated with the requestID, so we should not have a problem here.) I suggest you fill a JIRA expliciting the problem, we will investigate. In the mean ime, I assume it's safer to use one connection per thread. However in some cases it locks up (deadlock? race condition?) and the last logs I get is : ... 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-**dispatcher-14] INFO o.soluvas.ldaptools.cli.**PersonClear - Deleting uid=setsuna_hinagiku,ou=users,**dc=berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-**dispatcher-24] INFO o.soluvas.ldaptools.cli.**PersonClear - Deleting uid=rumah_amal_salman_itb,ou=**users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-**dispatcher-18] INFO o.soluvas.ldaptools.cli.**PersonClear - Deleting uid=setyo_rini,ou=users,dc=**berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-**dispatcher-1] INFO o.soluvas.ldaptools.cli.**PersonClear - Deleting uid=pipit_nugroho,ou=users,dc=**berbatik,dc=com 22:17:17 [ldap_cli-akka.actor.default-**dispatcher-15] INFO o.soluvas.ldaptools.cli.**PersonClear - Deleting uid=yuliana_riris_basaria,ou=**users,dc=berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [ldap_cli-akka.actor.default-**dispatcher-16] INFO o.soluvas.ldaptools.cli.**PersonClear - Deleting uid=setia_budi,ou=users,dc=**berbatik,dc=com 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 22:17:17 [NioProcessor-2] DEBUG o.a.m.f.codec.**ProtocolCodecFilter - Processing a MESSAGE_RECEIVED for session 1 I also experience similar issues doing concurrent add()s. Is LdapNetworkConnection meant to be thread-safe? Or should I just use separate LdapConnection for each thread? -- Regards, Cordialement, Emmanuel Lécharny www.iktek.com -- Hendy Irawan - on Twitter http://twitter.com/hendybippo - on LinkedInhttp://id.linkedin.com/in/hendyirawan Web Developer | Bippo Indonesia http://www.bippo.co.id/ | Akselerator Bisnis | Bandung
Re: Can't connect w/ encryption after loading a custom certificate
Figured it out. In this case the handshake error means that the data being served does not cryptographically correspond to the trusted certificate. The value of userCertificate must be derived from the value of privateKey. If you change userCertificate on uid=admin,ou=system you must also change privateKey to the corresponding private key. Getting a private key and corresponding certificate is a bit difficult with keytool (the Java key/certificate tool) because keytool does not expose private keys. Here is the procedure I came up with (copied from another document): *** Installing a Certificate Generated By Keytool *** When you create a new server, a private key and certificate are automatically created on the admin entry (uid=admin,ou=system). Unfortunately, the certificate references an non-existant issuer. This means that clients which expect a valid certificate cannot connect to the server. In this procedure we will: 1. create a keystore containing a private key certificate. 2. export the certificate 3. export the public key to X.509/DER format 4. export the private key to PKCS#8/DER format 5. import the keys and certificate to ApacheDS # create a PKCS#12 keystore containing a 2048 bit RSA private key and a certificate for localhost # the CN must match the host name of the server. A CN of localhost will not work for ldaps://my-server:389 or vice-versa # we create a keystore in PKCS#12 format for consumption by OpenSSL keytool -genkeypair -keyalg RSA -keysize 2048 -validity 365 -alias ldap -dname cn=localhost -keypass changeit -keystore ldap.p12 -storepass changeit -storetype PKCS12 # extract a certificate from the keystore keytool -exportcert -alias ldap -rfc -keystore ldap.p12 -storepass changeit -storetype PKCS12 -file ldap.cer # extract the private key from the keystore openssl pkcs12 -in ldap.p12 -passin pass:changeit -nodes -nocerts | openssl rsa | openssl pkcs8 -topk8 -nocrypt -outform DER -out ldap-privatekey.der # derive a public key from the private key in the keystore (this may be incorrect, but it does not seem to matter for ApacheDS) openssl pkcs12 -in ldap.p12 -passin pass:changeit -nodes -nocerts | openssl rsa -pubout -outform DER -out ldap-publickey.der # import the server certificate to the truststore for V-Flex to use # this is a self-signed (root) certificate, so you be asked to confirm that you trust it keytool -importcert -alias ldap -keystore .truststore -storepass changeit -keypass changeit -file ldap.cer To utilize the keys and certificate in ApacheDS: 1. browse to uid=admin,ou=system in the LDAP Browser 2. double-click on privateKey, click Load Data..., select ldap-privatekey.der and click OK 3. double-click on publicKey, click Load Data..., select ldap-publickey.der and click OK 4. double-click on userCertificate, click Load Certificate..., select ldap.cer and click OK 5. disconnect from the server 6. stop the server 7. restart the server 8. connect to the server 9. accept the new certificate as trusted -- Mat Gessel http://www.asquare.net On Tue, Jun 26, 2012 at 12:30 PM, Mat Gessel mat.ges...@gmail.com wrote: However, I am unable to connect when I specify a self-signed certificate for the server (via uid=admin,ou=system).
