Figured it out. In this case the handshake error means that the data being served does not cryptographically correspond to the trusted certificate. The value of "userCertificate" must be derived from the value of "privateKey". If you change "userCertificate" on "uid=admin,ou=system" you must also change "privateKey" to the corresponding private key.
Getting a private key and corresponding certificate is a bit difficult with keytool (the Java key/certificate tool) because keytool does not expose private keys. Here is the procedure I came up with (copied from another document): *** Installing a Certificate Generated By Keytool *** When you create a new server, a private key and certificate are automatically created on the admin entry (uid=admin,ou=system). Unfortunately, the certificate references an non-existant issuer. This means that clients which expect a valid certificate cannot connect to the server. In this procedure we will: 1. create a keystore containing a private key & certificate. 2. export the certificate 3. export the public key to X.509/DER format 4. export the private key to PKCS#8/DER format 5. import the keys and certificate to ApacheDS # create a PKCS#12 keystore containing a 2048 bit RSA private key and a certificate for localhost # the CN must match the host name of the server. A CN of "localhost" will not work for ldaps://my-server:389 or vice-versa # we create a keystore in PKCS#12 format for consumption by OpenSSL keytool -genkeypair -keyalg RSA -keysize 2048 -validity 365 -alias ldap -dname "cn=localhost" -keypass changeit -keystore ldap.p12 -storepass changeit -storetype PKCS12 # extract a certificate from the keystore keytool -exportcert -alias ldap -rfc -keystore ldap.p12 -storepass changeit -storetype PKCS12 -file ldap.cer # extract the private key from the keystore openssl pkcs12 -in ldap.p12 -passin pass:changeit -nodes -nocerts | openssl rsa | openssl pkcs8 -topk8 -nocrypt -outform DER -out ldap-privatekey.der # derive a public key from the private key in the keystore (this may be incorrect, but it does not seem to matter for ApacheDS) openssl pkcs12 -in ldap.p12 -passin pass:changeit -nodes -nocerts | openssl rsa -pubout -outform DER -out ldap-publickey.der # import the server certificate to the truststore for V-Flex to use # this is a self-signed (root) certificate, so you be asked to confirm that you trust it keytool -importcert -alias ldap -keystore .truststore -storepass changeit -keypass changeit -file ldap.cer To utilize the keys and certificate in ApacheDS: 1. browse to uid=admin,ou=system in the LDAP Browser 2. double-click on privateKey, click Load Data..., select ldap-privatekey.der and click OK 3. double-click on publicKey, click Load Data..., select ldap-publickey.der and click OK 4. double-click on userCertificate, click Load Certificate..., select ldap.cer and click OK 5. disconnect from the server 6. stop the server 7. restart the server 8. connect to the server 9. accept the new certificate as trusted -- Mat Gessel http://www.asquare.net On Tue, Jun 26, 2012 at 12:30 PM, Mat Gessel <mat.ges...@gmail.com> wrote: > However, I am unable to connect > when I specify a self-signed certificate for the server (via > uid=admin,ou=system).
