I created a self signed certificate and used it to sign a server's certificate. I put root in the client's store, using a custom X509_LOOKUP, overriding the get_by_subject() method. I see that I'm indeed returning the correct certificate in the return object from get_by_subject() and also a return value of X509_LU_X509. I also compared the issuer name of the server certificate and the subject name of the root certificate by printing then using X509_NAME_print_ex().
But what seems to be failing is X509_NAME_cmp(). I saw that the canon_enclen's of the root and server were different by 12 bytes. To be clear, I'm getting the server certificate (which the server loads from a PEM file) during the SSL handshake, while the root certificate was loaded from a DER file. I'm also attaching the certificates in question. The password for the Server.pem file is 'particle'. Any clues?
ServerRoot.pem
Description: Binary data
Server.der
Description: Binary data
Server.pem
Description: Binary data
ServerRoot.der
Description: Binary data
