SSL connection with client certificate authentication closing randomly

[Michael Lam]

    Hi all,

    We have been running into an issue lately where our client certificate
authenticated SSL connections are randomly closing with a TLS alert
"close_notify".  The strange thing is that out of 10 tries, the connection
may work maybe around 2-3 times.  All other times, the connections will
receive a "close_notify" and the connection will close.  For comparison
purposes, we tried using command-line CURL to submit the same request with
client certificate authentication and we were able to connect and send data
without issues every time.

Below is the debug (with -Djavax.net.debug=all) output from a sample
session. I have removed the actual data and replaced them with place
holders.

    ... <more data and messages>
    Client MAC write Secret:
    <data>
    Server MAC write Secret:
    <data>
    Client write key:
    <data>
    Server write key:
    <data>
    ... no IV used for this cipher
    Padded plaintext before ENCRYPTION:  len = 17
    <data>
    main, WRITE: TLSv1 Change Cipher Spec, length = 17
    [Raw write]: length = 22
    <data>
    *** Finished
    verify_data:  <data>
    ***
    [write] MD5 and SHA1 hashes:  len = 16
    <data>
    Padded plaintext before ENCRYPTION:  len = 32
    <data>
    main, WRITE: TLSv1 Handshake, length = 32
    <data>
    main, received EOFException: ignored
    main, called closeInternal(false)
    main, SEND TLSv1 ALERT:  warning, description = close_notify
    Padded plaintext before ENCRYPTION:  len = 18
    <data>
    main, WRITE: TLSv1 Alert, length = 18
    main, Exception sending alert: java.net.SocketException: Software caused
connection abort: socket write error
    2011-01-12 11:20:59,908 DEBUG
org.apache.commons.httpclient.HttpMethodDirector - Closing the connection.
    2011-01-12 11:20:59,908 DEBUG
org.apache.commons.httpclient.HttpConnection - enter HttpConnection.close()
    2011-01-12 11:20:59,908 DEBUG
org.apache.commons.httpclient.HttpConnection - enter
HttpConnection.closeSockedAndStreams()
    main, called close()
    main, called closeInternal(true)
    main, called close()
    main, called closeInternal(true)
    main, called close()
    main, called closeInternal(true)
    2011-01-12 11:20:59,909 INFO
org.apache.commons.httpclient.HttpMethodDirector - I/O exception
(org.apache.commons.httpclient.NoHttpResponseException) caught when
processing request: The server <host> failed to respond
    2011-01-12 11:20:59,912 DEBUG
org.apache.commons.httpclient.HttpMethodDirector - The server <host> failed
to respond
    org.apache.commons.httpclient.NoHttpResponseException: The server
www.callit.com failed to respond
    at
org.apache.commons.httpclient.HttpMethodBase.readStatusLine(HttpMethodBase.java:1976)
    at
org.apache.commons.httpclient.HttpMethodBase.readResponse(HttpMethodBase.java:1735)
    at
org.apache.commons.httpclient.HttpMethodBase.execute(HttpMethodBase.java:1098)
    at
org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:398)
    at
org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
    at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
    at
org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
            at SSLConnectionTest.main(SSLConnectionTest.java:83)


Here is the code I used to connect to the host:

        HttpClientParams params = new
HttpClientParams(DefaultHttpParams.getDefaultParams());
        HttpClient httpclient = new HttpClient(params);

        AuthSSLProtocolSocketFactory socketFactory = new
AuthSSLProtocolSocketFactory(keyStoreFileUrl.toURL(), keyStorePwd, null,
null);
        Protocol httpsProtocol = new Protocol("https", socketFactory, 443);
        httpclient.getHostConfiguration().setHost("www.myhost.com", 443,
httpsProtocol);

        PostMethod httppost = new PostMethod("/vl/feature.asp");

        NameValuePair[] data = {
                  new NameValuePair("Query", "function"),
                };
        try {
          httppost.setRequestBody(data);
          httpclient.executeMethod(httppost);
          System.out.println(httppost.getResponseBodyAsString());
        } catch (HttpException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        } finally {
            httppost.releaseConnection();
        }

Within the AuthSSLProtocolSocketFactory, we also use the
AuthSSLX509TrustManager and a custom KeyManager that stores the client
certificate and private key.  Any pointers or tips to help debug this issue
will be greatly appreciated.

Regards,
Mike