I think I found the problem in Java, and a partial workaround. Essentially the java HTTP client only sends the certificate if the server requests it in NeedClientAuth, not in WantClientAuth mode. This is a subtle bug and difficult to explain, or rather it requires moving all the way from the lowest levels of TLS to the user experience in a browser. Java used to have the ambition to become a language to write a browser (hotjava) and I think this is still a valid aim - in any case every application long term will become a browser application.
So what is the UI problem? The difference between WANT and NEED mode is that in WANT TLS mode the server can ask the client for a certificate, and if the client does not have one, or does not wish to continue, the connection will be pursued but without client authentication. This means that the browser can for example return an HTTP 400 page with a human readable explanation of the problem, or better even redirect the user to a account creation page, or to another site, or whatever... If the server requests the certificate in NEED mode, then if the client does not send the certificate the TLS connection is abruptly and finally closed leaving the user with a not so beautiful and not very explanatory error message page. It's the TLS equivalent of having the door shut in one's face without an explanation. So Java clients currently, I discovered - but I may still be shown some to be wrong - only seems to send a certificate when requested in NEED mode. I hope we can fix this or find a way for java clients to send certificates in WANT mode. We should certainly open a bug report on this. But I have no idea where one reports Java bugs nowadays. bugs.sun.com seems to be very broken. I really needed the Java clients to work at least in order to run the test suite to test the http://webid.info/ authentication in the read-write-web project. After all it would be silly not to be able to test the code that works for most desktop browsers in Java! So I had to find a workaround. Here is the code that allows me to test the connection to a TLS protected resource that needs authentication. In line 40 I enable the TLS client certificate to user "JoeLambda", so that if the client code ever gets requested a certificate in a TLS connection it can return that users cert. We then connect to the protected resource, using the basic http libraries. 240 testKeyManager.setId("JoeLambda") 245 val scon =webidProfile.secure.to_uri.toURL.openConnection().asInstanceOf[HttpsURLConnection] 246 scon.setSSLSocketFactory(sslContext.getSocketFactory) 247 scon.setRequestProperty("Content-Type",Post.SPARQL) 248 scon.setRequestProperty("User-Agent" , "Java/1.7.0") 249 scon.setRequestMethod("POST") 250 val msg = updateFriend.format("http://bblfish.net/#hjs";).getBytes("UTF-8") 251 scon.setRequestProperty("Content-Length",msg.length.toString) 252 scon.setDoOutput(true) 253 scon.setDoInput(true) 254 255 val out = scon.getOutputStream 256 out.write(msg) 257 out.flush() 258 out.close() 259 scon.connect() 260 261 val httpCode = scon.getResponseCode 262 263 264 val req =webidProfile.secure.PUT <:< Map("User-Agent" -> "Java/1.7.0","Content-Type"->Post.SPARQL) 265 val req2 = req.copy( 266 method="POST", 267 body=Some(new RefStringEntity(updateFriend.format(webID.toExternalForm),Post.SPARQL,"UTF-8")) 268 ) 269 270 val httpCode = Http( req2 get_statusCode ) 271 httpCode must_== 200 Because in this test case the resource is protected for POSTs the server ends up calling the authentication module, which tries to find a cert for the user. This then ends up calling the netty code below. On line 182 the server looks at the HTTP headers of the request, and if that header contains "Java" it sets the ssl engine in NeedClientAuth mode (line 182). For desktop browser the WantClientAuth mode is better, as that allows a user who does not have a certificate to log in with other technologies such as OAuth, OpenID, WebId etc... It then renegotiates the connection in line 186, and the waits for a maximum of 30 seconds for a response. By line 190 the server has the client certificate now, and can proceed with authorisation. 173 r.underlying.context.getPipeline.get(classOf[SslHandler]) match { 174 case sslh: SslHandler => try { 175 //return the client certificate in the existing session if one exists 176 Some(sslh.getEngine.getSession.getPeerCertificates) 177 } catch { 178 case e => { 179 // request a certificate from the user 180 sslh.setEnableRenegotiation(true) 181 r match { 182 case UserAgent(agent) if needAuth(agent) => sslh.getEngine.setNeedClientAuth(true) 183 case _ => sslh.getEngine.setWantClientAuth(true) 184 } 185 186 val future = sslh.handshake() 187 future.await(30000) //that's certainly way too long. 188 if (future.isDone) { 189 if (future.isSuccess) try { 190 Some(sslh.getEngine.getSession.getPeerCertificates) 191 } catch { 192 case e => None 193 } else { 194 None 195 } 196 } else { 197 None 198 } 199 } 200 } 201 case _ => None 202 } So in summary: does anyone know if there is a way to avoid this behaviour in the TLS client? And if not how do I put together a bug report, and to whom do I send it to? Henry On 28 Oct 2011, at 17:17, Oleg Kalnichevski wrote: > On Fri, Oct 28, 2011 at 04:44:59PM +0200, Henry Story wrote: >> >> On 28 Oct 2011, at 16:01, Oleg Kalnichevski wrote: >> >>> On Fri, Oct 28, 2011 at 10:21:42AM +0200, Henry Story wrote: >>>> I have a bit more support now in thinking that this is an issue with lack >>>> of support for TLS renegotiation. >>>> I added the following code to my test, which calls a non TLS renegotiating >>>> server >>>> >>>> "testing client certs" should { >>>> "connect to foafssl.org and ask for cert" in { >>>> keyManager.setId("JoeLambda") >>>> val foafssl = :/("foafssl.org",443)/"test/WebId" secure >>>> val model = Http(foafssl as_model(baseURI(foafssl),TURTLE) ) >>>> model.write(System.out,TURTLE.jenaLang) >>>> model.size() must_==10 //should be greater than, but anyway >>>> } >>>> } >>>> >>>> When I connect to foafssl.org - but any non TLS renegotiating server would >>>> do I believe - then the methods in my FlexiKeyManager get called in the >>>> order expected: namely first it gets asked for the aliases, then for a >>>> certificate for that alias, and finally for the private key for that >>>> alias. This does not happen when connecting to the server that does >>>> renegotiation. >>>> >>>> class FlexiKeyManager extends X509ExtendedKeyManager { >>>> val keys = mutable.Map[String, Pair[Array[X509Certificate],PrivateKey]]() >>>> >>>> def addClientCert(alias: String,certs: Array[X509Certificate], privateKey: >>>> PrivateKey) { >>>> keys.put(alias,Pair(certs,privateKey)) >>>> } >>>> >>>> var currentId: String = null >>>> >>>> def setId(alias: String) { currentId = if (keys.contains(alias)) alias >>>> else null } >>>> def getClientAliases(keyType: String, issuers: Array[Principal]) = >>>> if (currentId!=null) Array(currentId) else null >>>> def chooseClientAlias(keyType: Array[String], issuers: Array[Principal], >>>> socket: Socket) = >>>> currentId >>>> def getServerAliases(keyType: String, issuers: Array[Principal]) = null >>>> def chooseServerAlias(keyType: String, issuers: Array[Principal], socket: >>>> Socket) = "" >>>> def getCertificateChain(alias: String) = keys.get(alias) match { >>>> case Some(certNKey) => certNKey._1; >>>> case None => null >>>> } >>>> def getPrivateKey(alias: String) = >>>> keys.get(alias).map(ck=>ck._2).getOrElse(null) >>>> >>>> override def chooseEngineClientAlias(keyType: Array[String], issuers: >>>> Array[Principal], engine: SSLEngine): String = currentId >>>> } >>>> >>>> >>> >>> Hi Henry >>> >>> HttpClient has absolutely no control over TLS protocol aspects. It merely >>> leverages TLS/SSL capabilities provided by Java JSSE. As far as I know the >>> latest Java releases ship with support for the TLS renegotiation disabled. >>> You can try enabling it using instructions below or consider using an >>> alternative JSSE implementation. >>> >>> http://download.oracle.com/javase/6/docs/technotes/guides/security/jsse/JSSERefGuide.html#tlsRenegotiation >>> http://java.sun.com/javase/javaseforbusiness/docs/TLSReadme.html >> >> Thanks Oleg, >> >> I am already running with those following properties >> >> -Dsun.security.ssl.allowUnsafeRenegotiation=true >> -Dsun.security.ssl.allowLegacyHelloMessages=true >> >> set. I know I am doing this because I am also running the server in the same >> VM as the client here: the testing engine. I also just printed out all the >> system properties in the client code just to make sure, and they were still >> there. >> >> Btw, on the latest JVMs TLS renegotiation is back on by default >> >> http://download.oracle.com/javase/7/docs/technotes/guides/security/jsse/JSSERefGuide.html#descPhase2 >> >> So could it be that there is a bug in the Java code? That would be >> interesting, because it would show that they had not tested their server >> with their own client libraries, which would be somewhat odd in fact. >> Perhaps I'll post a bug report then on the oracle web site. >> >> Henry >> >> > > I think that with SSL debug one one should be able to see whether or not the > TLS renegotiation is enabled and whether or not the client attempts to > renegotiate in case the support for renegotiation is active. Thanks that was helpful. > > Oleg > > > >>> >>> Oleg >>> >>>> >>>> >>>> On 28 Oct 2011, at 00:49, Henry Story wrote: >>>> >>>>> Hello, >>>>> >>>>> I am working on a server that tries to ask the client for his X509 >>>>> certificate only when it is sure that it will be needed. This can be done >>>>> very neatly using TLS renegotiation: the server can analysing the HTTP >>>>> request to see if action requested on the resource needs authentication >>>>> at all. If so it requests a TLS renegotiations as show in this mini netty >>>>> server written in one page of Scala [1]. >>>>> >>>>> I am now trying to test this. Most desktop browsers accept some form of >>>>> TLS renegotiation - except Opera 11 I think. But I am not sure that java >>>>> http client does. I am using the dispatch scala wrapping of the >>>>> httpclient, and so I am cling them this too. >>>>> >>>>> The code for these tests is here: >>>>> >>>>> https://dvcs.w3.org/hg/read-write-web/file/c0bf9b280888/src/test/scala/auth/CreateWebIDSpec.scala >>>>> >>>>> The test after line 234 does not return the right result. After a lot of >>>>> stepping through code it occurred to me that perhaps httpclient does not >>>>> do renegotiation. Perhaps I have not set it up properly to do this. But >>>>> it could also be another issue. As it is late, I thought I'd ask before >>>>> going to sleep. >>>>> >>>>> Thanks in advance, >>>>> >>>>> Henry >>>>> >>>>> >>>>> [1] in the webid branch of the read-write-web project around line 64 >>>>> https://dvcs.w3.org/hg/read-write-web/file/9ca474c333e8/src/main/scala/netty/SslLoginTest.scala >>>>> [2] http://dispatch.databinder.net/Dispatch.html >>>>> >>>>> >>>>> Social Web Architect >>>>> http://bblfish.net/
