Also if I use keytool on the truststore I can see the entry:
()[root@overcloud-controller-0 ssl]# keytool -list -v -keystore
truststore.jks
[34/1923]
Enter keystore password:
Keystore type: JKS
Keystore provider: SUN
Your keystore contains 1 entry
Alias name: overcloud-controller-0
Creation date: Mar 20, 2018
Entry type: trustedCertEntry
Owner: CN=overcloud-controller-0.internalapi.opnfvlf.org, O=OPNFVLF.ORG
Issuer: CN=Certificate Authority, O=OPNFVLF.ORG
Serial number: 12b
Valid from: Tue Mar 20 09:34:11 UTC 2018 until: Fri Mar 20 09:34:11 UTC
2020
Certificate fingerprints:
MD5: 46:1D:AB:A3:66:06:59:3C:1F:E6:83:6F:99:86:14:6A
SHA1: EB:0A:5D:8D:4C:BB:DA:D1:76:DD:52:5D:4B:B5:13:4F:09:EE:6B:4E
SHA256:
CC:13:42:AE:75:9B:00:58:18:05:3B:A0:BE:A6:02:CF:98:54:E7:13:3E:27:51:DB:36:76:22:B7:B1:1C:B3:A0
Signature algorithm name: SHA256withRSA
Subject Public Key Algorithm: 2048-bit RSA key
Version: 3
Tim Rozet
Red Hat SDN Team
On Tue, Mar 20, 2018 at 4:03 PM, Tim Rozet <[email protected]> wrote:
> Hi Mohamed,
> Just to clarify...we do create the ctl.jks and place it in
> configuration/ssl before ODL boots. The truststore is created on first
> boot, I can see truststore.jks in the same directory. I also believe that
> the certificate for OVS is in the truststore:
>
> [root@overcloud-controller-0 ~]# curl -k -X POST --fail --silent -H
> 'Content-Type: application/json' -H 'Cache-Control: no-cache' -u
> admin:admin -d '{ "aaa-cert-rpc:input": { "aaa-cert-rpc:node-alias":
> "overcloud-controller-0" }}' https://192.0.2.10:8081/
> restconf/operations/aaa-cert-rpc:getNodeCertifcate
> {"output":{"node-cert":"MIIFGjCCBAKgAwIBAgICASswDQYJKo
> ZIhvcNAQELBQAwNjEUMBIGA1UECgwLT1BORlZMRi5PUkcxHjAcBgNVBAMMFU
> NlcnRpZmljYXRlIEF1dGhvcml0eTAeFw0xODAzMjAwOTM0MTFaFw0yMDAzMj
> AwOTM0MTFaME8xFDASBgNVBAoMC09QTkZWTEYuT1JHMTcwNQYDVQQDDC5vdm
> VyY2xvdWQtY29udHJvbGxlci0wLmludGVybmFsYXBpLm9wbmZ2bGYub3JnMI
> IBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoLDetkUZscbmllZNKc
> 9u1B9eDNW4dptIHn8PpUNG6cHnwLhmuynRDwgI0zYPaoZnFy6QzyeCtpVARg
> qpUDGB0SKyHPoO38YYwXuEUfK27mpfCFsZ0vKwmt4827fqglUQXMNlypyaPK
> drcx1w0z++E8x5DUixgL+trjjWtDBlooFkGs/qVoWFgySUvw/+CZ/
> mfg3G3W7oVX1k3zqf75wxsqzM7ZePGn7W0UPuPjNkkOOI9Hl8PFoLeLwZXuY
> aAaJmz3O6UffvRZgl3aSgPuCK31fM5tIfeCNnlQMxMdZZiPrbRpAi9O5vpaX
> xM/d6r7HkTI6N9l0g/hX7P53hearImQIDAQABo4ICFzCCAhM
> wHwYDVR0jBBgwFoAUEGBKlCQqJ05za3ZwFRfuCW0ae0owPQYIKwYBBQUHAQE
> EMTAvMC0GCCsGAQUFBzABhiFodHRwOi8vaXBhLWNhLm9wbmZ2bGYub3JnL2N
> hL29jc3AwDgYDVR0PAQH/BAQDAgTwMB0GA1UdJQQWMBQGCCsGAQ
> UFBwMBBggrBgEFBQcDAjB2BgNVHR8EbzBtMGugM6Axhi9odHRwOi8vaXBhLW
> NhLm9wbmZ2bGYub3JnL2lwYS9jcmwvTWFzdGVyQ1JMLmJpbqI0pDIwMDEOMA
> wGA1UECgwFaXBhY2ExHjAcBgNVBAMMFUNlcnRpZmljYXRlIEF1dGhvcml0eT
> AdBgNVHQ4EFgQUcySBEFDXvFw45hSfRhLgTlpxcrowgeoGA1UdEQSB4jCB34
> Iub3ZlcmNsb3VkLWNvbnRyb2xsZXItMC5pbnRlcm5hbGFwaS5vcG5mdmxmLm
> 9yZ6BOBgorBgEEAYI3FAIDoEAMPm92cy9vdmVyY2xvdWQtY29udHJvbGxlci
> 0wLmludGVybmFsYXBpLm9wbmZ2bGYub3JnQE9QTkZWTEYuT1JHoF0GBisGAQ
> UCAqBTMFGgDRsLT1BORlZMRi5PUkehQDA+oAMCAQGhNzA1GwNvdnMbLm92ZXJjbG
> 91ZC1jb250cm9sbGVyLTAuaW50ZXJuYWxhcGkub3BuZnZsZi5vcmcwDQYJKo
> ZIhvcNAQELBQADggEBAF+7W//ICAw0i+FxPr9hSG2rlWcauo0Zllk2ff5e9NGb
> f7uu7aJVINYRVpGW7IN0kgsq63QuCGZu7y/zCiPb5hmWuBGHKgqHFXTjLUXGmSQXJ
> 5yEuE7yLTmN5oHbXxJgbXgJoy7PDOLziAnWRQkrScBvLNZur7+
> 06FurN2UPtCMfrnn8cmbiteLBW6sdnzCFEUlrkU6bHOFPJWwAGski6YpdHYK
> cnw4p3J22phABCSRe8dq7xbIZx0wBg6WaSGTlwshqUSFEDX4VrjtVoWE06ip
> TSbHR3Ra93kh1dEd3VdMPND/CdflsIOcsiykD7SZl7oli9c35asqjipOAvI8HCn8="}}
>
>
> Let me know if there are any other debug outputs that could be helpful.
>
> Thanks,
>
> Tim Rozet
> Red Hat SDN Team
>
> On Tue, Mar 20, 2018 at 2:17 PM, Mohamed El-Serngawy <
> [email protected]> wrote:
>
>> Sorry Tim, just confused ignore my previous email the keystore will be
>> created anyway [1]. "didn't maintain this code for a long time"
>>
>> [1] https://github.com/opendaylight/aaa/blob/master/aaa-
>> cert/src/main/java/org/opendaylight/aaa/cert/impl/AaaCertPro
>> vider.java#L89
>>
>> On Tue, Mar 20, 2018 at 2:10 PM, Mohamed El-Serngawy <
>> [email protected]> wrote:
>>
>>> Hi Tim,
>>>
>>> This most properly mean that the ovs certificate is not in the trust
>>> keystore. Just need to clarify things, based on the your previous email
>>> you mentioned
>>> "We create a JKS for the controller keystore. For the trust store I
>>> believe ODL creates it on boot"
>>>
>>> Is this mean you create the ctl.jks file place it under /configuration/ssl/
>>> ? if yes, so I guess the issue at [0]. The certificate manager at the first
>>> time start check if the keystores files are there, If not create them. Let
>>> me know if my understand is correct ? You may do one more thing to confirm,
>>> just create an empty trust keystore file with the respect to the config at
>>> aaa-cert-config.xml and check if it will gonna work without restart.
>>>
>>>
>>> [0] https://github.com/opendaylight/aaa/blob/master/aaa-cert
>>> /src/main/java/org/opendaylight/aaa/cert/impl/CertificateMan
>>> agerService.java#L70
>>>
>>>
>>> Thanks
>>>
>>> On Tue, Mar 20, 2018 at 12:19 PM, Tim Rozet <[email protected]> wrote:
>>>
>>>> Hi Mohamed,
>>>> I managed to reproduce the issue. I flipped on debugging and I see:
>>>>
>>>> 2018-03-20 11:07:09,492 | INFO | entLoopGroup-4-1 | LoggingHandler
>>>> | 60 - io.netty.common - 4.1.16.Final | [id: 0xfe0d155e, L:/
>>>> 0.0.0.0:6640] READ COMPLETE
>>>> 2018-03-20 11:07:09,492 | DEBUG | entLoopGroup-5-4 |
>>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>>> 1.6.0.SNAPSHOT | New Passive channel created : [id: 0xae5053bf, L:/
>>>> 192.0.2.8:6640 - R:/192.0.2.8:51224]
>>>> 2018-03-20 11:07:09,593 | DEBUG | assiveConnServ-5 |
>>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>>> 1.6.0.SNAPSHOT | Handshake status NEED_UNWRAP
>>>> 2018-03-20 11:07:09,593 | DEBUG | assiveConnServ-5 |
>>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>>> 1.6.0.SNAPSHOT | handshake not done yet NEED_UNWRAP
>>>> 2018-03-20 11:07:09,693 | DEBUG | assiveConnServ-5 |
>>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>>> 1.6.0.SNAPSHOT | Handshake status NEED_UNWRAP
>>>> 2018-03-20 11:07:09,693 | DEBUG | assiveConnServ-5 |
>>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>>> 1.6.0.SNAPSHOT | handshake not done yet NEED_UNWRAP
>>>> 2018-03-20 11:07:09,793 | DEBUG | assiveConnServ-5 |
>>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>>> 1.6.0.SNAPSHOT | Handshake status NEED_UNWRAP
>>>> 2018-03-20 11:07:09,794 | DEBUG | assiveConnServ-5 |
>>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>>> 1.6.0.SNAPSHOT | handshake not done yet NEED_UNWRAP
>>>> 2018-03-20 11:07:09,894 | DEBUG | assiveConnServ-5 |
>>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>>> 1.6.0.SNAPSHOT | Handshake status NEED_UNWRAP
>>>> 2018-03-20 11:07:09,894 | DEBUG | assiveConnServ-5 |
>>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>>> 1.6.0.SNAPSHOT | handshake not done yet NEED_UNWRAP
>>>> 2018-03-20 11:07:09,894 | DEBUG | assiveConnServ-5 |
>>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>>> 1.6.0.SNAPSHOT | channel closed [id: 0xae5053bf, L:/192.0.2.8:6640 !
>>>> R:/192.0.2.8:51224]
>>>>
>>>> Which looks like it is coming from here:
>>>> https://github.com/opendaylight/ovsdb/blob/e6b469e18d5f72402
>>>> ccb817ce1fb1469dd2a9d6c/library/impl/src/main/java/org/opend
>>>> aylight/ovsdb/lib/impl/OvsdbConnectionService.java#L439
>>>>
>>>> Tim Rozet
>>>> Red Hat SDN Team
>>>>
>>>> On Mon, Mar 19, 2018 at 11:53 AM, Mohamed El-Serngawy <
>>>> [email protected]> wrote:
>>>>
>>>>> Hi Sam
>>>>>
>>>>> I'm not quit sure what could be causing this issue, suppose there is
>>>>> no need to restart. "getServerContext()" will be executed every
>>>>> time a connection established. Will try ti reproduce it today.
>>>>>
>>>>> thanks
>>>>>
>>>>>
>>>>> On Sun, Mar 18, 2018 at 11:01 AM, Sam Hague <[email protected]> wrote:
>>>>>
>>>>>> Mohamed,
>>>>>>
>>>>>> Tim is using the JKS - he pushes all that before connecting the OVS
>>>>>> nodes to ODL.
>>>>>>
>>>>>> Do you know if there are any timing with the JKS when ODL starts
>>>>>> compared to when certs are added via rest? ovsdb southbound stats up and
>>>>>> has the certificateManager
>>>>>> which it uses to start the netty listening on 6640. Then client certs
>>>>>> are included to the ODL via rest. Then connections attempted from the ovs
>>>>>> nodes but they never connect. Reboot ODL and the connections then work.
>>>>>>
>>>>>> Could there be something in the reboot which actally gets the client
>>>>>> certs applied?
>>>>>>
>>>>>> Or does the server context change when cert are applied? At startup
>>>>>> the ovsdb southbound does certManagerSrv.getServerContext() and
>>>>>> opens the listening channel. That same context is used when the incoming
>>>>>> connections come in - ovsdb does not do another read of that context.
>>>>>>
>>>>>> Thanks, Sam
>>>>>>
>>>>>> On Thu, Mar 15, 2018 at 3:03 PM, Tim Rozet <[email protected]> wrote:
>>>>>>
>>>>>>> Hi Mohamed,
>>>>>>> Right, that is one of the wiki pages I followed. There are several
>>>>>>> that I kind of had to merge the info together to get it to all work.
>>>>>>> The
>>>>>>> read from the trust store should work. I tested it manually and we
>>>>>>> have an
>>>>>>> unless here in puppet so we do not re-add the cert:
>>>>>>> https://github.com/openstack/puppet-neutron/blob/master/mani
>>>>>>> fests/plugins/ovs/opendaylight.pp#L191
>>>>>>>
>>>>>>> We create a JKS for the controller keystore. For the trust store I
>>>>>>> believe ODL creates it on boot based on this config:
>>>>>>> https://git.opendaylight.org/gerrit/gitweb?p=integration/pac
>>>>>>> kaging/puppet-opendaylight.git;a=blob;f=templates/aaa-cert-c
>>>>>>> onfig.xml.erb;h=d6faa891630cba1c4747f64ea977d07de08c6b65;hb=
>>>>>>> refs/heads/master
>>>>>>>
>>>>>>>
>>>>>>> Tim Rozet
>>>>>>> Red Hat SDN Team
>>>>>>>
>>>>>>> On Thu, Mar 15, 2018 at 2:41 PM, Mohamed El-Serngawy <
>>>>>>> [email protected]> wrote:
>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> The logs attached with the bug is not really showing errors, Just
>>>>>>>> the aaa-cert service waiting for aaa-encryption service then it starts
>>>>>>>> fine.
>>>>>>>>
>>>>>>>> Tim,
>>>>>>>>
>>>>>>>> I assume you followed the link at [0] to configure the ssl. After
>>>>>>>> you add the OVS certificate using the REST API, can you just confirm
>>>>>>>> that
>>>>>>>> you are able to read the certificate from the trust-store ? are you
>>>>>>>> using
>>>>>>>> MDSAL or java Key Store files ?
>>>>>>>>
>>>>>>>> [0] https://wiki.opendaylight.org/view/OVSDB_Integration:TLS
>>>>>>>> _Communication
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Thu, Mar 15, 2018 at 2:27 PM, Luis Gomez <[email protected]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> I do not remember that issue when I tested OF TLS in the past, I
>>>>>>>>> will have to retest to confirm.
>>>>>>>>>
>>>>>>>>> On Mar 15, 2018, at 11:24 AM, Tim Rozet <[email protected]> wrote:
>>>>>>>>>
>>>>>>>>> Hi Luis,
>>>>>>>>> To clarify we are not talking about SSL configuration here. We
>>>>>>>>> indeed configure the file you mentioned along with other config files
>>>>>>>>> pax
>>>>>>>>> web, ovsdb to only allow SSL/TLS, creating controller and trust
>>>>>>>>> stores.
>>>>>>>>> This is all done prior to ODL starting. The failure here is that ODL
>>>>>>>>> allows a REST implementation to add certificates to the trust store
>>>>>>>>> for OVS
>>>>>>>>> switches (which obviously implies ODL is up and running). At deploy
>>>>>>>>> time,
>>>>>>>>> we generate certificates for OVS and then add them via REST to ODL.
>>>>>>>>> At
>>>>>>>>> that point ODL should trust the switch and allow connections.
>>>>>>>>> However,
>>>>>>>>> OVSDB never seems to read again from the trust store (unless
>>>>>>>>> rebooted) and
>>>>>>>>> does not allow the switch to connect.
>>>>>>>>>
>>>>>>>>> Tim Rozet
>>>>>>>>> Red Hat SDN Team
>>>>>>>>>
>>>>>>>>> On Thu, Mar 15, 2018 at 1:55 PM, Luis Gomez <[email protected]>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> AFAIR for ofp you need to modify this config file:
>>>>>>>>>>
>>>>>>>>>> /etc/opendaylight/datastore/initial/config/default-openflow-
>>>>>>>>>> connection-config.xml
>>>>>>>>>>
>>>>>>>>>> which means you have to reboot the controller after.
>>>>>>>>>>
>>>>>>>>>> BR/Luis
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Mar 15, 2018, at 10:42 AM, Sam Hague <[email protected]>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> Mo, and ofp devs,
>>>>>>>>>>
>>>>>>>>>> how do you handle openflow connections using ssl? We have the bug
>>>>>>>>>> below where ODL is required to be restarted to pick up connections
>>>>>>>>>> over ssl.
>>>>>>>>>>
>>>>>>>>>> Is that a design requirement that ODL has to be restarted or is
>>>>>>>>>> there a different config that can be used?
>>>>>>>>>>
>>>>>>>>>> Thanks, Sam
>>>>>>>>>>
>>>>>>>>>> https://jira.opendaylight.org/browse/OVSDB-449
>>>>>>>>>> _______________________________________________
>>>>>>>>>> integration-dev mailing list
>>>>>>>>>> [email protected]
>>>>>>>>>> https://lists.opendaylight.org/mailman/listinfo/integration-dev
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Mohamed ElSerngawy
>>>>>>>>
>>>>>>>> +1 438 993 2462 <(438)%20993-2462>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Mohamed ElSerngawy
>>>>>
>>>>> +1 438 993 2462 <(438)%20993-2462>
>>>>>
>>>>
>>>>
>>>
>>>
>>> --
>>> Mohamed ElSerngawy
>>>
>>> +1 438 993 2462 <(438)%20993-2462>
>>>
>>
>>
>>
>> --
>> Mohamed ElSerngawy
>>
>> +1 438 993 2462 <(438)%20993-2462>
>>
>
>
_______________________________________________
openflowplugin-dev mailing list
[email protected]
https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
