Hi Mohamed,
Just to clarify...we do create the ctl.jks and place it in
configuration/ssl before ODL boots. The truststore is created on first
boot, I can see truststore.jks in the same directory. I also believe that
the certificate for OVS is in the truststore:
[root@overcloud-controller-0 ~]# curl -k -X POST --fail --silent -H
'Content-Type: application/json' -H 'Cache-Control: no-cache' -u
admin:admin -d '{ "aaa-cert-rpc:input": { "aaa-cert-rpc:node-alias":
"overcloud-controller-0" }}'
https://192.0.2.10:8081/restconf/operations/aaa-cert-rpc:getNodeCertifcate
{"output":{"node-cert":"MIIFGjCCBAKgAwIBAgICASswDQYJKoZIhvcNAQELBQAwNjEUMBIGA1UECgwLT1BORlZMRi5PUkcxHjAcBgNVBAMMFUNlcnRpZmljYXRlIEF1dGhvcml0eTAeFw0xODAzMjAwOTM0MTFaFw0yMDAzMjAwOTM0MTFaME8xFDASBgNVBAoMC09QTkZWTEYuT1JHMTcwNQYDVQQDDC5vdmVyY2xvdWQtY29udHJvbGxlci0wLmludGVybmFsYXBpLm9wbmZ2bGYub3JnMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAoLDetkUZscbmllZNKc9u1B9eDNW4dptIHn8PpUNG6cHnwLhmuynRDwgI0zYPaoZnFy6QzyeCtpVARgqpUDGB0SKyHPoO38YYwXuEUfK27mpfCFsZ0vKwmt4827fqglUQXMNlypyaPKdrcx1w0z++E8x5DUixgL+trjjWtDBlooFkGs/qVoWFgySUvw/+CZ/mfg3G3W7oVX1k3zqf75wxsqzM7ZePGn7W0UPuPjNkkOOI9Hl8PFoLeLwZXuYaAaJmz3O6UffvRZgl3aSgPuCK31fM5tIfeCNnlQMxMdZZiPrbRpAi9O5vpaXxM/d6r7HkTI6N9l0g/hX7P53hearImQIDAQABo4ICFzCCAhMwHwYDVR0jBBgwFoAUEGBKlCQqJ05za3ZwFRfuCW0ae0owPQYIKwYBBQUHAQEEMTAvMC0GCCsGAQUFBzABhiFodHRwOi8vaXBhLWNhLm9wbmZ2bGYub3JnL2NhL29jc3AwDgYDVR0PAQH/BAQDAgTwMB0GA1UdJQQWMBQGCCsGAQUFBwMBBggrBgEFBQcDAjB2BgNVHR8EbzBtMGugM6Axhi9odHRwOi8vaXBhLWNhLm9wbmZ2bGYub3JnL2lwYS9jcmwvTWFzdGVyQ1JMLmJpbqI0pDIwMDEOMAwGA1UECgwFaXBhY2ExHjAcBgNVBAMMFUNlcnRpZmljYXRlIEF1dGhvcml0eTAdBgNVHQ4EFgQUcySBEFDXvFw45hSfRhLgTlpxcrowgeoGA1UdEQSB4jCB34Iub3ZlcmNsb3VkLWNvbnRyb2xsZXItMC5pbnRlcm5hbGFwaS5vcG5mdmxmLm9yZ6BOBgorBgEEAYI3FAIDoEAMPm92cy9vdmVyY2xvdWQtY29udHJvbGxlci0wLmludGVybmFsYXBpLm9wbmZ2bGYub3JnQE9QTkZWTEYuT1JHoF0GBisGAQUCAqBTMFGgDRsLT1BORlZMRi5PUkehQDA+oAMCAQGhNzA1GwNvdnMbLm92ZXJjbG91ZC1jb250cm9sbGVyLTAuaW50ZXJuYWxhcGkub3BuZnZsZi5vcmcwDQYJKoZIhvcNAQELBQADggEBAF+7W//ICAw0i+FxPr9hSG2rlWcauo0Zllk2ff5e9NGbf7uu7aJVINYRVpGW7IN0kgsq63QuCGZu7y/zCiPb5hmWuBGHKgqHFXTjLUXGmSQXJ5yEuE7yLTmN5oHbXxJgbXgJoy7PDOLziAnWRQkrScBvLNZur7+06FurN2UPtCMfrnn8cmbiteLBW6sdnzCFEUlrkU6bHOFPJWwAGski6YpdHYKcnw4p3J22phABCSRe8dq7xbIZx0wBg6WaSGTlwshqUSFEDX4VrjtVoWE06ipTSbHR3Ra93kh1dEd3VdMPND/CdflsIOcsiykD7SZl7oli9c35asqjipOAvI8HCn8="}}
Let me know if there are any other debug outputs that could be helpful.
Thanks,
Tim Rozet
Red Hat SDN Team
On Tue, Mar 20, 2018 at 2:17 PM, Mohamed El-Serngawy <[email protected]
> wrote:
> Sorry Tim, just confused ignore my previous email the keystore will be
> created anyway [1]. "didn't maintain this code for a long time"
>
> [1] https://github.com/opendaylight/aaa/blob/master/
> aaa-cert/src/main/java/org/opendaylight/aaa/cert/impl/
> AaaCertProvider.java#L89
>
> On Tue, Mar 20, 2018 at 2:10 PM, Mohamed El-Serngawy <
> [email protected]> wrote:
>
>> Hi Tim,
>>
>> This most properly mean that the ovs certificate is not in the trust
>> keystore. Just need to clarify things, based on the your previous email
>> you mentioned
>> "We create a JKS for the controller keystore. For the trust store I
>> believe ODL creates it on boot"
>>
>> Is this mean you create the ctl.jks file place it under /configuration/ssl/
>> ? if yes, so I guess the issue at [0]. The certificate manager at the first
>> time start check if the keystores files are there, If not create them. Let
>> me know if my understand is correct ? You may do one more thing to confirm,
>> just create an empty trust keystore file with the respect to the config at
>> aaa-cert-config.xml and check if it will gonna work without restart.
>>
>>
>> [0] https://github.com/opendaylight/aaa/blob/master/aaa-
>> cert/src/main/java/org/opendaylight/aaa/cert/impl/Certificat
>> eManagerService.java#L70
>>
>>
>> Thanks
>>
>> On Tue, Mar 20, 2018 at 12:19 PM, Tim Rozet <[email protected]> wrote:
>>
>>> Hi Mohamed,
>>> I managed to reproduce the issue. I flipped on debugging and I see:
>>>
>>> 2018-03-20 11:07:09,492 | INFO | entLoopGroup-4-1 | LoggingHandler
>>> | 60 - io.netty.common - 4.1.16.Final | [id: 0xfe0d155e, L:/
>>> 0.0.0.0:6640] READ COMPLETE
>>> 2018-03-20 11:07:09,492 | DEBUG | entLoopGroup-5-4 |
>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>> 1.6.0.SNAPSHOT | New Passive channel created : [id: 0xae5053bf, L:/
>>> 192.0.2.8:6640 - R:/192.0.2.8:51224]
>>> 2018-03-20 11:07:09,593 | DEBUG | assiveConnServ-5 |
>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>> 1.6.0.SNAPSHOT | Handshake status NEED_UNWRAP
>>> 2018-03-20 11:07:09,593 | DEBUG | assiveConnServ-5 |
>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>> 1.6.0.SNAPSHOT | handshake not done yet NEED_UNWRAP
>>> 2018-03-20 11:07:09,693 | DEBUG | assiveConnServ-5 |
>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>> 1.6.0.SNAPSHOT | Handshake status NEED_UNWRAP
>>> 2018-03-20 11:07:09,693 | DEBUG | assiveConnServ-5 |
>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>> 1.6.0.SNAPSHOT | handshake not done yet NEED_UNWRAP
>>> 2018-03-20 11:07:09,793 | DEBUG | assiveConnServ-5 |
>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>> 1.6.0.SNAPSHOT | Handshake status NEED_UNWRAP
>>> 2018-03-20 11:07:09,794 | DEBUG | assiveConnServ-5 |
>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>> 1.6.0.SNAPSHOT | handshake not done yet NEED_UNWRAP
>>> 2018-03-20 11:07:09,894 | DEBUG | assiveConnServ-5 |
>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>> 1.6.0.SNAPSHOT | Handshake status NEED_UNWRAP
>>> 2018-03-20 11:07:09,894 | DEBUG | assiveConnServ-5 |
>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>> 1.6.0.SNAPSHOT | handshake not done yet NEED_UNWRAP
>>> 2018-03-20 11:07:09,894 | DEBUG | assiveConnServ-5 |
>>> OvsdbConnectionService | 399 - org.opendaylight.ovsdb.library -
>>> 1.6.0.SNAPSHOT | channel closed [id: 0xae5053bf, L:/192.0.2.8:6640 ! R:/
>>> 192.0.2.8:51224]
>>>
>>> Which looks like it is coming from here:
>>> https://github.com/opendaylight/ovsdb/blob/e6b469e18d5f72402
>>> ccb817ce1fb1469dd2a9d6c/library/impl/src/main/java/org/
>>> opendaylight/ovsdb/lib/impl/OvsdbConnectionService.java#L439
>>>
>>> Tim Rozet
>>> Red Hat SDN Team
>>>
>>> On Mon, Mar 19, 2018 at 11:53 AM, Mohamed El-Serngawy <
>>> [email protected]> wrote:
>>>
>>>> Hi Sam
>>>>
>>>> I'm not quit sure what could be causing this issue, suppose there is no
>>>> need to restart. "getServerContext()" will be executed every time a
>>>> connection established. Will try ti reproduce it today.
>>>>
>>>> thanks
>>>>
>>>>
>>>> On Sun, Mar 18, 2018 at 11:01 AM, Sam Hague <[email protected]> wrote:
>>>>
>>>>> Mohamed,
>>>>>
>>>>> Tim is using the JKS - he pushes all that before connecting the OVS
>>>>> nodes to ODL.
>>>>>
>>>>> Do you know if there are any timing with the JKS when ODL starts
>>>>> compared to when certs are added via rest? ovsdb southbound stats up and
>>>>> has the certificateManager
>>>>> which it uses to start the netty listening on 6640. Then client certs
>>>>> are included to the ODL via rest. Then connections attempted from the ovs
>>>>> nodes but they never connect. Reboot ODL and the connections then work.
>>>>>
>>>>> Could there be something in the reboot which actally gets the client
>>>>> certs applied?
>>>>>
>>>>> Or does the server context change when cert are applied? At startup
>>>>> the ovsdb southbound does certManagerSrv.getServerContext() and opens
>>>>> the listening channel. That same context is used when the incoming
>>>>> connections come in - ovsdb does not do another read of that context.
>>>>>
>>>>> Thanks, Sam
>>>>>
>>>>> On Thu, Mar 15, 2018 at 3:03 PM, Tim Rozet <[email protected]> wrote:
>>>>>
>>>>>> Hi Mohamed,
>>>>>> Right, that is one of the wiki pages I followed. There are several
>>>>>> that I kind of had to merge the info together to get it to all work. The
>>>>>> read from the trust store should work. I tested it manually and we have
>>>>>> an
>>>>>> unless here in puppet so we do not re-add the cert:
>>>>>> https://github.com/openstack/puppet-neutron/blob/master/mani
>>>>>> fests/plugins/ovs/opendaylight.pp#L191
>>>>>>
>>>>>> We create a JKS for the controller keystore. For the trust store I
>>>>>> believe ODL creates it on boot based on this config:
>>>>>> https://git.opendaylight.org/gerrit/gitweb?p=integration/pac
>>>>>> kaging/puppet-opendaylight.git;a=blob;f=templates/aaa-cert-c
>>>>>> onfig.xml.erb;h=d6faa891630cba1c4747f64ea977d07de08c6b65;hb=
>>>>>> refs/heads/master
>>>>>>
>>>>>>
>>>>>> Tim Rozet
>>>>>> Red Hat SDN Team
>>>>>>
>>>>>> On Thu, Mar 15, 2018 at 2:41 PM, Mohamed El-Serngawy <
>>>>>> [email protected]> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> The logs attached with the bug is not really showing errors, Just
>>>>>>> the aaa-cert service waiting for aaa-encryption service then it starts
>>>>>>> fine.
>>>>>>>
>>>>>>> Tim,
>>>>>>>
>>>>>>> I assume you followed the link at [0] to configure the ssl. After
>>>>>>> you add the OVS certificate using the REST API, can you just confirm
>>>>>>> that
>>>>>>> you are able to read the certificate from the trust-store ? are you
>>>>>>> using
>>>>>>> MDSAL or java Key Store files ?
>>>>>>>
>>>>>>> [0] https://wiki.opendaylight.org/view/OVSDB_Integration:TLS
>>>>>>> _Communication
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Thu, Mar 15, 2018 at 2:27 PM, Luis Gomez <[email protected]>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> I do not remember that issue when I tested OF TLS in the past, I
>>>>>>>> will have to retest to confirm.
>>>>>>>>
>>>>>>>> On Mar 15, 2018, at 11:24 AM, Tim Rozet <[email protected]> wrote:
>>>>>>>>
>>>>>>>> Hi Luis,
>>>>>>>> To clarify we are not talking about SSL configuration here. We
>>>>>>>> indeed configure the file you mentioned along with other config files
>>>>>>>> pax
>>>>>>>> web, ovsdb to only allow SSL/TLS, creating controller and trust stores.
>>>>>>>> This is all done prior to ODL starting. The failure here is that ODL
>>>>>>>> allows a REST implementation to add certificates to the trust store
>>>>>>>> for OVS
>>>>>>>> switches (which obviously implies ODL is up and running). At deploy
>>>>>>>> time,
>>>>>>>> we generate certificates for OVS and then add them via REST to ODL. At
>>>>>>>> that point ODL should trust the switch and allow connections. However,
>>>>>>>> OVSDB never seems to read again from the trust store (unless rebooted)
>>>>>>>> and
>>>>>>>> does not allow the switch to connect.
>>>>>>>>
>>>>>>>> Tim Rozet
>>>>>>>> Red Hat SDN Team
>>>>>>>>
>>>>>>>> On Thu, Mar 15, 2018 at 1:55 PM, Luis Gomez <[email protected]>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> AFAIR for ofp you need to modify this config file:
>>>>>>>>>
>>>>>>>>> /etc/opendaylight/datastore/initial/config/default-openflow-
>>>>>>>>> connection-config.xml
>>>>>>>>>
>>>>>>>>> which means you have to reboot the controller after.
>>>>>>>>>
>>>>>>>>> BR/Luis
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Mar 15, 2018, at 10:42 AM, Sam Hague <[email protected]> wrote:
>>>>>>>>>
>>>>>>>>> Mo, and ofp devs,
>>>>>>>>>
>>>>>>>>> how do you handle openflow connections using ssl? We have the bug
>>>>>>>>> below where ODL is required to be restarted to pick up connections
>>>>>>>>> over ssl.
>>>>>>>>>
>>>>>>>>> Is that a design requirement that ODL has to be restarted or is
>>>>>>>>> there a different config that can be used?
>>>>>>>>>
>>>>>>>>> Thanks, Sam
>>>>>>>>>
>>>>>>>>> https://jira.opendaylight.org/browse/OVSDB-449
>>>>>>>>> _______________________________________________
>>>>>>>>> integration-dev mailing list
>>>>>>>>> [email protected]
>>>>>>>>> https://lists.opendaylight.org/mailman/listinfo/integration-dev
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Mohamed ElSerngawy
>>>>>>>
>>>>>>> +1 438 993 2462 <(438)%20993-2462>
>>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Mohamed ElSerngawy
>>>>
>>>> +1 438 993 2462 <(438)%20993-2462>
>>>>
>>>
>>>
>>
>>
>> --
>> Mohamed ElSerngawy
>>
>> +1 438 993 2462 <(438)%20993-2462>
>>
>
>
>
> --
> Mohamed ElSerngawy
>
> +1 438 993 2462 <(438)%20993-2462>
>
_______________________________________________
openflowplugin-dev mailing list
[email protected]
https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
