[GitHub] rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership
rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership URL: https://github.com/apache/cloudstack/issues/2930#issuecomment-445014904 @DennisKonrad 1. Not virtual routers, only ssvm, cpvm and kvm hosts 2. Yes, the APi accepts hostid and reconnect params 3. N/A virtual routers don't need to be touched, VRs don't run any java agent This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership
rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership URL: https://github.com/apache/cloudstack/issues/2930#issuecomment-438518018 @DennisKonrad can you check in the logs if your kvm hosts are trying connect at all? You can try to delete the old keystore file at /etc/cloudstack/agent/cloud.jks and restart agent, then try the provision API. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership
rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership URL: https://github.com/apache/cloudstack/issues/2930#issuecomment-437840211 @DennisKonrad can you at least stop all your mgmt servers and start one by one? It's possible that you had a conflict b/w root ca cert/priv/pub keys, you can temporarily disable auth strictness, then remove the global settings and use provisionCertificate API to re-provision certificate with a new keypair/cert. You're currently using a messed up (sounds like) unstable/master branch and we cannot help once you fix your env. Also, please re-read my comments I'm not going to restate the same things again and again. About IPs, mgmt server when it starts discovers the IPs it needs to use to create a self-signed cert, and NO you don't need to delete the certs/keypair every time you add a new mgmt server. I advised that because you may have a case where you started multiple mgmt server during time of install/setup at once which stepped on each other and wrote incorrect ca keypair/cert in db (i.e. conflict+concurrent issue). This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership
rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership URL: https://github.com/apache/cloudstack/issues/2930#issuecomment-434166435 Given this is a setup /env issue, if the advise process fixes your issue and you're unable to reproduce the errors please close this issue @DennisKonrad This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership
rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership URL: https://github.com/apache/cloudstack/issues/2930#issuecomment-433872021 @DennisKonrad I've already replied to you how cert generation works, tl;dr - each mgmt server generates its own cert on startup based on keypair/ca-cert from the db, the alt names are obtained by mgmt server by reading ips on network interfaces. The certs of mgmt server are only used when they peer/cluster with other mgmt servers, if you re-generated core keypair/ca-cert then kvm hosts certs will need to be re-provisioned. Please go figure: https://github.com/apache/cloudstack/blob/master/plugins/ca/root-ca/src/main/java/org/apache/cloudstack/ca/provider/RootCAProvider.java#L409 This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership
rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership URL: https://github.com/apache/cloudstack/issues/2930#issuecomment-433854973 @DennisKonrad thanks. When a new management server starts, it first upgrades the DB and then various managers/components start. During this start stage, first the `ConfigurationManagerImpl` starts which configures various default settings, offerings, accounts (such as system, admin etc). When the CA manager starts, it asks the configured/default plugin to initialize which is the RootCA plugin by default. This plugin would check db is keypair (private/public keypair) exists and uses the CA cert to create and sign a self-signed cert for the mgmt server host. If during the initializing process, another mgmt server is started it might screw up big time conflicting for operations/default created by both ConfigurationManager and CAManager. Therefore, based on the output you've shared it's more of an env/setup issue than a bug. Please re-test master again, this time making sure that the first/primary mgmt server initializes/completes before secondary mgmt servers are added. For an existing env, to force re-kick of cert generation, shutdown all mgmt servers, then in the db and set these global settings in cloud.configurations table to null: - ca.plugin.root.private.key - ca.plugin.root.public.key - ca.plugin.root.ca.certificate And start first mgmt server, let it complete initialization and start other mgmt servers. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership
rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership URL: https://github.com/apache/cloudstack/issues/2930#issuecomment-433784424 @DennisKonrad did you deploy multiple management servers concurrently? Ideally you should wait for the first management server to fully start before starting secondary management server. From the screenshot tthe certificate was generated without IPs of the mgmt server, therefore the certificate validation logic failed the SSL connection (as the certificate's alt name/ip should match the connecting agent/mgmt server's address). For example the following is a valid mgmt server cert that has ipv4/v6 address in its alt name: ``` Certificate [1] : Serial: da32e26467ff7a4d Not Before:Sat Oct 27 07:44:03 UTC 2018 Not After:Mon Oct 19 19:44:03 UTC 2048 Signature Algorithm:SHA256withRSA Version:3 Subject DN:CN=pr2376-t3127-kvm-centos7-mgmt2 Issuer DN:CN=ca.cloudstack.apache.org Alternative Names:[[7, fe80:0:0:0:4af:4ff:fe01:7a8], [7, 10.2.2.176], [2, pr2376-t3127-kvm-centos7-mgmt2]] ``` I could not reproduce this with 4.11 branch, so will move to milestone 4.12.0.0/master. Please re-test and keep us posted, thanks. This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership
rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership URL: https://github.com/apache/cloudstack/issues/2930#issuecomment-433493484 @DennisKonrad can you also describe your setup, is it kvm/xenserver/vmware etc? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
[GitHub] rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership
rhtyd commented on issue #2930: HA for Management Server - roundrobin: Certificate ownership URL: https://github.com/apache/cloudstack/issues/2930#issuecomment-433492999 @DennisKonrad are you also able to reproduce this for 4.11? This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
