Please, forget my previous mail... The problem was Tomcat7...
I'm going on cert issue... Nicolas Bouige DIMSI cloud.dimsi.fr<http://www.cloud.dimsi.fr> 4, avenue Laurent Cely Tour d’Asnière – 92600 Asnière sur Seine T/ +33 (0)6 28 98 53 40 ________________________________ De : Nicolas Bouige <n.bou...@dimsi.fr> Envoyé : vendredi 4 mai 2018 15:33:43 À : users@cloudstack.apache.org Objet : RE: certificate issue second mgmt-server Hi All, So, i was working on my primary issue with the certificate on my second node...i've stopped and restart the first node and now the cloudstack-management is in status failed :'( I got an "invalid specified classpath" and "Cannot parse command line arguments". I modified the file cloudstack-management.service" to manually set up these varirable $JAR:$CLASSPATH. After a deamon-reload and a restart of the service, it's run around 30 sec and fail again, here the log from the status service: mai 04 20:43:24 ASPRCSMGMT01 systemd[1]: Starting CloudStack Management Server... mai 04 20:43:24 ASPRCSMGMT01 systemd[1]: Started CloudStack Management Server. mai 04 20:44:00 ASPRCSMGMT01 sudo[5033]: cloud : TTY=unknown ; PWD=/var/log/cloudstack/management ; USER=root ; COMMAND=/bin/mkdir -p /systemvm_mnt mai 04 20:44:00 ASPRCSMGMT01 sudo[5036]: cloud : TTY=unknown ; PWD=/var/log/cloudstack/management ; USER=root ; COMMAND=/bin/mount -o loop /usr/share/cloudstack-common/vms/systemvm.iso /systemvm_mnt mai 04 20:44:00 ASPRCSMGMT01 sudo[5041]: cloud : TTY=unknown ; PWD=/var/log/cloudstack/management ; USER=root ; COMMAND=/bin/umount /systemvm_mnt mai 04 20:44:15 ASPRCSMGMT01 systemd[1]: cloudstack-management.service: main process exited, code=exited, status=1/FAILURE mai 04 20:44:15 ASPRCSMGMT01 systemd[1]: cloudstack-management.service: control process exited, code=exited status=255 mai 04 20:44:15 ASPRCSMGMT01 systemd[1]: Unit cloudstack-management.service entered failed state. mai 04 20:44:15 ASPRCSMGMT01 systemd[1]: cloudstack-management.service failed. >From /var/log/messages : May 4 20:43:28 ASPRCSMGMT01 systemd: Configuration file /usr/lib/systemd/system/cloudstack-management.service is marked executable. Please remove executable permission bits. Proceeding anyway. May 4 20:44:15 ASPRCSMGMT01 systemd: cloudstack-management.service: main process exited, code=exited, status=1/FAILURE May 4 20:44:15 ASPRCSMGMT01 systemd: cloudstack-management.service: control process exited, code=exited status=255 May 4 20:44:15 ASPRCSMGMT01 systemd: Unit cloudstack-management.service entered failed state. This management was working fine for one month and i restarted it without any problem before... If one of you have any idea, it would be appreciated, Thanks upfront for any help, Best regards, Nicolas Bouige DIMSI cloud.dimsi.fr<http://www.cloud.dimsi.fr> 4, avenue Laurent Cely Tour d’Asnière – 92600 Asnière sur Seine T/ +33 (0)6 28 98 53 40 ________________________________ De : Nicolas Bouige <n.bou...@dimsi.fr> Envoyé : mercredi 2 mai 2018 10:54:36 À : users@cloudstack.apache.org Objet : RE: certificate issue second mgmt-server Rohit, Thanks for the details, i'll keep you update if its work. Best regards, N.B -----Message d'origine----- De : Rohit Yadav [mailto:rohit.ya...@shapeblue.com] Envoyé : mercredi 2 mai 2018 10:39 À : users@cloudstack.apache.org Objet : Re: certificate issue second mgmt-server Nicolas, Yes, if you've existing systemvms and KVM hosts changing the ca private/public key could cause system-wide cert issue. You can retry shutting down both management server(s), start the primary mgmt server to come up first and then start/deploy other mgmt servers one by one. - Rohit <https://cloudstack.apache.org> ________________________________ From: Nicolas Bouige <n.bou...@dimsi.fr> Sent: Wednesday, May 2, 2018 1:57:07 PM To: users@cloudstack.apache.org Subject: RE: certificate issue second mgmt-server Hi Rohit, Thanks for your answer, i can't remember if i added the second node before the end of the initialization maybe i was too impatient :/ I'll give a try this week with your workaround. Your workaround will affect also KVM server and System-VM ? So, i guess, it's not enought to delete the second node and redeploy it ? Best regards, N.B rohit.ya...@shapeblue.com www.shapeblue.com<http://www.shapeblue.com> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue -----Message d'origine----- De : Rohit Yadav [mailto:rohit.ya...@shapeblue.com] Envoyé : mardi 1 mai 2018 12:56 À : users@cloudstack.apache.org Objet : Re: certificate issue second mgmt-server Hi Nicolas, Did you deploy multiple managements at the same time? When you deploy multiple management server(s), wait for the first management server to initialize database where it sets up some default offerings, global settings and the root CA keypair and certificate. Only when you see the first management server's UI in browser, proceed with deployment of other management server(s). For your environment, you can test this workaround and let me know if that works for you: 1. Shutdown all the management server(s). 2. Delete ca keypair and cert: delete from configuration where name like "ca.plugin.root.private.key"; delete from configuration where name like "ca.plugin.root.public.key"; delete from configuration where name="ca.plugin.root.ca.certificate"; 3. Start one management server and wait for it to complete internal setup, until you see the UI. 4. Start all the other management server(s). - Rohit <https://cloudstack.apache.org> ________________________________ From: Nicolas Bouige <n.bou...@dimsi.fr> Sent: Monday, April 30, 2018 2:59:29 PM To: users@cloudstack.apache.org Subject: certificate issue second mgmt-server Hello All, I have an issue with one of my Cloudstack mgmt-server (4.11) The second node has been deployed with the command "cloudstack-setup-databases cloud:dbpassword@dbhost" i didnt have any problem during few days and now sometimes i got an error on web GUI when i perfom some basic task, the error is "Resource [Host:1] is unreachable: Host 1: Unable to reach the peer that the agent is connected" After a quick investigation, i had to stop cloudstack-management service from second mgmt-server and i noticed a lot of messages related with ca-certificate used by cloudstack : 2018-04-27 11:18:24,076 ERROR [c.c.u.n.Link] (StatsCollector-1:ctx-82335701) (logid:95fda6d7) SSL error caught during unwrap data: Received fatal alert: certificate_unknown, for local address=/172.16.22.61:60128, remote address=/172.16.22.60:8250. The client may have invalid ca-certificates. 2018-04-27 11:18:24,076 WARN [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-1:ctx-82335701) (logid:95fda6d7) Unable to connect to peer management server: 130719784044197, ip: 172.16.22.60 due to SSL: Fail to init SSL! java.io.IOException: SSL: Handshake failed with peer management server '130719784044197' on 172.16.22.60:8250 java.io.IOException: SSL: Fail to init SSL! java.io.IOException: SSL: Handshake failed with peer management server '130719784044197' on 172.16.22.60:8250 at com.cloud.agent.manager.ClusteredAgentManagerImpl.connectToPeer(ClusteredAgentManagerImpl.java:529) at com.cloud.agent.manager.ClusteredAgentAttache.send(ClusteredAgentAttache.java:177) at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:398) at com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:456) at com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:362) at com.cloud.agent.manager.AgentManagerImpl.easySend(AgentManagerImpl.java:954) at com.cloud.resource.ResourceManagerImpl.getHostStatistics(ResourceManagerImpl.java:2645) at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338) at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185) at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) at com.sun.proxy.$Proxy178.getHostStatistics(Unknown Source) at com.cloud.server.StatsCollector$HostCollector.runInContext(StatsCollector.java:438) at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748) 2018-04-27 11:18:24,077 DEBUG [c.c.a.m.ClusteredAgentAttache] (StatsCollector-1:ctx-82335701) (logid:95fda6d7) Seq 9-9075597674081682614: Unable to forward null 2018-04-27 11:18:24,177 ERROR [c.c.u.n.Link] (StatsCollector-1:ctx-82335701) (logid:95fda6d7) SSL error caught during unwrap data: Received fatal alert: certificate_unknown, for local address=/172.16.22.61:60130, remote address=/172.16.22.60:8250. The client may have invalid ca-certificates. 2018-04-27 11:18:24,177 WARN [c.c.a.m.ClusteredAgentManagerImpl] (StatsCollector-1:ctx-82335701) (logid:95fda6d7) Unable to connect to peer management server: 130719784044197, ip: 172.16.22.60 due to SSL: Fail to init SSL! java.io.IOException: SSL: Handshake failed with peer management server '130719784044197' on 172.16.22.60:8250 Im not familiar with the using of self-signed certificate in cloudstack, do you know where i can find out more information to investigate deeper ? or if you have any idea ? I tried to check keystore on both mgmt-server but i need a password i havnt... Thanks upfront, Have a nice day, Best regards, Nicolas Bouige DIMSI cloud.dimsi.fr<http://www.cloud.dimsi.fr> 4, avenue Laurent Cely Tour d'Asnière - 92600 Asnière sur Seine T/ +33 (0)6 28 98 53 40 rohit.ya...@shapeblue.com www.shapeblue.com<http://www.shapeblue.com> 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue
