[ https://issues.apache.org/jira/browse/CLOUDSTACK-5707?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13869670#comment-13869670 ]
ASF subversion and git services commented on CLOUDSTACK-5707: ------------------------------------------------------------- Commit 6bb15a00050c87204960e90b58d9e0bf1dd7720c in branch refs/heads/4.3 from [~likithas] [ https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;h=6bb15a0 ] CLOUDSTACK-5707. Hitting multiple 'User unauthenticated' exceptions when vCenter is upgraded to 5.5 from 5.1. We hit these excptions whenever a management server held session that was with the old 5.1 vCenter server is used to make resource calls to the new 5.5 vCenter. Validate a vCenter session context before it is being used to make a resource call. And if the context is invalid then discard the context and retrieve a new one. During the invalidation of an old context handle the context disconnect better by catching the appropriate exception and returning a newly created context. > Hitting multiple exceptions when the Vsphere client is upgraded to 5.5 from > 5.1 > ------------------------------------------------------------------------------- > > Key: CLOUDSTACK-5707 > URL: https://issues.apache.org/jira/browse/CLOUDSTACK-5707 > Project: CloudStack > Issue Type: Bug > Security Level: Public(Anyone can view this level - this is the > default.) > Components: VMware > Affects Versions: 4.3.0 > Environment: Upgraded the CS 4.2.1 to 4.3 with ESXi5.1 using Vsphere > client 5.1 > Upgraded the Vsphere client to 5.5 > Reporter: manasaveloori > Assignee: Likitha Shetty > Priority: Critical > Fix For: 4.3.0 > > Attachments: management-server.rar > > > Steps followed: > 1. Deployed CS 4.2.1 GA build with ESXi5.1(using Vsphere 5.1 client) > 2. Deployed some VMs. > 3. Upgraded the CS to 4.3. > 4. Now unmanaged the cluster, kept the host in maintenance . > 5. Upgraded the Vsphere client to 5.5 > (http://www.vmadmin.co.uk/vmware/36-virtualcenter/284-vsphere4to5upgradevcenter41to5) > 6. Manage the cluster and cancelled the maintenance mode on host. > 7. Host is connected. > Observing the following exceptions in Ms logs: > 014-01-01 20:03:35,467 DEBUG [c.c.s.StatsCollector] > (StatsCollector-3:ctx-a4d2e607) StorageCollector is running... > 2014-01-01 20:03:35,497 INFO [c.c.a.m.AgentAttache] > (StatsCollector-3:ctx-a4d2e607) Seq 2-1062210638: Unable to send due to > Resource [Host:2] is unreachable: Host 2: Channel is closed > 2014-01-01 20:03:35,497 DEBUG [c.c.a.m.AgentAttache] > (StatsCollector-3:ctx-a4d2e607) Seq 2-1062210638: Cancelling. > 2014-01-01 20:03:35,498 DEBUG [o.a.c.s.RemoteHostEndPoint] > (StatsCollector-3:ctx-a4d2e607) Failed to send command, due to Agent:2, > com.cloud.exception.AgentUnavailableException: Resource [Host:2] is > unreachable: Host 2: Channel is closed > 2014-01-01 20:03:35,498 ERROR [c.c.s.StatsCollector] > (StatsCollector-3:ctx-a4d2e607) Error trying to retrieve storage stats > com.cloud.utils.exception.CloudRuntimeException: Failed to send command, due > to Agent:2, com.cloud.exception.AgentUnavailableException: Resource [Host:2] > is unreachable: Host 2: Channel is closed > at > org.apache.cloudstack.storage.RemoteHostEndPoint.sendMessage(RemoteHostEndPoint.java:116) > at > com.cloud.server.StatsCollector$StorageCollector.runInContext(StatsCollector.java:550) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at > java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351) > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:722) > 2014-01-01 20:03:35,506 DEBUG [c.c.s.StatsCollector] > (StatsCollector-1:ctx-df84e56c) HostStatsCollector is running... > 2014-01-01 20:03:35,534 DEBUG [c.c.a.m.DirectAgentAttache] > (DirectAgent-374:ctx-c3b00e50) Seq 4-378077280: Executing request > 2014-01-01 20:03:35,638 ERROR [c.c.h.v.r.VmwareResource] > (DirectAgent-374:ctx-c3b00e50 10.147.40.28) Unable to execute > GetHostStatsCommand due to Exception: javax.xml.ws.soap.SOAPFaultException > Message: The session is not authenticated. > javax.xml.ws.soap.SOAPFaultException: The session is not authenticated. > at > com.sun.xml.internal.ws.fault.SOAP11Fault.getProtocolException(SOAP11Fault.java:178) > at > com.sun.xml.internal.ws.fault.SOAPFaultBuilder.createException(SOAPFaultBuilder.java:119) > at > com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:108) > at > com.sun.xml.internal.ws.client.sei.SyncMethodHandler.invoke(SyncMethodHandler.java:78) > at com.sun.xml.internal.ws.client.sei.SEIStub.invoke(SEIStub.java:129) > at $Proxy388.queryHostConnectionInfo(Unknown Source) > at > com.cloud.hypervisor.vmware.mo.HostMO.getHostHardwareSummary(HostMO.java:87) > at > com.cloud.hypervisor.vmware.mo.HostMO.getHyperHostHardwareSummary(HostMO.java:990) > at > com.cloud.hypervisor.vmware.resource.VmwareResource.getHyperHostStats(VmwareResource.java:6728) > at > com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:3805) > at > com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:435) > at > com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:216) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103) > at > org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53) > at > org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334) > at java.util.concurrent.FutureTask.run(FutureTask.java:166) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178) > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) > at java.lang.Thread.run(Thread.java:722) > 2014-01-01 20:03:35,641 DEBUG [c.c.a.m.DirectAgentAttache] > (DirectAgent-374:ctx-c3b00e50) Seq 4-378077280: Response Received: > 2014-01-01 20:03:35,641 DEBUG [c.c.a.t.Request] > (StatsCollector-1:ctx-df84e56c) Seq 4-378077280: Received: { Ans: , MgmtId: > 6758231703598, via: 4, Ver: v1, Flags: 10, { GetHostStatsAnswer } } > 2014-01-01 20:03:39,891 DEBUG [c.c.s.StatsCollector] > (StatsCollector-2:ctx-7c49b70a) VmStatsCollector is running... > Also the operations on the VMs are not consistently successful. > Details of the Environment: > MS: > 10.147.38.164:8080/client(admin/password) > CLI:root/password > Host 10.147.40.28(ESXI5.1)—root/password > Vcenter:10.147.60.9(upgraded from 5.1->5.5)(administrator/password_123) > Attaching the Ms logs -- This message was sent by Atlassian JIRA (v6.1.5#6160)