在vm_instance表里 s-909-VM/ s-889-VM是Running ，但在 host表里显示状态是Alerting，销毁重建后还是一样 ------------------ 原始邮件 ------------------ 发件人: "YUAN, GUI -HBIS";<yuan-...@hbis.com.cn>; 发送时间: 2014年7月15日(星期二) 下午2:58 收件人: "users-cn"<users-cn@cloudstack.apache.org>;
主题: RE: 注册模板后出现状态：Storage agent or storage VM disconnected 1.SSVM没有启动，可能是对应的AgentHost连不上。确认有可用的HOST 如：s-909-VM/ s-889-VM 2.如果SSVM正常，模板添加时，提示SSVM- response time out 则需要确认NFS服务器的NFS服务状态及防火墙设置 -----Original Message----- From: 赵涛 [mailto:zhaotao...@qq.com] Sent: Tuesday, July 15, 2014 12:46 PM To: users-cn Subject: 注册模板后出现状态：Storage agent or storage VM disconnected 注册模板后出现状态：Storage agent or storage VM disconnected,删除SSVM后问题依旧，请大神指导 删除SSVM虚拟机后重建的SSVM虚拟机上报日志： 2014-07-15 10:41:46,679 DEBUG [agent.manager.ClusteredAgentManagerImpl] (AgentManager-Handler-7:null) create ClusteredAgentAttache for 41 2014-07-15 10:41:46,685 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Connect to listener: XcpServerDiscoverer$$EnhancerByCGLIB$$18f3e761 2014-07-15 10:41:46,685 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Connect to listener: VmwareManagerImpl$$EnhancerByCGLIB$$4ad0e44c 2014-07-15 10:41:46,685 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Connect to listener: ClusteredVirtualMachineManagerImpl$$EnhancerByCGLIB$$15bc7206 2014-07-15 10:41:46,685 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Connect to listener: SecurityGroupListener 2014-07-15 10:41:46,685 INFO [network.security.SecurityGroupListener] (AgentManager-Handler-7:null) Received a host startup notification 2014-07-15 10:41:46,687 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Connect to listener: StoragePoolMonitor 2014-07-15 10:41:46,687 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Connect to listener: SecondaryStorageListener 2014-07-15 10:41:46,687 INFO [storage.secondary.SecondaryStorageListener] (AgentManager-Handler-7:null) Received a host startup notification com.cloud.agent.api.StartupSecondaryStorageCommand 2014-07-15 10:41:46,714 DEBUG [agent.transport.Request] (AgentManager-Handler-7:null) Seq 41-766640129: Sending { Cmd , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 100111, [{"SecStorageSetupCommand":{"secUrl":"nfs://10.11.18.7/export/d2secondary","certs":{},"wait":0}}] } 2014-07-15 10:41:46,727 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-14:null) Ping from 41 2014-07-15 10:41:46,776 DEBUG [agent.transport.Request] (AgentManager-Handler-10:null) Seq 42-1808072705: Processing: { Ans: , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110, [{"SecStorageSetupAnswer":{"_dir":"a35b9ab9-d061-3ed7-bbe7-82022eec78e4","result":true,"details":"success","wait":0}}] } 2014-07-15 10:41:46,776 DEBUG [agent.transport.Request] (AgentManager-Handler-2:null) Seq 42-1808072705: Received: { Ans: , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110, { SecStorageSetupAnswer } } 2014-07-15 10:41:46,777 DEBUG [agent.manager.AgentAttache] (AgentManager-Handler-10:null) Seq 42-1808072705: No more commands found 2014-07-15 10:41:46,777 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Details from executing class com.cloud.agent.api.SecStorageSetupCommand: success 2014-07-15 10:41:46,809 DEBUG [storage.secondary.SecondaryStorageManagerImpl] (AgentManager-Handler-2:null) Successfully programmed secondary storage nfs://10.11.18.7/export/d3secondary in secondary storage VM s-909-VM 2014-07-15 10:41:46,831 DEBUG [agent.transport.Request] (AgentManager-Handler-2:null) Seq 41-766640130: Sending { Cmd , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 100011, [{"SecStorageFirewallCfgCommand":{"portConfigs":[{"add":true,"sourceIp":"10.11.17.1","port":"443","intf":"eth2"}],"isAppendAIp":true,"wait":0}}] } 2014-07-15 10:41:46,880 DEBUG [agent.transport.Request] (AgentManager-Handler-4:null) Seq 41-766640130: Processing: { Ans: , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 10, [{"Answer":{"result":true,"wait":0}}] } 2014-07-15 10:41:46,881 DEBUG [agent.transport.Request] (AgentManager-Handler-2:null) Seq 41-766640130: Received: { Ans: , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 10, { Answer } } 2014-07-15 10:41:46,881 DEBUG [storage.secondary.SecondaryStorageManagerImpl] (AgentManager-Handler-2:null) Successfully programmed firewall rules into SSVM s-889-VM 2014-07-15 10:41:46,891 DEBUG [agent.transport.Request] (AgentManager-Handler-2:null) Seq 42-1808072706: Sending { Cmd , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 100011, [{"SecStorageFirewallCfgCommand":{"portConfigs":[{"add":true,"sourceIp":"10.11.19.8","port":"443","intf":"eth2"}],"isAppendAIp":false,"wait":0}}] } 2014-07-15 10:41:46,901 DEBUG [agent.transport.Request] (AgentManager-Handler-8:null) Seq 42-1808072706: Processing: { Ans: , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 10, [{"Answer":{"result":true,"wait":0}}] } 2014-07-15 10:41:46,902 DEBUG [agent.transport.Request] (AgentManager-Handler-2:null) Seq 42-1808072706: Received: { Ans: , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 10, { Answer } } 2014-07-15 10:41:46,902 DEBUG [storage.secondary.SecondaryStorageManagerImpl] (AgentManager-Handler-2:null) Successfully programmed firewall rules into s-909-VM 2014-07-15 10:41:46,929 DEBUG [agent.transport.Request] (AgentManager-Handler-2:null) Seq 42-1808072707: Sending { Cmd , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 100111, [{"SecStorageVMSetupCommand":{"allowedInternalSites":["10.11.18.0/24","10.11.19.0/24","10.11.16.0/24","10.11.17.0/24","10.11.24.0/24","192.168.1.0/24"],"copyUserName":"cloud","copyPassword":"nD7iafqpzcsxqme","wait":0}}] } 2014-07-15 10:41:47,083 DEBUG [agent.transport.Request] (AgentManager-Handler-11:null) Seq 42-1808072707: Processing: { Ans: , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110, [{"Answer":{"result":true,"details":"","wait":0}}] } 2014-07-15 10:41:47,083 DEBUG [agent.manager.AgentAttache] (AgentManager-Handler-11:null) Seq 42-1808072707: No more commands found 2014-07-15 10:41:47,083 DEBUG [agent.transport.Request] (AgentManager-Handler-2:null) Seq 42-1808072707: Received: { Ans: , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110, { Answer } } 2014-07-15 10:41:47,083 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Details from executing class com.cloud.agent.api.SecStorageVMSetupCommand: 2014-07-15 10:41:47,083 DEBUG [storage.secondary.SecondaryStorageManagerImpl] (AgentManager-Handler-2:null) Successfully programmed http auth into s-909-VM 2014-07-15 10:41:47,083 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Connect to listener: NetworkManagerImpl$$EnhancerByCGLIB$$bf7c5122 2014-07-15 10:41:47,083 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Connect to listener: AgentMonitor$$EnhancerByCGLIB$$301cd382 2014-07-15 10:41:47,084 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Connect to listener: DownloadListener 2014-07-15 10:41:47,101 DEBUG [agent.transport.Request] (AgentManager-Handler-2:null) Seq 42-1808072708: Sending { Cmd , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 100111, [{"storage.ListTemplateCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","wait":0}}] } 2014-07-15 10:41:47,629 DEBUG [agent.transport.Request] (AgentManager-Handler-3:null) Seq 42-1808072708: Processing: { Ans: , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110, [{"storage.ListTemplateAnswer":{"secUrl":"nfs://10.11.18.7/export/d3secondary","templateInfos":{"221-2-3706787c-d305-39e0-9680-9f84359df07a":{"templateName":"221-2-3706787c-d305-39e0-9680-9f84359df07a","installPath":"template/tmpl/2/221//221-2-3706787c-d305-39e0-9680-9f84359df07a.iso","size":45762560,"physicalSize":45762560,"id":221,"isPublic":true,"isCorrupted":false},"275-2-bb1fbd41-a14c-3f06-ba59-f815a250b45c":{"templateName":"275-2-bb1fbd41-a14c-3f06-ba59-f815a250b45c","installPath":"template/tmpl/2/275//90034c67-a7d5-33cb-be93-b3d0bd02de9d.qcow2","size":292001153024,"physicalSize":17294753792,"id":275,"isPublic":true,"isCorrupted":false},"217-2-0245468c-5e25-3513-92a4-89b4a045751f":{"templateName":"217-2-0245468c-5e25-3513-92a4-89b4a045751f","installPath":"template/tmpl/2/217//3da99fcd-4447-3e84-98c1-6ae7424ff649.qcow2","size":107374182400,"physicalSize":4641128448,"id":217,"isPublic":true,"isCorrupted":false},"routing-3":{"templateName":"routing-3","installPath":"template/tmpl/1/3//6c37c8df-1225-4bb5-bd0f-bf47eead5d09.qcow2","size":725811200,"physicalSize":725811200,"id":3,"isPublic":true,"isCorrupted":false},"centos55-x86_64":{"templateName":"centos55-x86_64","installPath":"template/tmpl/1/4//e970d345-e30b-30c4-879c-88d63200a721.qcow2","size":8589934592,"physicalSize":1769537536,"id":4,"isPublic":true,"isCorrupted":false},"218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1":{"templateName":"218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1","installPath":"template/tmpl/2/218//d3bfbf4b-902c-38db-9d45-2e500ac81dd7.qcow2","size":107374182400,"physicalSize":15100084224,"id":218,"isPublic":true,"isCorrupted":false}},"result":true,"details":"success","wait":0}}] } 2014-07-15 10:41:47,630 DEBUG [agent.manager.AgentAttache] (AgentManager-Handler-3:null) Seq 42-1808072708: No more commands found 2014-07-15 10:41:47,630 DEBUG [agent.transport.Request] (AgentManager-Handler-2:null) Seq 42-1808072708: Received: { Ans: , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110, { ListTemplateAnswer } } 2014-07-15 10:41:47,630 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Details from executing class com.cloud.agent.api.storage.ListTemplateCommand: success 2014-07-15 10:41:47,653 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-2:null) Template Sync found routing-3 already in the template host table 2014-07-15 10:41:47,705 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-2:null) Template Sync found centos55-x86_64 already in the template host table 2014-07-15 10:41:47,771 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-2:null) Template Sync did not find centos56-x86_64-xen ready on server 11, will request download to start/resume shortly 2014-07-15 10:41:47,775 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-2:null) Template Sync did not find centos53-x64 ready on server 11, will request download to start/resume shortly 2014-07-15 10:41:47,785 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-2:null) Template Sync found 217-2-0245468c-5e25-3513-92a4-89b4a045751f already in the template host table 2014-07-15 10:41:47,837 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-2:null) Template Sync found 218-2-00f60a5c-4771-3f10-a3f7-baae34372eb1 already in the template host table 2014-07-15 10:41:47,885 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-2:null) Template Sync found 221-2-3706787c-d305-39e0-9680-9f84359df07a already in the template host table 2014-07-15 10:41:47,913 DEBUG [agent.transport.Request] (AgentManager-Handler-6:null) Seq 41-766640129: Processing: { Ans: , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 110, [{"SecStorageSetupAnswer":{"_dir":"a5fd862f-5d3e-3c41-b912-3918446c341a","result":true,"details":"success","wait":0}}] } 2014-07-15 10:41:47,913 DEBUG [agent.manager.AgentAttache] (AgentManager-Handler-6:null) Seq 41-766640129: No more commands found 2014-07-15 10:41:47,913 DEBUG [agent.transport.Request] (AgentManager-Handler-7:null) Seq 41-766640129: Received: { Ans: , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 110, { SecStorageSetupAnswer } } 2014-07-15 10:41:47,913 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Details from executing class com.cloud.agent.api.SecStorageSetupCommand: success 2014-07-15 10:41:47,933 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-2:null) Template Sync found 275-2-bb1fbd41-a14c-3f06-ba59-f815a250b45c already in the template host table 2014-07-15 10:41:47,946 DEBUG [storage.secondary.SecondaryStorageManagerImpl] (AgentManager-Handler-7:null) Successfully programmed secondary storage nfs://10.11.18.7/export/d2secondary in secondary storage VM s-889-VM 2014-07-15 10:41:47,968 DEBUG [agent.transport.Request] (AgentManager-Handler-7:null) Seq 42-1808072709: Sending { Cmd , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 100011, [{"SecStorageFirewallCfgCommand":{"portConfigs":[{"add":true,"sourceIp":"10.11.19.8","port":"443","intf":"eth2"}],"isAppendAIp":true,"wait":0}}] } 2014-07-15 10:41:47,976 DEBUG [agent.transport.Request] (AgentManager-Handler-13:null) Seq 42-1808072709: Processing: { Ans: , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 10, [{"Answer":{"result":true,"wait":0}}] } 2014-07-15 10:41:47,977 DEBUG [agent.transport.Request] (AgentManager-Handler-7:null) Seq 42-1808072709: Received: { Ans: , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 10, { Answer } } 2014-07-15 10:41:47,977 DEBUG [storage.secondary.SecondaryStorageManagerImpl] (AgentManager-Handler-7:null) Successfully programmed firewall rules into SSVM s-909-VM 2014-07-15 10:41:47,985 DEBUG [agent.transport.Request] (AgentManager-Handler-7:null) Seq 41-766640131: Sending { Cmd , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 100011, [{"SecStorageFirewallCfgCommand":{"portConfigs":[{"add":true,"sourceIp":"10.11.17.1","port":"443","intf":"eth2"}],"isAppendAIp":false,"wait":0}}] } 2014-07-15 10:41:48,016 DEBUG [agent.transport.Request] (AgentManager-Handler-2:null) Seq 42-1808072710: Sending { Cmd , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 100111, [{"storage.ListVolumeCommand":{"secUrl":"nfs://10.11.18.7/export/d3secondary","wait":0}}] } 2014-07-15 10:41:48,033 DEBUG [agent.transport.Request] (AgentManager-Handler-5:null) Seq 41-766640131: Processing: { Ans: , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 10, [{"Answer":{"result":true,"wait":0}}] } 2014-07-15 10:41:48,033 DEBUG [agent.transport.Request] (AgentManager-Handler-7:null) Seq 41-766640131: Received: { Ans: , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 10, { Answer } } 2014-07-15 10:41:48,033 DEBUG [storage.secondary.SecondaryStorageManagerImpl] (AgentManager-Handler-7:null) Successfully programmed firewall rules into s-889-VM 2014-07-15 10:41:48,059 DEBUG [agent.transport.Request] (AgentManager-Handler-7:null) Seq 41-766640132: Sending { Cmd , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 100111, [{"SecStorageVMSetupCommand":{"allowedInternalSites":["10.11.18.0/24","10.11.19.0/24","10.11.16.0/24","10.11.17.0/24","10.11.24.0/24","192.168.1.0/24"],"copyUserName":"cloud","copyPassword":"nD7iafqpzcsxqme","wait":0}}] } 2014-07-15 10:41:48,077 DEBUG [agent.transport.Request] (AgentManager-Handler-12:null) Seq 42-1808072710: Processing: { Ans: , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110, [{"storage.ListVolumeAnswer":{"secUrl":"nfs://10.11.18.7/export/d3secondary","templateInfos":{},"result":true,"details":"success","wait":0}}] } 2014-07-15 10:41:48,077 DEBUG [agent.manager.AgentAttache] (AgentManager-Handler-12:null) Seq 42-1808072710: No more commands found 2014-07-15 10:41:48,077 DEBUG [agent.transport.Request] (AgentManager-Handler-2:null) Seq 42-1808072710: Received: { Ans: , MgmtId: 73548179740843, via: 42, Ver: v1, Flags: 110, { ListVolumeAnswer } } 2014-07-15 10:41:48,077 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Details from executing class com.cloud.agent.api.storage.ListVolumeCommand: success 2014-07-15 10:41:48,085 ERROR [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Monitor DownloadListener says there is an error in the connect process for 42 due to null java.lang.NullPointerException at com.cloud.storage.download.DownloadMonitorImpl.handleVolumeSync(DownloadMonitorImpl.java:704) at com.cloud.storage.download.DownloadMonitorImpl.handleSync(DownloadMonitorImpl.java:630) at com.cloud.storage.download.DownloadListener.processConnect(DownloadListener.java:385) at com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:605) at com.cloud.agent.manager.AgentManagerImpl.handleConnectedAgent(AgentManagerImpl.java:1157) at com.cloud.agent.manager.AgentManagerImpl.access$100(AgentManagerImpl.java:142) at com.cloud.agent.manager.AgentManagerImpl$AgentHandler.processRequest(AgentManagerImpl.java:1235) at com.cloud.agent.manager.AgentManagerImpl$AgentHandler.doTask(AgentManagerImpl.java:1374) at com.cloud.agent.manager.ClusteredAgentManagerImpl$ClusteredAgentHandler.doTask(ClusteredAgentManagerImpl.java:618) at com.cloud.utils.nio.Task.run(Task.java:83) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) 2014-07-15 10:41:48,089 INFO [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Host 42 is disconnecting with event AgentDisconnected 2014-07-15 10:41:48,093 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) The next status of agent 42is Alert, current status is Connecting 2014-07-15 10:41:48,093 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Deregistering link for 42 with state Alert 2014-07-15 10:41:48,093 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Remove Agent : 42 2014-07-15 10:41:48,093 DEBUG [agent.manager.ConnectedAgentAttache] (AgentManager-Handler-2:null) Processing Disconnect. 2014-07-15 10:41:48,094 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer$$EnhancerByCGLIB$$18f3e761 2014-07-15 10:41:48,095 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.hypervisor.vmware.manager.VmwareManagerImpl$$EnhancerByCGLIB$$4ad0e44c 2014-07-15 10:41:48,095 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.vm.ClusteredVirtualMachineManagerImpl$$EnhancerByCGLIB$$15bc7206 2014-07-15 10:41:48,095 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.network.security.SecurityGroupListener 2014-07-15 10:41:48,095 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.storage.listener.StoragePoolMonitor 2014-07-15 10:41:48,095 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.storage.secondary.SecondaryStorageListener 2014-07-15 10:41:48,095 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.network.NetworkManagerImpl$$EnhancerByCGLIB$$bf7c5122 2014-07-15 10:41:48,095 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.agent.manager.AgentMonitor$$EnhancerByCGLIB$$301cd382 2014-07-15 10:41:48,096 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.storage.download.DownloadListener 2014-07-15 10:41:48,096 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.storage.upload.UploadListener 2014-07-15 10:41:48,096 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.storage.LocalStoragePoolListener$$EnhancerByCGLIB$$c00af9b6 2014-07-15 10:41:48,096 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.capacity.StorageCapacityListener 2014-07-15 10:41:48,096 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.capacity.ComputeCapacityListener 2014-07-15 10:41:48,096 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.consoleproxy.ConsoleProxyListener 2014-07-15 10:41:48,101 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.network.SshKeysDistriMonitor 2014-07-15 10:41:48,101 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl$$EnhancerByCGLIB$$c27578aa 2014-07-15 10:41:48,101 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Sending Disconnect to listener: com.cloud.network.NetworkUsageManagerImpl$DirectNetworkStatsListener 2014-07-15 10:41:48,101 DEBUG [cloud.network.NetworkUsageManagerImpl] (AgentManager-Handler-2:null) Disconnected called on 42 with status Alert 2014-07-15 10:41:48,101 DEBUG [cloud.host.Status] (AgentManager-Handler-2:null) Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host id = 42, name = s-909-VM] 2014-07-15 10:41:48,163 DEBUG [cloud.host.Status] (AgentManager-Handler-2:null) Agent status update: [id = 42; name = s-909-VM; old status = Connecting; event = AgentDisconnected; new status = Alert; old update count = 209; new update count = 210] 2014-07-15 10:41:48,163 DEBUG [agent.manager.ClusteredAgentManagerImpl] (AgentManager-Handler-2:null) Notifying other nodes of to disconnect 2014-07-15 10:41:48,178 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Can not send command com.cloud.agent.api.ReadyCommand due to Host 42 is not up 2014-07-15 10:41:48,178 WARN [agent.manager.AgentManagerImpl] (AgentManager-Handler-2:null) Unable to create attache for agent: Seq 42-208: { Cmd , MgmtId: -1, via: 42, Ver: v1, Flags: 101, [{"StartupSecondaryStorageCommand":{"type":"SecondaryStorage","dataCenter":"2","pod":"2","guid":"s-909-VM-NfsSecondaryStorageResource","name":"s-909-VM","id":42,"version":"2.2.20130913151423","iqn":"NoIqn","publicIpAddress":"10.11.17.1","publicNetmask":"255.255.255.0","publicMacAddress":"06:ed:f0:00:00:29","privateIpAddress":"10.11.16.70","privateMacAddress":"06:5c:62:00:00:14","privateNetmask":"255.255.255.0","storageIpAddress":"10.11.16.70","storageNetmask":"255.255.255.0","storageMacAddress":"06:5c:62:00:00:14","resourceName":"NfsSecondaryStorageResource","wait":0}}] } 2014-07-15 10:41:48,208 DEBUG [agent.transport.Request] (AgentManager-Handler-9:null) Seq 41-766640132: Processing: { Ans: , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 110, [{"Answer":{"result":true,"details":"","wait":0}}] } 2014-07-15 10:41:48,208 DEBUG [agent.manager.AgentAttache] (AgentManager-Handler-9:null) Seq 41-766640132: No more commands found 2014-07-15 10:41:48,208 DEBUG [agent.transport.Request] (AgentManager-Handler-7:null) Seq 41-766640132: Received: { Ans: , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 110, { Answer } } 2014-07-15 10:41:48,208 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Details from executing class com.cloud.agent.api.SecStorageVMSetupCommand: 2014-07-15 10:41:48,208 DEBUG [storage.secondary.SecondaryStorageManagerImpl] (AgentManager-Handler-7:null) Successfully programmed http auth into s-889-VM 2014-07-15 10:41:48,209 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Connect to listener: NetworkManagerImpl$$EnhancerByCGLIB$$bf7c5122 2014-07-15 10:41:48,209 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Connect to listener: AgentMonitor$$EnhancerByCGLIB$$301cd382 2014-07-15 10:41:48,209 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Connect to listener: DownloadListener 2014-07-15 10:41:48,227 DEBUG [agent.transport.Request] (AgentManager-Handler-7:null) Seq 41-766640133: Sending { Cmd , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 100111, [{"storage.ListTemplateCommand":{"secUrl":"nfs://10.11.18.7/export/d2secondary","wait":0}}] } 2014-07-15 10:41:48,498 DEBUG [agent.transport.Request] (AgentManager-Handler-1:null) Seq 41-766640133: Processing: { Ans: , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 110, [{"storage.ListTemplateAnswer":{"secUrl":"nfs://10.11.18.7/export/d2secondary","templateInfos":{"221-2-3706787c-d305-39e0-9680-9f84359df07a":{"templateName":"221-2-3706787c-d305-39e0-9680-9f84359df07a","installPath":"template/tmpl/2/221//221-2-3706787c-d305-39e0-9680-9f84359df07a.iso","size":45762560,"physicalSize":45762560,"id":221,"isPublic":true,"isCorrupted":false},"206-2-33e4a593-fa36-3fb6-b68e-a587a8b1af80":{"templateName":"206-2-33e4a593-fa36-3fb6-b68e-a587a8b1af80","installPath":"template/tmpl/2/206//206-2-33e4a593-fa36-3fb6-b68e-a587a8b1af80.iso","size":3548971164,"physicalSize":3548971164,"id":206,"isPublic":true,"isCorrupted":false},"a1e5cdd4-b892-45a0-aeb4-a337cc96b442":{"templateName":"a1e5cdd4-b892-45a0-aeb4-a337cc96b442","installPath":"template/tmpl/2/209//a1e5cdd4-b892-45a0-aeb4-a337cc96b442.qcow2","size":107374182400,"physicalSize":15100084224,"id":1,"isPublic":true,"isCorrupted":false},"routing-3":{"templateName":"routing-3","installPath":"template/tmpl/1/3//3e38f707-5006-4d73-92a5-5be21b6f178f.qcow2","size":725811200,"physicalSize":725811200,"id":3,"isPublic":true,"isCorrupted":false},"5deebbfd-e603-4d09-912d-de5848d7118e":{"templateName":"5deebbfd-e603-4d09-912d-de5848d7118e","installPath":"template/tmpl/2/208//5deebbfd-e603-4d09-912d-de5848d7118e.qcow2","size":107374182400,"physicalSize":4641128448,"id":1,"isPublic":true,"isCorrupted":false},"205-2-5ffdab64-da2e-3afc-b166-061397e0de7c":{"templateName":"205-2-5ffdab64-da2e-3afc-b166-061397e0de7c","installPath":"template/tmpl/2/205//205-2-5ffdab64-da2e-3afc-b166-061397e0de7c.iso","size":4239566848,"physicalSize":4239566848,"id":205,"isPublic":true,"isCorrupted":false},"centos55-x86_64":{"templateName":"centos55-x86_64","installPath":"template/tmpl/1/4//20fb26b3-a01a-39ab-a39b-3b459dc862ed.qcow2","size":8589934592,"physicalSize":1769537536,"id":4,"isPublic":true,"isCorrupted":false}},"result":true,"details":"success","wait":0}}] } 2014-07-15 10:41:48,498 DEBUG [agent.manager.AgentAttache] (AgentManager-Handler-1:null) Seq 41-766640133: No more commands found 2014-07-15 10:41:48,498 DEBUG [agent.transport.Request] (AgentManager-Handler-7:null) Seq 41-766640133: Received: { Ans: , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 110, { ListTemplateAnswer } } 2014-07-15 10:41:48,498 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Details from executing class com.cloud.agent.api.storage.ListTemplateCommand: success 2014-07-15 10:41:48,521 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-7:null) Template Sync found routing-3 already in the template host table 2014-07-15 10:41:48,567 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-7:null) Template Sync found centos55-x86_64 already in the template host table 2014-07-15 10:41:48,609 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-7:null) Template Sync did not find centos56-x86_64-xen ready on server 2, will request download to start/resume shortly 2014-07-15 10:41:48,613 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-7:null) Template Sync did not find centos53-x64 ready on server 2, will request download to start/resume shortly 2014-07-15 10:41:48,625 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-7:null) Template Sync found 205-2-5ffdab64-da2e-3afc-b166-061397e0de7c already in the template host table 2014-07-15 10:41:48,687 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-7:null) Template Sync found 206-2-33e4a593-fa36-3fb6-b68e-a587a8b1af80 already in the template host table 2014-07-15 10:41:48,747 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-7:null) Template Sync found 5deebbfd-e603-4d09-912d-de5848d7118e already in the template host table 2014-07-15 10:41:48,813 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-7:null) Template Sync found a1e5cdd4-b892-45a0-aeb4-a337cc96b442 already in the template host table 2014-07-15 10:41:48,860 INFO [storage.download.DownloadMonitorImpl] (AgentManager-Handler-7:null) Template Sync found 221-2-3706787c-d305-39e0-9680-9f84359df07a already in the template host table 2014-07-15 10:41:48,922 DEBUG [agent.transport.Request] (AgentManager-Handler-7:null) Seq 41-766640134: Sending { Cmd , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 100111, [{"storage.ListVolumeCommand":{"secUrl":"nfs://10.11.18.7/export/d2secondary","wait":0}}] } 2014-07-15 10:41:48,939 DEBUG [agent.transport.Request] (AgentManager-Handler-15:null) Seq 41-766640134: Processing: { Ans: , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 110, [{"storage.ListVolumeAnswer":{"secUrl":"nfs://10.11.18.7/export/d2secondary","templateInfos":{},"result":true,"details":"success","wait":0}}] } 2014-07-15 10:41:48,939 DEBUG [agent.manager.AgentAttache] (AgentManager-Handler-15:null) Seq 41-766640134: No more commands found 2014-07-15 10:41:48,939 DEBUG [agent.transport.Request] (AgentManager-Handler-7:null) Seq 41-766640134: Received: { Ans: , MgmtId: 73548179740843, via: 41, Ver: v1, Flags: 110, { ListVolumeAnswer } } 2014-07-15 10:41:48,940 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Details from executing class com.cloud.agent.api.storage.ListVolumeCommand: success 2014-07-15 10:41:48,948 ERROR [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Monitor DownloadListener says there is an error in the connect process for 41 due to null java.lang.NullPointerException at com.cloud.storage.download.DownloadMonitorImpl.handleVolumeSync(DownloadMonitorImpl.java:704) at com.cloud.storage.download.DownloadMonitorImpl.handleSync(DownloadMonitorImpl.java:630) at com.cloud.storage.download.DownloadListener.processConnect(DownloadListener.java:385) at com.cloud.agent.manager.AgentManagerImpl.notifyMonitorsOfConnection(AgentManagerImpl.java:605) at com.cloud.agent.manager.AgentManagerImpl.handleConnectedAgent(AgentManagerImpl.java:1157) at com.cloud.agent.manager.AgentManagerImpl.access$100(AgentManagerImpl.java:142) at com.cloud.agent.manager.AgentManagerImpl$AgentHandler.processRequest(AgentManagerImpl.java:1235) at com.cloud.agent.manager.AgentManagerImpl$AgentHandler.doTask(AgentManagerImpl.java:1374) at com.cloud.agent.manager.ClusteredAgentManagerImpl$ClusteredAgentHandler.doTask(ClusteredAgentManagerImpl.java:618) at com.cloud.utils.nio.Task.run(Task.java:83) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603) at java.lang.Thread.run(Thread.java:679) 2014-07-15 10:41:48,951 INFO [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Host 41 is disconnecting with event AgentDisconnected 2014-07-15 10:41:48,956 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) The next status of agent 41is Alert, current status is Connecting 2014-07-15 10:41:48,956 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Deregistering link for 41 with state Alert 2014-07-15 10:41:48,956 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Remove Agent : 41 2014-07-15 10:41:48,956 DEBUG [agent.manager.ConnectedAgentAttache] (AgentManager-Handler-7:null) Processing Disconnect. 2014-07-15 10:41:48,957 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.hypervisor.xen.discoverer.XcpServerDiscoverer$$EnhancerByCGLIB$$18f3e761 2014-07-15 10:41:48,958 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.hypervisor.vmware.manager.VmwareManagerImpl$$EnhancerByCGLIB$$4ad0e44c 2014-07-15 10:41:48,958 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.vm.ClusteredVirtualMachineManagerImpl$$EnhancerByCGLIB$$15bc7206 2014-07-15 10:41:48,958 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.network.security.SecurityGroupListener 2014-07-15 10:41:48,958 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.storage.listener.StoragePoolMonitor 2014-07-15 10:41:48,958 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.storage.secondary.SecondaryStorageListener 2014-07-15 10:41:48,958 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.network.NetworkManagerImpl$$EnhancerByCGLIB$$bf7c5122 2014-07-15 10:41:48,958 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.agent.manager.AgentMonitor$$EnhancerByCGLIB$$301cd382 2014-07-15 10:41:48,959 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.storage.download.DownloadListener 2014-07-15 10:41:48,959 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.storage.upload.UploadListener 2014-07-15 10:41:48,959 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.storage.LocalStoragePoolListener$$EnhancerByCGLIB$$c00af9b6 2014-07-15 10:41:48,959 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.capacity.StorageCapacityListener 2014-07-15 10:41:48,959 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.capacity.ComputeCapacityListener 2014-07-15 10:41:48,959 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.consoleproxy.ConsoleProxyListener 2014-07-15 10:41:48,963 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.network.SshKeysDistriMonitor 2014-07-15 10:41:48,963 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.network.router.VpcVirtualNetworkApplianceManagerImpl$$EnhancerByCGLIB$$c27578aa 2014-07-15 10:41:48,964 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Sending Disconnect to listener: com.cloud.network.NetworkUsageManagerImpl$DirectNetworkStatsListener 2014-07-15 10:41:48,964 DEBUG [cloud.network.NetworkUsageManagerImpl] (AgentManager-Handler-7:null) Disconnected called on 41 with status Alert 2014-07-15 10:41:48,964 DEBUG [cloud.host.Status] (AgentManager-Handler-7:null) Transition:[Resource state = Enabled, Agent event = AgentDisconnected, Host id = 41, name = s-889-VM] 2014-07-15 10:41:49,006 DEBUG [cloud.host.Status] (AgentManager-Handler-7:null) Agent status update: [id = 41; name = s-889-VM; old status = Connecting; event = AgentDisconnected; new status = Alert; old update count = 391; new update count = 392] 2014-07-15 10:41:49,006 DEBUG [agent.manager.ClusteredAgentManagerImpl] (AgentManager-Handler-7:null) Notifying other nodes of to disconnect 2014-07-15 10:41:49,014 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Can not send command com.cloud.agent.api.ReadyCommand due to Host 41 is not up 2014-07-15 10:41:49,015 WARN [agent.manager.AgentManagerImpl] (AgentManager-Handler-7:null) Unable to create attache for agent: Seq 41-390: { Cmd , MgmtId: -1, via: 41, Ver: v1, Flags: 101, [{"StartupSecondaryStorageCommand":{"type":"SecondaryStorage","dataCenter":"1","pod":"1","guid":"s-889-VM-NfsSecondaryStorageResource","name":"s-889-VM","id":41,"version":"2.2.20130913151423","iqn":"NoIqn","publicIpAddress":"10.11.19.8","publicNetmask":"255.255.255.0","publicMacAddress":"06:fb:ee:00:00:1c","privateIpAddress":"10.11.18.67","privateMacAddress":"06:89:b0:00:00:11","privateNetmask":"255.255.255.0","storageIpAddress":"10.11.18.67","storageNetmask":"255.255.255.0","storageMacAddress":"06:89:b0:00:00:11","resourceName":"NfsSecondaryStorageResource","wait":0}}] } 2014-07-15 10:41:50,555 DEBUG [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) Zone 1 is ready to launch secondary storage VM 2014-07-15 10:41:50,609 DEBUG [storage.secondary.SecondaryStorageManagerImpl] (secstorage-1:null) Zone 2 is ready to launch secondary storage VM 2014-07-15 10:41:50,851 DEBUG [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null) Zone 1 is ready to launch console proxy 2014-07-15 10:41:50,868 DEBUG [cloud.consoleproxy.ConsoleProxyManagerImpl] (consoleproxy-1:null) Zone 2 is ready to launch console proxy 2014-07-15 10:41:51,848 DEBUG [cloud.server.StatsCollector] (StatsCollector-1:null) StorageCollector is running... 2014-07-15 10:41:51,864 DEBUG [cloud.server.StatsCollector] (StatsCollector-1:null) There is no secondary storage VM for secondary storage host nfs://10.11.18.7/export/d2secondary 2014-07-15 10:41:51,869 DEBUG [cloud.server.StatsCollector] (StatsCollector-1:null) There is no secondary storage VM for secondary storage host nfs://10.11.18.7/export/d3secondary 2014-07-15 10:41:51,928 DEBUG [network.router.VirtualNetworkApplianceManagerImpl] (RouterStatusMonitor-1:null) Found 0 routers. 2014-07-15 10:41:51,991 DEBUG [agent.transport.Request] (StatsCollector-1:null) Seq 7-513215195: Received: { Ans: , MgmtId: 73548179740843, via: 7, Ver: v1, Flags: 10, { GetStorageStatsAnswer } } 2014-07-15 10:41:52,059 DEBUG [agent.transport.Request] (StatsCollector-1:null) Seq 9-1410599594: Received: { Ans: , MgmtId: 73548179740843, via: 9, Ver: v1, Flags: 10, { GetStorageStatsAnswer } } 2014-07-15 10:41:52,128 DEBUG [agent.transport.Request] (StatsCollector-1:null) Seq 14-1320747056: Received: { Ans: , MgmtId: 73548179740843, via: 14, Ver: v1, Flags: 10, { GetStorageStatsAnswer } } 2014-07-15 10:41:52,150 DEBUG [agent.transport.Request] (StatsCollector-1:null) Seq 16-940116823: Received: { Ans: , MgmtId: 73548179740843, via: 16, Ver: v1, Flags: 10, { GetStorageStatsAnswer } } 2014-07-15 10:41:52,231 DEBUG [agent.transport.Request] (StatsCollector-1:null) Seq 14-1320747057: Received: { Ans: , MgmtId: 73548179740843, via: 14, Ver: v1, Flags: 10, { GetStorageStatsAnswer } } 2014-07-15 10:41:52,312 DEBUG [agent.transport.Request] (StatsCollector-1:null) Seq 7-513215196: Received: { Ans: , MgmtId: 73548179740843, via: 7, Ver: v1, Flags: 10, { GetStorageStatsAnswer } } 2014-07-15 10:41:53,351 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-6:null) Ping from 23 2014-07-15 10:41:54,315 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-13:null) Ping from 8 2014-07-15 10:41:54,532 DEBUG [agent.manager.AgentManagerImpl] (AgentManager-Handler-5:null) Ping from 40 Disclaimer: The contents of this e-mail, and its attachments, if any, are confidential and may be protected by law against any unauthorized use. If you have received this e-mail by mistake or have reason to believe that you are not the intended recipient, please notify the sender by reply e-mail as soon as possible and delete it from your computer system immediately thereafter. If you are not the intended recipient, you must not copy this e-mail or attachment or disclose the contents to any other person. While we have made every effort to keep our network virus free, we take no responsibility for any computer virus which might be transferred by way of this e-mail.
