Ah, now I see what's going on. The agent starts, but it doesn't work at all!
2015-05-12 18:12:49,493 INFO [cloud.agent.AgentShell] (Thread-1:null) Agent started 2015-05-12 18:12:49,495 INFO [cloud.agent.AgentShell] (Thread-1:null) Implementation Version is 4.2.1 2015-05-12 18:12:49,496 INFO [cloud.agent.AgentShell] (Thread-1:null) agent.properties found at /etc/cloudstack/agent/agent.properties 2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found property: workers 2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found property: port 2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found property: pod 2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found property: resource 2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found property: private.network.device 2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found property: zone 2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found property: guid 2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found property: guest.network.device 2015-05-12 18:12:49,498 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found property: cluster 2015-05-12 18:12:49,499 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found property: domr.scripts.dir 2015-05-12 18:12:49,499 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found property: local.storage.uuid 2015-05-12 18:12:49,499 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found property: public.network.device 2015-05-12 18:12:49,499 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Found property: host 2015-05-12 18:12:49,499 INFO [cloud.agent.AgentShell] (Thread-1:null) Defaulting to using properties file for storage 2015-05-12 18:12:49,500 INFO [cloud.agent.AgentShell] (Thread-1:null) Defaulting to the constant time backoff algorithm 2015-05-12 18:12:49,501 INFO [cloud.utils.LogUtils] (Thread-1:null) log4j configuration found at /etc/cloudstack/agent/log4j-cloud.xml 2015-05-12 18:12:49,614 DEBUG [cloud.agent.AgentShell] (Thread-1:null) Checking to see if agent.pid exists. 2015-05-12 18:12:49,622 DEBUG [cloud.utils.ProcessUtil] (Thread-1:null) Executing: bash -c echo $PPID And this is the last line I see in the log-file. The file /var/log/agent.pid is being created, but it has zero length. So, the agent's process is still "working" (I can see it by "ps"), but it doesn't do anything. Any ideas on the cause of such an odd behavior? -----Original Message----- From: Vladimir Melnik [mailto:v.mel...@uplink.ua] Sent: Tuesday, May 12, 2015 5:57 PM To: users@cloudstack.apache.org Subject: The agent doesn't restart :( Hello! I encountered quite an odd problem: the agent doesn't restart on KVM host. Here is what's going on. When it starts for the first time it creates /var/run/agent.pid file, but there are no numbers, the file has zero length. When I restart it (e.g. by service cloudstack-agent restart command), it's being shut down, but file isn't being removed. So it can't start again, here is what I see in /var/log/cloudstack/agent/cloudstack-agent.out file: 2015-05-12 17:04:37,504{GMT} INFO [cloud.agent.AgentShell] (Thread-1:) Agent started 2015-05-12 17:04:37,506{GMT} INFO [cloud.agent.AgentShell] (Thread-1:) Implementation Version is 4.2.1 2015-05-12 17:04:37,507{GMT} INFO [cloud.agent.AgentShell] (Thread-1:) agent.properties found at /etc/cloudstack/agent/agent.properties 2015-05-12 17:04:37,508{GMT} INFO [cloud.agent.AgentShell] (Thread-1:) Defaulting to using properties file for storage 2015-05-12 17:04:37,509{GMT} INFO [cloud.agent.AgentShell] (Thread-1:) Defaulting to the constant time backoff algorithm 2015-05-12 17:04:37,510{GMT} INFO [cloud.utils.LogUtils] (Thread-1:) log4j configuration found at /etc/cloudstack/agent/log4j-cloud.xml 2015-05-12 17:04:37,626{GMT} ERROR [cloud.agent.AgentShell] (Thread-1:) Unable to start agent: Java process is being started twice. If this is not true, remove /var/run/agent.pid Unable to start agent: Java process is being started twice. If this is not true, remove /var/run/agent.pid I remove this file by hands and do "restart" again. The agent starts, but /var/run/agent.pid has zero length again. This server is just the same as other CentOS hosts (there are many of them in my farm), but other hosts don't seem to be having this issue. I opened a ticket (https://issues.apache.org/jira/browse/CLOUDSTACK-8456, there are more details about the environment), but if anyone has some clues, hints or ideas, please, share your thoughts on this topic. Any help will be greatly appreciated! Thanks! -- V.Melnik
