I've already had to flatten and start again so I'd rather avoid it - but
my suspicion is that all of this is related to the kvm host's networking
somehow. I followed the instructions on the cloudstack install guide,
and ended up with the below - does it look right to you guys?
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet manual
auto cloudbr0
iface cloudbr0 inet static
bridge_ports eth0
bridge_fd 5
bridge_stp off
bridge_maxwait 1
address 10.4.0.2
netmask 255.0.0.0
network 10.0.0.0
broadcast 10.255.255.255
gateway 10.0.0.1
# dns-* options are implemented by the resolvconf package, if
installed
dns-nameservers 10.0.0.12
dns-search avco
auto cloudbr1
iface cloudbr1 inet manual
bridge_ports eth0
bridge_fd 5
bridge_stp off
bridge_maxwait 1
Many Thanks,
Elliot
Elliot Berg wrote:
Hi,
Cloud.log contains the following just after the machine's rebooted;
Mon Jul 14 16:01:06 UTC 2014 checking that eth0 has IP
Mon Jul 14 16:01:07 UTC 2014 waiting for eth0 interface setup with ip
timer=0
Mon Jul 14 16:01:08 UTC 2014 waiting for eth0 interface setup with ip
timer=1
Mon Jul 14 16:01:09 UTC 2014 waiting for eth0 interface setup with ip
timer=2
Mon Jul 14 16:01:10 UTC 2014 waiting for eth0 interface setup with ip
timer=3
Mon Jul 14 16:01:11 UTC 2014 waiting for eth0 interface setup with ip
timer=4
Mon Jul 14 16:01:12 UTC 2014 waiting for eth0 interface setup with ip
timer=5
Mon Jul 14 16:01:13 UTC 2014 waiting for eth0 interface setup with ip
timer=6
Mon Jul 14 16:01:14 UTC 2014 waiting for eth0 interface setup with ip
timer=7
Mon Jul 14 16:01:15 UTC 2014 waiting for eth0 interface setup with ip
timer=8
Mon Jul 14 16:01:16 UTC 2014 waiting for eth0 interface setup with ip
timer=9
Mon Jul 14 16:01:17 UTC 2014 waiting for eth0 interface setup with ip
timer=10
Mon Jul 14 16:01:18 UTC 2014 waiting for eth0 interface setup with ip
timer=11
Mon Jul 14 16:01:19 UTC 2014 waiting for eth0 interface setup with ip
timer=12
Mon Jul 14 16:01:20 UTC 2014 waiting for eth0 interface setup with ip
timer=13
Mon Jul 14 16:01:21 UTC 2014 waiting for eth0 interface setup with ip
timer=14
Mon Jul 14 16:01:22 UTC 2014 waiting for eth0 interface setup with ip
timer=15
Mon Jul 14 16:01:23 UTC 2014 waiting for eth0 interface setup with ip
timer=16
Mon Jul 14 16:01:23 UTC 2014 interface eth0 is not set up with ip...
exiting
As I say, I'm wondering whether this indicates a more general
networking issue on the host, as I'd have expected the virtual router
to sort its own networking assuming the host's is fine?
Thanks,
Elliot
Jayapal Reddy Uradi wrote:
Hi,
Check the logs while the router is booting. Also check
/var/log/cloud.log
Thanks,
Jayapal
On 14-Jul-2014, at 2:39 PM, Elliot Berg<elliot.b...@avcosystems.com>
wrote:
Hi,
I did that earlier as part of the troubleshooting when it was stuck
- so I've just looked at the logs instead of recreating it again as
that was only just done. When you say the router logs, do you mean
general logs on the virtual router machine? If so,
syslog/messages/kern.log/daemon.log are all empty?
Elliot
Jayapal Reddy Uradi wrote:
Hi Elliot,
Try recreating router (destroy the router and deploy new vm, router
get recreated).
After recreation if the problem still exists, check the router logs
to see why the interfaces are brought up.
Thanks,
jayapal
On 11-Jul-2014, at 1:38 PM, Elliot
Berg<elliot.b...@avcosystems.com> wrote:
So, I'm wondering whether the guest not having the interfaces
configured correctly (i.e. not having an IP) is just a symptom of
more generally broken networking - my interfaces file for the KVM
host is below, does anyone spot any issues?
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet manual
auto cloudbr0
iface cloudbr0 inet static
bridge_ports eth0
bridge_fd 5
bridge_stp off
bridge_maxwait 1
address 10.4.0.2
netmask 255.0.0.0
network 10.0.0.0
broadcast 10.255.255.255
gateway 10.0.0.1
# dns-* options are implemented by the resolvconf package,
if installed
dns-nameservers 10.0.0.12
dns-search avco
auto cloudbr1
iface cloudbr1 inet manual
bridge_ports eth0
bridge_fd 5
bridge_stp off
bridge_maxwait 1
Thanks,
Elliot
Elliot Berg wrote:
Doh! I did, but forgot about it being on a funny port. Now that
I'm into the VM I can see that it's not running, and fails to
start when it tries to bind to the address that it should have on
the guest range. I notice that "ifconfig -a" shows two NICs, only
one of which is up (the one with the link local IP). I'm
guessing that indicates a more general networking issue?
I think how it's laid out is 10.4.0.0-255 for physical machines
(1 is the management server, 2 is the first host), 10.4.1.0-255
is the management network and 10.4.2.0-255 is the guest
network...but it's possible I've misunderstood the networking
config during setup? What I really wanted was hosts on
10.4.0.0-255 and guests on 10.4.1.0-255 (and beyond), as in the
future I'd like it to co-exist with our existing infrastructure
while we migrate things - but I kept being told about conflicts
etc when I tried to set up cloudstack like that during the
initial set up process?
Thanks,
Elliot
Jayapal Reddy Uradi wrote:
Hi Elliot,
Did you ssh to VR using the ssh key ?
Ex: ssh -i /root/.ssh/id_rsa.cloud -p3922root@169.254.3.196
If it is failed to ssh, then there is issue with the ssh keys.
Thanks,
Jayapal
On 09-Jul-2014, at 4:43 PM, Harikrishna
Patnala<harikrishna.patn...@citrix.com> wrote:
1) Log into your KVM host.
2) Use command “virsh list”. This gives the list of VMs on the
host.
3) Use command “virsh console<VirtualRouterId>” to log into the
VR.
-Harikrishna
On 09-Jul-2014, at 3:52 pm, Elliot
Berg<elliot.b...@avcosystems.com> wrote:
I don't know - I can't seem to ssh to the link local IP. It
pings, but ssh times out. If I try and use the "connect to
console" button in the gui, that too times out :(
Elliot
Harikrishna Patnala wrote:
From the logs
2014-07-08 12:08:56,218 DEBUG [agent.transport.Request]
(AgentManager-Handler-1:null) Seq 1-277348416: Processing: {
Ans: , MgmtId: 159320647860937, via: 1, Ver: v1, Flags: 110,
[{"com.cloud.agent.api.Answer":{"result":false,"details":"grep:
/var/lib/misc/dnsmasq.leases: No such file or
directory","wait":0}}] }
Can you check whether dnsmasq service is running in the
Virtual Router ? if not, start the service and check for
“/var/lib/misc/dnsmasq.leases”
-Harikrishna
On 08-Jul-2014, at 3:47 pm, Elliot
Berg<elliot.b...@avcosystems.com> wrote:
Hi,
I've done that, and now there's a new virtual router which
says it's running, however a deployment still fails. My
latest lot of logs are available
athttps://dl.dropboxusercontent.com/u/47728104/management-server.log.gz,
and there's now one thing in the op_it_work table with a
step != 'Done', which is a ConsoleProxy.
Interestingly if I look at the console proxy vm in the
cloudstack management gui it says it's running, though.
Thanks,
Elliot
Harikrishna Patnala wrote:
Yes mark the VR to stopped, destroy VR, mark the VR entry
in op_it_work to “Done” and try deploying VM.
-Harikrishna
On 08-Jul-2014, at 12:44 pm, Elliot
Berg<elliot.b...@avcosystems.com> wrote:
Hi,
It appears to be stuck in the "starting" state - so I
don't get the option to reboot it or anything. If I change
the state to stopped in the database directly will the
management server attempt to start it again or do I need
to do something more?
Thanks!
Elliot
Harikrishna Patnala wrote:
Is your Virtual Router up and running ? If is in running
state you can mark it Done and deploy a VM.
If it is in stopped state try restarting it. You can try
updating the field as well.
-Harikrishna
On 07-Jul-2014, at 7:10 pm, Elliot
Berg<elliot.b...@avcosystems.com> wrote:
I can see two entries that have the "step" field set to
something other than "Done", one of them is
ConsoleProxy | Starting
and the other is
DomainRouter | Prepare
Am I safe to just delete the rows, or should I just
update the field?
Thanks,
Elliot
Harikrishna Patnala wrote:
Do you see any work item pending for Virtual Router
r-4-VM in “op_it_work” table ?
If there are any, remove those entries and try VM
deployment again.
I see in the logs that VR has a task pending
2014-07-07 10:28:15,934 WARN
[cloud.vm.VirtualMachineManagerImpl]
(Job-Executor-5:job-48 = [
22369802-b5aa-4b5a-a26d-1fab11241551 ]) The task item
for vm VM[DomainRouter|r-4-VM] has been inactive for
418531
-Harikrishna
On 07-Jul-2014, at 2:18 pm, Elliot
Berg<elliot.b...@avcosystems.com<mailto:elliot.b...@avcosystems.com>>
wrote:
I'm still not really spotting anything indicating why
it's not using the host, but I suspect that's just
because I don't really know what I'm looking for - so
I've zipped the whole log for today and stuffed it on
dropbox
athttps://dl.dropboxusercontent.com/u/47728104/management-server.log.gz.
Hopefully someone who's used cloudstack a lot more will
have more success!
Thanks,
Elliot
Elliot Berg wrote:
I'm going back over everything and I've noticed
something else - everywhere I've looked for how to use
local storage says I should change two global settings;
* system.vm.use.local.storage = true
* use.local.storage = true
However I'm looking at my global settings and only the
first exists (which I have set to true).
Elliot
Elliot Berg wrote:
Ah, so when looking back a bit further before (I was
kind of only looking for exceptions higher up before
now), I've just spotted this...
2014-07-03 10:48:28,765 DEBUG
[allocator.impl.FirstFitAllocator]
(Job-Executor-3:job-46 = [ 92fb959d-edc5-4fe2-84a0-5
6001226e4ac ] FirstFitRoutingAllocator) Looking for
speed=1000Mhz, Ram=1024
2014-07-03 10:48:28,765 DEBUG
[allocator.impl.FirstFitAllocator]
(Job-Executor-3:job-46 = [ 92fb959d-edc5-4fe2-84a0-5
6001226e4ac ] FirstFitRoutingAllocator) Host name:
cloudstack-host1, hostId: 1 is in avoid set, skipping
this and try
ing other available hosts
That's the one and only host - so I'm guessing that has
something to do with it!
Elliot
--
Elliot Berg | Analyst Programmer/Network Team
Email:elliot.b...@avcosystems.com<mailto:elliot.b...@avcosystems.com>
| Tel: 01753 213700 |
Web:www.avcosystems.com<http://www.avcosystems.com/>
<image.png>
Avco Systems Ltd, Registered in England& Wales,
Registration Number 1976620
Registered Office: Avco Systems | 17 Bath Road | Slough
| SL1 3UF
ilya musayev wrote:
Elliot,
When you see such an error - there usually a
predecessor message that says CloudStack checked for X,
Y and Z and found no suitable resources based on your
configuration.
Put the logs on pastebin or some other site (strip out
any private info you dont want to share). I would also
recommend cloudstack 4.3.1 (which is not officially out
yet) but should come thru in the next several weeks.
Its latest stable release of CloudStack 4.3.0 - with
latest bug fixes.
I've put a build for folks who want to try it out until
we complete official release of ACS 4.3.1 process.
Unzip tgz and it should have required RPMs with both
Open Source and Non-Open Source modules.
http://www.cloudsand.com/cloudstack-4.3.0-1.tgz
Regards
ilya
On 7/2/14, 1:06 AM, Elliot Berg wrote:
Hi,
I've been putting together a cloudstack set-up for
experimentation purposes - right now we're just trying
to compare different platforms for private cloud
infrastructure before we start getting too in depth
with any of them.
I've added the cloudstack 4.2 apt repository, and I'm
running on Ubuntu 12.04 LTS, and I believe I've
followed all the installation guides correctly at the
various stages.
We've set up a management server, which is also an NFS
server, however we're interested in using local storage
for the majority of things, and have also set up a
single KVM host which I believe is all configured
correctly to use local storage. If I look at the
dashboard, I'm told I have more than enough resource in
every section to create an instance the size I want to
- which is a small offering I've created with just
1.0GHz and 1GB of RAM, with local storage. The host's
not very powerful, but according to the dashboard I am
using 1.50GHz/5.87GHz, 1.38GB/7.80GB, 3.55GB/285.95GB
Secondary Storage, 1.03GB/450.99GB Local Storage and
0.00KB/571.90GB Primary Storage (I'm assuming that's
meant to be a combination of the NFS server's primary
storage offering and the local storage on the host,
though the numbers don't quite make sense at first
glance).
However, when I try to add an instance, I receive an
InsufficientServerCapacityException and I'm struggling
to work out why. I can't add an instance using a small
shared storage offering either, but if I'm not mistaken
that's expected because the zone and host are
configured to use local storage. The only thing I can
think of is that the local storage isn't properly
configured, but when I've looked it seems to be.
Any pointers for how I can further diagnose this would
be great - thanks in advance!
Elliot
