Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Daan Hoogland]

ood to hear Jevgeni, good floating..

On Fri, Feb 9, 2018 at 5:42 PM, Jevgeni Zolotarjov 
wrote:

> To finalize, IPs recovered too. I think I started my VMs too early before
> router was up.
>
> Now its all fine.
>
> On Fri, Feb 9, 2018 at 6:36 PM, Paul Angus 
> wrote:
>
> > No problem, I'm pleased that you're sorted.
> >
> >
> > Kind regards,
> >
> > Paul Angus
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > Sent: 09 February 2018 16:34
> > To: users@cloudstack.apache.org
> > Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> > 4.11
> >
> > destroyed virtual router
> >
> > it got recreated.
> > On NOW. My VMs start!!!
> >
> > All IPs got changed, but I can manage with that.
> >
> > Thank you for your support
> >
> > On Fri, Feb 9, 2018 at 6:21 PM, Paul Angus 
> > wrote:
> >
> > > Have you done the same for the virtual routers?
> > >
> > > I'll look at your log in the meantime
> > >
> > >
> > > Kind regards,
> > >
> > > Paul Angus
> > >
> > > paul.an...@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> > >
> > >
> > >
> > >
> > > -Original Message-
> > > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > > Sent: 09 February 2018 16:19
> > > To: users@cloudstack.apache.org
> > > Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> > > 4.11
> > >
> > > I destroyed system VMs and they got recreated automatically.
> > > They are running. I can verify that by
> > > virsh list --all
> > >
> > > and
> > > I can see their console and it sugests that it is Cloudstack 4.11
> > systemVM
> > >
> > > BUT
> > > it didn't solve the problem. None of my own VM is listed by "virsh list
> > > -all". They do not start.
> > > I tried to create new VM, it does not start either, due to to the same
> > > problem - insufficient capacity
> > >
> > > On Fri, Feb 9, 2018 at 6:02 PM, Daan Hoogland  >
> > > wrote:
> > >
> > > > listen to Paul, not to me.
> > > > He's an operator i'm just impatient
> > > >
> > > > On Fri, Feb 9, 2018 at 5:00 PM, Paul Angus  >
> > > > wrote:
> > > >
> > > > > After upgrading the code of the mgmt. server you need upgrade your
> > > > > system VMs from the old template to ones using the new templates.
> > > > >
> > > > > This needs to be done for the SSVM, CPVM and all of your virtual
> > > routers.
> > > > >
> > > > > For the SSVM & CPVM you do this by destroying them and CloudStack
> > > > > will recreate them with the new template.
> > > > > For the virtual routers, you can go to each of them in turn in the
> > > > > UI and there will be a upgrade router button.
> > > > >
> > > > > You get to these through the infrastructure tab in the UI.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > paul.an...@shapeblue.com
> > > > > www.shapeblue.com
> > > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > -Original Message-
> > > > > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > > > > Sent: 09 February 2018 15:55
> > > > > To: users@cloudstack.apache.org
> > > > > Subject: Re: cloudstack-management fails to start after upgrade
> 4.10
> > ->
> > > > > 4.11
> > > > >
> > > > > Destroyed VMs? No. Definitely not.
> > > > >
> > > > > At least I can see them all under web console Home->Instances. Can
> > make
> > > > > snapshots even
> > > > >
> > > > > My todays management server log https://www.sendspace.com/
> > file/nxxgg0
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Feb 9, 2018 at 5:33 PM, Paul Angus <
> paul.an...@shapeblue.com
> > >
> > > > > wrote:
> > > > >
> > > > > > Hi Jevgeni,
> > > > > >
> > > > > > Can I take you off at a slight tangent...
> > > > > >
> > > > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > > > (logid:0afb959d) DataCenter id = '1' provided is in avoid set,
> > > > > > DeploymentPlanner cannot allocate the VM, returning.
> > > > > >
> > > > > > Have you destroyed your system vms so that new 4.11 system vms
> have
> > > > > > been deployed?
> > > > > >
> > > > > > It may help us if you can paste all of your management log from
> > today
> > > > > > into https://pastebin.com (or similar) to share with us.
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > paul.an...@shapeblue.com
> > > > > > www.shapeblue.com
> > > > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> > > > > >
> > > > > >
> > > > > >
> > > > > >
> > > > > > -Original Message-
> > > > > > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > > > > > Sent: 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Jevgeni Zolotarjov]

To finalize, IPs recovered too. I think I started my VMs too early before
router was up.

Now its all fine.

On Fri, Feb 9, 2018 at 6:36 PM, Paul Angus  wrote:

> No problem, I'm pleased that you're sorted.
>
>
> Kind regards,
>
> Paul Angus
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> Sent: 09 February 2018 16:34
> To: users@cloudstack.apache.org
> Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> 4.11
>
> destroyed virtual router
>
> it got recreated.
> On NOW. My VMs start!!!
>
> All IPs got changed, but I can manage with that.
>
> Thank you for your support
>
> On Fri, Feb 9, 2018 at 6:21 PM, Paul Angus 
> wrote:
>
> > Have you done the same for the virtual routers?
> >
> > I'll look at your log in the meantime
> >
> >
> > Kind regards,
> >
> > Paul Angus
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > Sent: 09 February 2018 16:19
> > To: users@cloudstack.apache.org
> > Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> > 4.11
> >
> > I destroyed system VMs and they got recreated automatically.
> > They are running. I can verify that by
> > virsh list --all
> >
> > and
> > I can see their console and it sugests that it is Cloudstack 4.11
> systemVM
> >
> > BUT
> > it didn't solve the problem. None of my own VM is listed by "virsh list
> > -all". They do not start.
> > I tried to create new VM, it does not start either, due to to the same
> > problem - insufficient capacity
> >
> > On Fri, Feb 9, 2018 at 6:02 PM, Daan Hoogland 
> > wrote:
> >
> > > listen to Paul, not to me.
> > > He's an operator i'm just impatient
> > >
> > > On Fri, Feb 9, 2018 at 5:00 PM, Paul Angus 
> > > wrote:
> > >
> > > > After upgrading the code of the mgmt. server you need upgrade your
> > > > system VMs from the old template to ones using the new templates.
> > > >
> > > > This needs to be done for the SSVM, CPVM and all of your virtual
> > routers.
> > > >
> > > > For the SSVM & CPVM you do this by destroying them and CloudStack
> > > > will recreate them with the new template.
> > > > For the virtual routers, you can go to each of them in turn in the
> > > > UI and there will be a upgrade router button.
> > > >
> > > > You get to these through the infrastructure tab in the UI.
> > > >
> > > >
> > > >
> > > >
> > > > paul.an...@shapeblue.com
> > > > www.shapeblue.com
> > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> > > >
> > > >
> > > >
> > > >
> > > > -Original Message-
> > > > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > > > Sent: 09 February 2018 15:55
> > > > To: users@cloudstack.apache.org
> > > > Subject: Re: cloudstack-management fails to start after upgrade 4.10
> ->
> > > > 4.11
> > > >
> > > > Destroyed VMs? No. Definitely not.
> > > >
> > > > At least I can see them all under web console Home->Instances. Can
> make
> > > > snapshots even
> > > >
> > > > My todays management server log https://www.sendspace.com/
> file/nxxgg0
> > > >
> > > >
> > > >
> > > > On Fri, Feb 9, 2018 at 5:33 PM, Paul Angus  >
> > > > wrote:
> > > >
> > > > > Hi Jevgeni,
> > > > >
> > > > > Can I take you off at a slight tangent...
> > > > >
> > > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > > (logid:0afb959d) DataCenter id = '1' provided is in avoid set,
> > > > > DeploymentPlanner cannot allocate the VM, returning.
> > > > >
> > > > > Have you destroyed your system vms so that new 4.11 system vms have
> > > > > been deployed?
> > > > >
> > > > > It may help us if you can paste all of your management log from
> today
> > > > > into https://pastebin.com (or similar) to share with us.
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > paul.an...@shapeblue.com
> > > > > www.shapeblue.com
> > > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > -Original Message-
> > > > > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > > > > Sent: 09 February 2018 15:31
> > > > > To: users@cloudstack.apache.org
> > > > > Subject: Re: cloudstack-management fails to start after upgrade
> 4.10
> > ->
> > > > > 4.11
> > > > >
> > > > > the same result
> > > > >
> > > > > [root@mtl1-apphst03 management]# virsh list --all
> > > > >  IdName   State
> > > > > 
> > > > >  1 v-1-VM running
> > > > >  2 

RE: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Paul Angus]

No problem, I'm pleased that you're sorted.


Kind regards,

Paul Angus

paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com] 
Sent: 09 February 2018 16:34
To: users@cloudstack.apache.org
Subject: Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

destroyed virtual router

it got recreated.
On NOW. My VMs start!!!

All IPs got changed, but I can manage with that.

Thank you for your support

On Fri, Feb 9, 2018 at 6:21 PM, Paul Angus  wrote:

> Have you done the same for the virtual routers?
>
> I'll look at your log in the meantime
>
>
> Kind regards,
>
> Paul Angus
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>
>
>
>
> -Original Message-
> From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> Sent: 09 February 2018 16:19
> To: users@cloudstack.apache.org
> Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> 4.11
>
> I destroyed system VMs and they got recreated automatically.
> They are running. I can verify that by
> virsh list --all
>
> and
> I can see their console and it sugests that it is Cloudstack 4.11 systemVM
>
> BUT
> it didn't solve the problem. None of my own VM is listed by "virsh list
> -all". They do not start.
> I tried to create new VM, it does not start either, due to to the same
> problem - insufficient capacity
>
> On Fri, Feb 9, 2018 at 6:02 PM, Daan Hoogland 
> wrote:
>
> > listen to Paul, not to me.
> > He's an operator i'm just impatient
> >
> > On Fri, Feb 9, 2018 at 5:00 PM, Paul Angus 
> > wrote:
> >
> > > After upgrading the code of the mgmt. server you need upgrade your
> > > system VMs from the old template to ones using the new templates.
> > >
> > > This needs to be done for the SSVM, CPVM and all of your virtual
> routers.
> > >
> > > For the SSVM & CPVM you do this by destroying them and CloudStack
> > > will recreate them with the new template.
> > > For the virtual routers, you can go to each of them in turn in the
> > > UI and there will be a upgrade router button.
> > >
> > > You get to these through the infrastructure tab in the UI.
> > >
> > >
> > >
> > >
> > > paul.an...@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> > >
> > >
> > >
> > >
> > > -Original Message-
> > > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > > Sent: 09 February 2018 15:55
> > > To: users@cloudstack.apache.org
> > > Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> > > 4.11
> > >
> > > Destroyed VMs? No. Definitely not.
> > >
> > > At least I can see them all under web console Home->Instances. Can make
> > > snapshots even
> > >
> > > My todays management server log https://www.sendspace.com/file/nxxgg0
> > >
> > >
> > >
> > > On Fri, Feb 9, 2018 at 5:33 PM, Paul Angus 
> > > wrote:
> > >
> > > > Hi Jevgeni,
> > > >
> > > > Can I take you off at a slight tangent...
> > > >
> > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > (logid:0afb959d) DataCenter id = '1' provided is in avoid set,
> > > > DeploymentPlanner cannot allocate the VM, returning.
> > > >
> > > > Have you destroyed your system vms so that new 4.11 system vms have
> > > > been deployed?
> > > >
> > > > It may help us if you can paste all of your management log from today
> > > > into https://pastebin.com (or similar) to share with us.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > paul.an...@shapeblue.com
> > > > www.shapeblue.com
> > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> > > >
> > > >
> > > >
> > > >
> > > > -Original Message-
> > > > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > > > Sent: 09 February 2018 15:31
> > > > To: users@cloudstack.apache.org
> > > > Subject: Re: cloudstack-management fails to start after upgrade 4.10
> ->
> > > > 4.11
> > > >
> > > > the same result
> > > >
> > > > [root@mtl1-apphst03 management]# virsh list --all
> > > >  IdName   State
> > > > 
> > > >  1 v-1-VM running
> > > >  2 s-2-VM running
> > > >
> > > >
> > > >
> > > > On Fri, Feb 9, 2018 at 5:28 PM, Daan Hoogland <
> daan.hoogl...@gmail.com
> > >
> > > > wrote:
> > > >
> > > > >  so try
> > > > > # virsh list --all
> > > > >
> > > > > On Fri, Feb 9, 2018 at 4:27 PM, Jevgeni Zolotarjov
> > > > >  > > > > >
> > > > > wrote:
> > > > >
> > > > > > I guess, these are system VMs.
> > > > > > none of my own created instances can start.
> > > > > >  IdName   State

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Jevgeni Zolotarjov]

destroyed virtual router

it got recreated.
On NOW. My VMs start!!!

All IPs got changed, but I can manage with that.

Thank you for your support

On Fri, Feb 9, 2018 at 6:21 PM, Paul Angus  wrote:

> Have you done the same for the virtual routers?
>
> I'll look at your log in the meantime
>
>
> Kind regards,
>
> Paul Angus
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> Sent: 09 February 2018 16:19
> To: users@cloudstack.apache.org
> Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> 4.11
>
> I destroyed system VMs and they got recreated automatically.
> They are running. I can verify that by
> virsh list --all
>
> and
> I can see their console and it sugests that it is Cloudstack 4.11 systemVM
>
> BUT
> it didn't solve the problem. None of my own VM is listed by "virsh list
> -all". They do not start.
> I tried to create new VM, it does not start either, due to to the same
> problem - insufficient capacity
>
> On Fri, Feb 9, 2018 at 6:02 PM, Daan Hoogland 
> wrote:
>
> > listen to Paul, not to me.
> > He's an operator i'm just impatient
> >
> > On Fri, Feb 9, 2018 at 5:00 PM, Paul Angus 
> > wrote:
> >
> > > After upgrading the code of the mgmt. server you need upgrade your
> > > system VMs from the old template to ones using the new templates.
> > >
> > > This needs to be done for the SSVM, CPVM and all of your virtual
> routers.
> > >
> > > For the SSVM & CPVM you do this by destroying them and CloudStack
> > > will recreate them with the new template.
> > > For the virtual routers, you can go to each of them in turn in the
> > > UI and there will be a upgrade router button.
> > >
> > > You get to these through the infrastructure tab in the UI.
> > >
> > >
> > >
> > >
> > > paul.an...@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> > >
> > >
> > >
> > >
> > > -Original Message-
> > > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > > Sent: 09 February 2018 15:55
> > > To: users@cloudstack.apache.org
> > > Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> > > 4.11
> > >
> > > Destroyed VMs? No. Definitely not.
> > >
> > > At least I can see them all under web console Home->Instances. Can make
> > > snapshots even
> > >
> > > My todays management server log https://www.sendspace.com/file/nxxgg0
> > >
> > >
> > >
> > > On Fri, Feb 9, 2018 at 5:33 PM, Paul Angus 
> > > wrote:
> > >
> > > > Hi Jevgeni,
> > > >
> > > > Can I take you off at a slight tangent...
> > > >
> > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > (logid:0afb959d) DataCenter id = '1' provided is in avoid set,
> > > > DeploymentPlanner cannot allocate the VM, returning.
> > > >
> > > > Have you destroyed your system vms so that new 4.11 system vms have
> > > > been deployed?
> > > >
> > > > It may help us if you can paste all of your management log from today
> > > > into https://pastebin.com (or similar) to share with us.
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > paul.an...@shapeblue.com
> > > > www.shapeblue.com
> > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> > > >
> > > >
> > > >
> > > >
> > > > -Original Message-
> > > > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > > > Sent: 09 February 2018 15:31
> > > > To: users@cloudstack.apache.org
> > > > Subject: Re: cloudstack-management fails to start after upgrade 4.10
> ->
> > > > 4.11
> > > >
> > > > the same result
> > > >
> > > > [root@mtl1-apphst03 management]# virsh list --all
> > > >  IdName   State
> > > > 
> > > >  1 v-1-VM running
> > > >  2 s-2-VM running
> > > >
> > > >
> > > >
> > > > On Fri, Feb 9, 2018 at 5:28 PM, Daan Hoogland <
> daan.hoogl...@gmail.com
> > >
> > > > wrote:
> > > >
> > > > >  so try
> > > > > # virsh list --all
> > > > >
> > > > > On Fri, Feb 9, 2018 at 4:27 PM, Jevgeni Zolotarjov
> > > > >  > > > > >
> > > > > wrote:
> > > > >
> > > > > > I guess, these are system VMs.
> > > > > > none of my own created instances can start.
> > > > > >  IdName   State
> > > > > > 
> > > > > >  1 v-1-VM running
> > > > > >  2 s-2-VM running
> > > > > >
> > > > > > virsh start  3> gives error
> > > > > > error: failed to get domain '8'
> > > > > > error: Domain not found: no domain with matching name '8'
> > > > > >
> > > > > >
> > > > > > And yes, cloudstack and 

RE: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Paul Angus]

I'm a little confused about what’s working at what isn't, but...
2018-02-09 12:49:16,657 DEBUG [c.c.d.DeploymentPlanningManagerImpl] 
(Work-Job-Executor-10:ctx-a7c0188f job-581/job-793 ctx-babd6f7e) 
(logid:3bc0581f) The last host of this VM is not UP or is not enabled, host 
status is: Disconnected, host resource state is: Enabled

Your host is disconnected.

Did you yum update cloudstack-agent on the kvm host?

If you did, can you paste the cloudstack-agent log for us to see?


Kind regards,

Paul Angus

paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: Paul Angus [mailto:paul.an...@shapeblue.com] 
Sent: 09 February 2018 16:21
To: users@cloudstack.apache.org
Subject: RE: cloudstack-management fails to start after upgrade 4.10 -> 4.11

Have you done the same for the virtual routers?

I'll look at your log in the meantime


Kind regards,

Paul Angus

paul.an...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
  
 


-Original Message-
From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
Sent: 09 February 2018 16:19
To: users@cloudstack.apache.org
Subject: Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

I destroyed system VMs and they got recreated automatically.
They are running. I can verify that by
virsh list --all

and
I can see their console and it sugests that it is Cloudstack 4.11 systemVM

BUT
it didn't solve the problem. None of my own VM is listed by "virsh list -all". 
They do not start.
I tried to create new VM, it does not start either, due to to the same problem 
- insufficient capacity

On Fri, Feb 9, 2018 at 6:02 PM, Daan Hoogland 
wrote:

> listen to Paul, not to me.
> He's an operator i'm just impatient
>
> On Fri, Feb 9, 2018 at 5:00 PM, Paul Angus 
> wrote:
>
> > After upgrading the code of the mgmt. server you need upgrade your 
> > system VMs from the old template to ones using the new templates.
> >
> > This needs to be done for the SSVM, CPVM and all of your virtual routers.
> >
> > For the SSVM & CPVM you do this by destroying them and CloudStack 
> > will recreate them with the new template.
> > For the virtual routers, you can go to each of them in turn in the 
> > UI and there will be a upgrade router button.
> >
> > You get to these through the infrastructure tab in the UI.
> >
> >
> >
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > Sent: 09 February 2018 15:55
> > To: users@cloudstack.apache.org
> > Subject: Re: cloudstack-management fails to start after upgrade 4.10 
> > ->
> > 4.11
> >
> > Destroyed VMs? No. Definitely not.
> >
> > At least I can see them all under web console Home->Instances. Can 
> > make snapshots even
> >
> > My todays management server log 
> > https://www.sendspace.com/file/nxxgg0
> >
> >
> >
> > On Fri, Feb 9, 2018 at 5:33 PM, Paul Angus 
> > 
> > wrote:
> >
> > > Hi Jevgeni,
> > >
> > > Can I take you off at a slight tangent...
> > >
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > (logid:0afb959d) DataCenter id = '1' provided is in avoid set, 
> > > DeploymentPlanner cannot allocate the VM, returning.
> > >
> > > Have you destroyed your system vms so that new 4.11 system vms 
> > > have been deployed?
> > >
> > > It may help us if you can paste all of your management log from 
> > > today into https://pastebin.com (or similar) to share with us.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > paul.an...@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> > >
> > >
> > >
> > >
> > > -Original Message-
> > > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > > Sent: 09 February 2018 15:31
> > > To: users@cloudstack.apache.org
> > > Subject: Re: cloudstack-management fails to start after upgrade 
> > > 4.10 ->
> > > 4.11
> > >
> > > the same result
> > >
> > > [root@mtl1-apphst03 management]# virsh list --all
> > >  IdName   State
> > > 
> > >  1 v-1-VM running
> > >  2 s-2-VM running
> > >
> > >
> > >
> > > On Fri, Feb 9, 2018 at 5:28 PM, Daan Hoogland 
> > >  >
> > > wrote:
> > >
> > > >  so try
> > > > # virsh list --all
> > > >
> > > > On Fri, Feb 9, 2018 at 4:27 PM, Jevgeni Zolotarjov 
> > > >  > > > >
> > > > wrote:
> > > >
> > > > > I guess, these are system VMs.
> > > > > none of my own created instances can start.
> > > > >  IdName   State
> > > > > 

RE: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Paul Angus]

Have you done the same for the virtual routers?

I'll look at your log in the meantime


Kind regards,

Paul Angus

paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com] 
Sent: 09 February 2018 16:19
To: users@cloudstack.apache.org
Subject: Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

I destroyed system VMs and they got recreated automatically.
They are running. I can verify that by
virsh list --all

and
I can see their console and it sugests that it is Cloudstack 4.11 systemVM

BUT
it didn't solve the problem. None of my own VM is listed by "virsh list -all". 
They do not start.
I tried to create new VM, it does not start either, due to to the same problem 
- insufficient capacity

On Fri, Feb 9, 2018 at 6:02 PM, Daan Hoogland 
wrote:

> listen to Paul, not to me.
> He's an operator i'm just impatient
>
> On Fri, Feb 9, 2018 at 5:00 PM, Paul Angus 
> wrote:
>
> > After upgrading the code of the mgmt. server you need upgrade your 
> > system VMs from the old template to ones using the new templates.
> >
> > This needs to be done for the SSVM, CPVM and all of your virtual routers.
> >
> > For the SSVM & CPVM you do this by destroying them and CloudStack 
> > will recreate them with the new template.
> > For the virtual routers, you can go to each of them in turn in the 
> > UI and there will be a upgrade router button.
> >
> > You get to these through the infrastructure tab in the UI.
> >
> >
> >
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > Sent: 09 February 2018 15:55
> > To: users@cloudstack.apache.org
> > Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> > 4.11
> >
> > Destroyed VMs? No. Definitely not.
> >
> > At least I can see them all under web console Home->Instances. Can make
> > snapshots even
> >
> > My todays management server log https://www.sendspace.com/file/nxxgg0
> >
> >
> >
> > On Fri, Feb 9, 2018 at 5:33 PM, Paul Angus 
> > wrote:
> >
> > > Hi Jevgeni,
> > >
> > > Can I take you off at a slight tangent...
> > >
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > (logid:0afb959d) DataCenter id = '1' provided is in avoid set,
> > > DeploymentPlanner cannot allocate the VM, returning.
> > >
> > > Have you destroyed your system vms so that new 4.11 system vms have
> > > been deployed?
> > >
> > > It may help us if you can paste all of your management log from today
> > > into https://pastebin.com (or similar) to share with us.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > paul.an...@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> > >
> > >
> > >
> > >
> > > -Original Message-
> > > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > > Sent: 09 February 2018 15:31
> > > To: users@cloudstack.apache.org
> > > Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> > > 4.11
> > >
> > > the same result
> > >
> > > [root@mtl1-apphst03 management]# virsh list --all
> > >  IdName   State
> > > 
> > >  1 v-1-VM running
> > >  2 s-2-VM running
> > >
> > >
> > >
> > > On Fri, Feb 9, 2018 at 5:28 PM, Daan Hoogland  >
> > > wrote:
> > >
> > > >  so try
> > > > # virsh list --all
> > > >
> > > > On Fri, Feb 9, 2018 at 4:27 PM, Jevgeni Zolotarjov
> > > >  > > > >
> > > > wrote:
> > > >
> > > > > I guess, these are system VMs.
> > > > > none of my own created instances can start.
> > > > >  IdName   State
> > > > > 
> > > > >  1 v-1-VM running
> > > > >  2 s-2-VM running
> > > > >
> > > > > virsh start  3> gives error
> > > > > error: failed to get domain '8'
> > > > > error: Domain not found: no domain with matching name '8'
> > > > >
> > > > >
> > > > > And yes, cloudstack and mysql - everything locally
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Feb 9, 2018 at 5:15 PM, Daan Hoogland
> > > > > 
> > > > > wrote:
> > > > >
> > > > > > On Fri, Feb 9, 2018 at 3:58 PM, Jevgeni Zolotarjov <
> > > > > j.zolotar...@gmail.com
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > * how to start instance with virsh?
> > > > > > >
> > > > > > ​ah, usually something like
> > > > > > # virsh start 
> > > > > > or
> > > > > > # virsh start <​number>
> > > > > >
> > > > > >
> 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Jevgeni Zolotarjov]

I destroyed system VMs and they got recreated automatically.
They are running. I can verify that by
virsh list --all

and
I can see their console and it sugests that it is Cloudstack 4.11 systemVM

BUT
it didn't solve the problem. None of my own VM is listed by "virsh list
-all". They do not start.
I tried to create new VM, it does not start either, due to to the same
problem - insufficient capacity

On Fri, Feb 9, 2018 at 6:02 PM, Daan Hoogland 
wrote:

> listen to Paul, not to me.
> He's an operator i'm just impatient
>
> On Fri, Feb 9, 2018 at 5:00 PM, Paul Angus 
> wrote:
>
> > After upgrading the code of the mgmt. server you need upgrade your system
> > VMs from the old template to ones using the new templates.
> >
> > This needs to be done for the SSVM, CPVM and all of your virtual routers.
> >
> > For the SSVM & CPVM you do this by destroying them and CloudStack will
> > recreate them with the new template.
> > For the virtual routers, you can go to each of them in turn in the UI and
> > there will be a upgrade router button.
> >
> > You get to these through the infrastructure tab in the UI.
> >
> >
> >
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > Sent: 09 February 2018 15:55
> > To: users@cloudstack.apache.org
> > Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> > 4.11
> >
> > Destroyed VMs? No. Definitely not.
> >
> > At least I can see them all under web console Home->Instances. Can make
> > snapshots even
> >
> > My todays management server log https://www.sendspace.com/file/nxxgg0
> >
> >
> >
> > On Fri, Feb 9, 2018 at 5:33 PM, Paul Angus 
> > wrote:
> >
> > > Hi Jevgeni,
> > >
> > > Can I take you off at a slight tangent...
> > >
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > (logid:0afb959d) DataCenter id = '1' provided is in avoid set,
> > > DeploymentPlanner cannot allocate the VM, returning.
> > >
> > > Have you destroyed your system vms so that new 4.11 system vms have
> > > been deployed?
> > >
> > > It may help us if you can paste all of your management log from today
> > > into https://pastebin.com (or similar) to share with us.
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > > paul.an...@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> > >
> > >
> > >
> > >
> > > -Original Message-
> > > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > > Sent: 09 February 2018 15:31
> > > To: users@cloudstack.apache.org
> > > Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> > > 4.11
> > >
> > > the same result
> > >
> > > [root@mtl1-apphst03 management]# virsh list --all
> > >  IdName   State
> > > 
> > >  1 v-1-VM running
> > >  2 s-2-VM running
> > >
> > >
> > >
> > > On Fri, Feb 9, 2018 at 5:28 PM, Daan Hoogland  >
> > > wrote:
> > >
> > > >  so try
> > > > # virsh list --all
> > > >
> > > > On Fri, Feb 9, 2018 at 4:27 PM, Jevgeni Zolotarjov
> > > >  > > > >
> > > > wrote:
> > > >
> > > > > I guess, these are system VMs.
> > > > > none of my own created instances can start.
> > > > >  IdName   State
> > > > > 
> > > > >  1 v-1-VM running
> > > > >  2 s-2-VM running
> > > > >
> > > > > virsh start  3> gives error
> > > > > error: failed to get domain '8'
> > > > > error: Domain not found: no domain with matching name '8'
> > > > >
> > > > >
> > > > > And yes, cloudstack and mysql - everything locally
> > > > >
> > > > >
> > > > >
> > > > > On Fri, Feb 9, 2018 at 5:15 PM, Daan Hoogland
> > > > > 
> > > > > wrote:
> > > > >
> > > > > > On Fri, Feb 9, 2018 at 3:58 PM, Jevgeni Zolotarjov <
> > > > > j.zolotar...@gmail.com
> > > > > > >
> > > > > > wrote:
> > > > > >
> > > > > > > * how to start instance with virsh?
> > > > > > >
> > > > > > ​ah, usually something like
> > > > > > # virsh start 
> > > > > > or
> > > > > > # virsh start <​number>
> > > > > >
> > > > > >
> > > > > > > * starting in debugger? I would rather not do it :)
> > > > > > >
> > > > > > ​ok, i expected as much. It will slow our solution process
> > > > unfortunately
> > > > > > ​
> > > > > >
> > > > > > ​In the below at the bottom it says you have two VMs and both are
> > > > > running.
> > > > > > Is that correct?​
> > > > > >
> > > > > >
> > > > > > > * this is the log around GetHostStatsAnswer
> > > > > > > 2018-02-09 14:05:30,274 DEBUG 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Daan Hoogland]

listen to Paul, not to me.
He's an operator i'm just impatient

On Fri, Feb 9, 2018 at 5:00 PM, Paul Angus  wrote:

> After upgrading the code of the mgmt. server you need upgrade your system
> VMs from the old template to ones using the new templates.
>
> This needs to be done for the SSVM, CPVM and all of your virtual routers.
>
> For the SSVM & CPVM you do this by destroying them and CloudStack will
> recreate them with the new template.
> For the virtual routers, you can go to each of them in turn in the UI and
> there will be a upgrade router button.
>
> You get to these through the infrastructure tab in the UI.
>
>
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> Sent: 09 February 2018 15:55
> To: users@cloudstack.apache.org
> Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> 4.11
>
> Destroyed VMs? No. Definitely not.
>
> At least I can see them all under web console Home->Instances. Can make
> snapshots even
>
> My todays management server log https://www.sendspace.com/file/nxxgg0
>
>
>
> On Fri, Feb 9, 2018 at 5:33 PM, Paul Angus 
> wrote:
>
> > Hi Jevgeni,
> >
> > Can I take you off at a slight tangent...
> >
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > (logid:0afb959d) DataCenter id = '1' provided is in avoid set,
> > DeploymentPlanner cannot allocate the VM, returning.
> >
> > Have you destroyed your system vms so that new 4.11 system vms have
> > been deployed?
> >
> > It may help us if you can paste all of your management log from today
> > into https://pastebin.com (or similar) to share with us.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > Sent: 09 February 2018 15:31
> > To: users@cloudstack.apache.org
> > Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> > 4.11
> >
> > the same result
> >
> > [root@mtl1-apphst03 management]# virsh list --all
> >  IdName   State
> > 
> >  1 v-1-VM running
> >  2 s-2-VM running
> >
> >
> >
> > On Fri, Feb 9, 2018 at 5:28 PM, Daan Hoogland 
> > wrote:
> >
> > >  so try
> > > # virsh list --all
> > >
> > > On Fri, Feb 9, 2018 at 4:27 PM, Jevgeni Zolotarjov
> > >  > > >
> > > wrote:
> > >
> > > > I guess, these are system VMs.
> > > > none of my own created instances can start.
> > > >  IdName   State
> > > > 
> > > >  1 v-1-VM running
> > > >  2 s-2-VM running
> > > >
> > > > virsh start  3> gives error
> > > > error: failed to get domain '8'
> > > > error: Domain not found: no domain with matching name '8'
> > > >
> > > >
> > > > And yes, cloudstack and mysql - everything locally
> > > >
> > > >
> > > >
> > > > On Fri, Feb 9, 2018 at 5:15 PM, Daan Hoogland
> > > > 
> > > > wrote:
> > > >
> > > > > On Fri, Feb 9, 2018 at 3:58 PM, Jevgeni Zolotarjov <
> > > > j.zolotar...@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > * how to start instance with virsh?
> > > > > >
> > > > > ​ah, usually something like
> > > > > # virsh start 
> > > > > or
> > > > > # virsh start <​number>
> > > > >
> > > > >
> > > > > > * starting in debugger? I would rather not do it :)
> > > > > >
> > > > > ​ok, i expected as much. It will slow our solution process
> > > unfortunately
> > > > > ​
> > > > >
> > > > > ​In the below at the bottom it says you have two VMs and both are
> > > > running.
> > > > > Is that correct?​
> > > > >
> > > > >
> > > > > > * this is the log around GetHostStatsAnswer
> > > > > > 2018-02-09 14:05:30,274 DEBUG [c.c.s.StatsCollector]
> > > > > > (StatsCollector-4:ctx-167407f0) (logid:0c470e95)
> > > > > > HostStatsCollector
> > > is
> > > > > > running...
> > > > > > 2018-02-09 14:05:30,322 DEBUG [c.c.a.t.Request]
> > > > > > (StatsCollector-4:ctx-167407f0) (logid:0c470e95) Seq
> > > > > 3-613896924205940798:
> > > > > > Received:  { Ans: , MgmtId: 264216221068220, via:
> > > > > > 3(mtl1-apphst03),
> > > > Ver:
> > > > > > v1, Flags: 10, { GetHostStatsAnswer } }
> > > > > > 2018-02-09 14:05:31,388 DEBUG [c.c.s.StatsCollector]
> > > > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
> > > > > > StorageCollector is running...
> > > > > > 2018-02-09 14:05:31,396 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > > > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
> > > > > 

RE: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Paul Angus]

After upgrading the code of the mgmt. server you need upgrade your system VMs 
from the old template to ones using the new templates.

This needs to be done for the SSVM, CPVM and all of your virtual routers.

For the SSVM & CPVM you do this by destroying them and CloudStack will recreate 
them with the new template.
For the virtual routers, you can go to each of them in turn in the UI and there 
will be a upgrade router button.

You get to these through the infrastructure tab in the UI.




paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com] 
Sent: 09 February 2018 15:55
To: users@cloudstack.apache.org
Subject: Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

Destroyed VMs? No. Definitely not.

At least I can see them all under web console Home->Instances. Can make 
snapshots even

My todays management server log https://www.sendspace.com/file/nxxgg0



On Fri, Feb 9, 2018 at 5:33 PM, Paul Angus  wrote:

> Hi Jevgeni,
>
> Can I take you off at a slight tangent...
>
> (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> (logid:0afb959d) DataCenter id = '1' provided is in avoid set, 
> DeploymentPlanner cannot allocate the VM, returning.
>
> Have you destroyed your system vms so that new 4.11 system vms have 
> been deployed?
>
> It may help us if you can paste all of your management log from today 
> into https://pastebin.com (or similar) to share with us.
>
>
>
>
>
>
>
>
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>
>
>
>
> -Original Message-
> From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> Sent: 09 February 2018 15:31
> To: users@cloudstack.apache.org
> Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> 4.11
>
> the same result
>
> [root@mtl1-apphst03 management]# virsh list --all
>  IdName   State
> 
>  1 v-1-VM running
>  2 s-2-VM running
>
>
>
> On Fri, Feb 9, 2018 at 5:28 PM, Daan Hoogland 
> wrote:
>
> >  so try
> > # virsh list --all
> >
> > On Fri, Feb 9, 2018 at 4:27 PM, Jevgeni Zolotarjov
> >  > >
> > wrote:
> >
> > > I guess, these are system VMs.
> > > none of my own created instances can start.
> > >  IdName   State
> > > 
> > >  1 v-1-VM running
> > >  2 s-2-VM running
> > >
> > > virsh start  3> gives error
> > > error: failed to get domain '8'
> > > error: Domain not found: no domain with matching name '8'
> > >
> > >
> > > And yes, cloudstack and mysql - everything locally
> > >
> > >
> > >
> > > On Fri, Feb 9, 2018 at 5:15 PM, Daan Hoogland
> > > 
> > > wrote:
> > >
> > > > On Fri, Feb 9, 2018 at 3:58 PM, Jevgeni Zolotarjov <
> > > j.zolotar...@gmail.com
> > > > >
> > > > wrote:
> > > >
> > > > > * how to start instance with virsh?
> > > > >
> > > > ​ah, usually something like
> > > > # virsh start 
> > > > or
> > > > # virsh start <​number>
> > > >
> > > >
> > > > > * starting in debugger? I would rather not do it :)
> > > > >
> > > > ​ok, i expected as much. It will slow our solution process
> > unfortunately
> > > > ​
> > > >
> > > > ​In the below at the bottom it says you have two VMs and both are
> > > running.
> > > > Is that correct?​
> > > >
> > > >
> > > > > * this is the log around GetHostStatsAnswer
> > > > > 2018-02-09 14:05:30,274 DEBUG [c.c.s.StatsCollector]
> > > > > (StatsCollector-4:ctx-167407f0) (logid:0c470e95)
> > > > > HostStatsCollector
> > is
> > > > > running...
> > > > > 2018-02-09 14:05:30,322 DEBUG [c.c.a.t.Request]
> > > > > (StatsCollector-4:ctx-167407f0) (logid:0c470e95) Seq
> > > > 3-613896924205940798:
> > > > > Received:  { Ans: , MgmtId: 264216221068220, via:
> > > > > 3(mtl1-apphst03),
> > > Ver:
> > > > > v1, Flags: 10, { GetHostStatsAnswer } }
> > > > > 2018-02-09 14:05:31,388 DEBUG [c.c.s.StatsCollector]
> > > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
> > > > > StorageCollector is running...
> > > > > 2018-02-09 14:05:31,396 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
> > > > getCommandHostDelegation:
> > > > > class com.cloud.agent.api.GetStorageStatsCommand
> > > > > 2018-02-09 14:05:31,396 DEBUG [c.c.h.XenServerGuru]
> > > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) We are
> > > > > returning
> > the
> > > > > default host to execute commands because the command is not of
> > > > > Copy
> > > type.
> > > > > 2018-02-09 14:05:31,445 DEBUG [c.c.a.t.Request]
> > > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) Seq

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Daan Hoogland]

or better through cloudstack

On Fri, Feb 9, 2018 at 4:59 PM, Daan Hoogland 
wrote:

> Jevgeni,
>
> What Paul means is: Those two VMs are probably the old ones base on the
> 4.10 systemvm template. These need to be upgraded before you can deploy
> anything. The way to do that is to destroy them and let cloudstack recreate
> them. They won't allow you to instantiate anything anyway.
>
> so destroy
> v-1-VM and
> s-2-VM
>
> for instance with virsh
>
> On Fri, Feb 9, 2018 at 4:54 PM, Jevgeni Zolotarjov  > wrote:
>
>> Destroyed VMs? No. Definitely not.
>>
>> At least I can see them all under web console Home->Instances. Can make
>> snapshots even
>>
>> My todays management server log https://www.sendspace.com/file/nxxgg0
>>
>>
>>
>> On Fri, Feb 9, 2018 at 5:33 PM, Paul Angus 
>> wrote:
>>
>> > Hi Jevgeni,
>> >
>> > Can I take you off at a slight tangent...
>> >
>> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
>> > (logid:0afb959d) DataCenter id = '1' provided is in avoid set,
>> > DeploymentPlanner cannot allocate the VM, returning.
>> >
>> > Have you destroyed your system vms so that new 4.11 system vms have been
>> > deployed?
>> >
>> > It may help us if you can paste all of your management log from today
>> into
>> > https://pastebin.com (or similar) to share with us.
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> >
>> > paul.an...@shapeblue.com
>> > www.shapeblue.com
>> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> > @shapeblue
>> >
>> >
>> >
>> >
>> > -Original Message-
>> > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
>> > Sent: 09 February 2018 15:31
>> > To: users@cloudstack.apache.org
>> > Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
>> > 4.11
>> >
>> > the same result
>> >
>> > [root@mtl1-apphst03 management]# virsh list --all
>> >  IdName   State
>> > 
>> >  1 v-1-VM running
>> >  2 s-2-VM running
>> >
>> >
>> >
>> > On Fri, Feb 9, 2018 at 5:28 PM, Daan Hoogland 
>> > wrote:
>> >
>> > >  so try
>> > > # virsh list --all
>> > >
>> > > On Fri, Feb 9, 2018 at 4:27 PM, Jevgeni Zolotarjov
>> > > > > > >
>> > > wrote:
>> > >
>> > > > I guess, these are system VMs.
>> > > > none of my own created instances can start.
>> > > >  IdName   State
>> > > > 
>> > > >  1 v-1-VM running
>> > > >  2 s-2-VM running
>> > > >
>> > > > virsh start  3> gives error
>> > > > error: failed to get domain '8'
>> > > > error: Domain not found: no domain with matching name '8'
>> > > >
>> > > >
>> > > > And yes, cloudstack and mysql - everything locally
>> > > >
>> > > >
>> > > >
>> > > > On Fri, Feb 9, 2018 at 5:15 PM, Daan Hoogland
>> > > > 
>> > > > wrote:
>> > > >
>> > > > > On Fri, Feb 9, 2018 at 3:58 PM, Jevgeni Zolotarjov <
>> > > > j.zolotar...@gmail.com
>> > > > > >
>> > > > > wrote:
>> > > > >
>> > > > > > * how to start instance with virsh?
>> > > > > >
>> > > > > ​ah, usually something like
>> > > > > # virsh start 
>> > > > > or
>> > > > > # virsh start <​number>
>> > > > >
>> > > > >
>> > > > > > * starting in debugger? I would rather not do it :)
>> > > > > >
>> > > > > ​ok, i expected as much. It will slow our solution process
>> > > unfortunately
>> > > > > ​
>> > > > >
>> > > > > ​In the below at the bottom it says you have two VMs and both are
>> > > > running.
>> > > > > Is that correct?​
>> > > > >
>> > > > >
>> > > > > > * this is the log around GetHostStatsAnswer
>> > > > > > 2018-02-09 14:05:30,274 DEBUG [c.c.s.StatsCollector]
>> > > > > > (StatsCollector-4:ctx-167407f0) (logid:0c470e95)
>> > > > > > HostStatsCollector
>> > > is
>> > > > > > running...
>> > > > > > 2018-02-09 14:05:30,322 DEBUG [c.c.a.t.Request]
>> > > > > > (StatsCollector-4:ctx-167407f0) (logid:0c470e95) Seq
>> > > > > 3-613896924205940798:
>> > > > > > Received:  { Ans: , MgmtId: 264216221068220, via:
>> > > > > > 3(mtl1-apphst03),
>> > > > Ver:
>> > > > > > v1, Flags: 10, { GetHostStatsAnswer } }
>> > > > > > 2018-02-09 14:05:31,388 DEBUG [c.c.s.StatsCollector]
>> > > > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
>> > > > > > StorageCollector is running...
>> > > > > > 2018-02-09 14:05:31,396 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
>> > > > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
>> > > > > getCommandHostDelegation:
>> > > > > > class com.cloud.agent.api.GetStorageStatsCommand
>> > > > > > 2018-02-09 14:05:31,396 DEBUG [c.c.h.XenServerGuru]
>> > > > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) We are
>> > > > > > returning
>> > > the
>> > > > > > default host to execute commands because the command is not of

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Daan Hoogland]

Jevgeni,

What Paul means is: Those two VMs are probably the old ones base on the
4.10 systemvm template. These need to be upgraded before you can deploy
anything. The way to do that is to destroy them and let cloudstack recreate
them. They won't allow you to instantiate anything anyway.

so destroy
v-1-VM and
s-2-VM

for instance with virsh

On Fri, Feb 9, 2018 at 4:54 PM, Jevgeni Zolotarjov 
wrote:

> Destroyed VMs? No. Definitely not.
>
> At least I can see them all under web console Home->Instances. Can make
> snapshots even
>
> My todays management server log https://www.sendspace.com/file/nxxgg0
>
>
>
> On Fri, Feb 9, 2018 at 5:33 PM, Paul Angus 
> wrote:
>
> > Hi Jevgeni,
> >
> > Can I take you off at a slight tangent...
> >
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > (logid:0afb959d) DataCenter id = '1' provided is in avoid set,
> > DeploymentPlanner cannot allocate the VM, returning.
> >
> > Have you destroyed your system vms so that new 4.11 system vms have been
> > deployed?
> >
> > It may help us if you can paste all of your management log from today
> into
> > https://pastebin.com (or similar) to share with us.
> >
> >
> >
> >
> >
> >
> >
> >
> >
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com]
> > Sent: 09 February 2018 15:31
> > To: users@cloudstack.apache.org
> > Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> > 4.11
> >
> > the same result
> >
> > [root@mtl1-apphst03 management]# virsh list --all
> >  IdName   State
> > 
> >  1 v-1-VM running
> >  2 s-2-VM running
> >
> >
> >
> > On Fri, Feb 9, 2018 at 5:28 PM, Daan Hoogland 
> > wrote:
> >
> > >  so try
> > > # virsh list --all
> > >
> > > On Fri, Feb 9, 2018 at 4:27 PM, Jevgeni Zolotarjov
> > >  > > >
> > > wrote:
> > >
> > > > I guess, these are system VMs.
> > > > none of my own created instances can start.
> > > >  IdName   State
> > > > 
> > > >  1 v-1-VM running
> > > >  2 s-2-VM running
> > > >
> > > > virsh start  3> gives error
> > > > error: failed to get domain '8'
> > > > error: Domain not found: no domain with matching name '8'
> > > >
> > > >
> > > > And yes, cloudstack and mysql - everything locally
> > > >
> > > >
> > > >
> > > > On Fri, Feb 9, 2018 at 5:15 PM, Daan Hoogland
> > > > 
> > > > wrote:
> > > >
> > > > > On Fri, Feb 9, 2018 at 3:58 PM, Jevgeni Zolotarjov <
> > > > j.zolotar...@gmail.com
> > > > > >
> > > > > wrote:
> > > > >
> > > > > > * how to start instance with virsh?
> > > > > >
> > > > > ​ah, usually something like
> > > > > # virsh start 
> > > > > or
> > > > > # virsh start <​number>
> > > > >
> > > > >
> > > > > > * starting in debugger? I would rather not do it :)
> > > > > >
> > > > > ​ok, i expected as much. It will slow our solution process
> > > unfortunately
> > > > > ​
> > > > >
> > > > > ​In the below at the bottom it says you have two VMs and both are
> > > > running.
> > > > > Is that correct?​
> > > > >
> > > > >
> > > > > > * this is the log around GetHostStatsAnswer
> > > > > > 2018-02-09 14:05:30,274 DEBUG [c.c.s.StatsCollector]
> > > > > > (StatsCollector-4:ctx-167407f0) (logid:0c470e95)
> > > > > > HostStatsCollector
> > > is
> > > > > > running...
> > > > > > 2018-02-09 14:05:30,322 DEBUG [c.c.a.t.Request]
> > > > > > (StatsCollector-4:ctx-167407f0) (logid:0c470e95) Seq
> > > > > 3-613896924205940798:
> > > > > > Received:  { Ans: , MgmtId: 264216221068220, via:
> > > > > > 3(mtl1-apphst03),
> > > > Ver:
> > > > > > v1, Flags: 10, { GetHostStatsAnswer } }
> > > > > > 2018-02-09 14:05:31,388 DEBUG [c.c.s.StatsCollector]
> > > > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
> > > > > > StorageCollector is running...
> > > > > > 2018-02-09 14:05:31,396 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > > > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
> > > > > getCommandHostDelegation:
> > > > > > class com.cloud.agent.api.GetStorageStatsCommand
> > > > > > 2018-02-09 14:05:31,396 DEBUG [c.c.h.XenServerGuru]
> > > > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) We are
> > > > > > returning
> > > the
> > > > > > default host to execute commands because the command is not of
> > > > > > Copy
> > > > type.
> > > > > > 2018-02-09 14:05:31,445 DEBUG [c.c.a.t.Request]
> > > > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) Seq
> > > > > > 4-1941614389350105109:
> > > > > > Received:  { Ans: , MgmtId: 264216221068220, via: 4(s-2-VM),
> > > > > > Ver: 

RE: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Paul Angus]

Hi Jevgeni, 

Can I take you off at a slight tangent...

(Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
(logid:0afb959d) DataCenter id = '1' provided is in avoid set,
DeploymentPlanner cannot allocate the VM, returning.

Have you destroyed your system vms so that new 4.11 system vms have been 
deployed?

It may help us if you can paste all of your management log from today into 
https://pastebin.com (or similar) to share with us.










paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com] 
Sent: 09 February 2018 15:31
To: users@cloudstack.apache.org
Subject: Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

the same result

[root@mtl1-apphst03 management]# virsh list --all
 IdName   State

 1 v-1-VM running
 2 s-2-VM running



On Fri, Feb 9, 2018 at 5:28 PM, Daan Hoogland 
wrote:

>  so try
> # virsh list --all
>
> On Fri, Feb 9, 2018 at 4:27 PM, Jevgeni Zolotarjov 
>  >
> wrote:
>
> > I guess, these are system VMs.
> > none of my own created instances can start.
> >  IdName   State
> > 
> >  1 v-1-VM running
> >  2 s-2-VM running
> >
> > virsh start  3> gives error
> > error: failed to get domain '8'
> > error: Domain not found: no domain with matching name '8'
> >
> >
> > And yes, cloudstack and mysql - everything locally
> >
> >
> >
> > On Fri, Feb 9, 2018 at 5:15 PM, Daan Hoogland 
> > 
> > wrote:
> >
> > > On Fri, Feb 9, 2018 at 3:58 PM, Jevgeni Zolotarjov <
> > j.zolotar...@gmail.com
> > > >
> > > wrote:
> > >
> > > > * how to start instance with virsh?
> > > >
> > > ​ah, usually something like
> > > # virsh start 
> > > or
> > > # virsh start <​number>
> > >
> > >
> > > > * starting in debugger? I would rather not do it :)
> > > >
> > > ​ok, i expected as much. It will slow our solution process
> unfortunately
> > > ​
> > >
> > > ​In the below at the bottom it says you have two VMs and both are
> > running.
> > > Is that correct?​
> > >
> > >
> > > > * this is the log around GetHostStatsAnswer
> > > > 2018-02-09 14:05:30,274 DEBUG [c.c.s.StatsCollector]
> > > > (StatsCollector-4:ctx-167407f0) (logid:0c470e95) 
> > > > HostStatsCollector
> is
> > > > running...
> > > > 2018-02-09 14:05:30,322 DEBUG [c.c.a.t.Request]
> > > > (StatsCollector-4:ctx-167407f0) (logid:0c470e95) Seq
> > > 3-613896924205940798:
> > > > Received:  { Ans: , MgmtId: 264216221068220, via: 
> > > > 3(mtl1-apphst03),
> > Ver:
> > > > v1, Flags: 10, { GetHostStatsAnswer } }
> > > > 2018-02-09 14:05:31,388 DEBUG [c.c.s.StatsCollector]
> > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) 
> > > > StorageCollector is running...
> > > > 2018-02-09 14:05:31,396 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
> > > getCommandHostDelegation:
> > > > class com.cloud.agent.api.GetStorageStatsCommand
> > > > 2018-02-09 14:05:31,396 DEBUG [c.c.h.XenServerGuru]
> > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) We are 
> > > > returning
> the
> > > > default host to execute commands because the command is not of 
> > > > Copy
> > type.
> > > > 2018-02-09 14:05:31,445 DEBUG [c.c.a.t.Request]
> > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) Seq
> > > > 4-1941614389350105109:
> > > > Received:  { Ans: , MgmtId: 264216221068220, via: 4(s-2-VM), 
> > > > Ver: v1,
> > > > Flags: 10, { GetStorageStatsAnswer } }
> > > > 2018-02-09 14:05:31,447 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
> > > getCommandHostDelegation:
> > > > class com.cloud.agent.api.GetStorageStatsCommand
> > > > 2018-02-09 14:05:31,447 DEBUG [c.c.h.XenServerGuru]
> > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) We are 
> > > > returning
> the
> > > > default host to execute commands because the command is not of 
> > > > Copy
> > type.
> > > > 2018-02-09 14:05:31,517 DEBUG [c.c.a.t.Request]
> > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) Seq
> > > 3-613896924205940799:
> > > > Received:  { Ans: , MgmtId: 264216221068220, via: 
> > > > 3(mtl1-apphst03),
> > Ver:
> > > > v1, Flags: 10, { GetStorageStatsAnswer } }
> > > > 2018-02-09 14:05:34,504 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > > (AsyncJobMgr-Heartbeat-1:ctx-6955e300) (logid:300ab196) Begin
> cleanup
> > > > expired async-jobs
> > > > 2018-02-09 14:05:34,506 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > > (AsyncJobMgr-Heartbeat-1:ctx-6955e300) (logid:300ab196) End 
> > > > cleanup expired async-jobs
> > > > 2018-02-09 14:05:35,979 DEBUG 
> > > > 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Jevgeni Zolotarjov]

the same result

[root@mtl1-apphst03 management]# virsh list --all
 IdName   State

 1 v-1-VM running
 2 s-2-VM running



On Fri, Feb 9, 2018 at 5:28 PM, Daan Hoogland 
wrote:

>  so try
> # virsh list --all
>
> On Fri, Feb 9, 2018 at 4:27 PM, Jevgeni Zolotarjov  >
> wrote:
>
> > I guess, these are system VMs.
> > none of my own created instances can start.
> >  IdName   State
> > 
> >  1 v-1-VM running
> >  2 s-2-VM running
> >
> > virsh start  3> gives error
> > error: failed to get domain '8'
> > error: Domain not found: no domain with matching name '8'
> >
> >
> > And yes, cloudstack and mysql - everything locally
> >
> >
> >
> > On Fri, Feb 9, 2018 at 5:15 PM, Daan Hoogland 
> > wrote:
> >
> > > On Fri, Feb 9, 2018 at 3:58 PM, Jevgeni Zolotarjov <
> > j.zolotar...@gmail.com
> > > >
> > > wrote:
> > >
> > > > * how to start instance with virsh?
> > > >
> > > ​ah, usually something like
> > > # virsh start 
> > > or
> > > # virsh start <​number>
> > >
> > >
> > > > * starting in debugger? I would rather not do it :)
> > > >
> > > ​ok, i expected as much. It will slow our solution process
> unfortunately
> > > ​
> > >
> > > ​In the below at the bottom it says you have two VMs and both are
> > running.
> > > Is that correct?​
> > >
> > >
> > > > * this is the log around GetHostStatsAnswer
> > > > 2018-02-09 14:05:30,274 DEBUG [c.c.s.StatsCollector]
> > > > (StatsCollector-4:ctx-167407f0) (logid:0c470e95) HostStatsCollector
> is
> > > > running...
> > > > 2018-02-09 14:05:30,322 DEBUG [c.c.a.t.Request]
> > > > (StatsCollector-4:ctx-167407f0) (logid:0c470e95) Seq
> > > 3-613896924205940798:
> > > > Received:  { Ans: , MgmtId: 264216221068220, via: 3(mtl1-apphst03),
> > Ver:
> > > > v1, Flags: 10, { GetHostStatsAnswer } }
> > > > 2018-02-09 14:05:31,388 DEBUG [c.c.s.StatsCollector]
> > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) StorageCollector is
> > > > running...
> > > > 2018-02-09 14:05:31,396 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
> > > getCommandHostDelegation:
> > > > class com.cloud.agent.api.GetStorageStatsCommand
> > > > 2018-02-09 14:05:31,396 DEBUG [c.c.h.XenServerGuru]
> > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) We are returning
> the
> > > > default host to execute commands because the command is not of Copy
> > type.
> > > > 2018-02-09 14:05:31,445 DEBUG [c.c.a.t.Request]
> > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) Seq
> > > > 4-1941614389350105109:
> > > > Received:  { Ans: , MgmtId: 264216221068220, via: 4(s-2-VM), Ver: v1,
> > > > Flags: 10, { GetStorageStatsAnswer } }
> > > > 2018-02-09 14:05:31,447 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
> > > getCommandHostDelegation:
> > > > class com.cloud.agent.api.GetStorageStatsCommand
> > > > 2018-02-09 14:05:31,447 DEBUG [c.c.h.XenServerGuru]
> > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) We are returning
> the
> > > > default host to execute commands because the command is not of Copy
> > type.
> > > > 2018-02-09 14:05:31,517 DEBUG [c.c.a.t.Request]
> > > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) Seq
> > > 3-613896924205940799:
> > > > Received:  { Ans: , MgmtId: 264216221068220, via: 3(mtl1-apphst03),
> > Ver:
> > > > v1, Flags: 10, { GetStorageStatsAnswer } }
> > > > 2018-02-09 14:05:34,504 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > > (AsyncJobMgr-Heartbeat-1:ctx-6955e300) (logid:300ab196) Begin
> cleanup
> > > > expired async-jobs
> > > > 2018-02-09 14:05:34,506 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > > (AsyncJobMgr-Heartbeat-1:ctx-6955e300) (logid:300ab196) End cleanup
> > > > expired
> > > > async-jobs
> > > > 2018-02-09 14:05:35,979 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> > > > (secstorage-1:ctx-e83be1a3) (logid:60b9de64) Zone 1 is ready to
> launch
> > > > secondary storage VM
> > > > 2018-02-09 14:05:36,002 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> > > > (consoleproxy-1:ctx-508f06d4) (logid:fea1da44) Zone 1 is ready to
> > launch
> > > > console proxy
> > > > 2018-02-09 14:05:36,697 DEBUG [c.c.a.m.AgentManagerImpl]
> > > > (AgentManager-Handler-14:null) (logid:) SeqA 5-273: Processing Seq
> > 5-273:
> > > > { Cmd , MgmtId: -1, via: 5, Ver: v1, Flags: 11,
> > > > [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> > > > :{"_proxyVmId":1,"_loadInfo":"{\n
> > > > \"connections\": []\n}","wait":0}}] }
> > > > 2018-02-09 14:05:36,699 DEBUG [c.c.a.m.AgentManagerImpl]
> > > > (AgentManager-Handler-14:null) (logid:) SeqA 5-273: Sending Seq
> > 5-273:  {
> > > > Ans: , MgmtId: 264216221068220, via: 5, Ver: v1, Flags: 100010,
> 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Daan Hoogland]

 so try
# virsh list --all

On Fri, Feb 9, 2018 at 4:27 PM, Jevgeni Zolotarjov 
wrote:

> I guess, these are system VMs.
> none of my own created instances can start.
>  IdName   State
> 
>  1 v-1-VM running
>  2 s-2-VM running
>
> virsh start  3> gives error
> error: failed to get domain '8'
> error: Domain not found: no domain with matching name '8'
>
>
> And yes, cloudstack and mysql - everything locally
>
>
>
> On Fri, Feb 9, 2018 at 5:15 PM, Daan Hoogland 
> wrote:
>
> > On Fri, Feb 9, 2018 at 3:58 PM, Jevgeni Zolotarjov <
> j.zolotar...@gmail.com
> > >
> > wrote:
> >
> > > * how to start instance with virsh?
> > >
> > ​ah, usually something like
> > # virsh start 
> > or
> > # virsh start <​number>
> >
> >
> > > * starting in debugger? I would rather not do it :)
> > >
> > ​ok, i expected as much. It will slow our solution process unfortunately
> > ​
> >
> > ​In the below at the bottom it says you have two VMs and both are
> running.
> > Is that correct?​
> >
> >
> > > * this is the log around GetHostStatsAnswer
> > > 2018-02-09 14:05:30,274 DEBUG [c.c.s.StatsCollector]
> > > (StatsCollector-4:ctx-167407f0) (logid:0c470e95) HostStatsCollector is
> > > running...
> > > 2018-02-09 14:05:30,322 DEBUG [c.c.a.t.Request]
> > > (StatsCollector-4:ctx-167407f0) (logid:0c470e95) Seq
> > 3-613896924205940798:
> > > Received:  { Ans: , MgmtId: 264216221068220, via: 3(mtl1-apphst03),
> Ver:
> > > v1, Flags: 10, { GetHostStatsAnswer } }
> > > 2018-02-09 14:05:31,388 DEBUG [c.c.s.StatsCollector]
> > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) StorageCollector is
> > > running...
> > > 2018-02-09 14:05:31,396 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
> > getCommandHostDelegation:
> > > class com.cloud.agent.api.GetStorageStatsCommand
> > > 2018-02-09 14:05:31,396 DEBUG [c.c.h.XenServerGuru]
> > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) We are returning the
> > > default host to execute commands because the command is not of Copy
> type.
> > > 2018-02-09 14:05:31,445 DEBUG [c.c.a.t.Request]
> > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) Seq
> > > 4-1941614389350105109:
> > > Received:  { Ans: , MgmtId: 264216221068220, via: 4(s-2-VM), Ver: v1,
> > > Flags: 10, { GetStorageStatsAnswer } }
> > > 2018-02-09 14:05:31,447 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
> > getCommandHostDelegation:
> > > class com.cloud.agent.api.GetStorageStatsCommand
> > > 2018-02-09 14:05:31,447 DEBUG [c.c.h.XenServerGuru]
> > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) We are returning the
> > > default host to execute commands because the command is not of Copy
> type.
> > > 2018-02-09 14:05:31,517 DEBUG [c.c.a.t.Request]
> > > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) Seq
> > 3-613896924205940799:
> > > Received:  { Ans: , MgmtId: 264216221068220, via: 3(mtl1-apphst03),
> Ver:
> > > v1, Flags: 10, { GetStorageStatsAnswer } }
> > > 2018-02-09 14:05:34,504 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (AsyncJobMgr-Heartbeat-1:ctx-6955e300) (logid:300ab196) Begin cleanup
> > > expired async-jobs
> > > 2018-02-09 14:05:34,506 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (AsyncJobMgr-Heartbeat-1:ctx-6955e300) (logid:300ab196) End cleanup
> > > expired
> > > async-jobs
> > > 2018-02-09 14:05:35,979 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> > > (secstorage-1:ctx-e83be1a3) (logid:60b9de64) Zone 1 is ready to launch
> > > secondary storage VM
> > > 2018-02-09 14:05:36,002 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> > > (consoleproxy-1:ctx-508f06d4) (logid:fea1da44) Zone 1 is ready to
> launch
> > > console proxy
> > > 2018-02-09 14:05:36,697 DEBUG [c.c.a.m.AgentManagerImpl]
> > > (AgentManager-Handler-14:null) (logid:) SeqA 5-273: Processing Seq
> 5-273:
> > > { Cmd , MgmtId: -1, via: 5, Ver: v1, Flags: 11,
> > > [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> > > :{"_proxyVmId":1,"_loadInfo":"{\n
> > > \"connections\": []\n}","wait":0}}] }
> > > 2018-02-09 14:05:36,699 DEBUG [c.c.a.m.AgentManagerImpl]
> > > (AgentManager-Handler-14:null) (logid:) SeqA 5-273: Sending Seq
> 5-273:  {
> > > Ans: , MgmtId: 264216221068220, via: 5, Ver: v1, Flags: 100010,
> > > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}]
> }
> > > 2018-02-09 14:05:44,502 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (AsyncJobMgr-Heartbeat-1:ctx-61f44674) (logid:94748b08) Begin cleanup
> > > expired async-jobs
> > > 2018-02-09 14:05:44,505 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> > > (AsyncJobMgr-Heartbeat-1:ctx-61f44674) (logid:94748b08) End cleanup
> > > expired
> > > async-jobs
> > > 2018-02-09 14:05:44,582 DEBUG [c.c.n.r.VirtualNetworkApplianceManager
> > Impl]
> > > (RouterStatusMonitor-1:ctx-9eea60cf) (logid:171472b2) Found 0 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Jevgeni Zolotarjov]

I guess, these are system VMs.
none of my own created instances can start.
 IdName   State

 1 v-1-VM running
 2 s-2-VM running

virsh start  3> gives error
error: failed to get domain '8'
error: Domain not found: no domain with matching name '8'


And yes, cloudstack and mysql - everything locally



On Fri, Feb 9, 2018 at 5:15 PM, Daan Hoogland 
wrote:

> On Fri, Feb 9, 2018 at 3:58 PM, Jevgeni Zolotarjov  >
> wrote:
>
> > * how to start instance with virsh?
> >
> ​ah, usually something like
> # virsh start 
> or
> # virsh start <​number>
>
>
> > * starting in debugger? I would rather not do it :)
> >
> ​ok, i expected as much. It will slow our solution process unfortunately
> ​
>
> ​In the below at the bottom it says you have two VMs and both are running.
> Is that correct?​
>
>
> > * this is the log around GetHostStatsAnswer
> > 2018-02-09 14:05:30,274 DEBUG [c.c.s.StatsCollector]
> > (StatsCollector-4:ctx-167407f0) (logid:0c470e95) HostStatsCollector is
> > running...
> > 2018-02-09 14:05:30,322 DEBUG [c.c.a.t.Request]
> > (StatsCollector-4:ctx-167407f0) (logid:0c470e95) Seq
> 3-613896924205940798:
> > Received:  { Ans: , MgmtId: 264216221068220, via: 3(mtl1-apphst03), Ver:
> > v1, Flags: 10, { GetHostStatsAnswer } }
> > 2018-02-09 14:05:31,388 DEBUG [c.c.s.StatsCollector]
> > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) StorageCollector is
> > running...
> > 2018-02-09 14:05:31,396 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
> getCommandHostDelegation:
> > class com.cloud.agent.api.GetStorageStatsCommand
> > 2018-02-09 14:05:31,396 DEBUG [c.c.h.XenServerGuru]
> > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) We are returning the
> > default host to execute commands because the command is not of Copy type.
> > 2018-02-09 14:05:31,445 DEBUG [c.c.a.t.Request]
> > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) Seq
> > 4-1941614389350105109:
> > Received:  { Ans: , MgmtId: 264216221068220, via: 4(s-2-VM), Ver: v1,
> > Flags: 10, { GetStorageStatsAnswer } }
> > 2018-02-09 14:05:31,447 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> > (StatsCollector-1:ctx-308796ad) (logid:9e87351b)
> getCommandHostDelegation:
> > class com.cloud.agent.api.GetStorageStatsCommand
> > 2018-02-09 14:05:31,447 DEBUG [c.c.h.XenServerGuru]
> > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) We are returning the
> > default host to execute commands because the command is not of Copy type.
> > 2018-02-09 14:05:31,517 DEBUG [c.c.a.t.Request]
> > (StatsCollector-1:ctx-308796ad) (logid:9e87351b) Seq
> 3-613896924205940799:
> > Received:  { Ans: , MgmtId: 264216221068220, via: 3(mtl1-apphst03), Ver:
> > v1, Flags: 10, { GetStorageStatsAnswer } }
> > 2018-02-09 14:05:34,504 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (AsyncJobMgr-Heartbeat-1:ctx-6955e300) (logid:300ab196) Begin cleanup
> > expired async-jobs
> > 2018-02-09 14:05:34,506 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (AsyncJobMgr-Heartbeat-1:ctx-6955e300) (logid:300ab196) End cleanup
> > expired
> > async-jobs
> > 2018-02-09 14:05:35,979 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> > (secstorage-1:ctx-e83be1a3) (logid:60b9de64) Zone 1 is ready to launch
> > secondary storage VM
> > 2018-02-09 14:05:36,002 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> > (consoleproxy-1:ctx-508f06d4) (logid:fea1da44) Zone 1 is ready to launch
> > console proxy
> > 2018-02-09 14:05:36,697 DEBUG [c.c.a.m.AgentManagerImpl]
> > (AgentManager-Handler-14:null) (logid:) SeqA 5-273: Processing Seq 5-273:
> > { Cmd , MgmtId: -1, via: 5, Ver: v1, Flags: 11,
> > [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> > :{"_proxyVmId":1,"_loadInfo":"{\n
> > \"connections\": []\n}","wait":0}}] }
> > 2018-02-09 14:05:36,699 DEBUG [c.c.a.m.AgentManagerImpl]
> > (AgentManager-Handler-14:null) (logid:) SeqA 5-273: Sending Seq 5-273:  {
> > Ans: , MgmtId: 264216221068220, via: 5, Ver: v1, Flags: 100010,
> > [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> > 2018-02-09 14:05:44,502 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (AsyncJobMgr-Heartbeat-1:ctx-61f44674) (logid:94748b08) Begin cleanup
> > expired async-jobs
> > 2018-02-09 14:05:44,505 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> > (AsyncJobMgr-Heartbeat-1:ctx-61f44674) (logid:94748b08) End cleanup
> > expired
> > async-jobs
> > 2018-02-09 14:05:44,582 DEBUG [c.c.n.r.VirtualNetworkApplianceManager
> Impl]
> > (RouterStatusMonitor-1:ctx-9eea60cf) (logid:171472b2) Found 0 routers to
> > update status.
> > 2018-02-09 14:05:44,583 DEBUG [c.c.n.r.VirtualNetworkApplianceManager
> Impl]
> > (RouterStatusMonitor-1:ctx-9eea60cf) (logid:171472b2) Found 0 VPC
> networks
> > to update Redundant State.
> > 2018-02-09 14:05:44,584 DEBUG [c.c.n.r.VirtualNetworkApplianceManager
> Impl]
> > 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Daan Hoogland]

On Fri, Feb 9, 2018 at 3:58 PM, Jevgeni Zolotarjov 
wrote:

> * how to start instance with virsh?
>
​ah, usually something like
# virsh start 
or
# virsh start <​number>


> * starting in debugger? I would rather not do it :)
>
​ok, i expected as much. It will slow our solution process unfortunately
​

​In the below at the bottom it says you have two VMs and both are running.
Is that correct?​


> * this is the log around GetHostStatsAnswer
> 2018-02-09 14:05:30,274 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-4:ctx-167407f0) (logid:0c470e95) HostStatsCollector is
> running...
> 2018-02-09 14:05:30,322 DEBUG [c.c.a.t.Request]
> (StatsCollector-4:ctx-167407f0) (logid:0c470e95) Seq 3-613896924205940798:
> Received:  { Ans: , MgmtId: 264216221068220, via: 3(mtl1-apphst03), Ver:
> v1, Flags: 10, { GetHostStatsAnswer } }
> 2018-02-09 14:05:31,388 DEBUG [c.c.s.StatsCollector]
> (StatsCollector-1:ctx-308796ad) (logid:9e87351b) StorageCollector is
> running...
> 2018-02-09 14:05:31,396 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> (StatsCollector-1:ctx-308796ad) (logid:9e87351b) getCommandHostDelegation:
> class com.cloud.agent.api.GetStorageStatsCommand
> 2018-02-09 14:05:31,396 DEBUG [c.c.h.XenServerGuru]
> (StatsCollector-1:ctx-308796ad) (logid:9e87351b) We are returning the
> default host to execute commands because the command is not of Copy type.
> 2018-02-09 14:05:31,445 DEBUG [c.c.a.t.Request]
> (StatsCollector-1:ctx-308796ad) (logid:9e87351b) Seq
> 4-1941614389350105109:
> Received:  { Ans: , MgmtId: 264216221068220, via: 4(s-2-VM), Ver: v1,
> Flags: 10, { GetStorageStatsAnswer } }
> 2018-02-09 14:05:31,447 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
> (StatsCollector-1:ctx-308796ad) (logid:9e87351b) getCommandHostDelegation:
> class com.cloud.agent.api.GetStorageStatsCommand
> 2018-02-09 14:05:31,447 DEBUG [c.c.h.XenServerGuru]
> (StatsCollector-1:ctx-308796ad) (logid:9e87351b) We are returning the
> default host to execute commands because the command is not of Copy type.
> 2018-02-09 14:05:31,517 DEBUG [c.c.a.t.Request]
> (StatsCollector-1:ctx-308796ad) (logid:9e87351b) Seq 3-613896924205940799:
> Received:  { Ans: , MgmtId: 264216221068220, via: 3(mtl1-apphst03), Ver:
> v1, Flags: 10, { GetStorageStatsAnswer } }
> 2018-02-09 14:05:34,504 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-6955e300) (logid:300ab196) Begin cleanup
> expired async-jobs
> 2018-02-09 14:05:34,506 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-6955e300) (logid:300ab196) End cleanup
> expired
> async-jobs
> 2018-02-09 14:05:35,979 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
> (secstorage-1:ctx-e83be1a3) (logid:60b9de64) Zone 1 is ready to launch
> secondary storage VM
> 2018-02-09 14:05:36,002 DEBUG [c.c.c.ConsoleProxyManagerImpl]
> (consoleproxy-1:ctx-508f06d4) (logid:fea1da44) Zone 1 is ready to launch
> console proxy
> 2018-02-09 14:05:36,697 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-14:null) (logid:) SeqA 5-273: Processing Seq 5-273:
> { Cmd , MgmtId: -1, via: 5, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> :{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2018-02-09 14:05:36,699 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-14:null) (logid:) SeqA 5-273: Sending Seq 5-273:  {
> Ans: , MgmtId: 264216221068220, via: 5, Ver: v1, Flags: 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2018-02-09 14:05:44,502 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-61f44674) (logid:94748b08) Begin cleanup
> expired async-jobs
> 2018-02-09 14:05:44,505 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
> (AsyncJobMgr-Heartbeat-1:ctx-61f44674) (logid:94748b08) End cleanup
> expired
> async-jobs
> 2018-02-09 14:05:44,582 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-9eea60cf) (logid:171472b2) Found 0 routers to
> update status.
> 2018-02-09 14:05:44,583 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-9eea60cf) (logid:171472b2) Found 0 VPC networks
> to update Redundant State.
> 2018-02-09 14:05:44,584 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
> (RouterStatusMonitor-1:ctx-9eea60cf) (logid:171472b2) Found 0 networks to
> update RvR status.
> 2018-02-09 14:05:46,700 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-15:null) (logid:) SeqA 5-274: Processing Seq 5-274:
> { Cmd , MgmtId: -1, via: 5, Ver: v1, Flags: 11,
> [{"com.cloud.agent.api.ConsoleProxyLoadReportCommand"
> :{"_proxyVmId":1,"_loadInfo":"{\n
> \"connections\": []\n}","wait":0}}] }
> 2018-02-09 14:05:46,702 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentManager-Handler-15:null) (logid:) SeqA 5-274: Sending Seq 5-274:  {
> Ans: , MgmtId: 264216221068220, via: 5, Ver: v1, Flags: 100010,
> [{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
> 2018-02-09 14:05:49,610 DEBUG [c.c.h.d.HostDaoImpl] 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Jevgeni Zolotarjov]

* how to start instance with virsh?

* starting in debugger? I would rather not do it :)

* this is the log around GetHostStatsAnswer
2018-02-09 14:05:30,274 DEBUG [c.c.s.StatsCollector]
(StatsCollector-4:ctx-167407f0) (logid:0c470e95) HostStatsCollector is
running...
2018-02-09 14:05:30,322 DEBUG [c.c.a.t.Request]
(StatsCollector-4:ctx-167407f0) (logid:0c470e95) Seq 3-613896924205940798:
Received:  { Ans: , MgmtId: 264216221068220, via: 3(mtl1-apphst03), Ver:
v1, Flags: 10, { GetHostStatsAnswer } }
2018-02-09 14:05:31,388 DEBUG [c.c.s.StatsCollector]
(StatsCollector-1:ctx-308796ad) (logid:9e87351b) StorageCollector is
running...
2018-02-09 14:05:31,396 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(StatsCollector-1:ctx-308796ad) (logid:9e87351b) getCommandHostDelegation:
class com.cloud.agent.api.GetStorageStatsCommand
2018-02-09 14:05:31,396 DEBUG [c.c.h.XenServerGuru]
(StatsCollector-1:ctx-308796ad) (logid:9e87351b) We are returning the
default host to execute commands because the command is not of Copy type.
2018-02-09 14:05:31,445 DEBUG [c.c.a.t.Request]
(StatsCollector-1:ctx-308796ad) (logid:9e87351b) Seq 4-1941614389350105109:
Received:  { Ans: , MgmtId: 264216221068220, via: 4(s-2-VM), Ver: v1,
Flags: 10, { GetStorageStatsAnswer } }
2018-02-09 14:05:31,447 DEBUG [c.c.h.o.r.Ovm3HypervisorGuru]
(StatsCollector-1:ctx-308796ad) (logid:9e87351b) getCommandHostDelegation:
class com.cloud.agent.api.GetStorageStatsCommand
2018-02-09 14:05:31,447 DEBUG [c.c.h.XenServerGuru]
(StatsCollector-1:ctx-308796ad) (logid:9e87351b) We are returning the
default host to execute commands because the command is not of Copy type.
2018-02-09 14:05:31,517 DEBUG [c.c.a.t.Request]
(StatsCollector-1:ctx-308796ad) (logid:9e87351b) Seq 3-613896924205940799:
Received:  { Ans: , MgmtId: 264216221068220, via: 3(mtl1-apphst03), Ver:
v1, Flags: 10, { GetStorageStatsAnswer } }
2018-02-09 14:05:34,504 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-6955e300) (logid:300ab196) Begin cleanup
expired async-jobs
2018-02-09 14:05:34,506 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-6955e300) (logid:300ab196) End cleanup expired
async-jobs
2018-02-09 14:05:35,979 DEBUG [o.a.c.s.SecondaryStorageManagerImpl]
(secstorage-1:ctx-e83be1a3) (logid:60b9de64) Zone 1 is ready to launch
secondary storage VM
2018-02-09 14:05:36,002 DEBUG [c.c.c.ConsoleProxyManagerImpl]
(consoleproxy-1:ctx-508f06d4) (logid:fea1da44) Zone 1 is ready to launch
console proxy
2018-02-09 14:05:36,697 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-14:null) (logid:) SeqA 5-273: Processing Seq 5-273:
{ Cmd , MgmtId: -1, via: 5, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2018-02-09 14:05:36,699 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-14:null) (logid:) SeqA 5-273: Sending Seq 5-273:  {
Ans: , MgmtId: 264216221068220, via: 5, Ver: v1, Flags: 100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2018-02-09 14:05:44,502 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-61f44674) (logid:94748b08) Begin cleanup
expired async-jobs
2018-02-09 14:05:44,505 INFO  [o.a.c.f.j.i.AsyncJobManagerImpl]
(AsyncJobMgr-Heartbeat-1:ctx-61f44674) (logid:94748b08) End cleanup expired
async-jobs
2018-02-09 14:05:44,582 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-9eea60cf) (logid:171472b2) Found 0 routers to
update status.
2018-02-09 14:05:44,583 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-9eea60cf) (logid:171472b2) Found 0 VPC networks
to update Redundant State.
2018-02-09 14:05:44,584 DEBUG [c.c.n.r.VirtualNetworkApplianceManagerImpl]
(RouterStatusMonitor-1:ctx-9eea60cf) (logid:171472b2) Found 0 networks to
update RvR status.
2018-02-09 14:05:46,700 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-15:null) (logid:) SeqA 5-274: Processing Seq 5-274:
{ Cmd , MgmtId: -1, via: 5, Ver: v1, Flags: 11,
[{"com.cloud.agent.api.ConsoleProxyLoadReportCommand":{"_proxyVmId":1,"_loadInfo":"{\n
\"connections\": []\n}","wait":0}}] }
2018-02-09 14:05:46,702 DEBUG [c.c.a.m.AgentManagerImpl]
(AgentManager-Handler-15:null) (logid:) SeqA 5-274: Sending Seq 5-274:  {
Ans: , MgmtId: 264216221068220, via: 5, Ver: v1, Flags: 100010,
[{"com.cloud.agent.api.AgentControlAnswer":{"result":true,"wait":0}}] }
2018-02-09 14:05:49,610 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-6233f81e) (logid:42d0cd7b) Resetting hosts suitable for reconnect
2018-02-09 14:05:49,611 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-6233f81e) (logid:42d0cd7b) Completed resetting hosts suitable for
reconnect
2018-02-09 14:05:49,611 DEBUG [c.c.h.d.HostDaoImpl] (ClusteredAgentManager
Timer:ctx-6233f81e) (logid:42d0cd7b) Acquiring hosts for clusters already
owned by this management server
2018-02-09 14:05:49,611 DEBUG [c.c.h.d.HostDaoImpl] 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Daan Hoogland]

and with cloudstack (and mysql) running locally, right?
So my next step would be to stop cloudstack and see if you can start the
image with virsh?
Cloudstack thinks for some reason the host is not a suitable target. I have
no clue why that is from your log. You can start cloudstack in a debugger
but given you are asking on users@, i don't think you are familiar with
that kind of work are, you?
You might also want to look in the management (and agent) log to see if you
can find anything about the capacity reported. Look for GetHostStatsAnswer.

On Fri, Feb 9, 2018 at 3:11 PM, Jevgeni Zolotarjov 
wrote:

> Yes, its KVM
>
> On Fri, Feb 9, 2018 at 4:06 PM, Daan Hoogland 
> wrote:
>
> > I am at a loss but really would like to help you.
> > is it a KVM with the management server running locally?
> >
> > On Fri, Feb 9, 2018 at 3:02 PM, Jevgeni Zolotarjov <
> j.zolotar...@gmail.com
> > >
> > wrote:
> >
> > > My Host is only 1 machine atm:
> > > Dell PowerEdge610
> > > OS: CentOS7 (latest)
> > > CPU: 2x12 cores (24 cores in total)
> > > RAM: 192 GB
> > > Storage: 3+ TB
>


-- 
Daan

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Jevgeni Zolotarjov]

Yes, its KVM

On Fri, Feb 9, 2018 at 4:06 PM, Daan Hoogland 
wrote:

> I am at a loss but really would like to help you.
> is it a KVM with the management server running locally?
>
> On Fri, Feb 9, 2018 at 3:02 PM, Jevgeni Zolotarjov  >
> wrote:
>
> > My Host is only 1 machine atm:
> > Dell PowerEdge610
> > OS: CentOS7 (latest)
> > CPU: 2x12 cores (24 cores in total)
> > RAM: 192 GB
> > Storage: 3+ TB
> >
> > I was running cloudstack-4.10 without problems since it was released
> >
> > Your help is appreciated
> >
> > On Fri, Feb 9, 2018 at 3:57 PM, Daan Hoogland 
> > wrote:
> >
> > > Jevgeni,
> > >
> > > can you describe your environment in more detail? the host[3] that is
> in
> > > the avoid list, is it the host that VM was/is originally running on? Is
> > > anything running on it atm? what is it's distribution and version?
> > etc
> > >
> > > On Fri, Feb 9, 2018 at 2:44 PM, Jevgeni Zolotarjov <
> > j.zolotar...@gmail.com
> > > >
> > > wrote:
> > >
> > > > To follow up:
> > > > this is what I have in log:
> > > > 2018-02-09 13:31:48,836 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > (logid:0afb959d) DeploymentPlanner allocation algorithm: null
> > > > 2018-02-09 13:31:48,836 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > (logid:0afb959d) Trying to allocate a host and storage pools from
> dc:1,
> > > > pod:1,cluster:null, requested cpu: 500, requested ram: 2147483648
> > > > 2018-02-09 13:31:48,836 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > (logid:0afb959d) Is ROOT volume READY (pool already allocated)?: No
> > > > 2018-02-09 13:31:48,840 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > (logid:0afb959d) Deploy avoids pods: [], clusters: [], hosts: [3]
> > > > 2018-02-09 13:31:48,840 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > (logid:0afb959d) DataCenter id = '1' provided is in avoid set,
> > > > DeploymentPlanner cannot allocate the VM, returning.
> > > > 2018-02-09 13:31:48,843 DEBUG [c.c.c.CapacityManagerImpl]
> > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > (logid:0afb959d) VM state transitted from :Starting to Stopped with
> > > event:
> > > > OperationFailedvm's original host id: 3 new host id: null host id
> > before
> > > > state transition: 3
> > > > 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > (logid:0afb959d) Hosts's actual total CPU: 70224 and CPU after
> applying
> > > > overprovisioning: 70224
> > > > 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > (logid:0afb959d) Hosts's actual total RAM: 205071147008 and RAM after
> > > > applying overprovisioning: 205071155200
> > > > 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > (logid:0afb959d) release cpu from host: 3, old used: 1500,reserved:
> > > 26100,
> > > > actual total: 70224, total with overprovisioning: 70224; new used:
> > > > 1000,reserved:26100; movedfromreserved: false,moveToReserveredfalse
> > > > 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > (logid:0afb959d) release mem from host: 3, old used:
> > 3758096384,reserved:
> > > > 81335943168, total: 205071155200; new used:
> > > > 1610612736,reserved:81335943168; movedfromreserved:
> > > > false,moveToReserveredfalse
> > > > 2018-02-09 13:31:48,854 ERROR [c.c.v.VmWorkJobHandlerProxy]
> > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > (logid:0afb959d) Invocation exception, caused by:
> > > > com.cloud.exception.InsufficientServerCapacityException: Unable to
> > > create
> > > > a
> > > > deployment for VM[User|i-2-11-VM]Scope=interface
> > > com.cloud.dc.DataCenter;
> > > > id=1
> > > > 2018-02-09 13:31:48,854 INFO  [c.c.v.VmWorkJobHandlerProxy]
> > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > > (logid:0afb959d) Rethrow exception
> > > > com.cloud.exception.InsufficientServerCapacityException: Unable to
> > > create
> > > > a
> > > > deployment for VM[User|i-2-11-VM]Scope=interface
> > > com.cloud.dc.DataCenter;
> > > > id=1
> > > > 2018-02-09 13:31:48,854 DEBUG [c.c.v.VmWorkJobDispatcher]
> > > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896) (logid:0afb959d)
> > Done
> > > > with run of VM work job: com.cloud.vm.VmWorkStart for VM 11, job
> > origin:
> > > > 893
> > > > 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Daan Hoogland]

I am at a loss but really would like to help you.
is it a KVM with the management server running locally?

On Fri, Feb 9, 2018 at 3:02 PM, Jevgeni Zolotarjov 
wrote:

> My Host is only 1 machine atm:
> Dell PowerEdge610
> OS: CentOS7 (latest)
> CPU: 2x12 cores (24 cores in total)
> RAM: 192 GB
> Storage: 3+ TB
>
> I was running cloudstack-4.10 without problems since it was released
>
> Your help is appreciated
>
> On Fri, Feb 9, 2018 at 3:57 PM, Daan Hoogland 
> wrote:
>
> > Jevgeni,
> >
> > can you describe your environment in more detail? the host[3] that is in
> > the avoid list, is it the host that VM was/is originally running on? Is
> > anything running on it atm? what is it's distribution and version?
> etc
> >
> > On Fri, Feb 9, 2018 at 2:44 PM, Jevgeni Zolotarjov <
> j.zolotar...@gmail.com
> > >
> > wrote:
> >
> > > To follow up:
> > > this is what I have in log:
> > > 2018-02-09 13:31:48,836 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > (logid:0afb959d) DeploymentPlanner allocation algorithm: null
> > > 2018-02-09 13:31:48,836 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > (logid:0afb959d) Trying to allocate a host and storage pools from dc:1,
> > > pod:1,cluster:null, requested cpu: 500, requested ram: 2147483648
> > > 2018-02-09 13:31:48,836 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > (logid:0afb959d) Is ROOT volume READY (pool already allocated)?: No
> > > 2018-02-09 13:31:48,840 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > (logid:0afb959d) Deploy avoids pods: [], clusters: [], hosts: [3]
> > > 2018-02-09 13:31:48,840 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > (logid:0afb959d) DataCenter id = '1' provided is in avoid set,
> > > DeploymentPlanner cannot allocate the VM, returning.
> > > 2018-02-09 13:31:48,843 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > (logid:0afb959d) VM state transitted from :Starting to Stopped with
> > event:
> > > OperationFailedvm's original host id: 3 new host id: null host id
> before
> > > state transition: 3
> > > 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > (logid:0afb959d) Hosts's actual total CPU: 70224 and CPU after applying
> > > overprovisioning: 70224
> > > 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > (logid:0afb959d) Hosts's actual total RAM: 205071147008 and RAM after
> > > applying overprovisioning: 205071155200
> > > 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > (logid:0afb959d) release cpu from host: 3, old used: 1500,reserved:
> > 26100,
> > > actual total: 70224, total with overprovisioning: 70224; new used:
> > > 1000,reserved:26100; movedfromreserved: false,moveToReserveredfalse
> > > 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > (logid:0afb959d) release mem from host: 3, old used:
> 3758096384,reserved:
> > > 81335943168, total: 205071155200; new used:
> > > 1610612736,reserved:81335943168; movedfromreserved:
> > > false,moveToReserveredfalse
> > > 2018-02-09 13:31:48,854 ERROR [c.c.v.VmWorkJobHandlerProxy]
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > (logid:0afb959d) Invocation exception, caused by:
> > > com.cloud.exception.InsufficientServerCapacityException: Unable to
> > create
> > > a
> > > deployment for VM[User|i-2-11-VM]Scope=interface
> > com.cloud.dc.DataCenter;
> > > id=1
> > > 2018-02-09 13:31:48,854 INFO  [c.c.v.VmWorkJobHandlerProxy]
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > > (logid:0afb959d) Rethrow exception
> > > com.cloud.exception.InsufficientServerCapacityException: Unable to
> > create
> > > a
> > > deployment for VM[User|i-2-11-VM]Scope=interface
> > com.cloud.dc.DataCenter;
> > > id=1
> > > 2018-02-09 13:31:48,854 DEBUG [c.c.v.VmWorkJobDispatcher]
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896) (logid:0afb959d)
> Done
> > > with run of VM work job: com.cloud.vm.VmWorkStart for VM 11, job
> origin:
> > > 893
> > > 2018-02-09 13:31:48,854 ERROR [c.c.v.VmWorkJobDispatcher]
> > > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896) (logid:0afb959d)
> > Unable
> > > to complete AsyncJobVO {id:896, userId: 2, accountId: 2, instanceType:
> > > null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
> > > 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Jevgeni Zolotarjov]

My Host is only 1 machine atm:
Dell PowerEdge610
OS: CentOS7 (latest)
CPU: 2x12 cores (24 cores in total)
RAM: 192 GB
Storage: 3+ TB

I was running cloudstack-4.10 without problems since it was released

Your help is appreciated

On Fri, Feb 9, 2018 at 3:57 PM, Daan Hoogland 
wrote:

> Jevgeni,
>
> can you describe your environment in more detail? the host[3] that is in
> the avoid list, is it the host that VM was/is originally running on? Is
> anything running on it atm? what is it's distribution and version? etc
>
> On Fri, Feb 9, 2018 at 2:44 PM, Jevgeni Zolotarjov  >
> wrote:
>
> > To follow up:
> > this is what I have in log:
> > 2018-02-09 13:31:48,836 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > (logid:0afb959d) DeploymentPlanner allocation algorithm: null
> > 2018-02-09 13:31:48,836 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > (logid:0afb959d) Trying to allocate a host and storage pools from dc:1,
> > pod:1,cluster:null, requested cpu: 500, requested ram: 2147483648
> > 2018-02-09 13:31:48,836 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > (logid:0afb959d) Is ROOT volume READY (pool already allocated)?: No
> > 2018-02-09 13:31:48,840 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > (logid:0afb959d) Deploy avoids pods: [], clusters: [], hosts: [3]
> > 2018-02-09 13:31:48,840 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > (logid:0afb959d) DataCenter id = '1' provided is in avoid set,
> > DeploymentPlanner cannot allocate the VM, returning.
> > 2018-02-09 13:31:48,843 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > (logid:0afb959d) VM state transitted from :Starting to Stopped with
> event:
> > OperationFailedvm's original host id: 3 new host id: null host id before
> > state transition: 3
> > 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > (logid:0afb959d) Hosts's actual total CPU: 70224 and CPU after applying
> > overprovisioning: 70224
> > 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > (logid:0afb959d) Hosts's actual total RAM: 205071147008 and RAM after
> > applying overprovisioning: 205071155200
> > 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > (logid:0afb959d) release cpu from host: 3, old used: 1500,reserved:
> 26100,
> > actual total: 70224, total with overprovisioning: 70224; new used:
> > 1000,reserved:26100; movedfromreserved: false,moveToReserveredfalse
> > 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > (logid:0afb959d) release mem from host: 3, old used: 3758096384,reserved:
> > 81335943168, total: 205071155200; new used:
> > 1610612736,reserved:81335943168; movedfromreserved:
> > false,moveToReserveredfalse
> > 2018-02-09 13:31:48,854 ERROR [c.c.v.VmWorkJobHandlerProxy]
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > (logid:0afb959d) Invocation exception, caused by:
> > com.cloud.exception.InsufficientServerCapacityException: Unable to
> create
> > a
> > deployment for VM[User|i-2-11-VM]Scope=interface
> com.cloud.dc.DataCenter;
> > id=1
> > 2018-02-09 13:31:48,854 INFO  [c.c.v.VmWorkJobHandlerProxy]
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> > (logid:0afb959d) Rethrow exception
> > com.cloud.exception.InsufficientServerCapacityException: Unable to
> create
> > a
> > deployment for VM[User|i-2-11-VM]Scope=interface
> com.cloud.dc.DataCenter;
> > id=1
> > 2018-02-09 13:31:48,854 DEBUG [c.c.v.VmWorkJobDispatcher]
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896) (logid:0afb959d) Done
> > with run of VM work job: com.cloud.vm.VmWorkStart for VM 11, job origin:
> > 893
> > 2018-02-09 13:31:48,854 ERROR [c.c.v.VmWorkJobDispatcher]
> > (Work-Job-Executor-7:ctx-a92d467b job-893/job-896) (logid:0afb959d)
> Unable
> > to complete AsyncJobVO {id:896, userId: 2, accountId: 2, instanceType:
> > null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
> > rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oA
> > BGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50
> > UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5n
> > L0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYX
> > ZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-
> > AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS
> > 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Daan Hoogland]

Jevgeni,

can you describe your environment in more detail? the host[3] that is in
the avoid list, is it the host that VM was/is originally running on? Is
anything running on it atm? what is it's distribution and version? etc

On Fri, Feb 9, 2018 at 2:44 PM, Jevgeni Zolotarjov 
wrote:

> To follow up:
> this is what I have in log:
> 2018-02-09 13:31:48,836 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> (logid:0afb959d) DeploymentPlanner allocation algorithm: null
> 2018-02-09 13:31:48,836 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> (logid:0afb959d) Trying to allocate a host and storage pools from dc:1,
> pod:1,cluster:null, requested cpu: 500, requested ram: 2147483648
> 2018-02-09 13:31:48,836 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> (logid:0afb959d) Is ROOT volume READY (pool already allocated)?: No
> 2018-02-09 13:31:48,840 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> (logid:0afb959d) Deploy avoids pods: [], clusters: [], hosts: [3]
> 2018-02-09 13:31:48,840 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
> (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> (logid:0afb959d) DataCenter id = '1' provided is in avoid set,
> DeploymentPlanner cannot allocate the VM, returning.
> 2018-02-09 13:31:48,843 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> (logid:0afb959d) VM state transitted from :Starting to Stopped with event:
> OperationFailedvm's original host id: 3 new host id: null host id before
> state transition: 3
> 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> (logid:0afb959d) Hosts's actual total CPU: 70224 and CPU after applying
> overprovisioning: 70224
> 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> (logid:0afb959d) Hosts's actual total RAM: 205071147008 and RAM after
> applying overprovisioning: 205071155200
> 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> (logid:0afb959d) release cpu from host: 3, old used: 1500,reserved: 26100,
> actual total: 70224, total with overprovisioning: 70224; new used:
> 1000,reserved:26100; movedfromreserved: false,moveToReserveredfalse
> 2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
> (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> (logid:0afb959d) release mem from host: 3, old used: 3758096384,reserved:
> 81335943168, total: 205071155200; new used:
> 1610612736,reserved:81335943168; movedfromreserved:
> false,moveToReserveredfalse
> 2018-02-09 13:31:48,854 ERROR [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> (logid:0afb959d) Invocation exception, caused by:
> com.cloud.exception.InsufficientServerCapacityException: Unable to create
> a
> deployment for VM[User|i-2-11-VM]Scope=interface com.cloud.dc.DataCenter;
> id=1
> 2018-02-09 13:31:48,854 INFO  [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
> (logid:0afb959d) Rethrow exception
> com.cloud.exception.InsufficientServerCapacityException: Unable to create
> a
> deployment for VM[User|i-2-11-VM]Scope=interface com.cloud.dc.DataCenter;
> id=1
> 2018-02-09 13:31:48,854 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-a92d467b job-893/job-896) (logid:0afb959d) Done
> with run of VM work job: com.cloud.vm.VmWorkStart for VM 11, job origin:
> 893
> 2018-02-09 13:31:48,854 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-7:ctx-a92d467b job-893/job-896) (logid:0afb959d) Unable
> to complete AsyncJobVO {id:896, userId: 2, accountId: 2, instanceType:
> null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
> rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oA
> BGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50
> UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5n
> L0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYX
> ZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-
> AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS
> 91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-
> ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-
> AAN4cAACAAIAC3QAGVZpcnR1YWxNYWNoaW5l
> TWFuYWdlckltcGwAAHBwcHBwcHBwc3IAEWphdmEudXRpbC5IYXNo
> TWFwBQfawcMWYNEDAAJGAApsb2FkRmFjdG9ySQAJdGhyZXNob2xkeHA_
> QAAADHcIEAF0AApWbVBhc3N3b3JkdAAcck8wQUJYUUFEbk5o
> ZG1Wa1gzQmhjM04zYjNKa3hw,
> cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
> 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Jevgeni Zolotarjov]

To follow up:
this is what I have in log:
2018-02-09 13:31:48,836 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
(logid:0afb959d) DeploymentPlanner allocation algorithm: null
2018-02-09 13:31:48,836 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
(logid:0afb959d) Trying to allocate a host and storage pools from dc:1,
pod:1,cluster:null, requested cpu: 500, requested ram: 2147483648
2018-02-09 13:31:48,836 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
(logid:0afb959d) Is ROOT volume READY (pool already allocated)?: No
2018-02-09 13:31:48,840 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
(logid:0afb959d) Deploy avoids pods: [], clusters: [], hosts: [3]
2018-02-09 13:31:48,840 DEBUG [c.c.d.DeploymentPlanningManagerImpl]
(Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
(logid:0afb959d) DataCenter id = '1' provided is in avoid set,
DeploymentPlanner cannot allocate the VM, returning.
2018-02-09 13:31:48,843 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
(logid:0afb959d) VM state transitted from :Starting to Stopped with event:
OperationFailedvm's original host id: 3 new host id: null host id before
state transition: 3
2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
(logid:0afb959d) Hosts's actual total CPU: 70224 and CPU after applying
overprovisioning: 70224
2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
(logid:0afb959d) Hosts's actual total RAM: 205071147008 and RAM after
applying overprovisioning: 205071155200
2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
(logid:0afb959d) release cpu from host: 3, old used: 1500,reserved: 26100,
actual total: 70224, total with overprovisioning: 70224; new used:
1000,reserved:26100; movedfromreserved: false,moveToReserveredfalse
2018-02-09 13:31:48,847 DEBUG [c.c.c.CapacityManagerImpl]
(Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
(logid:0afb959d) release mem from host: 3, old used: 3758096384,reserved:
81335943168, total: 205071155200; new used:
1610612736,reserved:81335943168; movedfromreserved:
false,moveToReserveredfalse
2018-02-09 13:31:48,854 ERROR [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
(logid:0afb959d) Invocation exception, caused by:
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[User|i-2-11-VM]Scope=interface com.cloud.dc.DataCenter;
id=1
2018-02-09 13:31:48,854 INFO  [c.c.v.VmWorkJobHandlerProxy]
(Work-Job-Executor-7:ctx-a92d467b job-893/job-896 ctx-9ffd0cf1)
(logid:0afb959d) Rethrow exception
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[User|i-2-11-VM]Scope=interface com.cloud.dc.DataCenter;
id=1
2018-02-09 13:31:48,854 DEBUG [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-7:ctx-a92d467b job-893/job-896) (logid:0afb959d) Done
with run of VM work job: com.cloud.vm.VmWorkStart for VM 11, job origin: 893
2018-02-09 13:31:48,854 ERROR [c.c.v.VmWorkJobDispatcher]
(Work-Job-Executor-7:ctx-a92d467b job-893/job-896) (logid:0afb959d) Unable
to complete AsyncJobVO {id:896, userId: 2, accountId: 2, instanceType:
null, instanceId: null, cmd: com.cloud.vm.VmWorkStart, cmdInfo:
rO0ABXNyABhjb20uY2xvdWQudm0uVm1Xb3JrU3RhcnR9cMGsvxz73gIAC0oABGRjSWRMAAZhdm9pZHN0ADBMY29tL2Nsb3VkL2RlcGxveS9EZXBsb3ltZW50UGxhbm5lciRFeGNsdWRlTGlzdDtMAAljbHVzdGVySWR0ABBMamF2YS9sYW5nL0xvbmc7TAAGaG9zdElkcQB-AAJMAAtqb3VybmFsTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO0wAEXBoeXNpY2FsTmV0d29ya0lkcQB-AAJMAAdwbGFubmVycQB-AANMAAVwb2RJZHEAfgACTAAGcG9vbElkcQB-AAJMAAlyYXdQYXJhbXN0AA9MamF2YS91dGlsL01hcDtMAA1yZXNlcnZhdGlvbklkcQB-AAN4cgATY29tLmNsb3VkLnZtLlZtV29ya5-ZtlbwJWdrAgAESgAJYWNjb3VudElkSgAGdXNlcklkSgAEdm1JZEwAC2hhbmRsZXJOYW1lcQB-AAN4cAACAAIAC3QAGVZpcnR1YWxNYWNoaW5lTWFuYWdlckltcGwAAHBwcHBwcHBwc3IAEWphdmEudXRpbC5IYXNoTWFwBQfawcMWYNEDAAJGAApsb2FkRmFjdG9ySQAJdGhyZXNob2xkeHA_QAAADHcIEAF0AApWbVBhc3N3b3JkdAAcck8wQUJYUUFEbk5oZG1Wa1gzQmhjM04zYjNKa3hw,
cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0,
result: null, initMsid: 264216221068220, completeMsid: null, lastUpdated:
null, lastPolled: null, created: Fri Feb 09 13:31:46 GMT 2018}, job
origin:893
com.cloud.exception.InsufficientServerCapacityException: Unable to create a
deployment for VM[User|i-2-11-VM]Scope=interface com.cloud.dc.DataCenter;
id=1
at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1072)
at

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Jevgeni Zolotarjov]

OK, I pushed through and got management server running.
The steps I made:
* recovered DB from backup
* downgraded everything 4.11->4.10
* installed systemvmtemplates
* upgraded 4.10 -> 4.11

THEN!!! To make it run I had to go through all manual DB altering. Meaning,
there is definitely something wrong in shapeblue packages. I could not
locate the necessary DB scripts in the package. So I fetched an update from
github repo.

Even then it failed to start and I had to do some more DB altering from
file
https://github.com/apache/cloudstack/blob/4.11/engine/schema/resources/META-INF/db/schema-4930to41000.sql
Namely:
ALTER TABLE `cloud`.`sslcerts` ADD COLUMN `name` varchar(255) NULL default
NULL COMMENT 'Name of the Certificate';
ALTER TABLE `cloud`.`network_offerings` ADD COLUMN `service_package_id`
varchar(255) NULL default NULL COMMENT 'Netscaler ControlCenter Service
Package';

OK. Here I am.

BUT! Not all my VMs are starting. On attempt to start them, manually, it
says

"Unable to start a VM due to insufficient capacity"

OK, I am having VMs taking more cores, than actual physical number of
cores.
VM cores - 35,
physical cores - 24

It was not the problem with previous version 4.10 - and it was correct.  I
want to decide myself if I want to allow cpu overprovisioning.

I consider this a critical bug.

On Fri, Feb 9, 2018 at 11:51 AM, Jevgeni Zolotarjov 
wrote:

> my host in Centos7. I upgraded just by changing the URL in repository and
> yum update
>
> How do I upgrade systemvm templates now?
>
> On Fri, Feb 9, 2018 at 11:49 AM, Daan Hoogland 
> wrote:
>
>> ok, this is also an upgrade issue. If all is well you installed them
>> first.
>> You must have downloaded the new systemvm templates and installed them as
>> per the upgrade instructions (didn't read them myself this time so i'd
>> have
>> to check) and the upgrade will then mark them as the current templates for
>> systemVMs.
>> This error indicates that something in that process went wrong.
>>
>> On Fri, Feb 9, 2018 at 10:44 AM, Jevgeni Zolotarjov <
>> j.zolotar...@gmail.com>
>> wrote:
>>
>> > I have a backup and noone is accessing the installation during upgrade
>> > process
>> >
>> > Now:
>> > I moved 1 bit further.
>> >
>> > Now I am having this error
>> > 2018-02-09 09:38:05,475 DEBUG [c.c.u.d.ScriptRunner] (main:null)
>> (logid:)
>> > ALTER TABLE cloud.ldap_configuration ADD COLUMN domain_id BIGINT(20)
>> > DEFAULT NULL
>> > 2018-02-09 09:38:05,476 DEBUG [c.c.u.d.ScriptRunner] (main:null)
>> (logid:)
>> > ALTER TABLE cloud.ldap_trust_map ADD COLUMN account_id BIGINT(20)
>> DEFAULT 0
>> > 2018-02-09 09:38:05,478 DEBUG [c.c.u.d.ScriptRunner] (main:null)
>> (logid:)
>> > ALTER TABLE cloud.ldap_trust_map DROP FOREIGN KEY
>> > fk_ldap_trust_map__domain_id
>> > 2018-02-09 09:38:05,481 DEBUG [c.c.u.d.ScriptRunner] (main:null)
>> (logid:)
>> > DROP INDEX uk_ldap_trust_map__domain_id ON cloud.ldap_trust_map
>> > 2018-02-09 09:38:05,482 DEBUG [c.c.u.d.ScriptRunner] (main:null)
>> (logid:)
>> > CREATE UNIQUE INDEX uk_ldap_trust_map__bind_location ON ldap_trust_map
>> > (domain_id, account_id)
>> > 2018-02-09 09:38:05,488 ERROR [c.c.u.PropertiesUtil] (main:null)
>> (logid:)
>> > Unable to find properties file: commands.properties
>> > 2018-02-09 09:38:05,489 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
>> > (logid:) No commands.properties file was found, enabling dynamic roles
>> by
>> > setting dynamic.apichecker.enabled to true if not already enabled.
>> > 2018-02-09 09:38:05,490 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
>> > (logid:) Done validating base64 content of user data
>> > 2018-02-09 09:38:05,490 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
>> > (logid:) Updating System Vm template IDs
>> > 2018-02-09 09:38:05,493 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
>> > (logid:) Updating LXC System Vms
>> > 2018-02-09 09:38:05,493 WARN  [c.c.u.d.Upgrade41000to41100] (main:null)
>> > (logid:) 4.11.0.0LXC SystemVm template not found. LXC hypervisor is not
>> > used, so not failing upgrade
>> > 2018-02-09 09:38:05,494 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
>> > (logid:) Updating Hyperv System Vms
>> > 2018-02-09 09:38:05,495 WARN  [c.c.u.d.Upgrade41000to41100] (main:null)
>> > (logid:) 4.11.0.0Hyperv SystemVm template not found. Hyperv hypervisor
>> is
>> > not used, so not failing upgrade
>> > 2018-02-09 09:38:05,496 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
>> > (logid:) Updating KVM System Vms
>> > 2018-02-09 09:38:05,497 ERROR [c.c.u.DatabaseUpgradeChecker] (main:null)
>> > (logid:) Unable to upgrade the database
>> > com.cloud.utils.exception.CloudRuntimeException: 4.11.0.0KVM SystemVm
>> > template not found. Cannot upgrade system Vms
>> > at
>> > com.cloud.upgrade.dao.Upgrade41000to41100.updateSystemVmTemplates(
>> > Upgrade41000to41100.java:270)
>> > at
>> > com.cloud.upgrade.dao.Upgrade41000to41100.performDataMigration(
>> > Upgrade41000to41100.java:71)

Re: iso not downloading from local path

[Dag Sonstebo]

Check the ACL on your NFS share – remember your SSVM is using an IP address in 
the management range of your zone to access NFS, so we sometimes see issues 
where hypervisors are on the ACL, but not the management IP range.

Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue

On 09/02/2018, 11:15, "Swastik Mittal"  wrote:

Hey,

I checked accessing nfs server from my ssvm. I tried the following

root@s-1-VM:~# rpcinfo -p
   program vers proto   port  service
104   tcp111  portmapper
103   tcp111  portmapper
102   tcp111  portmapper
104   udp111  portmapper
103   udp111  portmapper
102   udp111  portmapper
1000241   udp  58883  status
1000241   tcp  48454  status

I have only portmapper running. Thats it. No nfs ports found.

root@s-1-VM:~# mount -t nfs

above gives no result.

I manually mounted ssvm to /export/secondary of secondary storage nfs
server. But again all the above commands give the same result.

root@s-1-VM:~# df -k
Filesystem 1K-blocks Used
Available Use% Mounted on
rootfs472036
160666286999  36% /
udev   10240
0 10240   0% /dev
tmpfs  50896
256 50640   1% /run
/dev/disk/by-uuid/30c81d3d-ee9f-4a88-81c1-5f349b22ba1d472036
160666286999  36% /
tmpfs   5120
0  5120   0% /run/lock
tmpfs 211960
0211960   0% /run/shm
/dev/vda1  74367
22706 47821  33% /boot
/dev/vda6  93207
5654 82741   7% /home
/dev/vda8 188403
6091172584   4% /opt
/dev/vda11 93207
5679 82716   7% /tmp
/dev/vda7 768016
565696163308  78% /usr
/dev/vda9 575500
235564310704  44% /var
/dev/vda10188403
10441168234   6% /var/log
10.1.0.77:/export/secondary708019200 12345344
659686400   2% /mnt/SecStorage/15b578c2-972f-3178-ac74-acc61e05ec25
10.1.0.77:/export/secondary708019200 12345344
659686400   2% /mnt/SecStorage/15b578c2-972f-3178-ac74-acc61e05ec25
root@s-1-VM:~#

(10.1.0.77 - management server and also secondary and primary storage
server)

I checked cloud.out and it states "no credentials for host found".

I cannot ssh into management from ssvm. I tried adding management server
(also my storage server) in known_host of ssvm but it din't work.
@Dag I guess you were right about SSVM not able to access Secondary
Storage. I tried mounting SSVM manually to Secondary storage server but it
din't work.
Any idea?

cheers
Swastik


On Fri, Feb 9, 2018 at 11:56 AM, Swastik Mittal 
wrote:

> Hey,
>
> I took a lookat my NFS storage.
>
> root@MY-PC:~# mount -t nfs
> 10.1.0.77:/export/primary on /mnt/9ebaae44-dc8f-33d5-8fb6-90c193e69a22
> type nfs (rw,vers=4,addr=10.1.0.77,clientaddr=10.1.0.77)
> 10.1.0.77:/export/secondary on /mnt/secondary type nfs
> (rw,rsize=8192,wsize=8192,timeo=14,intr,vers=3,addr=10.1.0.77)
> 10.1.0.77:/export/primary on /mnt/primary type nfs
> (rw,rsize=8192,wsize=8192,timeo=14,intr,vers=3,addr=10.1.0.77)
> root@MY-PC:~# df -h
> Filesystem  Size  Used  Avail  Use%   Mounted on
> /dev/sda4   676G   12G  630G2%  /
> none   4.0K 0 4.0K0%  /sys/fs/cgroup
> udev   7.8G  4.0K   7.8G1% /dev
> tmpfs  1.6G  1.9M  1.6G1% /run
> none   5.0M 0 5.0M   0%  /run/lock
> none   7.9G  208K  7.9G1%  /run/shm
> none   100M   48K  100M   1%/run/user
> 10.1.0.77:/export/primary  676G   12G  630G   2%
> /mnt/9ebaae44-dc8f-33d5-8fb6-90c193e69a22
> 10.1.0.77:/export/primary  676G   12G  630G   2% /mnt/primary
>
> My secondary storage is mounted as /mnt/secondary but it does not show as
> the diskspace available like the primary memory.
>
> Cheers
> Swastik
>
>
> On Fri, Feb 9, 2018 at 9:23 AM, Swastik Mittal 

Re: iso not downloading from local path

[Swastik Mittal]

Hey,

I checked accessing nfs server from my ssvm. I tried the following

root@s-1-VM:~# rpcinfo -p
   program vers proto   port  service
104   tcp111  portmapper
103   tcp111  portmapper
102   tcp111  portmapper
104   udp111  portmapper
103   udp111  portmapper
102   udp111  portmapper
1000241   udp  58883  status
1000241   tcp  48454  status

I have only portmapper running. Thats it. No nfs ports found.

root@s-1-VM:~# mount -t nfs

above gives no result.

I manually mounted ssvm to /export/secondary of secondary storage nfs
server. But again all the above commands give the same result.

root@s-1-VM:~# df -k
Filesystem 1K-blocks Used
Available Use% Mounted on
rootfs472036
160666286999  36% /
udev   10240
0 10240   0% /dev
tmpfs  50896
256 50640   1% /run
/dev/disk/by-uuid/30c81d3d-ee9f-4a88-81c1-5f349b22ba1d472036
160666286999  36% /
tmpfs   5120
0  5120   0% /run/lock
tmpfs 211960
0211960   0% /run/shm
/dev/vda1  74367
22706 47821  33% /boot
/dev/vda6  93207
5654 82741   7% /home
/dev/vda8 188403
6091172584   4% /opt
/dev/vda11 93207
5679 82716   7% /tmp
/dev/vda7 768016
565696163308  78% /usr
/dev/vda9 575500
235564310704  44% /var
/dev/vda10188403
10441168234   6% /var/log
10.1.0.77:/export/secondary708019200 12345344
659686400   2% /mnt/SecStorage/15b578c2-972f-3178-ac74-acc61e05ec25
10.1.0.77:/export/secondary708019200 12345344
659686400   2% /mnt/SecStorage/15b578c2-972f-3178-ac74-acc61e05ec25
root@s-1-VM:~#

(10.1.0.77 - management server and also secondary and primary storage
server)

I checked cloud.out and it states "no credentials for host found".

I cannot ssh into management from ssvm. I tried adding management server
(also my storage server) in known_host of ssvm but it din't work.
@Dag I guess you were right about SSVM not able to access Secondary
Storage. I tried mounting SSVM manually to Secondary storage server but it
din't work.
Any idea?

cheers
Swastik


On Fri, Feb 9, 2018 at 11:56 AM, Swastik Mittal 
wrote:

> Hey,
>
> I took a lookat my NFS storage.
>
> root@MY-PC:~# mount -t nfs
> 10.1.0.77:/export/primary on /mnt/9ebaae44-dc8f-33d5-8fb6-90c193e69a22
> type nfs (rw,vers=4,addr=10.1.0.77,clientaddr=10.1.0.77)
> 10.1.0.77:/export/secondary on /mnt/secondary type nfs
> (rw,rsize=8192,wsize=8192,timeo=14,intr,vers=3,addr=10.1.0.77)
> 10.1.0.77:/export/primary on /mnt/primary type nfs
> (rw,rsize=8192,wsize=8192,timeo=14,intr,vers=3,addr=10.1.0.77)
> root@MY-PC:~# df -h
> Filesystem  Size  Used  Avail  Use%   Mounted on
> /dev/sda4   676G   12G  630G2%  /
> none   4.0K 0 4.0K0%  /sys/fs/cgroup
> udev   7.8G  4.0K   7.8G1% /dev
> tmpfs  1.6G  1.9M  1.6G1% /run
> none   5.0M 0 5.0M   0%  /run/lock
> none   7.9G  208K  7.9G1%  /run/shm
> none   100M   48K  100M   1%/run/user
> 10.1.0.77:/export/primary  676G   12G  630G   2%
> /mnt/9ebaae44-dc8f-33d5-8fb6-90c193e69a22
> 10.1.0.77:/export/primary  676G   12G  630G   2% /mnt/primary
>
> My secondary storage is mounted as /mnt/secondary but it does not show as
> the diskspace available like the primary memory.
>
> Cheers
> Swastik
>
>
> On Fri, Feb 9, 2018 at 9:23 AM, Swastik Mittal 
> wrote:
>
>> Hey Dag,
>>
>> On running ssvm-check I got :
>>
>> root@s-1-VM:/usr/local/cloud/systemvm# ./ssvm-check.sh
>> 
>> First DNS server is  8.8.8.8
>> PING 8.8.8.8 (8.8.8.8): 48 data bytes
>> --- 8.8.8.8 ping statistics ---
>> 2 packets transmitted, 0 packets received, 100% packet loss
>> WARNING: cannot ping DNS server
>> route follows
>> Kernel IP routing table
>> Destination Gateway Genmask Flags Metric RefUse
>> Iface
>> 0.0.0.0   10.1.0.20.0.0.0 UG0  00
>> eth2
>> 8.8.8.8   10.1.0.2255.255.255.255 UGH   0  00
>> eth1
>> 10.1.0.0  0.0.0.0 255.255.255.0   U 0  00
>> eth1
>> 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Jevgeni Zolotarjov]

my host in Centos7. I upgraded just by changing the URL in repository and
yum update

How do I upgrade systemvm templates now?

On Fri, Feb 9, 2018 at 11:49 AM, Daan Hoogland 
wrote:

> ok, this is also an upgrade issue. If all is well you installed them first.
> You must have downloaded the new systemvm templates and installed them as
> per the upgrade instructions (didn't read them myself this time so i'd have
> to check) and the upgrade will then mark them as the current templates for
> systemVMs.
> This error indicates that something in that process went wrong.
>
> On Fri, Feb 9, 2018 at 10:44 AM, Jevgeni Zolotarjov <
> j.zolotar...@gmail.com>
> wrote:
>
> > I have a backup and noone is accessing the installation during upgrade
> > process
> >
> > Now:
> > I moved 1 bit further.
> >
> > Now I am having this error
> > 2018-02-09 09:38:05,475 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > ALTER TABLE cloud.ldap_configuration ADD COLUMN domain_id BIGINT(20)
> > DEFAULT NULL
> > 2018-02-09 09:38:05,476 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > ALTER TABLE cloud.ldap_trust_map ADD COLUMN account_id BIGINT(20)
> DEFAULT 0
> > 2018-02-09 09:38:05,478 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > ALTER TABLE cloud.ldap_trust_map DROP FOREIGN KEY
> > fk_ldap_trust_map__domain_id
> > 2018-02-09 09:38:05,481 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > DROP INDEX uk_ldap_trust_map__domain_id ON cloud.ldap_trust_map
> > 2018-02-09 09:38:05,482 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > CREATE UNIQUE INDEX uk_ldap_trust_map__bind_location ON ldap_trust_map
> > (domain_id, account_id)
> > 2018-02-09 09:38:05,488 ERROR [c.c.u.PropertiesUtil] (main:null) (logid:)
> > Unable to find properties file: commands.properties
> > 2018-02-09 09:38:05,489 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
> > (logid:) No commands.properties file was found, enabling dynamic roles by
> > setting dynamic.apichecker.enabled to true if not already enabled.
> > 2018-02-09 09:38:05,490 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
> > (logid:) Done validating base64 content of user data
> > 2018-02-09 09:38:05,490 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
> > (logid:) Updating System Vm template IDs
> > 2018-02-09 09:38:05,493 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
> > (logid:) Updating LXC System Vms
> > 2018-02-09 09:38:05,493 WARN  [c.c.u.d.Upgrade41000to41100] (main:null)
> > (logid:) 4.11.0.0LXC SystemVm template not found. LXC hypervisor is not
> > used, so not failing upgrade
> > 2018-02-09 09:38:05,494 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
> > (logid:) Updating Hyperv System Vms
> > 2018-02-09 09:38:05,495 WARN  [c.c.u.d.Upgrade41000to41100] (main:null)
> > (logid:) 4.11.0.0Hyperv SystemVm template not found. Hyperv hypervisor is
> > not used, so not failing upgrade
> > 2018-02-09 09:38:05,496 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
> > (logid:) Updating KVM System Vms
> > 2018-02-09 09:38:05,497 ERROR [c.c.u.DatabaseUpgradeChecker] (main:null)
> > (logid:) Unable to upgrade the database
> > com.cloud.utils.exception.CloudRuntimeException: 4.11.0.0KVM SystemVm
> > template not found. Cannot upgrade system Vms
> > at
> > com.cloud.upgrade.dao.Upgrade41000to41100.updateSystemVmTemplates(
> > Upgrade41000to41100.java:270)
> > at
> > com.cloud.upgrade.dao.Upgrade41000to41100.performDataMigration(
> > Upgrade41000to41100.java:71)
> > at
> > com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(
> > DatabaseUpgradeChecker.java:561)
> > at
> > com.cloud.upgrade.DatabaseUpgradeChecker.check(
> > DatabaseUpgradeChecker.java:641)
> > at
> > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.
> > checkIntegrity(CloudStackExtendedLifeCycle.java:65)
> > at
> > org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.
> start(
> > CloudStackExtendedLifeCycle.java:55)
> > at
> > org.springframework.context.support.DefaultLifecycleProcessor.doStart(
> > DefaultLifecycleProcessor.java:183)
> > at
> > org.springframework.context.support.DefaultLifecycleProcessor.
> access$200(
> > DefaultLifecycleProcessor.java:52)
> > at
> > org.springframework.context.support.DefaultLifecycleProcessor$
> > LifecycleGroup.start(DefaultLifecycleProcessor.java:358)
> > at
> > org.springframework.context.support.DefaultLifecycleProcessor.
> startBeans(
> > DefaultLifecycleProcessor.java:159)
> > at
> > org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(
> > DefaultLifecycleProcessor.java:123)
> > at
> > org.springframework.context.support.AbstractApplicationContext.
> > finishRefresh(AbstractApplicationContext.java:884)
> > at
> > org.springframework.context.support.AbstractApplicationContext.refresh(
> > AbstractApplicationContext.java:552)
> > at
> > org.apache.cloudstack.spring.module.model.impl.
> 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Daan Hoogland]

ok, this is also an upgrade issue. If all is well you installed them first.
You must have downloaded the new systemvm templates and installed them as
per the upgrade instructions (didn't read them myself this time so i'd have
to check) and the upgrade will then mark them as the current templates for
systemVMs.
This error indicates that something in that process went wrong.

On Fri, Feb 9, 2018 at 10:44 AM, Jevgeni Zolotarjov 
wrote:

> I have a backup and noone is accessing the installation during upgrade
> process
>
> Now:
> I moved 1 bit further.
>
> Now I am having this error
> 2018-02-09 09:38:05,475 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> ALTER TABLE cloud.ldap_configuration ADD COLUMN domain_id BIGINT(20)
> DEFAULT NULL
> 2018-02-09 09:38:05,476 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> ALTER TABLE cloud.ldap_trust_map ADD COLUMN account_id BIGINT(20) DEFAULT 0
> 2018-02-09 09:38:05,478 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> ALTER TABLE cloud.ldap_trust_map DROP FOREIGN KEY
> fk_ldap_trust_map__domain_id
> 2018-02-09 09:38:05,481 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> DROP INDEX uk_ldap_trust_map__domain_id ON cloud.ldap_trust_map
> 2018-02-09 09:38:05,482 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> CREATE UNIQUE INDEX uk_ldap_trust_map__bind_location ON ldap_trust_map
> (domain_id, account_id)
> 2018-02-09 09:38:05,488 ERROR [c.c.u.PropertiesUtil] (main:null) (logid:)
> Unable to find properties file: commands.properties
> 2018-02-09 09:38:05,489 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
> (logid:) No commands.properties file was found, enabling dynamic roles by
> setting dynamic.apichecker.enabled to true if not already enabled.
> 2018-02-09 09:38:05,490 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
> (logid:) Done validating base64 content of user data
> 2018-02-09 09:38:05,490 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
> (logid:) Updating System Vm template IDs
> 2018-02-09 09:38:05,493 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
> (logid:) Updating LXC System Vms
> 2018-02-09 09:38:05,493 WARN  [c.c.u.d.Upgrade41000to41100] (main:null)
> (logid:) 4.11.0.0LXC SystemVm template not found. LXC hypervisor is not
> used, so not failing upgrade
> 2018-02-09 09:38:05,494 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
> (logid:) Updating Hyperv System Vms
> 2018-02-09 09:38:05,495 WARN  [c.c.u.d.Upgrade41000to41100] (main:null)
> (logid:) 4.11.0.0Hyperv SystemVm template not found. Hyperv hypervisor is
> not used, so not failing upgrade
> 2018-02-09 09:38:05,496 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
> (logid:) Updating KVM System Vms
> 2018-02-09 09:38:05,497 ERROR [c.c.u.DatabaseUpgradeChecker] (main:null)
> (logid:) Unable to upgrade the database
> com.cloud.utils.exception.CloudRuntimeException: 4.11.0.0KVM SystemVm
> template not found. Cannot upgrade system Vms
> at
> com.cloud.upgrade.dao.Upgrade41000to41100.updateSystemVmTemplates(
> Upgrade41000to41100.java:270)
> at
> com.cloud.upgrade.dao.Upgrade41000to41100.performDataMigration(
> Upgrade41000to41100.java:71)
> at
> com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(
> DatabaseUpgradeChecker.java:561)
> at
> com.cloud.upgrade.DatabaseUpgradeChecker.check(
> DatabaseUpgradeChecker.java:641)
> at
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.
> checkIntegrity(CloudStackExtendedLifeCycle.java:65)
> at
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(
> CloudStackExtendedLifeCycle.java:55)
> at
> org.springframework.context.support.DefaultLifecycleProcessor.doStart(
> DefaultLifecycleProcessor.java:183)
> at
> org.springframework.context.support.DefaultLifecycleProcessor.access$200(
> DefaultLifecycleProcessor.java:52)
> at
> org.springframework.context.support.DefaultLifecycleProcessor$
> LifecycleGroup.start(DefaultLifecycleProcessor.java:358)
> at
> org.springframework.context.support.DefaultLifecycleProcessor.startBeans(
> DefaultLifecycleProcessor.java:159)
> at
> org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(
> DefaultLifecycleProcessor.java:123)
> at
> org.springframework.context.support.AbstractApplicationContext.
> finishRefresh(AbstractApplicationContext.java:884)
> at
> org.springframework.context.support.AbstractApplicationContext.refresh(
> AbstractApplicationContext.java:552)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.
> loadContext(DefaultModuleDefinitionSet.java:145)
> at
> org.apache.cloudstack.spring.module.model.impl.
> DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
> at
> org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.
> withModule(DefaultModuleDefinitionSet.java:245)
> at
> 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Raja Pullela]

Jevgeni, the error shows that the MS is not able to find the 4.11 system 
template.  Did you seed the 4.11 KVM system template? 

Best,
Raja
Engineering, Accelerite,
2055, Laurelwood Road, Santa Clara, CA, 95054
www.accelerite.com


On 2/9/18, 3:15 PM, "Jevgeni Zolotarjov"  wrote:

 Unable to upgrade the database
com.cloud.utils.exception.CloudRuntimeException: 4.11.0.0KVM SystemVm
template not found. Cannot upgrade system Vms

DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Accelerite, a Persistent Systems business. It is intended only for 
the use of the individual or entity to which it is addressed. If you are not 
the intended recipient, you are not authorized to read, retain, copy, print, 
distribute or use this message. If you have received this communication in 
error, please notify the sender and delete all copies of this message. 
Accelerite, a Persistent Systems business does not accept any liability for 
virus infected mails.

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Jevgeni Zolotarjov]

I have a backup and noone is accessing the installation during upgrade
process

Now:
I moved 1 bit further.

Now I am having this error
2018-02-09 09:38:05,475 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
ALTER TABLE cloud.ldap_configuration ADD COLUMN domain_id BIGINT(20)
DEFAULT NULL
2018-02-09 09:38:05,476 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
ALTER TABLE cloud.ldap_trust_map ADD COLUMN account_id BIGINT(20) DEFAULT 0
2018-02-09 09:38:05,478 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
ALTER TABLE cloud.ldap_trust_map DROP FOREIGN KEY
fk_ldap_trust_map__domain_id
2018-02-09 09:38:05,481 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
DROP INDEX uk_ldap_trust_map__domain_id ON cloud.ldap_trust_map
2018-02-09 09:38:05,482 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
CREATE UNIQUE INDEX uk_ldap_trust_map__bind_location ON ldap_trust_map
(domain_id, account_id)
2018-02-09 09:38:05,488 ERROR [c.c.u.PropertiesUtil] (main:null) (logid:)
Unable to find properties file: commands.properties
2018-02-09 09:38:05,489 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
(logid:) No commands.properties file was found, enabling dynamic roles by
setting dynamic.apichecker.enabled to true if not already enabled.
2018-02-09 09:38:05,490 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
(logid:) Done validating base64 content of user data
2018-02-09 09:38:05,490 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
(logid:) Updating System Vm template IDs
2018-02-09 09:38:05,493 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
(logid:) Updating LXC System Vms
2018-02-09 09:38:05,493 WARN  [c.c.u.d.Upgrade41000to41100] (main:null)
(logid:) 4.11.0.0LXC SystemVm template not found. LXC hypervisor is not
used, so not failing upgrade
2018-02-09 09:38:05,494 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
(logid:) Updating Hyperv System Vms
2018-02-09 09:38:05,495 WARN  [c.c.u.d.Upgrade41000to41100] (main:null)
(logid:) 4.11.0.0Hyperv SystemVm template not found. Hyperv hypervisor is
not used, so not failing upgrade
2018-02-09 09:38:05,496 DEBUG [c.c.u.d.Upgrade41000to41100] (main:null)
(logid:) Updating KVM System Vms
2018-02-09 09:38:05,497 ERROR [c.c.u.DatabaseUpgradeChecker] (main:null)
(logid:) Unable to upgrade the database
com.cloud.utils.exception.CloudRuntimeException: 4.11.0.0KVM SystemVm
template not found. Cannot upgrade system Vms
at
com.cloud.upgrade.dao.Upgrade41000to41100.updateSystemVmTemplates(Upgrade41000to41100.java:270)
at
com.cloud.upgrade.dao.Upgrade41000to41100.performDataMigration(Upgrade41000to41100.java:71)
at
com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpgradeChecker.java:561)
at
com.cloud.upgrade.DatabaseUpgradeChecker.check(DatabaseUpgradeChecker.java:641)
at
org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.checkIntegrity(CloudStackExtendedLifeCycle.java:65)
at
org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:55)
at
org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:183)
at
org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:52)
at
org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:358)
at
org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:159)
at
org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123)
at
org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:884)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:552)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
at
org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
at

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Daan Hoogland]

before you continue Jevgeni,

Do you have a good backup and is no-one able to access this installation
with UI or API, when you roll back?

On Fri, Feb 9, 2018 at 10:15 AM, Ernie Janse van Rensburg <
ernie.jvrensb...@shapeblue.com> wrote:

> engine/schema/resources/META-INF/db/schema-41000to41100.sql:
>
> 
> From: Jevgeni Zolotarjov 
> Sent: Friday, February 9, 2018 10:59:27 AM
> To: users@cloudstack.apache.org
> Subject: Re: cloudstack-management fails to start after upgrade 4.10 ->
> 4.11
>
> I dropped the column. But then another error like this appeared
>
> Please advise, where are the DB update script located? so I can manually
> inspect that.
>
> On Fri, Feb 9, 2018 at 10:29 AM, Ernie Janse van Rensburg <
> ernie.jvrensb...@shapeblue.com> wrote:
>
> > Hi Jevgeni
> >
> >
> > It looks like there was a database error during the upgrade process.
> >
> >
> > A SQL script that is trying to add a column 'for_vpc' to the TABLE
> > 'cloud.network_offerings' but the column already exists, for some reason,
> > so it fails because mysql does not allow 2 columns with the same name in
> > the same table.
> >
> >
> > The column might have already existed from a previous upgrade or from
> > running the 4.11 upgrade more than once, and perhaps the SQL script is
> not
> > idempotent.
> >
> >
> > I suggest to manually drop the column on the table and run the 4.11
> > upgrade process again.
> >
> >
> > Regards
> >
> >
> > Ernie
> >
> > 
> > From: Jevgeni Zolotarjov 
> > Sent: Friday, February 9, 2018 10:10:59 AM
> > To: users@cloudstack.apache.org
> > Subject: cloudstack-management fails to start after upgrade 4.10 -> 4.11
> >
> > cloudstack-management fails to start after upgrade from 4.10 to 4.11
> >
> > management-server.log:
> > 2018-02-09 07:49:50,842 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null)
> > (logid:) Running upgrade Upgrade41000to41100 to upgrade from
> > 4.10.0.0-4.11.0.0 to 4.11.0.0
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > -- Licensed to the Apache Software Foundation (ASF) under one
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > -- or more contributor license agreements.  See the NOTICE file
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > -- distributed with this work for additional information
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > -- regarding copyright ownership.  The ASF licenses this file
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > -- to you under the Apache License, Version 2.0 (the
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > -- "License"); you may not use this file except in compliance
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > -- with the License.  You may obtain a copy of the License at
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > --
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > --   http://www.apache.org/licenses/LICENSE-2.0
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > --
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > -- Unless required by applicable law or agreed to in writing,
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > -- software distributed under the License is distributed on an
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > -- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > -- KIND, either express or implied.  See the License for the
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > -- specific language governing permissions and limitations
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > -- under the License.
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > --;
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > -- Schema upgrade from 4.10.0.0 to 4.11.0.0
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > --;
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > -- Add For VPC flag
> > 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > ALTER TABLE cloud.network_offerings ADD COLUMN for_vpc INT(1) NOT NULL
> > DEFAULT 0
> > 2018-02-09 07:49:50,847 ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > Error executing: ALTER TABLE cloud.network_offerings ADD COLUMN for_vpc
> > INT(1) NOT NULL DEFAULT 0
> > 2018-02-09 07:49:50,848 ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:)
> > 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Ernie Janse van Rensburg]

engine/schema/resources/META-INF/db/schema-41000to41100.sql:


From: Jevgeni Zolotarjov 
Sent: Friday, February 9, 2018 10:59:27 AM
To: users@cloudstack.apache.org
Subject: Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

I dropped the column. But then another error like this appeared

Please advise, where are the DB update script located? so I can manually
inspect that.

On Fri, Feb 9, 2018 at 10:29 AM, Ernie Janse van Rensburg <
ernie.jvrensb...@shapeblue.com> wrote:

> Hi Jevgeni
>
>
> It looks like there was a database error during the upgrade process.
>
>
> A SQL script that is trying to add a column 'for_vpc' to the TABLE
> 'cloud.network_offerings' but the column already exists, for some reason,
> so it fails because mysql does not allow 2 columns with the same name in
> the same table.
>
>
> The column might have already existed from a previous upgrade or from
> running the 4.11 upgrade more than once, and perhaps the SQL script is not
> idempotent.
>
>
> I suggest to manually drop the column on the table and run the 4.11
> upgrade process again.
>
>
> Regards
>
>
> Ernie
>
> 
> From: Jevgeni Zolotarjov 
> Sent: Friday, February 9, 2018 10:10:59 AM
> To: users@cloudstack.apache.org
> Subject: cloudstack-management fails to start after upgrade 4.10 -> 4.11
>
> cloudstack-management fails to start after upgrade from 4.10 to 4.11
>
> management-server.log:
> 2018-02-09 07:49:50,842 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null)
> (logid:) Running upgrade Upgrade41000to41100 to upgrade from
> 4.10.0.0-4.11.0.0 to 4.11.0.0
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- Licensed to the Apache Software Foundation (ASF) under one
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- or more contributor license agreements.  See the NOTICE file
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- distributed with this work for additional information
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- regarding copyright ownership.  The ASF licenses this file
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- to you under the Apache License, Version 2.0 (the
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- "License"); you may not use this file except in compliance
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- with the License.  You may obtain a copy of the License at
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> --
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> --   http://www.apache.org/licenses/LICENSE-2.0
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> --
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- Unless required by applicable law or agreed to in writing,
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- software distributed under the License is distributed on an
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- KIND, either express or implied.  See the License for the
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- specific language governing permissions and limitations
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- under the License.
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> --;
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- Schema upgrade from 4.10.0.0 to 4.11.0.0
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> --;
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- Add For VPC flag
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> ALTER TABLE cloud.network_offerings ADD COLUMN for_vpc INT(1) NOT NULL
> DEFAULT 0
> 2018-02-09 07:49:50,847 ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:)
> Error executing: ALTER TABLE cloud.network_offerings ADD COLUMN for_vpc
> INT(1) NOT NULL DEFAULT 0
> 2018-02-09 07:49:50,848 ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:)
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Duplicate
> column
> name 'for_vpc'
> 2018-02-09 07:49:50,849 ERROR [c.c.u.DatabaseUpgradeChecker] (main:null)
> (logid:) Unable to execute upgrade script
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Duplicate
> column
> name 'for_vpc'
> at com.cloud.utils.db.ScriptRunner.runScript(
> ScriptRunner.java:185)
> at 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Jevgeni Zolotarjov]

I dropped the column. But then another error like this appeared

Please advise, where are the DB update script located? so I can manually
inspect that.

On Fri, Feb 9, 2018 at 10:29 AM, Ernie Janse van Rensburg <
ernie.jvrensb...@shapeblue.com> wrote:

> Hi Jevgeni
>
>
> It looks like there was a database error during the upgrade process.
>
>
> A SQL script that is trying to add a column 'for_vpc' to the TABLE
> 'cloud.network_offerings' but the column already exists, for some reason,
> so it fails because mysql does not allow 2 columns with the same name in
> the same table.
>
>
> The column might have already existed from a previous upgrade or from
> running the 4.11 upgrade more than once, and perhaps the SQL script is not
> idempotent.
>
>
> I suggest to manually drop the column on the table and run the 4.11
> upgrade process again.
>
>
> Regards
>
>
> Ernie
>
> 
> From: Jevgeni Zolotarjov 
> Sent: Friday, February 9, 2018 10:10:59 AM
> To: users@cloudstack.apache.org
> Subject: cloudstack-management fails to start after upgrade 4.10 -> 4.11
>
> cloudstack-management fails to start after upgrade from 4.10 to 4.11
>
> management-server.log:
> 2018-02-09 07:49:50,842 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null)
> (logid:) Running upgrade Upgrade41000to41100 to upgrade from
> 4.10.0.0-4.11.0.0 to 4.11.0.0
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- Licensed to the Apache Software Foundation (ASF) under one
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- or more contributor license agreements.  See the NOTICE file
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- distributed with this work for additional information
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- regarding copyright ownership.  The ASF licenses this file
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- to you under the Apache License, Version 2.0 (the
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- "License"); you may not use this file except in compliance
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- with the License.  You may obtain a copy of the License at
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> --
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> --   http://www.apache.org/licenses/LICENSE-2.0
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> --
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- Unless required by applicable law or agreed to in writing,
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- software distributed under the License is distributed on an
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- KIND, either express or implied.  See the License for the
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- specific language governing permissions and limitations
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- under the License.
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> --;
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- Schema upgrade from 4.10.0.0 to 4.11.0.0
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> --;
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- Add For VPC flag
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> ALTER TABLE cloud.network_offerings ADD COLUMN for_vpc INT(1) NOT NULL
> DEFAULT 0
> 2018-02-09 07:49:50,847 ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:)
> Error executing: ALTER TABLE cloud.network_offerings ADD COLUMN for_vpc
> INT(1) NOT NULL DEFAULT 0
> 2018-02-09 07:49:50,848 ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:)
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Duplicate
> column
> name 'for_vpc'
> 2018-02-09 07:49:50,849 ERROR [c.c.u.DatabaseUpgradeChecker] (main:null)
> (logid:) Unable to execute upgrade script
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Duplicate
> column
> name 'for_vpc'
> at com.cloud.utils.db.ScriptRunner.runScript(
> ScriptRunner.java:185)
> at com.cloud.utils.db.ScriptRunner.runScript(ScriptRunner.java:87)
> at
> com.cloud.upgrade.DatabaseUpgradeChecker.runScript(
> DatabaseUpgradeChecker.java:459)
> at
> com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(
> DatabaseUpgradeChecker.java:557)
> at
> com.cloud.upgrade.DatabaseUpgradeChecker.check(
> 

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Ernie Janse van Rensburg]

Hi Jevgeni


It looks like there was a database error during the upgrade process.


A SQL script that is trying to add a column 'for_vpc' to the TABLE 
'cloud.network_offerings' but the column already exists, for some reason, so it 
fails because mysql does not allow 2 columns with the same name in the same 
table.


The column might have already existed from a previous upgrade or from running 
the 4.11 upgrade more than once, and perhaps the SQL script is not idempotent.


I suggest to manually drop the column on the table and run the 4.11 upgrade 
process again.


Regards


Ernie


From: Jevgeni Zolotarjov 
Sent: Friday, February 9, 2018 10:10:59 AM
To: users@cloudstack.apache.org
Subject: cloudstack-management fails to start after upgrade 4.10 -> 4.11

cloudstack-management fails to start after upgrade from 4.10 to 4.11

management-server.log:
2018-02-09 07:49:50,842 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null)
(logid:) Running upgrade Upgrade41000to41100 to upgrade from
4.10.0.0-4.11.0.0 to 4.11.0.0
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- Licensed to the Apache Software Foundation (ASF) under one
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- or more contributor license agreements.  See the NOTICE file
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- distributed with this work for additional information
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- regarding copyright ownership.  The ASF licenses this file
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- to you under the Apache License, Version 2.0 (the
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- "License"); you may not use this file except in compliance
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- with the License.  You may obtain a copy of the License at
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
--   http://www.apache.org/licenses/LICENSE-2.0
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- Unless required by applicable law or agreed to in writing,
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- software distributed under the License is distributed on an
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- KIND, either express or implied.  See the License for the
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- specific language governing permissions and limitations
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- under the License.
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
--;
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- Schema upgrade from 4.10.0.0 to 4.11.0.0
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
--;
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- Add For VPC flag
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
ALTER TABLE cloud.network_offerings ADD COLUMN for_vpc INT(1) NOT NULL
DEFAULT 0
2018-02-09 07:49:50,847 ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:)
Error executing: ALTER TABLE cloud.network_offerings ADD COLUMN for_vpc
INT(1) NOT NULL DEFAULT 0
2018-02-09 07:49:50,848 ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:)
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Duplicate column
name 'for_vpc'
2018-02-09 07:49:50,849 ERROR [c.c.u.DatabaseUpgradeChecker] (main:null)
(logid:) Unable to execute upgrade script
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Duplicate column
name 'for_vpc'
at com.cloud.utils.db.ScriptRunner.runScript(ScriptRunner.java:185)
at com.cloud.utils.db.ScriptRunner.runScript(ScriptRunner.java:87)
at
com.cloud.upgrade.DatabaseUpgradeChecker.runScript(DatabaseUpgradeChecker.java:459)
at
com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpgradeChecker.java:557)
at
com.cloud.upgrade.DatabaseUpgradeChecker.check(DatabaseUpgradeChecker.java:641)
at
org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.checkIntegrity(CloudStackExtendedLifeCycle.java:65)
at
org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:55)
at
org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:183)
at

Re: cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Daan Hoogland]

Jevgeni,

It looks like the db upgrade went wrong. You'll have to restore a backup,
or be very savvy about what you do next.
For some reason the new column, 'for_vpc' was already defined. The
management server saw that the db was version 4.10 and self is 4.11. It
then starts the run the required upgrade scripts. In this case just one
set, 4.10.0.0 -> 4.11.0.0.

You can restore the db and retrieve the db scripts from the jar or from
github.
The you can either
1. check what needs to be done for an upgrade and make sure there are no
conflicts and restat themanagement server or
2. just run the required steps by hand including the entry in the version
table.

I have no doubt you have further question if you are new to this, feel free
regards,

On Fri, Feb 9, 2018 at 9:10 AM, Jevgeni Zolotarjov 
wrote:

> cloudstack-management fails to start after upgrade from 4.10 to 4.11
>
> management-server.log:
> 2018-02-09 07:49:50,842 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null)
> (logid:) Running upgrade Upgrade41000to41100 to upgrade from
> 4.10.0.0-4.11.0.0 to 4.11.0.0
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- Licensed to the Apache Software Foundation (ASF) under one
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- or more contributor license agreements.  See the NOTICE file
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- distributed with this work for additional information
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- regarding copyright ownership.  The ASF licenses this file
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- to you under the Apache License, Version 2.0 (the
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- "License"); you may not use this file except in compliance
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- with the License.  You may obtain a copy of the License at
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> --
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> --   http://www.apache.org/licenses/LICENSE-2.0
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> --
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- Unless required by applicable law or agreed to in writing,
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- software distributed under the License is distributed on an
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- KIND, either express or implied.  See the License for the
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- specific language governing permissions and limitations
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- under the License.
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> --;
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- Schema upgrade from 4.10.0.0 to 4.11.0.0
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> --;
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> -- Add For VPC flag
> 2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
> ALTER TABLE cloud.network_offerings ADD COLUMN for_vpc INT(1) NOT NULL
> DEFAULT 0
> 2018-02-09 07:49:50,847 ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:)
> Error executing: ALTER TABLE cloud.network_offerings ADD COLUMN for_vpc
> INT(1) NOT NULL DEFAULT 0
> 2018-02-09 07:49:50,848 ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:)
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Duplicate
> column
> name 'for_vpc'
> 2018-02-09 07:49:50,849 ERROR [c.c.u.DatabaseUpgradeChecker] (main:null)
> (logid:) Unable to execute upgrade script
> com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Duplicate
> column
> name 'for_vpc'
> at com.cloud.utils.db.ScriptRunner.runScript(
> ScriptRunner.java:185)
> at com.cloud.utils.db.ScriptRunner.runScript(ScriptRunner.java:87)
> at
> com.cloud.upgrade.DatabaseUpgradeChecker.runScript(
> DatabaseUpgradeChecker.java:459)
> at
> com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(
> DatabaseUpgradeChecker.java:557)
> at
> com.cloud.upgrade.DatabaseUpgradeChecker.check(
> DatabaseUpgradeChecker.java:641)
> at
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.
> checkIntegrity(CloudStackExtendedLifeCycle.java:65)
> at
> org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(
> CloudStackExtendedLifeCycle.java:55)
> at

cloudstack-management fails to start after upgrade 4.10 -> 4.11

[Jevgeni Zolotarjov]

cloudstack-management fails to start after upgrade from 4.10 to 4.11

management-server.log:
2018-02-09 07:49:50,842 DEBUG [c.c.u.DatabaseUpgradeChecker] (main:null)
(logid:) Running upgrade Upgrade41000to41100 to upgrade from
4.10.0.0-4.11.0.0 to 4.11.0.0
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- Licensed to the Apache Software Foundation (ASF) under one
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- or more contributor license agreements.  See the NOTICE file
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- distributed with this work for additional information
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- regarding copyright ownership.  The ASF licenses this file
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- to you under the Apache License, Version 2.0 (the
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- "License"); you may not use this file except in compliance
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- with the License.  You may obtain a copy of the License at
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
--   http://www.apache.org/licenses/LICENSE-2.0
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:) --
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- Unless required by applicable law or agreed to in writing,
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- software distributed under the License is distributed on an
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- KIND, either express or implied.  See the License for the
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- specific language governing permissions and limitations
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- under the License.
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
--;
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- Schema upgrade from 4.10.0.0 to 4.11.0.0
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
--;
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
-- Add For VPC flag
2018-02-09 07:49:50,846 DEBUG [c.c.u.d.ScriptRunner] (main:null) (logid:)
ALTER TABLE cloud.network_offerings ADD COLUMN for_vpc INT(1) NOT NULL
DEFAULT 0
2018-02-09 07:49:50,847 ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:)
Error executing: ALTER TABLE cloud.network_offerings ADD COLUMN for_vpc
INT(1) NOT NULL DEFAULT 0
2018-02-09 07:49:50,848 ERROR [c.c.u.d.ScriptRunner] (main:null) (logid:)
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Duplicate column
name 'for_vpc'
2018-02-09 07:49:50,849 ERROR [c.c.u.DatabaseUpgradeChecker] (main:null)
(logid:) Unable to execute upgrade script
com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Duplicate column
name 'for_vpc'
at com.cloud.utils.db.ScriptRunner.runScript(ScriptRunner.java:185)
at com.cloud.utils.db.ScriptRunner.runScript(ScriptRunner.java:87)
at
com.cloud.upgrade.DatabaseUpgradeChecker.runScript(DatabaseUpgradeChecker.java:459)
at
com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpgradeChecker.java:557)
at
com.cloud.upgrade.DatabaseUpgradeChecker.check(DatabaseUpgradeChecker.java:641)
at
org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.checkIntegrity(CloudStackExtendedLifeCycle.java:65)
at
org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLifeCycle.start(CloudStackExtendedLifeCycle.java:55)
at
org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:183)
at
org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:52)
at
org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:358)
at
org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:159)
at
org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:123)
at
org.springframework.context.support.AbstractApplicationContext.finishRefresh(AbstractApplicationContext.java:884)
at
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:552)
at
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
at