RE: 1 click deploy VPC network

[Nicolas Bouige]

Hi Richard,

Why don't try with Terraform ?
You can provide same terraform file for all of your student.
Just need to automate (or not) the users creation with api key/secret key if 
you plan to use specific ID for each student.
Or create a bunch of user like User01, User02...and reuse them for the next 
classroom.

Best regards,
N.B

-Message d'origine-
De : Ivan Kudryavtsev [mailto:kudryavtsev...@bw-sw.com] 
Envoyé : mardi 20 novembre 2018 19:55
À : users 
Objet : Re: 1 click deploy VPC network

Hello, Richard.

Well, there are two options:
1. Extend frontend with a plugin which will do everything you need.
2. Implement callback app which will be bound to account 1st login or 
registration and automatically creates everything you need.

Anyway, the customization is required. If you decide to go with the second 
option, you may contact me directly as we have implemented CS self-registration 
service which can do everything you need:
https://bitworks.software/en/products/cloudstack-self-registration-service/



вт, 20 нояб. 2018 г. в 13:47, Richard Persaud :

> Hello,
>
> We plan on using cloudStack for classroom style deployments, where 
> each user will get the same setup but in different VPCs.
>
> How can I set up cloudStack so when a user logs into their account, 
> there is a 1 click deploy template (VPC, tiers and VMs)?
>
> Regards,
> Richard Persaud
> Sys Spec, Info Security Del | Macy's, Inc.
> 5985 State Bridge Rd. | Johns Creek, GA 30097
> Office: 678-474-2357
> https://macyspartners.com/PublishingImages/MakeLifeShineBrighter.png
>


--
With best regards, Ivan Kudryavtsev
Bitworks LLC
Cell RU: +7-923-414-1515
Cell USA: +1-201-257-1512
WWW: http://bitworks.software/ 

RE: Upload Data Volume

[Nicolas Bouige]

olumeToVM(VolumeApiServiceImpl.java:1428)
at 
com.cloud.storage.VolumeApiServiceImpl.orchestrateAttachVolumeToVM(VolumeApiServiceImpl.java:3145)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
com.cloud.vm.VmWorkJobHandlerProxy.handleVmWorkJob(VmWorkJobHandlerProxy.java:107)
at 
com.cloud.storage.VolumeApiServiceImpl.handleVmWorkJob(VolumeApiServiceImpl.java:3178)
at sun.reflect.GeneratedMethodAccessor559.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy196.handleVmWorkJob(Unknown Source)
at com.cloud.vm.VmWorkJobDispatcher.runJob(VmWorkJobDispatcher.java:102)
at 
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:581)
at 
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at 
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at 
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:529)
at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

Best regards,
N.B

-Message d'origine-
De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com] 
Envoyé : vendredi 2 novembre 2018 19:00
À : users 
Objet : Re: Upload Data Volume

Do you have a Java stack trace for these errors?

On Fri, Nov 2, 2018 at 2:55 PM Nicolas Bouige  wrote:

> I missed those messages from mgmt-server :
>
> 2018-11-02 17:41:53,779 ERROR [o.a.c.s.v.VolumeServiceImpl]
> (Work-Job-Executor-66:ctx-15d61c99 job-6732/job-6733 ctx-b786c44b)
> (logid:e218d7a4) failed to copy volume from image store
> com.cloud.utils.exception.CloudRuntimeException: Failed to update
> state:com.cloud.utils.exception.CloudRuntimeException: Failed to 
> transit
> volume: 623, due to: com.cloud.utils.fsm.NoTransitionException: Unable 
> to transition to a new state from Copying via CopyRequested
>
> 018-11-02 17:41:53,805 INFO  [c.c.v.VmWorkJobHandlerProxy]
> (Work-Job-Executor-66:ctx-15d61c99 job-6732/job-6733 ctx-b786c44b)
> (logid:e218d7a4) Rethrow exception
> com.cloud.utils.exception.CloudRuntimeException: Failed to update
> state:com.cloud.utils.exception.CloudRuntimeException: Failed to 
> transit
> volume: 623, due to: com.cloud.utils.fsm.NoTransitionException: Unable 
> to transition to a new state from Uploaded via OperationFailed
>
> 2018-11-02 17:41:53,805 DEBUG [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-66:ctx-15d61c99 job-6732/job-6733) (logid:e218d7a4) 
> Done with run of VM work job: com.cloud.vm.VmWorkAttachVolume for VM 
> 532, job
> origin: 6732
>
> 2018-11-02 17:41:53,806 ERROR [c.c.v.VmWorkJobDispatcher]
> (Work-Job-Executor-66:ctx-15d61c99 job-6732/job-6733) (logid:e218d7a4) 
> Unable to complete AsyncJobVO {id:6733, userId: 15, accountId: 12,
> instanceType: null, instanceId: null, cmd: 
> com.cloud.vm.VmWorkAttachVolume,
> cmdInfo:
> rO0ABXNyAB9jb20uY2xvdWQudm0uVm1Xb3JrQXR0YWNoVm9sdW1lB62v-WGH4hwCAAJMAA
> hkZXZpY2VJZHQAEExqYXZhL2xhbmcvTG9uZztMAAh2b2x1bWVJZHEAfgABeHIAE2NvbS5j
>

RE: Upload Data Volume

[Nicolas Bouige]

I missed those messages from mgmt-server :

2018-11-02 17:41:53,779 ERROR [o.a.c.s.v.VolumeServiceImpl] 
(Work-Job-Executor-66:ctx-15d61c99 job-6732/job-6733 ctx-b786c44b) 
(logid:e218d7a4) failed to copy volume from image store
com.cloud.utils.exception.CloudRuntimeException: Failed to update 
state:com.cloud.utils.exception.CloudRuntimeException: Failed to transit 
volume: 623, due to: com.cloud.utils.fsm.NoTransitionException: Unable to 
transition to a new state from Copying via CopyRequested

018-11-02 17:41:53,805 INFO  [c.c.v.VmWorkJobHandlerProxy] 
(Work-Job-Executor-66:ctx-15d61c99 job-6732/job-6733 ctx-b786c44b) 
(logid:e218d7a4) Rethrow exception 
com.cloud.utils.exception.CloudRuntimeException: Failed to update 
state:com.cloud.utils.exception.CloudRuntimeException: Failed to transit 
volume: 623, due to: com.cloud.utils.fsm.NoTransitionException: Unable to 
transition to a new state from Uploaded via OperationFailed

2018-11-02 17:41:53,805 DEBUG [c.c.v.VmWorkJobDispatcher] 
(Work-Job-Executor-66:ctx-15d61c99 job-6732/job-6733) (logid:e218d7a4) Done 
with run of VM work job: com.cloud.vm.VmWorkAttachVolume for VM 532, job 
origin: 6732

2018-11-02 17:41:53,806 ERROR [c.c.v.VmWorkJobDispatcher] 
(Work-Job-Executor-66:ctx-15d61c99 job-6732/job-6733) (logid:e218d7a4) Unable 
to complete AsyncJobVO {id:6733, userId: 15, accountId: 12, instanceType: null, 
instanceId: null, cmd: com.cloud.vm.VmWorkAttachVolume, cmdInfo: 
rO0ABXNyAB9jb20uY2xvdWQudm0uVm1Xb3JrQXR0YWNoVm9sdW1lB62v-WGH4hwCAAJMAAhkZXZpY2VJZHQAEExqYXZhL2xhbmcvTG9uZztMAAh2b2x1bWVJZHEAfgABeHIAE2NvbS5jbG91ZC52bS5WbVdvcmufmbZW8CVnawIABEoACWFjY291bnRJZEoABnVzZXJJZEoABHZtSWRMAAtoYW5kbGVyTmFtZXQAEkxqYXZhL2xhbmcvU3RyaW5nO3hwAAwADwIUdAAUVm9sdW1lQXBpU2VydmljZUltcGxwc3IADmphdmEubGFuZy5Mb25nO4vkkMyPI98CAAFKAAV2YWx1ZXhyABBqYXZhLmxhbmcuTnVtYmVyhqyVHQuU4IsCAAB4cAJv,
 cmdVersion: 0, status: IN_PROGRESS, processStatus: 0, resultCode: 0, result: 
null, initMsid: 2886211232, completeMsid: null, lastUpdated: null, lastPolled: 
null, created: Fri Nov 02 17:41:52 CET 2018}, job origin:6732
com.cloud.utils.exception.CloudRuntimeException: Failed to update 
state:com.cloud.utils.exception.CloudRuntimeException: Failed to transit 
volume: 623, due to: com.cloud.utils.fsm.NoTransitionException: Unable to 
transition to a new state from Uploaded via OperationFailed

Does the attach disk process works on the same way than template/iso ? 

De : Nicolas Bouige [mailto:n.bou...@dimsi.fr] 
Envoyé : vendredi 2 novembre 2018 17:26
À : users 
Objet : RE: Upload Data Volume

Hi Rafael,


I know this is not a proper way to work directly on the DB. But sometimes 
that's permit me to "unblock"  situations specially with VMs...so why not give 
a try ?


I used the link created by Cloudpatform and i uploaded it in Cloudstack using 
the "upload" button in the storage tabs.


The .vhd is present into the cloudstack SSVM with his volume.properties file 
and correct informations...(that's why i just tried to change the flag..  )


I did not see any related error messages, i going to try again and check in 
real time just in case...thanks !


Best regards,
N.B




De : Rafael Weingärtner  Envoyé : vendredi 2 
novembre 2018 16:53:54 À : users Objet : Re: Upload Data Volume

There is a joke I always tell people here; database interventions (in
CloudStack) is forbidden, unless when I execute them. It does not work like 
that, going to the DB and simply changing a flag. You need to understand why 
the status is stuck in “uploaded”, and is not changing to ready.

There is something that called my attention now. Are you uploading the file 
(using your browser)? Or, are you configuring the template/data disk as link, 
and then the SSVM is downloading them? These are two distinct operations. I was 
under the impression that you were using the latter. So, which one of these 
methods are you using? If nothing is working, I would expect some error 
messages either in the CloudStack management server, system VMs or in the 
hosts. Did you check them?

On Fri, Nov 2, 2018 at 12:32 PM Nicolas Bouige  wrote:

> Hi,
>
> SSVM works fine and we dont use any redirection.
> We are able to Upload the volume-data after some modifications 
> directly on the cloudplatform SSVM.
>
> But the problem is still there as we can't attach the volume to a VM. 
> It is in a state "Uploaded", i tried to modify manually the state into 
> the DB (Uploaded --> Ready) but i ran into a new issue.
> Exception com.xensource.xenapi.Types$UuidInvalid :VDI getByUuid for uuid:
> null failed due to The uuid you supplied was invalid.
>
> -Message d'origine-
> De : Andrija Panic [mailto:andrija.pa...@gmail.com] Envoyé : vendredi 
> 2 novembre 2018 16:03 À : users  Objet : 
> Re: Upload Data Volume
>
> Ok, so SSVM can access remote web server properly, that is good - but 
&

RE: Upload Data Volume

[Nicolas Bouige]

Hi Rafael,


I know this is not a proper way to work directly on the DB. But sometimes 
that's permit me to "unblock"  situations specially with VMs...so why not give 
a try ?


I used the link created by Cloudpatform and i uploaded it in Cloudstack using 
the "upload" button in the storage tabs.


The .vhd is present into the cloudstack SSVM with his volume.properties file 
and correct informations...(that's why i just tried to change the flag..  )


I did not see any related error messages, i going to try again and check in 
real time just in case...thanks !


Best regards,
N.B




De : Rafael Weingärtner 
Envoyé : vendredi 2 novembre 2018 16:53:54
À : users
Objet : Re: Upload Data Volume

There is a joke I always tell people here; database interventions (in
CloudStack) is forbidden, unless when I execute them. It does not work like
that, going to the DB and simply changing a flag. You need to understand
why the status is stuck in “uploaded”, and is not changing to ready.

There is something that called my attention now. Are you uploading the file
(using your browser)? Or, are you configuring the template/data disk as link,
and then the SSVM is downloading them? These are two distinct operations. I
was under the impression that you were using the latter. So, which one of
these methods are you using? If nothing is working, I would expect some
error messages either in the CloudStack management server, system VMs or in
the hosts. Did you check them?

On Fri, Nov 2, 2018 at 12:32 PM Nicolas Bouige  wrote:

> Hi,
>
> SSVM works fine and we dont use any redirection.
> We are able to Upload the volume-data after some modifications directly on
> the cloudplatform SSVM.
>
> But the problem is still there as we can't attach the volume to a VM. It
> is in a state "Uploaded", i tried to modify manually the state into the DB
> (Uploaded --> Ready) but i ran into a new issue.
> Exception com.xensource.xenapi.Types$UuidInvalid :VDI getByUuid for uuid:
> null failed due to The uuid you supplied was invalid.
>
> -Message d'origine-
> De : Andrija Panic [mailto:andrija.pa...@gmail.com]
> Envoyé : vendredi 2 novembre 2018 16:03
> À : users 
> Objet : Re: Upload Data Volume
>
> Ok, so SSVM can access remote web server properly, that is good - but
> please note, any http redirections are not allowed nor httpS (ssl) when
> downloading volume/template from remote locations via CloudStack, just
> plain http and http response/status code 200...
>
> Can you please also verify health of the SSVM with the script that exists
> inside SSVM: /usr/local/cloud/system.../ssvm-check or similar named...there
> should be no errors...
>
> Best,
> Andrija
>
>
>
> On Fri, Nov 2, 2018, 11:46 Nicolas Bouige 
> > Hi Andrija,
> >
> > I can download the .vhd directly on the SSVM.
> > I tried with the link provided by CP and link from my webserver.
> >
> > Both works fine.
> >
> > From CP, to make it work,  we had to modifiy symbolic link and add
> > permission on the .vhd.
> > 49085a35-73bc-4e11-a40b-c7547c798787.vhd ->
> > /mnt/SecStorage/null/volumes/4/29907/7f9a2e42-04e1-4733-be3e-a00b41cb2
> > f08.vhd
> > ln -sfn
> > /mnt/SecStorage/1766927c-d83e-3423-aa68-87a5d0c71771/volumes/4/29907/7
> > f9a2e42-04e1-4733-be3e-a00b41cb2f08.vhd
> > 49085a35-73bc-4e11-a40b-c7547c798787.vhd
> >
> > Now, the DATA volume is staying in state "Uploaded" and we are not
> > able to make any operations with it , like attach disk to a VM.
> >
> > Best regards,
> > N.B
> >
> > -Message d'origine-
> > De : Andrija Panic [mailto:andrija.pa...@gmail.com] Envoyé : mercredi
> > 31 octobre 2018 18:45 À : users  Objet :
> > Re: Upload Data Volume
> >
> > Login inside SSVM in destination CloudStack and try to wget the volume
> > as suggested in previous email. See if that works first.
> >
> > Best
> > Andrija
> >
> > On Wed, Oct 31, 2018, 18:36 Nicolas Bouige  wrote:
> >
> > > Nope, i exported the .vhd manually from my Xen pool  and put it in a
> > > webserver to be download by Cloudstack.
> > > This step works fine. So that's not a problem if the download volume
> > > feature from CloudPlatform didn' work.
> > >
> > > i'm in trouble with the second part, to import the .vhd hosted on my
> > > webserver  to cloudstack.
> > >
> > >
> > > -Message d'origine-
> > > De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com]
> > > Envoyé : mercredi 31 octobre 2018 18:26 À : users
> > >  Objet : Re: Upload Data Volume
> > >
> > > Ah, 

RE: Upload Data Volume

[Nicolas Bouige]

Hi,

SSVM works fine and we dont use any redirection.
We are able to Upload the volume-data after some modifications directly on the 
cloudplatform SSVM.

But the problem is still there as we can't attach the volume to a VM. It is in 
a state "Uploaded", i tried to modify manually the state into the DB (Uploaded 
--> Ready) but i ran into a new issue.
Exception com.xensource.xenapi.Types$UuidInvalid :VDI getByUuid for uuid: null 
failed due to The uuid you supplied was invalid.

-Message d'origine-
De : Andrija Panic [mailto:andrija.pa...@gmail.com] 
Envoyé : vendredi 2 novembre 2018 16:03
À : users 
Objet : Re: Upload Data Volume

Ok, so SSVM can access remote web server properly, that is good - but please 
note, any http redirections are not allowed nor httpS (ssl) when downloading 
volume/template from remote locations via CloudStack, just plain http and http 
response/status code 200...

Can you please also verify health of the SSVM with the script that exists 
inside SSVM: /usr/local/cloud/system.../ssvm-check or similar named...there 
should be no errors...

Best,
Andrija



On Fri, Nov 2, 2018, 11:46 Nicolas Bouige  Hi Andrija,
>
> I can download the .vhd directly on the SSVM.
> I tried with the link provided by CP and link from my webserver.
>
> Both works fine.
>
> From CP, to make it work,  we had to modifiy symbolic link and add 
> permission on the .vhd.
> 49085a35-73bc-4e11-a40b-c7547c798787.vhd -> 
> /mnt/SecStorage/null/volumes/4/29907/7f9a2e42-04e1-4733-be3e-a00b41cb2
> f08.vhd
> ln -sfn
> /mnt/SecStorage/1766927c-d83e-3423-aa68-87a5d0c71771/volumes/4/29907/7
> f9a2e42-04e1-4733-be3e-a00b41cb2f08.vhd
> 49085a35-73bc-4e11-a40b-c7547c798787.vhd
>
> Now, the DATA volume is staying in state "Uploaded" and we are not 
> able to make any operations with it , like attach disk to a VM.
>
> Best regards,
> N.B
>
> -Message d'origine-
> De : Andrija Panic [mailto:andrija.pa...@gmail.com] Envoyé : mercredi 
> 31 octobre 2018 18:45 À : users  Objet : 
> Re: Upload Data Volume
>
> Login inside SSVM in destination CloudStack and try to wget the volume 
> as suggested in previous email. See if that works first.
>
> Best
> Andrija
>
> On Wed, Oct 31, 2018, 18:36 Nicolas Bouige  wrote:
>
> > Nope, i exported the .vhd manually from my Xen pool  and put it in a 
> > webserver to be download by Cloudstack.
> > This step works fine. So that's not a problem if the download volume 
> > feature from CloudPlatform didn' work.
> >
> > i'm in trouble with the second part, to import the .vhd hosted on my 
> > webserver  to cloudstack.
> >
> >
> > -Message d'origine-
> > De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com]
> > Envoyé : mercredi 31 octobre 2018 18:26 À : users 
> >  Objet : Re: Upload Data Volume
> >
> > Ah, so the cloud platform is not working properly. I mean, it is not 
> > exporting the VHDs already. That is your problem, right?
> >
> > On Wed, Oct 31, 2018 at 2:24 PM Nicolas Bouige 
> wrote:
> >
> > > Hi Rafael,
> > >
> > > Did not try as i got an access denied via my web browser...that's 
> > > why i exported the .vhd on my own webserver instead to use this
> feature.
> > >
> > > Best regards,
> > > N.B
> > >
> > > -Message d'origine-
> > > De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com]
> > > Envoyé : mercredi 31 octobre 2018 17:36 À : users 
> > >  Objet : Re: Upload Data Volume
> > >
> > > Did you log in the SSVM of your ACS 4.11, and tried to wget the 
> > > template/data disk using the URL provided by the Citrix/Accelerite 
> > > cloud platform?
> > >
> > > On Wed, Oct 31, 2018 at 1:33 PM Nicolas Bouige 
> > wrote:
> > >
> > > > Hello All,
> > > >
> > > >
> > > >
> > > > I’m currently trying to migrate some VMs from Cloudplatform 4.7
> > > > (XEN) to Cloudstack 4.11.1 (XCP)
> > > >
> > > >
> > > >
> > > > Im stuck with the DATA Volume, there are exported on a web 
> > > > server and accessible by the  management-server.
> > > >
> > > > Webserver is working  fine as i can upload ROOT disk from it.
> > > >
> > > >
> > > >
> > > > The only difference is for upload ROOT DISK as template SSVM is 
> > > > involved with the public ip  and  DATA DISK, the management 
> > > > server is used with private ip.
> > > > Both ip (private and public) are « pingable » from mgmt-server 
> 

RE: Upload Data Volume

[Nicolas Bouige]

Hi Andrija,

I can download the .vhd directly on the SSVM.
I tried with the link provided by CP and link from my webserver.

Both works fine.

From CP, to make it work,  we had to modifiy symbolic link and add permission 
on the .vhd.
49085a35-73bc-4e11-a40b-c7547c798787.vhd -> 
/mnt/SecStorage/null/volumes/4/29907/7f9a2e42-04e1-4733-be3e-a00b41cb2f08.vhd
ln -sfn 
/mnt/SecStorage/1766927c-d83e-3423-aa68-87a5d0c71771/volumes/4/29907/7f9a2e42-04e1-4733-be3e-a00b41cb2f08.vhd
 49085a35-73bc-4e11-a40b-c7547c798787.vhd

Now, the DATA volume is staying in state "Uploaded" and we are not able to make 
any operations with it , like attach disk to a VM.

Best regards,
N.B

-Message d'origine-
De : Andrija Panic [mailto:andrija.pa...@gmail.com] 
Envoyé : mercredi 31 octobre 2018 18:45
À : users 
Objet : Re: Upload Data Volume

Login inside SSVM in destination CloudStack and try to wget the volume as 
suggested in previous email. See if that works first.

Best
Andrija

On Wed, Oct 31, 2018, 18:36 Nicolas Bouige  wrote:

> Nope, i exported the .vhd manually from my Xen pool  and put it in a 
> webserver to be download by Cloudstack.
> This step works fine. So that's not a problem if the download volume 
> feature from CloudPlatform didn' work.
>
> i'm in trouble with the second part, to import the .vhd hosted on my 
> webserver  to cloudstack.
>
>
> -Message d'origine-
> De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com]
> Envoyé : mercredi 31 octobre 2018 18:26 À : users 
>  Objet : Re: Upload Data Volume
>
> Ah, so the cloud platform is not working properly. I mean, it is not 
> exporting the VHDs already. That is your problem, right?
>
> On Wed, Oct 31, 2018 at 2:24 PM Nicolas Bouige  wrote:
>
> > Hi Rafael,
> >
> > Did not try as i got an access denied via my web browser...that's 
> > why i exported the .vhd on my own webserver instead to use this feature.
> >
> > Best regards,
> > N.B
> >
> > -Message d'origine-
> > De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com]
> > Envoyé : mercredi 31 octobre 2018 17:36 À : users 
> >  Objet : Re: Upload Data Volume
> >
> > Did you log in the SSVM of your ACS 4.11, and tried to wget the 
> > template/data disk using the URL provided by the Citrix/Accelerite 
> > cloud platform?
> >
> > On Wed, Oct 31, 2018 at 1:33 PM Nicolas Bouige 
> wrote:
> >
> > > Hello All,
> > >
> > >
> > >
> > > I’m currently trying to migrate some VMs from Cloudplatform 4.7
> > > (XEN) to Cloudstack 4.11.1 (XCP)
> > >
> > >
> > >
> > > Im stuck with the DATA Volume, there are exported on a web server 
> > > and accessible by the  management-server.
> > >
> > > Webserver is working  fine as i can upload ROOT disk from it.
> > >
> > >
> > >
> > > The only difference is for upload ROOT DISK as template SSVM is 
> > > involved with the public ip  and  DATA DISK, the management server 
> > > is used with private ip.
> > > Both ip (private and public) are « pingable » from mgmt-server and 
> > > SSVM and i can download .vhd.
> > >
> > > Unfortunately, each time i try to upload DATA disk, i got this error :
> > >
> > >
> > >
> > > 2018-10-31 15:26:09,493 DEBUG [c.c.a.t.Request]
> > > (AgentManager-Handler-15:null) (logid:) Seq 34-3493104460979266769:
> > > Processing:  { Ans: , MgmtId: 2886211232, via: 34, Ver: v1, Flags:
> > > 10,
> > > [{"com.cloud.agent.api.storage.DownloadAnswer":{"jobId":"1866970c-
> > > b1
> > > 9f
> > > -427a-b957-3738bbd50d0e","downloadPct":0,"errorString":"Connection
> > > refused (Connection
> > > refused)","downloadStatus":"DOWNLOAD_ERROR","downloadPath":"/mnt/S
> > > ec
> > > St
> > > orage/2949d66c-3490-3636-93f7-f2299fa5f4b2/volumes/12/615/dnld3626
> > > 14
> > > 18
> > > 85766458800tmp_","installPath":"volumes/12/615","templateSize":0,"
> > > te mp latePhySicalSize":0,"result":true,"details":"Connection
> > > refused (Connection refused)","wait":0}}] }
> > >
> > > 2018-10-31 15:26:09,563 WARN  [o.a.c.alerts]
> > > (RemoteHostEndPoint-15:ctx-28d55a70) (logid:ebfb4bd8) AlertType:: 
> > > 28
> > > |
> > > dataCenterId:: 0 | podId:: null | clusterId:: null | message::
> > > Failed to upload volume: ce2747c9-468a-4d1f-9651-8b6e2373079b with
> error:
> > > Connection refused (Connection refused)
> > >
> > > 2018-10-31 15:26:09,573 ERROR [o.a.c.s.i.BaseImageStoreDriverImpl]
> > > (RemoteHostEndPoint-15:ctx-28d55a70) (logid:ebfb4bd8) Failed to 
> > > upload
> > > volume: ce2747c9-468a-4d1f-9651-8b6e2373079b with error: 
> > > Connection refused (Connection refused)
> > >
> > >
> > >
> > > I don’t know where the « connection refused » comes from.
> > >
> > >
> > >
> > > Someone knows the exact process in cloudstack to upload volumes 
> > > from an URL ?
> > >
> > > Best regards,
> > >
> > >
> > >
> > > [image: SignatureBouige]
> > >
> > >
> > >
> >
> >
> > --
> > Rafael Weingärtner
> >
>
>
> --
> Rafael Weingärtner
>

RE: Upload Data Volume

[Nicolas Bouige]

Nope, i exported the .vhd manually from my Xen pool  and put it in a webserver 
to be download by Cloudstack. 
This step works fine. So that's not a problem if the download volume feature 
from CloudPlatform didn' work.

i'm in trouble with the second part, to import the .vhd hosted on my webserver  
to cloudstack.


-Message d'origine-
De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com] 
Envoyé : mercredi 31 octobre 2018 18:26
À : users 
Objet : Re: Upload Data Volume

Ah, so the cloud platform is not working properly. I mean, it is not exporting 
the VHDs already. That is your problem, right?

On Wed, Oct 31, 2018 at 2:24 PM Nicolas Bouige  wrote:

> Hi Rafael,
>
> Did not try as i got an access denied via my web browser...that's why 
> i exported the .vhd on my own webserver instead to use this feature.
>
> Best regards,
> N.B
>
> -Message d'origine-
> De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com]
> Envoyé : mercredi 31 octobre 2018 17:36 À : users 
>  Objet : Re: Upload Data Volume
>
> Did you log in the SSVM of your ACS 4.11, and tried to wget the 
> template/data disk using the URL provided by the Citrix/Accelerite 
> cloud platform?
>
> On Wed, Oct 31, 2018 at 1:33 PM Nicolas Bouige  wrote:
>
> > Hello All,
> >
> >
> >
> > I’m currently trying to migrate some VMs from Cloudplatform 4.7 
> > (XEN) to Cloudstack 4.11.1 (XCP)
> >
> >
> >
> > Im stuck with the DATA Volume, there are exported on a web server 
> > and accessible by the  management-server.
> >
> > Webserver is working  fine as i can upload ROOT disk from it.
> >
> >
> >
> > The only difference is for upload ROOT DISK as template SSVM is 
> > involved with the public ip  and  DATA DISK, the management server 
> > is used with private ip.
> > Both ip (private and public) are « pingable » from mgmt-server and 
> > SSVM and i can download .vhd.
> >
> > Unfortunately, each time i try to upload DATA disk, i got this error :
> >
> >
> >
> > 2018-10-31 15:26:09,493 DEBUG [c.c.a.t.Request]
> > (AgentManager-Handler-15:null) (logid:) Seq 34-3493104460979266769:
> > Processing:  { Ans: , MgmtId: 2886211232, via: 34, Ver: v1, Flags: 
> > 10, 
> > [{"com.cloud.agent.api.storage.DownloadAnswer":{"jobId":"1866970c-b1
> > 9f 
> > -427a-b957-3738bbd50d0e","downloadPct":0,"errorString":"Connection
> > refused (Connection
> > refused)","downloadStatus":"DOWNLOAD_ERROR","downloadPath":"/mnt/Sec
> > St
> > orage/2949d66c-3490-3636-93f7-f2299fa5f4b2/volumes/12/615/dnld362614
> > 18 
> > 85766458800tmp_","installPath":"volumes/12/615","templateSize":0,"te
> > mp latePhySicalSize":0,"result":true,"details":"Connection
> > refused (Connection refused)","wait":0}}] }
> >
> > 2018-10-31 15:26:09,563 WARN  [o.a.c.alerts]
> > (RemoteHostEndPoint-15:ctx-28d55a70) (logid:ebfb4bd8) AlertType:: 28 
> > |
> > dataCenterId:: 0 | podId:: null | clusterId:: null | message:: 
> > Failed to upload volume: ce2747c9-468a-4d1f-9651-8b6e2373079b with error:
> > Connection refused (Connection refused)
> >
> > 2018-10-31 15:26:09,573 ERROR [o.a.c.s.i.BaseImageStoreDriverImpl]
> > (RemoteHostEndPoint-15:ctx-28d55a70) (logid:ebfb4bd8) Failed to 
> > upload
> > volume: ce2747c9-468a-4d1f-9651-8b6e2373079b with error: Connection 
> > refused (Connection refused)
> >
> >
> >
> > I don’t know where the « connection refused » comes from.
> >
> >
> >
> > Someone knows the exact process in cloudstack to upload volumes from 
> > an URL ?
> >
> > Best regards,
> >
> >
> >
> > [image: SignatureBouige]
> >
> >
> >
>
>
> --
> Rafael Weingärtner
>


--
Rafael Weingärtner

RE: Upload Data Volume

[Nicolas Bouige]

Hi Rafael,

Did not try as i got an access denied via my web browser...that's why i 
exported the .vhd on my own webserver instead to use this feature.

Best regards,
N.B 

-Message d'origine-
De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com] 
Envoyé : mercredi 31 octobre 2018 17:36
À : users 
Objet : Re: Upload Data Volume

Did you log in the SSVM of your ACS 4.11, and tried to wget the template/data 
disk using the URL provided by the Citrix/Accelerite cloud platform?

On Wed, Oct 31, 2018 at 1:33 PM Nicolas Bouige  wrote:

> Hello All,
>
>
>
> I’m currently trying to migrate some VMs from Cloudplatform 4.7 (XEN) 
> to Cloudstack 4.11.1 (XCP)
>
>
>
> Im stuck with the DATA Volume, there are exported on a web server and 
> accessible by the  management-server.
>
> Webserver is working  fine as i can upload ROOT disk from it.
>
>
>
> The only difference is for upload ROOT DISK as template SSVM is 
> involved with the public ip  and  DATA DISK, the management server is 
> used with private ip.
> Both ip (private and public) are « pingable » from mgmt-server and 
> SSVM and i can download .vhd.
>
> Unfortunately, each time i try to upload DATA disk, i got this error :
>
>
>
> 2018-10-31 15:26:09,493 DEBUG [c.c.a.t.Request]
> (AgentManager-Handler-15:null) (logid:) Seq 34-3493104460979266769:
> Processing:  { Ans: , MgmtId: 2886211232, via: 34, Ver: v1, Flags: 10, 
> [{"com.cloud.agent.api.storage.DownloadAnswer":{"jobId":"1866970c-b19f
> -427a-b957-3738bbd50d0e","downloadPct":0,"errorString":"Connection
> refused (Connection
> refused)","downloadStatus":"DOWNLOAD_ERROR","downloadPath":"/mnt/SecSt
> orage/2949d66c-3490-3636-93f7-f2299fa5f4b2/volumes/12/615/dnld36261418
> 85766458800tmp_","installPath":"volumes/12/615","templateSize":0,"temp
> latePhySicalSize":0,"result":true,"details":"Connection
> refused (Connection refused)","wait":0}}] }
>
> 2018-10-31 15:26:09,563 WARN  [o.a.c.alerts]
> (RemoteHostEndPoint-15:ctx-28d55a70) (logid:ebfb4bd8) AlertType:: 28 |
> dataCenterId:: 0 | podId:: null | clusterId:: null | message:: Failed 
> to upload volume: ce2747c9-468a-4d1f-9651-8b6e2373079b with error: 
> Connection refused (Connection refused)
>
> 2018-10-31 15:26:09,573 ERROR [o.a.c.s.i.BaseImageStoreDriverImpl]
> (RemoteHostEndPoint-15:ctx-28d55a70) (logid:ebfb4bd8) Failed to upload
> volume: ce2747c9-468a-4d1f-9651-8b6e2373079b with error: Connection 
> refused (Connection refused)
>
>
>
> I don’t know where the « connection refused » comes from.
>
>
>
> Someone knows the exact process in cloudstack to upload volumes from 
> an URL ?
>
> Best regards,
>
>
>
> [image: SignatureBouige]
>
>
>


--
Rafael Weingärtner

Upload Data Volume

[Nicolas Bouige]

Hello All,

I'm currently trying to migrate some VMs from Cloudplatform 4.7 (XEN) to 
Cloudstack 4.11.1 (XCP)

Im stuck with the DATA Volume, there are exported on a web server and 
accessible by the  management-server.
Webserver is working  fine as i can upload ROOT disk from it.

The only difference is for upload ROOT DISK as template SSVM is involved with 
the public ip  and  DATA DISK, the management server is used with  private ip.
Both ip (private and public) are < pingable > from mgmt-server and SSVM and i 
can download .vhd.

Unfortunately, each time i try to upload DATA disk, i got this error :

2018-10-31 15:26:09,493 DEBUG [c.c.a.t.Request] (AgentManager-Handler-15:null) 
(logid:) Seq 34-3493104460979266769: Processing:  { Ans: , MgmtId: 2886211232, 
via: 34, Ver: v1, Flags: 10, 
[{"com.cloud.agent.api.storage.DownloadAnswer":{"jobId":"1866970c-b19f-427a-b957-3738bbd50d0e","downloadPct":0,"errorString":"Connection
 refused (Connection 
refused)","downloadStatus":"DOWNLOAD_ERROR","downloadPath":"/mnt/SecStorage/2949d66c-3490-3636-93f7-f2299fa5f4b2/volumes/12/615/dnld3626141885766458800tmp_","installPath":"volumes/12/615","templateSize":0,"templatePhySicalSize":0,"result":true,"details":"Connection
 refused (Connection refused)","wait":0}}] }
2018-10-31 15:26:09,563 WARN  [o.a.c.alerts] 
(RemoteHostEndPoint-15:ctx-28d55a70) (logid:ebfb4bd8) AlertType:: 28 | 
dataCenterId:: 0 | podId:: null | clusterId:: null | message:: Failed to upload 
volume: ce2747c9-468a-4d1f-9651-8b6e2373079b with error: Connection refused 
(Connection refused)
2018-10-31 15:26:09,573 ERROR [o.a.c.s.i.BaseImageStoreDriverImpl] 
(RemoteHostEndPoint-15:ctx-28d55a70) (logid:ebfb4bd8) Failed to upload volume: 
ce2747c9-468a-4d1f-9651-8b6e2373079b with error: Connection refused (Connection 
refused)

I don't know where the < connection refused > comes from.

Someone knows the exact process in cloudstack to upload volumes from an URL ?

Best regards,

[SignatureBouige]

RE: Build template with Packer

[Nicolas Bouige]

Hi René,

Thanks, there are a lot of cool stuff there  ;)

Regards,
N.B

-Message d'origine-
De : Rene Moser [mailto:m...@renemoser.net] 
Envoyé : lundi 8 octobre 2018 11:19
À : users@cloudstack.apache.org
Objet : Re: Build template with Packer

Hi Nicolas

On 10/08/2018 11:10 AM, Nicolas Bouige wrote:
> Did anyone try to set up template for cloudstack with Packer ?
> 
> Especially, to install all prerequisites and update.
> 
> If someone have some json example for cloudstack, so  i could work on 
> it and set up my own build

Also see https://github.com/resmo/awesome-cloudstack#templates

Regards
René

Build template with Packer

[Nicolas Bouige]

Hello All,

Did anyone try to set up template for cloudstack with Packer ?
Especially, to install all prerequisites and update.

If someone have some json example for cloudstack, so  i could work on it and 
set up my own build

Any help would be appreciate.

Thanks,
N.B

[SignatureBouige]

RE: Container service CS 4.11.1

[Nicolas Bouige]

Hi Boris,

Thanks for the update !
I saw the update on the github repository so i tought it was okay ;)

Best regards,
N.B

-Message d'origine-
De : Boris Stoyanov [mailto:boris.stoya...@shapeblue.com] 
Envoyé : jeudi 4 octobre 2018 09:58
À : users@cloudstack.apache.org
Objet : Re: Container service CS 4.11.1

Hi Nicolas,

CCS for 4.11.1 is currently under development and is yet to be released to 
community. I think last supported version is 4.9.3 but that’s not supporting 
XCP-NG unfortunately. We still have work to do, but keep your head up, we might 
be able to release it in the next month or so..

Bobby.


boris.stoya...@shapeblue.com
www.shapeblue.com
Amadeus House, Floral Street, London  WC2E 9DPUK @shapeblue
  
 

On 3 Oct 2018, at 19:34, Nicolas Bouige 
mailto:n.bou...@dimsi.fr>> wrote:

Hello All,

Did anyone try to install the Cloudstack Container Service, without any 
specific troubles,  on Cloudstack 4.11.1 and XCP-NG 7.4 ?
And if yes, with which usage case ?

Thanks !

Best regards,
N.B

[SignatureBouige]

Container service CS 4.11.1

[Nicolas Bouige]

Hello All,

Did anyone try to install the Cloudstack Container Service, without any 
specific troubles,  on Cloudstack 4.11.1 and XCP-NG 7.4 ?
And if yes, with which usage case ?

Thanks !

Best regards,
N.B

[SignatureBouige]

RE: Unable to deploy/restart VM

[Nicolas Bouige]

Hi Ivan and Rohit,

Thanks for your answer, The issue is closed, i juste added a new Host to CS ( 
move from three host to four host)
the CPU allocated was 150% on two of three  host on my cluster and CS didn't 
allow to deploy vm on the whole cluster , even if one host as enought 
ressources, as the threshold is for the cluster and not for the hosts.

Best regards,
N.B
-Message d'origine-
De : Ivan Kudryavtsev [mailto:kudryavtsev...@bw-sw.com] 
Envoyé : mercredi 3 octobre 2018 17:06
À : users 
Objet : Re: Unable to deploy/restart VM

Hi, Nicolas. Any chance you are using local storage for VM volumes and 
decreased memory on a certain node? Local storage-based VMs don't have certain 
migration features to be rebalanced across the nodes. E.g. if you have 
DATA-volumes attached it cannot be migrated.

ср, 3 окт. 2018 г. в 8:35, Rohit Yadav :

> Hi Nicolas,
>
>
> What is the version of CloudStack you're using? You get this error 
> when the cluster cpu/memory thresholds have been exceeded and the 
> cluster will be skipped by the deployment planner for deploying the 
> VM. Check if the cluster where you were trying to deploy the VM has 
> enough CPU and memory for the guest VM to run.
>
>
> - Rohit
>
> <https://cloudstack.apache.org>
>
>
>
> 
> From: Nicolas Bouige 
> Sent: Monday, September 17, 2018 7:57:08 PM
> To: users@cloudstack.apache.org
> Subject: Unable to deploy/restart VM
>
>
> Hello All,
>
>
>
> On one of my cluster i can’t deploy and/or restart (after cpu/memory 
> update), the only information i’ve got about the probleme is this message :
>
>
>
> Cannot allocate cluster list [1] for vm creation since their allocated 
> percentage crosses the disable capacity threshold defined at each 
> cluster at global value for capacity Type : 1, skipping these clusters.
>
> I checked cluster and global configuration cpu threshold, there are 
> both with the same value.
>
>
> I was able to restart the impacted Vms with the global admin account 
> as i could chose the host.
>
> So why the VMs did not start on the available host ? and Why the error 
> log appears event if the threshold is not reach on, at least, one of my 
> server ?
>
>
>
> Did i miss something ?  or do i need to check something else ?
>
>
> Best regards,
>
> [SignatureBouige]
>
>
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London  WC2E 9DPUK @shapeblue
>
>
>
>

-- 
With best regards, Ivan Kudryavtsev
Bitworks LLC
Cell: +7-923-414-1515
WWW: http://bitworks.software/ <http://bw-sw.com/>

RE: Add NIC and public IP

[Nicolas Bouige]

Hi,

It's okay..
So i have to delete the old primary NIC to force CS to update the new primary 
card and add the old network..

Best regards,
N.B
De : Nicolas Bouige [mailto:n.bou...@dimsi.fr]
Envoyé : lundi 24 septembre 2018 14:25
À : users@cloudstack.apache.org
Objet : Add NIC and public IP

Hello All,

I'm trying to add an ip public to one of my VM (Windows 2016)  with a new added 
network without success :/
The goal is to put this vm in a DMZ with a NIC in DMZ and the other one on the 
« internal » Tier Network.
The public ip has to be associate to the IP from the DMZ.

So i added the new DMZ network and setted up as the default NIC.
When i choose the Tier network and the VM, CS displays the old primary address 
from the first network.
And after a reboot, none of the NIC card is recognized by Windows...

However, in cloud Database, the VM is correctly updated.

Do i need to do something else to add a new NIC on an existing VM ?
We're using CS 4.11

[SignatureBouige]

Add NIC and public IP

[Nicolas Bouige]

Hello All,

I'm trying to add an ip public to one of my VM (Windows 2016)  with a new added 
network without success :/
The goal is to put this vm in a DMZ with a NIC in DMZ and the other one on the 
< internal > Tier Network.
The public ip has to be associate to the IP from the DMZ.

So i added the new DMZ network and setted up as the default NIC.
When i choose the Tier network and the VM, CS displays the old primary address 
from the first network.
And after a reboot, none of the NIC card is recognized by Windows...

However, in cloud Database, the VM is correctly updated.

Do i need to do something else to add a new NIC on an existing VM ?
We're using CS 4.11

[SignatureBouige]

RE: Unable to migrate instance to new host

[Nicolas Bouige]

Hello Jevgeni,

Whats is your linux distribution ?
On ubuntu 16.04,  We ran into the same problem few month ago and we had to 
modify the libvirt-bin.service as well.

'ExecStart=/usr/sbin/libvirtd $libvirtd_opts' >> 'ExecStart=/usr/sbin/libvirtd 
-l $libvirtd_opts'

Add the "-l" before $libvirtd_opts

Best regards,
N.B 

-Message d'origine-
De : Jevgeni Zolotarjov [mailto:j.zolotar...@gmail.com] 
Envoyé : mardi 18 septembre 2018 18:10
À : users@cloudstack.apache.org
Objet : Unable to migrate instance to new host

We were running cloudstack 4.11.1 with 1 host.
Now we added another identical host.

The procedure completed successfully.

But the attempt to migrate instance to this new host fails with error
message:

Migration was refused connection to destination: qemu+tcp://A.B.C.D/system.
Please check libvirt configuration compatibility and firewall rules on the 
source and destination hosts.

iptables configuration on both hosts is the one suggested here 
http://docs.cloudstack.apache.org/projects/archived-cloudstack-installation/en/4.11/hypervisor/kvm.html#configuring-the-firewall

Please help.

Unable to deploy/restart VM

[Nicolas Bouige]

Hello All,

On one of my cluster i can't deploy and/or restart (after cpu/memory update), 
the only information i've got about the probleme is this message :

Cannot allocate cluster list [1] for vm creation since their allocated 
percentage crosses the disable capacity threshold defined at each cluster at 
global value for capacity Type : 1, skipping these clusters.

I checked cluster and global configuration cpu threshold, there are both with 
the same value.

I was able to restart the impacted Vms with the global admin account as i could 
chose the host.
So why the VMs did not start on the available host ? and Why the error log 
appears event if the threshold is not reach on, at least, one of my server ?

Did i miss something ?  or do i need to check something else ?

Best regards,
[SignatureBouige]

RE: qemu2 images are being corrupted

[Nicolas Bouige]

Hi All,

Maybe this is not related but that's seem know qemu corrup .qcow2 image with 
internal snapshot

https://www.linux-kvm.org/images/6/65/02x08B-Max_Reitz-Backups_with_QEMU.pdf 
(slide 13/15)

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40



De : cloudstack-fan 
Envoyé : samedi 18 août 2018 13:06:08
À : users@cloudstack.apache.org
Objet : Re: qemu2 images are being corrupted

Dear colleagues,

You might find it interesting:
https://forum.proxmox.com/threads/qcow2-corruption-after-snapshot-or-heavy-disk-i-o.32865/

It seems that qemu-kvm really could corrupt a QCOW2 image. :-(

What do you think, is that possible to avoid that? Maybe there's an option to 
use RAW forman instead of QCOW2?

Thanks!

‐‐‐ Original Message ‐‐‐
On 2 July 2018 12:21 PM, cloudstack-fan  wrote:

> Dear colleagues,
>
> I'm posting as an anonymous user, because there's a thing that concerns me a 
> little and I'd like to share my experience with you, so maybe some people 
> could relate to the same. ACS is amazing, it solves my tasks for 6 years, I'm 
> running a few ACS-backed clouds that contain hundreds and hundreds of VMs. 
> I'm enjoying ACS really much, but there's a thing that scares me sometimes.
>
> It happens pretty seldom, but the more VMs you have is the more chances you 
> run into this glitch. It usually happens on the sly and you don't get any 
> error messages in log-files of your cloudstack-management server or a 
> cloudstack-agent, so you don't even know that something had happened until 
> you see that a virtual machine is having major problems. If you're lucky, you 
> see it on the same day when it happens, but if you aren't - you won't suspect 
> anything unusual for a week, but at some moment you realize that the 
> filesystem had become a mess and you can't do anything to restore it. You're 
> trying to restore it from a snapshot, but if you don't have a snapshot that 
> would be created before the incident, your snapshots won't help. :-(
>
> I experienced it for about 5-7 times during the last 5-6 years and there are 
> a few conditions that always present:
>  * it happens on KVM-based hosts (I experienced itt with CentOS 6 and CentOS 
> 7) with qcow2-images (either 0.10 and 1.1 versions);
>  * it happens on primary storages running different filesystems (I 
> experiences it with local XFS and network-based GFS2 and NFS);
>  * it happens when a volume snapshot is being made, according to the 
> log-files inside of a VM (guest's operating system's kernel starts 
> complaining on a filesystem errors);
>  * at the same time, as I wrote before, there are NO error messages in the 
> log-files outside of a VM which disk image is corrupted;
>  * but when you run `qemu-img check ...` to check the image, you may see a 
> lot of leaked clusters (that's why I'd strongly advice to check each and 
> every image one each and every primary storage at least once per hour by a 
> script being run by your monitoring system, something kind of `for imagefile 
> in $(find /var/lib/libvirt/images -maxdepth 1 -type f); do { 
> /usr/bin/qemu-img check "${imagfile}"; if [[ ${?} -ne 0 ]]; then { ... } fi; 
> } done`);
>  * when it happens you can also find a record in the snapshot_store_ref table 
> that refers to the snapshot on a primary storage (see an example here 
> https://pastebin.com/BuxCXVSq) - this record should have been removed when 
> the snapshot's state is being changed from "BackingUp" to "BackedUp", but it 
> isn't being removed in this case. At the same time, this snapshot isn't being 
> listed in the output of `qemu-img snapshot -l ...`, so that's why I suppose 
> that the image is being corrupted when ACS deletes the snapshot that has been 
> backed up (it tries to delete the snapshot, but something goes wrong, image 
> is being corrupted, but ACS thinks that everything's fine and changes the 
> status to "BackedUp" without a bit of qualm);
>  * if you're trying to restore this VM's image from the same snapshot that 
> has caused destruction or any other snapshot that has been made after that, 
> you'll find the same corrupted filesystem inside, but the snapshot's image 
> that is stored in your secondary storage doesn't show anything wrong when you 
> run `qemu-img check ...` (so you can restore your image only if you have a 
> snapshot that had been created AND stored before the incident).
>
> As I wrote, I saw several times in different environments and different 
> versions of ACS. I'm pretty sure that it's not only me who had such a luck to 
> experience the same glitch, so let's share our stories. Maybe together we'll 
> find out why does it happen and how to prevent that in future.
>
> Thanks in advance,
> An Anonymous ACS Fan

RE: add new ip range to zone

[Nicolas Bouige]

Hi Rafael,

yes, we had the situation but both ranges of public IPs was already in the same 
VLAN.

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40



De : Rafael Weingärtner 
Envoyé : mercredi 18 juillet 2018 13:02:41
À : users
Objet : Re: add new ip range to zone

Nicolas, have you had the situation where 2 public IPs from different
ranges were assigned to the same VPC(VR)?
Here we had this situation, and it was not playing well in our network
setup. We had to move everything to the same VLAN.

On Wed, Jul 18, 2018 at 7:25 AM, Nicolas Bouige  wrote:

> Hello Swen,
>
> In our deployment, we are using two different network for public ip in the
> same vlan without any problem.
>
> Best regards,
> N.B
>
> -Message d'origine-
> De : Swen - swen.io [mailto:m...@swen.io]
> Envoyé : mercredi 18 juillet 2018 11:26
> À : users@cloudstack.apache.org
> Objet : add new ip range to zone
>
> Hello all,
>
> We are using advanced networking in our installation and we need to add
> new public ips.
> Should we configure the new /24 network in the same vlan as the other one
> or should we use a vlan-id for each /24 network? What do you recommend?
>
> I am concerned what is happening if a VR gets public ips from different
> /24 networks within the same or in different vlans? Is there a chance that
> a routing problem of any kind will accrue?
>
> Thank you for your 2 cents (or more) on this!
>
> Best regards,
> Swen
>
>
>
>


--
Rafael Weingärtner

RE: add new ip range to zone

[Nicolas Bouige]

Hello Swen,

In our deployment, we are using two different network for public ip in the same 
vlan without any problem.

Best regards,
N.B

-Message d'origine-
De : Swen - swen.io [mailto:m...@swen.io] 
Envoyé : mercredi 18 juillet 2018 11:26
À : users@cloudstack.apache.org
Objet : add new ip range to zone

Hello all,

We are using advanced networking in our installation and we need to add new 
public ips.
Should we configure the new /24 network in the same vlan as the other one or 
should we use a vlan-id for each /24 network? What do you recommend?

I am concerned what is happening if a VR gets public ips from different /24 
networks within the same or in different vlans? Is there a chance that a 
routing problem of any kind will accrue?

Thank you for your 2 cents (or more) on this!

Best regards,
Swen
 

RE: VPC vitual router stuck in starting

[Nicolas Bouige]

Hello Jon,


okay, good news :)


For the futur, you have to use this credentials to connect to the VR in console 
:
root
password


have a great day,

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40



De : Jon Marshall 
Envoyé : mercredi 18 juillet 2018 10:58:32
À : users@cloudstack.apache.org
Objet : Re: VPC vitual router stuck in starting

The virtual router for the VPC finally went to stopped and I did a restart VPC 
and did a clean up and the VR restarted. I could then restart the VMs.





From: Jon Marshall 
Sent: 17 July 2018 13:46
To: users@cloudstack.apache.org
Subject: RE: VPC vitual router stuck in starting

Hi Jon,

It is possible to connect directly to the VR via console KVM ? (virsh console 
r-XXX-VM)
If yes, please check cloud.log, State "starting"  from CS doesn't mean it's not 
okay from KVM

The cloud-agent log on KVM host could be useful as well.


Best regards,
N.B


-Message d'origine-
De : Jon Marshall [mailto:jms@hotmail.co.uk]
Envoyé : mardi 17 juillet 2018 12:28
À : users@cloudstack.apache.org
Objet : VPC vitual router stuck in starting

Testing with advanced networking v4.11 using KVM.


I setup some isolated networks (2) and then a VPC which all worked fine. I then 
rebooted compute nodes (x3) and manager and when it all came back the VPC 
virtual router is stuck in starting as are the VMs in the VPC.


I have checked the management server logs and I see a lot of -


com.cloud.utils.exception.ExecutionException: Unable to start  
VM:ed9a140a-9cd6-47e9-a2b7-8d34ca5b6ca7 due to error in finalizeStart, not 
retrying
com.cloud.exception.AgentUnavailableException: Resource [Host:4] is 
unreachable: Host 4: Unable to start instance due to Unable to start  
VM:ed9a140a-9cd6-47e9-a2b7-8d34ca5b6ca7 due to error in finalizeStart, not 
retrying Caused by: com.cloud.utils.exception.ExecutionException: Unable to 
start  VM:ed9a140a-9cd6-47e9-a2b7-8d34ca5b6ca7 due to error in finalizeStart, 
not retrying
com.cloud.utils.exception.ExecutionException: Unable to start  
VM:ed9a140a-9cd6-47e9-a2b7-8d34ca5b6ca7 due to error in finalizeStart, not 
retrying

it says Host 4 is not reachable but I have another virtual router for one of 
the isolated networks and some guest VMs running on the same host.


1) Does anyone have any suggestions as to how to troubleshoot this beyond 
looking through the logs ?


2) how can I stop the vritual router, destroy it and recreate it as while it is 
starting you cannot do anything with it ?


thanks

RE: VPC vitual router stuck in starting

[Nicolas Bouige]

Hi Jon,

It is possible to connect directly to the VR via console KVM ? (virsh console 
r-XXX-VM)
If yes, please check cloud.log, State "starting"  from CS doesn't mean it's not 
okay from KVM

The cloud-agent log on KVM host could be useful as well.


Best regards,
N.B


-Message d'origine-
De : Jon Marshall [mailto:jms@hotmail.co.uk] 
Envoyé : mardi 17 juillet 2018 12:28
À : users@cloudstack.apache.org
Objet : VPC vitual router stuck in starting

Testing with advanced networking v4.11 using KVM.


I setup some isolated networks (2) and then a VPC which all worked fine. I then 
rebooted compute nodes (x3) and manager and when it all came back the VPC 
virtual router is stuck in starting as are the VMs in the VPC.


I have checked the management server logs and I see a lot of -


com.cloud.utils.exception.ExecutionException: Unable to start  
VM:ed9a140a-9cd6-47e9-a2b7-8d34ca5b6ca7 due to error in finalizeStart, not 
retrying
com.cloud.exception.AgentUnavailableException: Resource [Host:4] is 
unreachable: Host 4: Unable to start instance due to Unable to start  
VM:ed9a140a-9cd6-47e9-a2b7-8d34ca5b6ca7 due to error in finalizeStart, not 
retrying Caused by: com.cloud.utils.exception.ExecutionException: Unable to 
start  VM:ed9a140a-9cd6-47e9-a2b7-8d34ca5b6ca7 due to error in finalizeStart, 
not retrying
com.cloud.utils.exception.ExecutionException: Unable to start  
VM:ed9a140a-9cd6-47e9-a2b7-8d34ca5b6ca7 due to error in finalizeStart, not 
retrying

it says Host 4 is not reachable but I have another virtual router for one of 
the isolated networks and some guest VMs running on the same host.


1) Does anyone have any suggestions as to how to troubleshoot this beyond 
looking through the logs ?


2) how can I stop the vritual router, destroy it and recreate it as while it is 
starting you cannot do anything with it ?


thanks

RE: Isolated network VR

[Nicolas Bouige]

Hello All,


I downgraded one of my kvm to the version 1.5 from base depot.
i create one VR and its working fine with this qemu version.


The issue seems to come from the communication between KVM and guest via the 
vports


XML configuration on both VR are the same (except one line  in channel type = unix)


>From KVM side on netstat, we can see :

qemu/kvm 2.3 :

unix  2  [ ] STREAM CONNECTE  39748167 
/var/lib/libvirt/qemu/r-73-VM.agent

unix  2  [ ] STREAM ENCONNEXION   0
/var/lib/libvirt/qemu/r-73-VM.agent


qemu/kvm 1.5:

unix  2  [ ] STREAM CONNECTE  38420851 
/var/lib/libvirt/qemu/r-74-VM.agent


>From guest side (faulty VR) :

root@systemvm:/sys/kernel/debug/virtio-ports# cat vport3p1
name: r-73-VM.vport
guest_connected: 0
host_connected: 1
outvq_full: 0
bytes_sent: 0
bytes_received: 0
bytes_discarded: 0
is_console: no
console_vtermno: 0

>From guest side (good VR) :

root@systemvm:/sys/kernel/debug/virtio-ports# cat vport1p1

name: r-74-VM.vport

guest_connected: 0
host_connected: 1
outvq_full: 0
bytes_sent: 0
bytes_received: 864
bytes_discarded: 0
is_console: no
console_vtermno: 0

As we can see the working VR have "bytes_received" not the faulty one, i'm 
guessing the bytes correspond at the succes of the patchviasocket.py


Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40


________
De : Nicolas Bouige 
Envoyé : mercredi 20 juin 2018 11:32:38
À : users@cloudstack.apache.org
Objet : Isolated network VR

Hello All,


i'm facing a new issue with new VR creation (Guest network/VPC network)

it's look like the configuration is not push to the virtual router...


>From GUI web, creation is working fine except the router remains in state 
>"starting"


log from KVM :

2018-06-19 16:10:18,020 WARN  [kvm.resource.LibvirtComputingResource] 
(agentRequest-Handler-3:null) (logid:d9e07a00) Timed out: 
/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patchviasocket.py -n 
r-256-VM -p 
%template=domP%name=r-256-VM%eth2ip=159.180.X.X%eth2mask=255.255.255.X%gateway=159.180.X.X%eth0ip=10.1.1.1%eth0mask=255.255.255.0%domain=cs5cloud.internal%cidrsize=24%dhcprange=10.1.1.1%eth1ip=169.254.0.148%eth1mask=255.255.0.0%type=router%disable_rp_filter=true%dns1=8.8.8.8%baremetalnotificationsecuritykey=BjMd_9Gs0da26DHEHBNAmSDK64_4s4rwhuNtiQY0vk3rpR1ICQelMbWgPI-YMA1PZxvH4hHDTgoDOKxw7AigkQ%baremetalnotificationapikey=QbcXriZUmi6z-hCAKIBswy47CenZYTvIbsYsYbfcWdQXn8iSEpjjQoarOKRWf4GDWJDUfdCs_pLH-bfCyTrxIg%host=172.8.X.X%port=8080
 .  Output is:

2018-06-19 16:10:18,020 ERROR [kvm.resource.LibvirtComputingResource] 
(agentRequest-Handler-3:null) (logid:d9e07a00) passcmd failed:timeout


log from Virtual-Router (script can't detect the system-vm as a virtual-router):

Tue Jun 19 16:13:18 UTC 2018 Executing cloud-early-config

Tue Jun 19 16:13:18 UTC 2018 Detected that we are running inside kvm
Tue Jun 19 16:13:49 UTC 2018 Scripts checksum detected: 
oldmd5=3abb34cc8308c19c684676a7deaf86c8 newmd5=3abb34cc8308c19c684676a7deaf86c8
Tue Jun 19 16:13:49 UTC 2018 Configuring systemvm type=
Tue Jun 19 16:13:49 UTC 2018 Finished setting up systemvm

i updated qemu-kvm from qemu-kvm 1.5.3 to qemu-kvm-ev 2.3, do you think this 
version could be incompatible with CS 4.11 ?

QEMU emulator version 2.3.0 (qemu-kvm-ev-2.3.0-29.1.el7)

i made this update to perfom --quiesce backup of the VMs.


Best regards,

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40

Isolated network VR

[Nicolas Bouige]

Hello All,


i'm facing a new issue with new VR creation (Guest network/VPC network)

it's look like the configuration is not push to the virtual router...


>From GUI web, creation is working fine except the router remains in state 
>"starting"


log from KVM :

2018-06-19 16:10:18,020 WARN  [kvm.resource.LibvirtComputingResource] 
(agentRequest-Handler-3:null) (logid:d9e07a00) Timed out: 
/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patchviasocket.py -n 
r-256-VM -p 
%template=domP%name=r-256-VM%eth2ip=159.180.X.X%eth2mask=255.255.255.X%gateway=159.180.X.X%eth0ip=10.1.1.1%eth0mask=255.255.255.0%domain=cs5cloud.internal%cidrsize=24%dhcprange=10.1.1.1%eth1ip=169.254.0.148%eth1mask=255.255.0.0%type=router%disable_rp_filter=true%dns1=8.8.8.8%baremetalnotificationsecuritykey=BjMd_9Gs0da26DHEHBNAmSDK64_4s4rwhuNtiQY0vk3rpR1ICQelMbWgPI-YMA1PZxvH4hHDTgoDOKxw7AigkQ%baremetalnotificationapikey=QbcXriZUmi6z-hCAKIBswy47CenZYTvIbsYsYbfcWdQXn8iSEpjjQoarOKRWf4GDWJDUfdCs_pLH-bfCyTrxIg%host=172.8.X.X%port=8080
 .  Output is:

2018-06-19 16:10:18,020 ERROR [kvm.resource.LibvirtComputingResource] 
(agentRequest-Handler-3:null) (logid:d9e07a00) passcmd failed:timeout


log from Virtual-Router (script can't detect the system-vm as a virtual-router):

Tue Jun 19 16:13:18 UTC 2018 Executing cloud-early-config

Tue Jun 19 16:13:18 UTC 2018 Detected that we are running inside kvm
Tue Jun 19 16:13:49 UTC 2018 Scripts checksum detected: 
oldmd5=3abb34cc8308c19c684676a7deaf86c8 newmd5=3abb34cc8308c19c684676a7deaf86c8
Tue Jun 19 16:13:49 UTC 2018 Configuring systemvm type=
Tue Jun 19 16:13:49 UTC 2018 Finished setting up systemvm

i updated qemu-kvm from qemu-kvm 1.5.3 to qemu-kvm-ev 2.3, do you think this 
version could be incompatible with CS 4.11 ?

QEMU emulator version 2.3.0 (qemu-kvm-ev-2.3.0-29.1.el7)

i made this update to perfom --quiesce backup of the VMs.


Best regards,

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40

RE: DB Communication Link Failure

[Nicolas Bouige]

Hello All,


My DB is now up and running.

i just executed the following commands :


mysql > alter table cloud.event engine = innoDB

mysql> check table cloud.event


The check command give me an "OK", so i restarted the cloudstack-management.

Now, all is good.

Thanks you all for your help !

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40


________
De : Nicolas Bouige
Envoyé : lundi 18 juin 2018 14:38:28
À : users@cloudstack.apache.org
Objet : RE: DB Communication Link Failure


Hi Leandro,


i will take a look to this tool, thanks for the information !


Nicolas Bouige

DIMSI

cloud.dimsi.fr<http://www.cloud.dimsi.fr>

4, avenue Laurent Cely

Tour d’Asnière – 92600 Asnière sur Seine

T/ +33 (0)6 28 98 53 40



De : Leandro Mendes 
Envoyé : lundi 18 juin 2018 14:32:17
À : users@cloudstack.apache.org
Objet : Re: DB Communication Link Failure

Nicolas,

I did not follow the thread properly, but as i saw this line about no
backup and mysql corrupted files it got my attention.

I had a similar problem once and i had used the Percona toolkit. It will
not fix your DB but dump the data so you can reimport it

Good luck.

On Mon, Jun 18, 2018 at 2:18 PM Nicolas Bouige  wrote:

> Stephan,
>
>
> Thanks for your help,
>
> Unfortunately, the --auto-repair switch doesnt work as it's  not support
> by the storage engine...and yes i dont have any backup without the
> corrupted tables.
>
> Nicolas Bouige
> DIMSI
> cloud.dimsi.fr<http://www.cloud.dimsi.fr>
> 4, avenue Laurent Cely
> Tour d’Asnière – 92600 Asnière sur Seine
> T/ +33 (0)6 28 98 53 40
>
>
> 
> De : Stephan Seitz 
> Envoyé : lundi 18 juin 2018 14:07:50
> À : users@cloudstack.apache.org
> Objet : Re: DB Communication Link Failure
>
> Hi!
>
> there's also a --auto-repair switch that could be added to mysqlcheck
> --all-databases.
>
> But to be honest, you can't guarantee the content will match. So
> references to
> other tablefields might not match afterwards (well, i expect these
> references don't match right now either)
>
> As far as your resultset shows, the corruption "only" happened to the
> index-space so your data
> "could" be fine.
>
> Normally, I'ld suggest to revert to a backup, but as this question has
> been around for a few days here,
> I assume your last uncorrupted backup could be far too old.
>
>
>
> Am Montag, den 18.06.2018, 11:56 + schrieb Nicolas Bouige:
> > Hi Stephan,
> >
> >
> > thanks for the command, i could spot which tables is corrupted :
> >
> >
> > cloud.event
> > Warning  : InnoDB: Index 'i_event__created' contains 548 entries, should
> be 542.
> > Warning  : InnoDB: Index 'i_event__user_id' contains 547 entries, should
> be 542.
> > Warning  : InnoDB: Index 'i_event__account_id' contains 547 entries,
> should be 542.
> > Warning  : InnoDB: Index 'i_event__level_id' contains 547 entries,
> should be 542.
> > Warning  : InnoDB: Index 'i_event__type_id' contains 548 entries, should
> be 542.
> > error: Corrupt
> >
> > Now, i supposed i have to delete the entries
> >
> > Nicolas Bouige
> > DIMSI
> > cloud.dimsi.fr<http://www.cloud.dimsi.fr>
> > 4, avenue Laurent Cely
> > Tour d’Asnière – 92600 Asnière sur Seine
> > T/ +33 (0)6 28 98 53 40
> >
> >
> > 
> > De : Stephan Seitz 
> > Envoyé : lundi 18 juin 2018 13:30:43
> > À : users@cloudstack.apache.org
> > Objet : Re: DB Communication Link Failure
> >
> > Hi!
> >
> > This sound's like a corrupted database table. It's not that unusual
> mysqld are
> > restarting after a query reqeuests values from a corrupted table space.
> That
> > behaviour subsequently results in aborted connections.
> >
> > I'ld double check database consistency. The easist way to check against
> > (at least physical) corruption should be mysqlcheckk --all-databases
> >
> > cheers,
> >
> > Stephan
> >
> >
> > Am Montag, den 18.06.2018, 12:47 +0200 schrieb Rafael Weingärtner:
> > >
> > > Your timeout configuration seems fine. There must be something wrong in
> > > your network. Or maybe in your MySQL service; as you said, it is
> restarting
> > > when you run commands against it. Therefore, it might be better to
> > > eliminate these issues first.
> > >
> > > On Mon, Jun 18, 2018 at 11:56 A

RE: DB Communication Link Failure

[Nicolas Bouige]

Hi Leandro,


i will take a look to this tool, thanks for the information !

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40



De : Leandro Mendes 
Envoyé : lundi 18 juin 2018 14:32:17
À : users@cloudstack.apache.org
Objet : Re: DB Communication Link Failure

Nicolas,

I did not follow the thread properly, but as i saw this line about no
backup and mysql corrupted files it got my attention.

I had a similar problem once and i had used the Percona toolkit. It will
not fix your DB but dump the data so you can reimport it

Good luck.

On Mon, Jun 18, 2018 at 2:18 PM Nicolas Bouige  wrote:

> Stephan,
>
>
> Thanks for your help,
>
> Unfortunately, the --auto-repair switch doesnt work as it's  not support
> by the storage engine...and yes i dont have any backup without the
> corrupted tables.
>
> Nicolas Bouige
> DIMSI
> cloud.dimsi.fr<http://www.cloud.dimsi.fr>
> 4, avenue Laurent Cely
> Tour d’Asnière – 92600 Asnière sur Seine
> T/ +33 (0)6 28 98 53 40
>
>
> 
> De : Stephan Seitz 
> Envoyé : lundi 18 juin 2018 14:07:50
> À : users@cloudstack.apache.org
> Objet : Re: DB Communication Link Failure
>
> Hi!
>
> there's also a --auto-repair switch that could be added to mysqlcheck
> --all-databases.
>
> But to be honest, you can't guarantee the content will match. So
> references to
> other tablefields might not match afterwards (well, i expect these
> references don't match right now either)
>
> As far as your resultset shows, the corruption "only" happened to the
> index-space so your data
> "could" be fine.
>
> Normally, I'ld suggest to revert to a backup, but as this question has
> been around for a few days here,
> I assume your last uncorrupted backup could be far too old.
>
>
>
> Am Montag, den 18.06.2018, 11:56 + schrieb Nicolas Bouige:
> > Hi Stephan,
> >
> >
> > thanks for the command, i could spot which tables is corrupted :
> >
> >
> > cloud.event
> > Warning  : InnoDB: Index 'i_event__created' contains 548 entries, should
> be 542.
> > Warning  : InnoDB: Index 'i_event__user_id' contains 547 entries, should
> be 542.
> > Warning  : InnoDB: Index 'i_event__account_id' contains 547 entries,
> should be 542.
> > Warning  : InnoDB: Index 'i_event__level_id' contains 547 entries,
> should be 542.
> > Warning  : InnoDB: Index 'i_event__type_id' contains 548 entries, should
> be 542.
> > error: Corrupt
> >
> > Now, i supposed i have to delete the entries
> >
> > Nicolas Bouige
> > DIMSI
> > cloud.dimsi.fr<http://www.cloud.dimsi.fr>
> > 4, avenue Laurent Cely
> > Tour d’Asnière – 92600 Asnière sur Seine
> > T/ +33 (0)6 28 98 53 40
> >
> >
> > 
> > De : Stephan Seitz 
> > Envoyé : lundi 18 juin 2018 13:30:43
> > À : users@cloudstack.apache.org
> > Objet : Re: DB Communication Link Failure
> >
> > Hi!
> >
> > This sound's like a corrupted database table. It's not that unusual
> mysqld are
> > restarting after a query reqeuests values from a corrupted table space.
> That
> > behaviour subsequently results in aborted connections.
> >
> > I'ld double check database consistency. The easist way to check against
> > (at least physical) corruption should be mysqlcheckk --all-databases
> >
> > cheers,
> >
> > Stephan
> >
> >
> > Am Montag, den 18.06.2018, 12:47 +0200 schrieb Rafael Weingärtner:
> > >
> > > Your timeout configuration seems fine. There must be something wrong in
> > > your network. Or maybe in your MySQL service; as you said, it is
> restarting
> > > when you run commands against it. Therefore, it might be better to
> > > eliminate these issues first.
> > >
> > > On Mon, Jun 18, 2018 at 11:56 AM, Nicolas Bouige 
> wrote:
> > >
> > > >
> > > >
> > > > Hello Dag,
> > > >
> > > > Im not trying to do a multi-master setup, just recover my DB :/
> > > > I have installed  a second node and connect it to the DB and it's not
> > > > possible to connect to the database server automatically (but
> manually
> > > > yes..)
> > > > On the first node at each sql query sent, the service mysql restart
> on db
> > > > server...
> > > >
> > > >
> > > > @Rafael, the timeout value is 28800
> > > >
> > > >

RE: DB Communication Link Failure

[Nicolas Bouige]

Stephan,


Thanks for your help,

Unfortunately, the --auto-repair switch doesnt work as it's  not support by the 
storage engine...and yes i dont have any backup without the corrupted tables.

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40



De : Stephan Seitz 
Envoyé : lundi 18 juin 2018 14:07:50
À : users@cloudstack.apache.org
Objet : Re: DB Communication Link Failure

Hi!

there's also a --auto-repair switch that could be added to mysqlcheck 
--all-databases.

But to be honest, you can't guarantee the content will match. So references to
other tablefields might not match afterwards (well, i expect these references 
don't match right now either)

As far as your resultset shows, the corruption "only" happened to the 
index-space so your data
"could" be fine.

Normally, I'ld suggest to revert to a backup, but as this question has been 
around for a few days here,
I assume your last uncorrupted backup could be far too old.



Am Montag, den 18.06.2018, 11:56 +0000 schrieb Nicolas Bouige:
> Hi Stephan,
>
>
> thanks for the command, i could spot which tables is corrupted :
>
>
> cloud.event
> Warning  : InnoDB: Index 'i_event__created' contains 548 entries, should be 
> 542.
> Warning  : InnoDB: Index 'i_event__user_id' contains 547 entries, should be 
> 542.
> Warning  : InnoDB: Index 'i_event__account_id' contains 547 entries, should 
> be 542.
> Warning  : InnoDB: Index 'i_event__level_id' contains 547 entries, should be 
> 542.
> Warning  : InnoDB: Index 'i_event__type_id' contains 548 entries, should be 
> 542.
> error    : Corrupt
>
> Now, i supposed i have to delete the entries
>
> Nicolas Bouige
> DIMSI
> cloud.dimsi.fr<http://www.cloud.dimsi.fr>
> 4, avenue Laurent Cely
> Tour d’Asnière – 92600 Asnière sur Seine
> T/ +33 (0)6 28 98 53 40
>
>
> 
> De : Stephan Seitz 
> Envoyé : lundi 18 juin 2018 13:30:43
> À : users@cloudstack.apache.org
> Objet : Re: DB Communication Link Failure
>
> Hi!
>
> This sound's like a corrupted database table. It's not that unusual mysqld are
> restarting after a query reqeuests values from a corrupted table space. That
> behaviour subsequently results in aborted connections.
>
> I'ld double check database consistency. The easist way to check against
> (at least physical) corruption should be mysqlcheckk --all-databases
>
> cheers,
>
> Stephan
>
>
> Am Montag, den 18.06.2018, 12:47 +0200 schrieb Rafael Weingärtner:
> >
> > Your timeout configuration seems fine. There must be something wrong in
> > your network. Or maybe in your MySQL service; as you said, it is restarting
> > when you run commands against it. Therefore, it might be better to
> > eliminate these issues first.
> >
> > On Mon, Jun 18, 2018 at 11:56 AM, Nicolas Bouige  wrote:
> >
> > >
> > >
> > > Hello Dag,
> > >
> > > Im not trying to do a multi-master setup, just recover my DB :/
> > > I have installed  a second node and connect it to the DB and it's not
> > > possible to connect to the database server automatically (but manually
> > > yes..)
> > > On the first node at each sql query sent, the service mysql restart on db
> > > server...
> > >
> > >
> > > @Rafael, the timeout value is 28800
> > >
> > >
> > > mysql> SHOW VARIABLES LIKE 'wait_timeout';
> > > +---+---+
> > > >
> > > >
> > > > Variable_name | Value |
> > > +---+---+
> > > >
> > > >
> > > > wait_timeout  | 28800 |
> > > +---+---+
> > >
> > > Best regards,
> > >
> > >
> > > Nicolas Bouige
> > > DIMSI
> > > cloud.dimsi.fr<http://www.cloud.dimsi.fr>
> > > 4, avenue Laurent Cely
> > > Tour d’Asnière – 92600 Asnière sur Seine
> > > T/ +33 (0)6 28 98 53 40
> > >
> > >
> > > 
> > > De : Dag Sonstebo 
> > > Envoyé : jeudi 14 juin 2018 10:32:13
> > > À : users@cloudstack.apache.org
> > > Objet : Re: DB Communication Link Failure
> > >
> > > What Rafael said…
> > >
> > > In addition – can you confirm you aren’t trying something like a
> > > multi-master MySQL setup? I have seen this cause similar issues.
> > >
> > > Regards,
> > > Dag Sonstebo
> > > Cloud Architect
> > > ShapeBlue
> >

RE: DB Communication Link Failure

[Nicolas Bouige]

Hi Stephan,


thanks for the command, i could spot which tables is corrupted :


cloud.event
Warning  : InnoDB: Index 'i_event__created' contains 548 entries, should be 542.
Warning  : InnoDB: Index 'i_event__user_id' contains 547 entries, should be 542.
Warning  : InnoDB: Index 'i_event__account_id' contains 547 entries, should be 
542.
Warning  : InnoDB: Index 'i_event__level_id' contains 547 entries, should be 
542.
Warning  : InnoDB: Index 'i_event__type_id' contains 548 entries, should be 542.
error: Corrupt

Now, i supposed i have to delete the entries

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40



De : Stephan Seitz 
Envoyé : lundi 18 juin 2018 13:30:43
À : users@cloudstack.apache.org
Objet : Re: DB Communication Link Failure

Hi!

This sound's like a corrupted database table. It's not that unusual mysqld are
restarting after a query reqeuests values from a corrupted table space. That
behaviour subsequently results in aborted connections.

I'ld double check database consistency. The easist way to check against
(at least physical) corruption should be mysqlcheckk --all-databases

cheers,

Stephan


Am Montag, den 18.06.2018, 12:47 +0200 schrieb Rafael Weingärtner:
> Your timeout configuration seems fine. There must be something wrong in
> your network. Or maybe in your MySQL service; as you said, it is restarting
> when you run commands against it. Therefore, it might be better to
> eliminate these issues first.
>
> On Mon, Jun 18, 2018 at 11:56 AM, Nicolas Bouige  wrote:
>
> >
> > Hello Dag,
> >
> > Im not trying to do a multi-master setup, just recover my DB :/
> > I have installed  a second node and connect it to the DB and it's not
> > possible to connect to the database server automatically (but manually
> > yes..)
> > On the first node at each sql query sent, the service mysql restart on db
> > server...
> >
> >
> > @Rafael, the timeout value is 28800
> >
> >
> > mysql> SHOW VARIABLES LIKE 'wait_timeout';
> > +---+---+
> > >
> > > Variable_name | Value |
> > +---+---+
> > >
> > > wait_timeout  | 28800 |
> > +---+---+
> >
> > Best regards,
> >
> >
> > Nicolas Bouige
> > DIMSI
> > cloud.dimsi.fr<http://www.cloud.dimsi.fr>
> > 4, avenue Laurent Cely
> > Tour d’Asnière – 92600 Asnière sur Seine
> > T/ +33 (0)6 28 98 53 40
> >
> >
> > 
> > De : Dag Sonstebo 
> > Envoyé : jeudi 14 juin 2018 10:32:13
> > À : users@cloudstack.apache.org
> > Objet : Re: DB Communication Link Failure
> >
> > What Rafael said…
> >
> > In addition – can you confirm you aren’t trying something like a
> > multi-master MySQL setup? I have seen this cause similar issues.
> >
> > Regards,
> > Dag Sonstebo
> > Cloud Architect
> > ShapeBlue
> >
> > On 13/06/2018, 18:44, "Rafael Weingärtner" 
> > wrote:
> >
> > In this case, I would say that you might be either having some problem
> > in
> > your network, or maybe some timeout in the mysql server.
> > Can you check the following variable?
> > >
> > > show variables like "%timeout%";
> > >
> > >
> >
> > dag.sonst...@shapeblue.com
> > www.shapeblue.com<http://www.shapeblue.com>;
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> > On Wed, Jun 13, 2018 at 6:05 PM, Nicolas Bouige  wrote:
> >
> > > Hello Dag, Rafael,
> > >
> > > Thanks for your answer, i knonw this seems to be a "simple" issue
> > but, as
> > > i said in my previous mail, i checked connectivity between both
> > server
> > > without any problem (ping, telnet, firewall policies, remote mysql
> > > connection...) and yes i restarted mgmt server (and it's not a silly
> > > question...;)..)
> > >
> > > Someone knows something about the rollback transactions ?
> > >
> > > Best regards,
> > >
> > > N.B
> > >
> > > -Message d'origine-
> > > De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com]
> > > Envoyé : mercredi 13 juin 2018 16:59
> > > À : users 
> > > Objet : Re: DB Communication Link Failure
> > >
> > > This might be a silly 

RE: DB Communication Link Failure

[Nicolas Bouige]

Hello Dag,

Im not trying to do a multi-master setup, just recover my DB :/
I have installed  a second node and connect it to the DB and it's not possible 
to connect to the database server automatically (but manually yes..)
On the first node at each sql query sent, the service mysql restart on db 
server...


@Rafael, the timeout value is 28800


mysql> SHOW VARIABLES LIKE 'wait_timeout';
+---+---+
| Variable_name | Value |
+---+---+
| wait_timeout  | 28800 |
+---+---+

Best regards,


Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40



De : Dag Sonstebo 
Envoyé : jeudi 14 juin 2018 10:32:13
À : users@cloudstack.apache.org
Objet : Re: DB Communication Link Failure

What Rafael said…

In addition – can you confirm you aren’t trying something like a multi-master 
MySQL setup? I have seen this cause similar issues.

Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue

On 13/06/2018, 18:44, "Rafael Weingärtner"  wrote:

In this case, I would say that you might be either having some problem in
your network, or maybe some timeout in the mysql server.
Can you check the following variable?
>
> show variables like "%timeout%";
>
>

dag.sonst...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue



On Wed, Jun 13, 2018 at 6:05 PM, Nicolas Bouige  wrote:

> Hello Dag, Rafael,
>
> Thanks for your answer, i knonw this seems to be a "simple" issue but, as
> i said in my previous mail, i checked connectivity between both server
> without any problem (ping, telnet, firewall policies, remote mysql
> connection...) and yes i restarted mgmt server (and it's not a silly
> question...;)..)
>
> Someone knows something about the rollback transactions ?
>
> Best regards,
>
> N.B
>
> -Message d'origine-
> De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com]
> Envoyé : mercredi 13 juin 2018 16:59
> À : users 
> Objet : Re: DB Communication Link Failure
>
> This might be a silly question, but... Have you restarted the management
> server?
> This problem may be caused by the connection pool. In theory, it should
> re-create the connection, but you know, sometimes things just break.
>
> On Wed, Jun 13, 2018 at 4:56 PM, Dag Sonstebo 
> wrote:
>
> > Hi Nicolas – not to dig too far into your analysis or log entries –
> > but in my experience any time we see “Communications link failure” the
> > problem is exactly that – a communications issue between management
    > > and DB, rather than an internal DB issue.
> >
> > Regards,
> > Dag Sonstebo
> > Cloud Architect
> > ShapeBlue
> >
> > From: Nicolas Bouige 
> > Reply-To: "users@cloudstack.apache.org" 
> > Date: Wednesday, 13 June 2018 at 15:04
> > To: "users@cloudstack.apache.org" 
> > Subject: DB Communication Link Failure
> >
> > Hello All,
> >
> > Tonight du to a network issue our management cloudstack server and
> > MariaDb server have been shutdowned and restarted this morning.
> >
> > Unfortunately, we  get some issues of « connectivity » between the
> > mgmt server and db server.
> > Both service (cloudstack-management/mysqld) are up and running without
> > any errors.
> > Db server are available from mgmt server (telnet OK, mysql –h
> > dbserver…ok, firewall policies are good from both sides)
> >
> > However, when we want to login to the GUI cloudstack an execption is
> > raised in mgmt-server.log :
> >
> > 2018-06-13 15:28:41,019 DEBUG [c.c.u.d.T.Transaction]
> > (qtp1796488937-13:ctx-88e441c4) (logid:f9e9b399) Rolling back the
> > transaction: Time = 97 Name =  qtp1796488937-13; called by
> > -TransactionLegacy.rollback:889-TransactionLegacy.removeUpTo:832-
> > TransactionLegacy.close:656-TransactionContextInterceptor.invoke:36-
> > ReflectiveMethodInvocation.proceed:174-ExposeInvocationInterceptor.
> > invoke:92-ReflectiveMethodInvocation.proceed:185-
> > JdkDynamicAopProxy.invoke:212-$Proxy121.persist:-1-ActionEventUtils.
> > persistActionEvent:186-ActionEventUtils.onActionEvent:98-
> > AccountManagerImpl.logoutUser:2096
> > 2018-06-13 15:28:41,020 WARN  [c.c.u.d.T.Transaction]
> > (qtp

RE: DB Communication Link Failure

[Nicolas Bouige]

Hello Dag, Rafael,

Thanks for your answer, i knonw this seems to be a "simple" issue but, as i 
said in my previous mail, i checked connectivity between both server without 
any problem (ping, telnet, firewall policies, remote mysql connection...) and 
yes i restarted mgmt server (and it's not a silly question...;)..)

Someone knows something about the rollback transactions ? 

Best regards,

N.B

-Message d'origine-
De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com] 
Envoyé : mercredi 13 juin 2018 16:59
À : users 
Objet : Re: DB Communication Link Failure

This might be a silly question, but... Have you restarted the management server?
This problem may be caused by the connection pool. In theory, it should 
re-create the connection, but you know, sometimes things just break.

On Wed, Jun 13, 2018 at 4:56 PM, Dag Sonstebo 
wrote:

> Hi Nicolas – not to dig too far into your analysis or log entries – 
> but in my experience any time we see “Communications link failure” the 
> problem is exactly that – a communications issue between management 
> and DB, rather than an internal DB issue.
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
> From: Nicolas Bouige 
> Reply-To: "users@cloudstack.apache.org" 
> Date: Wednesday, 13 June 2018 at 15:04
> To: "users@cloudstack.apache.org" 
> Subject: DB Communication Link Failure
>
> Hello All,
>
> Tonight du to a network issue our management cloudstack server and 
> MariaDb server have been shutdowned and restarted this morning.
>
> Unfortunately, we  get some issues of « connectivity » between the 
> mgmt server and db server.
> Both service (cloudstack-management/mysqld) are up and running without 
> any errors.
> Db server are available from mgmt server (telnet OK, mysql –h 
> dbserver…ok, firewall policies are good from both sides)
>
> However, when we want to login to the GUI cloudstack an execption is 
> raised in mgmt-server.log :
>
> 2018-06-13 15:28:41,019 DEBUG [c.c.u.d.T.Transaction]
> (qtp1796488937-13:ctx-88e441c4) (logid:f9e9b399) Rolling back the
> transaction: Time = 97 Name =  qtp1796488937-13; called by
> -TransactionLegacy.rollback:889-TransactionLegacy.removeUpTo:832-
> TransactionLegacy.close:656-TransactionContextInterceptor.invoke:36-
> ReflectiveMethodInvocation.proceed:174-ExposeInvocationInterceptor.
> invoke:92-ReflectiveMethodInvocation.proceed:185-
> JdkDynamicAopProxy.invoke:212-$Proxy121.persist:-1-ActionEventUtils.
> persistActionEvent:186-ActionEventUtils.onActionEvent:98-
> AccountManagerImpl.logoutUser:2096
> 2018-06-13 15:28:41,020 WARN  [c.c.u.d.T.Transaction]
> (qtp1796488937-13:ctx-88e441c4) (logid:f9e9b399) Unable to rollback
> com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException:
> Communications link failure during rollback(). Transaction resolution 
> unknown.
> at 
> sun.reflect.GeneratedConstructorAccessor98.newInstance(Unknown
> Source)
> at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(
> DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
> at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
> at com.mysql.jdbc.Util.getInstance(Util.java:386)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1015)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975)
> at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920)
> at com.mysql.jdbc.ConnectionImpl.rollback(ConnectionImpl.java:
> 5067)
> at org.apache.commons.dbcp.DelegatingConnection.rollback(
> DelegatingConnection.java:368)
> at org.apache.commons.dbcp.PoolingDataSource$
> PoolGuardConnectionWrapper.rollback(PoolingDataSource.java:323)
> at com.cloud.utils.db.TransactionLegacy.rollbackTransaction(
> TransactionLegacy.java:851)
> at com.cloud.utils.db.TransactionLegacy.rollback(
> TransactionLegacy.java:889)
> at com.cloud.utils.db.TransactionLegacy.removeUpTo(
> TransactionLegacy.java:832)
> at com.cloud.utils.db.TransactionLegacy.close(
> TransactionLegacy.java:656)
> at com.cloud.utils.db.TransactionContextInterceptor.invoke(
> TransactionContextInterceptor.java:36)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.
> proceed(ReflectiveMethodInvocation.java:174)
> at org.springframework.aop.interceptor.
> ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.
> proceed(ReflectiveMethodInvocation

RE: SSVM's not starting, timeout for libvirt python script in agent.log

[Nicolas Bouige]

159%eth0mask=255.255.0.0%eth1ip=6.
> >>> 6.6.6%eth1mask=255.255.255.0%mgmtcidr=
> >>> 9.9.9.0/24%localgw=1.2.3.4%internaldns1=1.2.3.4%dns1=8.8.8.8
> >>> %dns2=8.8.4.4
> >>>
> >>> It could be you have edited the above IP addresses to mask your 
> >>> real addresses – if so ignore this.
> >>>
> >>> If not then the above points to:
> >>> - Management host is on 1.1.1.1
> >>> - Eth2 which for a console proxy is public traffic is on 
> >>> 9.9.9.9/24
> >>> - Eth0 which is the link local management interface is on
> >>> 169.254.3.159/16 (system generated)
> >>> - Eth1 is the main management interface on 6.6.6.6/24
> >>> - You have a gateway address of 1.2.3.4
> >>>
> >>> So in this case – the CPVM can not check in to the management host 
> >>> on
> >>> 1.1.1.1 -  It’s got no interface on that subnet and it also has a
> gateway
> >>> it’s not able to reach.
> >>>
> >>> Regards,
> >>> Dag Sonstebo
> >>> Cloud Architect
> >>> ShapeBlue
> >>>
> >>> On 12/06/2018, 13:12, "Nicolas Bouige"  wrote:
> >>>
> >>> Hi Ivan,
> >>>
> >>>
> >>> Are you talking about this global parameters :
> >>>
> >>> router.aggregation.command.each.timeout
> >>>
> >>>
> >>>
> >>> Best regards,
> >>>
> >>> Nicolas Bouige
> >>> DIMSI
> >>> cloud.dimsi.fr<http://www.cloud.dimsi.fr>
> >>> 4, avenue Laurent Cely
> >>> <https://maps.google.com/?q=4,+avenue+Laurent+Cely=
> gmail=g>
> >>> Tour d’Asnière – 92600 Asnière sur Seine
> >>> T/ +33 (0)6 28 98 53 40
> >>>
> >>>
> >>> 
> >>> De : Ivan Kudryavtsev 
> >>> Envoyé : mardi 12 juin 2018 13:59:39
> >>> À : users
> >>> Objet : Re: SSVM's not starting, timeout for libvirt python 
> >>> script in agent.log
> >>>
> >>> Increasing command timeouts in global parameters can work 
> >>> here. At least I
> >>> met similar behaviour with VR.
> >>>
> >>> вт, 12 июн. 2018 г., 14:39 Christoffer Pedersen :
> >>>
> >>> > Hi Nicolas,
> >>> >
> >>> > I did a apt show qemu and it gave me this version:
> >>> >
> >>> > Version: 1:2.5+dfsg-5ubuntu10.29
> >>> >
> >>> > So I guess tha would be version 2.5?
> >>> >
> >>>
> >>> dag.sonst...@shapeblue.com
> >>> www.shapeblue.com
> >>> 53 Chandos Place, Covent Garden, London 
> >>> <https://maps.google.com/?q=53+Chandos+Place,+Covent+
> Garden,+London=gmail=g>
> >>> WC2N 4HSUK
> >>> @shapeblue
> >>>
> >>>
> >>>
> >>> > On Tue, Jun 12, 2018 at 1:04 PM, Nicolas Bouige 
> >>> > 
> >>> wrote:
> >>> >
> >>> > > Hello Christoffer,
> >>> > >
> >>> > >
> >>> > > Could you tell us wich qemu version are you using ?
> >>> > >
> >>> > > Nicolas Bouige
> >>> > > DIMSI
> >>> > > cloud.dimsi.fr<http://www.cloud.dimsi.fr>
> >>> > > 4, avenue Laurent Cely
> >>> <https://maps.google.com/?q=4,+avenue+Laurent+Cely=
> gmail=g>
> >>> > > Tour d’Asnière – 92600 Asnière sur Seine
> >>> > > T/ +33 (0)6 28 98 53 40
> >>> > >
> >>> > >
> >>> > > 
> >>> > > De : Christoffer Pedersen 
> >>> > > Envoyé : mardi 12 juin 2018 12:30:48
> >>> > > À : users@cloudstack.apache.org
> >>> > > Objet : SSVM's not starting, timeout for libvirt python 
> >>> script
> in
> >>> > agent.log
> >>> > >
> >>> > > Hi all,
> >>> > >
> >>> > > I have an issue regarding the system VMs. After deploying 
> >>> an advanced
> >>> > zone,
> >>>

DB Communication Link Failure

[Nicolas Bouige]

Hello All,

Tonight du to a network issue our management cloudstack server and MariaDb 
server have been shutdowned and restarted this morning.

Unfortunately, we  get some issues of < connectivity > between the mgmt server 
and db server.
Both service (cloudstack-management/mysqld) are up and running without any 
errors.
Db server are available from mgmt server (telnet OK, mysql -h dbserver...ok, 
firewall policies are good from both sides)

However, when we want to login to the GUI cloudstack an execption is raised in 
mgmt-server.log :

2018-06-13 15:28:41,019 DEBUG [c.c.u.d.T.Transaction] 
(qtp1796488937-13:ctx-88e441c4) (logid:f9e9b399) Rolling back the transaction: 
Time = 97 Name =  qtp1796488937-13; called by 
-TransactionLegacy.rollback:889-TransactionLegacy.removeUpTo:832-TransactionLegacy.close:656-TransactionContextInterceptor.invoke:36-ReflectiveMethodInvocation.proceed:174-ExposeInvocationInterceptor.invoke:92-ReflectiveMethodInvocation.proceed:185-JdkDynamicAopProxy.invoke:212-$Proxy121.persist:-1-ActionEventUtils.persistActionEvent:186-ActionEventUtils.onActionEvent:98-AccountManagerImpl.logoutUser:2096
2018-06-13 15:28:41,020 WARN  [c.c.u.d.T.Transaction] 
(qtp1796488937-13:ctx-88e441c4) (logid:f9e9b399) Unable to rollback
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: 
Communications link failure during rollback(). Transaction resolution unknown.
at sun.reflect.GeneratedConstructorAccessor98.newInstance(Unknown 
Source)
at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
at com.mysql.jdbc.Util.getInstance(Util.java:386)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1015)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:989)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:975)
at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:920)
at com.mysql.jdbc.ConnectionImpl.rollback(ConnectionImpl.java:5067)
at 
org.apache.commons.dbcp.DelegatingConnection.rollback(DelegatingConnection.java:368)
at 
org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.rollback(PoolingDataSource.java:323)
at 
com.cloud.utils.db.TransactionLegacy.rollbackTransaction(TransactionLegacy.java:851)
at 
com.cloud.utils.db.TransactionLegacy.rollback(TransactionLegacy.java:889)
at 
com.cloud.utils.db.TransactionLegacy.removeUpTo(TransactionLegacy.java:832)
at 
com.cloud.utils.db.TransactionLegacy.close(TransactionLegacy.java:656)
at 
com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:36)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:174)
at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy121.persist(Unknown Source)
at 
com.cloud.event.ActionEventUtils.persistActionEvent(ActionEventUtils.java:186)
at 
com.cloud.event.ActionEventUtils.onActionEvent(ActionEventUtils.java:98)
at 
com.cloud.user.AccountManagerImpl.logoutUser(AccountManagerImpl.java:2096)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy40.logoutUser(Unknown Source)
at com.cloud.api.ApiServer.logoutUser(ApiServer.java:1104)
at com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:234)
at com.cloud.api.ApiServlet$1.run(ApiServlet.java:130)
at 

RE: SSVM's not starting, timeout for libvirt python script in agent.log

[Nicolas Bouige]

Hi Ivan,


Are you talking about this global parameters :

router.aggregation.command.each.timeout



Best regards,

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40



De : Ivan Kudryavtsev 
Envoyé : mardi 12 juin 2018 13:59:39
À : users
Objet : Re: SSVM's not starting, timeout for libvirt python script in agent.log

Increasing command timeouts in global parameters can work here. At least I
met similar behaviour with VR.

вт, 12 июн. 2018 г., 14:39 Christoffer Pedersen :

> Hi Nicolas,
>
> I did a apt show qemu and it gave me this version:
>
> Version: 1:2.5+dfsg-5ubuntu10.29
>
> So I guess tha would be version 2.5?
>
> On Tue, Jun 12, 2018 at 1:04 PM, Nicolas Bouige  wrote:
>
> > Hello Christoffer,
> >
> >
> > Could you tell us wich qemu version are you using ?
> >
> > Nicolas Bouige
> > DIMSI
> > cloud.dimsi.fr<http://www.cloud.dimsi.fr>
> > 4, avenue Laurent Cely
> > Tour d’Asnière – 92600 Asnière sur Seine
> > T/ +33 (0)6 28 98 53 40
> >
> >
> > 
> > De : Christoffer Pedersen 
> > Envoyé : mardi 12 juin 2018 12:30:48
> > À : users@cloudstack.apache.org
> > Objet : SSVM's not starting, timeout for libvirt python script in
> agent.log
> >
> > Hi all,
> >
> > I have an issue regarding the system VMs. After deploying an advanced
> zone,
> > the system VMs are trying to be created but gets stuck in a "Starting"
> > state, however the Agent state is "Up". I have these logs in the
> agent.log
> > (sorry for the formatting)
> >
> > 2018-06-12 12:22:06,354 WARN  [kvm.resource.LibvirtComputingResource]
> > (Script-8:null) (logid:) Interrupting script.
> > 2018-06-12 12:22:06,355 WARN  [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-4:null) (logid:ea9cb55a) Timed out:
> > /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patchviasocket.py
> > -n
> > v-1-VM -p
> > %template=domP%type=consoleproxy%host=1.1.1.1%
> > port=8250%name=v-1-VM%zone=1%pod=1%guid=Proxy.1%proxy_vm=1%
> > disable_rp_filter=true%eth2ip=9.9.9.9%eth2mask=255.255.255.
> > 0%gateway=9.9.9.1%eth0ip=169.254.3.159%eth0mask=255.255.0.
> > 0%eth1ip=6.6.6.6%eth1mask=255.255.255.0%mgmtcidr=
> >
> 9.9.9.0/24%localgw=1.2.3.4%internaldns1=1.2.3.4%dns1=8.8.8.8%dns2=8.8.4.4
> > .  Output is:
> > 2018-06-12 12:22:06,355 ERROR [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-4:null) (logid:ea9cb55a) passcmd failed:timeout
> > 2018-06-12 12:22:08,914 WARN  [kvm.resource.LibvirtComputingResource]
> > (Script-4:null) (logid:) Interrupting script.
> > 2018-06-12 12:22:08,915 WARN  [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-5:null) (logid:8e44093e) Timed out:
> > /usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patchviasocket.py
> > -n
> > s-2-VM -p
> > %template=domP%type=secstorage%host=1.1.1.1%port=
> > 8250%name=s-2-VM%zone=1%pod=1%guid=s-2-VM%workers=5%resource=org.apache.
> > cloudstack.storage.resource.NfsSecondaryStorageResource%
> > instance=SecStorage%sslcopy=false%role=templateProcessor%
> > mtu=1500%eth2ip=7.7.7.7%eth2mask=255.255.255.0%gateway=9.9.9.1%public.
> > network.device=eth2%eth0ip=169.254.2.193%eth0mask=255.
> > 255.0.0%eth1ip=10.120.0.61%eth1mask=255.255.255.0%mgmtcidr=
> > 9.9.9.0/24%localgw=1.2.3.4%private.network.device=eth1%
> > internaldns1=1.2.3.4%dns1=8.8.8.8%dns2=8.8.4.4%nfsVersion=null
> > .  Output is:
> > 2018-06-12 12:22:08,915 ERROR [kvm.resource.LibvirtComputingResource]
> > (agentRequest-Handler-5:null) (logid:8e44093e) passcmd failed:timeout
> >
> > I have seen this error around but did not really find a solution to it. I
> > am not exactly sure whats "timing" out? I can ping both SSVM's on their
> > private and public interface.
> >
> > I hope someone can help me out here. :)
> >
> > --
> > Thanks,
> > Chris pedersen
> >
>
>
>
> --
> Thanks,
> Chris pedersen
>

RE: SSVM's not starting, timeout for libvirt python script in agent.log

[Nicolas Bouige]

Hello Christoffer,


Could you tell us wich qemu version are you using ?

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40



De : Christoffer Pedersen 
Envoyé : mardi 12 juin 2018 12:30:48
À : users@cloudstack.apache.org
Objet : SSVM's not starting, timeout for libvirt python script in agent.log

Hi all,

I have an issue regarding the system VMs. After deploying an advanced zone,
the system VMs are trying to be created but gets stuck in a "Starting"
state, however the Agent state is "Up". I have these logs in the agent.log
(sorry for the formatting)

2018-06-12 12:22:06,354 WARN  [kvm.resource.LibvirtComputingResource]
(Script-8:null) (logid:) Interrupting script.
2018-06-12 12:22:06,355 WARN  [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-4:null) (logid:ea9cb55a) Timed out:
/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patchviasocket.py -n
v-1-VM -p
%template=domP%type=consoleproxy%host=1.1.1.1%port=8250%name=v-1-VM%zone=1%pod=1%guid=Proxy.1%proxy_vm=1%disable_rp_filter=true%eth2ip=9.9.9.9%eth2mask=255.255.255.0%gateway=9.9.9.1%eth0ip=169.254.3.159%eth0mask=255.255.0.0%eth1ip=6.6.6.6%eth1mask=255.255.255.0%mgmtcidr=
9.9.9.0/24%localgw=1.2.3.4%internaldns1=1.2.3.4%dns1=8.8.8.8%dns2=8.8.4.4
.  Output is:
2018-06-12 12:22:06,355 ERROR [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-4:null) (logid:ea9cb55a) passcmd failed:timeout
2018-06-12 12:22:08,914 WARN  [kvm.resource.LibvirtComputingResource]
(Script-4:null) (logid:) Interrupting script.
2018-06-12 12:22:08,915 WARN  [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-5:null) (logid:8e44093e) Timed out:
/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patchviasocket.py -n
s-2-VM -p
%template=domP%type=secstorage%host=1.1.1.1%port=8250%name=s-2-VM%zone=1%pod=1%guid=s-2-VM%workers=5%resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource%instance=SecStorage%sslcopy=false%role=templateProcessor%mtu=1500%eth2ip=7.7.7.7%eth2mask=255.255.255.0%gateway=9.9.9.1%public.network.device=eth2%eth0ip=169.254.2.193%eth0mask=255.255.0.0%eth1ip=10.120.0.61%eth1mask=255.255.255.0%mgmtcidr=
9.9.9.0/24%localgw=1.2.3.4%private.network.device=eth1%internaldns1=1.2.3.4%dns1=8.8.8.8%dns2=8.8.4.4%nfsVersion=null
.  Output is:
2018-06-12 12:22:08,915 ERROR [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-5:null) (logid:8e44093e) passcmd failed:timeout

I have seen this error around but did not really find a solution to it. I
am not exactly sure whats "timing" out? I can ping both SSVM's on their
private and public interface.

I hope someone can help me out here. :)

--
Thanks,
Chris pedersen

RE: certificate issue second mgmt-server

[Nicolas Bouige]

Please, forget my previous mail...
The problem was Tomcat7...


I'm going on cert issue...

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40



De : Nicolas Bouige <n.bou...@dimsi.fr>
Envoyé : vendredi 4 mai 2018 15:33:43
À : users@cloudstack.apache.org
Objet : RE: certificate issue second mgmt-server

Hi All,


So, i was working on my primary issue with the certificate on my second 
node...i've stopped and restart the first node and now the 
cloudstack-management is in status failed :'(


I got an "invalid specified classpath" and "Cannot parse command line 
arguments".

I modified the file cloudstack-management.service" to manually set up these 
varirable $JAR:$CLASSPATH.


After a deamon-reload and a restart of the service, it's  run around 30 sec and 
fail again, here the log from the status service:


mai 04 20:43:24 ASPRCSMGMT01 systemd[1]: Starting CloudStack Management 
Server...
mai 04 20:43:24 ASPRCSMGMT01 systemd[1]: Started CloudStack Management Server.
mai 04 20:44:00 ASPRCSMGMT01 sudo[5033]:cloud : TTY=unknown ; 
PWD=/var/log/cloudstack/management ; USER=root ; COMMAND=/bin/mkdir -p 
/systemvm_mnt
mai 04 20:44:00 ASPRCSMGMT01 sudo[5036]:cloud : TTY=unknown ; 
PWD=/var/log/cloudstack/management ; USER=root ; COMMAND=/bin/mount -o loop 
/usr/share/cloudstack-common/vms/systemvm.iso /systemvm_mnt
mai 04 20:44:00 ASPRCSMGMT01 sudo[5041]:cloud : TTY=unknown ; 
PWD=/var/log/cloudstack/management ; USER=root ; COMMAND=/bin/umount 
/systemvm_mnt
mai 04 20:44:15 ASPRCSMGMT01 systemd[1]: cloudstack-management.service: main 
process exited, code=exited, status=1/FAILURE
mai 04 20:44:15 ASPRCSMGMT01 systemd[1]: cloudstack-management.service: control 
process exited, code=exited status=255
mai 04 20:44:15 ASPRCSMGMT01 systemd[1]: Unit cloudstack-management.service 
entered failed state.
mai 04 20:44:15 ASPRCSMGMT01 systemd[1]: cloudstack-management.service failed.

>From /var/log/messages :

May  4 20:43:28 ASPRCSMGMT01 systemd: Configuration file 
/usr/lib/systemd/system/cloudstack-management.service is marked executable. 
Please remove executable permission bits. Proceeding anyway.
May  4 20:44:15 ASPRCSMGMT01 systemd: cloudstack-management.service: main 
process exited, code=exited, status=1/FAILURE
May  4 20:44:15 ASPRCSMGMT01 systemd: cloudstack-management.service: control 
process exited, code=exited status=255
May  4 20:44:15 ASPRCSMGMT01 systemd: Unit cloudstack-management.service 
entered failed state.



This management was working fine for one month and i restarted it without any 
problem before...


If one of you have any idea, it would be appreciated,

Thanks upfront for any help,

Best regards,

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40


____
De : Nicolas Bouige <n.bou...@dimsi.fr>
Envoyé : mercredi 2 mai 2018 10:54:36
À : users@cloudstack.apache.org
Objet : RE: certificate issue second mgmt-server

Rohit,

Thanks for the details, i'll keep you update if its work.

Best regards,
N.B

-Message d'origine-
De : Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
Envoyé : mercredi 2 mai 2018 10:39
À : users@cloudstack.apache.org
Objet : Re: certificate issue second mgmt-server

Nicolas,

Yes, if you've existing systemvms and KVM hosts changing the ca private/public 
key could cause system-wide cert issue. You can retry shutting down both 
management server(s), start the primary mgmt server to come up first and then 
start/deploy other mgmt servers one by one.


- Rohit

<https://cloudstack.apache.org>




From: Nicolas Bouige <n.bou...@dimsi.fr>
Sent: Wednesday, May 2, 2018 1:57:07 PM
To: users@cloudstack.apache.org
Subject: RE: certificate issue second mgmt-server

Hi Rohit,

Thanks for your answer, i can't remember if i added the second node before the 
end of the initialization maybe i was too impatient :/ I'll give a try this 
week with your workaround.
Your workaround will affect also KVM server and System-VM ?

So, i guess, it's not enought to delete the second node and redeploy it ?

Best regards,
N.B

rohit.ya...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue




-Message d'origine-
De : Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
Envoyé : mardi 1 mai 2018 12:56
À : users@cloudstack.apache.org
Objet : Re: certificate issue second mgmt-server

Hi Nicolas,


Did you deploy multiple managements at the same time? When you deploy multiple 
management server(s), wait for the first management server to initialize 
database where it sets up some default offerings, global settings and the root 
CA keypair and certificate. O

RE: certificate issue second mgmt-server

[Nicolas Bouige]

Hi All,


So, i was working on my primary issue with the certificate on my second 
node...i've stopped and restart the first node and now the 
cloudstack-management is in status failed :'(


I got an "invalid specified classpath" and "Cannot parse command line 
arguments".

I modified the file cloudstack-management.service" to manually set up these 
varirable $JAR:$CLASSPATH.


After a deamon-reload and a restart of the service, it's  run around 30 sec and 
fail again, here the log from the status service:


mai 04 20:43:24 ASPRCSMGMT01 systemd[1]: Starting CloudStack Management 
Server...
mai 04 20:43:24 ASPRCSMGMT01 systemd[1]: Started CloudStack Management Server.
mai 04 20:44:00 ASPRCSMGMT01 sudo[5033]:cloud : TTY=unknown ; 
PWD=/var/log/cloudstack/management ; USER=root ; COMMAND=/bin/mkdir -p 
/systemvm_mnt
mai 04 20:44:00 ASPRCSMGMT01 sudo[5036]:cloud : TTY=unknown ; 
PWD=/var/log/cloudstack/management ; USER=root ; COMMAND=/bin/mount -o loop 
/usr/share/cloudstack-common/vms/systemvm.iso /systemvm_mnt
mai 04 20:44:00 ASPRCSMGMT01 sudo[5041]:cloud : TTY=unknown ; 
PWD=/var/log/cloudstack/management ; USER=root ; COMMAND=/bin/umount 
/systemvm_mnt
mai 04 20:44:15 ASPRCSMGMT01 systemd[1]: cloudstack-management.service: main 
process exited, code=exited, status=1/FAILURE
mai 04 20:44:15 ASPRCSMGMT01 systemd[1]: cloudstack-management.service: control 
process exited, code=exited status=255
mai 04 20:44:15 ASPRCSMGMT01 systemd[1]: Unit cloudstack-management.service 
entered failed state.
mai 04 20:44:15 ASPRCSMGMT01 systemd[1]: cloudstack-management.service failed.

>From /var/log/messages :

May  4 20:43:28 ASPRCSMGMT01 systemd: Configuration file 
/usr/lib/systemd/system/cloudstack-management.service is marked executable. 
Please remove executable permission bits. Proceeding anyway.
May  4 20:44:15 ASPRCSMGMT01 systemd: cloudstack-management.service: main 
process exited, code=exited, status=1/FAILURE
May  4 20:44:15 ASPRCSMGMT01 systemd: cloudstack-management.service: control 
process exited, code=exited status=255
May  4 20:44:15 ASPRCSMGMT01 systemd: Unit cloudstack-management.service 
entered failed state.



This management was working fine for one month and i restarted it without any 
problem before...


If one of you have any idea, it would be appreciated,

Thanks upfront for any help,

Best regards,

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40


________
De : Nicolas Bouige <n.bou...@dimsi.fr>
Envoyé : mercredi 2 mai 2018 10:54:36
À : users@cloudstack.apache.org
Objet : RE: certificate issue second mgmt-server

Rohit,

Thanks for the details, i'll keep you update if its work.

Best regards,
N.B

-Message d'origine-
De : Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
Envoyé : mercredi 2 mai 2018 10:39
À : users@cloudstack.apache.org
Objet : Re: certificate issue second mgmt-server

Nicolas,

Yes, if you've existing systemvms and KVM hosts changing the ca private/public 
key could cause system-wide cert issue. You can retry shutting down both 
management server(s), start the primary mgmt server to come up first and then 
start/deploy other mgmt servers one by one.


- Rohit

<https://cloudstack.apache.org>



____
From: Nicolas Bouige <n.bou...@dimsi.fr>
Sent: Wednesday, May 2, 2018 1:57:07 PM
To: users@cloudstack.apache.org
Subject: RE: certificate issue second mgmt-server

Hi Rohit,

Thanks for your answer, i can't remember if i added the second node before the 
end of the initialization maybe i was too impatient :/ I'll give a try this 
week with your workaround.
Your workaround will affect also KVM server and System-VM ?

So, i guess, it's not enought to delete the second node and redeploy it ?

Best regards,
N.B

rohit.ya...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue




-Message d'origine-
De : Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
Envoyé : mardi 1 mai 2018 12:56
À : users@cloudstack.apache.org
Objet : Re: certificate issue second mgmt-server

Hi Nicolas,


Did you deploy multiple managements at the same time? When you deploy multiple 
management server(s), wait for the first management server to initialize 
database where it sets up some default offerings, global settings and the root 
CA keypair and certificate. Only when you see the first management server's UI 
in browser, proceed with deployment of other management server(s).


For your environment, you can test this workaround and let me know if that 
works for you:


  1.  Shutdown all the management server(s).
  2.  Delete ca keypair and cert:
 delete from configuration where name like "ca.plugin.root.private.key";  
delete from configuration where name like "ca.plugin.root.public.key"

RE: certificate issue second mgmt-server

[Nicolas Bouige]

Rohit,

Thanks for the details, i'll keep you update if its work.

Best regards,
N.B

-Message d'origine-
De : Rohit Yadav [mailto:rohit.ya...@shapeblue.com] 
Envoyé : mercredi 2 mai 2018 10:39
À : users@cloudstack.apache.org
Objet : Re: certificate issue second mgmt-server

Nicolas,

Yes, if you've existing systemvms and KVM hosts changing the ca private/public 
key could cause system-wide cert issue. You can retry shutting down both 
management server(s), start the primary mgmt server to come up first and then 
start/deploy other mgmt servers one by one.


- Rohit

<https://cloudstack.apache.org>




From: Nicolas Bouige <n.bou...@dimsi.fr>
Sent: Wednesday, May 2, 2018 1:57:07 PM
To: users@cloudstack.apache.org
Subject: RE: certificate issue second mgmt-server

Hi Rohit,

Thanks for your answer, i can't remember if i added the second node before the 
end of the initialization maybe i was too impatient :/ I'll give a try this 
week with your workaround.
Your workaround will affect also KVM server and System-VM ?

So, i guess, it's not enought to delete the second node and redeploy it ?

Best regards,
N.B

rohit.ya...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
  
 


-Message d'origine-
De : Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
Envoyé : mardi 1 mai 2018 12:56
À : users@cloudstack.apache.org
Objet : Re: certificate issue second mgmt-server

Hi Nicolas,


Did you deploy multiple managements at the same time? When you deploy multiple 
management server(s), wait for the first management server to initialize 
database where it sets up some default offerings, global settings and the root 
CA keypair and certificate. Only when you see the first management server's UI 
in browser, proceed with deployment of other management server(s).


For your environment, you can test this workaround and let me know if that 
works for you:


  1.  Shutdown all the management server(s).
  2.  Delete ca keypair and cert:
 delete from configuration where name like "ca.plugin.root.private.key";  
delete from configuration where name like "ca.plugin.root.public.key";  delete 
from configuration where name="ca.plugin.root.ca.certificate";
  3.  Start one management server and wait for it to complete internal setup, 
until you see the UI.
  4.  Start all the other management server(s).



- Rohit

<https://cloudstack.apache.org>



____
From: Nicolas Bouige <n.bou...@dimsi.fr>
Sent: Monday, April 30, 2018 2:59:29 PM
To: users@cloudstack.apache.org
Subject: certificate issue second mgmt-server

Hello All,


I have an issue with one of my Cloudstack mgmt-server (4.11)

The second node has been deployed with the command  "cloudstack-setup-databases 
cloud:dbpassword@dbhost"


i didnt have any problem during few days and now sometimes i got an error on 
web GUI when i perfom some basic task, the error is "Resource [Host:1] is 
unreachable: Host 1: Unable to reach the peer that the agent is connected"


After a quick investigation, i had to stop cloudstack-management service from 
second mgmt-server and i noticed a lot of messages related with ca-certificate 
used by cloudstack :


2018-04-27 11:18:24,076 ERROR [c.c.u.n.Link] (StatsCollector-1:ctx-82335701) 
(logid:95fda6d7) SSL error caught during unwrap data: Received fatal alert: 
certificate_unknown, for local address=/172.16.22.61:60128, remote 
address=/172.16.22.60:8250. The client may have invalid ca-certificates.
2018-04-27 11:18:24,076 WARN  [c.c.a.m.ClusteredAgentManagerImpl] 
(StatsCollector-1:ctx-82335701) (logid:95fda6d7) Unable to connect to peer 
management server: 130719784044197, ip: 172.16.22.60 due to SSL: Fail to init 
SSL! java.io.IOException: SSL: Handshake failed with peer management server 
'130719784044197' on 172.16.22.60:8250
java.io.IOException: SSL: Fail to init SSL! java.io.IOException: SSL: Handshake 
failed with peer management server '130719784044197' on 172.16.22.60:8250
at 
com.cloud.agent.manager.ClusteredAgentManagerImpl.connectToPeer(ClusteredAgentManagerImpl.java:529)
at 
com.cloud.agent.manager.ClusteredAgentAttache.send(ClusteredAgentAttache.java:177)
at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:398)
at 
com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:456)
at 
com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:362)
at 
com.cloud.agent.manager.AgentManagerImpl.easySend(AgentManagerImpl.java:954)
at 
com.cloud.resource.ResourceManagerImpl.getHostStatistics(ResourceManagerImpl.java:2645)
at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
a

RE: certificate issue second mgmt-server

[Nicolas Bouige]

Hi Rohit,

Thanks for your answer, i can't remember if i added the second node before the 
end of the initialization maybe i was too impatient :/
I'll give a try this week with your workaround.
Your workaround will affect also KVM server and System-VM ?

So, i guess, it's not enought to delete the second node and redeploy it ?

Best regards,
N.B

-Message d'origine-
De : Rohit Yadav [mailto:rohit.ya...@shapeblue.com] 
Envoyé : mardi 1 mai 2018 12:56
À : users@cloudstack.apache.org
Objet : Re: certificate issue second mgmt-server

Hi Nicolas,


Did you deploy multiple managements at the same time? When you deploy multiple 
management server(s), wait for the first management server to initialize 
database where it sets up some default offerings, global settings and the root 
CA keypair and certificate. Only when you see the first management server's UI 
in browser, proceed with deployment of other management server(s).


For your environment, you can test this workaround and let me know if that 
works for you:


  1.  Shutdown all the management server(s).
  2.  Delete ca keypair and cert:
 delete from configuration where name like "ca.plugin.root.private.key";  
delete from configuration where name like "ca.plugin.root.public.key";  delete 
from configuration where name="ca.plugin.root.ca.certificate";
  3.  Start one management server and wait for it to complete internal setup, 
until you see the UI.
  4.  Start all the other management server(s).



- Rohit

<https://cloudstack.apache.org>



____________
From: Nicolas Bouige <n.bou...@dimsi.fr>
Sent: Monday, April 30, 2018 2:59:29 PM
To: users@cloudstack.apache.org
Subject: certificate issue second mgmt-server

Hello All,


I have an issue with one of my Cloudstack mgmt-server (4.11)

The second node has been deployed with the command  "cloudstack-setup-databases 
cloud:dbpassword@dbhost"


i didnt have any problem during few days and now sometimes i got an error on 
web GUI when i perfom some basic task, the error is "Resource [Host:1] is 
unreachable: Host 1: Unable to reach the peer that the agent is connected"


After a quick investigation, i had to stop cloudstack-management service from 
second mgmt-server and i noticed a lot of messages related with ca-certificate 
used by cloudstack :


2018-04-27 11:18:24,076 ERROR [c.c.u.n.Link] (StatsCollector-1:ctx-82335701) 
(logid:95fda6d7) SSL error caught during unwrap data: Received fatal alert: 
certificate_unknown, for local address=/172.16.22.61:60128, remote 
address=/172.16.22.60:8250. The client may have invalid ca-certificates.
2018-04-27 11:18:24,076 WARN  [c.c.a.m.ClusteredAgentManagerImpl] 
(StatsCollector-1:ctx-82335701) (logid:95fda6d7) Unable to connect to peer 
management server: 130719784044197, ip: 172.16.22.60 due to SSL: Fail to init 
SSL! java.io.IOException: SSL: Handshake failed with peer management server 
'130719784044197' on 172.16.22.60:8250
java.io.IOException: SSL: Fail to init SSL! java.io.IOException: SSL: Handshake 
failed with peer management server '130719784044197' on 172.16.22.60:8250
at 
com.cloud.agent.manager.ClusteredAgentManagerImpl.connectToPeer(ClusteredAgentManagerImpl.java:529)
at 
com.cloud.agent.manager.ClusteredAgentAttache.send(ClusteredAgentAttache.java:177)
at com.cloud.agent.manager.AgentAttache.send(AgentAttache.java:398)
at 
com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:456)
at 
com.cloud.agent.manager.AgentManagerImpl.send(AgentManagerImpl.java:362)
at 
com.cloud.agent.manager.AgentManagerImpl.easySend(AgentManagerImpl.java:954)
at 
com.cloud.resource.ResourceManagerImpl.getHostStatistics(ResourceManagerImpl.java:2645)
at sun.reflect.GeneratedMethodAccessor96.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:338)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:197)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:185)
at 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212)
at com.sun.proxy.$Proxy178.getHostStatistics(Unknown Source)
at 
com.cloud.server.StatsCollector$HostCollector.runInContext(StatsCollector.java:438)
at 
org.apache.cloudstack.managed.conte

RE: New to Cloudstack - Issues going in to production - Public and Guest bridges not up KVM

[Nicolas Bouige]

Adam,


Yes ;)

Sorry for my misunderstanding, i saw that when i check on my own KVM 
configuration.


Do you have any log from KVM agent cloudstack or MGMT ?

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40




De : Adam Witwicki <awitwi...@oakfordis.com>
Envoyé : lundi 30 avril 2018 14:36
À : users@cloudstack.apache.org
Objet : New to Cloudstack - Issues going in to production - Public and Guest 
bridges not up KVM

Hello

I have managed to test Cloudstack in the Lab with very little issues, but now 
going into production (adding Loadbalanced Cloudstack management servers and 
LACP nic bonding) I am having a few issues.

When starting my first instance the network bridges stay down

13: brbond0-999: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000

If I bring this up using ip link set dev brbond0-999 up the bridge comes up 
fine and the systemVMs work.

Any ideas?

Kind Regards

Adam Witwicki


Centos 7
Cloudstack 4.9.3.1
KVM 3.2.0

/etc/sysconfig/network-scripts/ifcfg-enp4s0f0
TYPE="Ethernet"
USERCTL=no
MASTER=bond0
SLAVE=yes
NAME="enp4s0f0"
UUID="abe1d2f9-253c-4246-a7d6-b7e7e396ce8f"
DEVICE="enp4s0f0"
ONBOOT="yes"
NM_CONTROLLED=no
BOOTPROTO=none


/etc/sysconfig/network-scripts/ifcfg-enp4s0f0
TYPE=Ethernet
BOOTPROTO=none
USERCTL=no
MASTER=bond0
SLAVE=yes
NAME=enp4s0f1
UUID=f6cec93b-e271-44d4-818b-c94393133d23
DEVICE=enp4s0f1
ONBOOT=yes
NM_CONTROLLED=no


/etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
ONBOOT=yes
BONDING_OPTS='mode=4 miimon=100'
BRIDGE=cloudbr0
NM_CONTROLLED=no


/etc/sysconfig/network-scripts/ifcfg-cloudbr0
DEVICE=cloudbr0
TYPE=Bridge
ONBOOT=yes
BOOTPROTO=static
IPV6INIT=no
IPV6_AUTOCONF=no
DELAY=5
STP=yes
IPADDR= x.x.x.x
GATEWAY= x.x.x.x
PREFIX=xx
DNS1= x.x.x.x
DNS2= x.x.x.x

bridge name bridge id   STP enabled interfaces
brbond0-999 8000.a0369f20953c   no  bond0.999
vnet4
vnet5
cloud0  8000.fe00a9fe0006   no  vnet0
vnet1
cloudbr08000.a0369f20953c   yes bond0
vnet2
vnet3
virbr0  8000.525400a48eba   yes virbr0-nic






Disclaimer Notice:
This email has been sent by Oakford Technology Limited, while we have checked 
this e-mail and any attachments for viruses, we can not guarantee that they are 
virus-free. You must therefore take full responsibility for virus checking.
This message and any attachments are confidential and should only be read by 
those to whom they are addressed. If you are not the intended recipient, please 
contact us, delete the message from your computer and destroy any copies. Any 
distribution or copying without our prior permission is prohibited.
Internet communications are not always secure and therefore Oakford Technology 
Limited does not accept legal responsibility for this message. The recipient is 
responsible for verifying its authenticity before acting on the contents. Any 
views or opinions presented are solely those of the author and do not 
necessarily represent those of Oakford Technology Limited.
Registered address: Oakford Technology Limited, 10 Prince Maurice Court, 
Devizes, Wiltshire. SN10 2RT.
Registered in England and Wales No. 5971519

RE: New to Cloudstack - Issues going in to production - Public and Guest bridges not up KVM

[Nicolas Bouige]

Hi Adam,


Your network bridge with KVM to be used with cloudstack need to be named like 
this  :  cloudbr0, cloudbr1


So you have to name your  bridge brbond0-999 -> clourbr1


Do you have only one bond ? (bond0)

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40



De : Adam Witwicki <awitwi...@oakfordis.com>
Envoyé : lundi 30 avril 2018 14:36:44
À : users@cloudstack.apache.org
Objet : New to Cloudstack - Issues going in to production - Public and Guest 
bridges not up KVM

Hello

I have managed to test Cloudstack in the Lab with very little issues, but now 
going into production (adding Loadbalanced Cloudstack management servers and 
LACP nic bonding) I am having a few issues.

When starting my first instance the network bridges stay down

13: brbond0-999: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN qlen 1000

If I bring this up using ip link set dev brbond0-999 up the bridge comes up 
fine and the systemVMs work.

Any ideas?

Kind Regards

Adam Witwicki


Centos 7
Cloudstack 4.9.3.1
KVM 3.2.0

/etc/sysconfig/network-scripts/ifcfg-enp4s0f0
TYPE="Ethernet"
USERCTL=no
MASTER=bond0
SLAVE=yes
NAME="enp4s0f0"
UUID="abe1d2f9-253c-4246-a7d6-b7e7e396ce8f"
DEVICE="enp4s0f0"
ONBOOT="yes"
NM_CONTROLLED=no
BOOTPROTO=none


/etc/sysconfig/network-scripts/ifcfg-enp4s0f0
TYPE=Ethernet
BOOTPROTO=none
USERCTL=no
MASTER=bond0
SLAVE=yes
NAME=enp4s0f1
UUID=f6cec93b-e271-44d4-818b-c94393133d23
DEVICE=enp4s0f1
ONBOOT=yes
NM_CONTROLLED=no


/etc/sysconfig/network-scripts/ifcfg-bond0
DEVICE=bond0
ONBOOT=yes
BONDING_OPTS='mode=4 miimon=100'
BRIDGE=cloudbr0
NM_CONTROLLED=no


/etc/sysconfig/network-scripts/ifcfg-cloudbr0
DEVICE=cloudbr0
TYPE=Bridge
ONBOOT=yes
BOOTPROTO=static
IPV6INIT=no
IPV6_AUTOCONF=no
DELAY=5
STP=yes
IPADDR= x.x.x.x
GATEWAY= x.x.x.x
PREFIX=xx
DNS1= x.x.x.x
DNS2= x.x.x.x

bridge name bridge id   STP enabled interfaces
brbond0-999 8000.a0369f20953c   no  bond0.999
vnet4
vnet5
cloud0  8000.fe00a9fe0006   no  vnet0
vnet1
cloudbr08000.a0369f20953c   yes bond0
vnet2
vnet3
virbr0  8000.525400a48eba   yes virbr0-nic






Disclaimer Notice:
This email has been sent by Oakford Technology Limited, while we have checked 
this e-mail and any attachments for viruses, we can not guarantee that they are 
virus-free. You must therefore take full responsibility for virus checking.
This message and any attachments are confidential and should only be read by 
those to whom they are addressed. If you are not the intended recipient, please 
contact us, delete the message from your computer and destroy any copies. Any 
distribution or copying without our prior permission is prohibited.
Internet communications are not always secure and therefore Oakford Technology 
Limited does not accept legal responsibility for this message. The recipient is 
responsible for verifying its authenticity before acting on the contents. Any 
views or opinions presented are solely those of the author and do not 
necessarily represent those of Oakford Technology Limited.
Registered address: Oakford Technology Limited, 10 Prince Maurice Court, 
Devizes, Wiltshire. SN10 2RT.
Registered in England and Wales No. 5971519

certificate issue second mgmt-server

[Nicolas Bouige]

ip: 172.16.22.60 due to SSL: Fail to init 
SSL! java.io.IOException: SSL: Handshake failed with peer management server 
'130719784044197' on 172.16.22.60:8250

Im not familiar with the using of self-signed certificate in cloudstack, do you 
know where i can find out more information to investigate deeper ? or if you 
have any idea ?
I tried to check keystore on both mgmt-server but i need a password i havnt...


Thanks upfront,
Have a nice day,

Best regards,

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine

T/ +33 (0)6 28 98 53 40

RE: KVM in Xenserver advanced networking environement

[Nicolas Bouige]

Hi Benjamin,

We were in the same stuation few month ago ;)

You have to create team/bond and attach cloud bridge to them.
You have to name cloud bridge like : cloubr0, cloudbr1, cloudbrX

For example for the management, (im using team instead of bond) :
Create a team  named teamMGMT 
Create  a bridge named cloudbr0
Link teamMGMT with cloudbr0  (command "brctl show" is useful)
Create a vlan with ID 133 
Create a bridge named cloudbr0.133
Link vlan id 133 to cloudbr0.133

For guest network, juste create a trunk  and link it to a cloud bridge named 
"cloudbr1". You will need to put the vlan range during the zone configuration.

i advise to create the network config manually and avoid to use Network Manager.
Tell me if you need more detail or config files.

Best regards,
N.B



-Message d'origine-
De : Benjamin Naber [mailto:benjamin.na...@coders-area.de] 
Envoyé : vendredi 27 avril 2018 10:18
À : Cloudstack Mailinglist 
Objet : KVM in Xenserver advanced networking environement

Hi together,

currently we have a cloudstack 4.10 environment with Xenserver and the 
following network configuration:

Physical (Bond) Interfaces:

1 management -> untagged vlan for management / tagged vlan 133 for secondary 
storage.
2 guest -> tagged vlans 2000-2699
3 public -> tagged vlans 200-250
4 ISCSI -> untagged

I am now on eval for using KVM in this environment.
I am also new to KVM.
My question is how do i have to setup the cloud bridges for using this network 
layout on KVM. And How do i have to name the cloudbridges in KVM ?

Kind regards Ben

RE: ssvm NFS public ip

[Nicolas Bouige]

Hi Swatisk,


Yep, here the repo/url :


KVM  (centos7):

[cloudstack]
name=cloudstack
baseurl=http://cloudstack.apt-get.eu/centos/7/4.11/
enabled=1
gpgcheck=0

MGMT-SRV (centos7) :

[cloudstack]
name=cloudstack
baseurl=http://cloudstack.apt-get.eu/centos/7/4.11/
enabled=1
gpgcheck=0

Systemvm-template :

http://cloudstack.apt-get.eu/systemvm/4.11/systemvmtemplate-4.11.0-kvm.qcow2.bz2


Best regards,
N.B



De : Swastik Mittal <mittal.swas...@gmail.com>
Envoyé : vendredi 13 avril 2018 10:19:03
À : users@cloudstack.apache.org
Objet : Re: ssvm NFS public ip

Hey Nicolas,

Can you mention the urls's you used finally to get everything working.

Regards
Swastik
<https://mailtrack.io/> Sent with Mailtrack
<https://chrome.google.com/webstore/detail/mailtrack-for-gmail-inbox/ndnaehgpjlnokgebbaldlmgkapkpjkkb?utm_source=gmail_medium=signature_campaign=signaturevirality>

On Thu, Apr 12, 2018 at 3:10 PM, Nicolas Bouige <n.bou...@dimsi.fr> wrote:

> Hello All,
>
> With this new deployement my ssvm is now okay.
> I saw my mistake...the URLs were good for mgmt and systemvm template...but
> not for the cloudstack-agent installed on KVM...
>
> Best regards,
>
> N.B
>
> -Message d'origine-
> De : Nicolas Bouige [mailto:n.bou...@dimsi.fr]
> Envoyé : jeudi 12 avril 2018 09:56
> À : users@cloudstack.apache.org
> Objet : RE: ssvm NFS public ip
>
> Hi Paul,
>
> I used repository from shapeblue for mgmt-server and systemvm-template.
> Im going to make a new fresh install (again :/) and double check the URL.
>
> Thanks,
>
> Best regards,
> N.B
>
>
> -Message d'origine-
> De : Paul Angus [mailto:paul.an...@shapeblue.com] Envoyé : jeudi 12 avril
> 2018 09:12 À : users@cloudstack.apache.org Objet : RE: ssvm NFS public ip
>
> Hi Nicolas, this usually happens if you mix redist and noredist builds of
> mgmt. server and system vm code.
>
> What URLs did you use for the management server repo and the system VM
> templates?
>
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com<http://www.shapeblue.com>
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>
>
>
>
> -Original Message-
> From: Nicolas Bouige <n.bou...@dimsi.fr>
> Sent: 11 April 2018 19:44
> To: users@cloudstack.apache.org
> Subject: RE: ssvm NFS public ip
>
> Hi ilya,
>
>
> Thanks for your answer
>
> Yes i did it already with no effect :/
>
>
> Best regards,
>
> N.B
>
> 
> De : ilya musayev <ilya.mailing.li...@gmail.com> Envoyé : mercredi 11
> avril 2018 20:26:34 À : users@cloudstack.apache.org Objet : Re: ssvm NFS
> public ip
>
> There is a global setting you have to set to use internal non public IP.
>
> Try setting sec.storage.allow.internal.site to an internal network cidr.
>
> You may need to destroy ssvm for settings to take effect. In my case there
> is some sort of minor bug where it takes upward of 5 minutes for ssvm to
> program the internal routes, but since this is onetime operation - I just
> live with it.
>
> On Wed, Apr 11, 2018 at 10:50 AM Nicolas Bouige <n.bou...@dimsi.fr> wrote:
>
> > a small update about my problem.
> >
> > I 've recreated the zone from scratch this morning and  one of my
> > "cloudbr" used for the secondary storage was misconfigured.
> >
> > So Now, I can ping the secondary storage from KVM host, CS-MGMT,  SSVM
> > and mount the nfs on them...but...the agent still not going up and the
> > ssvm_check.sh give me an ip public for the nfs instead of the private.
> >
> >
> > i got only this error in /var/log:cloud.log :
> >
> > 17:30:22,600 ERROR AgentShell:477 - Unable to start agent: Resource
> > class not found: com.cloud.storage.resource.
> PremiumSecondaryStorageResource due
> > to: java.lang.ClassNotFoundException:
> > com.cloud.storage.resource.PremiumSecondaryStorageReso
> >   urce
> > Unable to start agent: Resource class not found:
> > com.cloud.storage.resource.Prem
> > iumSecondaryStorageResource due to: java.lang.ClassNotFoundException:
> > com.cloud.
> > storage.resource.PremiumSecondaryStorageResource
> >
> > i've tried to update a differente systemvm template with no success...
> > I 've compared the configuration of the ssvm with one of our
> > deployment and i juste noticed the "resource"
> > (org.apache.cloudstack.storage.resource.NfsSecondaryResource) was not
> > the same.
> > i changed it but still the same error...
> >
> > There is a way to find java binaries/script and  upload them on the ssvm
> ?
> >
> > If one of you have an idea, it would be appreciate.
> >
> > Thanks !
> >
> > Best regards,
> > N.B
> >
> >
>
>

RE: ssvm NFS public ip

[Nicolas Bouige]

Hello All,

With this new deployement my ssvm is now okay.
I saw my mistake...the URLs were good for mgmt and systemvm template...but not 
for the cloudstack-agent installed on KVM...

Best regards,

N.B

-Message d'origine-
De : Nicolas Bouige [mailto:n.bou...@dimsi.fr] 
Envoyé : jeudi 12 avril 2018 09:56
À : users@cloudstack.apache.org
Objet : RE: ssvm NFS public ip

Hi Paul,

I used repository from shapeblue for mgmt-server and systemvm-template.
Im going to make a new fresh install (again :/) and double check the URL.

Thanks,

Best regards,
N.B


-Message d'origine-
De : Paul Angus [mailto:paul.an...@shapeblue.com] Envoyé : jeudi 12 avril 2018 
09:12 À : users@cloudstack.apache.org Objet : RE: ssvm NFS public ip

Hi Nicolas, this usually happens if you mix redist and noredist builds of mgmt. 
server and system vm code.

What URLs did you use for the management server repo and the system VM 
templates?



paul.an...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
  
 


-Original Message-
From: Nicolas Bouige <n.bou...@dimsi.fr>
Sent: 11 April 2018 19:44
To: users@cloudstack.apache.org
Subject: RE: ssvm NFS public ip

Hi ilya,


Thanks for your answer

Yes i did it already with no effect :/


Best regards,

N.B


De : ilya musayev <ilya.mailing.li...@gmail.com> Envoyé : mercredi 11 avril 
2018 20:26:34 À : users@cloudstack.apache.org Objet : Re: ssvm NFS public ip

There is a global setting you have to set to use internal non public IP.

Try setting sec.storage.allow.internal.site to an internal network cidr.

You may need to destroy ssvm for settings to take effect. In my case there is 
some sort of minor bug where it takes upward of 5 minutes for ssvm to program 
the internal routes, but since this is onetime operation - I just live with it.

On Wed, Apr 11, 2018 at 10:50 AM Nicolas Bouige <n.bou...@dimsi.fr> wrote:

> a small update about my problem.
>
> I 've recreated the zone from scratch this morning and  one of my 
> "cloudbr" used for the secondary storage was misconfigured.
>
> So Now, I can ping the secondary storage from KVM host, CS-MGMT,  SSVM 
> and mount the nfs on them...but...the agent still not going up and the 
> ssvm_check.sh give me an ip public for the nfs instead of the private.
>
>
> i got only this error in /var/log:cloud.log :
>
> 17:30:22,600 ERROR AgentShell:477 - Unable to start agent: Resource 
> class not found: com.cloud.storage.resource.PremiumSecondaryStorageResource 
> due
> to: java.lang.ClassNotFoundException:
> com.cloud.storage.resource.PremiumSecondaryStorageReso
>   urce
> Unable to start agent: Resource class not found:
> com.cloud.storage.resource.Prem
> iumSecondaryStorageResource due to: java.lang.ClassNotFoundException:
> com.cloud.
> storage.resource.PremiumSecondaryStorageResource
>
> i've tried to update a differente systemvm template with no success...
> I 've compared the configuration of the ssvm with one of our 
> deployment and i juste noticed the "resource"
> (org.apache.cloudstack.storage.resource.NfsSecondaryResource) was not 
> the same.
> i changed it but still the same error...
>
> There is a way to find java binaries/script and  upload them on the ssvm ?
>
> If one of you have an idea, it would be appreciate.
>
> Thanks !
>
> Best regards,
> N.B
>
>

RE: ssvm NFS public ip

[Nicolas Bouige]

Hi Paul,

I used repository from shapeblue for mgmt-server and systemvm-template.
Im going to make a new fresh install (again :/) and double check the URL.

Thanks,

Best regards,
N.B


-Message d'origine-
De : Paul Angus [mailto:paul.an...@shapeblue.com] 
Envoyé : jeudi 12 avril 2018 09:12
À : users@cloudstack.apache.org
Objet : RE: ssvm NFS public ip

Hi Nicolas, this usually happens if you mix redist and noredist builds of mgmt. 
server and system vm code.

What URLs did you use for the management server repo and the system VM 
templates?



paul.an...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
  
 


-Original Message-
From: Nicolas Bouige <n.bou...@dimsi.fr>
Sent: 11 April 2018 19:44
To: users@cloudstack.apache.org
Subject: RE: ssvm NFS public ip

Hi ilya,


Thanks for your answer

Yes i did it already with no effect :/


Best regards,

N.B


De : ilya musayev <ilya.mailing.li...@gmail.com> Envoyé : mercredi 11 avril 
2018 20:26:34 À : users@cloudstack.apache.org Objet : Re: ssvm NFS public ip

There is a global setting you have to set to use internal non public IP.

Try setting sec.storage.allow.internal.site to an internal network cidr.

You may need to destroy ssvm for settings to take effect. In my case there is 
some sort of minor bug where it takes upward of 5 minutes for ssvm to program 
the internal routes, but since this is onetime operation - I just live with it.

On Wed, Apr 11, 2018 at 10:50 AM Nicolas Bouige <n.bou...@dimsi.fr> wrote:

> a small update about my problem.
>
> I 've recreated the zone from scratch this morning and  one of my 
> "cloudbr" used for the secondary storage was misconfigured.
>
> So Now, I can ping the secondary storage from KVM host, CS-MGMT,  SSVM 
> and mount the nfs on them...but...the agent still not going up and the 
> ssvm_check.sh give me an ip public for the nfs instead of the private.
>
>
> i got only this error in /var/log:cloud.log :
>
> 17:30:22,600 ERROR AgentShell:477 - Unable to start agent: Resource 
> class not found: com.cloud.storage.resource.PremiumSecondaryStorageResource 
> due
> to: java.lang.ClassNotFoundException:
> com.cloud.storage.resource.PremiumSecondaryStorageReso
>   urce
> Unable to start agent: Resource class not found:
> com.cloud.storage.resource.Prem
> iumSecondaryStorageResource due to: java.lang.ClassNotFoundException:
> com.cloud.
> storage.resource.PremiumSecondaryStorageResource
>
> i've tried to update a differente systemvm template with no success...
> I 've compared the configuration of the ssvm with one of our 
> deployment and i juste noticed the "resource"
> (org.apache.cloudstack.storage.resource.NfsSecondaryResource) was not 
> the same.
> i changed it but still the same error...
>
> There is a way to find java binaries/script and  upload them on the ssvm ?
>
> If one of you have an idea, it would be appreciate.
>
> Thanks !
>
> Best regards,
> N.B
>
>

RE: ssvm NFS public ip

[Nicolas Bouige]

Hi ilya,


Thanks for your answer

Yes i did it already with no effect :/


Best regards,

N.B


De : ilya musayev <ilya.mailing.li...@gmail.com>
Envoyé : mercredi 11 avril 2018 20:26:34
À : users@cloudstack.apache.org
Objet : Re: ssvm NFS public ip

There is a global setting you have to set to use internal non public IP.

Try setting sec.storage.allow.internal.site to an internal network cidr.

You may need to destroy ssvm for settings to take effect. In my case there
is some sort of minor bug where it takes upward of 5 minutes for ssvm to
program the internal routes, but since this is onetime operation - I just
live with it.

On Wed, Apr 11, 2018 at 10:50 AM Nicolas Bouige <n.bou...@dimsi.fr> wrote:

> a small update about my problem.
>
> I 've recreated the zone from scratch this morning and  one of my
> "cloudbr" used for the secondary storage was misconfigured.
>
> So Now, I can ping the secondary storage from KVM host, CS-MGMT,  SSVM and
> mount the nfs on them...but...the agent still not going up and the
> ssvm_check.sh give me an ip public for the nfs instead of the private.
>
>
> i got only this error in /var/log:cloud.log :
>
> 17:30:22,600 ERROR AgentShell:477 - Unable to start agent: Resource class
> not found: com.cloud.storage.resource.PremiumSecondaryStorageResource due
> to: java.lang.ClassNotFoundException:
> com.cloud.storage.resource.PremiumSecondaryStorageReso
>   urce
> Unable to start agent: Resource class not found:
> com.cloud.storage.resource.Prem
> iumSecondaryStorageResource due to: java.lang.ClassNotFoundException:
> com.cloud.
> storage.resource.PremiumSecondaryStorageResource
>
> i've tried to update a differente systemvm template with no success...
> I 've compared the configuration of the ssvm with one of our deployment
> and i juste noticed the "resource"
> (org.apache.cloudstack.storage.resource.NfsSecondaryResource) was not the
> same.
> i changed it but still the same error...
>
> There is a way to find java binaries/script and  upload them on the ssvm ?
>
> If one of you have an idea, it would be appreciate.
>
> Thanks !
>
> Best regards,
> N.B
>
>

RE: ssvm NFS public ip

[Nicolas Bouige]

a small update about my problem.

I 've recreated the zone from scratch this morning and  one of my "cloudbr" 
used for the secondary storage was misconfigured.

So Now, I can ping the secondary storage from KVM host, CS-MGMT,  SSVM and 
mount the nfs on them...but...the agent still not going up and the 
ssvm_check.sh give me an ip public for the nfs instead of the private.


i got only this error in /var/log:cloud.log :

17:30:22,600 ERROR AgentShell:477 - Unable to start agent: Resource class not 
found: com.cloud.storage.resource.PremiumSecondaryStorageResource due to: 
java.lang.ClassNotFoundException: 
com.cloud.storage.resource.PremiumSecondaryStorageReso  
  urce
Unable to start agent: Resource class not found: 
com.cloud.storage.resource.Prem
iumSecondaryStorageResource due to: java.lang.ClassNotFoundException: 
com.cloud.
storage.resource.PremiumSecondaryStorageResource

i've tried to update a differente systemvm template with no success...
I 've compared the configuration of the ssvm with one of our deployment and i 
juste noticed the "resource" 
(org.apache.cloudstack.storage.resource.NfsSecondaryResource) was not the same.
i changed it but still the same error...

There is a way to find java binaries/script and  upload them on the ssvm ?

If one of you have an idea, it would be appreciate.

Thanks !

Best regards,
N.B

ssvm NFS public ip

[Nicolas Bouige]

Hello All,

I' m currently set up a fresh installation of cloucdstack 4.11 on CentOS7/KVM 
server.

All was working good until the systemvm installation...the ssvm is using the 
public ip to contact the nfs instead of the internal ip adresse.
I had this issue before and it was a ip route and/or arp records problem, but 
not this time.

>From ssvm_check script, i got two error

-  NFS not mounted

-  Java process not running

I don't know if the java process error is related to the NFS not mounted.

Here the result  :
First DNS server is  8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=48 time=54.150 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=48 time=45.959 ms
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 45.959/50.055/54.150/4.096 ms
Good: Can ping DNS server

Good: DNS resolves cloudstack.apache.org

ERROR: NFS is not currently mounted
Try manually mounting from inside the VM
NFS server is  77.88.44.225
PING 77.88.44.225 (77.88.44.225): 56 data bytes
64 bytes from 77.88.44.225: icmp_seq=0 ttl=64 time=1.132 ms
64 bytes from 77.88.44.225: icmp_seq=1 ttl=64 time=1.123 ms
--- 77.88.44.225 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 1.123/1.127/1.132/0.000 ms
Good: Can ping nfs server

Management server is 172.16.22.60. Checking connectivity.
./ssvm-check.sh: line 121: warning: command substitution: ignored null byte in 
input
Good: Can connect to management server port 8250

ERROR: Java process not running.  Try restarting the SSVM.

>From the /var/log/cloud.log, i've noticed this error :
2018-04-10 10:24:22,062 INFO  [cloud.agent.AgentShell] (main:null) Agent started
2018-04-10 10:24:22,066 INFO  [cloud.agent.AgentShell] (main:null) 
Implementation Version is 4.11.0.0
2018-04-10 10:24:22,067 INFO  [cloud.agent.AgentShell] (main:null) 
agent.properties found at /usr/local/cloud/systemvm/conf/agent.properties
2018-04-10 10:24:22,077 INFO  [cloud.agent.AgentShell] (main:null) Defaulting 
to using properties file for storage
2018-04-10 10:24:22,084 INFO  [cloud.agent.AgentShell] (main:null) Defaulting 
to the constant time backoff algorithm
2018-04-10 10:24:22,114 INFO  [cloud.utils.LogUtils] (main:null) log4j 
configuration found at /usr/local/cloud/systemvm/conf/log4j-cloud.xml
2018-04-10 10:24:22,168 INFO  [cloud.agent.AgentShell] (main:null) Using 
default Java settings for IPv6 preference for agent connection
2018-04-10 10:24:22,323 ERROR [cloud.agent.AgentShell] (main:null) Unable to 
start agent: Resource class not found: 
com.cloud.storage.resource.PremiumSecondaryStorageResource due to: 
java.lang.ClassNotFoundException: 
com.cloud.storage.resource.PremiumSecondaryStorageResource


>From the host, i've created cloudbr0 (mgmt) and cloudbr1 with a bridge/vlan  
>connected to the secondary storage network.
###ifcfg-cloudbr1
DEVICE=cloudbr1
NM_CONTROLLED=no
TYPE=Bridge
BOOTPROTO=none
ONBOOT=yes
DEFROUTE=yes
ZONE=public

###ifcfg-cloudbr1.103
DEVICE=cloudbr1.103
ONBOOT=yes
TYPE=Bridge
IPADDR=172.16.3.217
NETMASK=255.255.255.0
GATEWAY=172.16.3.254
NM_CONTROLLED=no
BOOTPROTO=static
ZONE=public

###ifcfg-vlan103
VLAN=yes
TYPE=vlan
DEVICE=vlan103
PHYSDEV=teamTRUNK
VLAN_ID=103
NAME=VLAN103
ONBOOT=yes
BRIDGE=cloudbr1.103

###brctl show result###
bridge name bridge id   STP enabled interfaces
brteamTRUNK-500 8000.80c16e791e14   no  teamTRUNK.500

 vnet2

  vnet5
cloud0 8000.fe00a9fe0101   no   
  vnet0

vnet3
cloudbr0 8000.80c16e791e10   no  
teamMGMT

  vnet1

 vnet4
cloudbr1 8000.80c16e791e14   no  
teamTRUNK

  vnet6
cloudbr1.103  8000.80c16e791e14   no  vlan103
virbr0  8000.525400663967   yes virbr0-nic

>From the host i can ping and attached the NFS.

Im running out of idea, any help 

RE: host KVM unable to find cloudbr0

[Nicolas Bouige]

Hello,


We finally found the solution.


We have checked the source code to know exactly how the network settings are 
detected by cloudstack.


##

String [] _ifNamePatterns = {
   "^eth",
   "^bond",
   "^vlan",
   "^vx",
   "^em",
   "^ens",
   "^eno",
   "^enp",
   "^team",
   "^enx",
   "^p\\d+p\\d+"
   };
   /**
* @param fname
* @return
*/
   boolean isInterface(final String fname) {
   StringBuffer commonPattern = new StringBuffer();
   for (final String ifNamePattern : _ifNamePatterns) {
   commonPattern.append("|(").append(ifNamePattern).append(".*)");
   }
   if(fname.matches(commonPattern.toString())) {
   return true;
   }
  return false;
   }



As you can see cloudstack check only the list above of name so if your device 
name doesn't match it fails.


our device team name were MGMT and TRUNK, we just added team.

MGMT --> teamMGMT

TRUNK --> teamTRUNK


(team must be in minus and in first)


now KVM host is up on cloudstack GUI.


So,for the moment,  KVM works fine with teamed NIC configured with nmcli


Best regards,
N.B



De : Dag Sonstebo <dag.sonst...@shapeblue.com>
Envoyé : mardi 6 février 2018 13:55:28
À : users@cloudstack.apache.org
Objet : Re: host KVM unable to find cloudbr0

Hi Nicolas,

Yes I would do a double test with both bonding and teaming and see if the agent 
simply doesn’t like teaming at all.
You can obviously also change the agent logs to trace and see if that sheds 
more light on it.

With regards to naming convention I know this is a contested issue – we do the 
same as you and change it back to the legacy ethX naming convention to simplify 
our build scripts, but overall I would expect it to work with the new world 
naming convention.

Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue

On 06/02/2018, 12:47, "Nicolas Bouige" <n.bou...@dimsi.fr> wrote:

Dag,


okay, i got it, thanks a lot for the details and your help.
As im  stuck with the current configuration with nmcli, im going to try 
without on an another host see if i have more success.


Do you know if someone success to set up KVM networking with the new naming 
convention on centOS7 ? (ensX, enpX..etc)


because i renamed the NICs with ethX but don't know if it was really 
necessary.


Best regards,


N.B

De : Dag Sonstebo <dag.sonst...@shapeblue.com>
Envoyé : mardi 6 février 2018 12:40:19
À : users@cloudstack.apache.org
Objet : Re: host KVM unable to find cloudbr0

Hi Nicolas

These two settings are mutually exclusive – you are controlling your 
networking with NetworkManager (NM) through nmcli. My personal preference is to 
leave NM out of the equation and do all configuration manually (or with 
Ansible, Chef or whatever tool you choose) – hence I mark the different 
interfaces with "NM_CONTROLLED=no" to stop NM ever trying to interfere if 
someone starts the NM service up.

So – if you want to use nmcli then remove "NM_CONTROLLED=no" from your 
config files.

As I said – this is a personal preference only though – you will probably 
manage to get it to work with NM, I just find it too intrusive.

Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue

On 06/02/2018, 11:15, "Nicolas Bouige" <n.bou...@dimsi.fr> wrote:

Hi Dag,


You are right, and i did it, it was not clear enought on my first mail.
I add the ethX interface to the team-MGMT with this command :


nmcli con add type ethernet con-name MGMT-port1 ifname eth0 master MGMT


Here the configuration :

### MGMT-port1 

NAME=MGMT-port1

UUID=-x...etc

DEVICE=eth0

TEAM_MASTER=MGMT

DEVICETYPE=TeamPort



 i just tried with adding "NM_CONTROLLED=no" but its worse, now, i 
can't even contact cloudstack management-server :/

And "ip a" tell me cloudbr0 is down...


So, there is a real difference between :

- create the networking configuration with "nmcli command" and add 
"nm_controlled=no"

- create the networking configuration manually and directly with 
"nm_controlled=no"

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40



De : Dag Sonstebo <dag.sonst...@shapeblue.com>

RE: host KVM unable to find cloudbr0

[Nicolas Bouige]

Dag,


okay, i got it, thanks a lot for the details and your help.
As im  stuck with the current configuration with nmcli, im going to try without 
on an another host see if i have more success.


Do you know if someone success to set up KVM networking with the new naming 
convention on centOS7 ? (ensX, enpX..etc)


because i renamed the NICs with ethX but don't know if it was really necessary.


Best regards,


N.B

De : Dag Sonstebo <dag.sonst...@shapeblue.com>
Envoyé : mardi 6 février 2018 12:40:19
À : users@cloudstack.apache.org
Objet : Re: host KVM unable to find cloudbr0

Hi Nicolas

These two settings are mutually exclusive – you are controlling your networking 
with NetworkManager (NM) through nmcli. My personal preference is to leave NM 
out of the equation and do all configuration manually (or with Ansible, Chef or 
whatever tool you choose) – hence I mark the different interfaces with 
"NM_CONTROLLED=no" to stop NM ever trying to interfere if someone starts the NM 
service up.

So – if you want to use nmcli then remove "NM_CONTROLLED=no" from your config 
files.

As I said – this is a personal preference only though – you will probably 
manage to get it to work with NM, I just find it too intrusive.

Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue

On 06/02/2018, 11:15, "Nicolas Bouige" <n.bou...@dimsi.fr> wrote:

Hi Dag,


You are right, and i did it, it was not clear enought on my first mail.
I add the ethX interface to the team-MGMT with this command :


nmcli con add type ethernet con-name MGMT-port1 ifname eth0 master MGMT


Here the configuration :

### MGMT-port1 

NAME=MGMT-port1

UUID=-x...etc

DEVICE=eth0

TEAM_MASTER=MGMT

DEVICETYPE=TeamPort



 i just tried with adding "NM_CONTROLLED=no" but its worse, now, i can't 
even contact cloudstack management-server :/

And "ip a" tell me cloudbr0 is down...


So, there is a real difference between :

- create the networking configuration with "nmcli command" and add 
"nm_controlled=no"

- create the networking configuration manually and directly with 
"nm_controlled=no"

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40



De : Dag Sonstebo <dag.sonst...@shapeblue.com>
Envoyé : mardi 6 février 2018 11:56:46
À : users@cloudstack.apache.org
Objet : Re: host KVM unable to find cloudbr0

Hi Nicolas,

First of all – you learn something new every day – I didn’t realise there 
was a difference between a team and a bond – but there is: 
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/networking_guide/sec-comparison_of_network_teaming_to_bonding
 So with regards to this – I can’t comment – suggest you test with both, but I 
would expect teaming to work just as well looking through the comparison table 
above.

I may be missing something – but to me it looks like your main problem is 
this:

eth0/eth1 ---X---> teamed NIC (mgmt) -->  cloudbr0

i.e. you have eth0 and eth1 – but they are not linked to the team in any 
way – I would expect to see a master/slave type configuration in your ifcfg-* 
files. The odd thing here is obviously that you can ping the host and speak to 
it in the first place – which would point to cloudbr0 somehow being online – 
hence my suspicion may be wrong.

With regards to nmcli – personally this has caused me too much trouble 
through the years – hence I never use it and just mark my interfaces as 
NM_CONTROLLED=no.


Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue

On 06/02/2018, 10:24, "Nicolas Bouige" <n.bou...@dimsi.fr> wrote:

Hello Dag,


Thanks for your help,


Here the informations :


## IP A RESULT ###

root@ASPRKVM06 network-scripts]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
qlen 1
 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
 inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
  valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast 
master MGMT state UP qlen 1000
link/ether 00:1b:78:2b:3a:de brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast 
master TRUNK state UP qlen 1000
link/ether 00:1b:78:2b:3a:df brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master 
MGMT portid 002481adfe90 state UP qlen 1000
link/ether 00:1b:78:2b:3a:de brd ff

RE: host KVM unable to find cloudbr0

[Nicolas Bouige]

Hi Dag,


You are right, and i did it, it was not clear enought on my first mail.
I add the ethX interface to the team-MGMT with this command :


nmcli con add type ethernet con-name MGMT-port1 ifname eth0 master MGMT


Here the configuration :

### MGMT-port1 

NAME=MGMT-port1

UUID=-x...etc

DEVICE=eth0

TEAM_MASTER=MGMT

DEVICETYPE=TeamPort



 i just tried with adding "NM_CONTROLLED=no" but its worse, now, i can't even 
contact cloudstack management-server :/

And "ip a" tell me cloudbr0 is down...


So, there is a real difference between :

- create the networking configuration with "nmcli command" and add 
"nm_controlled=no"

- create the networking configuration manually and directly with 
"nm_controlled=no"

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d’Asnière – 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40



De : Dag Sonstebo <dag.sonst...@shapeblue.com>
Envoyé : mardi 6 février 2018 11:56:46
À : users@cloudstack.apache.org
Objet : Re: host KVM unable to find cloudbr0

Hi Nicolas,

First of all – you learn something new every day – I didn’t realise there was a 
difference between a team and a bond – but there is: 
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/networking_guide/sec-comparison_of_network_teaming_to_bonding
 So with regards to this – I can’t comment – suggest you test with both, but I 
would expect teaming to work just as well looking through the comparison table 
above.

I may be missing something – but to me it looks like your main problem is this:

eth0/eth1 ---X---> teamed NIC (mgmt) -->  cloudbr0

i.e. you have eth0 and eth1 – but they are not linked to the team in any way – 
I would expect to see a master/slave type configuration in your ifcfg-* files. 
The odd thing here is obviously that you can ping the host and speak to it in 
the first place – which would point to cloudbr0 somehow being online – hence my 
suspicion may be wrong.

With regards to nmcli – personally this has caused me too much trouble through 
the years – hence I never use it and just mark my interfaces as 
NM_CONTROLLED=no.


Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue

On 06/02/2018, 10:24, "Nicolas Bouige" <n.bou...@dimsi.fr> wrote:

Hello Dag,


Thanks for your help,


Here the informations :


## IP A RESULT ###

root@ASPRKVM06 network-scripts]# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1
 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
 inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
  valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master 
MGMT state UP qlen 1000
link/ether 00:1b:78:2b:3a:de brd ff:ff:ff:ff:ff:ff
3: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast master 
TRUNK state UP qlen 1000
link/ether 00:1b:78:2b:3a:df brd ff:ff:ff:ff:ff:ff
4: eth2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master MGMT 
portid 002481adfe90 state UP qlen 1000
link/ether 00:1b:78:2b:3a:de brd ff:ff:ff:ff:ff:ff
5: eth3: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master TRUNK 
portid 002481adfe94 state UP qlen 1000
link/ether 00:1b:78:2b:3a:df brd ff:ff:ff:ff:ff:ff
12: virbr0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue 
state DOWN qlen 1000
link/ether 52:54:00:41:c3:2f brd ff:ff:ff:ff:ff:ff
inet 192.168.122.1/24 brd 192.168.122.255 scope global virbr0
valid_lft forever preferred_lft forever
13: virbr0-nic: <BROADCAST,MULTICAST> mtu 1500 qdisc pfifo_fast master 
virbr0 state DOWN qlen 1000
link/ether 52:54:00:41:c3:2f brd ff:ff:ff:ff:ff:ff
20: cloud0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state 
UNKNOWN qlen 1000
link/ether 56:1e:58:2b:a4:95 brd ff:ff:ff:ff:ff:ff
  inet 169.254.0.1/16 scope global cloud0
 valid_lft forever preferred_lft forever
inet6 fe80::541e:58ff:fe2b:a495/64 scope link
valid_lft forever preferred_lft forever
39: TRUNK: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue master 
cloudbr1 state UP qlen 1000
   link/ether 00:1b:78:2b:3a:df brd ff:ff:ff:ff:ff:ff
   inet6 fe80::21b:78ff:fe2b:3adf/64 scope link
valid_lft forever preferred_lft forever
40: TRUNK103@TRUNK: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc 
noqueue master cloudbr1.103 state UP qlen 1000
  link/ether 00:1b:78:2b:3a:df brd ff:ff:ff:ff:ff:ff
41: cloudbr1.103: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue 
state UP qlen 1000
   link/ether 00:1b:78:2b:3a:df brd ff:ff:ff:ff:ff:ff
   inet 172.16.3.216/24 brd 172.16.3.255 s

RE: host KVM unable to find cloudbr0

[Nicolas Bouige]

n-name MGMT-port1 ifname eth0 master MGMT

nmcli con add type ethernet con-name MGMT-port2 ifname eth2 master MGMT

nmcli conn add type bridge con-name cloudbr0 ifname cloudbr0

all device are up and connected

Best regards,
N.B


De : Dag Sonstebo <dag.sonst...@shapeblue.com>
Envoyé : mardi 6 février 2018 10:26
À : users@cloudstack.apache.org
Objet : Re: host KVM unable to find cloudbr0

Hi Nicolas,

OK I’m with you. Sounds like you have an underlying network issue on your KVM 
host.

Can you post up an ifconfig / ip a from your KVM host?
Can you also post up the contents of the ifcfg-eth0 + ifcfg-eth1 as well as as 
ifcfg- and ifcfg-cloudbr0?


Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue

On 05/02/2018, 20:06, "Nicolas Bouige" <n.bou...@dimsi.fr> wrote:

Hello Dag and Andrija,


Thanks for your answer,


@Andrija, we are using advanced zone and yes, we have specified the 
traffics label and the agent on the host has retrieved the informations.

@Dag, its the documentation i followed, just, instead of bond i used team 
NIC.



Best regards,

N.B



De : Dag Sonstebo <dag.sonst...@shapeblue.com>
Envoyé : lundi 5 février 2018 20:01:17
À : users@cloudstack.apache.org
Objet : Re: host KVM unable to find cloudbr0

Hi Nicolas,

Take a look at the following blog article – it’s a couple of years old but 
should still be valid:

http://www.shapeblue.com/networking-kvm-for-cloudstack/


Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue

On 05/02/2018, 18:51, "Andrija Panic" <andrija.pa...@gmail.com> wrote:

Hi Nicolas,

what does your zone networking look like ?
For every network you setup in the Zone (are you using advanced zones, 
vlan
isolation method ???) you need to specify "KVM traffic label" - this
actually tells ACS what parent interface to look for...

Cheers


dag.sonst...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue




dag.sonst...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
[http://www.shapeblue.com/wp-content/uploads/2017/06/logo.png]<http://www.shapeblue.com/>

Shapeblue - The CloudStack Company<http://www.shapeblue.com/>
www.shapeblue.com
Rapid deployment framework for Apache CloudStack IaaS Clouds. CSForge is a 
framework developed by ShapeBlue to deliver the rapid deployment of a 
standardised ...


53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue



On 5 February 2018 at 18:12, Nicolas Bouige <n.bou...@dimsi.fr> wrote:

> To complete my previous  mail:
>
>
> we are running KVM on CentOS7
>
>
> Here the exact error message from the cloudstack GUI :
>
> incorrect Network setup on agentReinitialize agent after network 
names are
> setupdetails : Can not find network: cloudbr0
>
> 
> De : Nicolas Bouige <n.bou...@dimsi.fr>
> Envoyé : lundi 5 février 2018 18:02:19
> À : users@cloudstack.apache.org
> Objet : host KVM unable to find cloudbr0
>
> Hello all,
>
>
> As a lot of people, we are trying to switch our hypervisor and so our
> cloudstack platform from Xenserver to KVM.
>
>
> And we dont have a lot of experience with the duo cloudstack/KVM, we 
are
> facing some issues and one of them is about the network.
>
> In the official documentation we have to create two bridges called
> cloudbr0 and cloudbr1.
>
> That's what we did.
>
>
> eth0/eth1 --> teamed NIC (mgmt) -->  cloudbr0
>
> eth2/eth3 --> teamed NIC (trunk) --> cloudbr1
>
>
> we add a vlan on teamed NIC (trunk) with the id of the storage 
network.
>
> --> teamed NIC (trunk) --> trunk103 (vlan 103) --> cloudbr1.103
>
>
> The configuration sound good, we can ping each host/storage and web.
>
> cloudbr0 is configured with an IP address and cloudbr1.103 as well.
>
>
> During zone configuration we have added  cloudbr0 for admin traffic  
and
> cloudbr1 for storage/guest/public.
>
>
> We are able to add the host and the agent get all the informations 
needed :
>
> guest.network.device=cloudbr1
>
> workers=5
> private.network.device=cloudbr0
> port=8250

RE: host KVM unable to find cloudbr0

[Nicolas Bouige]

Hello Dag and Andrija,


Thanks for your answer,


@Andrija, we are using advanced zone and yes, we have specified the traffics 
label and the agent on the host has retrieved the informations.

@Dag, its the documentation i followed, just, instead of bond i used team NIC.



Best regards,

N.B



De : Dag Sonstebo <dag.sonst...@shapeblue.com>
Envoyé : lundi 5 février 2018 20:01:17
À : users@cloudstack.apache.org
Objet : Re: host KVM unable to find cloudbr0

Hi Nicolas,

Take a look at the following blog article – it’s a couple of years old but 
should still be valid:

http://www.shapeblue.com/networking-kvm-for-cloudstack/


Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue

On 05/02/2018, 18:51, "Andrija Panic" <andrija.pa...@gmail.com> wrote:

Hi Nicolas,

what does your zone networking look like ?
For every network you setup in the Zone (are you using advanced zones, vlan
isolation method ???) you need to specify "KVM traffic label" - this
actually tells ACS what parent interface to look for...

Cheers


dag.sonst...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue



On 5 February 2018 at 18:12, Nicolas Bouige <n.bou...@dimsi.fr> wrote:

> To complete my previous  mail:
>
>
> we are running KVM on CentOS7
>
>
> Here the exact error message from the cloudstack GUI :
>
> incorrect Network setup on agentReinitialize agent after network names are
> setupdetails : Can not find network: cloudbr0
>
> 
> De : Nicolas Bouige <n.bou...@dimsi.fr>
> Envoyé : lundi 5 février 2018 18:02:19
> À : users@cloudstack.apache.org
> Objet : host KVM unable to find cloudbr0
>
> Hello all,
>
>
> As a lot of people, we are trying to switch our hypervisor and so our
> cloudstack platform from Xenserver to KVM.
>
>
> And we dont have a lot of experience with the duo cloudstack/KVM, we are
> facing some issues and one of them is about the network.
>
> In the official documentation we have to create two bridges called
> cloudbr0 and cloudbr1.
>
> That's what we did.
>
>
> eth0/eth1 --> teamed NIC (mgmt) -->  cloudbr0
>
> eth2/eth3 --> teamed NIC (trunk) --> cloudbr1
>
>
> we add a vlan on teamed NIC (trunk) with the id of the storage network.
>
> --> teamed NIC (trunk) --> trunk103 (vlan 103) --> cloudbr1.103
>
>
> The configuration sound good, we can ping each host/storage and web.
>
> cloudbr0 is configured with an IP address and cloudbr1.103 as well.
>
>
> During zone configuration we have added  cloudbr0 for admin traffic  and
> cloudbr1 for storage/guest/public.
>
>
> We are able to add the host and the agent get all the informations needed 
:
>
> guest.network.device=cloudbr1
>
> workers=5
> private.network.device=cloudbr0
> port=8250
> resource=com.cloud.hypervisor.kvm.resource.LibvirtComputingResource
> pod=2
> zone=2
> hypervisor.type=kvm
> guid=6ce7dedb-0b21-31ed-b7f8-4141613c0946
> public.network.device=cloudbr1
> cluster=3
> local.storage.uuid=dbd798f9-b7ca-4022-943d-9dd2cd8b2bfa
> domr.scripts.dir=scripts/network/domr/kvm
> LibvirtComputingResource.id=0
> host=XXX.XXX.XXX.XXX
>
>
> network cloud0 has been created automatically.
>
> For information, we have followed this ticket as well but nothing changed.
>
> https://issues.apache.org/jira/browse/CLOUDSTACK-8838
>
>
> i guess i misunderstood something during the network configuration but i'm
> running out of idea.
>
>
> Any help will be appreciated ;)
>
>
> Have a great day,
>
> Best regards,
>
>
> N.B
>
>
>


--

Andrija Panić

RE: host KVM unable to find cloudbr0

[Nicolas Bouige]

To complete my previous  mail:


we are running KVM on CentOS7


Here the exact error message from the cloudstack GUI :

incorrect Network setup on agentReinitialize agent after network names are 
setupdetails : Can not find network: cloudbr0


De : Nicolas Bouige <n.bou...@dimsi.fr>
Envoyé : lundi 5 février 2018 18:02:19
À : users@cloudstack.apache.org
Objet : host KVM unable to find cloudbr0

Hello all,


As a lot of people, we are trying to switch our hypervisor and so our 
cloudstack platform from Xenserver to KVM.


And we dont have a lot of experience with the duo cloudstack/KVM, we are facing 
some issues and one of them is about the network.

In the official documentation we have to create two bridges called cloudbr0 and 
cloudbr1.

That's what we did.


eth0/eth1 --> teamed NIC (mgmt) -->  cloudbr0

eth2/eth3 --> teamed NIC (trunk) --> cloudbr1


we add a vlan on teamed NIC (trunk) with the id of the storage network.

--> teamed NIC (trunk) --> trunk103 (vlan 103) --> cloudbr1.103


The configuration sound good, we can ping each host/storage and web.

cloudbr0 is configured with an IP address and cloudbr1.103 as well.


During zone configuration we have added  cloudbr0 for admin traffic  and 
cloudbr1 for storage/guest/public.


We are able to add the host and the agent get all the informations needed :

guest.network.device=cloudbr1

workers=5
private.network.device=cloudbr0
port=8250
resource=com.cloud.hypervisor.kvm.resource.LibvirtComputingResource
pod=2
zone=2
hypervisor.type=kvm
guid=6ce7dedb-0b21-31ed-b7f8-4141613c0946
public.network.device=cloudbr1
cluster=3
local.storage.uuid=dbd798f9-b7ca-4022-943d-9dd2cd8b2bfa
domr.scripts.dir=scripts/network/domr/kvm
LibvirtComputingResource.id=0
host=XXX.XXX.XXX.XXX


network cloud0 has been created automatically.

For information, we have followed this ticket as well but nothing changed.

https://issues.apache.org/jira/browse/CLOUDSTACK-8838


i guess i misunderstood something during the network configuration but i'm 
running out of idea.


Any help will be appreciated ;)


Have a great day,

Best regards,


N.B

host KVM unable to find cloudbr0

[Nicolas Bouige]

Hello all,


As a lot of people, we are trying to switch our hypervisor and so our 
cloudstack platform from Xenserver to KVM.


And we dont have a lot of experience with the duo cloudstack/KVM, we are facing 
some issues and one of them is about the network.

In the official documentation we have to create two bridges called cloudbr0 and 
cloudbr1.

That's what we did.


eth0/eth1 --> teamed NIC (mgmt) -->  cloudbr0

eth2/eth3 --> teamed NIC (trunk) --> cloudbr1


we add a vlan on teamed NIC (trunk) with the id of the storage network.

--> teamed NIC (trunk) --> trunk103 (vlan 103) --> cloudbr1.103


The configuration sound good, we can ping each host/storage and web.

cloudbr0 is configured with an IP address and cloudbr1.103 as well.


During zone configuration we have added  cloudbr0 for admin traffic  and 
cloudbr1 for storage/guest/public.


We are able to add the host and the agent get all the informations needed :

guest.network.device=cloudbr1

workers=5
private.network.device=cloudbr0
port=8250
resource=com.cloud.hypervisor.kvm.resource.LibvirtComputingResource
pod=2
zone=2
hypervisor.type=kvm
guid=6ce7dedb-0b21-31ed-b7f8-4141613c0946
public.network.device=cloudbr1
cluster=3
local.storage.uuid=dbd798f9-b7ca-4022-943d-9dd2cd8b2bfa
domr.scripts.dir=scripts/network/domr/kvm
LibvirtComputingResource.id=0
host=XXX.XXX.XXX.XXX


network cloud0 has been created automatically.

For information, we have followed this ticket as well but nothing changed.

https://issues.apache.org/jira/browse/CLOUDSTACK-8838


i guess i misunderstood something during the network configuration but i'm 
running out of idea.


Any help will be appreciated ;)


Have a great day,

Best regards,


N.B

RE: HOST/CPU ERROR on Web UI

[Nicolas Bouige]

Hello,

After some debug and thanks to your help the issue came from an old PR-STRORE 
which was deleted.
But the database was not properly updated.

mysql> select * from storage_pool_host_ref where pool_id=21;

| 25 |  22 |  21 | 2014-10-24 08:57:50 | NULL| 
/mnt/6866120f-9592-3904-a487-f32bee9d |
| 26 |  26 |  21 | 2014-10-24 08:57:51 | NULL| 
/mnt/6866120f-9592-3904-a487-f32bee9d |
| 27 |  27 |  21 | 2014-10-24 08:57:52 | NULL| 
/mnt/6866120f-9592-3904-a487-f32bee9d |
| 53 |  97 |  21 | 2015-05-28 21:41:11 | NULL| 
/mnt/6866120f-9592-3904-a487-f32bee9d |
| 57 |104 |  21 | 2015-09-01 21:56:02 | NULL| 
/mnt/6866120f-9592-3904-a487-f32bee9d |
| 61 |116 |  21 | 2016-11-30 11:16:58 | NULL| 
/mnt/6866120f-9592-3904-a487-f32bee9d |
| 63 |117 |  21 | 2016-11-30 16:52:54 | NULL| 
/mnt/6866120f-9592-3904-a487-f32bee9d |

mysql> delete from storage_pool_host_ref where pool_id=21;

Now, all is working fine.

Best Regards,
Nicolas Bouige

-Message d'origine-
De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com] 
Envoyé : vendredi 13 janvier 2017 14:30
À : users@cloudstack.apache.org
Objet : Re: HOST/CPU ERROR on Web UI

UPDATE:
Are you sure it is ACS 4.7 you are using?
Because the line that shows the error in your log is this one: "
List volumes = _volsDao.search(volumeSC, null);".
The only way to happen a NPE there, is if "volsDao" is null, which does not 
happen.

On Fri, Jan 13, 2017 at 11:19 AM, Rafael Weingärtner < 
rafaelweingart...@gmail.com> wrote:

> What 4.7 version are you using?
> 4.7.1? 4.7.1.1?
>
> On Fri, Jan 13, 2017 at 7:32 AM, Nicolas Bouige <n.bou...@dimsi.fr> wrote:
>
>> Hello,
>>
>> Sorry typo, we are under Xen 6.5.
>> We are using XenServer 7 in our test lab with 4.9 to prepare the 
>> migration.
>>
>> Best regards,
>> N.B
>> -Message d'origine-
>> De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com]
>> Envoyé : jeudi 12 janvier 2017 19:47
>> À : users@cloudstack.apache.org
>> Objet : Re: HOST/CPU ERROR on Web UI
>>
>> Is this a new environment or a production one? I was under the 
>> impression that the version 4.7 does not support XenServer7.
>> I thought the support was going to be added only on 4.9+.
>>
>> On Thu, Jan 12, 2017 at 10:14 AM, Nicolas Bouige <n.bou...@dimsi.fr>
>> wrote:
>>
>> > Hello Rafael,
>> >
>> > We're using 4.7 and XenServer 7
>> >
>> > Thanks,
>> > N.B
>> >
>> > -Message d'origine-
>> > De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com]
>> > Envoyé : jeudi 12 janvier 2017 15:58 À : 
>> > users@cloudstack.apache.org Objet : Re: HOST/CPU ERROR on Web UI
>> >
>> > It seems to be related.
>> >
>> > What version of ACS are you using?
>> >
>> > On Thu, Jan 12, 2017 at 12:52 PM, Nicolas Bouige 
>> > <n.bou...@dimsi.fr>
>> > wrote:
>> >
>> > > Hello All,
>> > >
>> > > I'm facing an issue with the displaying of information about HOST 
>> > > and
>> > CPU.
>> > > On CPU web page there are no information display and on HOST page 
>> > > only < ERROR >.
>> > >
>> > > I dont know if its related but there is this error on 
>> > > management-server.log :
>> > >
>> > > 2017-01-12 15:23:14,934 DEBUG [c.c.a.ApiServlet]
>> > > (catalina-exec-17:ctx-b4b62069)
>> > > (logid:45883c7d) ===START===  XX.XX.XX.XX -- GET 
>> > > command=listHosts=json=routing=
>> > > true=1=20&_=1484230881538
>> > > 2017-01-12 15:23:14,940 WARN  
>> > > [c.c.a.d.ParamGenericValidationWorker]
>> > > (catalina-exec-17:ctx-b4b62069 ctx-48ead17f) (logid:45883c7d) 
>> > > Received unknown parameters for command listHosts. Unknown 
>> > > parameters : listall
>> > > 2017-01-12 15:23:14,941 DEBUG [c.c.a.q.QueryManagerImpl]
>> > > (catalina-exec-17:ctx-b4b62069 ctx-48ead17f) (logid:45883c7d)
>> > > >>>Searching for hosts>>>
>> > > 2017-01-12 15:23:14,949 DEBUG [c.c.a.q.QueryManagerImpl]
>> > > (catalina-exec-17:ctx-b4b62069 ctx-48ead17f) (logid:45883c7d)
>> > > >>>Generating
>> > > Response>>>
>> > > 2017-01-12 15:23:14,955 ERROR [c.c.a.ApiServer]
>> > > (catalina-exec-17:ctx-b4b62069 ctx-48ead17f) (logid:45883c7d) 
>> > > unhandled exception executing a

RE: HOST/CPU ERROR on Web UI

[Nicolas Bouige]

Hello Rafael,

We're using 4.7 and XenServer 7

Thanks,
N.B

-Message d'origine-
De : Rafael Weingärtner [mailto:rafaelweingart...@gmail.com] 
Envoyé : jeudi 12 janvier 2017 15:58
À : users@cloudstack.apache.org
Objet : Re: HOST/CPU ERROR on Web UI

It seems to be related.

What version of ACS are you using?

On Thu, Jan 12, 2017 at 12:52 PM, Nicolas Bouige <n.bou...@dimsi.fr> wrote:

> Hello All,
>
> I'm facing an issue with the displaying of information about HOST and CPU.
> On CPU web page there are no information display and on HOST page only 
> < ERROR >.
>
> I dont know if its related but there is this error on 
> management-server.log :
>
> 2017-01-12 15:23:14,934 DEBUG [c.c.a.ApiServlet] 
> (catalina-exec-17:ctx-b4b62069)
> (logid:45883c7d) ===START===  XX.XX.XX.XX -- GET 
> command=listHosts=json=routing=
> true=1=20&_=1484230881538
> 2017-01-12 15:23:14,940 WARN  [c.c.a.d.ParamGenericValidationWorker]
> (catalina-exec-17:ctx-b4b62069 ctx-48ead17f) (logid:45883c7d) Received 
> unknown parameters for command listHosts. Unknown parameters : listall
> 2017-01-12 15:23:14,941 DEBUG [c.c.a.q.QueryManagerImpl]
> (catalina-exec-17:ctx-b4b62069 ctx-48ead17f) (logid:45883c7d) 
> >>>Searching for hosts>>>
> 2017-01-12 15:23:14,949 DEBUG [c.c.a.q.QueryManagerImpl]
> (catalina-exec-17:ctx-b4b62069 ctx-48ead17f) (logid:45883c7d) 
> >>>Generating
> Response>>>
> 2017-01-12 15:23:14,955 ERROR [c.c.a.ApiServer]
> (catalina-exec-17:ctx-b4b62069 ctx-48ead17f) (logid:45883c7d) 
> unhandled exception executing api command: 
> [Ljava.lang.String;@59a40f5c java.lang.NullPointerException
>  at com.cloud.storage.StorageManagerImpl.
> isLocalStorageActiveOnHost(StorageManagerImpl.java:391)
>  at com.cloud.api.ApiDBUtils.isLocalStorageActiveOnHost(
> ApiDBUtils.java:908)
>  at 
> com.cloud.api.query.dao.HostJoinDaoImpl.newHostResponse(
> HostJoinDaoImpl.java:214)
>  at sun.reflect.GeneratedMethodAccessor338.invoke(Unknown
> Source)
>  at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:498)
>  at org.springframework.aop.support.AopUtils.
> invokeJoinpointUsingReflection(AopUtils.java:317)
>  at org.springframework.aop.framework.
> ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.
> java:183)
>  at org.springframework.aop.framework.
> ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
>  at 
> com.cloud.utils.db.TransactionContextInterceptor.invoke(
> TransactionContextInterceptor.java:34)
>  at org.springframework.aop.framework.
> ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
>  at org.springframework.aop.interceptor.
> ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
>  at org.springframework.aop.framework.
> ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
>  at org.springframework.aop.framework.JdkDynamicAopProxy.
> invoke(JdkDynamicAopProxy.java:204)
>  at com.sun.proxy.$Proxy284.newHostResponse(Unknown Source)
>  at com.cloud.api.ApiDBUtils.newHostResponse(ApiDBUtils.
> java:1781)
>  at 
> com.cloud.api.query.ViewResponseHelper.createHostResponse(
> ViewResponseHelper.java:252)
>  at com.cloud.api.query.QueryManagerImpl.searchForServers(
> QueryManagerImpl.java:1655)
>  at org.apache.cloudstack.api.command.admin.host.
> ListHostsCmd.execute(ListHostsCmd.java:186)
>  at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:
> 132)
>  at com.cloud.api.ApiServer.queueCommand(ApiServer.java:707)
>  at com.cloud.api.ApiServer.handleRequest(ApiServer.java:538)
>  at com.cloud.api.ApiServlet.processRequestInContext(
> ApiServlet.java:297)
>  at com.cloud.api.ApiServlet$1.run(ApiServlet.java:129)
>  at org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext$1.call(DefaultManagedContext.java:56)
>  at org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
>  at org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
>  at com.cloud.api.ApiServlet.processRequest(ApiServlet.
> java:126)
>  at com.cloud.api.ApiServlet.doGet(ApiServlet.java:86)
>  at javax.servlet.http.HttpServlet.service(
> HttpServlet.java:617)
>

HOST/CPU ERROR on Web UI

[Nicolas Bouige]

Hello All,

I'm facing an issue with the displaying of information about HOST and CPU.
On CPU web page there are no information display and on HOST page only < ERROR 
>.

I dont know if its related but there is this error on management-server.log :

2017-01-12 15:23:14,934 DEBUG [c.c.a.ApiServlet] 
(catalina-exec-17:ctx-b4b62069) (logid:45883c7d) ===START===  XX.XX.XX.XX -- 
GET  
command=listHosts=json=routing=true=1=20&_=1484230881538
2017-01-12 15:23:14,940 WARN  [c.c.a.d.ParamGenericValidationWorker] 
(catalina-exec-17:ctx-b4b62069 ctx-48ead17f) (logid:45883c7d) Received unknown 
parameters for command listHosts. Unknown parameters : listall
2017-01-12 15:23:14,941 DEBUG [c.c.a.q.QueryManagerImpl] 
(catalina-exec-17:ctx-b4b62069 ctx-48ead17f) (logid:45883c7d) >>>Searching for 
hosts>>>
2017-01-12 15:23:14,949 DEBUG [c.c.a.q.QueryManagerImpl] 
(catalina-exec-17:ctx-b4b62069 ctx-48ead17f) (logid:45883c7d) >>>Generating 
Response>>>
2017-01-12 15:23:14,955 ERROR [c.c.a.ApiServer] (catalina-exec-17:ctx-b4b62069 
ctx-48ead17f) (logid:45883c7d) unhandled exception executing api command: 
[Ljava.lang.String;@59a40f5c
java.lang.NullPointerException
 at 
com.cloud.storage.StorageManagerImpl.isLocalStorageActiveOnHost(StorageManagerImpl.java:391)
 at 
com.cloud.api.ApiDBUtils.isLocalStorageActiveOnHost(ApiDBUtils.java:908)
 at 
com.cloud.api.query.dao.HostJoinDaoImpl.newHostResponse(HostJoinDaoImpl.java:214)
 at sun.reflect.GeneratedMethodAccessor338.invoke(Unknown Source)
 at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
 at java.lang.reflect.Method.invoke(Method.java:498)
 at 
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
 at 
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
 at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
 at 
com.cloud.utils.db.TransactionContextInterceptor.invoke(TransactionContextInterceptor.java:34)
 at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:161)
 at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
 at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
 at 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
 at com.sun.proxy.$Proxy284.newHostResponse(Unknown Source)
 at com.cloud.api.ApiDBUtils.newHostResponse(ApiDBUtils.java:1781)
 at 
com.cloud.api.query.ViewResponseHelper.createHostResponse(ViewResponseHelper.java:252)
 at 
com.cloud.api.query.QueryManagerImpl.searchForServers(QueryManagerImpl.java:1655)
 at 
org.apache.cloudstack.api.command.admin.host.ListHostsCmd.execute(ListHostsCmd.java:186)
 at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:132)
 at com.cloud.api.ApiServer.queueCommand(ApiServer.java:707)
 at com.cloud.api.ApiServer.handleRequest(ApiServer.java:538)
 at 
com.cloud.api.ApiServlet.processRequestInContext(ApiServlet.java:297)
 at com.cloud.api.ApiServlet$1.run(ApiServlet.java:129)
 at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
 at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
 at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
 at com.cloud.api.ApiServlet.processRequest(ApiServlet.java:126)
 at com.cloud.api.ApiServlet.doGet(ApiServlet.java:86)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
 at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
 at 
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290)
 at 
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
 at 
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
 at 
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
 at 
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:127)
 at 
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
 at 
org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:555)
 at 
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
 at 

RE: SSVM add new SecStr

[Nicolas Bouige]

Hello Dag,


Thanks for the details.

i misconfigured the port on the switch...wrong VLAN :/

Now all works fine.


Thanks a lot for your help and i wish you an happy new year :)


Nicolas Bouige

DIMSI

cloud.dimsi.fr<http://www.cloud.dimsi.fr>

4, avenue Laurent Cely

Tour d’Asnière – 92600 Asnière sur Seine

T/ +33 (0)6 28 98 53 40



De : Dag Sonstebo <dag.sonst...@shapeblue.com>
Envoyé : jeudi 29 décembre 2016 13:09:53
À : users@cloudstack.apache.org
Objet : Re: SSVM add new SecStr

Hi Nicolas,

Is your new secondary storage share is on a different server / appliance than 
your first secondary storage pool?

In short the best way to troubleshoot this is to increase logging verbosity on 
your NFS server and see if this highlights what the issue is.

One other thing to check is the actual folder permissions on the share (rather 
than the NFS ACLs) – can you e.g. briefly test what happens if you fully open 
this up (chmod –R 777 /folder…)?

We have also seen issues in the past where the user UID used by CloudStack – 
SSVM and hypervisors – is mismatched on additional secondary storage shares. In 
short connections come in from e.g. user UID 1000, which matches the 
permissions on the first secondary storage pool. However on the second share 
the permissions are set up for e.g. UID 1003 – and since inbound connections 
still use UID 1000 you have a permission mismatch. The cases I’ve seen have 
generally been when the new share is hosted on a new NFS server, otherwise this 
scenario would not normally apply. I believe various NFS flavours have ways of 
dealing with this though.

Hope this helps,

Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue

On 29/12/2016, 10:07, "Nicolas Bouige" <n.bou...@dimsi.fr> wrote:

Hello Dag,


I tried yesterday but it can't connect the share to the SSVM...

Permission denied.

On the mgmt-server its okay and there are the same permission rules.

i added the ip range vlan-storage on the NFS Share


BR,

Nicolas Bouige

DIMSI

cloud.dimsi.fr<http://www.cloud.dimsi.fr>

4, avenue Laurent Cely

Tour d’Asnière – 92600 Asnière sur Seine

T/ +33 (0)6 28 98 53 40



De : Dag Sonstebo <dag.sonst...@shapeblue.com>
Envoyé : mercredi 28 décembre 2016 20:21:14
À : users@cloudstack.apache.org
Objet : Re: SSVM add new SecStr

Hi Nicolas,

Can you mount the new secstorage share manually on the SSVM? If so can you 
write to it?

Regards,
Dag Sonstebo
Cloud Architect
    ShapeBlue

    On 28/12/2016, 17:04, "Nicolas Bouige" <n.bou...@dimsi.fr> wrote:

Hi Dag,


thanks for your answer,

Yes, i opened up the permissions to the full range IP management and 
storage vlan.



BR

Nicolas Bouige




De : Dag Sonstebo <dag.sonst...@shapeblue.com>
Envoyé : mercredi 28 décembre 2016 17:19:24
À : users@cloudstack.apache.org
Objet : Re: SSVM add new SecStr

Hi Nicolas,

Have you opened up the permissions on your new secondary storage share 
to the full management IP range?

Regards,
Dag Sonstebo
Cloud Architect
    ShapeBlue

On 28/12/2016, 16:02, "Nicolas Bouige" <n.bou...@dimsi.fr> wrote:

Hello All,


Im trying to add a new secondary storage on my cloudstack 
envoronment (4.7) but it seems to not working.
The SSVM works fine as i already have a SecStr configured and on 
the GUI  VM is running and agent is up.


I checked my NFS by added it directly on the mgmt server and all 
works fine, i can create file..etc

I used the script "/usr/local/cloud/systemvm/ssvm-check.sh" and the 
script see only one SecStr and i got an error :

the script doesn't have permission to create file.
I exported the NFS with "norootsquash" and as i said i can create 
file on it directly.


For information,  the first SecStr is full, do you think there is a 
relation ?


Thanks for your help


BR,
Nicolas Bouige





dag.sonst...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue






dag.sonst...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue






dag.sonst...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue

RE: SSVM add new SecStr

[Nicolas Bouige]

Hello Dag,


I tried yesterday but it can't connect the share to the SSVM...

Permission denied.

On the mgmt-server its okay and there are the same permission rules.

i added the ip range vlan-storage on the NFS Share


BR,

Nicolas Bouige

DIMSI

cloud.dimsi.fr<http://www.cloud.dimsi.fr>

4, avenue Laurent Cely

Tour d’Asnière – 92600 Asnière sur Seine

T/ +33 (0)6 28 98 53 40



De : Dag Sonstebo <dag.sonst...@shapeblue.com>
Envoyé : mercredi 28 décembre 2016 20:21:14
À : users@cloudstack.apache.org
Objet : Re: SSVM add new SecStr

Hi Nicolas,

Can you mount the new secstorage share manually on the SSVM? If so can you 
write to it?

Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue

On 28/12/2016, 17:04, "Nicolas Bouige" <n.bou...@dimsi.fr> wrote:

Hi Dag,


thanks for your answer,

Yes, i opened up the permissions to the full range IP management and 
storage vlan.



BR

Nicolas Bouige




De : Dag Sonstebo <dag.sonst...@shapeblue.com>
Envoyé : mercredi 28 décembre 2016 17:19:24
À : users@cloudstack.apache.org
Objet : Re: SSVM add new SecStr

Hi Nicolas,

Have you opened up the permissions on your new secondary storage share to 
the full management IP range?

Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue

    On 28/12/2016, 16:02, "Nicolas Bouige" <n.bou...@dimsi.fr> wrote:

Hello All,


Im trying to add a new secondary storage on my cloudstack envoronment 
(4.7) but it seems to not working.
The SSVM works fine as i already have a SecStr configured and on the 
GUI  VM is running and agent is up.


I checked my NFS by added it directly on the mgmt server and all works 
fine, i can create file..etc

I used the script "/usr/local/cloud/systemvm/ssvm-check.sh" and the 
script see only one SecStr and i got an error :

the script doesn't have permission to create file.
I exported the NFS with "norootsquash" and as i said i can create file 
on it directly.


For information,  the first SecStr is full, do you think there is a 
relation ?


Thanks for your help


BR,
Nicolas Bouige





dag.sonst...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue






dag.sonst...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue

RE: SSVM add new SecStr

[Nicolas Bouige]

Hi Dag,


thanks for your answer,

Yes, i opened up the permissions to the full range IP management and storage 
vlan.



BR

Nicolas Bouige




De : Dag Sonstebo <dag.sonst...@shapeblue.com>
Envoyé : mercredi 28 décembre 2016 17:19:24
À : users@cloudstack.apache.org
Objet : Re: SSVM add new SecStr

Hi Nicolas,

Have you opened up the permissions on your new secondary storage share to the 
full management IP range?

Regards,
Dag Sonstebo
Cloud Architect
ShapeBlue

On 28/12/2016, 16:02, "Nicolas Bouige" <n.bou...@dimsi.fr> wrote:

Hello All,


Im trying to add a new secondary storage on my cloudstack envoronment (4.7) 
but it seems to not working.
The SSVM works fine as i already have a SecStr configured and on the GUI  
VM is running and agent is up.


I checked my NFS by added it directly on the mgmt server and all works 
fine, i can create file..etc

I used the script "/usr/local/cloud/systemvm/ssvm-check.sh" and the script 
see only one SecStr and i got an error :

the script doesn't have permission to create file.
I exported the NFS with "norootsquash" and as i said i can create file on 
it directly.


For information,  the first SecStr is full, do you think there is a 
relation ?


Thanks for your help


BR,
Nicolas Bouige





dag.sonst...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue

SSVM add new SecStr

[Nicolas Bouige]

Hello All,


Im trying to add a new secondary storage on my cloudstack envoronment (4.7) but 
it seems to not working.
The SSVM works fine as i already have a SecStr configured and on the GUI  VM is 
running and agent is up.


I checked my NFS by added it directly on the mgmt server and all works fine, i 
can create file..etc

I used the script "/usr/local/cloud/systemvm/ssvm-check.sh" and the script see 
only one SecStr and i got an error :

the script doesn't have permission to create file.
I exported the NFS with "norootsquash" and as i said i can create file on it 
directly.


For information,  the first SecStr is full, do you think there is a relation ?


Thanks for your help


BR,
Nicolas Bouige

RE: Volume migration unable to serialize

[Nicolas Bouige]

Juste one more question :

These settings works for a live migration as well ?

Nicolas Bouige
DIMSI
cloud.dimsi.fr<http://www.cloud.dimsi.fr>
4, avenue Laurent Cely
Tour d'Asnière - 92600 Asnière sur Seine
T/ +33 (0)6 28 98 53 40



De : Nicolas Bouige
Envoyé : vendredi 2 décembre 2016 09:10:28
À : users@cloudstack.apache.org
Objet : RE: Volume migration unable to serialize


Hello,


Thanks a lot Makrand and Sergey, im going to try today !


Have a good day,


Best regards,

Nicolas Bouige




De : Makrand <makrandsa...@gmail.com>
Envoyé : vendredi 2 décembre 2016 06:38:49
À : users@cloudstack.apache.org
Objet : Re: Volume migration unable to serialize

Hi Nicolas,

With xenserver its always issue with moving bigger volumes.

Here are the parameters that worked for me for bigger volume migrations.
Just need to put 1 additional zero at end really.

migratewait: 36000
storage.pool.max.waitseconds: 36000
wait:18000

*wait *is most critical of all and bit confusing in name.

--
Makrand


On Fri, Dec 2, 2016 at 4:32 AM, Sergey Levitskiy <
sergey.levits...@autodesk.com> wrote:

> You need to adjust this threshold
>
> job.cancel.threshold.minutes
>
>
>
> Thanks,
> Sergey
>
>
>

RE: Volume migration unable to serialize

[Nicolas Bouige]

Hello,


Thanks a lot Makrand and Sergey, im going to try today !


Have a good day,


Best regards,

Nicolas Bouige




De : Makrand <makrandsa...@gmail.com>
Envoyé : vendredi 2 décembre 2016 06:38:49
À : users@cloudstack.apache.org
Objet : Re: Volume migration unable to serialize

Hi Nicolas,

With xenserver its always issue with moving bigger volumes.

Here are the parameters that worked for me for bigger volume migrations.
Just need to put 1 additional zero at end really.

migratewait: 36000
storage.pool.max.waitseconds: 36000
wait:18000

*wait *is most critical of all and bit confusing in name.

--
Makrand


On Fri, Dec 2, 2016 at 4:32 AM, Sergey Levitskiy <
sergey.levits...@autodesk.com> wrote:

> You need to adjust this threshold
>
> job.cancel.threshold.minutes
>
>
>
> Thanks,
> Sergey
>
>
>

Volume migration unable to serialize

[Nicolas Bouige]

Hello team,

We are trying to migrate a volume (1TB) from a primary storage to antoher one.

The VM is stopped but we got this error log :

2016-12-01 20:31:42,629 ERROR [c.c.a.ApiAsyncJobDispatcher] 
(API-Job-Executor-43:ctx-5883082d job-89033) (logid:df7aaf8e) Unexpected 
exception while executing 
org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd
com.cloud.utils.exception.CloudRuntimeException: Unable to serialize: Job is 
cancelled as it has been blocking others for too long
at 
org.apache.cloudstack.framework.jobs.impl.JobSerializerHelper.fromObjectSerializedString(JobSerializerHelper.java:134)
at 
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl.unmarshallResultObject(AsyncJobManagerImpl.java:759)
at 
com.cloud.vm.VirtualMachineManagerImpl.storageMigration(VirtualMachineManagerImpl.java:1854)
at 
com.cloud.vm.UserVmManagerImpl.vmStorageMigration(UserVmManagerImpl.java:4390)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:317)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:183)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:150)
at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at 
org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy198.vmStorageMigration(Unknown Source)
at 
org.apache.cloudstack.api.command.admin.vm.MigrateVMCmd.execute(MigrateVMCmd.java:176)
at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:132)
at 
com.cloud.api.ApiAsyncJobDispatcher.runJob(ApiAsyncJobDispatcher.java:109)
at 
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.runInContext(AsyncJobManagerImpl.java:594)
at 
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)
at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)
at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)
at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)
at 
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)
at 
org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:541)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.StreamCorruptedException: invalid stream header: 2686E2B1
at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:806)
at java.io.ObjectInputStream.(ObjectInputStream.java:299)
at 
org.apache.cloudstack.framework.jobs.impl.JobSerializerHelper.fromObjectSerializedString(JobSerializerHelper.java:128)
... 29 more
2016-12-01 20:31:42,630 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl] 
(API-Job-Executor-43:ctx-5883082d job-89033) (logid:df7aaf8e) Complete async 
job-89033, jobStatus: FAILED, resultCode: 530, result: 
org.apache.cloudstack.api.response.ExceptionResponse/null/{"uuidList":[],"errorcode":530,"errortext":"Unable
 to serialize: Job is cancelled as it has been blocking others for too long"}

The probleme seems to be only a kind of timeout threshold, we have changed 
these global options to "0" :
migratewait: 3600
storage.pool.max.waitseconds: 3600
vm.op.cancel.interval: 3600
vm.op.cleanup.wait: 3600
wait:1800
vm.tranisition.wait.interval:3600

But we still get the error after one hour and on xenserver side the job is 
pending...

There is way to migrate big volume between two primary storage ? or to increase 
the threshold ?

Thanks,
N.B


Nicolas Bouige

RE: Migrate VM system

[Nicolas Bouige]

Hello Dag,

Could be a good idea for the next time.
Finally, the system VM migration is okay ;)

Have a great day,
N.B

-Message d'origine-
De : Dag Sonstebo [mailto:dag.sonst...@shapeblue.com] 
Envoyé : jeudi 24 novembre 2016 14:44
À : users@cloudstack.apache.org
Objet : Re: Migrate VM system

Hi Nicolas,

This seems a lot of effort - have you considered just putting the primary 
storage into maintenance mode and destroying the system Vms, thereby forcing 
them to be recreated on the new storage pool?
 
Regards, 
Dag Sonstebo
Cloud Architect
ShapeBlue








On 24/11/2016, 09:18, "Nicolas Bouige" <n.bou...@dimsi.fr> wrote:

>Dear All,
>
>
>I have a CS 4.7 advanced zone working fine with two primary storage.
>
>we are planning to change one of our primary storage but first we need to 
>delete the second one.
>
>
>We have migrated almost all the vm from the second primary storage to the 
>first primary storage.
>
>To migrate storage VM on Xenserver we need to install Xentool on each VMs.
>
>
>Now, it remains to migrate only system VM from the second primary storage to 
>finalize our operation.
>
>Unfortunately, as the Systems VM have not Xentool installed we cannot perform 
>a storage migration.
>
>
>Do you know if its possible to install xentool on a system VM in use ? 
>(Virtual router and Secondary storage VM)
>
>Or if there is a another way to migrate/change primary storage's system VM ?
>
>
>Any help would be highly appreciated.
>
>
>Best Regards,
>Nicolas Bouige
>
>

dag.sonst...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 

RE: Migrate VM system

[Nicolas Bouige]

Hello, 

Thanks for your answer.
Yes, i read the documentation but my xenserver (6.5) doesnt see the xentools 
are installed on the system VM

N.B

-Message d'origine-
De : Abhinandan Prateek [mailto:abhinandan.prat...@shapeblue.com] 
Envoyé : jeudi 24 novembre 2016 11:09
À : users@cloudstack.apache.org
Objet : Re: Migrate VM system

AFAIK xentools are installed on system vm template released by cloudstack.




On 24/11/16, 2:48 PM, "Nicolas Bouige" <n.bou...@dimsi.fr> wrote:

>Dear All,
>
>
>I have a CS 4.7 advanced zone working fine with two primary storage.
>
>we are planning to change one of our primary storage but first we need to 
>delete the second one.
>
>
>We have migrated almost all the vm from the second primary storage to the 
>first primary storage.
>
>To migrate storage VM on Xenserver we need to install Xentool on each VMs.
>
>
>Now, it remains to migrate only system VM from the second primary storage to 
>finalize our operation.
>
>Unfortunately, as the Systems VM have not Xentool installed we cannot perform 
>a storage migration.
>
>
>Do you know if its possible to install xentool on a system VM in use ? 
>(Virtual router and Secondary storage VM)
>
>Or if there is a another way to migrate/change primary storage's system VM ?
>
>
>Any help would be highly appreciated.
>
>
>Best Regards,
>Nicolas Bouige
>
>

abhinandan.prat...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
  
 

Migrate VM system

[Nicolas Bouige]

Dear All,


I have a CS 4.7 advanced zone working fine with two primary storage.

we are planning to change one of our primary storage but first we need to 
delete the second one.


We have migrated almost all the vm from the second primary storage to the first 
primary storage.

To migrate storage VM on Xenserver we need to install Xentool on each VMs.


Now, it remains to migrate only system VM from the second primary storage to 
finalize our operation.

Unfortunately, as the Systems VM have not Xentool installed we cannot perform a 
storage migration.


Do you know if its possible to install xentool on a system VM in use ? (Virtual 
router and Secondary storage VM)

Or if there is a another way to migrate/change primary storage's system VM ?


Any help would be highly appreciated.


Best Regards,
Nicolas Bouige