SSVM and CPVM agent unable to start after console proxy SSL certificate update

[Cloud List]

Hi,

Merry Christmas to all.

We are using Cloudstack with KVM hypervisor. Since our console proxy SSL
certificate has expired, we updated our new SSL certificate using below
method:

http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.9/systemvm.html#using-a-ssl-certificate-for-the-console-proxy

We have done the above method in the past years without any issues, however
this time round, both the SSVM and CPVM agents are not able to start after
the update.

The state for both VMs are up but agents are in "disconnected" state. We
are still able to login to the SSVM, and found out that the cloud service
is not running.

root@s-4200-VM:~# service cloud status
CloudStack cloud service is not running

Tried to start the service:

root@s-4200-VM:~# service cloud start
Starting CloudStack cloud service (type=secstorage) Success

But the service is not started:

root@s-4200-VM:~# service cloud status
CloudStack cloud service is not running

Below is the logs from /var/log/cloud.log:

=
Sat Dec 26 03:45:04 UTC 2020 Executing cloud-early-config
Sat Dec 26 03:45:04 UTC 2020 Detected that we are running inside kvm guest
Sat Dec 26 03:45:04 UTC 2020 Found a non empty cmdline file. Will now exit
the loop and proceed with configuration.
Sat Dec 26 03:45:04 UTC 2020 Patching  cloud service
Sat Dec 26 03:45:10 UTC 2020 Updating log4j-cloud.xml
Sat Dec 26 03:45:10 UTC 2020 Setting up secondary storage system vm
Sat Dec 26 03:45:10 UTC 2020 checking that eth0 has IP
Sat Dec 26 03:45:11 UTC 2020 waiting for eth0 interface setup with ip
timer=0
Sat Dec 26 03:45:11 UTC 2020 checking that eth1 has IP
Sat Dec 26 03:45:11 UTC 2020 checking that eth2 has IP
Sat Dec 26 03:45:20 UTC 2020 checking that eth3 has IP
Sat Dec 26 03:45:20 UTC 2020 Successfully setup storage network with
STORAGE_IP:10.19.22.67, STORAGE_NETMASK:255.255.240.0, STORAGE_CIDR:
Sat Dec 26 03:45:20 UTC 2020 Setting up route of RFC1918 space to 10.19.16.1
Sat Dec 26 03:45:20 UTC 2020 Setting up apache web server
Sat Dec 26 03:45:20 UTC 2020 setting up apache2 for post upload of
volume/template
Sat Dec 26 03:45:20 UTC 2020 rewrite rules already exist in file
/etc/apache2/sites-available/default-ssl
Sat Dec 26 03:45:20 UTC 2020 adding cors rules to file:
/etc/apache2/sites-available/default-ssl
Sat Dec 26 03:45:21 UTC 2020 cloud: disable rp_filter
Sat Dec 26 03:45:21 UTC 2020 disable rpfilter
Sat Dec 26 03:45:21 UTC 2020 cloud: enable_fwding = 0
Sat Dec 26 03:45:21 UTC 2020 enable_fwding = 0
Sat Dec 26 03:45:21 UTC 2020 Enable service haproxy = 0
Sat Dec 26 03:45:21 UTC 2020 Processors = 1  Enable service  = 0
Sat Dec 26 03:45:21 UTC 2020 Enable service dnsmasq = 0
Sat Dec 26 03:45:21 UTC 2020 Enable service cloud-passwd-srvr = 0
Sat Dec 26 03:45:21 UTC 2020 Enable service cloud = 1
=

Result of /usr/local/cloud/systemvm/ssvm-check.sh:

=
root@s-4200-VM:/var/log# /usr/local/cloud/systemvm/ssvm-check.sh

First DNS server is  8.8.8.8
PING 8.8.8.8 (8.8.8.8): 48 data bytes
56 bytes from 8.8.8.8: icmp_seq=0 ttl=122 time=0.531 ms
56 bytes from 8.8.8.8: icmp_seq=1 ttl=122 time=0.676 ms
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.531/0.604/0.676/0.073 ms
Good: Can ping DNS server

Good: DNS resolves download.cloud.com

ERROR: NFS is not currently mounted
Try manually mounting from inside the VM
NFS server is  X.X.201.1
PING X.X.201.1 (X.X.201.1): 48 data bytes
56 bytes from X.X.201.1: icmp_seq=0 ttl=255 time=0.463 ms
56 bytes from X.X.201.1: icmp_seq=1 ttl=255 time=0.482 ms
--- X.X.201.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.463/0.473/0.482/0.000 ms
Good: Can ping nfs server

Management server is 10.237.3.8. Checking connectivity.
Good: Can connect to management server port 8250

ERROR: Java process not running.  Try restarting the SSVM.
root@s-4200-VM:/var/log#
=

The result is OK except the NFS test, but we checked the IP address is not
correct (X.X.201.1 which is the public IP address of the gateway rather
than the actual NFS server IP). We tested mounting to the actual NFS server
and it works fine.

Have tried stopping and starting back the SSVM and the issue still persists.

Anyone can help to advice how we can resolve the problem?

Looking forward to your reply, thank you.

-ip-


Virus-free.
www.avg.com

<#DAB4FAD8-2DD7-40BB-A1B8-4E2AA1F9FDF2>

Re: Automatically initialize data-disk on Windows Server 2016 OS template

[Cloud List]

Dear all,

Anyone who has created a Windows Server 2016 OS template on CloudStack can
advise on what's the best way to auto-initialize the data-disk upon VM
provisioning?

Any advice is highly appreciated.

Thank you.



On Mon, Aug 14, 2017 at 1:32 PM, Cloud List <cloud-l...@sg.or.id> wrote:

> Dear all,
>
> I am using CloudStack 4.8.1 and KVM hypervisor. When creating my Windows
> Server 2016 OS template, I tried to automate the initialization and
> formatting of the data-disk so that it will automatically be put online,
> initialized with MBR partition, formatted and assigned the E: drive letter,
> without the user having to manually initialize the disk using Computer
> Management > Disk Management.
>
> I tried to do so by preparing 2 scripts:
>
> 1. A PowerShell script to automatically initialize and format all disks
> with RAW partition, which I am taking from here:
>
> https://blogs.technet.microsoft.com/heyscriptingguy/
> 2013/05/29/use-powershell-to-initialize-raw-disks-and-to-
> partition-and-format-volumes/
>
> I saved it on C:\script\InitializePartitionFormatNewDisks.ps1.
>
> 2. A command script to automatically run the above PowerShell script when
> the Administrator user logs in. I follow the instructions on below URL:
>
> https://stackoverflow.com/questions/20575257/how-to-run-
> powershell-script-when-computer-starts
>
> to create a command script to set PowerShell execution policy to
> "unrestricted" and then run the script.
>
> ===
> PowerShell -Command "Set-ExecutionPolicy Unrestricted" >>
> "%TEMP%\StartupLog.txt" 2>&1 PowerShell C:\script\
> InitializePartitionFormatNewDisks.ps1 >> "%TEMP%\StartupLog.txt" 2>&1
> ===
>
> I saved the command script onto: C:\Users\Administrator\
> AppData\Roaming\Microsoft\Windows\Start Menu\Programs\Startup\startup.cmd
>
> The issue I face is that the command script I put onto
> C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start
> Menu\Programs\Startup\startup.cmd is deleted during the "sysprep"
> process, as part of the Windows template creation on CloudStack as per:
>
> http://docs.cloudstack.apache.org/projects/cloudstack-
> administration/en/4.8/templates/_create_windows.html
>
> Is there a way I can put the startup.cmd file somewhere else so that it
> will not be deleted during the sysprep process but will still be executed
> when the VM boots up or when the Administrator user logs in? Or is there
> any better approach on how to automate the data-disk initialization?
>
> Any advice is highly appreciated.
>
> Looking forward to your reply, thank you.
>
> Cheers.
>

Automatically initialize data-disk on Windows Server 2016 OS template

[Cloud List]

Dear all,

I am using CloudStack 4.8.1 and KVM hypervisor. When creating my Windows
Server 2016 OS template, I tried to automate the initialization and
formatting of the data-disk so that it will automatically be put online,
initialized with MBR partition, formatted and assigned the E: drive letter,
without the user having to manually initialize the disk using Computer
Management > Disk Management.

I tried to do so by preparing 2 scripts:

1. A PowerShell script to automatically initialize and format all disks
with RAW partition, which I am taking from here:

https://blogs.technet.microsoft.com/heyscriptingguy/2013/05/29/use-powershell-to-initialize-raw-disks-and-to-partition-and-format-volumes/

I saved it on C:\script\InitializePartitionFormatNewDisks.ps1.

2. A command script to automatically run the above PowerShell script when
the Administrator user logs in. I follow the instructions on below URL:

https://stackoverflow.com/questions/20575257/how-to-run-powershell-script-when-computer-starts

to create a command script to set PowerShell execution policy to
"unrestricted" and then run the script.

===
PowerShell -Command "Set-ExecutionPolicy Unrestricted" >>
"%TEMP%\StartupLog.txt" 2>&1 PowerShell
C:\script\InitializePartitionFormatNewDisks.ps1 >> "%TEMP%\StartupLog.txt"
2>&1
===

I saved the command script onto:
C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start
Menu\Programs\Startup\startup.cmd

The issue I face is that the command script I put onto
C:\Users\Administrator\AppData\Roaming\Microsoft\Windows\Start
Menu\Programs\Startup\startup.cmd is deleted during the "sysprep" process,
as part of the Windows template creation on CloudStack as per:

http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.8/templates/_create_windows.html

Is there a way I can put the startup.cmd file somewhere else so that it
will not be deleted during the sysprep process but will still be executed
when the VM boots up or when the Administrator user logs in? Or is there
any better approach on how to automate the data-disk initialization?

Any advice is highly appreciated.

Looking forward to your reply, thank you.

Cheers.

Re: Creating Windows Server 2016 template for CloudStack

[Cloud List]

Hi Todd,

Just a quick update -- below solution works! Thanks for pointing me to the
right direction.

Cheers.

-ip-


On Mon, Aug 14, 2017 at 8:50 AM, Cloud List <cloud-l...@sg.or.id> wrote:

> Thanks Todd.
>
> I found out that the correct syntax is:
>
> cd c:\windows\system32\sysprep
> sysprep.exe /oobe /generalize /shutdown /unattend:c:\windows\system32\
> sysprep\unattend.xml
>
> Trying this now and will share the outcome shortly.
>
> Thank you for your advice.
>
> Cheers.
>
> -ip-
>
>
> On Fri, Aug 11, 2017 at 6:21 PM, Todd Pigram <t...@toddpigram.com> wrote:
>
>> You need to specify the xml in the command.
>>
>> sysprep.exe /oobe /generalize /shutdown c:\system32\sysprep\unattend.xml.
>>
>> On w2016, its Autounattend.xml for the file name.
>>
>> On Fri, Aug 11, 2017 at 5:28 AM, Cloud List <cloud-l...@sg.or.id> wrote:
>>
>> > Dear all,
>> >
>> > I am using CloudStack 4.8 and KVM hypervisor, andI tried creating a
>> Windows
>> > Server 2016 OS template using sysprep following the instruction on the
>> > "System Preparation for Windows Server 2008 R2" section of the
>> > documentation at below URL:
>> >
>> > http://docs.cloudstack.apache.org/projects/cloudstack-
>> > administration/en/4.8/templates/_create_windows.html
>> >
>> > However, after the template is created and I tried to spin off a new VM
>> > using the template, the console is still prompting the Language and
>> > Country/Region selection, EULA and change password selection when
>> > booting-up. It seems that the sysprep process to automate those steps
>> > didn't work.
>> >
>> > I did saved the unattend.xml file into the C:\Windows\System32\sysprep
>> > folder and run "sysprep.exe /oobe /generalize /shutdown" before creating
>> > the template from the main VM.
>> >
>> > Did I miss anything, or is the sysprep process for Windows Server 2016
>> is
>> > different from 2008? The same method worked for 2012 last time.
>> >
>> > Any advice is appreciated, thank you.
>> >
>> > Cheers.
>> >
>> > -ip-
>> >
>>
>>
>>
>> --
>>
>>
>> Todd Pigram
>> http://about.me/ToddPigram
>> www.linkedin.com/in/toddpigram/
>> @pigram86 on twitter
>> https://plus.google.com/+ToddPigram86
>> Mobile - 216-224-5769
>>
>> PGP Public Key
>> <http://pgp.mit.edu/pks/lookup?op=get=0x96B7B0F0C55933BB>
>>
>
>

Creating Windows Server 2016 template for CloudStack

[Cloud List]

Dear all,

I am using CloudStack 4.8 and KVM hypervisor, andI tried creating a Windows
Server 2016 OS template using sysprep following the instruction on the
"System Preparation for Windows Server 2008 R2" section of the
documentation at below URL:

http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.8/templates/_create_windows.html

However, after the template is created and I tried to spin off a new VM
using the template, the console is still prompting the Language and
Country/Region selection, EULA and change password selection when
booting-up. It seems that the sysprep process to automate those steps
didn't work.

I did saved the unattend.xml file into the C:\Windows\System32\sysprep
folder and run "sysprep.exe /oobe /generalize /shutdown" before creating
the template from the main VM.

Did I miss anything, or is the sysprep process for Windows Server 2016 is
different from 2008? The same method worked for 2012 last time.

Any advice is appreciated, thank you.

Cheers.

-ip-

Re: Windows Server 2016 - unable to boot ISO for installation

[Cloud List]

Hi Lucian and all,

Just a quick update, the problem was resolved after I added below lines
into the host's agent.properties:

===
guest.cpu.mode=custom
guest.cpu.model=core2duo
===

and restarted CloudStack agent. The VM is now able to boot-up from the
Windows Server 2016 ISO without experiencing the BSOD issue.

Many thanks for all the help in pointing me to the right direction! :)

Cheers.

-ip-


On Thu, Aug 3, 2017 at 12:04 PM, Cloud List <cloud-l...@sg.or.id> wrote:

> Hi Lucian,
>
> Good day to you, and thank you for your reply.
>
> From what I understand, once the hypervisor is managed by CloudStack
> (using cloudstack-agent), we shouldn't use virt-manager to create VMs and
> all VMs will have to be created through CloudStack, correct?
>
> I read further and based on below documentation:
>
> https://social.technet.microsoft.com/Forums/en-US/695c8997-52cf-4c30-a3f7-
> f26a40dc703a/failed-install-of-build-10041-in-the-kvm-
> virtual-machine-system-thread-exception-not-handled?forum=
> WinPreview2014Setup
>
> I am using Sandy Bridge CPU and based on the above, it seems that we can
> resolve the problem by setting the CPU type as follows: 'core2duo'.
>
> The question is how to change it through CloudStack, can I confirm that we
> can do so by following the "Configure CPU model for KVM guest (Optional)"
> instruction at below URL?
>
> http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.8/
> hypervisor/kvm.html
>
> Based on the above instruction, I would need to add some lines on
> agent.properties file as follows:
>
> ===
> guest.cpu.mode=custom
> guest.cpu.model=core2duo
> ===
>
> and then restart CloudStack agent for the changes to take affect. Is it
> correct? Please correct me if I'm wrong.
>
> Looking forward to your reply, thank you.
>
> Cheers.
>
> -ip-
>
>
>
>
>
> On Tue, Aug 1, 2017 at 11:26 PM, Nux! <n...@li.nux.ro> wrote:
>
>> Hello,
>>
>> I am not in a position to test, but it should be fairly easy to; just
>> power up virt-manager on the said Ubuntu hypervisor and try to create a
>> Windows 2016 VM with  
>>
>> Let us know how it went :)
>>
>> --
>> Sent from the Delta quadrant using Borg technology!
>>
>> Nux!
>> www.nux.ro
>>
>> - Original Message -
>> > From: "Cloud List" <cloud-l...@sg.or.id>
>> > To: "users" <us...@cloudstack.apache.org>
>> > Cc: "dev" <dev@cloudstack.apache.org>
>> > Sent: Tuesday, 1 August, 2017 04:11:06
>> > Subject: Re: Windows Server 2016 - unable to boot ISO for installation
>>
>> > Hi Lucian,
>> >
>> > Nice to hear from you again. Below is the KVM version (QEMU version, to
>> be
>> > exact) I am using:
>> >
>> > # kvm --version
>> > QEMU emulator version 2.4.1, Copyright (c) 2003-2008 Fabrice Bellard
>> >
>> > I am using Ubuntu instead of CentOS for the KVM host, may I know if this
>> > affects KVM running on Ubuntu as well? I am using Intel Xeon E5 series
>> for
>> > the hypervisor hosts, not too sure if it's supported and part of the
>> Xeon E
>> > series on the list?
>> >
>> > Looking forward to your reply, thank you.
>> >
>> > Cheers.
>> >
>> > -ip-
>> >
>> >
>> > On Tue, Aug 1, 2017 at 12:45 AM, Nux! <n...@li.nux.ro> wrote:
>> >
>> >> It could be this
>> >> https://access.redhat.com/documentation/en-US/Red_Hat_
>> >> Enterprise_Linux/6/html/6.8_Release_Notes/known_issues_virtu
>> alization.html
>> >>
>> >> --
>> >> Sent from the Delta quadrant using Borg technology!
>> >>
>> >> Nux!
>> >> www.nux.ro
>> >>
>> >> - Original Message -
>> >> > From: "Nux!" <n...@li.nux.ro>
>> >> > To: "users" <us...@cloudstack.apache.org>
>> >> > Cc: "dev" <dev@cloudstack.apache.org>
>> >> > Sent: Monday, 31 July, 2017 16:54:51
>> >> > Subject: Re: Windows Server 2016 - unable to boot ISO for
>> installation
>> >>
>> >> > What version of KVM?
>> >> >
>> >> > --
>> >> > Sent from the Delta quadrant using Borg technology!
>> >> >
>> >> > Nux!
>> >> > www.nux.ro
>> >> >
>> >> > - Original Message -
>> >> >> From: "Clou

Re: Windows Server 2016 - unable to boot ISO for installation

[Cloud List]

Hi Lucian,

Good day to you, and thank you for your reply.

>From what I understand, once the hypervisor is managed by CloudStack (using
cloudstack-agent), we shouldn't use virt-manager to create VMs and all VMs
will have to be created through CloudStack, correct?

I read further and based on below documentation:

https://social.technet.microsoft.com/Forums/en-US/695c8997-52cf-4c30-a3f7-f26a40dc703a/failed-install-of-build-10041-in-the-kvm-virtual-machine-system-thread-exception-not-handled?forum=WinPreview2014Setup

I am using Sandy Bridge CPU and based on the above, it seems that we can
resolve the problem by setting the CPU type as follows: 'core2duo'.

The question is how to change it through CloudStack, can I confirm that we
can do so by following the "Configure CPU model for KVM guest (Optional)"
instruction at below URL?

http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.8/hypervisor/kvm.html

Based on the above instruction, I would need to add some lines on
agent.properties file as follows:

===
guest.cpu.mode=custom
guest.cpu.model=core2duo
===

and then restart CloudStack agent for the changes to take affect. Is it
correct? Please correct me if I'm wrong.

Looking forward to your reply, thank you.

Cheers.

-ip-




On Tue, Aug 1, 2017 at 11:26 PM, Nux! <n...@li.nux.ro> wrote:

> Hello,
>
> I am not in a position to test, but it should be fairly easy to; just
> power up virt-manager on the said Ubuntu hypervisor and try to create a
> Windows 2016 VM with  
>
> Let us know how it went :)
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> - Original Message -
> > From: "Cloud List" <cloud-l...@sg.or.id>
> > To: "users" <us...@cloudstack.apache.org>
> > Cc: "dev" <dev@cloudstack.apache.org>
> > Sent: Tuesday, 1 August, 2017 04:11:06
> > Subject: Re: Windows Server 2016 - unable to boot ISO for installation
>
> > Hi Lucian,
> >
> > Nice to hear from you again. Below is the KVM version (QEMU version, to
> be
> > exact) I am using:
> >
> > # kvm --version
> > QEMU emulator version 2.4.1, Copyright (c) 2003-2008 Fabrice Bellard
> >
> > I am using Ubuntu instead of CentOS for the KVM host, may I know if this
> > affects KVM running on Ubuntu as well? I am using Intel Xeon E5 series
> for
> > the hypervisor hosts, not too sure if it's supported and part of the
> Xeon E
> > series on the list?
> >
> > Looking forward to your reply, thank you.
> >
> > Cheers.
> >
> > -ip-
> >
> >
> > On Tue, Aug 1, 2017 at 12:45 AM, Nux! <n...@li.nux.ro> wrote:
> >
> >> It could be this
> >> https://access.redhat.com/documentation/en-US/Red_Hat_
> >> Enterprise_Linux/6/html/6.8_Release_Notes/known_issues_
> virtualization.html
> >>
> >> --
> >> Sent from the Delta quadrant using Borg technology!
> >>
> >> Nux!
> >> www.nux.ro
> >>
> >> - Original Message -
> >> > From: "Nux!" <n...@li.nux.ro>
> >> > To: "users" <us...@cloudstack.apache.org>
> >> > Cc: "dev" <dev@cloudstack.apache.org>
> >> > Sent: Monday, 31 July, 2017 16:54:51
> >> > Subject: Re: Windows Server 2016 - unable to boot ISO for installation
> >>
> >> > What version of KVM?
> >> >
> >> > --
> >> > Sent from the Delta quadrant using Borg technology!
> >> >
> >> > Nux!
> >> > www.nux.ro
> >> >
> >> > - Original Message -
> >> >> From: "Cloud List" <cloud-l...@sg.or.id>
> >> >> To: "users" <us...@cloudstack.apache.org>
> >> >> Cc: "dev" <dev@cloudstack.apache.org>
> >> >> Sent: Monday, 31 July, 2017 03:31:45
> >> >> Subject: Windows Server 2016 - unable to boot ISO for installation
> >> >
> >> >> Dear all,
> >> >>
> >> >> I am using CloudStack 4.8.1.1 with KVM hypervisor. I have downloaded
> >> >> Windows Server 2016 ISO but when I tried to boot up the ISO for
> >> >> installation and template creation, it immediately goes into Blue
> >> Screen /
> >> >> BSOD with below error messages:
> >> >>
> >> >> ===
> >> >> Your PC ran into a problem and needs a restart. We'll restart for
> you.
> >> >>
> >> >> For more information about this issue and possible fixes, visit
> 

Re: Windows Server 2016 - unable to boot ISO for installation

[Cloud List]

Hi Lucian,

Nice to hear from you again. Below is the KVM version (QEMU version, to be
exact) I am using:

# kvm --version
QEMU emulator version 2.4.1, Copyright (c) 2003-2008 Fabrice Bellard

I am using Ubuntu instead of CentOS for the KVM host, may I know if this
affects KVM running on Ubuntu as well? I am using Intel Xeon E5 series for
the hypervisor hosts, not too sure if it's supported and part of the Xeon E
series on the list?

Looking forward to your reply, thank you.

Cheers.

-ip-


On Tue, Aug 1, 2017 at 12:45 AM, Nux! <n...@li.nux.ro> wrote:

> It could be this
> https://access.redhat.com/documentation/en-US/Red_Hat_
> Enterprise_Linux/6/html/6.8_Release_Notes/known_issues_virtualization.html
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> - Original Message -
> > From: "Nux!" <n...@li.nux.ro>
> > To: "users" <us...@cloudstack.apache.org>
> > Cc: "dev" <dev@cloudstack.apache.org>
> > Sent: Monday, 31 July, 2017 16:54:51
> > Subject: Re: Windows Server 2016 - unable to boot ISO for installation
>
> > What version of KVM?
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> > - Original Message -
> >> From: "Cloud List" <cloud-l...@sg.or.id>
> >> To: "users" <us...@cloudstack.apache.org>
> >> Cc: "dev" <dev@cloudstack.apache.org>
> >> Sent: Monday, 31 July, 2017 03:31:45
> >> Subject: Windows Server 2016 - unable to boot ISO for installation
> >
> >> Dear all,
> >>
> >> I am using CloudStack 4.8.1.1 with KVM hypervisor. I have downloaded
> >> Windows Server 2016 ISO but when I tried to boot up the ISO for
> >> installation and template creation, it immediately goes into Blue
> Screen /
> >> BSOD with below error messages:
> >>
> >> ===
> >> Your PC ran into a problem and needs a restart. We'll restart for you.
> >>
> >> For more information about this issue and possible fixes, visit
> >> http://windows.com/stopcode
> >>
> >> If you call a support person, give them this info:
> >> Stop Code: SYSTEM THREAD EXCEPTION NOT HANDLED
> >> ===
> >>
> >> and then it will go into restart, hit with the same error and keep on
> >> restarting on an infinite loop.
> >>
> >> I browse around and it seems the error is related to outdated or wrong
> >> driver (most probably display driver) or due to resources issues. Have
> >> tried to increase the resources to 8 GB RAM, 4 CPU cores and 60 GB of
> disk
> >> space, and the problem still persists. Have tried different OS types:
> >> "Other PV (64-bit)", "Other (64-bit)" and "Windows Server 2012 R2
> >> (64-bit"), and problem still persists.
> >>
> >> I understand that Windows Server 2016 should work and some people have
> >> managed to get Windows Server 2016 VM and template up and running (see
> >> below email thread for example) -- anyone encountered similar issues and
> >> can provide advise?
> >>
> >> Any advice is greatly appreciated.
> >>
> >> Thanks in advance.
> >>
> >> Cheers.
> >>
> >>
> >>
> >> -- Forwarded message --
> >> From: Andrija Panic <andrija.pa...@gmail.com>
> >> Date: Thu, May 4, 2017 at 4:49 AM
> >> Subject: Re: OS Type: Windows Server 2016 - CS 4.9.2
> >> To: "us...@cloudstack.apache.org" <us...@cloudstack.apache.org>
> >>
> >>
> >> We used to provide 2016 template for our users, with VirtIO preinstalled
> >> inside OS (during building template), and then we simply choose OS type
> of
> >> "Windows PV" - not related to Xen (we use KVM), but just FYI as
> parallel.
> >>
> >> On 12 April 2017 at 15:57, Pierre-Luc Dion <pd...@cloudops.com> wrote:
> >>
> >>> Hi David,
> >>>
> >>> When using pre xenserver 7, CloudStack deploy Windows 2016 vm as "Other
> >> (64
> >>> bit)".
> >>> I've made some test with XenServer 7.1  and Windows 2016 are default
> HVM
> >>> when using the new xenserver template  for Windows 2016.
> >>> My understanding is that if you run a VM in XenServer as HVM and you
> have
> >>> xentools installed and up-to-date, your VM run in PVHVM mode which
> seems
> >> to

Windows Server 2016 - unable to boot ISO for installation

[Cloud List]

Dear all,

I am using CloudStack 4.8.1.1 with KVM hypervisor. I have downloaded
Windows Server 2016 ISO but when I tried to boot up the ISO for
installation and template creation, it immediately goes into Blue Screen /
BSOD with below error messages:

===
Your PC ran into a problem and needs a restart. We'll restart for you.

For more information about this issue and possible fixes, visit
http://windows.com/stopcode

If you call a support person, give them this info:
Stop Code: SYSTEM THREAD EXCEPTION NOT HANDLED
===

and then it will go into restart, hit with the same error and keep on
restarting on an infinite loop.

I browse around and it seems the error is related to outdated or wrong
driver (most probably display driver) or due to resources issues. Have
tried to increase the resources to 8 GB RAM, 4 CPU cores and 60 GB of disk
space, and the problem still persists. Have tried different OS types:
"Other PV (64-bit)", "Other (64-bit)" and "Windows Server 2012 R2
(64-bit"), and problem still persists.

I understand that Windows Server 2016 should work and some people have
managed to get Windows Server 2016 VM and template up and running (see
below email thread for example) -- anyone encountered similar issues and
can provide advise?

Any advice is greatly appreciated.

Thanks in advance.

Cheers.



-- Forwarded message --
From: Andrija Panic 
Date: Thu, May 4, 2017 at 4:49 AM
Subject: Re: OS Type: Windows Server 2016 - CS 4.9.2
To: "us...@cloudstack.apache.org" 


We used to provide 2016 template for our users, with VirtIO preinstalled
inside OS (during building template), and then we simply choose OS type of
"Windows PV" - not related to Xen (we use KVM), but just FYI as parallel.

On 12 April 2017 at 15:57, Pierre-Luc Dion  wrote:

> Hi David,
>
> When using pre xenserver 7, CloudStack deploy Windows 2016 vm as "Other
(64
> bit)".
> I've made some test with XenServer 7.1  and Windows 2016 are default HVM
> when using the new xenserver template  for Windows 2016.
> My understanding is that if you run a VM in XenServer as HVM and you have
> xentools installed and up-to-date, your VM run in PVHVM mode which seems
to
> be the best of both world.
>
> The coming release of ACS (4.10) should have updated ostype so windows
2016
> wouid use Windows 2016 template in xenserver 7+ instead of "Other (64
> bit)".
>
>
>
>
>
> On Wed, Apr 12, 2017 at 4:14 AM, David Amorín <
> david.amo...@adderglobal.com>
> wrote:
>
> > Hi all,
> > In our test environment based on CS 4.9.2 and XS 7.0, we are deploying
> VMs
> > with Windows Server 2016 as guest installation, using OS Type: "Other
> > (64-bit)" and so far everything is working fine. We understand that
using
> > this option we are deploying full virtualized (HVM)
> >
> >
> > Do you know if there is any issue (error, performance, etc.) for using
> > "Other (64-bit))=HVM for Windows Server 2016 VMs?
> >
> >
> > Do you know when will be available Windows Server 2016 as OS Type?
> >
> >
> > Thanks
> >
> >
> > David
> >
> >
>



--

Andrija Panić

Maximum number of site-to-site VPNs on Cloudstack

[Cloud List]

Dear all,

On below documentation:

http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.8/networking/site_to_site_vpn.html

It says that CloudStack supports creating up to 8 VPN connections. May I
know if it's 8 VPN connections per VPC, or globally within a Cloudstack
cluster or zone?

Looking forward to your reply, thank you.

Cheers.

Re: ACS 4.8.1: Virtual router - /var/log always full

[Cloud List]

Thanks Wei for your tips, greatly appreciated. Will do what you have
suggested.

Hi Will, we are using KVM, so not too sure if it's related to the bug you
just found.

Thank you.


On Mon, Nov 28, 2016 at 2:02 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote:

> This is because the free space in systemvm template is not big, and
> logrotate rules are not wise enough to hold the logs.
>
> you can remove the "delaycompress" and reduce the "size" in
> systemvm/patches/debian/config/etc/logrotate.d/rsyslog
> (/etc/logrotate.d/rsyslog in VR/systemvms), then restart logrotate to fix
> it.
>
> -Wei
>
> 2016-11-28 5:45 GMT+01:00 Will Stevens <wstev...@cloudops.com>:
>
> > Are you using XenServer?  We recently found a bug in XenServer which has
> > similar characteristics.  I can put together details of the problem if
> > yes...
> >
> > Cheers,
> >
> > Will
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > <https://goo.gl/NYZ8KK>
> >
> > On Sun, Nov 27, 2016 at 11:36 PM, Cloud List <cloud-l...@sg.or.id>
> wrote:
> >
> > > Dear all,
> > >
> > > Please ignore this, managed to find the reason - paswd_server_ip
> process
> > is
> > > holding to a log file which has been deleted earlier to fix the disk
> > space
> > > issue. Restarted the process and the disk space is fine now.
> > >
> > > ===
> > > passwd_se  2199   root1w  REG 254,10
> > > 171802407 15 /var/log/cloud.log (deleted)
> > > passwd_se  2199   root2w  REG 254,10
> > > 171802407 15 /var/log/cloud.log (deleted)
> > > passwd_se  2199   root3w  REG 254,10
> > > 171802407 15 /var/log/cloud.log (deleted)
> > > python 2202   root3w  REG 254,10
> > > 171802407     15 /var/log/cloud.log (deleted)
> > >
> > >  2199 ?S  0:00 /bin/bash /opt/cloud/bin/passwd_server_ip
> > > X.X.X.2 dummy
> > >  2202 ?S  7:13 python /opt/cloud/bin/passwd_server_ip.py
> > > X.X.X.2
> > > ===
> > >
> > > Thank you.
> > >
> > >
> > > On Mon, Nov 28, 2016 at 11:46 AM, Cloud List <cloud-l...@sg.or.id>
> > wrote:
> > >
> > > > Dear all,
> > > >
> > > > After upgrading to ACS 4.8.1, one of our virtual router's /var/log
> > > > partition is always full and used up quite fast. This caused the VR
> not
> > > > able to serve DHCP and password requests from VM.
> > > >
> > > > root@r-4155-VM:/var/log# df -h
> > > > Filesystem  Size  Used
> > Avail
> > > > Use% Mounted on
> > > > rootfs  461M  158M
> > 280M
> > > > 37% /
> > > > udev 10M 0
> > > > 10M   0% /dev
> > > > tmpfs25M  236K
> > > > 25M   1% /run
> > > > /dev/disk/by-uuid/30c81d3d-ee9f-4a88-81c1-5f349b22ba1d  461M  158M
> > 280M
> > > > 37% /
> > > > tmpfs   5.0M 0
> > > > 5.0M   0% /run/lock
> > > > tmpfs   157M 0
> > > > 157M   0% /run/shm
> > > > /dev/vda173M   23M
> >  47M
> > > > 33% /boot
> > > > /dev/vda692M  5.6M
> > > > 81M   7% /home
> > > > /dev/vda8   184M  6.2M
> > > > 169M   4% /opt
> > > > /dev/vda11   92M  5.6M
> > > > 81M   7% /tmp
> > > > /dev/vda7   751M  493M
> > 219M
> > > > 70% /usr
> > > > /dev/vda9   563M  282M
> > 252M
> > > > 53% /var
> > > > /dev/vda10  184M  176M
> >  0
> > > > 100% /var/log
> > > >
> > > > Even after rotating and clearing the logs, the usage of /var/log is
> > only
> > > > 4.7M so I am not too sure where is the 176M coming from.
> > > >
> > > > root@r-4155-VM:/var/log# du -h
> > > > 1.0K./samba
> > > > 3.8M./sysstat
> > > > 68K ./apt
> > > > 7.0K./apache2
> > > > 3.0K./fsck
> > > > 317K./installer/cdebconf
> > > > 809K./installer
> > > > 1.0K./news
> > > > 12K ./lost+found
> > > > 1.0K./ntpstats
> > > > 4.7M.
> > > >
> > > > /dev/vda10  184M  175M
> > 475K
> > > > 100% /var/log
> > > >
> > > > I would need to clear the logs and do a "service dnsmasq restart"
> > > > regularly to make the VR functioning again, which is quite
> troublesome.
> > > >
> > > > Any advice is greatly appreciated.
> > > >
> > > > Looking forward to your reply, thank you.
> > > >
> > > > Cheers.
> > > >
> > >
> >
>

Re: ACS 4.8.1: Virtual router - /var/log always full

[Cloud List]

Dear all,

Please ignore this, managed to find the reason - paswd_server_ip process is
holding to a log file which has been deleted earlier to fix the disk space
issue. Restarted the process and the disk space is fine now.

===
passwd_se  2199   root1w  REG 254,10
171802407 15 /var/log/cloud.log (deleted)
passwd_se  2199   root2w  REG 254,10
171802407 15 /var/log/cloud.log (deleted)
passwd_se  2199   root3w  REG 254,10
171802407 15 /var/log/cloud.log (deleted)
python 2202   root3w  REG 254,10
171802407 15 /var/log/cloud.log (deleted)

 2199 ?S  0:00 /bin/bash /opt/cloud/bin/passwd_server_ip
X.X.X.2 dummy
 2202 ?S  7:13 python /opt/cloud/bin/passwd_server_ip.py X.X.X.2
===

Thank you.


On Mon, Nov 28, 2016 at 11:46 AM, Cloud List <cloud-l...@sg.or.id> wrote:

> Dear all,
>
> After upgrading to ACS 4.8.1, one of our virtual router's /var/log
> partition is always full and used up quite fast. This caused the VR not
> able to serve DHCP and password requests from VM.
>
> root@r-4155-VM:/var/log# df -h
> Filesystem  Size  Used Avail
> Use% Mounted on
> rootfs  461M  158M  280M
> 37% /
> udev 10M 0
> 10M   0% /dev
> tmpfs25M  236K
> 25M   1% /run
> /dev/disk/by-uuid/30c81d3d-ee9f-4a88-81c1-5f349b22ba1d  461M  158M  280M
> 37% /
> tmpfs   5.0M 0
> 5.0M   0% /run/lock
> tmpfs   157M 0
> 157M   0% /run/shm
> /dev/vda173M   23M   47M
> 33% /boot
> /dev/vda692M  5.6M
> 81M   7% /home
> /dev/vda8   184M  6.2M
> 169M   4% /opt
> /dev/vda11   92M  5.6M
> 81M   7% /tmp
> /dev/vda7   751M  493M  219M
> 70% /usr
> /dev/vda9   563M  282M  252M
> 53% /var
> /dev/vda10  184M  176M 0
> 100% /var/log
>
> Even after rotating and clearing the logs, the usage of /var/log is only
> 4.7M so I am not too sure where is the 176M coming from.
>
> root@r-4155-VM:/var/log# du -h
> 1.0K./samba
> 3.8M./sysstat
> 68K ./apt
> 7.0K./apache2
> 3.0K./fsck
> 317K./installer/cdebconf
> 809K./installer
> 1.0K./news
> 12K ./lost+found
> 1.0K./ntpstats
> 4.7M.
>
> /dev/vda10  184M  175M  475K
> 100% /var/log
>
> I would need to clear the logs and do a "service dnsmasq restart"
> regularly to make the VR functioning again, which is quite troublesome.
>
> Any advice is greatly appreciated.
>
> Looking forward to your reply, thank you.
>
> Cheers.
>

ACS 4.8.1: Virtual router - /var/log always full

[Cloud List]

Dear all,

After upgrading to ACS 4.8.1, one of our virtual router's /var/log
partition is always full and used up quite fast. This caused the VR not
able to serve DHCP and password requests from VM.

root@r-4155-VM:/var/log# df -h
Filesystem  Size  Used Avail
Use% Mounted on
rootfs  461M  158M  280M
37% /
udev 10M 0   10M
0% /dev
tmpfs25M  236K   25M
1% /run
/dev/disk/by-uuid/30c81d3d-ee9f-4a88-81c1-5f349b22ba1d  461M  158M  280M
37% /
tmpfs   5.0M 0  5.0M
0% /run/lock
tmpfs   157M 0  157M
0% /run/shm
/dev/vda173M   23M   47M
33% /boot
/dev/vda692M  5.6M   81M
7% /home
/dev/vda8   184M  6.2M  169M
4% /opt
/dev/vda11   92M  5.6M   81M
7% /tmp
/dev/vda7   751M  493M  219M
70% /usr
/dev/vda9   563M  282M  252M
53% /var
/dev/vda10  184M  176M 0
100% /var/log

Even after rotating and clearing the logs, the usage of /var/log is only
4.7M so I am not too sure where is the 176M coming from.

root@r-4155-VM:/var/log# du -h
1.0K./samba
3.8M./sysstat
68K ./apt
7.0K./apache2
3.0K./fsck
317K./installer/cdebconf
809K./installer
1.0K./news
12K ./lost+found
1.0K./ntpstats
4.7M.

/dev/vda10  184M  175M  475K
100% /var/log

I would need to clear the logs and do a "service dnsmasq restart" regularly
to make the VR functioning again, which is quite troublesome.

Any advice is greatly appreciated.

Looking forward to your reply, thank you.

Cheers.

Re: ACS - Some VMs unable to get DHCP IP from VR

[Cloud List]

Hi Wei,

Just to let you know that the workaround you have suggested fixes the
problem.

/bin/bash /opt/cloud/bin/passwd_server_ip X.X.203.2 &

Many thanks! Greatly appreciated.

Cheers.


On Fri, Nov 11, 2016 at 3:49 AM, Cloud List <cloud-l...@sg.or.id> wrote:

> Hi Wei,
>
> Thanks for your reply.
>
> I ran the command and it seems to be running very long without any
> response. Does it take time to execute, or is it supposed to run in
> background? e.g. shall I run with & option instead?
>
> /bin/bash /opt/cloud/bin/passwd_server_ip X.X.203.2 &
>
> Here's the result:
>
> root@r-4155-VM:/opt/cloud/bin# /bin/bash /opt/cloud/bin/passwd_server_ip
> X.X.203.2
> [no response]
>
> Because when I grep the process using ps aux, I can see that it seems to
> be running as daemon for the first subnet:
>
>  2199 ?S  0:00 /bin/bash /opt/cloud/bin/passwd_server_ip
> X.X.202.2 dummy
>  2202 ?S  0:12 python /opt/cloud/bin/passwd_server_ip.py
> X.X.202.2
>
> Do I also need to add the "dummy" option for the second subnet onwards?
>
> Looking forward to your reply, thank you.
>
> Cheers.
>
> On Thu, Nov 10, 2016 at 9:28 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote:
>
>> Sorry what I said in previous reply is wrong. The 80 port /apache2 are
>> used
>> for ssh-key server, not password server.
>>
>> For password server, there should be a passwd_server_ip.py running with
>> corresponding ip.
>> The password server will listen the 8080 port, and cache the vm password
>> in
>> /var/cache/cloud/passwords-X.X.X.X
>>
>> In a word, you need to run the following command:
>> /bin/bash /opt/cloud/bin/passwd_server_ip X.X.203.2 (the vr ip in second
>> ip
>> subnet)
>>
>> -Wei
>>
>> 2016-11-10 13:07 GMT+01:00 Cloud List <cloud-l...@sg.or.id>:
>>
>> > Hi Wei,
>> >
>> > Thanks for your reply.
>> >
>> > You are right, when I tried to access the other subnet IP on port 80, it
>> > got connection refused. I have modified the Apache configuration to add
>> the
>> > port and virtualhost configuration and we are now able to access the
>> other
>> > subnet IP on port 80.
>> >
>> > root@test-40g-root-disk:~# telnet X.X.203.2 80
>> > Trying X.X.203.2...
>> > Connected to X.X.203.2.
>> > Escape character is '^]'.
>> > ^]
>> > telnet> quit
>> > Connection closed.
>> >
>> > However, password reset still doesn't work. Not too sure what's wrong.
>> >
>> > I looked at the python script on /opt/cloud/bin/passwd_server_ip.py, it
>> > seems to be looking into just one file called
>> > /var/cache/cloud/passwords-X.X.202.2 although I see some IPs from other
>> > subnets as well inside the file. Is there something we need to modify on
>> > the area as well?
>> >
>> > Any further advice is greatly appreciated.
>> >
>> > Thank you.
>> >
>> >
>> >
>>
>
>

Re: ACS - Some VMs unable to get DHCP IP from VR

[Cloud List]

Hi Wei,

Thanks for your reply.

I ran the command and it seems to be running very long without any
response. Does it take time to execute, or is it supposed to run in
background? e.g. shall I run with & option instead?

/bin/bash /opt/cloud/bin/passwd_server_ip X.X.203.2 &

Here's the result:

root@r-4155-VM:/opt/cloud/bin# /bin/bash /opt/cloud/bin/passwd_server_ip
X.X.203.2
[no response]

Because when I grep the process using ps aux, I can see that it seems to be
running as daemon for the first subnet:

 2199 ?S  0:00 /bin/bash /opt/cloud/bin/passwd_server_ip
X.X.202.2 dummy
 2202 ?S  0:12 python /opt/cloud/bin/passwd_server_ip.py
X.X.202.2

Do I also need to add the "dummy" option for the second subnet onwards?

Looking forward to your reply, thank you.

Cheers.

On Thu, Nov 10, 2016 at 9:28 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote:

> Sorry what I said in previous reply is wrong. The 80 port /apache2 are used
> for ssh-key server, not password server.
>
> For password server, there should be a passwd_server_ip.py running with
> corresponding ip.
> The password server will listen the 8080 port, and cache the vm password in
> /var/cache/cloud/passwords-X.X.X.X
>
> In a word, you need to run the following command:
> /bin/bash /opt/cloud/bin/passwd_server_ip X.X.203.2 (the vr ip in second
> ip
> subnet)
>
> -Wei
>
> 2016-11-10 13:07 GMT+01:00 Cloud List <cloud-l...@sg.or.id>:
>
> > Hi Wei,
> >
> > Thanks for your reply.
> >
> > You are right, when I tried to access the other subnet IP on port 80, it
> > got connection refused. I have modified the Apache configuration to add
> the
> > port and virtualhost configuration and we are now able to access the
> other
> > subnet IP on port 80.
> >
> > root@test-40g-root-disk:~# telnet X.X.203.2 80
> > Trying X.X.203.2...
> > Connected to X.X.203.2.
> > Escape character is '^]'.
> > ^]
> > telnet> quit
> > Connection closed.
> >
> > However, password reset still doesn't work. Not too sure what's wrong.
> >
> > I looked at the python script on /opt/cloud/bin/passwd_server_ip.py, it
> > seems to be looking into just one file called
> > /var/cache/cloud/passwords-X.X.202.2 although I see some IPs from other
> > subnets as well inside the file. Is there something we need to modify on
> > the area as well?
> >
> > Any further advice is greatly appreciated.
> >
> > Thank you.
> >
> >
> >
>

Re: ACS - Some VMs unable to get DHCP IP from VR

[Cloud List]

Hi Wei,

Thanks for your reply.

You are right, when I tried to access the other subnet IP on port 80, it
got connection refused. I have modified the Apache configuration to add the
port and virtualhost configuration and we are now able to access the other
subnet IP on port 80.

root@test-40g-root-disk:~# telnet X.X.203.2 80
Trying X.X.203.2...
Connected to X.X.203.2.
Escape character is '^]'.
^]
telnet> quit
Connection closed.

However, password reset still doesn't work. Not too sure what's wrong.

I looked at the python script on /opt/cloud/bin/passwd_server_ip.py, it
seems to be looking into just one file called
/var/cache/cloud/passwords-X.X.202.2 although I see some IPs from other
subnets as well inside the file. Is there something we need to modify on
the area as well?

Any further advice is greatly appreciated.

Thank you.


On Thu, Nov 10, 2016 at 7:46 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote:

> The password server  in VR is implemented by apache2.
> you can have a look at configuration files in /etc/apache2/ (ports.conf,
> sites-available/default, etc)
>
> you will notice that the server listens at the 80 port of default IP of VR.
> However, the VM in non-default ip subnets cannot reach the default IP.
>
> you need to create another configuration file in /etc/apache2/conf.d/ to
> support password servers on non-default ip subnets.
>
> -Wei
>
> 2016-11-10 12:23 GMT+01:00 Cloud List <cloud-l...@sg.or.id>:
>
> > Hi Wei,
> >
> > In this case, dhcp works, and the server is up and running. But password
> > reset doesn't work. What could be the reason?
> >
> > Thanks.
> >
> >
>

Re: ACS - Some VMs unable to get DHCP IP from VR

[Cloud List]

Hi Wei,

In this case, dhcp works, and the server is up and running. But password
reset doesn't work. What could be the reason?

Thanks.

On Thu, Nov 10, 2016 at 7:01 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote:

> The password server depends on the dhcp services.
> The vm fetch the dhcp ip at first, then get the password from dhcp server
> (=password server).
> If dhcp does not work, then password server will not work as well.
>
> -Wei
>
> 2016-11-10 11:29 GMT+01:00 Cloud List <cloud-l...@sg.or.id>:
>
> > Hi Wei,
> >
> > Thanks for your suggestion, will test.
> >
> > I found another problem -- it seems that password reset function also
> works
> > only for VMs on the first subnet (X.X.202.*) but not on second subnet
> > (X.X.203.*) onwards, any file I can modify to add the additional subnets
> so
> > that password reset can work? Which service is doing the password reset
> on
> > VR and which files it read?
> >
> > Looking forward to your reply, thank you.
> >
> > Cheers.
> >
> >
> > On Wed, Nov 9, 2016 at 5:59 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote:
> >
> > > Hi,
> > >
> > > I meant some development in cloudstack to add new nic when add new ip
> > > range. it is not easy.
> > >
> > > You can have multiple shared networks untagged. As far as I see, the
> > shared
> > > networks can also have same vlan but with different subnets. I suggest
> > you
> > > to test this solution.
> > >
> > > -Wei
> > >
> > >
> > > 2016-11-09 4:24 GMT+01:00 Cloud List <cloud-l...@sg.or.id>:
> > >
> > > > Hi Wei,
> > > >
> > > > Any documentation on how to implement your suggestion -- adding a new
> > NIC
> > > > for new IP range, while leaving the shared network untagged? Does
> this
> > > mean
> > > > that the different IP subnets can still share the same VLAN and
> > broadcast
> > > > domain?
> > > >
> > > > Looking forward to your reply, thank you.
> > > >
> > > > Cheers.
> > > >
> > > > -ip-
> > > >
> > > >
> > > >
> > > >
> > >
> >
>

Re: ACS - Some VMs unable to get DHCP IP from VR

[Cloud List]

Hi Wei,

Thanks for your suggestion, will test.

I found another problem -- it seems that password reset function also works
only for VMs on the first subnet (X.X.202.*) but not on second subnet
(X.X.203.*) onwards, any file I can modify to add the additional subnets so
that password reset can work? Which service is doing the password reset on
VR and which files it read?

Looking forward to your reply, thank you.

Cheers.


On Wed, Nov 9, 2016 at 5:59 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote:

> Hi,
>
> I meant some development in cloudstack to add new nic when add new ip
> range. it is not easy.
>
> You can have multiple shared networks untagged. As far as I see, the shared
> networks can also have same vlan but with different subnets. I suggest you
> to test this solution.
>
> -Wei
>
>
> 2016-11-09 4:24 GMT+01:00 Cloud List <cloud-l...@sg.or.id>:
>
> > Hi Wei,
> >
> > Any documentation on how to implement your suggestion -- adding a new NIC
> > for new IP range, while leaving the shared network untagged? Does this
> mean
> > that the different IP subnets can still share the same VLAN and broadcast
> > domain?
> >
> > Looking forward to your reply, thank you.
> >
> > Cheers.
> >
> > -ip-
> >
> >
> >
> >
>

Re: ACS - Some VMs unable to get DHCP IP from VR

[Cloud List]

Hi Wei,

Adding new subnet on a a new shared network, meaning a new VLAN? But that
means the IPs will not be interchange-able between networks?

I have managed to find the workaround to the problem. On the VR, I only see
the IP address of the first subnet on the eth0 interface. The remaining
subnets are not there. This was not the case on ACS 4.2 where each of the
subnet's IP has "representation" on the VR's /etc/network/interface as
eth0's subinterfaces (eth0:0, eth0:1, etc).

This is what I did:

- Add the rest of the multiple subnets as eth0 subinterfaces on
/etc/network/interfaces:

iface eth0:0 inet static
  address X.X.203.2
  netmask 255.255.255.0

iface eth0:1 inet static
  address X.X.152.2
  netmask 255.255.255.0

iface eth0:2 inet static
  address X.X.153.2
  netmask 255.255.255.0

- Bring up the sub-interfaces:

ifup eth0:0
ifup eth0:1
ifup eth0:2

- Add below lines on /etc/dnsmasq.conf:

dhcp-range=X.X.203.1,static
dhcp-range=X.X.152.1,static
dhcp-range=X.X.153.1,static

- Add below files on /etc/dnsmasq.d folder:

cloud203.conf

dhcp-range=interface:eth0:0,set:interface-eth0:0,X.X.203.2,static
dhcp-option=tag:interface-eth0:0,15,cs1cloud.internal
dhcp-option=tag:interface-eth0:0,6,8.8.8.8,8.8.4.4
dhcp-option=tag:interface-eth0:0,3,X,X.203.1
dhcp-option=tag:interface-eth0:0,1,255.255.255.0

cloud152.conf, cloud153.conf etc

- Restart dnsmasq:

service dnsmasq restart

Do you think the above workaround will work and there will not be any side
effect? We understand that it means we have to re-apply the configuration
again in the event the VR needs to be re-created.

Looking forward to your reply, thank you.

Cheers.

-ip-



On Tue, Nov 8, 2016 at 4:17 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote:

> Hi,
>
> I think it is related to multiple ip ranges in a shared network.
> If I change the ip in /etc/dhcphosts.txt to another ip in other range, I
> got the same error log in dnsmasq.log
>
> We do not use multiple vlan or ip ranges in a shared network. I believe few
> of us have the same use case.
> If possible, it would be better to add the new ip range as a new shared
> network.
>
> -Wei
>
>
> 2016-11-07 21:35 GMT+01:00 Cloud List <cloud-l...@sg.or.id>:
>
> > Hi Chiradeep and Wei, thanks for your reply.
> >
> > Wei, here's the result you requested:
> >
> > root@r-4155-VM:/var/log2# cat /etc/dnsmasq.conf |grep -v "^#" |grep -v
> > "^$"
> > domain-needed
> > bogus-priv
> > resolv-file=/etc/dnsmasq-resolv.conf
> > local=/cs1cloud.internal/
> > except-interface=eth1
> > except-interface=eth2
> > except-interface=lo
> > no-dhcp-interface=eth1
> > no-dhcp-interface=eth2
> > expand-hosts
> > domain=cs1cloud.internal
> > domain=cs1cloud.internal
> > domain=cs1cloud.internal
> > dhcp-range=X.Y.202.1,static
> > dhcp-hostsfile=/etc/dhcphosts.txt
> > dhcp-ignore=tag:!known
> > dhcp-option=15,"cs1cloud.internal"
> > dhcp-option=vendor:MSFT,2,1i
> > dhcp-boot=pxelinux.0
> > enable-tftp
> > tftp-root=/opt/tftpboot
> > dhcp-lease-max=2100
> > domain=cs1cloud.internal
> > log-dhcp
> > log-facility=/var/log2/dnsmasq.log
> > conf-dir=/etc/dnsmasq.d
> > dhcp-optsfile=/etc/dhcpopts.txt
> > dhcp-option=option:router,X.Y.202.1
> > dhcp-option=6,X.Y.202.2,8.8.8.8,8.8.4.4
> > dhcp-client-update
> >
> > We actually have 3 class-C subnets assigned to the network.
> >
> > X.Y.202.0/24
> > X.Y.203.0/24
> > Z.A.107.0/24
> >
> > After enabling log-dhcp as per Chiradeep Vittal, I can see that the
> > available DHCP subnet is only the first subnet.
> >
> > Nov  7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 available DHCP subnet:
> > X.Y.202.2/255.255.255.0
> > Nov  7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 available DHCP subnet:
> > X.Y.202.1/255.255.255.0
> > Nov  7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 client provides name:
> > Debian-81-64b
> > Nov  7 20:27:49 dnsmasq-dhcp[22462]: 1979424125 DHCPDISCOVER(eth0)
> > 06:c5:38:01:13:40 ignored
> > Nov  7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 available DHCP subnet:
> > X.Y.202.2/255.255.255.0
> > Nov  7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 available DHCP subnet:
> > X.Y.202.1/255.255.255.0
> > Nov  7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 client provides name:
> > WIN-H4INMOBBRJA
> > Nov  7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 vendor class: MSFT 5.0
> > Nov  7 20:27:51 dnsmasq-dhcp[22462]: 2670909966 DHCPDISCOVER(eth0)
> > 06:31:ac:01:13:f0 ignored
> >
> > Coincidentally, all affected VMs are coming from the second subnet:
> > X.Y.203.0/24, while the third subnet is rarely used.

Re: ACS - Some VMs unable to get DHCP IP from VR

[Cloud List]

Hi Wei,

In addition,

The VR is serving a shared not isolated network, meaning the IP it serves
is 'guest' not 'public' IP. Will that make a difference on the iptables
command we need to execute?

Looking forward to your reply, thank you.

Cheers.


On Tue, Nov 8, 2016 at 3:19 AM, Cloud List <cloud-l...@sg.or.id> wrote:

> Hi Wei and Ozhan,
>
> Thanks for your reply.
>
> The problem doesn't affect only Debian-based guest VMs, but also affected
> some Windows and Ubuntu-based VMs as well. I have executed the command on
> the VR and reset the NIC of the guest VM, but unfortunately the issue still
> persists.
>
> iptables -t mangle -A POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM
> --checksum-fill
>
> After issuing the above command on VR and reset the NIC on guest vm
> (ifdown eth0, ifup eth0):
>
> On VR's /var/log/dnsmasq.log:
>
> Nov  7 19:09:22 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:b1:22:01:13:17
> ignored
> Nov  7 19:09:25 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:b1:22:01:13:17
> ignored
> Nov  7 19:09:30 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:b1:22:01:13:17
> ignored
> Nov  7 19:09:36 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:b1:22:01:13:17
> ignored
>
> On the guest VM:
>
> root@vm:~# ifup eth0
> Internet Systems Consortium DHCP Client 4.2.2
> Copyright 2004-2011 Internet Systems Consortium.
> All rights reserved.
> For info, please visit https://www.isc.org/software/dhcp
>
> Listening on LPF/eth0/06:b1:22:01:13:17
> Sending on LPF/eth0/06:b1:22:01:13:17
> Sending on Socket/fallback
> DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 4
> DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 5
> DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 64DHCPDISCOVER on
> eth0 to 255.255.255.255 port 67 interval 14
> DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 9
> DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 10
> DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 13
> No DHCOFFERS received.
> No working leases in persistent database - sleeping.
>
> I also tried to execute similar hotfix as mentioned on the bug report:
>
> iptables -A POSTROUTING -t mangle -p udp --dport bootpc -j CHECKSUM
> --checksum-fill
>
> The problem still persists. Any further advice on this is highly
> appreciated.
>
> Looking forward to your reply, thank you.
>
> Cheers.
>
>
> On Tue, Nov 8, 2016 at 2:41 AM, Wei ZHOU <ustcweiz...@gmail.com> wrote:
>
>> GOOD point.
>>
>> @CloudList, can you try again after executing the following command in VR
>> ?
>>
>> iptables -t mangle -A POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM
>> --checksum-fill
>>
>> -Wei
>>
>> 2016-11-07 14:46 GMT+01:00 Özhan Rüzgar Karaman <oruzgarkara...@gmail.com
>> >:
>>
>> > Hi;
>> > Whats your problematic vm's type is it Debian? Maybe you are affected
>> from
>> > the bug CLOUDSTACK-8326? I do not know if this bug has effected on ACS
>> 4.2
>> > release but i know that it effects release 4.8.x 4.9.x
>> >
>> > Thanks
>> > Özhan
>> >
>> > On Mon, Nov 7, 2016 at 4:36 PM, Cloud List <cloud-l...@sg.or.id> wrote:
>> >
>> > > Hi Wei,
>> > >
>> > > Thanks for your reply.
>> > >
>> > > I checked and I can confirm that the mac address is listed on
>> > > /etc/dhcphosts.txt in VR.
>> > >
>> > > For example:
>> > >
>> > > Nov  7 13:30:19 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
>> 06:31:ac:01:13:YY
>> > > ignored
>> > > Nov  7 13:30:24 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
>> 06:31:ac:01:13:YY
>> > > ignored
>> > > Nov  7 13:30:33 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
>> 06:31:ac:01:13:YY
>> > > ignored
>> > >
>> > > root@r-4155-VM:/var/log# grep 06:31:ac:01:13:f0 /etc/dhcphosts.txt
>> > > 06:31:ac:01:13:YY,X.X.X.X,vm-name,infinite
>> > >
>> > > YY - last two digits of the mac address
>> > > X.X.X.X - ip address which is supposed to be allocated to the VM
>> > >
>> > > Any advice how can I troubleshoot this further?
>> > >
>> > > Looking forward to your reply, thank you.
>> > >
>> > > Cheers.
>> > >
>> > >
>> > > On Mon, Nov 7, 2016 at 9:22 PM, Wei ZHOU <ustcweiz...@gmail.com>
>> wrote:
>> > >
>> > > > If the mac address is not listed in /etc/dhcphosts.txt in VR, the
>> > request
>> > > > wil

Re: ACS - Some VMs unable to get DHCP IP from VR

[Cloud List]

Hi Wei and Ozhan,

Thanks for your reply.

The problem doesn't affect only Debian-based guest VMs, but also affected
some Windows and Ubuntu-based VMs as well. I have executed the command on
the VR and reset the NIC of the guest VM, but unfortunately the issue still
persists.

iptables -t mangle -A POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM
--checksum-fill

After issuing the above command on VR and reset the NIC on guest vm (ifdown
eth0, ifup eth0):

On VR's /var/log/dnsmasq.log:

Nov  7 19:09:22 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:b1:22:01:13:17
ignored
Nov  7 19:09:25 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:b1:22:01:13:17
ignored
Nov  7 19:09:30 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:b1:22:01:13:17
ignored
Nov  7 19:09:36 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:b1:22:01:13:17
ignored

On the guest VM:

root@vm:~# ifup eth0
Internet Systems Consortium DHCP Client 4.2.2
Copyright 2004-2011 Internet Systems Consortium.
All rights reserved.
For info, please visit https://www.isc.org/software/dhcp

Listening on LPF/eth0/06:b1:22:01:13:17
Sending on LPF/eth0/06:b1:22:01:13:17
Sending on Socket/fallback
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 4
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 5
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 64DHCPDISCOVER on
eth0 to 255.255.255.255 port 67 interval 14
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 9
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 10
DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 13
No DHCOFFERS received.
No working leases in persistent database - sleeping.

I also tried to execute similar hotfix as mentioned on the bug report:

iptables -A POSTROUTING -t mangle -p udp --dport bootpc -j CHECKSUM
--checksum-fill

The problem still persists. Any further advice on this is highly
appreciated.

Looking forward to your reply, thank you.

Cheers.


On Tue, Nov 8, 2016 at 2:41 AM, Wei ZHOU <ustcweiz...@gmail.com> wrote:

> GOOD point.
>
> @CloudList, can you try again after executing the following command in VR ?
>
> iptables -t mangle -A POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM
> --checksum-fill
>
> -Wei
>
> 2016-11-07 14:46 GMT+01:00 Özhan Rüzgar Karaman <oruzgarkara...@gmail.com
> >:
>
> > Hi;
> > Whats your problematic vm's type is it Debian? Maybe you are affected
> from
> > the bug CLOUDSTACK-8326? I do not know if this bug has effected on ACS
> 4.2
> > release but i know that it effects release 4.8.x 4.9.x
> >
> > Thanks
> > Özhan
> >
> > On Mon, Nov 7, 2016 at 4:36 PM, Cloud List <cloud-l...@sg.or.id> wrote:
> >
> > > Hi Wei,
> > >
> > > Thanks for your reply.
> > >
> > > I checked and I can confirm that the mac address is listed on
> > > /etc/dhcphosts.txt in VR.
> > >
> > > For example:
> > >
> > > Nov  7 13:30:19 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
> 06:31:ac:01:13:YY
> > > ignored
> > > Nov  7 13:30:24 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
> 06:31:ac:01:13:YY
> > > ignored
> > > Nov  7 13:30:33 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
> 06:31:ac:01:13:YY
> > > ignored
> > >
> > > root@r-4155-VM:/var/log# grep 06:31:ac:01:13:f0 /etc/dhcphosts.txt
> > > 06:31:ac:01:13:YY,X.X.X.X,vm-name,infinite
> > >
> > > YY - last two digits of the mac address
> > > X.X.X.X - ip address which is supposed to be allocated to the VM
> > >
> > > Any advice how can I troubleshoot this further?
> > >
> > > Looking forward to your reply, thank you.
> > >
> > > Cheers.
> > >
> > >
> > > On Mon, Nov 7, 2016 at 9:22 PM, Wei ZHOU <ustcweiz...@gmail.com>
> wrote:
> > >
> > > > If the mac address is not listed in /etc/dhcphosts.txt in VR, the
> > request
> > > > will be ignored.
> > > >
> > > > Can you give more details so we can reproduce it and fix it ?
> > > >
> > > > -Wei
> > > >
> > > > 2016-11-07 13:44 GMT+01:00 Cloud List <cloud-l...@sg.or.id>:
> > > >
> > > > > Hi,
> > > > >
> > > > > After our upgrade from CloudStack 4.2 to 4.8.1.1, we noted that
> some
> > > (but
> > > > > not all) of our VMs are not able to get DHCP from the VR. This
> gives
> > > > > problem when the VM is restarted and cannot get up and running
> > because
> > > > > unable to get IP.
> > > > >
> > > > > I logged in to the VR and found below messages showing that the
> DHCP
> > > > server
> > &

Re: ACS - Some VMs unable to get DHCP IP from VR

[Cloud List]

Hi Wei,

Thanks for your reply.

I checked and I can confirm that the mac address is listed on
/etc/dhcphosts.txt in VR.

For example:

Nov  7 13:30:19 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:31:ac:01:13:YY
ignored
Nov  7 13:30:24 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:31:ac:01:13:YY
ignored
Nov  7 13:30:33 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:31:ac:01:13:YY
ignored

root@r-4155-VM:/var/log# grep 06:31:ac:01:13:f0 /etc/dhcphosts.txt
06:31:ac:01:13:YY,X.X.X.X,vm-name,infinite

YY - last two digits of the mac address
X.X.X.X - ip address which is supposed to be allocated to the VM

Any advice how can I troubleshoot this further?

Looking forward to your reply, thank you.

Cheers.


On Mon, Nov 7, 2016 at 9:22 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote:

> If the mac address is not listed in /etc/dhcphosts.txt in VR, the request
> will be ignored.
>
> Can you give more details so we can reproduce it and fix it ?
>
> -Wei
>
> 2016-11-07 13:44 GMT+01:00 Cloud List <cloud-l...@sg.or.id>:
>
> > Hi,
> >
> > After our upgrade from CloudStack 4.2 to 4.8.1.1, we noted that some (but
> > not all) of our VMs are not able to get DHCP from the VR. This gives
> > problem when the VM is restarted and cannot get up and running because
> > unable to get IP.
> >
> > I logged in to the VR and found below messages showing that the DHCP
> server
> > is ignoring the request from the VM.
> >
> > Nov  7 12:19:43 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
> > ignored
> > Nov  7 12:19:52 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:05:64:01:13:d3
> > ignored
> > Nov  7 12:19:53 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
> > ignored
> > Nov  7 12:20:04 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
> > ignored
> > Nov  7 12:20:11 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:05:64:01:13:d3
> > ignored
> > Nov  7 12:20:22 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
> > ignored
> > Nov  7 12:20:30 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
> > ignored
> > Nov  7 12:20:42 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
> > ignored
> > Nov  7 12:21:02 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
> > ignored
> > Nov  7 12:21:19 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
> > ignored
> >
> > Any reason why the dnsmasq service ignored the request from the VM and
> how
> > can we fix that?
> >
> > At the moment, the only workaround we can do is to configure the IP
> address
> > statically to the servelet, which is not practical.
> >
> > Any advice is greatly appreciated.
> >
> > Thank you.
> >
>

Re: Long delay in listDomains API response

[Cloud List]

Thanks Sergey, appreciate your reply. Points noted.

Cheers.

On Mon, Nov 7, 2016 at 10:44 AM, Sergey Levitskiy <
sergey.levits...@autodesk.com> wrote:

> What essentially happens here that for each domain, you have access with
> your key, ACS executes 12 queries to get domain limits.  Even if such query
> for each resource limit takes 0.03 sec it translates into 0.4 sec for each
> domain. If you have more than few domains, you will see rather lengthy
> total execution of listDomains in spite of some parallel execution of such
> queries. This is apparently far from optimal but I don’t see if it can be
> improved by optimizing MySQL alone.   I looked through the ACS code and
> there is a way to improve this big time since all resource limits are
> already in domain_view . We will try to fix it soon for 4.9-4.10 versions.
> It usually takes at least 2-3 weeks before the code will be reviewed and
> committed. Until then I suggest to increase any client timeouts accordingly.
>
> Thanks,
> Sergey
>
>
> On 11/6/16, 5:52 PM, "Cloud List" <cloud-l...@sg.or.id> wrote:
>
> Hi Wei,
>
> In addition, can you also advice any specific tweak or tuning of the
> CloudStack management server and MySQL database's configurations to
> improve
> the time it takes to perform the listDomains API query and result? We
> have
> tried CloudStack and MySQL tunings as per below documentations and it
> doesn't seem to improve the listDomains API query time.
>
> http://docs.cloudstack.apache.org/projects/cloudstack-
> administration/en/4.8/tuning.html
> http://events.linuxfoundation.org/sites/events/files/slides/
> CloudSTack-Tuning-Collab-2015.pdf
> https://www.percona.com/blog/2014/01/28/10-mysql-
> performance-tuning-settings-after-installation/
> https://www.percona.com/blog/2012/01/06/mysql-high-number-
> connections-per-secon/
>
> Any advice on this is greatly appreciated.
>
> Looking forward to your reply, thank you.
>
> Cheers.
>
>
>
> On Mon, Nov 7, 2016 at 2:35 AM, Cloud List <cloud-l...@sg.or.id>
> wrote:
>
> > Hi Wei,
> >
> > Thanks for your reply. Is it possible for us to remove the resource
> > limitation feature -- any global settings that we can set to turn
> this off?
> > If not, would you be able to let us know where on the source code
> this is
> > being set, so that we can try changing it and recompile ACS to turn
> off the
> > feature?
> >
> > If it's possible, how would be the impact to CloudStack operations in
> > general? May I know what was the original purpose the limitation is
> being
> > put in the first place?
> >
> > Thanks for the workaround recommendation -- will forward to our
> > application vendor to see if the app can be fine-tuned following your
> > recommendation.
> >
> > Looking forward to your reply, thank you.
> >
> > Cheers.
> >
> > On Mon, Nov 7, 2016 at 12:42 AM, Wei ZHOU <ustcweiz...@gmail.com>
> wrote:
> >
> >> I cannot say it is unnecessary or not for now.
> >> There are indeed many db connections when get resource
> limtiation/count
> >> of domains.
> >>
> >> A workaound is (1) listDomainChildren gets id of domain and
> sub-domains,
> >> then (2) listDomains with specified id.
> >>
> >> -Wei
> >>
> >>
> >>
> >> 2016-11-06 17:23 GMT+01:00 Cloud List <cloud-l...@sg.or.id>:
> >>
> >>> Hi Sergey,
> >>>
> >>> Thanks for your reply. The grep result is a bit long, total 64K
> lines,
> >>> so I have uploaded it here:
> >>>
> >>> https://dl.dropboxusercontent.com/u/24788636/ctx-3f3ff343.txt
> >>>
> >>> It seems there are a lot of unnecessary creating and closing of
> >>> connections to the DB, causing the delay?
> >>>
> >>> Looking forward to your reply, thank you.
> >>>
> >>> Cheers.
> >>>
> >>>
> >>> On Sun, Nov 6, 2016 at 11:30 PM, Sergey Levitskiy <
> >>> sergey.levits...@autodesk.com> wrote:
> >>>
> >>>> Can you grep for this line ‘ctx-3f3ff343’ in your log and post it
> here?
> >>>>
> >>>>
> >>>> On 11/5/16, 11:01 PM, "Cloud List" <cloud-l...@sg.or.id> wrote:

ACS - Some VMs unable to get DHCP IP from VR

[Cloud List]

Hi,

After our upgrade from CloudStack 4.2 to 4.8.1.1, we noted that some (but
not all) of our VMs are not able to get DHCP from the VR. This gives
problem when the VM is restarted and cannot get up and running because
unable to get IP.

I logged in to the VR and found below messages showing that the DHCP server
is ignoring the request from the VM.

Nov  7 12:19:43 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
ignored
Nov  7 12:19:52 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:05:64:01:13:d3
ignored
Nov  7 12:19:53 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
ignored
Nov  7 12:20:04 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
ignored
Nov  7 12:20:11 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:05:64:01:13:d3
ignored
Nov  7 12:20:22 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
ignored
Nov  7 12:20:30 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
ignored
Nov  7 12:20:42 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
ignored
Nov  7 12:21:02 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
ignored
Nov  7 12:21:19 dnsmasq-dhcp[29938]: DHCPDISCOVER(eth0) 06:44:da:01:13:98
ignored

Any reason why the dnsmasq service ignored the request from the VM and how
can we fix that?

At the moment, the only workaround we can do is to configure the IP address
statically to the servelet, which is not practical.

Any advice is greatly appreciated.

Thank you.

Re: Long delay in listDomains API response

[Cloud List]

Hi Wei,

In addition, can you also advice any specific tweak or tuning of the
CloudStack management server and MySQL database's configurations to improve
the time it takes to perform the listDomains API query and result? We have
tried CloudStack and MySQL tunings as per below documentations and it
doesn't seem to improve the listDomains API query time.

http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.8/tuning.html
http://events.linuxfoundation.org/sites/events/files/slides/CloudSTack-Tuning-Collab-2015.pdf
https://www.percona.com/blog/2014/01/28/10-mysql-performance-tuning-settings-after-installation/
https://www.percona.com/blog/2012/01/06/mysql-high-number-connections-per-secon/

Any advice on this is greatly appreciated.

Looking forward to your reply, thank you.

Cheers.



On Mon, Nov 7, 2016 at 2:35 AM, Cloud List <cloud-l...@sg.or.id> wrote:

> Hi Wei,
>
> Thanks for your reply. Is it possible for us to remove the resource
> limitation feature -- any global settings that we can set to turn this off?
> If not, would you be able to let us know where on the source code this is
> being set, so that we can try changing it and recompile ACS to turn off the
> feature?
>
> If it's possible, how would be the impact to CloudStack operations in
> general? May I know what was the original purpose the limitation is being
> put in the first place?
>
> Thanks for the workaround recommendation -- will forward to our
> application vendor to see if the app can be fine-tuned following your
> recommendation.
>
> Looking forward to your reply, thank you.
>
> Cheers.
>
> On Mon, Nov 7, 2016 at 12:42 AM, Wei ZHOU <ustcweiz...@gmail.com> wrote:
>
>> I cannot say it is unnecessary or not for now.
>> There are indeed many db connections when get resource limtiation/count
>> of domains.
>>
>> A workaound is (1) listDomainChildren gets id of domain and sub-domains,
>> then (2) listDomains with specified id.
>>
>> -Wei
>>
>>
>>
>> 2016-11-06 17:23 GMT+01:00 Cloud List <cloud-l...@sg.or.id>:
>>
>>> Hi Sergey,
>>>
>>> Thanks for your reply. The grep result is a bit long, total 64K lines,
>>> so I have uploaded it here:
>>>
>>> https://dl.dropboxusercontent.com/u/24788636/ctx-3f3ff343.txt
>>>
>>> It seems there are a lot of unnecessary creating and closing of
>>> connections to the DB, causing the delay?
>>>
>>> Looking forward to your reply, thank you.
>>>
>>> Cheers.
>>>
>>>
>>> On Sun, Nov 6, 2016 at 11:30 PM, Sergey Levitskiy <
>>> sergey.levits...@autodesk.com> wrote:
>>>
>>>> Can you grep for this line ‘ctx-3f3ff343’ in your log and post it here?
>>>>
>>>>
>>>> On 11/5/16, 11:01 PM, "Cloud List" <cloud-l...@sg.or.id> wrote:
>>>>
>>>> Dear all,
>>>>
>>>> After researching further, I found out that our problem is exactly
>>>> the one
>>>> reported on this JIRA bug:
>>>>
>>>> https://issues.apache.org/jira/browse/CLOUDSTACK-9405
>>>>
>>>> I am cc-ing dsclose and Wei Zhou on this email. Anyone know if
>>>> there's any
>>>> resolution to the above problem/bug?
>>>>
>>>> On the bug report, Wei Zhou mentioned that he added domain level
>>>> resource
>>>> limitation, may I know if this limitation is configurable? I can't
>>>> find it
>>>> on the Global Settings.
>>>>
>>>> Looking forward to your reply, thank you.
>>>>
>>>> Cheers.
>>>>
>>>>
>>>> On Sun, Nov 6, 2016 at 10:30 AM, Cloud List <cloud-l...@sg.or.id>
>>>> wrote:
>>>>
>>>> > Thanks Sergey for your assistance.
>>>> >
>>>> > I have enabled the TRACE logs as requested. After the listDomains
>>>> API
>>>> > call, I see a lot of DB connections being created to the database,
>>>> > preparing SELECT statements and then immediately being closed
>>>> thereafter.
>>>> > This happens many many times in the loop in the course of 10-15
>>>> seconds.
>>>> > For example:
>>>> >
>>>> > Creating a DB connection with  no txn
>>>> > Preparing: SELECT
>>>> > Closing: com.mysql.jdbc.PreparedStatement
>>>> > Closing DB connection
>>&g

Re: Long delay in listDomains API response

[Cloud List]

Hi Sergey,

Thanks for your reply. The grep result is a bit long, total 64K lines, so I
have uploaded it here:

https://dl.dropboxusercontent.com/u/24788636/ctx-3f3ff343.txt

It seems there are a lot of unnecessary creating and closing of connections
to the DB, causing the delay?

Looking forward to your reply, thank you.

Cheers.


On Sun, Nov 6, 2016 at 11:30 PM, Sergey Levitskiy <
sergey.levits...@autodesk.com> wrote:

> Can you grep for this line ‘ctx-3f3ff343’ in your log and post it here?
>
>
> On 11/5/16, 11:01 PM, "Cloud List" <cloud-l...@sg.or.id> wrote:
>
> Dear all,
>
> After researching further, I found out that our problem is exactly the
> one
> reported on this JIRA bug:
>
> https://issues.apache.org/jira/browse/CLOUDSTACK-9405
>
> I am cc-ing dsclose and Wei Zhou on this email. Anyone know if there's
> any
> resolution to the above problem/bug?
>
> On the bug report, Wei Zhou mentioned that he added domain level
> resource
> limitation, may I know if this limitation is configurable? I can't
> find it
> on the Global Settings.
>
> Looking forward to your reply, thank you.
>
> Cheers.
>
>
> On Sun, Nov 6, 2016 at 10:30 AM, Cloud List <cloud-l...@sg.or.id>
> wrote:
>
> > Thanks Sergey for your assistance.
> >
> > I have enabled the TRACE logs as requested. After the listDomains API
> > call, I see a lot of DB connections being created to the database,
> > preparing SELECT statements and then immediately being closed
> thereafter.
> > This happens many many times in the loop in the course of 10-15
> seconds.
> > For example:
> >
> > Creating a DB connection with  no txn
> > Preparing: SELECT
> > Closing: com.mysql.jdbc.PreparedStatement
> > Closing DB connection
> >
> > The above continues in a loop for 10-15 seconds, sometimes more. Is
> it
> > normal? Could it be something wrong during the database schema
> upgrade when
> > I upgraded from 4.2 to 4.8?
> >
> > Excerpts of the TRACE logs are below. Any advice on how I can
> resolve the
> > problem is highly appreciated.
> >
> > ===
> > 2016-11-06 09:36:04,893 DEBUG [c.c.a.ApiServlet]
> > (http-8080-10:ctx-3f3ff343) (logid:0429eaf0) ===START===  X.X.X.X --
> GET
> > apikey==listDomains=json=
> > 2016-11-06 09:36:04,893 TRACE [c.c.u.d.T.Transaction]
> > (http-8080-10:ctx-3f3ff343 ctx-e486fd2b) (logid:0429eaf0) Using
> current
> > transaction: http-8080-10 : http-8080-10,
> > 2016-11-06 09:36:04,893 TRACE [c.c.u.d.T.Transaction]
> > (http-8080-10:ctx-3f3ff343 ctx-e486fd2b) (logid:0429eaf0) Using
> current
> > transaction: http-8080-10 : http-8080-10,
> > 2016-11-06 09:36:04,894 TRACE [c.c.u.d.T.Statement]
> (StatsCollector-2:ctx-293a2a9e)
> > (logid:725b6dd7) Closing: com.mysql.jdbc.JDBC4PreparedStatement@
> 1bda5f56:
> > SELECT vm_instance.id, vm_instance.name, vm_instance.vnc_password,
> > vm_instance.proxy_id, vm_instance.proxy_assign_time,
> vm_instance.state,
> > vm_instance.private_ip_address, vm_instance.instance_name,
> > vm_instance.vm_template_id, vm_instance.guest_os_id,
> vm_instance.host_id,
> > vm_instance.last_host_id, vm_instance.pod_id,
> vm_instance.private_mac_address,
> > vm_instance.data_center_id, vm_instance.vm_type,
> vm_instance.ha_enabled,
> > vm_instance.display_vm, vm_instance.limit_cpu_use,
> > vm_instance.update_count, vm_instance.created, vm_instance.removed,
> > vm_instance.update_time, vm_instance.domain_id,
> vm_instance.account_id,
> > vm_instance.user_id, vm_instance.service_offering_id,
> > vm_instance.reservation_id, vm_instance.hypervisor_type,
> > vm_instance.dynamically_scalable, vm_instance.uuid,
> > vm_instance.disk_offering_id, vm_instance.power_state,
> > vm_instance.power_state_update_time, vm_instance.power_state_
> update_count,
> > vm_instance.power_host, user_vm.iso_id, user_vm.user_data,
> > user_vm.display_name, user_vm.update_parameters FROM user_vm INNER
> JOIN
> > vm_instance ON user_vm.id=vm_instance.id  WHERE
> vm_instance.type='User'
> > AND user_vm.id = 4088  AND vm_instance.removed IS NULL
> > 2016-11-06 09:36:04,894 TRACE [c.c.u.d.T.Connection]
> (StatsCollector-2:ctx-293a2a9e)
> > (logid:725b6dd7) Closing DB connection: dbconn1208177431
> > 2016-11-06 09:36:04,894 TRACE [c.c.u.d.T.Transaction]
> > (StatsCollector-2:ctx-29

Re: Site 2 Site VPN on VPC not working after upgrading ACS from 4.2 to 4.8.1.1

[Cloud List]

Hi,

[RESOLVED] Would like to share that I managed to find the workaround to the
problem by emptying the password ipsec_psk field on the entry inside
s2s_customer_gateway table, delete the VPN connection, modify the VPN
customer gateway with the new password (we will notice that the encrypted
format of the password will be populated into the ipsec_psk field), and
then recreate the VPN connection. The VPN connection will then be
established.

Hope the above can help others.

Thank you.

On Sun, Nov 6, 2016 at 6:16 PM, Cloud List <cloud-l...@sg.or.id> wrote:

> Hi,
>
> Another issue we noted after upgrading ACS from 4.2 to 4.8.1.1 is that
> site-to-site VPN is not working. From GUI, I cannot even go to Home >
> Network > Select view: VPN customer gateway with below error messages shown
> on the GUI:
>
> 
> Caught: com.mysql.jdbc.JDBC4PreparedStatement@45ae1e69: SELECT
> s2s_customer_gateway.id, s2s_customer_gateway.uuid,
> s2s_customer_gateway.name, s2s_customer_gateway.gateway_ip,
> s2s_customer_gateway.guest_cidr_list, s2s_customer_gateway.ipsec_psk,
> s2s_customer_gateway.ike_policy, s2s_customer_gateway.esp_policy,
> s2s_customer_gateway.ike_lifetime, s2s_customer_gateway.esp_lifetime,
> s2s_customer_gateway.dpd, s2s_customer_gateway.force_encap,
> s2s_customer_gateway.domain_id, s2s_customer_gateway.account_id,
> s2s_customer_gateway.removed FROM s2s_customer_gateway INNER JOIN account
> ON s2s_customer_gateway.account_id=account.id WHERE
> s2s_customer_gateway.removed IS NULL AND (account.type != 5 ) ORDER BY
> s2s_customer_gateway.id DESC LIMIT 0, 20
> 
>
> This is the error messages on CloudStack management server:
>
> 
> 2016-11-06 18:00:30,218 DEBUG [c.c.u.c.DBEncryptionUtil]
> (http-8080-2:ctx-da483727 ctx-695addac) (logid:3d3a6225) Error while
> decrypting: 
> 2016-11-06 18:00:30,218 ERROR [c.c.a.ApiServer] (http-8080-2:ctx-da483727
> ctx-695addac) (logid:3d3a6225) unhandled exception executing api command:
> [Ljava.lang.String;@
> 6ad8a1d4
> com.cloud.utils.exception.CloudRuntimeException: Caught: com.mysql.jdbc.
> JDBC4PreparedStatement@55edbe9c: SELECT s2s_customer_gateway.id,
> s2s_customer_gateway.uuid, s2s_
> customer_gateway.name, s2s_customer_gateway.gateway_ip,
> s2s_customer_gateway.guest_cidr_list, s2s_customer_gateway.ipsec_psk,
> s2s_customer_gateway.ike_policy, s2s_custo
> mer_gateway.esp_policy, s2s_customer_gateway.ike_lifetime,
> s2s_customer_gateway.esp_lifetime, s2s_customer_gateway.dpd,
> s2s_customer_gateway.force_encap, s2s_customer_g
> ateway.domain_id, s2s_customer_gateway.account_id,
> s2s_customer_gateway.removed FROM s2s_customer_gateway  INNER JOIN account
> ON s2s_customer_gateway.account_id=account
> .id WHERE s2s_customer_gateway.removed IS NULL  AND  (account.type != 5 )
> ORDER BY s2s_customer_gateway.id DESC  LIMIT 0, 20
> 
>
> Note that on "Error while decrypting: ", I checked that 
> is the cleartext IPsec preshared key (ipsec_psk field on
> s2s_customer_gateway table on the database). It seems that CloudStack tries
> to decrypt a clear-text / non-encrypted password? Does the ipsec_psk field
> on the database supposed to store the encrypted IPsec pre-shared key for
> the VPN connection under CloudStack version 4.8.1.1?
>
> I am able to execute the SELECT command on the database directly without
> any issues.
>
> Any help is greatly appreciated.
>
> Thank you.
>

Site 2 Site VPN on VPC not working after upgrading ACS from 4.2 to 4.8.1.1

[Cloud List]

Hi,

Another issue we noted after upgrading ACS from 4.2 to 4.8.1.1 is that
site-to-site VPN is not working. From GUI, I cannot even go to Home >
Network > Select view: VPN customer gateway with below error messages shown
on the GUI:


Caught: com.mysql.jdbc.JDBC4PreparedStatement@45ae1e69: SELECT
s2s_customer_gateway.id, s2s_customer_gateway.uuid,
s2s_customer_gateway.name, s2s_customer_gateway.gateway_ip,
s2s_customer_gateway.guest_cidr_list, s2s_customer_gateway.ipsec_psk,
s2s_customer_gateway.ike_policy, s2s_customer_gateway.esp_policy,
s2s_customer_gateway.ike_lifetime, s2s_customer_gateway.esp_lifetime,
s2s_customer_gateway.dpd, s2s_customer_gateway.force_encap,
s2s_customer_gateway.domain_id, s2s_customer_gateway.account_id,
s2s_customer_gateway.removed FROM s2s_customer_gateway INNER JOIN account
ON s2s_customer_gateway.account_id=account.id WHERE
s2s_customer_gateway.removed IS NULL AND (account.type != 5 ) ORDER BY
s2s_customer_gateway.id DESC LIMIT 0, 20


This is the error messages on CloudStack management server:


2016-11-06 18:00:30,218 DEBUG [c.c.u.c.DBEncryptionUtil]
(http-8080-2:ctx-da483727 ctx-695addac) (logid:3d3a6225) Error while
decrypting: 
2016-11-06 18:00:30,218 ERROR [c.c.a.ApiServer] (http-8080-2:ctx-da483727
ctx-695addac) (logid:3d3a6225) unhandled exception executing api command:
[Ljava.lang.String;@
6ad8a1d4
com.cloud.utils.exception.CloudRuntimeException: Caught:
com.mysql.jdbc.JDBC4PreparedStatement@55edbe9c: SELECT
s2s_customer_gateway.id, s2s_customer_gateway.uuid, s2s_
customer_gateway.name, s2s_customer_gateway.gateway_ip,
s2s_customer_gateway.guest_cidr_list, s2s_customer_gateway.ipsec_psk,
s2s_customer_gateway.ike_policy, s2s_custo
mer_gateway.esp_policy, s2s_customer_gateway.ike_lifetime,
s2s_customer_gateway.esp_lifetime, s2s_customer_gateway.dpd,
s2s_customer_gateway.force_encap, s2s_customer_g
ateway.domain_id, s2s_customer_gateway.account_id,
s2s_customer_gateway.removed FROM s2s_customer_gateway  INNER JOIN account
ON s2s_customer_gateway.account_id=account
.id WHERE s2s_customer_gateway.removed IS NULL  AND  (account.type != 5 )
ORDER BY s2s_customer_gateway.id DESC  LIMIT 0, 20


Note that on "Error while decrypting: ", I checked that  is
the cleartext IPsec preshared key (ipsec_psk field on s2s_customer_gateway
table on the database). It seems that CloudStack tries to decrypt a
clear-text / non-encrypted password? Does the ipsec_psk field on the
database supposed to store the encrypted IPsec pre-shared key for the VPN
connection under CloudStack version 4.8.1.1?

I am able to execute the SELECT command on the database directly without
any issues.

Any help is greatly appreciated.

Thank you.

Re: Long delay in listDomains API response

[Cloud List]

Dear all,

After researching further, I found out that our problem is exactly the one
reported on this JIRA bug:

https://issues.apache.org/jira/browse/CLOUDSTACK-9405

I am cc-ing dsclose and Wei Zhou on this email. Anyone know if there's any
resolution to the above problem/bug?

On the bug report, Wei Zhou mentioned that he added domain level resource
limitation, may I know if this limitation is configurable? I can't find it
on the Global Settings.

Looking forward to your reply, thank you.

Cheers.


On Sun, Nov 6, 2016 at 10:30 AM, Cloud List <cloud-l...@sg.or.id> wrote:

> Thanks Sergey for your assistance.
>
> I have enabled the TRACE logs as requested. After the listDomains API
> call, I see a lot of DB connections being created to the database,
> preparing SELECT statements and then immediately being closed thereafter.
> This happens many many times in the loop in the course of 10-15 seconds.
> For example:
>
> Creating a DB connection with  no txn
> Preparing: SELECT
> Closing: com.mysql.jdbc.PreparedStatement
> Closing DB connection
>
> The above continues in a loop for 10-15 seconds, sometimes more. Is it
> normal? Could it be something wrong during the database schema upgrade when
> I upgraded from 4.2 to 4.8?
>
> Excerpts of the TRACE logs are below. Any advice on how I can resolve the
> problem is highly appreciated.
>
> ===
> 2016-11-06 09:36:04,893 DEBUG [c.c.a.ApiServlet]
> (http-8080-10:ctx-3f3ff343) (logid:0429eaf0) ===START===  X.X.X.X -- GET
> apikey==listDomains=json=
> 2016-11-06 09:36:04,893 TRACE [c.c.u.d.T.Transaction]
> (http-8080-10:ctx-3f3ff343 ctx-e486fd2b) (logid:0429eaf0) Using current
> transaction: http-8080-10 : http-8080-10,
> 2016-11-06 09:36:04,893 TRACE [c.c.u.d.T.Transaction]
> (http-8080-10:ctx-3f3ff343 ctx-e486fd2b) (logid:0429eaf0) Using current
> transaction: http-8080-10 : http-8080-10,
> 2016-11-06 09:36:04,894 TRACE [c.c.u.d.T.Statement] 
> (StatsCollector-2:ctx-293a2a9e)
> (logid:725b6dd7) Closing: com.mysql.jdbc.JDBC4PreparedStatement@1bda5f56:
> SELECT vm_instance.id, vm_instance.name, vm_instance.vnc_password,
> vm_instance.proxy_id, vm_instance.proxy_assign_time, vm_instance.state,
> vm_instance.private_ip_address, vm_instance.instance_name,
> vm_instance.vm_template_id, vm_instance.guest_os_id, vm_instance.host_id,
> vm_instance.last_host_id, vm_instance.pod_id, vm_instance.private_mac_address,
> vm_instance.data_center_id, vm_instance.vm_type, vm_instance.ha_enabled,
> vm_instance.display_vm, vm_instance.limit_cpu_use,
> vm_instance.update_count, vm_instance.created, vm_instance.removed,
> vm_instance.update_time, vm_instance.domain_id, vm_instance.account_id,
> vm_instance.user_id, vm_instance.service_offering_id,
> vm_instance.reservation_id, vm_instance.hypervisor_type,
> vm_instance.dynamically_scalable, vm_instance.uuid,
> vm_instance.disk_offering_id, vm_instance.power_state,
> vm_instance.power_state_update_time, vm_instance.power_state_update_count,
> vm_instance.power_host, user_vm.iso_id, user_vm.user_data,
> user_vm.display_name, user_vm.update_parameters FROM user_vm INNER JOIN
> vm_instance ON user_vm.id=vm_instance.id  WHERE vm_instance.type='User'
> AND user_vm.id = 4088  AND vm_instance.removed IS NULL
> 2016-11-06 09:36:04,894 TRACE [c.c.u.d.T.Connection] 
> (StatsCollector-2:ctx-293a2a9e)
> (logid:725b6dd7) Closing DB connection: dbconn1208177431
> 2016-11-06 09:36:04,894 TRACE [c.c.u.d.T.Transaction]
> (StatsCollector-2:ctx-293a2a9e) (logid:725b6dd7) Using current
> transaction: StatsCollector-2 : StatsCollector-2,
> 2016-11-06 09:36:04,895 TRACE [c.c.u.d.T.Connection]
> (http-8080-10:ctx-3f3ff343 ctx-e486fd2b) (logid:0429eaf0) Creating a DB
> connection with  no txn:  for 0: dbconn1379204259. Stack:
> -TransactionLegacy.prepareStatement:457-TransactionLegacy.
> prepareAutoCloseStatement:450-AccountDaoImpl.findUserAccountByApiKey:129-
> GeneratedMethodAccessor234.invoke:-1-DelegatingMethodAccessorImpl.
> invoke:43-Method.invoke:606-AopUtils.invokeJoinpointUsingReflection:317-
> ReflectiveMethodInvocation.invokeJoinpoint:183-ReflectiveMethodInvocation.
> proceed:150-TransactionContextInterceptor.invoke:34-
> ReflectiveMethodInvocation.proceed:161-ExposeInvocationInterceptor.
> invoke:91
> 2016-11-06 09:36:04,895 TRACE [c.c.u.d.T.Statement]
> (http-8080-10:ctx-3f3ff343 ctx-e486fd2b) (logid:0429eaf0) Preparing: SELECT
> u.id, u.username, u.account_id, u.secret_key, u.state, a.id,
> a.account_name, a.type, a.domain_id, a.state FROM `cloud`.`user` u,
> `cloud`.`account` a WHERE u.account_id = a.id AND u.api_key = ? and
> u.removed IS NULL
> 2016-11-06 09:36:04,895 TRACE [c.c.u.d.T.Connection] 
> (StatsCollector-2:ctx-293a2a9e)
> (logid:725b6dd7) Creating a DB connection with  no txn:  for 0:
> 

Re: Long delay in listDomains API response

[Cloud List]

: dbconn1688317537
2016-11-06 09:36:04,902 TRACE [c.c.u.d.T.Connection]
(StatsCollector-2:ctx-293a2a9e) (logid:725b6dd7) Creating a DB connection
with  no txn:  for 0: dbconn1632185236. Stack:
-TransactionLegacy.prepareStatement:457-TransactionLegacy.prepareAutoCloseStatement:450-GenericDaoBase.findById:1005-GenericDaoBase.lockRow:991-GenericDaoBase.findById:949-GeneratedMethodAccessor42.invoke:-1-DelegatingMethodAccessorImpl.invoke:43-Method.invoke:606-AopUtils.invokeJoinpointUsingReflection:317-ReflectiveMethodInvocation.invokeJoinpoint:183-ReflectiveMethodInvocation.proceed:150-TransactionContextInterceptor.invoke:34
2016-11-06 09:36:04,902 TRACE [c.c.u.AccountManagerImpl]
(http-8080-10:ctx-3f3ff343 ctx-e486fd2b ctx-f5ec429d) (logid:0429eaf0) Root
Access granted to Acct[0bb6751b-eaa1-11e2-b078-6d70ed52d122-admin] by
AffinityGroupAccessChecker
2016-11-06 09:36:04,902 TRACE [c.c.u.d.T.Statement]
(StatsCollector-2:ctx-293a2a9e) (logid:725b6dd7) Preparing: SELECT
vm_instance.id, vm_instance.name, vm_instance.vnc_password,
vm_instance.proxy_id, vm_instance.proxy_assign_time, vm_instance.state,
vm_instance.private_ip_address, vm_instance.instance_name,
vm_instance.vm_template_id, vm_instance.guest_os_id, vm_instance.host_id,
vm_instance.last_host_id, vm_instance.pod_id,
vm_instance.private_mac_address, vm_instance.data_center_id,
vm_instance.vm_type, vm_instance.ha_enabled, vm_instance.display_vm,
vm_instance.limit_cpu_use, vm_instance.update_count, vm_instance.created,
vm_instance.removed, vm_instance.update_time, vm_instance.domain_id,
vm_instance.account_id, vm_instance.user_id,
vm_instance.service_offering_id, vm_instance.reservation_id,
vm_instance.hypervisor_type, vm_instance.dynamically_scalable,
vm_instance.uuid, vm_instance.disk_offering_id, vm_instance.power_state,
vm_instance.power_state_update_time, vm_instance.power_state_update_count,
vm_instance.power_host, user_vm.iso_id, user_vm.user_data,
user_vm.display_name, user_vm.update_parameters FROM user_vm INNER JOIN
vm_instance ON user_vm.id=vm_instance.id  WHERE vm_instance.type='User' AND
user_vm.id = ?  AND vm_instance.removed IS NULL
2016-11-06 09:36:04,902 TRACE [c.c.u.d.T.Transaction]
(http-8080-10:ctx-3f3ff343 ctx-e486fd2b ctx-f5ec429d) (logid:0429eaf0)
Using current transaction: http-8080-10 : http-8080-10,
2016-11-06 09:36:04,903 TRACE [c.c.u.d.T.Statement]
(StatsCollector-2:ctx-293a2a9e) (logid:725b6dd7) Closing:
com.mysql.jdbc.JDBC4PreparedStatement@107795cb: SELECT vm_instance.id,
vm_instance.name, vm_instance.vnc_password, vm_instance.proxy_id,
vm_instance.proxy_assign_time, vm_instance.state,
vm_instance.private_ip_address, vm_instance.instance_name,
vm_instance.vm_template_id, vm_instance.guest_os_id, vm_instance.host_id,
vm_instance.last_host_id, vm_instance.pod_id,
vm_instance.private_mac_address, vm_instance.data_center_id,
vm_instance.vm_type, vm_instance.ha_enabled, vm_instance.display_vm,
vm_instance.limit_cpu_use, vm_instance.update_count, vm_instance.created,
vm_instance.removed, vm_instance.update_time, vm_instance.domain_id,
vm_instance.account_id, vm_instance.user_id,
vm_instance.service_offering_id, vm_instance.reservation_id,
vm_instance.hypervisor_type, vm_instance.dynamically_scalable,
vm_instance.uuid, vm_instance.disk_offering_id, vm_instance.power_state,
vm_instance.power_state_update_time, vm_instance.power_state_update_count,
vm_instance.power_host, user_vm.iso_id, user_vm.user_data,
user_vm.display_name, user_vm.update_parameters FROM user_vm INNER JOIN
vm_instance ON user_vm.id=vm_instance.id  WHERE vm_instance.type='User' AND
user_vm.id = 4221  AND vm_instance.removed IS NULL
2016-11-06 09:36:04,904 TRACE [c.c.u.d.T.Connection]
(StatsCollector-2:ctx-293a2a9e) (logid:725b6dd7) Closing DB connection:
dbconn1632185236



On Sun, Nov 6, 2016 at 9:37 AM, Sergey Levitskiy <
sergey.levits...@autodesk.com> wrote:

> Please change DEBUG to TRACE for the following classes so they look like
> below. You don’t need to restart Cloudstack since it will dynamically
> switch log level. Once switched it will start generating enormous amount of
> logging so once you capture the slowness switch it back to DEBUG.
>
>   
>  
>
>
>
>   
>
>
>
> Thanks,
> Sergey
>
>
>
>
> On 11/5/16, 6:11 PM, "Cloud List" <cloud-l...@sg.or.id> wrote:
>
> Hi Sergey,
>
> Thanks for your reply. May I know logging for which class I need to
> modify
> to TRACE? Is it for the c.c.a.ApiServelet class? Not too sure what
> does the
> c.c.a stands for -- how do I check that? So I can put the correct full
> name
> of the class category on /etc/cloudstack/management/log4j-cloud.xml.
>
> Looking forward to your reply, thank you.
>
> Cheers.
>
>
> On Sun, Nov 6, 2016 at 5:50 AM, Sergey Levitskiy <
> sergey.levits...@autodesk.com> wrote:
>
> > You will probably need 

Re: Long delay in listDomains API response

[Cloud List]

Hi Sergey,

Thanks for your reply. May I know logging for which class I need to modify
to TRACE? Is it for the c.c.a.ApiServelet class? Not too sure what does the
c.c.a stands for -- how do I check that? So I can put the correct full name
of the class category on /etc/cloudstack/management/log4j-cloud.xml.

Looking forward to your reply, thank you.

Cheers.


On Sun, Nov 6, 2016 at 5:50 AM, Sergey Levitskiy <
sergey.levits...@autodesk.com> wrote:

> You will probably need to enable TRACE logging and see what SQL query took
> long time to execute.
>
> On 11/5/16, 1:21 PM, "Cloud List" <cloud-l...@sg.or.id> wrote:
>
> Hi,
>
> We have upgraded ACS from 4.2 to 4.8.1.1. We noted that a
> "listDomains" API
> call from our billing system took 21 seconds for our CloudStack to
> reply,
> resulting in time-out.
>
> 
> 2016-11-06 03:40:42,891 DEBUG [c.c.a.ApiServlet]
> (http-8080-2:ctx-faa0bcbd)
> (logid:86b581fc) ===START===  X.X.X.X -- GET
> apikey=XX=
> listDomains=json=YYY
> 2016-11-06 03:41:03,850 DEBUG [c.c.a.ApiServlet]
> (http-8080-2:ctx-faa0bcbd
> ctx-5653f761 ctx-cb1e1f59) (logid:86b581fc) ===END===  X.X.X.X -- GET
> apikey=XX=
> listDomains=json=YYY
> 
>
> Any advice on what caused the delay in the API reply and how we can
> expedite?
>
> Thank you.
>
>
>

Long delay in listDomains API response

[Cloud List]

Hi,

We have upgraded ACS from 4.2 to 4.8.1.1. We noted that a "listDomains" API
call from our billing system took 21 seconds for our CloudStack to reply,
resulting in time-out.


2016-11-06 03:40:42,891 DEBUG [c.c.a.ApiServlet] (http-8080-2:ctx-faa0bcbd)
(logid:86b581fc) ===START===  X.X.X.X -- GET
apikey=XX=listDomains=json=YYY
2016-11-06 03:41:03,850 DEBUG [c.c.a.ApiServlet] (http-8080-2:ctx-faa0bcbd
ctx-5653f761 ctx-cb1e1f59) (logid:86b581fc) ===END===  X.X.X.X -- GET
apikey=XX=listDomains=json=YYY


Any advice on what caused the delay in the API reply and how we can
expedite?

Thank you.

Re: Unable to start agent after roll-back to previous ACS version

[Cloud List]

ot;private", _guestBridgeName)”. Does that exist
> for you?
>
> There is also another piece of code at 1059-1071 that may also be used to
> get the private PIF name. For that, it lists the files from
> “sys/devices/virtual/net”.
>
> Can you list directories “/sys/class/net/" and “/sys/devices/virtual/net”?
>
> I will now check the source code for ACS 4.2, and see if it is the same.
>
> On Thu, Oct 20, 2016 at 12:01 PM, Cloud List <cloud-l...@sg.or.id> wrote:
>
> > Hi Rafael,
> >
> > Thanks for your reply.
> >
> > Here's the output of the command:
> >
> > root@test-kvm-03:/var/log/cloudstack/agent# ovs-vsctl list-br | sed
> > '{:q;N;s/\\n/%/g;t q}'
> > The program 'ovs-vsctl' is currently not installed.  You can install it
> by
> > typing:
> > apt-get install openvswitch-switch
> >
> > I believe the command is only applicable if we are using OpenVSwitch. We
> > are using the normal Ubuntu network bridges rather than using
> OpenVSwitch.
> > Furthermore, we are trying to roll back to 4.2 and this issue happens
> when
> > I want to start the agent after I downgraded the agent to 4.2. Shouldn't
> we
> > be checking the 4.2 source code instead?
> >
> > Looking forward to your reply, thank you.
> >
> > Cheers.
> >
> >
> > On Thu, Oct 20, 2016 at 8:38 PM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > Hi, Anonymous fellow ;)
> > > Let’s see if I can help you a little bit. I am checking the ACS 4.9
> > source
> > > code.
> > >
> > > The error is thrown at line 896 of class
> > > “com.cloud.hypervisor.kvm.resource.LibvirtComputingResource”.
> > > The condition that causes the error is “_pifs.get("private") == null”.
> > > “_pifs” if a map. The key “private” is added to the map at line 1124,
> if
> > > condition “_guestBridgeName != null && bridge.equals(_
> guestBridgeName)”
> > is
> > > met.
> > > The variable “_guestBridgeName” is a String that can receive the value
> of
> > > “guest.network.device” parameter or “_privBridgeName” variable. This
> > > process happens at lines 752-755. The process of assigning a value to
> > > “_privBridgeName” happens at lines 747-750. The default value for
> > > “_privBridgeName” is “cloudbr1”. The default can be overridden by
> > > “private.network.device” parameter.
> > >
> > > Having detailed the parameter. Let's see how ACS gets the “bridge”
> value.
> > > It gets that value from code at line 1113 “cmdout.split("%")”. The
> > variable
> > > “cmdout” contains the output of the following OS command: “ovs-vsctl
> > > list-br | sed '{:q;N;s/\\n/%/g;t q}'”.
> > >
> > > Can you run the command and check its output?
> > >
> > >
> > > On Thu, Oct 20, 2016 at 8:49 AM, Cloud List <cloud-l...@sg.or.id>
> wrote:
> > >
> > > > Hi,
> > > >
> > > > We are using ACS version 4.2 / 4.9 on our test environment. We are
> > using
> > > > Ubuntu 12.04 as the operating system and KVM as the hypervisor.
> > > >
> > > > We are trying to simulate an upgrade from ACS 4.2 to 4.9 and
> roll-back
> > > from
> > > > 4.9 to 4.2 on our test environment. The upgrade went smooth, and the
> > > > roll-back went well as well except when we need to start the agent
> > after
> > > > downgrading the agent.
> > > >
> > > > After uninstalling cloudstack-agent version 4.9 and installing back
> > > > cloudstack-agent version 4.2, I am not able to start the agent with
> > below
> > > > error messages:
> > > >
> > > > 
> > > > 2016-10-20 17:32:28,187 DEBUG [kvm.resource.
> LibvirtComputingResource]
> > > > (main:null) (logid:) looking in file /sys/devices/virtual/net/
> > > > cloud0/bridge
> > > > 2016-10-20 17:32:28,187 DEBUG [kvm.resource.
> LibvirtComputingResource]
> > > > (main:null) (logid:) Found bridge cloud0
> > > > 2016-10-20 17:32:28,188 DEBUG [kvm.resource.
> LibvirtComputingResource]
> > > > (main:null) (logid:) looking in file /sys/devices/virtual/net/lo/
> > bridge
> > > > 2016-10-20 17:32:28,188 DEBUG [kvm.resource.
> LibvirtComputingResource]
> > > > (main:null) (logid:) looking in file
> > > > /sys/devices/virtual/net/cloudbr1/bridge
> > > > 2016-10-20 17:32:28,188 DEBUG [kvm.resource.
> Libvir

Re: Unable to start agent after roll-back to previous ACS version

[Cloud List]

Hi Rafael,

Thanks for your reply.

Here's the output of the command:

root@test-kvm-03:/var/log/cloudstack/agent# ovs-vsctl list-br | sed
'{:q;N;s/\\n/%/g;t q}'
The program 'ovs-vsctl' is currently not installed.  You can install it by
typing:
apt-get install openvswitch-switch

I believe the command is only applicable if we are using OpenVSwitch. We
are using the normal Ubuntu network bridges rather than using OpenVSwitch.
Furthermore, we are trying to roll back to 4.2 and this issue happens when
I want to start the agent after I downgraded the agent to 4.2. Shouldn't we
be checking the 4.2 source code instead?

Looking forward to your reply, thank you.

Cheers.


On Thu, Oct 20, 2016 at 8:38 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Hi, Anonymous fellow ;)
> Let’s see if I can help you a little bit. I am checking the ACS 4.9 source
> code.
>
> The error is thrown at line 896 of class
> “com.cloud.hypervisor.kvm.resource.LibvirtComputingResource”.
> The condition that causes the error is “_pifs.get("private") == null”.
> “_pifs” if a map. The key “private” is added to the map at line 1124, if
> condition “_guestBridgeName != null && bridge.equals(_guestBridgeName)” is
> met.
> The variable “_guestBridgeName” is a String that can receive the value of
> “guest.network.device” parameter or “_privBridgeName” variable. This
> process happens at lines 752-755. The process of assigning a value to
> “_privBridgeName” happens at lines 747-750. The default value for
> “_privBridgeName” is “cloudbr1”. The default can be overridden by
> “private.network.device” parameter.
>
> Having detailed the parameter. Let's see how ACS gets the “bridge” value.
> It gets that value from code at line 1113 “cmdout.split("%")”. The variable
> “cmdout” contains the output of the following OS command: “ovs-vsctl
> list-br | sed '{:q;N;s/\\n/%/g;t q}'”.
>
> Can you run the command and check its output?
>
>
> On Thu, Oct 20, 2016 at 8:49 AM, Cloud List <cloud-l...@sg.or.id> wrote:
>
> > Hi,
> >
> > We are using ACS version 4.2 / 4.9 on our test environment. We are using
> > Ubuntu 12.04 as the operating system and KVM as the hypervisor.
> >
> > We are trying to simulate an upgrade from ACS 4.2 to 4.9 and roll-back
> from
> > 4.9 to 4.2 on our test environment. The upgrade went smooth, and the
> > roll-back went well as well except when we need to start the agent after
> > downgrading the agent.
> >
> > After uninstalling cloudstack-agent version 4.9 and installing back
> > cloudstack-agent version 4.2, I am not able to start the agent with below
> > error messages:
> >
> > 
> > 2016-10-20 17:32:28,187 DEBUG [kvm.resource.LibvirtComputingResource]
> > (main:null) (logid:) looking in file /sys/devices/virtual/net/
> > cloud0/bridge
> > 2016-10-20 17:32:28,187 DEBUG [kvm.resource.LibvirtComputingResource]
> > (main:null) (logid:) Found bridge cloud0
> > 2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
> > (main:null) (logid:) looking in file /sys/devices/virtual/net/lo/bridge
> > 2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
> > (main:null) (logid:) looking in file
> > /sys/devices/virtual/net/cloudbr1/bridge
> > 2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
> > (main:null) (logid:) Found bridge cloudbr1
> > 2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
> > (main:null) (logid:) looking in file /sys/devices/virtual/net/
> vnet4/bridge
> > 2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
> > (main:null) (logid:) looking in file /sys/devices/virtual/net/
> vnet5/bridge
> > 2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
> > (main:null) (logid:) looking in file /sys/devices/virtual/net/
> vnet6/bridge
> > 2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
> > (main:null) (logid:) looking in file /sys/devices/virtual/net/
> vnet7/bridge
> > 2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
> > (main:null) (logid:) looking in file /sys/devices/virtual/net/
> vnet8/bridge
> > 2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
> > (main:null) (logid:) looking in file /sys/devices/virtual/net/
> vnet9/bridge
> > 2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
> > (main:null) (logid:) looking in file /sys/devices/virtual/net/
> vnet0/bridge
> > 2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
> > (main:null) (logid:) looking in file /sys/devices/virtual/net/
> vnet1/bridge
> > 20

Re: Avoid VR downtime when upgrading ACS

[Cloud List]

Hi,

Sorry to bring up an old question, just want to ask again if anyone knows
if we can upgrade ACS and avoid the downtime caused when the VRs are being
upgraded?

Looking forward to your reply, thank you.

Cheers.

-ip-


On Mon, Jul 4, 2016 at 10:57 AM, Cloud List <cloud-l...@sg.or.id> wrote:

> Dear all,
>
> I have done some extensive tests on upgrading ACS from 4.2 to 4.8, and one
> of the steps involves upgrading the virtual routers (VRs) which will cause
> downtime to network served by the VRs. Is there a way I can avoid such
> downtime?
>
> Will having redundant VRs for every network help to avoid the downtime?
>
> Looking forward to your reply, thank youi.
>
> Cheers.
>
> -ip-
>

Unable to start agent after roll-back to previous ACS version

[Cloud List]

Hi,

We are using ACS version 4.2 / 4.9 on our test environment. We are using
Ubuntu 12.04 as the operating system and KVM as the hypervisor.

We are trying to simulate an upgrade from ACS 4.2 to 4.9 and roll-back from
4.9 to 4.2 on our test environment. The upgrade went smooth, and the
roll-back went well as well except when we need to start the agent after
downgrading the agent.

After uninstalling cloudstack-agent version 4.9 and installing back
cloudstack-agent version 4.2, I am not able to start the agent with below
error messages:


2016-10-20 17:32:28,187 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/cloud0/bridge
2016-10-20 17:32:28,187 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) Found bridge cloud0
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/lo/bridge
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file
/sys/devices/virtual/net/cloudbr1/bridge
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) Found bridge cloudbr1
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet4/bridge
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet5/bridge
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet6/bridge
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet7/bridge
2016-10-20 17:32:28,188 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet8/bridge
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet9/bridge
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet0/bridge
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet1/bridge
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet2/bridge
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet3/bridge
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/virbr0/bridge
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) Found bridge virbr0
2016-10-20 17:32:28,189 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet11/bridge
2016-10-20 17:32:28,190 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet12/bridge
2016-10-20 17:32:28,190 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet13/bridge
2016-10-20 17:32:28,190 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet14/bridge
2016-10-20 17:32:28,190 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking in file /sys/devices/virtual/net/vnet10/bridge
2016-10-20 17:32:28,190 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking for pif for bridge cloud0
2016-10-20 17:32:28,190 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) matchPifFileInDirectory: file name 'vnet1'
2016-10-20 17:32:28,191 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) matchPifFileInDirectory: file name 'vnet2'
2016-10-20 17:32:28,191 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) matchPifFileInDirectory: file name 'vnet5'
2016-10-20 17:32:28,191 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) failing to get physical interface from bridge cloud0,
did not find an eth*, bond*, vlan*, em*, or p*p* in
/sys/devices/virtual/net/cloud0/brif
2016-10-20 17:32:28,191 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking for pif for bridge cloudbr1
2016-10-20 17:32:28,191 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) matchPifFileInDirectory: file name 'eth1'
2016-10-20 17:32:28,192 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) looking for pif for bridge virbr0
2016-10-20 17:32:28,192 DEBUG [kvm.resource.LibvirtComputingResource]
(main:null) (logid:) failing to get physical interface from bridge virbr0,
did not find an eth*, bond*, vlan*, em*, or p*p* in

Avoid VR downtime when upgrading ACS

[Cloud List]

Dear all,

I have done some extensive tests on upgrading ACS from 4.2 to 4.8, and one
of the steps involves upgrading the virtual routers (VRs) which will cause
downtime to network served by the VRs. Is there a way I can avoid such
downtime?

Will having redundant VRs for every network help to avoid the downtime?

Looking forward to your reply, thank youi.

Cheers.

-ip-

ERROR: Java process not running. Try restarting the SSVM.

[Cloud List]

Dear all,

I am running ACS 4.2.0 in our test environment and after restarting the KVM
host, I am not able to get SSVM agent up and running, despite destroying
and recreating it.

The problem is similar to what Mike Tutkowsi reported on below thread, same
symptoms but different cloud.out stack error message:

http://comments.gmane.org/gmane.comp.apache.cloudstack.devel/28827

I can SSH into SSVM but unable to start the Java agent. SSVM health check
result shows two error:
- Unable to connect NFS (wrong IP detected); and
- Unable to start Java.

Unable to connect NFS is because wrong IP is detected for the NFS ip. I can
manually mount the NFS server IP using correct IP.

Result:


root@s-121-VM:/var/log/cloud# /usr/local/cloud/systemvm/ssvm-check.sh

First DNS server is  8.8.8.8
PING 8.8.8.8 (8.8.8.8): 56 data bytes
64 bytes from 8.8.8.8: icmp_seq=0 ttl=49 time=4.572 ms
64 bytes from 8.8.8.8: icmp_seq=1 ttl=49 time=4.365 ms
--- 8.8.8.8 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 4.365/4.468/4.572/0.104 ms
Good: Can ping DNS server

Good: DNS resolves download.cloud.com

ERROR: NFS is not currently mounted
Try manually mounting from inside the VM
NFS server is  169.254.0.127
PING 169.254.0.127 (169.254.0.127): 56 data bytes
64 bytes from 169.254.0.127: icmp_seq=0 ttl=64 time=0.106 ms
64 bytes from 169.254.0.127: icmp_seq=1 ttl=64 time=0.040 ms
--- 169.254.0.127 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.040/0.073/0.106/0.033 ms
Good: Can ping NFS server

Management server is 192.168.0.222. Checking connectivity.
Good: Can connect to management server port 8250

ERROR: Java process not running.  Try restarting the SSVM.


Repeated error shown on /var/log/cloud/cloud.out:


+ java -Djavax.net.ssl.trustStore=./certs/realhostip.keystore
-Dlog.home=/var/log/cloud/ -mx-24m -cp
./:./conf:activation-1.1.jar:antisamy-1.4.3.jar:aopalliance-1.0.jar:apache-log4j-extras-1.1.jar:aspectjrt-1.7.1.jar:aspectjweaver-1.7.1.jar:aws-java-sdk-1.3.22.jar:backport-util-concurrent-3.1.jar:batik-css-1.7.jar:batik-ext-1.7.jar:batik-util-1.7.jar:bcprov-jdk16-1.46.jar:bsh-core-2.0b4.jar:c3p0-0.9.1.1.jar:cglib-nodep-2.2.2.jar:cloud-agent-4.2.0.jar:cloud-api-4.2.0.jar:cloud-console-proxy-4.2.0.jar:cloud-core-4.2.0.jar:cloud-engine-api-4.2.0.jar:cloud-engine-components-api-4.2.0.jar:cloud-engine-schema-4.2.0.jar:cloud-framework-events-4.2.0.jar:cloud-framework-ipc-4.2.0.jar:cloud-framework-jobs-4.2.0.jar:cloud-framework-rest-4.2.0.jar:cloud-secondary-storage-4.2.0.jar:cloud-server-4.2.0.jar:cloud-utils-4.2.0.jar:commons-beanutils-core-1.7.0.jar:commons-codec-1.6.jar:commons-collections-3.2.1.jar:commons-configuration-1.8.jar:commons-daemon-1.0.10.jar:commons-dbcp-1.4.jar:commons-discovery-0.5.jar:commons-fileupload-1.2.jar:commons-httpclient-3.1.jar:commons-io-1.4.jar:commons-lang-2.6.jar:commons-logging-1.1.1.jar:commons-pool-1.6.jar:cxf-bundle-jaxrs-2.7.0.jar:dom4j-1.6.1.jar:ehcache-1.5.0.jar:ejb-api-3.0.jar:esapi-2.0.1.jar:geronimo-javamail_1.4_spec-1.7.1.jar:geronimo-servlet_3.0_spec-1.0.jar:gson-1.7.1.jar:guava-11.0.2.jar:httpclient-4.2.1.jar:httpcore-4.2.1.jar:jackson-annotations-2.1.1.jar:jackson-core-2.1.1.jar:jackson-core-asl-1.8.9.jar:jackson-databind-2.1.1.jar:jackson-jaxrs-json-provider-2.1.1.jar:jackson-mapper-asl-1.8.9.jar:jackson-module-jaxb-annotations-2.1.1.jar:jasypt-1.9.0.jar:java-ipv6-0.10.jar:javassist-3.12.1.GA.jar:javax.inject-1.jar:javax.persistence-2.0.0.jar:javax.ws.rs-api-2.0-m10.jar:jaxen-1.1-beta-8.jar:jdom-1.0.jar:jetty-continuation-8.1.7.v20120910.jar:jetty-http-8.1.7.v20120910.jar:jetty-io-8.1.7.v20120910.jar:jetty-security-8.1.7.v20120910.jar:jetty-util-8.1.7.v20120910.jar:jsch-0.1.42.jar:jsr107cache-1.0.jar:jsr305-1.3.9.jar:jstl-1.2.jar:log4j-1.2.16.jar:mail-1.4.jar:nekohtml-1.9.12.jar:quartz-2.1.6.jar:reflections-0.9.8.jar:slf4j-api-1.6.2.jar:spring-aop-3.1.2.RELEASE.jar:spring-asm-3.1.2.RELEASE.jar:spring-beans-3.1.2.RELEASE.jar:spring-context-3.1.2.RELEASE.jar:spring-core-3.1.2.RELEASE.jar:spring-expression-3.1.2.RELEASE.jar:spring-web-3.1.2.RELEASE.jar:stax2-api-3.1.1.jar:trilead-ssh2-build213-svnkit-1.3-patch.jar:woodstox-core-asl-4.1.4.jar:wsdl4j-1.6.2.jar:xalan-2.7.0.jar:xercesImpl-2.6.2.jar:xml-apis-1.0.b2.jar:xml-apis-ext-1.3.04.jar:xmlParserAPIs-2.6.2.jar:xmlschema-core-2.0.3.jar:xom-1.1.jar:xpp3_min-1.1.4c.jar:xstream-1.3.1.jar
com.cloud.agent.AgentShell template=domP type=secstorage host=192.168.0.222
port=8250 name=s-121-VM zone=1 pod=1 guid=s-121-VM
resource=org.apache.cloudstack.storage.resource.NfsSecondaryStorageResource
instance=SecStorage sslcopy=true role=templateProcessor mtu=1500
eth2ip=192.168.5.170 

Re: CloudStack 4.8.0 with LXC - LibvirtException: unsupported configuration: System lacks NETNS support

[Cloud List]

Dear all,

Anyone implemented LXC on CloudStack before and can provide some advice?
Documentation is a bit limited on this area. Other than below
documentations, any other pointers anyone can provide?

http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.8/hypervisor/lxc.html
https://cwiki.apache.org/confluence/display/CLOUDSTACK/LXC+Template+creation
https://cwiki.apache.org/confluence/display/CLOUDSTACK/LXC+Support+in+Cloudstack
https://cwiki.apache.org/confluence/display/CLOUDSTACK/LXC+Enhancements

Looking forward to your reply, thank you.

Cheers.

-ip-


On Fri, Jun 10, 2016 at 6:39 PM, Cloud List <cloud-l...@sg.or.id> wrote:

> Dear all,
>
> We tried to setup an LXC cluster and add an LXC host into a test
> CloudStack 4.8.0.1 environment. The cluster is created and the LXC host is
> created successfully. We also tried to follow below documentation to create
> an Ubuntu LXC template -- we are running Ubuntu 12.04.5 LTS.
>
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/LXC+Template+creation
>
> ===
> Install lxc: apt-get install lxc
>
> Create  a container with any name, in our case 'ubuntu':
>
> sudo lxc-create -t download -n ubuntu -- --dist ubuntu --release trusty
> --arch amd64
>
> Creation of the LXC container will take a lot of time, it will be stored
> here: /var/lib/lxc/ubuntu
>
> Next stop the container:
> sudo lxc-stop -n ubuntu
>
> Next create a temp director, tar the rootfs and export the template:
> mkdir -p /tmp/lxc-template
> cd /tmp/lxc-template
> sudo tar --numeric-owner -czf /tmp/lxc-template/template.tar.gz
> /var/lib/lxc/ubuntu/rootfs/
> ===
>
> As per suggestion from Rohit on another thread, I saved the template file
> as .tar and register it to cloudstack with HVM disabled. I then tried to
> create an LXC container instance using the template, but after a long
> "Creating" process, the provisioning will fail with insufficient capacity
> error.
>
> ===
> 2016-06-10 18:24:45,468 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
> (API-Job-Executor-44:ctx-c8a4afdc job-232) (logid:523a0c37) Complete async
> job-232, jobStatus: FAILED, resultCode: 530, result:
> org.apache.cloudstack.api.response.ExceptionR
> esponse/null/{"uuidList":[],"errorcode":530,"errortext":"Unable to start a
> VM due to insufficient capacity"}
> ===
>
> Further check on the LXC host's agent.log shows that the provisioning
> failed due to libvirt exception, which says that the configuration is
> unsupported because the system lacks NETNS support
>
> "LibvirtException: unsupported configuration: System lacks NETNS support"
>
> See excerpts of the logs below:
>
> ===
> 2016-06-10 18:24:43,740 DEBUG
> [resource.wrapper.LibvirtStartCommandWrapper] (agentRequest-Handler-1:null)
> starting i-2-42-VM: 
> i-2-42-VM
> c29a74fb-e4bc-4aa2-bef4-3c8844de87df
> Ubuntu 14.04 (64-bit)
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
>dir='/mnt/d7678837-9f9f-32ab-99f3-b508feb3595a/6d26a879-999f-4cba-bfcc-1d2d9e70af5c'/>
>   
> 
> 
> 
> 
> 
> 
> 
> 
> 1048576
> 
> 
> 
> 1
> 
> exe
> /sbin/init
> 
> 
> 1000
> 
> restart
> destroy
> destroy
> 
>
> 2016-06-10 18:24:43,743 WARN
> [resource.wrapper.LibvirtStartCommandWrapper] (agentRequest-Handler-1:null)
> LibvirtException
> org.libvirt.LibvirtException: unsupported configuration: System lacks
> NETNS support
> at org.libvirt.ErrorHandler.processError(Unknown Source)
> at org.libvirt.ErrorHandler.processError(Unknown Source)
> at org.libvirt.Connect.domainCreateXML(Unknown Source)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.startVM(LibvirtComputingResource.java:1293)
> at
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStartCommandWrapper.execute(LibvirtStartCommandWrapper.java:82)
> at
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStartCommandWrapper.execute(LibvirtStartCommandWrapper.java:46)
> at
> com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:75)
> at
> com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1317)
> at com.cloud.agent.Agent.processRequest(Agent.java:522)
> at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:830)
> at com.cloud.utils.nio.Task.call(Task.java:83)
> at com.cloud.utils.nio.Task.call(Task.java:29)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(Thr

CloudStack 4.8.0 with LXC - LibvirtException: unsupported configuration: System lacks NETNS support

[Cloud List]

Dear all,

We tried to setup an LXC cluster and add an LXC host into a test CloudStack
4.8.0.1 environment. The cluster is created and the LXC host is created
successfully. We also tried to follow below documentation to create an
Ubuntu LXC template -- we are running Ubuntu 12.04.5 LTS.

https://cwiki.apache.org/confluence/display/CLOUDSTACK/LXC+Template+creation

===
Install lxc: apt-get install lxc

Create  a container with any name, in our case 'ubuntu':

sudo lxc-create -t download -n ubuntu -- --dist ubuntu --release trusty
--arch amd64

Creation of the LXC container will take a lot of time, it will be stored
here: /var/lib/lxc/ubuntu

Next stop the container:
sudo lxc-stop -n ubuntu

Next create a temp director, tar the rootfs and export the template:
mkdir -p /tmp/lxc-template
cd /tmp/lxc-template
sudo tar --numeric-owner -czf /tmp/lxc-template/template.tar.gz
/var/lib/lxc/ubuntu/rootfs/
===

As per suggestion from Rohit on another thread, I saved the template file
as .tar and register it to cloudstack with HVM disabled. I then tried to
create an LXC container instance using the template, but after a long
"Creating" process, the provisioning will fail with insufficient capacity
error.

===
2016-06-10 18:24:45,468 DEBUG [o.a.c.f.j.i.AsyncJobManagerImpl]
(API-Job-Executor-44:ctx-c8a4afdc job-232) (logid:523a0c37) Complete async
job-232, jobStatus: FAILED, resultCode: 530, result:
org.apache.cloudstack.api.response.ExceptionR
esponse/null/{"uuidList":[],"errorcode":530,"errortext":"Unable to start a
VM due to insufficient capacity"}
===

Further check on the LXC host's agent.log shows that the provisioning
failed due to libvirt exception, which says that the configuration is
unsupported because the system lacks NETNS support

"LibvirtException: unsupported configuration: System lacks NETNS support"

See excerpts of the logs below:

===
2016-06-10 18:24:43,740 DEBUG [resource.wrapper.LibvirtStartCommandWrapper]
(agentRequest-Handler-1:null) starting i-2-42-VM: 
i-2-42-VM
c29a74fb-e4bc-4aa2-bef4-3c8844de87df
Ubuntu 14.04 (64-bit)

















  
  








1048576



1

exe
/sbin/init


1000

restart
destroy
destroy


2016-06-10 18:24:43,743 WARN  [resource.wrapper.LibvirtStartCommandWrapper]
(agentRequest-Handler-1:null) LibvirtException
org.libvirt.LibvirtException: unsupported configuration: System lacks NETNS
support
at org.libvirt.ErrorHandler.processError(Unknown Source)
at org.libvirt.ErrorHandler.processError(Unknown Source)
at org.libvirt.Connect.domainCreateXML(Unknown Source)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.startVM(LibvirtComputingResource.java:1293)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStartCommandWrapper.execute(LibvirtStartCommandWrapper.java:82)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtStartCommandWrapper.execute(LibvirtStartCommandWrapper.java:46)
at
com.cloud.hypervisor.kvm.resource.wrapper.LibvirtRequestWrapper.execute(LibvirtRequestWrapper.java:75)
at
com.cloud.hypervisor.kvm.resource.LibvirtComputingResource.executeRequest(LibvirtComputingResource.java:1317)
at com.cloud.agent.Agent.processRequest(Agent.java:522)
at com.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:830)
at com.cloud.utils.nio.Task.call(Task.java:83)
at com.cloud.utils.nio.Task.call(Task.java:29)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-06-10 18:24:43,744 DEBUG [kvm.storage.KVMStoragePoolManager]
(agentRequest-Handler-1:null) Disconnecting disk
6d26a879-999f-4cba-bfcc-1d2d9e70af5c
2016-06-10 18:24:43,744 INFO  [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-1:null) Trying to fetch storage pool
d7678837-9f9f-32ab-99f3-b508feb3595a from libvirt
2016-06-10 18:24:43,744 DEBUG [kvm.resource.LibvirtConnection]
(agentRequest-Handler-1:null) Looking for libvirtd connection at: lxc:///
2016-06-10 18:24:43,751 DEBUG [kvm.storage.LibvirtStorageAdaptor]
(agentRequest-Handler-1:null) Succesfully refreshed pool
d7678837-9f9f-32ab-99f3-b508feb3595a Capacity: 483387244544 Used:
127107334144 Available: 356279910400
2016-06-10 18:24:43,757 DEBUG [cloud.agent.Agent]
(agentRequest-Handler-1:null) Seq 7-8790463522673787745:  { Ans: , MgmtId:
181122461655966, via: 7, Ver: v1, Flags: 10,
[{"com.cloud.agent.api.StartAnswer":{"vm":{"id":42,"name":"i-2-42-VM","type":"User","cpus":1,"minSpeed":1000,"maxSpeed":2000,"minRam":1073741824,"maxRam":1073741824,"arch":"x86_64","os":"Ubuntu
14.04 (64-bit)","platformEmulator":"Ubuntu

Re: LXC container template for CloudStack

[Cloud List]

Thanks Kishan.

Have tried downloading the template and create a container on my LXC
cluster (host is running Ubuntu), and it fails. Not too sure if it's
because I tried to spin off a CentOS 7 container on an Ubuntu 12.04 LXC
host?I understand that for container, the kernel of the guest container
must be the same as the host OS?

Thank you.


On Wed, Jun 8, 2016 at 4:40 PM, Kishan Kavala <kishan.kav...@accelerite.com>
wrote:

> You can try this centos 7 template for LXC:
> http://download.cloud.com/templates/builtin/centos-7-x86_64.tar.gz
>
> -Original Message-----
> From: Cloud List [mailto:cloud-l...@sg.or.id]
> Sent: Wednesday, June 08, 2016 1:35 PM
> To: us...@cloudstack.apache.org; dev@cloudstack.apache.org
> Subject: Re: LXC container template for CloudStack
>
> Dear all,
>
> Anyone can point me to any URLs containing ready-to-use LXC container
> template for Cloudstack, ready for download?
>
> Thank you.
>
> -ip-
>
>
> On Mon, Jun 6, 2016 at 2:21 PM, Cloud List <cloud-l...@sg.or.id> wrote:
>
> > Dear all,
> >
> > I am trying to add an LXC cluster in addition to the existing running
> > KVM cluster in my CloudStack 4.8.0 test environment.
> >
> >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/LXC+Support+in+
> > Cloudstack
> >
> > I understand that the LXC container template for CloudStack is not
> > provided / bundled by default, so we need to create one or download.
> > May I
> > know:
> >
> > (1) How do we create one? I follow the instruction on how to create
> > LXC template for Ubuntu here:
> >
> >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/LXC+Template+cr
> > eation#LXCTemplatecreation-CreatingtemplateonUbuntu
> >
> > It requires us to install lxc (apt-get install lxc) but doing so will
> > remove cloudstack-agent package hosted on the same host.
> >
> > (2) Where can we download ready-to-use LXC container template for
> > CloudStack?
> >
> > Looking forward to your reply, thank you.
> >
> > Cheers.
> >
> > -ip-
> >
>
>
>
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which is
> the property of Accelerite, a Persistent Systems business. It is intended
> only for the use of the individual or entity to which it is addressed. If
> you are not the intended recipient, you are not authorized to read, retain,
> copy, print, distribute or use this message. If you have received this
> communication in error, please notify the sender and delete all copies of
> this message. Accelerite, a Persistent Systems business does not accept any
> liability for virus infected mails.
>

Re: LXC container template for CloudStack

[Cloud List]

Dear all,

Anyone can point me to any URLs containing ready-to-use LXC container
template for Cloudstack, ready for download?

Thank you.

-ip-


On Mon, Jun 6, 2016 at 2:21 PM, Cloud List <cloud-l...@sg.or.id> wrote:

> Dear all,
>
> I am trying to add an LXC cluster in addition to the existing running KVM
> cluster in my CloudStack 4.8.0 test environment.
>
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/LXC+Support+in+Cloudstack
>
> I understand that the LXC container template for CloudStack is not
> provided / bundled by default, so we need to create one or download. May I
> know:
>
> (1) How do we create one? I follow the instruction on how to create LXC
> template for Ubuntu here:
>
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/LXC+Template+creation#LXCTemplatecreation-CreatingtemplateonUbuntu
>
> It requires us to install lxc (apt-get install lxc) but doing so will
> remove cloudstack-agent package hosted on the same host.
>
> (2) Where can we download ready-to-use LXC container template for
> CloudStack?
>
> Looking forward to your reply, thank you.
>
> Cheers.
>
> -ip-
>

Re: Upgrading from ACS 4.2 to 4.8 - VR upgrade issue

[Cloud List]

Dear all,

Could it be the VR is not able to use the new systemVM template because we
registered the template as Routing = NO? Will it resolve the problem if
during upgrade, we register the template as Routing = YES?

Based on the upgrade instruction at
http://cloudstack-release-notes.readthedocs.io/ja/stable/upgrade/upgrade-4.2.html
, it says that routing option should be "no", is this correct? Since the
systemVM template will also be used by VR?

Any advice is appreciated.

Looking forward to your reply, thank you.

Cheers.

-ip-


On Tue, Jun 7, 2016 at 10:22 AM, Cloud List <cloud-l...@sg.or.id> wrote:

> Hi Kirk,
>
> Thanks for your reply.
>
> Here's the status of the CPVM and SSVM in the UI:
>
> Namev-2-VM
> IDfc6d23bf-58ac-4641-af9b-506d1dc46df3
> StateRunning
> TypeConsole Proxy VM
> Zonesc-test-cs42-zone
> Public IP Address192.168.0.162
> Private IP Address192.168.0.160
> Link Local IP Address169.254.3.147
> Hostsc-test-kvm-03
> Gateway192.168.0.1
> Created31 May 2016 15:24:20
> Active Sessions0
>
> Names-29-VM
> IDd26f54f8-1c1f-4f3d-9862-fc0405159620
> StateRunning
> TypeSecondary Storage VM
> Zonesc-test-cs42-zone
> Public IP Address192.168.0.161
> Private IP Address192.168.0.151
> Link Local IP Address169.254.1.207
> Hostsc-test-kvm-03
> Gateway192.168.0.1
> Created06 Jun 2016 00:31:04
> Active Sessions
>
> Both system VMs seem to be on "Running" state but I can't verify what
> template version they are using, whether they are already on 4.6/4.8
> template or still using 4.2 template, are you able to advise how can I
> tell? The SSVM creation date seem to be after we performed the upgrade but
> the CPVM created date is before the upgrade.
>
> How to ensure that all systemVMs using the new version template? Shall I
> destroy the system VMs so new ones will be created, or shall I re-run the
> /usr/bin/cloudstack-sysvmadm again in full?
>
> Looking forward to your reply, thank you.
>
> Cheers.
>
> -ip-
>
>
>
> On Tue, Jun 7, 2016 at 7:22 AM, Kirk Kosinski <kirk.kosin...@shapeblue.com
> > wrote:
>
>> Hi, what is the status of the CPVM and SSVM in the UI?  Were they really
>> upgraded successfully?
>>
>> Also, are there any errors in the management-server.log?
>>
>> kirk.kosin...@shapeblue.com
>> www.shapeblue.com
>> 53 Chandos Place, Covent Garden, London  WC2N 4HS
>> @shapeblue
>>
>>
>>
>> -Original Message-
>> From: Cloud List [mailto:cloud-l...@sg.or.id]
>> Sent: Monday, June 06, 2016 9:51 AM
>> To: us...@cloudstack.apache.org; dev@cloudstack.apache.org
>> Subject: Upgrading from ACS 4.2 to 4.8 - VR upgrade issue
>>
>> Hi,
>>
>> I tried upgrading our CloudStack 4.2.0 on our test environment to 4.8.0,
>> and I am having issues on VR upgrade. I followed the upgrade instructions
>> at below URL:
>>
>>
>> http://cloudstack-release-notes.readthedocs.io/ja/stable/upgrade/upgrade-4.2.html
>>
>> On the last step to restart all the system VMs and VRs, it seems that the
>> cloudstack-sysvmadm script managed to restart the system VMs (SSVM and
>> CPVM) but not the VR.
>>
>> # nohup cloudstack-sysvmadm -d IPaddress -u cloud -p password -a >
>> sysvm.log 2>&1 &
>>
>> Here's the output of the sysvm.log file:
>>
>> 
>> nohup: ignoring input
>> /usr/bin/cloudstack-sysvmadm: line 21: /etc/rc.d/init.d/functions: No
>> such file or directory
>>
>> Stopping and starting 1 secondary storage vm(s)...
>> Done stopping and starting secondary storage vm(s)
>>
>> Stopping and starting 1 console proxy vm(s)...
>> Done stopping and starting console proxy vm(s) .
>>
>> Stopping and starting 1 running routing vm(s)...
>> *ERROR: Failed to restart domainRouter with id 5*
>>
>> Done restarting router(s).
>> 
>>
>> Output of the cloud.log file:
>>
>> 
>> [2016.06.05-22.28.40] Stopping and starting 1 secondary storage vm(s)...
>> [2016.06.05-22.28.40] INFO: Stopping secondary storage vm with id 3
>> [2016.06.05-22.28.46] INFO: Starting secondary storage vm with id 3
>> [2016.06.05-23.29.00] ERROR: Failed to startSystemVm id=3; jobId is
>> 121c5d89-745a-4e8e-9f96-0b348ef42abf
>> [2016.06.05-23.29.00] ERROR: Failed to start secondary storage vm with id
>> 3 [2016.06.05-23.29.00] ERROR: Failed to start secondary storage vm with id
>> 3 [2016.06.05-23.29.00] Done stopping and starting secondary storage vm(s).
>> [2016.06.05-23.29.00] Stopp

Upgrading from ACS 4.2 to 4.8 - VR upgrade issue

[Cloud List]

Hi,

I tried upgrading our CloudStack 4.2.0 on our test environment to 4.8.0,
and I am having issues on VR upgrade. I followed the upgrade instructions
at below URL:

http://cloudstack-release-notes.readthedocs.io/ja/stable/upgrade/upgrade-4.2.html

On the last step to restart all the system VMs and VRs, it seems that the
cloudstack-sysvmadm script managed to restart the system VMs (SSVM and
CPVM) but not the VR.

# nohup cloudstack-sysvmadm -d IPaddress -u cloud -p password -a >
sysvm.log 2>&1 &

Here's the output of the sysvm.log file:


nohup: ignoring input
/usr/bin/cloudstack-sysvmadm: line 21: /etc/rc.d/init.d/functions: No such
file or directory

Stopping and starting 1 secondary storage vm(s)...
Done stopping and starting secondary storage vm(s)

Stopping and starting 1 console proxy vm(s)...
Done stopping and starting console proxy vm(s) .

Stopping and starting 1 running routing vm(s)...
*ERROR: Failed to restart domainRouter with id 5*

Done restarting router(s).


Output of the cloud.log file:


[2016.06.05-22.28.40] Stopping and starting 1 secondary storage vm(s)...
[2016.06.05-22.28.40] INFO: Stopping secondary storage vm with id 3
[2016.06.05-22.28.46] INFO: Starting secondary storage vm with id 3
[2016.06.05-23.29.00] ERROR: Failed to startSystemVm id=3; jobId is
121c5d89-745a-4e8e-9f96-0b348ef42abf
[2016.06.05-23.29.00] ERROR: Failed to start secondary storage vm with id 3
[2016.06.05-23.29.00] ERROR: Failed to start secondary storage vm with id 3
[2016.06.05-23.29.00] Done stopping and starting secondary storage vm(s).
[2016.06.05-23.29.00] Stopping and starting 1 console proxy vm(s)...
[2016.06.05-23.29.00] INFO: Stopping console proxy with id 2
[2016.06.05-23.29.06] INFO: Starting console proxy vm with id 2
[2016.06.05-23.30.49] Done stopping and starting console proxy vm(s) .
[2016.06.05-23.30.49] Stopping and starting 1 running routing vm(s)...
[2016.06.05-23.30.49] INFO: Restarting router with id 5
[2016.06.06-00.30.59] ERROR: Failed to restart domainRouter with id 5;
jobId 4a3f681e-c4a6-4856-9d42-0d3b79bffdae
[2016.06.06-00.31.04] Done restarting router(s).


Initially I noted that the VR remains in 4.2.0 version (instead of the
expected 4.8.0 version), tried to destroy the router so it'll be recreated
but it doesn't get created successfully.

Tried to run the cloudstack-sysvmadm manually with the -r option to restart
only the VR, and the problem still persists.

At the moment, the version of the VR is "UNKNOWN" and the "Require Upgrade"
state is "Yes".

Any advice how to make the VR back up and running?

Looking forward to your reply, thank you.

Cheers.

-ip-

LXC container template for CloudStack

[Cloud List]

Dear all,

I am trying to add an LXC cluster in addition to the existing running KVM
cluster in my CloudStack 4.8.0 test environment.

https://cwiki.apache.org/confluence/display/CLOUDSTACK/LXC+Support+in+Cloudstack

I understand that the LXC container template for CloudStack is not provided
/ bundled by default, so we need to create one or download. May I know:

(1) How do we create one? I follow the instruction on how to create LXC
template for Ubuntu here:

https://cwiki.apache.org/confluence/display/CLOUDSTACK/LXC+Template+creation#LXCTemplatecreation-CreatingtemplateonUbuntu

It requires us to install lxc (apt-get install lxc) but doing so will
remove cloudstack-agent package hosted on the same host.

(2) Where can we download ready-to-use LXC container template for
CloudStack?

Looking forward to your reply, thank you.

Cheers.

-ip-

Re: Older Ceph packages for Ubuntu 12.04 (Precise Pangolin) to recompile libvirt with RBD support

[Cloud List]

Hi,

Anyone can assist on this?

Looking forward to your reply, thank you.

Cheers.


On Fri, Jun 3, 2016 at 11:56 AM, Cloud List <cloud-l...@sg.or.id> wrote:

> Dear all,
>
> I am trying to setup older version of CloudStack 4.2.0 on Ubuntu 12.04 to
> use Ceph RBD as primary storage for our upgrade testing purposes. Two of
> the steps involved were to add below repository to manually compile libvirt
> to have RBD support, since the default libvirt on Ubuntu 12.04 doesn't have
> RBD support by default, unlike on Ubuntu 14.04:
>
> 
> # wget -q -O- '
> https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc' | sudo
> apt-key add -
> OK
>
> # echo deb http://eu.ceph.com/debian-cuttlefish/ $(lsb_release -sc) main
> | sudo tee /etc/apt/sources.list.d/ceph.list
> deb http://eu.ceph.com/debian-cuttlefish/ precise main
> 
>
> But when I ran sudo apt-get update, I am receiving this error (excerpts):
>
> 
> Err http://eu.ceph.com precise/main amd64 Packages
>   404  Not Found
> Err http://eu.ceph.com precise/main i386 Packages
>   404  Not Found
>
> W: Failed to fetch
> http://eu.ceph.com/debian-cuttlefish/dists/precise/main/binary-amd64/Packages
> 404  Not Found
>
> W: Failed to fetch
> http://eu.ceph.com/debian-cuttlefish/dists/precise/main/binary-i386/Packages
> 404  Not Found
>
> E: Some index files failed to download. They have been ignored, or old
> ones used instead.
> 
>
> It seems that the repository for the particular required packages has been
> removed, anyone can advise if I can get the required packages, may be from
> a different location?
>
> Any help is greatly appreciated.
>
> Looking forward to your reply, thank you.
>
> Cheers.
>
> -ip-
>

Re: Older Ceph packages for Ubuntu 12.04 (Precise Pangolin) to recompile libvirt with RBD support

[Cloud List]

Hi Wido,

Thanks for your reply.

On Fri, Jun 3, 2016 at 3:27 PM, Wido den Hollander <w...@widodh.nl> wrote:

>
> > Op 3 juni 2016 om 5:56 schreef Cloud List <cloud-l...@sg.or.id>:
> >
> >
> > Dear all,
> >
> > I am trying to setup older version of CloudStack 4.2.0 on Ubuntu 12.04 to
> > use Ceph RBD as primary storage for our upgrade testing purposes. Two of
> > the steps involved were to add below repository to manually compile
> libvirt
> > to have RBD support, since the default libvirt on Ubuntu 12.04 doesn't
> have
> > RBD support by default, unlike on Ubuntu 14.04:
> >
>
> Why do you still want to use such old versions of software?
>
> I recommend that you use at least Ubuntu 14.04 and Ceph version Hammer
> (0.94).
>

The reason is because we are setting up a test / staging environment to
replicate the existing CloudStack 4.2.0 production setup that we have, so
that we can test upgrading from CloudStack 4.2.0 to 4.8.0 on the test
environment, before performing the same upgrade on the production
environment.

Looking forward to your reply, thank you.

Cheers.

-ip-

Older Ceph packages for Ubuntu 12.04 (Precise Pangolin) to recompile libvirt with RBD support

[Cloud List]

Dear all,

I am trying to setup older version of CloudStack 4.2.0 on Ubuntu 12.04 to
use Ceph RBD as primary storage for our upgrade testing purposes. Two of
the steps involved were to add below repository to manually compile libvirt
to have RBD support, since the default libvirt on Ubuntu 12.04 doesn't have
RBD support by default, unlike on Ubuntu 14.04:


# wget -q -O- '
https://ceph.com/git/?p=ceph.git;a=blob_plain;f=keys/release.asc' | sudo
apt-key add -
OK

# echo deb http://eu.ceph.com/debian-cuttlefish/ $(lsb_release -sc) main |
sudo tee /etc/apt/sources.list.d/ceph.list
deb http://eu.ceph.com/debian-cuttlefish/ precise main


But when I ran sudo apt-get update, I am receiving this error (excerpts):


Err http://eu.ceph.com precise/main amd64 Packages
  404  Not Found
Err http://eu.ceph.com precise/main i386 Packages
  404  Not Found

W: Failed to fetch
http://eu.ceph.com/debian-cuttlefish/dists/precise/main/binary-amd64/Packages
404  Not Found

W: Failed to fetch
http://eu.ceph.com/debian-cuttlefish/dists/precise/main/binary-i386/Packages
404  Not Found

E: Some index files failed to download. They have been ignored, or old ones
used instead.


It seems that the repository for the particular required packages has been
removed, anyone can advise if I can get the required packages, may be from
a different location?

Any help is greatly appreciated.

Looking forward to your reply, thank you.

Cheers.

-ip-

Re: SystemVM templates for older CloudStack version 4.2.0

[Cloud List]

Wei Zhou,

Many thanks! :)

Cheers.

-ip-

On Mon, May 30, 2016 at 7:26 PM, Wei ZHOU <ustcweiz...@gmail.com> wrote:

>
> http://download.cloud.com/templates/4.2/systemvmtemplate-2013-06-12-master-kvm.qcow2.bz2
>
> 2016-05-30 13:18 GMT+02:00 Cloud List <cloud-l...@sg.or.id>:
>
> > Dear all,
> >
> > We plan to upgrade our existing CloudStack setup from 4.2.0 to the latest
> > version 4.8.0, and I am setting up a test environment with CloudStack
> > version 4.2.0 for me to try upgrading it to 4.8.0. However, I can't seem
> to
> > find the systemVM templates for the older version 4.2.0 similar to what's
> > available for the current version 4.6.0 and above:
> >
> >
> >
> http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2
> >
> > We are using KVM -- if anyone can point me to the URL for the SystemVM64
> > 4.2.0 template for KVM, it will be very appreciated.
> >
> > Looking forward to your reply, thank you.
> >
> > Cheers.
> >
>

SystemVM templates for older CloudStack version 4.2.0

[Cloud List]

Dear all,

We plan to upgrade our existing CloudStack setup from 4.2.0 to the latest
version 4.8.0, and I am setting up a test environment with CloudStack
version 4.2.0 for me to try upgrading it to 4.8.0. However, I can't seem to
find the systemVM templates for the older version 4.2.0 similar to what's
available for the current version 4.6.0 and above:

http://cloudstack.apt-get.eu/systemvm/4.6/systemvm64template-4.6.0-kvm.qcow2.bz2

We are using KVM -- if anyone can point me to the URL for the SystemVM64
4.2.0 template for KVM, it will be very appreciated.

Looking forward to your reply, thank you.

Cheers.