Re: [PROPOSE] Combining Apache CloudStack Documentation

[ilya musayev]

I like it but wonder if Upgrade section needs to be added? ..

On Tue, Jul 24, 2018 at 2:25 AM Paul Angus  wrote:

> Hi All,
>
> We currently have four sources of documentation [1]. Which make managing
> the documentation convoluted, and worse, make navigating and searching the
> documentation really difficult.
>
> I have taken the current documentation and combined them into one repo,
> then created 7 sections:
>
> CloudStack Concepts and Terminology
> Quick Installation Guide
> Installation Guide
> Usage Guide
> Developers Guide
> Plugins Guide
> Release Notes
>
> I haven't changed any of the content, but I've moved some of it around to
> make more sense (to me).  You can see the result on RTD [2]
>
> I'd like to PROPOSE to move this demo version of the documentation over to
> the Apache repos and make it THE documentation source, update the website,
> and mark the current repos/sites as archive data.
>
> [1]
> https://github.com/apache/cloudstack-docs.git <
> https://github.com/apache/cloudstack-docs.git> is a bit of a dodge-podge
> of resources
> https://github.com/apache/cloudstack-docs-install.git is the install guide
> https://github.com/apache/cloudstack-docs-admin.git is the current admin
> manual.
> https://github.com/apache/cloudstack-docs-rn.git is the release notes for
> individual releases
>
> [2]  https://beta-cloudstack-docs.readthedocs.io/en/latest/
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>

Re: [DISCUSS] Blocking the creation of new Basic Networking zones

[ilya musayev]

I think the simplicity of Basic Zone was - you can get away with 1 VLAN for
everything (great for POC setup) where as Advanced Shared with VLAN
isolation requires several VLANs to get going.

How would we cover this use case?

On Wed, Jun 20, 2018 at 11:34 AM Tutkowski, Mike 
wrote:

> Also, yes, I agree with the list you provided, Wido. We might have to
> break “other fancy stuff” into more detail, though. ;)
>
> On 6/20/18, 12:32 PM, "Tutkowski, Mike"  wrote:
>
> Sorry, Wido :) I missed that part.
>
> On 6/20/18, 5:03 AM, "Wido den Hollander"  wrote:
>
>
>
> On 06/20/2018 12:31 AM, Tutkowski, Mike wrote:
> > If this initiative goes through, perhaps that’s a good time to
> bump CloudStack’s release number to 5.0.0?
> >
>
> That's what I said in my e-mail :-) But yes, I agree with you, this
> might be a good time to bump it to 5.0
>
> With that we would:
>
> - Drop creation of new Basic Networking Zones
> - Support IPv6 in shared IPv6 networks
> - Java 9?
> - Drop support for Ubuntu 12.04
> - Other fancy stuff?
> - Support ConfigDrive in all scenarios properly
>
> How would that sound?
>
> Wido
>
> >> On Jun 19, 2018, at 3:17 PM, Wido den Hollander 
> wrote:
> >>
> >>
> >>
> >>> On 06/19/2018 11:07 PM, Daan Hoogland wrote:
> >>> I like this initiative, and here comes the big but even though
> I myself
> >>> might think it is not valid; Basic zones are there to give a
> simple start
> >>> for new users. If we can give a one-knob start/one page wizard
> for creating
> >>> a shared network in advanced zone with security groups and
> userdata, great.
> >>
> >> That would be a UI thing, but it would be a matter of using VLAN
> >> isolation and giving in VLAN 0 or 'untagged', because that's
> basically
> >> what Basic Networking does.
> >>
> >> It plugs the VM on top of usually cloudbr0 (KVM).
> >>
> >> If you use vlan://untagged for the broadcast_uri in Advanced
> Networking
> >> you get exactly the same result.
> >>
> >>> And I really fancy this idea. let's make ACS more simple by
> throwing at as
> >>> much code as we can in a gradual and controlled way :+1:
> >>
> >> I would love to. But I'm a real novice when it comes to the UI
> though.
> >> So that would be something I wouldn't be good at doing.
> >>
> >> Blocking Basic Networking creation is a few if-statements at
> the right
> >> location and you're done.
> >>
> >> Wido
> >>
> >>>
>  On Tue, Jun 19, 2018 at 10:57 PM, Wido den Hollander <
> w...@widodh.nl> wrote:
> 
>  Hi,
> 
>  We (PCextreme) are a big-time user of Basic Networking and
> recently
>  started to look into Advanced Networking with VLAN isolation
> and a
>  shared network.
> 
>  This provides (from what we can see) all the features Basic
> Networking
>  provides, like the VR just doing DHCP and UserData while the
> Hypervisor
>  does the Security Grouping.
> 
>  That made me wonder why we still have Basic Networking.
> 
>  Dropping all the code would be a big problem for users as you
> can't
>  simply migrate from Basic to Advanced. In theory we found out
> that it's
>  possible by changing the database, but I wouldn't guarantee
> it works in
>  every use-case. So doing this automatically during a upgrade
> would be
>  difficult.
> 
>  To prevent us from having to maintain the Basic Networking
> code for ever
>  I would like to propose and discuss the matter of preventing
> the
>  creation of new Basic Networking zones.
> 
>  In the future this can get us rid of a lot of if-else
> statements in the
>  code and it would make testing also easier as we have few
> things to test.
> 
>  Most of the development also seems to go in the Advanced
> Networking
>  direction.
> 
>  We are currently also working on IPv6 in Advanced Shared
> Networks and
>  that's progressing very good as well.
> 
>  Would this be something to call the 5.0 release where we
> simplify the
>  networking and in the UI/API get rid of Basic Networking
> while keeping
>  it alive for existing users?
> 
>  Wido
> 
> >>>
> >>>
> >>>
>
>
>
>
>

Convert KVM Instance to CloudStack

[ilya musayev]

Hi Users and Dev

I apologize for cross posting.. 

I have bunch of VMs that were deployed by CloudStack - however - the management 
server along with a DB is no longer available.

This is a POC environment - but i would love not to loose and recreate the VMs 
if possible,

Hence i’m thinking of writing re-injestion process of existing running KVM 
instances back into new cloudstack - without doing template imports and such.

Has anyone create a tooling for this endevour by any chance? If not - i might 
have to create one :(


Thanks
ilya

Re: {ANNOUNCE] 4.11.1 RC2 cut

[ilya musayev]

Daan and Rohit

Come to think about it - you are correct, I looked through and noticed my
install does cloudstack-* against 4.11 repo and Marvin being in the repo -
is being installed.

I will change my process, thanks for the update

-ilya

On Fri, Jun 8, 2018 at 1:08 AM Rohit Yadav 
wrote:

> Hi Ilya,
>
>
> The cloudstack-marvin package is not needed to be installed for normal
> CloudStack setup/use, nor it is added as a dependency on any of the other
> production packages such as cloudstack-management, cloudstack-agent,
> cloudstack-common, cloudstack-usage.
>
>
> We created these additional packages (cloudstack-integration-tests and
> cloudstack-marvin) to make it easier for people to test a CloudStack
> release using release specific marvin and integration tests (as far as I
> know the current support/usage of these packages are with Trillian). We can
> discuss having a wheel/frozen marvin package that includes all python
> dependencies, but in a separate thread.
>
>
> - Rohit
>
> <https://cloudstack.apache.org>
>
>
>
> 
> From: ilya musayev 
> Sent: Friday, June 8, 2018 1:52:00 AM
> To: dev@cloudstack.apache.org
> Subject: Re: {ANNOUNCE] 4.11.1 RC2 cut
>
> Hi Daan
>
> I've tried installing 4.11 RC1 (i know i'm in the wrong thread) and noticed
> it does something funky - like installing marvin and going to pypi to get
> dependencies. If its already fixed in RC2 - please ignore this request -
> but if not...
>
> Unfortunately some environments dont have a luxury of open outbound
> internet connection.
>
> Do you think we can refine the requirements and drop packages that arent
> needed
>
> Here is an example:
>
>   Installing : cloudstack-marvin-4.11.1.0-rc1.el7.centos.x86_64
>
>
>
> 191/297
>
> Collecting
>
> http://cdn.mysql.com/Downloads/Connector-Python/mysql-connector-python-2.0.4.zip#md5=3df394d89300db95163f17c843ef49df
>
>   Downloading
>
> http://cdn.mysql.com/Downloads/Connector-Python/mysql-connector-python-2.0.4.zip
> (277kB)
>
> Installing collected packages: mysql-connector-python
>
>   Found existing installation: mysql-connector-python 1.1.6
>
> DEPRECATION: Uninstalling a distutils installed project
> (mysql-connector-python) has been deprecated and will be removed in a
> future version. This is due to the fact that uninstalling a distutils
> project will only partially uninstall the project.
>
> Uninstalling mysql-connector-python-1.1.6:
>
>   Successfully uninstalled mysql-connector-python-1.1.6
>
>   Running setup.py install for mysql-connector-python: started
>
> Running setup.py install for mysql-connector-python: finished with
> status 'done'
>
> Successfully installed mysql-connector-python-2.0.4
>
> You are using pip version 8.1.2, however version 10.0.1 is available.
>
> You should consider upgrading via the 'pip install --upgrade pip' command.
>
> Processing /usr/share/cloudstack-marvin/Marvin-4.11.1.0.tar.gz
>
> Collecting mysql-connector-python>=1.1.6 (from Marvin==4.11.1.0)
>
>   Downloading
>
> https://files.pythonhosted.org/packages/dc/48/32c715d2cef42d0791c5b2f21b4f1f280c8e45afa66a02f4d1828c77f3ea/mysql_connector_python-8.0.11-cp27-cp27mu-manylinux1_x86_64.whl
> (8.1MB)
>
> Collecting requests>=2.2.1 (from Marvin==4.11.1.0)
>
>   Downloading
>
> https://files.pythonhosted.org/packages/49/df/50aa1999ab9bde74656c2919d9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.18.4-py2.py3-none-any.whl
> (88kB
> <https://files.pythonhosted.org/packages/49/df/50aa1999ab9bde74656c2919d9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.18.4-py2.py3-none-any.whl(88kB>
> )
>
> Collecting paramiko>=1.13.0 (from Marvin==4.11.1.0)
>
>   Downloading
>
> https://files.pythonhosted.org/packages/3e/db/cb7b6656e0e7387637ce850689084dc0b94b44df31cc52e5fc5c2c4fd2c1/paramiko-2.4.1-py2.py3-none-any.whl
> (194kB
> <https://files.pythonhosted.org/packages/3e/db/cb7b6656e0e7387637ce850689084dc0b94b44df31cc52e5fc5c2c4fd2c1/paramiko-2.4.1-py2.py3-none-any.whl(194kB>
> )
>
> Collecting nose>=1.3.3 (from Marvin==4.11.1.0)
>
>   Downloading
>
> https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
> (154kB
> <https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl(154kB>
> )
>
> Collecting ddt>=0.4.0 (from Marvin==4.11.1.0)
>
>   Downloading
>
> https://files.pythonhosted.org/packages/54/eb/b39eec5f24414cb5d7393ed9cb1bafac740d005846019473d6fc8df18d

Re: {ANNOUNCE] 4.11.1 RC2 cut

[ilya musayev]

Hi Daan

I've tried installing 4.11 RC1 (i know i'm in the wrong thread) and noticed
it does something funky - like installing marvin and going to pypi to get
dependencies. If its already fixed in RC2 - please ignore this request -
but if not...

Unfortunately some environments dont have a luxury of open outbound
internet connection.

Do you think we can refine the requirements and drop packages that arent
needed

Here is an example:

  Installing : cloudstack-marvin-4.11.1.0-rc1.el7.centos.x86_64



191/297

Collecting
http://cdn.mysql.com/Downloads/Connector-Python/mysql-connector-python-2.0.4.zip#md5=3df394d89300db95163f17c843ef49df

  Downloading
http://cdn.mysql.com/Downloads/Connector-Python/mysql-connector-python-2.0.4.zip
(277kB)

Installing collected packages: mysql-connector-python

  Found existing installation: mysql-connector-python 1.1.6

DEPRECATION: Uninstalling a distutils installed project
(mysql-connector-python) has been deprecated and will be removed in a
future version. This is due to the fact that uninstalling a distutils
project will only partially uninstall the project.

Uninstalling mysql-connector-python-1.1.6:

  Successfully uninstalled mysql-connector-python-1.1.6

  Running setup.py install for mysql-connector-python: started

Running setup.py install for mysql-connector-python: finished with
status 'done'

Successfully installed mysql-connector-python-2.0.4

You are using pip version 8.1.2, however version 10.0.1 is available.

You should consider upgrading via the 'pip install --upgrade pip' command.

Processing /usr/share/cloudstack-marvin/Marvin-4.11.1.0.tar.gz

Collecting mysql-connector-python>=1.1.6 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/dc/48/32c715d2cef42d0791c5b2f21b4f1f280c8e45afa66a02f4d1828c77f3ea/mysql_connector_python-8.0.11-cp27-cp27mu-manylinux1_x86_64.whl
(8.1MB)

Collecting requests>=2.2.1 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/49/df/50aa1999ab9bde74656c2919d9c0c085fd2b3775fd3eca826012bef76d8c/requests-2.18.4-py2.py3-none-any.whl
(88kB)

Collecting paramiko>=1.13.0 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/3e/db/cb7b6656e0e7387637ce850689084dc0b94b44df31cc52e5fc5c2c4fd2c1/paramiko-2.4.1-py2.py3-none-any.whl
(194kB)

Collecting nose>=1.3.3 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl
(154kB)

Collecting ddt>=0.4.0 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/54/eb/b39eec5f24414cb5d7393ed9cb1bafac740d005846019473d6fc8df18db2/ddt-1.1.3-py2.py3-none-any.whl

Collecting pyvmomi>=5.5.0 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/ba/45/d6e4a87004f1c87bdee2942a8896289684e660dbd76e868047d3319b245f/pyvmomi-6.7.0-py2.py3-none-any.whl
(249kB)

Collecting netaddr>=0.7.14 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/ba/97/ce14451a9fd7bdb5a397abf99b24a1a6bb7a1a440b019bebd2e9a0dbec74/netaddr-0.7.19-py2.py3-none-any.whl
(1.6MB)

Collecting dnspython (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/a6/72/209e18bdfedfd78c6994e9ec96981624a5ad7738524dd474237268422cb8/dnspython-1.15.0-py2.py3-none-any.whl
(177kB)

Collecting ipmisim>=0.7 (from Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/58/95/6acd215ec4eaa523b1bfd3b9e16f1defaaf03717a2ed7193077ecf96fa7e/ipmisim-0.7.tar.gz

Collecting protobuf>=3.0.0 (from
mysql-connector-python>=1.1.6->Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/9d/61/54c3a9cfde6ffe0ca6a1786ddb8874263f4ca32e7693ad383bd8cf935015/protobuf-3.5.2.post1-cp27-cp27mu-manylinux1_x86_64.whl
(6.4MB)

Collecting urllib3<1.23,>=1.21.1 (from requests>=2.2.1->Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/63/cb/6965947c13a94236f6d4b8223e21beb4d576dc72e8130bd7880f600839b8/urllib3-1.22-py2.py3-none-any.whl
(132kB)

Collecting idna<2.7,>=2.5 (from requests>=2.2.1->Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/27/cc/6dd9a3869f15c2edfab863b992838277279ce92663d334df9ecf5106f5c6/idna-2.6-py2.py3-none-any.whl
(56kB)

Collecting chardet<3.1.0,>=3.0.2 (from requests>=2.2.1->Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
(133kB)

Collecting certifi>=2017.4.17 (from requests>=2.2.1->Marvin==4.11.1.0)

  Downloading
https://files.pythonhosted.org/packages/7c/e6/92ad559b7192d846975fc916b65f667c7b8c3a32bea7372340bfe9a15fa5/certifi-2018.4.16-py2.py3-none-any.whl
(150kB)

Collecting cryptography>=1.5 (from paramiko>=1.13.0->Marvin==4.11.1.0)

  Downloading

Re: Snapshots only on Primary Storage feature

[ilya musayev]

Perhaps bring it back into 4.11.1?

On Fri, May 18, 2018 at 9:28 AM Suresh Kumar Anaparti <
sureshkumar.anapa...@gmail.com> wrote:

> Si / Will,
>
> That is just FYI, if anyone uses VMware with that flag set to false. I'm
> neither against the feature nor telling to rip that out.
>
> You are correct, the PR 2081 supports KVM and Xen as the volume snapshots
> are directly supported on them and backup operation is not tightly coupled
> with the create operation.
>
> -Suresh
>
> On Fri, May 18, 2018 at 7:38 PM, Simon Weller 
> wrote:
>
> > There are plenty of features in ACS that are particular to a certain
> > hypervisor (or hypervisor set), including VMware specific items.
> >
> > It was never claimed this feature worked across all hypervisors. In
> > addition to that, the default was to leave the existing functionality
> > exactly the way it was originally implemented and if a user wished to
> > change the functionality they could via a global config variable.
> >
> > Your original spec for PR 2081 in confluence states that the PR was
> > targeted towards KVM and Xen, so I'm confused as to why VMware is even
> > being mentioned here.
> >
> >
> > This is a major feature regression that a number of organizations/service
> > providers are relying on and it wasn't called out when the PR was
> submitted.
> >
> >
> > 
> > From: Will Stevens 
> > Sent: Friday, May 18, 2018 6:12 AM
> > To: dev@cloudstack.apache.org
> > Subject: Re: Snapshots only on Primary Storage feature
> >
> > Just because it does not work for VMware should not a reason to rip out
> the
> > functionality for other hypervisors where it is being used though.
> >
> > I know we also have the requirement that snapshots are not automatically
> > replicated to secondary storage, so this feature is useful to us.
> >
> > I don't understand the rational for removing the feature just because it
> > does not work on VMware.
> >
> > On Fri, May 18, 2018, 6:27 AM Suresh Kumar Anaparti, <
> > sureshkumar.anapa...@gmail.com> wrote:
> >
> > > Si,
> > >
> > > The PR# 1697 with the global setting *snapshot.backup.rightafter** -
> > > false* doesn't
> > > work for VMware as create snapshot never takes a snapshot in Primary
> > pool,
> > > it just returns the snapshot uuid. The backup snapshot does the
> complete
> > > job - creates a VM snapshot with the uuid, extracts and exports the
> > target
> > > volume to secondary. On demand backup snapshot doesn't work as there is
> > no
> > > snapshot in primary. Also, there'll be only one entry with Primary
> store
> > > role in snapshot_store_ref, which is the latest snapshot taken for that
> > > volume.
> > >
> > > -Suresh
> > >
> > > On Fri, May 18, 2018 at 1:03 AM, Simon Weller  >
> > > wrote:
> > >
> > > > The whole point of the original PR was to optionally disable this
> > > > functionality.
> > > >
> > > > We don't expose views of the backup state to our customers (we have
> our
> > > > own customer interfaces) and it's a large waste of space for us to be
> > > > backing up tons of VM images when we have a solid primary storage
> > > > infrastructure that already has lots of resiliency.
> > > >
> > > >
> > > > I guess we're going to have to revisit this again before we can
> > consider
> > > > rebasing on 4.11.
> > > >
> > > > 
> > > > From: Suresh Kumar Anaparti 
> > > > Sent: Thursday, May 17, 2018 2:21 PM
> > > > To: dev
> > > > Subject: Re: Snapshots only on Primary Storage feature
> > > >
> > > > Hi Si,
> > > >
> > > > No. not possible to disable the backup to secondary. It copies the
> > volume
> > > > snapshot to secondary in a background thread using asyncBackup param
> > (set
> > > > to true) and allows other operations during that time.
> > > >
> > > > I understand that the backup was on demand when any operations are
> > > > performed on the snapshot. But, backup during that time may take
> > > > considerable time (depending on the snapshot size and the network
> > > > bandwidth), which can result in the job timeout and the User may
> assume
> > > > that it is already Backed up based on its state, unless it is
> > documented.
> > > >
> > > > -Suresh
> > > >
> > > > On Fri, May 18, 2018 at 12:23 AM, Simon Weller
>  > >
> > > > wrote:
> > > >
> > > > > Suresh,
> > > > >
> > > > >
> > > > > With this new merged  PR, is it possible to disable the backup to
> > > > > secondary completely? I can't tell from the reference spec and
> we're
> > > not
> > > > on
> > > > > a 4.10/4.11 base yet.
> > > > >
> > > > > For the record, in the instances where a volume or template from
> > > snapshot
> > > > > was required, the backup image was copied on demand to secondary.
> > > > >
> > > > > In an ideal world, secondary storage wouldn't even be involved in
> > most
> > > of
> > > > > these options, instead using the native 

Re: Dynamic roles question

[ilya musayev]

Ivan

This is already done in 4.11, I’m not next to comp to check but ShapeBlue
has a feature created that would allow for movement between different roles.

Regards
ilya

On Thu, May 17, 2018 at 6:03 AM Ivan Kudryavtsev 
wrote:

> Hello, community.
>
> I'm thinking about implementing the feature for accounts which permits to
> change account role. Basically, the rationale is trials or demonstration
> modes which restricts users from doing extra stuff, like VM creation,
> service offering changes, etc. Basically, after trial the account should be
> switched to a normal mode or removed. By permitting such role switching we
> can support the feature, otherwise we have to create unique role for every
> user and manage it separately. Please, let me know your thoughts about
> that. Have a good day.
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks Software, Ltd.
> Cell: +7-923-414-1515
> WWW: http://bitworks.software/ 
>

Re: [DISCUSS] CloudStack graceful shutdown

[ilya musayev]

Rafael

What you are suggesting - was already implemented. We've created Load
Balancing algorithms - but we did not take into account the LB algo for
maintenance (yet). Rohit and ShapeBlue were the developers behind the
feature.

What needs to happen is a tweak to LB Algorithms to become MS maintenance
aware - or create new LB Algos altogether. Essentially we need to merge
your work and this feature. Please read the FS below.

Functional Spec:


The new CA framework introduced basic support for comma-separated
list of management servers for agent, which makes an external LB
unnecessary.

This extends that feature to implement LB sorting algorithms that
sorts the management server list before they are sent to the agents.
This adds a central intelligence in the management server and adds
additional enhancements to Agent class to be algorithm aware and
have a background mechanism to check/fallback to preferred management
server (assumed as the first in the list). This is support for any
indirect agent such as the KVM, CPVM and SSVM agent, and would
provide support for management server host migration during upgrade
(when instead of in-place, new hosts are used to setup new mgmt server).

This FR introduces two new global settings:

   - indirect.agent.lb.algorithm: The algorithm for the indirect agent LB.
   - indirect.agent.lb.check.interval: The preferred host check interval
   for the agent's background task that checks and switches to agent's
   preferred host.

The indirect.agent.lb.algorithm supports following algorithm options:

   - static: use the list as provided.
   - roundrobin: evenly spreads hosts across management servers based on
   host's id.
   - shuffle: (pseudo) randomly sorts the list (not recommended for
   production).

Any changes to the global settings - indirect.agent.lb.algorithm and
host does not require restarting of the mangement server(s) and the
agents. A message bus based system dynamically reacts to change in these
global settings and propagates them to all connected agents.

Comma-separated management server list is propagated to agents on
following cases:

   - Addition of a host (including ssvm, cpvm systevms).
   - Connection or reconnection by the agents to a management server.
   - After admin changes the 'host' and/or the
   'indirect.agent.lb.algorithm' global settings.

On the agent side, the 'host' setting is saved in its properties file as:
host=@.

First the agent connects to the management server and sends its current
management server list, which is compared by the management server and
in case of failure a new/update list is sent for the agent to persist.

>From the agent's perspective, the first address in the propagated list
will be considered the preferred host. A new background task can be
activated by configuring the indirect.agent.lb.check.interval which is
a cluster level global setting from CloudStack and admins can also
override this by configuring the 'host.lb.check.interval' in the
agent.properties file.

Every time agent gets a ms-host list and the algorithm, the host specific
background check interval is also sent and it dynamically reconfigures
the background task without need to restart agents.

Note: The 'static' and 'roundrobin' algorithms, strictly checks for the
order as expected by them, however, the 'shuffle' algorithm just checks
for content and not the order of the comma separate ms host addresses.

Regards
ilya


On Fri, Apr 20, 2018 at 1:01 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Is that management server load balancing feature using static
> configurations? I heard about it on the mailing list, but I did not follow
> the implementation.
>
> I do not see many problems with agents reconnecting. We can implement in
> agents (not just KVM, but also system VMs) a logic that instead of using a
> static pool of management servers configured in a properties file, they
> dynamically request a list of available management servers via that list
> management servers API method. This would require us to configure agents
> with a load balancer URL that executes the balancing between multiple
> management servers.
>
> I am +1 to remove the need for that VIP, which executes the load balance
> for connecting agents to management servers.
>
> On Fri, Apr 20, 2018 at 4:41 PM, ilya musayev <
> ilya.mailing.li...@gmail.com>
> wrote:
>
> > Rafael and Community
> >
> > All is well and good and i think we are thinking along the similar lines
> -
> > the only issue that i see right now with any approach is KVM Agents (or
> > direct agents) and using LoadBalancer on 8250.
> >
> > Here is a scenario:
> >
> > You have 2 Management Server setup fronted with a VIP on 8250.
> > The LB Algorithm is either Round Robin or Least Connections used.
> > You initiate a maintenance mode operation on o

Re: [DISCUSS] CloudStack graceful shutdown

[ilya musayev]

Rafael and Community

All is well and good and i think we are thinking along the similar lines -
the only issue that i see right now with any approach is KVM Agents (or
direct agents) and using LoadBalancer on 8250.

Here is a scenario:

You have 2 Management Server setup fronted with a VIP on 8250.
The LB Algorithm is either Round Robin or Least Connections used.
You initiate a maintenance mode operation on one of the MS servers (call it
MS1) - assume you have a long running migration job that needs 60 minutes
to complete.
We attempt to evacuate the agents by telling them to disconnect and
reconnect again
If we are using LB on 8250 with
1) Least Connection used - then all agents will continuously try to connect
to a MS1 node that is attempting to go down for maintenance. Essentially
with this  LB configuration this operation will never
2) Round Robin - this will take a while - but eventually - you will get all
nodes connected to MS2

The current limitation is usage of external LB on 8250. For this operation
to work without issue - would mean agents must connect to MS server without
an LB. This is a recent feature we've developed with ShapeBlue - where we
maintain the list of CloudStack Management Servers in the agent.properties
file.

Unless you can think of other solution - it appears we may have to forced
to bypass the 8250 VIP LB and use the new feature to maintain the list of
management servers within agent.properties.


I need to run now, let me know what your thoughts are.

Regards
ilya



On Tue, Apr 17, 2018 at 8:27 AM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Ilya and others,
>
> We have been discussing this idea of graceful/nicely shutdown.  Our feeling
> is that we (in CloudStack community) might have been trying to solve this
> problem with too much scripting. What if we developed a more integrated
> (native) solution?
>
> Let me explain our idea.
>
> ACS has a table called “mshost”, which is used to store management server
> information. During balancing and when jobs are dispatched to other
> management servers this table is consulted/queried.  Therefore, we have
> been discussing the idea of creating a management API for management
> servers.  We could have an API method that changes the state of management
> servers to “prepare to maintenance” and then “maintenance” (as soon as all
> of the task/jobs it is managing finish). The idea is that during
> rebalancing we would remove the hosts of servers that are not in “Up” state
> (of course we would also ignore hosts in the aforementioned state to
> receive hosts to manage).  Moreover, when we send/dispatch jobs to other
> management servers, we could ignore the ones that are not in “Up” state
> (which is something already done).
>
> By doing this, the nicely shutdown could be executed in a few steps.
>
> 1 – issue the maintenance method for the management server you desire
> 2 – wait until the MS goes into maintenance mode, while there are still
> running jobs it (the management server) will be maintained in prepare for
> maintenance
> 3 – execute the Linux shutdown command
>
> We would need other APIs methods to manage MSs then. An (i) API method to
> list MSs, and we could even create an (ii) API to remove old/de-activated
> management servers, which we currently do not have (forcing users to apply
> changed directly in the database).
>
> Moreover, in this model, we would not kill hanging jobs; we would wait
> until they expire and ACS expunges them. Of course, it is possible to
> develop a forceful maintenance method as well. Then, when the “prepare for
> maintenance” takes longer than a parameter, we could kill hanging jobs.
>
> All of this would allow the MS to be kept up and receiving requests until
> it can be safely shutdown. What do you guys about this approach?
>
> On Tue, Apr 10, 2018 at 6:52 PM, Yiping Zhang <yzh...@marketo.com> wrote:
>
> > As a cloud admin, I would love to have this feature.
> >
> > It so happens that I just accidentally restarted my ACS management server
> > while two instances are migrating to another Xen cluster (via storage
> > migration, not live migration).  As results, both instances
> > ends up with corrupted data disk which can't be reattached or migrated.
> >
> > Any feature which prevents this from happening would be great.  A low
> > hanging fruit is simply checking for
> > if there are any async jobs running, especially any kind of migration
> jobs
> > or other known long running type of
> > jobs and warn the operator  so that he has a chance to abort server
> > shutdowns.
> >
> > Yiping
> >
> > On 4/5/18, 3:13 PM, "ilya musayev" <ilya.mailing.li...@gmail.com>
> wrote:
> >
> > Andrija
> >

Re: [DISCUSS] CloudMonkey 6.0.0-alpha (about six years after initial version in 2012)

[ilya musayev]

This is great news and cloud monkey is used more than you think :)

I will share the news with my team.

On Tue, Apr 10, 2018 at 5:07 AM Will Stevens  wrote:

> +1. It has been a great tool for years.  Looking forward to the golang
> version.
>
> On Apr 10, 2018 7:59 AM, "Rohit Yadav"  wrote:
>
> All,
>
>
> Few months ago, I started porting the current code to be compatible with
> both Python2 and Python3 to make it run with both Python2 (for older
> systems such as CentOS6 etc) and Python3 (for newer platforms). The work
> was not a success, another problem was that cloudmonkey was not easy to
> install and required several dependencies that would certainly fail on
> older systems with Python 2.6.x.
>
>
> Considering all things, I started working on an experimental golang port
> [2] and happy to announce that the initial alpha version shows a lot of
> promise and is 5-20x faster than the python based cli [1]. The compiled
> binary runs on several targets, including windows [1].
>
>
> I cannot commit to a timeline/release date yet but the aim of this thread
> is to discuss and propose the simplification of the CLI which may require
> removal of some features and some breaking changes may be introduced:
>
>
> - Make json the default output format
>
> - Remove coloured output
>
> - Remove unpopular, least user output formats? xml, default (line-separate
> key=value), table?
>
> - Remove `set` options: color, expires, (custom) prompt
>
> - Remove `paramcompletion` option, this will be true/enabled by default
>
> - Remove signature version and expires (I'm not sure why this is needed or
> used)
>
> - Remove history_file, cache_file, log_file options, use the default paths
> in folder at (user's  home directory)/.cloudmonkey.
>
> - Remove shell based execution from interactive interpreter mode (using !
> or shell keywords)
>
> - Remove support for CloudStack older than 4.5, i.e. it won't be tested
> against older cloudstacks.
>
> - Remove a default API cache with the client, for a fresh env without any
> ~/.cloudmonkey/cache; users can run `sync` command against a management
> server.
>
> - Interactive API parameter completion in CLI mode: the current API
> parameter completion requires the user to manually copy/paste the uuids, or
> autocomplete by typing parts of the uuids/option.
>
> - Improve how maps are passed.
>
> - Good to have: bash/zsh completion.
>
>
> Please share your thoughts, and objections (especially if you're using the
> proposed features to be removed in version 6.x).
>
>
> [1] https://twitter.com/rhtyd/status/983448788059770882
>
> [2] https://github.com/rhtyd/cmk
>
>
> - Rohit
>
> 
>
>
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>

Re: [DISCUSS] CloudStack graceful shutdown

[ilya musayev]

the content of the file back to "ready"
> > > again, instead of "maint" [2].
> > >
> > > [1] AsyncJobMgr-Heartbeat, CapacityChecker, StatsCollector
> > > [2] HA proxy documentation on agent checker: https://cbonte.github.io/
> > > haproxy-dconv/1.6/configuration.html#5.2-agent-check
> > >
> > > Regarding your issue on the port blocking, I think it's fair to
> consider
> > > that if you want to shutdown your server at some point, you have to
> stop
> > > serving (some) requests. Here the only way it's to stop serving
> > everything.
> > > If the API had a REST design, we could reject any POST/PUT/DELETE
> > > operations and allow GET ones. I don't know how hard it would be today
> to
> > > only allow listBaseCmd operations to be more friendly with the users.
> > >
> > > Marco
> > >
> > >
> > > On Thu, Apr 5, 2018 at 2:22 AM, Sergey Levitskiy <serg...@hotmail.com>
> > > wrote:
> > >
> > >> Now without spellchecking :)
> > >>
> > >> This is not simple e.g. for VMware. Each management server also acts
> as
> > an
> > >> agent proxy so tasks against a particular ESX host will be always
> > >> forwarded. That right answer will be to support a native “maintenance
> > mode”
> > >> for management server. When entered to such mode the management server
> > >> should release all agents including SSVM, block/redirect API calls and
> > >> login request and finish all async job it originated.
> > >>
> > >>
> > >>
> > >> On Apr 4, 2018, at 5:15 PM, Sergey Levitskiy <serg...@hotmail.com
> >  > >> serg...@hotmail.com>> wrote:
> > >>
> > >> This is not simple e.g. for VMware. Each management server also acts
> as
> > an
> > >> agent proxy so tasks against a particular ESX host will be always
> > >> forwarded. That right answer will be to a native support for
> > “maintenance
> > >> mode” for management server. When entered to such mode the management
> > >> server should release all agents including save, block/redirect API
> > calls
> > >> and login request and finish all a sync job it originated.
> > >>
> > >> Sent from my iPhone
> > >>
> > >> On Apr 4, 2018, at 3:31 PM, Rafael Weingärtner <
> > >> rafaelweingart...@gmail.com<mailto:rafaelweingart...@gmail.com>>
> wrote:
> > >>
> > >> Ilya, still regarding the management server that is being shut down
> > issue;
> > >> if other MSs/or maybe system VMs (I am not sure to know if they are
> > able to
> > >> do such tasks) can direct/redirect/send new jobs to this management
> > server
> > >> (the one being shut down), the process might never end because new
> tasks
> > >> are always being created for the management server that we want to
> shut
> > >> down. Is this scenario possible?
> > >>
> > >> That is why I mentioned blocking the port 8250 for the
> > “graceful-shutdown”.
> > >>
> > >> If this scenario is not possible, then everything s fine.
> > >>
> > >>
> > >> On Wed, Apr 4, 2018 at 7:14 PM, ilya musayev <
> > ilya.mailing.li...@gmail.com
> > >> <mailto:ilya.mailing.li...@gmail.com>>
> > >> wrote:
> > >>
> > >> I'm thinking of using a configuration from
> > "job.cancel.threshold.minutes" -
> > >> it will be the longest
> > >>
> > >>"category": "Advanced",
> > >>
> > >>"description": "Time (in minutes) for async-jobs to be forcely
> > >> cancelled if it has been in process for long",
> > >>
> > >>"name": "job.cancel.threshold.minutes",
> > >>
> > >>"value": "60"
> > >>
> > >>
> > >>
> > >>
> > >> On Wed, Apr 4, 2018 at 1:36 PM, Rafael Weingärtner <
> > >> rafaelweingart...@gmail.com<mailto:rafaelweingart...@gmail.com>>
> wrote:
> > >>
> > >> Big +1 for this feature; I only have a few doubts.
> > >>
> > >> * Regarding the tasks/jobs that management servers (MSs) execute; are
> > >> these
> > >> tasks originate from requests that come to the MS, or is it pos

Re: [DISCUSS] CloudStack graceful shutdown

[ilya musayev]

After much useful input from many of you - i realize my approach is
somewhat incomplete and possible very optimistic.

Speaking to Marcus, here is what we propose as alternate solution, i was
hoping to stay outside of the "core" - but it looks like there is no other
away around it.

Proposed functionality: Management Server functional to prepare for
maintenance
* i'm thinking this should be applicable to multinode setup only
drain all connection on 8250 for KVM and Other agents - by issuing a
reconnect command on agents
while 8250 is still listening, a new attempt to connect will be blocked and
agent will be asked to reconnect (if you have LB - it will route it to
another node and eventually reconnect all agents to other nodes - this
might be an area where Marc's HAProxy solution would plugin). In 4.11 -
there is a new framework for managing agent connectivity without needing
Load Balancer, need to investigate how this will work.
allow the existing running async tasks to complete - as per
"job.cancel.threshold.minutes"
max value
queue the new tasks and process them on the next management server

Still dont know what will happen to Xen or VMware in this case - perhaps
ShapeBlue team can help answer or fill in the blanks for us.

Regards,
ilya

On Thu, Apr 5, 2018 at 2:48 PM, ilya musayev <ilya.mailing.li...@gmail.com>
wrote:

> Hi Sergey
>
> Glad to see you are doing well,
>
> I was gonna say drop "enterprise virtualization company" and save a
> $fortune$ - but its not for everyone :)
>
> I'll post another proposed solution to bottom of this thread.
>
> Regards
> ilya
>
>
> On Wed, Apr 4, 2018 at 5:22 PM, Sergey Levitskiy <serg...@hotmail.com>
> wrote:
>
>> Now without spellchecking :)
>>
>> This is not simple e.g. for VMware. Each management server also acts as
>> an agent proxy so tasks against a particular ESX host will be always
>> forwarded. That right answer will be to support a native “maintenance mode”
>> for management server. When entered to such mode the management server
>> should release all agents including SSVM, block/redirect API calls and
>> login request and finish all async job it originated.
>>
>>
>>
>> On Apr 4, 2018, at 5:15 PM, Sergey Levitskiy <serg...@hotmail.com> serg...@hotmail.com>> wrote:
>>
>> This is not simple e.g. for VMware. Each management server also acts as
>> an agent proxy so tasks against a particular ESX host will be always
>> forwarded. That right answer will be to a native support for “maintenance
>> mode” for management server. When entered to such mode the management
>> server should release all agents including save, block/redirect API calls
>> and login request and finish all a sync job it originated.
>>
>> Sent from my iPhone
>>
>> On Apr 4, 2018, at 3:31 PM, Rafael Weingärtner <
>> rafaelweingart...@gmail.com<mailto:rafaelweingart...@gmail.com>> wrote:
>>
>> Ilya, still regarding the management server that is being shut down issue;
>> if other MSs/or maybe system VMs (I am not sure to know if they are able
>> to
>> do such tasks) can direct/redirect/send new jobs to this management server
>> (the one being shut down), the process might never end because new tasks
>> are always being created for the management server that we want to shut
>> down. Is this scenario possible?
>>
>> That is why I mentioned blocking the port 8250 for the
>> “graceful-shutdown”.
>>
>> If this scenario is not possible, then everything s fine.
>>
>>
>> On Wed, Apr 4, 2018 at 7:14 PM, ilya musayev <
>> ilya.mailing.li...@gmail.com<mailto:ilya.mailing.li...@gmail.com>>
>> wrote:
>>
>> I'm thinking of using a configuration from "job.cancel.threshold.minutes"
>> -
>> it will be the longest
>>
>> "category": "Advanced",
>>
>> "description": "Time (in minutes) for async-jobs to be forcely
>> cancelled if it has been in process for long",
>>
>> "name": "job.cancel.threshold.minutes",
>>
>> "value": "60"
>>
>>
>>
>>
>> On Wed, Apr 4, 2018 at 1:36 PM, Rafael Weingärtner <
>> rafaelweingart...@gmail.com<mailto:rafaelweingart...@gmail.com>> wrote:
>>
>> Big +1 for this feature; I only have a few doubts.
>>
>> * Regarding the tasks/jobs that management servers (MSs) execute; are
>> these
>> tasks originate from requests that come to the MS, or is it possible that
>> requests received by one management server to be executed by other? I
>> 

Re: [DISCUSS] CloudStack graceful shutdown

[ilya musayev]

Hi Sergey

Glad to see you are doing well,

I was gonna say drop "enterprise virtualization company" and save a
$fortune$ - but its not for everyone :)

I'll post another proposed solution to bottom of this thread.

Regards
ilya


On Wed, Apr 4, 2018 at 5:22 PM, Sergey Levitskiy <serg...@hotmail.com>
wrote:

> Now without spellchecking :)
>
> This is not simple e.g. for VMware. Each management server also acts as an
> agent proxy so tasks against a particular ESX host will be always
> forwarded. That right answer will be to support a native “maintenance mode”
> for management server. When entered to such mode the management server
> should release all agents including SSVM, block/redirect API calls and
> login request and finish all async job it originated.
>
>
>
> On Apr 4, 2018, at 5:15 PM, Sergey Levitskiy <serg...@hotmail.com serg...@hotmail.com>> wrote:
>
> This is not simple e.g. for VMware. Each management server also acts as an
> agent proxy so tasks against a particular ESX host will be always
> forwarded. That right answer will be to a native support for “maintenance
> mode” for management server. When entered to such mode the management
> server should release all agents including save, block/redirect API calls
> and login request and finish all a sync job it originated.
>
> Sent from my iPhone
>
> On Apr 4, 2018, at 3:31 PM, Rafael Weingärtner <
> rafaelweingart...@gmail.com<mailto:rafaelweingart...@gmail.com>> wrote:
>
> Ilya, still regarding the management server that is being shut down issue;
> if other MSs/or maybe system VMs (I am not sure to know if they are able to
> do such tasks) can direct/redirect/send new jobs to this management server
> (the one being shut down), the process might never end because new tasks
> are always being created for the management server that we want to shut
> down. Is this scenario possible?
>
> That is why I mentioned blocking the port 8250 for the “graceful-shutdown”.
>
> If this scenario is not possible, then everything s fine.
>
>
> On Wed, Apr 4, 2018 at 7:14 PM, ilya musayev <ilya.mailing.li...@gmail.com
> <mailto:ilya.mailing.li...@gmail.com>>
> wrote:
>
> I'm thinking of using a configuration from "job.cancel.threshold.minutes" -
> it will be the longest
>
> "category": "Advanced",
>
> "description": "Time (in minutes) for async-jobs to be forcely
> cancelled if it has been in process for long",
>
> "name": "job.cancel.threshold.minutes",
>
> "value": "60"
>
>
>
>
> On Wed, Apr 4, 2018 at 1:36 PM, Rafael Weingärtner <
> rafaelweingart...@gmail.com<mailto:rafaelweingart...@gmail.com>> wrote:
>
> Big +1 for this feature; I only have a few doubts.
>
> * Regarding the tasks/jobs that management servers (MSs) execute; are
> these
> tasks originate from requests that come to the MS, or is it possible that
> requests received by one management server to be executed by other? I
> mean,
> if I execute a request against MS1, will this request always be
> executed/threated by MS1, or is it possible that this request is executed
> by another MS (e.g. MS2)?
>
> * I would suggest that after we block traffic coming from
> 8080/8443/8250(we
> will need to block this as well right?), we can log the execution of
> tasks.
> I mean, something saying, there are XXX tasks (enumerate tasks) still
> being
> executed, we will wait for them to finish before shutting down.
>
> * The timeout (60 minutes suggested) could be global settings that we can
> load before executing the graceful-shutdown.
>
> On Wed, Apr 4, 2018 at 5:15 PM, ilya musayev <
> ilya.mailing.li...@gmail.com<mailto:ilya.mailing.li...@gmail.com>
>
> wrote:
>
> Use case:
> In any environment - time to time - administrator needs to perform a
> maintenance. Current stop sequence of cloudstack management server will
> ignore the fact that there may be long running async jobs - and
> terminate
> the process. This in turn can create a poor user experience and
> occasional
> inconsistency  in cloudstack db.
>
> This is especially painful in large environments where the user has
> thousands of nodes and there is a continuous patching that happens
> around
> the clock - that requires migration of workload from one node to
> another.
>
> With that said - i've created a script that monitors the async job
> queue
> for given MS and waits for it complete all jobs. More details are
> posted
> below.
>
> I'd like to introduce "graceful-shutdown" into the systemctl/service of
> cloudstack-management service.
>
&g

Re: [DISCUSS] CloudStack graceful shutdown

[ilya musayev]

Marc

Thank you posting the details on how your implementation works.
Unfortunately for us - HAproxy is not an option - hence we cant take
advantage of this implementation, but please do share with the community -
perhaps it will help someone else.

I'm going to post to the bottom of this thread with new proposed solution.

Regards
ilya

On Thu, Apr 5, 2018 at 2:36 AM, Marc-Aurèle Brothier <ma...@exoscale.ch>
wrote:

> Hi all,
>
> Good point ilya but as stated by Sergey there's more thing to consider
> before being able to do a proper shutdown. I augmented my script I gave you
> originally and changed code in CS. What we're doing for our environment is
> as follow:
>
> 1. the MGMT looks for a change in the file /etc/lb-agent which contains
> keywords for HAproxy[2] (ready, maint) so that HA-proxy can disable the
> mgmt on the keyword "maint" and the mgmt server stops a couple of
> threads[1] to stop processing async jobs in the queue
> 2. Looks for the async jobs and wait until there is none to ensure you can
> send the reconnect commands (if jobs are running, a reconnect will result
> in a failed job since the result will never reach the management server -
> the agent waits for the current job to be done before reconnecting, and
> discard the result... rooms for improvement here!)
> 3. Issue a reconnectHost command to all the hosts connected to the mgmt
> server so that they reconnect to another one, otherwise the mgmt must be up
> since it is used to forward commands to agents.
> 4. when all agents are reconnected, we can shutdown the management server
> and perform the maintenance.
>
> One issue remains for me, during the reconnect, the commands that are
> processed at the same time should be kept in a queue until the agents have
> finished any current jobs and have reconnected. Today the little time
> window during which the reconnect happens can lead to failed jobs due to
> the agent not being connected at the right moment.
>
> I could push a PR for the change to stop some processing threads based on
> the content of a file. It's possible also to cancel the drain of the
> management by simply changing the content of the file back to "ready"
> again, instead of "maint" [2].
>
> [1] AsyncJobMgr-Heartbeat, CapacityChecker, StatsCollector
> [2] HA proxy documentation on agent checker: https://cbonte.github.io/
> haproxy-dconv/1.6/configuration.html#5.2-agent-check
>
> Regarding your issue on the port blocking, I think it's fair to consider
> that if you want to shutdown your server at some point, you have to stop
> serving (some) requests. Here the only way it's to stop serving everything.
> If the API had a REST design, we could reject any POST/PUT/DELETE
> operations and allow GET ones. I don't know how hard it would be today to
> only allow listBaseCmd operations to be more friendly with the users.
>
> Marco
>
>
> On Thu, Apr 5, 2018 at 2:22 AM, Sergey Levitskiy <serg...@hotmail.com>
> wrote:
>
> > Now without spellchecking :)
> >
> > This is not simple e.g. for VMware. Each management server also acts as
> an
> > agent proxy so tasks against a particular ESX host will be always
> > forwarded. That right answer will be to support a native “maintenance
> mode”
> > for management server. When entered to such mode the management server
> > should release all agents including SSVM, block/redirect API calls and
> > login request and finish all async job it originated.
> >
> >
> >
> > On Apr 4, 2018, at 5:15 PM, Sergey Levitskiy <serg...@hotmail.com
>  > serg...@hotmail.com>> wrote:
> >
> > This is not simple e.g. for VMware. Each management server also acts as
> an
> > agent proxy so tasks against a particular ESX host will be always
> > forwarded. That right answer will be to a native support for “maintenance
> > mode” for management server. When entered to such mode the management
> > server should release all agents including save, block/redirect API calls
> > and login request and finish all a sync job it originated.
> >
> > Sent from my iPhone
> >
> > On Apr 4, 2018, at 3:31 PM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com<mailto:rafaelweingart...@gmail.com>> wrote:
> >
> > Ilya, still regarding the management server that is being shut down
> issue;
> > if other MSs/or maybe system VMs (I am not sure to know if they are able
> to
> > do such tasks) can direct/redirect/send new jobs to this management
> server
> > (the one being shut down), the process might never end because new tasks
> > are always being created for the management server that we want to shu

Re: [DISCUSS] CloudStack graceful shutdown

[ilya musayev]

I'm thinking of using a configuration from "job.cancel.threshold.minutes" -
it will be the longest

  "category": "Advanced",

  "description": "Time (in minutes) for async-jobs to be forcely
cancelled if it has been in process for long",

  "name": "job.cancel.threshold.minutes",

  "value": "60"




On Wed, Apr 4, 2018 at 1:36 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Big +1 for this feature; I only have a few doubts.
>
> * Regarding the tasks/jobs that management servers (MSs) execute; are these
> tasks originate from requests that come to the MS, or is it possible that
> requests received by one management server to be executed by other? I mean,
> if I execute a request against MS1, will this request always be
> executed/threated by MS1, or is it possible that this request is executed
> by another MS (e.g. MS2)?
>
> * I would suggest that after we block traffic coming from 8080/8443/8250(we
> will need to block this as well right?), we can log the execution of tasks.
> I mean, something saying, there are XXX tasks (enumerate tasks) still being
> executed, we will wait for them to finish before shutting down.
>
> * The timeout (60 minutes suggested) could be global settings that we can
> load before executing the graceful-shutdown.
>
> On Wed, Apr 4, 2018 at 5:15 PM, ilya musayev <ilya.mailing.li...@gmail.com
> >
> wrote:
>
> > Use case:
> > In any environment - time to time - administrator needs to perform a
> > maintenance. Current stop sequence of cloudstack management server will
> > ignore the fact that there may be long running async jobs - and terminate
> > the process. This in turn can create a poor user experience and
> occasional
> > inconsistency  in cloudstack db.
> >
> > This is especially painful in large environments where the user has
> > thousands of nodes and there is a continuous patching that happens around
> > the clock - that requires migration of workload from one node to another.
> >
> > With that said - i've created a script that monitors the async job queue
> > for given MS and waits for it complete all jobs. More details are posted
> > below.
> >
> > I'd like to introduce "graceful-shutdown" into the systemctl/service of
> > cloudstack-management service.
> >
> > The details of how it will work is below:
> >
> > Workflow for graceful shutdown:
> >   Using iptables/firewalld - block any connection attempts on 8080/8443
> (we
> > can identify the ports dynamically)
> >   Identify the MSID for the node, using the proper msid - query async_job
> > table for
> > 1) any jobs that are still running (or job_status=“0”)
> > 2) job_dispatcher not like “pseudoJobDispatcher"
> > 3) job_init_msid=$my_ms_id
> >
> > Monitor this async_job table for 60 minutes - until all async jobs for
> MSID
> > are done, then proceed with shutdown
> > If failed for any reason or terminated, catch the exit via trap
> command
> > and unblock the 8080/8443
> >
> > Comments are welcome
> >
> > Regards,
> > ilya
> >
>
>
>
> --
> Rafael Weingärtner
>

Re: [DISCUSS] CloudStack graceful shutdown

[ilya musayev]

Rafael

> * Regarding the tasks/jobs that management servers (MSs) execute; are
these
tasks originate from requests that come to the MS, or is it possible that
requests received by one management server to be executed by other? I mean,
if I execute a request against MS1, will this request always be
executed/threated by MS1, or is it possible that this request is executed
by another MS (e.g. MS2)?

Yes its possible, but it will be tracked under async_job with proper MS
that is responsible for this task.

My initial goal was to prevent user from creating more async jobs on the
node thats about to go down for maintenance - but as i'm thinking about it
- i dont know if it matters - since async job will be executed on the MS
node that tracks a specific hypervisor/agent - as defined in cloud.host
table.

Maybe i'll leave off the blocking off 8080/8443 and just focus on tracking
async_jobs instead. Assuming you are managing your MS with Load Balancer -
it should be smart enough to shift the user traffic to MS that is up.

> * I would suggest that after we block traffic coming from
8080/8443/8250(we
will need to block this as well right?), we can log the execution of tasks.
I mean, something saying, there are XXX tasks (enumerate tasks) still being
executed, we will wait for them to finish before shutting down

8250 - is a bit too aggressive in my opinion andwe dont want to do that. If
you block 8250 and you have a long running tasks - you are waiting on to
complete - then it may fail - because you block agent communication on 8250.

Thanks
ilya


On Wed, Apr 4, 2018 at 1:36 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Big +1 for this feature; I only have a few doubts.
>
> * Regarding the tasks/jobs that management servers (MSs) execute; are these
> tasks originate from requests that come to the MS, or is it possible that
> requests received by one management server to be executed by other? I mean,
> if I execute a request against MS1, will this request always be
> executed/threated by MS1, or is it possible that this request is executed
> by another MS (e.g. MS2)?
>
> * I would suggest that after we block traffic coming from 8080/8443/8250(we
> will need to block this as well right?), we can log the execution of tasks.
> I mean, something saying, there are XXX tasks (enumerate tasks) still being
> executed, we will wait for them to finish before shutting down.
>
> * The timeout (60 minutes suggested) could be global settings that we can
> load before executing the graceful-shutdown.
>
> On Wed, Apr 4, 2018 at 5:15 PM, ilya musayev <ilya.mailing.li...@gmail.com
> >
> wrote:
>
> > Use case:
> > In any environment - time to time - administrator needs to perform a
> > maintenance. Current stop sequence of cloudstack management server will
> > ignore the fact that there may be long running async jobs - and terminate
> > the process. This in turn can create a poor user experience and
> occasional
> > inconsistency  in cloudstack db.
> >
> > This is especially painful in large environments where the user has
> > thousands of nodes and there is a continuous patching that happens around
> > the clock - that requires migration of workload from one node to another.
> >
> > With that said - i've created a script that monitors the async job queue
> > for given MS and waits for it complete all jobs. More details are posted
> > below.
> >
> > I'd like to introduce "graceful-shutdown" into the systemctl/service of
> > cloudstack-management service.
> >
> > The details of how it will work is below:
> >
> > Workflow for graceful shutdown:
> >   Using iptables/firewalld - block any connection attempts on 8080/8443
> (we
> > can identify the ports dynamically)
> >   Identify the MSID for the node, using the proper msid - query async_job
> > table for
> > 1) any jobs that are still running (or job_status=“0”)
> > 2) job_dispatcher not like “pseudoJobDispatcher"
> > 3) job_init_msid=$my_ms_id
> >
> > Monitor this async_job table for 60 minutes - until all async jobs for
> MSID
> > are done, then proceed with shutdown
> > If failed for any reason or terminated, catch the exit via trap
> command
> > and unblock the 8080/8443
> >
> > Comments are welcome
> >
> > Regards,
> > ilya
> >
>
>
>
> --
> Rafael Weingärtner
>

Re: [DISCUSS] CloudStack graceful shutdown

[ilya musayev]

Andrija

This is the reason for this enhancement, snapshot, migration and others -
are all async jobs - and therefore should be tracked in async_job table
under specific MS.It is known they may take a while to complete and last
thing we want is to interrupt it.

Depending on what value you have set in Configurations - it may time out -
but continue working on the background.. meaning cloudstack will stop
tracking the async job beyond specific interval - but cloudstack agent will
push forward.

I dont see a harm of taking the server offline - if there are no jobs that
are being tracked.

However - we should not stop the server - if we identify any jobs that are
still active. The user can decide to append the forceful shutdown after the
graceful one if he feels like it. For example

[shell] # service cloudstack-management graceful-shutdown; service
cloudstack-management shutdown

For your issue,

Please check the value for "job.cancel.threshold.minutes"

  "category": "Advanced",

  "description": "Time (in minutes) for async-jobs to be forcely
cancelled if it has been in process for long",

  "name": "job.cancel.threshold.minutes",

  "value": "60"


I propose for the graceful shutdown command to source
"job.cancel.threshold.minutes"
as a max value - before giving up on the endeavor.


The only issue i'm on the fence about - is blocking access to 8080/8443 -
if you have a single node setup.


There is a chance you may block the access to cloudstack for over an hour -
and that may not be what you intended.


Perhaps we add a parameter in db.properties for
"graceful.shutdown.block.api.server = true/false"


Regards,

ilya

On Wed, Apr 4, 2018 at 2:22 PM, Andrija Panic <andrija.pa...@gmail.com>
wrote:

> One comment here (I had to shutdown whole DC for few hours recently),
> please make sure to perhaps at least consider snapshoting process as the
> special case - it can take few hours for snapshot to complete really (copy
> process from Primary to Secondary Storage)
>
> I did (in my recent unfortunate DC shutdown), actually stop MS (we also
> have script to identify running async jobs), so we stop it once safe, but
> any running qemu-img processes (we use kVM) need to be killed manually
> (ansbile) after MS is stopped, etc,etc...
>
> I can assume most jobs can take reasonable long time to complete, but
> snapshots are probably the biggest exceptions as can take extremely long
> time to complete...
>
> Cheers
>
> On 4 April 2018 at 22:46, Tutkowski, Mike <mike.tutkow...@netapp.com>
> wrote:
>
> > I may be remembering this incorrectly, but from what I recall, if a
> > resource is owned by one MS and a request related to that resource comes
> in
> > to another MS, the MS that received the request passes it on to the other
> > MS.
> >
> > > On Apr 4, 2018, at 2:36 PM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> > >
> > > Big +1 for this feature; I only have a few doubts.
> > >
> > > * Regarding the tasks/jobs that management servers (MSs) execute; are
> > these
> > > tasks originate from requests that come to the MS, or is it possible
> that
> > > requests received by one management server to be executed by other? I
> > mean,
> > > if I execute a request against MS1, will this request always be
> > > executed/threated by MS1, or is it possible that this request is
> executed
> > > by another MS (e.g. MS2)?
> > >
> > > * I would suggest that after we block traffic coming from
> > 8080/8443/8250(we
> > > will need to block this as well right?), we can log the execution of
> > tasks.
> > > I mean, something saying, there are XXX tasks (enumerate tasks) still
> > being
> > > executed, we will wait for them to finish before shutting down.
> > >
> > > * The timeout (60 minutes suggested) could be global settings that we
> can
> > > load before executing the graceful-shutdown.
> > >
> > > On Wed, Apr 4, 2018 at 5:15 PM, ilya musayev <
> > ilya.mailing.li...@gmail.com>
> > > wrote:
> > >
> > >> Use case:
> > >> In any environment - time to time - administrator needs to perform a
> > >> maintenance. Current stop sequence of cloudstack management server
> will
> > >> ignore the fact that there may be long running async jobs - and
> > terminate
> > >> the process. This in turn can create a poor user experience and
> > occasional
> > >> inconsistency  in cloudstack db.
> > >>
> > >> This is especially 

Re: [DISCUSS] New VPN implementation based on IKEv2 backed by Vault

[ilya musayev]

Khosrow

My 2c, little less than ideal to manage yet another external end point
like.

While i understand that it makes it easier to manage certificates - it also
means going forward - Vault implementation will become a requirement to
validate future ACS release.

With that said - i do like the proposal and not against it, but:
1) Please consider decoupling it from cloudstack-management server - and
release it as server plugin
2) Test coverage must be sufficient enough to validate the functionality
(perhaps mock vault endpoints and response)

Regards,
ilya

On Wed, Apr 4, 2018 at 10:49 AM, Khosrow Moossavi 
wrote:

> Thanks Paul, the proposed feature will enable the functionality to use
> Vault to
> act as CA if enabled in ACS, otherwise will fall back to "default"
> implementation
> which Rohit has already done.
>
>
> On Wed, Apr 4, 2018 at 12:29 PM, Paul Angus 
> wrote:
>
> > You guys should speak to Rohit about the CA framework.  CloudStack can
> > manage certificates now, including creating them itself and acting as a
> > root CA.
> >
> >
> >
> >
> > Kind regards,
> >
> > Paul Angus
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: Rafael Weingärtner 
> > Sent: 04 April 2018 16:51
> > To: dev 
> > Subject: Re: [DISCUSS] New VPN implementation based on IKEv2 backed by
> > Vault
> >
> > Thanks for sharing the details. Now I have a better perspective of the
> > proposal.It is an interesting integration of CloudStack VPN service with
> > Vault PKI feature.
> >
> > On Wed, Apr 4, 2018 at 12:38 PM, Khosrow Moossavi <
> kmooss...@cloudops.com>
> > wrote:
> >
> > > One of the things Vault does is essentially one of the thing Let's
> > > Encrypt does, acting as CA and generating/signing certificates.
> > >
> > > From the Vault website itself:
> > >
> > > "HashiCorp Vault secures, stores, and tightly controls access to
> > > tokens, passwords, certificates, API keys, and other secrets in modern
> > > computing. Vault handles leasing, key revocation, key rolling, and
> > > auditing. Through a unified API, users can access an encrypted
> > > Key/Value store and network encryption-as-a-service, or generate AWS
> > > IAM/STS credentials, SQL/NoSQL databases, X.509 certificates, SSH
> > > credentials, and more."
> > >
> > > In our case we are going to use Vault as PKI backend engine, to act as
> > > Root CA, sign certificates, handle CRL (Certificate Revocation List),
> > > etc.
> > > Technically we can
> > > do these with Let's Encrypt, but I haven't started exploring the
> > > possibilities or potential limitation. Using external services (such
> > > as Let's Encrypt) or going forward with Bring You Own Certificate
> > > model would be for future, it they ever made sense to do.
> > >
> > >
> > >
> > > On Wed, Apr 4, 2018 at 11:20 AM, Rafael Weingärtner <
> > > rafaelweingart...@gmail.com> wrote:
> > >
> > > > Got it. Thanks for the explanations.
> > > > There is one other thing I do not understand. This Vault thing that
> > > > you mention, how does it work? Is it similar to let's encrypt?
> > > >
> > > > On Wed, Apr 4, 2018 at 12:15 PM, Khosrow Moossavi <
> > > kmooss...@cloudops.com>
> > > > wrote:
> > > >
> > > > > On Wed, Apr 4, 2018 at 10:36 AM, Rafael Weingärtner <
> > > > > rafaelweingart...@gmail.com> wrote:
> > > > >
> > > > > > So, you need a certificate that is signed by the CA that is used
> > > > > > by
> > > the
> > > > > VPN
> > > > > > service. Is that it?
> > > > > >
> > > > > >
> > > > > Correct, a self signed "server certificate" against CA, to be
> > > > > installed directly on VR.
> > > > >
> > > > >
> > > > > >
> > > > > > It has been a while that I do not configure these VPN systems;
> > > > > > do you
> > > > > need
> > > > > > access to the private key of the CA? Or, does the program simply
> > > > validate
> > > > > > the user (VPN client) certificate to see if it is issued by a
> > > specific
> > > > > CA?
> > > > > > I believe it also needs the public key of the user to execute
> > > > > > the
> > > > > handshake
> > > > > > and create the connection.
> > > > > >
> > > > > >
> > > > > >
> > > > > No, end user only needs to have Root CA at hand, to *trust* it.
> > > > > Both
> > > the
> > > > > "Server
> > > > > Certificate" and "Server Private Key" are sensitive information
> > > > > and
> > > only
> > > > > exist  on
> > > > > VR.
> > > > >
> > > > > User then can go ahead and install the Root CA on their local
> > > > > machine
> > > and
> > > > > open
> > > > > up VPN connection with strongSwan client of the correspondning OS
> > > they're
> > > > > on
> > > > > import the Root CA, and their credential (EAP on VPN side), and
> > > > > that's
> > > > it.
> > > > >
> > > > >
> > > > > >
> > > > > >
> > > > > > On Wed, Apr 4, 2018 at 11:22 AM, Khosrow 

[DISCUSS] CloudStack graceful shutdown

[ilya musayev]

Use case:
In any environment - time to time - administrator needs to perform a
maintenance. Current stop sequence of cloudstack management server will
ignore the fact that there may be long running async jobs - and terminate
the process. This in turn can create a poor user experience and occasional
inconsistency  in cloudstack db.

This is especially painful in large environments where the user has
thousands of nodes and there is a continuous patching that happens around
the clock - that requires migration of workload from one node to another.

With that said - i've created a script that monitors the async job queue
for given MS and waits for it complete all jobs. More details are posted
below.

I'd like to introduce "graceful-shutdown" into the systemctl/service of
cloudstack-management service.

The details of how it will work is below:

Workflow for graceful shutdown:
  Using iptables/firewalld - block any connection attempts on 8080/8443 (we
can identify the ports dynamically)
  Identify the MSID for the node, using the proper msid - query async_job
table for
1) any jobs that are still running (or job_status=“0”)
2) job_dispatcher not like “pseudoJobDispatcher"
3) job_init_msid=$my_ms_id

Monitor this async_job table for 60 minutes - until all async jobs for MSID
are done, then proceed with shutdown
If failed for any reason or terminated, catch the exit via trap command
and unblock the 8080/8443

Comments are welcome

Regards,
ilya

Re: ConfigDrive status

[ilya musayev]

Lucian

We reported 3-4 issues with config drive. It’s being worked on. It does
work - but not per agreed upon specifications.

See below


https://issues.apache.org/jira/browse/CLOUDSTACK-10287
https://issues.apache.org/jira/browse/CLOUDSTACK-10288
https://issues.apache.org/jira/browse/CLOUDSTACK-10289
https://issues.apache.org/jira/browse/CLOUDSTACK-10290

Regards
Ilya

On Mon, Mar 26, 2018 at 8:58 AM Dag Sonstebo 
wrote:

> Hi Lucian,
>
> I’m maybe not the right person to answer this – but my understanding is it
> only kicks in when you have a network offering without a VR, at which point
> the metadata etc. is presented as the config drive. Happy to be corrected
> on this.
>
> We did however have a really major gotcha 18 months ago – when a customer
> did a CloudStack upgrade and ended up with new unexpected config drives
> causing changes to all the VMware disk controller addressing – meaning VMs
> wouldn’t boot, couldn’t see disks, etc. If you use VMware I would test
> beforehand.
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
> On 26/03/2018, 14:50, "Nux!"  wrote:
>
> Hi,
>
> I am interested in the ConfigDrive feature.
> Before I potentially waste time on it, is anyone around here using it
> or can clarify whether it's usable or not or gotchas etc?
>
> Regards,
> Lucian
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
>
>
> dag.sonst...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>

Re: Welcoming Mike as the new Apache CloudStack VP

[ilya musayev]

Welcome Mike, thank you Wido!

On Mon, Mar 26, 2018 at 8:59 AM Simon Weller 
wrote:

> Thanks for all of your hard work Wido, we really appreciate it.
>
>
> Congratulations Mike!
>
>
> - Si
>
> 
> From: Wido den Hollander 
> Sent: Monday, March 26, 2018 9:11 AM
> To: dev@cloudstack.apache.org; us...@cloudstack.apache.org
> Subject: Welcoming Mike as the new Apache CloudStack VP
>
> Hi all,
>
> It's been a great pleasure working with the CloudStack project as the
> ACS VP over the past year.
>
> A big thank you from my side for everybody involved with the project in
> the last year.
>
> Hereby I would like to announce that Mike Tutkowski has been elected to
> replace me as the Apache Cloudstack VP in our annual VP rotation.
>
> Mike has a long history with the project and I am are happy welcome him
> as the new VP for CloudStack.
>
> Welcome Mike!
>
> Thanks,
>
> Wido
>

Re: [DISCUSS] CloudStack Connection Pools

[ilya musayev]

When everything works smooth - the end user does not need to know whats
under the hood.

However, when things begin to swing and your hands are tied to just one CP
- what do you do?

On Wed, Mar 14, 2018 at 10:50 AM, Khosrow Moossavi 
wrote:

> Why would we want to expose this choice to administrator of Cloudstack
> whose responsibility
> is to keep it running and not knowing about the inner-mechanic of how it
> works. right? It's not
> like that we're giving them a choice of which database to connect to.
>
> So on that note, I would say we need to agree on any of those CP libraries
> and implement, the
> same way we chose for example log4j or slf4j over one another, or any other
> _library_ we use.
>
> Khosrow Moossavi
>
> CloudOps
>
>
>
> On Wed, Mar 14, 2018 at 10:36 AM, Nicolas Vazquez <
> nicolas.vazq...@shapeblue.com> wrote:
>
> > Thanks Khosrow and Rafael. You both agree on Spring Data as the best
> > option, I see it would require a big effort and commitment to migrate to
> > it, therefore it can take some (long) time to achieve it.
> >
> > As a more viable option, would you agree on supporting different
> > connection pool management libraries and letting the administrator choose
> > which one to use? (DBCP 1.4 as default)
> >
> > 
> > From: Rafael Weingärtner 
> > Sent: Tuesday, March 13, 2018 8:52:50 AM
> > To: dev
> > Subject: Re: [DISCUSS] CloudStack Connection Pools
> >
> > Spring data would be awesome. It is very flexible and has a very good
> API.
> > However, this would require commitment from our side to slowly migrate
> > things to it.
> >
> > Regarding the connection pool management libraries; I would prefer either
> > C3P0 or 2.* DBCP. The other two sound trendy, but I worry about this type
> > of project in the long run. Both DBCP from Apache and C3P0 from Hibernate
> > (RedHat) sound a more reasonable selection for me. They have been around
> > for years, and have a solid community base already.
> >
> > On Mon, Mar 12, 2018 at 11:31 PM, Khosrow Moossavi <
> kmooss...@cloudops.com
> > >
> > wrote:
> >
> > > Hi Nicolas
> > >
> > > From my past experiences, I prefer 1) HikariCP 2) Tomcat Pool 3) C3P0
> 4)
> > > DBCP in that order. Although I don't have
> > > any benchmark of my own to provide, and the ones you mentioned are
> really
> > > informative anyway.
> > >
> > > To me the broader subject is the _one_ who uses the pool, I mean if the
> > > transactions are handled in a faster way and
> > > released sooner and with shorter locks, generally speaking if it's more
> > > efficient, I don't think from ACS point of view
> > > there won't be much difference between the above mentioned options.
> > >
> > > On the same subject, it might be more interesting to use Spring Boot in
> > > general and Spring Boot Data in particular
> > > rather than only changing the CP functionality, and slowly
> > refactor/retire
> > > the DAO layer in favor of Spring Boot equivalent
> > > implementation.
> > >
> > >
> > > Khosrow Moossavi
> > >
> > > CloudOps
> > >
> > >
> > >
> > > On Mon, Mar 12, 2018 at 9:32 PM, Nicolas Vazquez <
> > > nicolas.vazq...@shapeblue.com> wrote:
> > >
> > > > Hi all,
> > > >
> > > >
> > > > I would like to introduce a topic for discussion, regarding DB
> > connection
> > > > pools used in CloudStack, currently Apache Commons DBCP 1.4 (
> > > > http://commons.apache.org/) is used. I've been investigating this
> > topic
> > > > as we are having complains of random issues on MySQL connection pool
> on
> > > > large environments. Please let me know if this topic has already been
> > > > discussed before.
> > > >
> > > >
> > > > First of all, DBCP 1.4 has been released on 2010 (
> > > > https://commons.apache.org/proper/commons-dbcp/changes-report.html),
> > and
> > > > no minor/patch version has been released since then. It seems to work
> > in
> > > > high performance with relatively low traffic and low load
> applications.
> > > > However, it is single threaded, and in order to be thread-safe, the
> > > entire
> > > > pool needs to be locked. It is also reported that an CPU and
> concurrent
> > > > threads increases, the performance gets affected. This is a serious
> > issue
> > > > on highly concurrent systems, such as CloudStack.
> > > >
> > > >
> > > > I've been investigating some options to replace it:
> > > > - The first option can be upgrading to version 2.x. Issues on
> > performance
> > > > and concurrency could be solved using this version.
> > > > - Tomcat JDBC Connection Pool. Please check:
> > https://tomcat.apache.org/
> > > > tomcat-7.0-doc/jdbc-pool.html.
> > > >
> > > > - Other replacement options found: BoneCP, C3P0, HikariCP
> > > >
> > > >
> > > > Given these options, I've been looking for benchmarks to compare them
> > > (*).
> > > > Looks like HikariCP (http://brettwooldridge.github.io/HikariCP/)
> could
> > > be
> > > > the best replacement, improving performance and 

Re: [DISCUSS] CloudStack Connection Pools

[ilya musayev]

Rafael and Khosrow,

I actually think quite the opposite on DBCP development. Please pull up
latest commits and release notes on DBCP.
An assumption that its an Apache project and therefore will live and
flourish - is a dangerous one. It comes down to who supports the project
and if there funding to back it.

https://github.com/apache/commons-dbcp

While there are some commits - there is no major innovation happening and
its limited to few individuals.

In terms of performance HikariCP blows all other contenders by leaps and
bounds. It also holds ASF license - so we should have no issues
intergrating.
HikariCP-bench-2.6.0.png


You can see active development and its backed by a company that is funded
sufficiently well.

C3P0 - is GNU licensed based - which if i understand correctly - cannot be
used with ASF project.

The reason it was proposed to allow a flexibility for connection pool
providers - is to allow for user to switch between one or the other - in
case you come across an issue with one of the two.

It would be something an advanced user knows and understands. Not providing
the flexibility to choose between different CP - makes troubleshooting
extremely difficult.

Lastly - DBCP 1.4 log4j support is none existent - therefore you cant
troubleshoot the connection pool issues with ease. We can upgrade to 2.x
and see if it helps - but given the slow development and performance
limitation - i dont see a huge benefit. Nevertheless, we know DBCP 2.x is
stable - it would be nice to have a fallback as an option in case the user
suspects his issues are due to connection pool handling.

Just point of reference, internally - some of apps switched from DBCP to
HikariCP and we saw tremendous boost in performance (10x+) and
troubleshooting became much easier. I dont want to assume we will get 10x
boost - but i think we will certainly do much better than what 8 year old
DBCP has to offer.

Regards,
ilya

On Wed, Mar 14, 2018 at 10:58 AM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> I agree with Khosrow. Even though the idea of externalizing a configuration
> like this seems interesting,  I believe that it would bring more
> complications than benefits. And, at the end of the day operators would
> only use the default.
>
> On Wed, Mar 14, 2018 at 2:50 PM, Khosrow Moossavi 
> wrote:
>
> > Why would we want to expose this choice to administrator of Cloudstack
> > whose responsibility
> > is to keep it running and not knowing about the inner-mechanic of how it
> > works. right? It's not
> > like that we're giving them a choice of which database to connect to.
> >
> > So on that note, I would say we need to agree on any of those CP
> libraries
> > and implement, the
> > same way we chose for example log4j or slf4j over one another, or any
> other
> > _library_ we use.
> >
> > Khosrow Moossavi
> >
> > CloudOps
> >
> >
> >
> > On Wed, Mar 14, 2018 at 10:36 AM, Nicolas Vazquez <
> > nicolas.vazq...@shapeblue.com> wrote:
> >
> > > Thanks Khosrow and Rafael. You both agree on Spring Data as the best
> > > option, I see it would require a big effort and commitment to migrate
> to
> > > it, therefore it can take some (long) time to achieve it.
> > >
> > > As a more viable option, would you agree on supporting different
> > > connection pool management libraries and letting the administrator
> choose
> > > which one to use? (DBCP 1.4 as default)
> > >
> > > 
> > > From: Rafael Weingärtner 
> > > Sent: Tuesday, March 13, 2018 8:52:50 AM
> > > To: dev
> > > Subject: Re: [DISCUSS] CloudStack Connection Pools
> > >
> > > Spring data would be awesome. It is very flexible and has a very good
> > API.
> > > However, this would require commitment from our side to slowly migrate
> > > things to it.
> > >
> > > Regarding the connection pool management libraries; I would prefer
> either
> > > C3P0 or 2.* DBCP. The other two sound trendy, but I worry about this
> type
> > > of project in the long run. Both DBCP from Apache and C3P0 from
> Hibernate
> > > (RedHat) sound a more reasonable selection for me. They have been
> around
> > > for years, and have a solid community base already.
> > >
> > > On Mon, Mar 12, 2018 at 11:31 PM, Khosrow Moossavi <
> > kmooss...@cloudops.com
> > > >
> > > wrote:
> > >
> > > > Hi Nicolas
> > > >
> > > > From my past experiences, I prefer 1) HikariCP 2) Tomcat Pool 3) C3P0
> > 4)
> > > > DBCP in that order. Although I don't have
> > > > any benchmark of my own to provide, and the ones you mentioned are
> > really
> > > > informative anyway.
> > > >
> > > > To me the broader subject is the _one_ who uses the pool, I mean if
> the
> > > > transactions are handled in a faster way and
> > > > released sooner and with shorter locks, generally speaking if it's
> more
> > > > efficient, I don't think from ACS point of view
> > > > there 

Re: I'd like to introduce you to Khosrow

[ilya musayev]

Great news!

On Thu, Feb 22, 2018 at 3:40 PM Pierre-Luc Dion  wrote:

> Hi fellow colleagues,
>
> I might be a bit late with this email...
>
> I'd like to introduce Khosrow Moossavi, who recently join our team and his
> focus is currently exclusively on dev for Cloudstack with cloud.ca.
>
> Our 2 current priorities are:
> -fixing VRs,SVMs to run has HVM VMs in xenserver.
> - redesign, or rewrite, the remote management vpn for vpc, poc in progress
> for IKEv2...
>
>
>
> Some of you might have interact with him already.
>
>
> Also, we are going to be more active for the upcomming 4.12 release.
>
>
> Cheers!
>

Re: Performance considerations related to Intel Meltdown on KVM CPU types

[ilya musayev]

Thanks for sharing

On Mon, Jan 8, 2018 at 7:11 AM Nux!  wrote:

> Hello,
>
> Just stumbled upon this
> https://twitter.com/berrange/status/950209752486817792
>
> "ensure KVM guest CPU model you choose has the "pcid" feature, otherwise
> guests will suffer terrible performance from the Meltdown fixes. This means
> using a named Haswell, Broadwell or Skylake based model or host passthrough"
>
>
> This means whoever is running with the KVM default CPU (like I do) as
> opposed to specific ones or host passthrough needs to change this in order
> to avoid bad performance once the new mitigating kernel is installed.
>
>
> Bad news is older Xeons do not support this, check if "invpcid" flag shows
> up in /proc/cpuinfo (you might see "pcid", that one is not enough).
>
>
>
>
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>

Re: [VOTE] Clean up old and obsolete branches.

[ilya musayev]

+1

On Tue, Jan 2, 2018 at 2:41 PM Boris Stoyanov 
wrote:

> 0
>
>
> boris.stoya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> > On 2 Jan 2018, at 22:13, Khosrow Moossavi 
> wrote:
> >
> > +1
> >
> > Khosrow Moossavi
> > CloudOps
> >
> > On Jan 2, 2018 14:07, "Nicolas Vazquez" 
> > wrote:
> >
> >> +1
> >>
> >> 
> >> From: Simon Weller 
> >> Sent: Tuesday, January 2, 2018 3:38:00 PM
> >> To: dev
> >> Subject: Re: [VOTE] Clean up old and obsolete branches.
> >>
> >> +0
> >>
> >> 
> >> From: Daan Hoogland 
> >> Sent: Tuesday, January 2, 2018 12:19 PM
> >> To: dev
> >> Subject: Re: [VOTE] Clean up old and obsolete branches.
> >>
> >> 0
> >>
> >> On Tue, Jan 2, 2018 at 1:51 PM, Gabriel Beims Bräscher <
> >> gabrasc...@gmail.com
> >>> wrote:
> >>
> >>> +1
> >>>
> >>> 2018-01-02 9:46 GMT-02:00 Rafael Weingärtner <
> >> rafaelweingart...@gmail.com
>  :
> >>>
>  Hope you guys had great holy days!
> 
>  Resuming the discussion we started last year in [1]. It is time to
> vote
> >>> and
>  then to push (if the vote is successful) the protocol defined to our
> >>> wiki.
>  Later we can start enforcing it.
>  I will summarize the protocol for branches in the official repository.
> 
>    1. We only maintain the master and major release branches. We
> >>> currently
>    have a system of X.Y.Z.S. I define major release here as a release
> >>> that
>    changes either ((X or Y) or (X and Y));
>    2. We will use tags for versioning. Therefore, all versions we
> >> release
>    are tagged accordingly, including minor and security releases;
>    3. When releasing the “SNAPSHOT” is removed and the branch of the
>    version is created (if the version is being cut from master). Rule
> >> (1)
>  one
>    is applied here; therefore, only major releases will receive
> >> branches.
>    Every release must have a tag according to the format X.Y.Z.S. After
>    releasing, we bump the POM of the version to next available
> >> SNAPSHOT;
>    4. If there's a need to fix an old version, we work on HEAD of
>    corresponding release branch. For instance, if we want to fix
> >>> something
>  in
>    release 4.1.1.0, we will work on branch 4.1, which will have the POM
>  set to
>    4.1.2.0-SNAPSHOT;
>    5. People should avoid (it is not forbidden though) using the
> >> official
>    apache repository to store working branches. If we want to work
>  together on
>    some issues, we can set up a fork and give permission to interested
>  parties
>    (the official repository is restricted to committers). If one uses
> >> the
>    official repository, the branch used must be cleaned right after
>  merging;
>    6. Branches not following these rules will be removed if they have
> >> not
>    received attention (commits) for over 6 (six) months;
>    7. Before the removal of a branch in the official repository it is
>    mandatory to create a Jira ticket and send a notification email to
>    CloudStack’s dev mailing list. If there are no objections, the
> >> branch
>  can
>    be deleted seven (7) business days after the notification email is
> >>> sent;
>    8. After the branch removal, the Jira ticket must be closed.
> 
>  Let’s go to the poll:
>  (+1) – I want to work using this protocol
>  (0) – Indifferent to me
>  (-1) – I prefer the way it is not, without any protocol/guidelines
> 
> 
>  [1]
>  http://mail-archives.apache.org/mod_mbox/cloudstack-dev/
>  201711.mbox/%3CCAHGRR8ozDBX%3DJJewLz_cu-YP9vA3TEmesvxGArTDBPerAOj8Cw%
>  40mail.gmail.com%3E
> 
>  --
>  Rafael Weingärtner
> 
> >>>
> >>
> >>
> >>
> >> --
> >> Daan
> >>
> >> nicolas.vazq...@shapeblue.com
> >> www.shapeblue.com
> >> ,
> >> @shapeblue
> >>
> >>
> >>
> >>
>
>

Re: [DISCUSS] Management server (pre-)shutdown to avoid killing jobs

[ilya musayev]

I very much agree with Paul, we should consider moving into resilient model
with least dependence I.e ha-proxy..

Send a notification to partner MS to take over the job management would be
ideal.

On Mon, Dec 18, 2017 at 9:28 AM Paul Angus  wrote:

> Hi Marc-Aurèle,
>
> Personally, my utopia would be to be able to pass async jobs between mgmt.
> servers.
> So rather than waiting in indeterminate time for a snapshot to complete,
> monitoring the job is passed to another management server.
>
> I would LOVE that something like Zookeeper monitored the state of the
> mgmt. servers, so that 'other' management servers could take over the async
> jobs in the (unlikely) event that a management server becomes unavailable.
>
>
>
> Kind regards,
>
> Paul Angus
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Marc-Aurèle Brothier [mailto:ma...@exoscale.ch]
> Sent: 18 December 2017 13:56
> To: dev@cloudstack.apache.org
> Subject: [DISCUSS] Management server (pre-)shutdown to avoid killing jobs
>
> Hi everyone,
>
> Another point, another thread. Currently when shutting down a management
> server, despite all the "stop()" method not being called as far as I know,
> the server could be in the middle of processing an async job task. It will
> lead to a failed job since the response won't be delivered to the correct
> management server even though the job might have succeed on the agent. To
> overcome this limitation due to our weekly production upgrades, we added a
> pre-shutdown mechanism which works along side HA-proxy. The management
> server keeps a eye onto a file "lb-agent" in which some keywords can be
> written following the HA proxy guide (
> https://cbonte.github.io/haproxy-dconv/1.9/configuration.html#5.2-agent-check
> ).
> When it finds "maint", "stopped" or "drain", it stops those threads:
>  - AsyncJobManager._heartbeatScheduler: responsible to fetch and start
> execution of AsyncJobs
>  - AlertManagerImpl._timer: responsible to send capacity check commands
>  - StatsCollector._executor: responsible to schedule stats command
>
> Then the management server stops most of its scheduled tasks. The correct
> thing to do before shutting down the server would be to send
> "rebalance/reconnect" commands to all agents connected on that management
> server to ensure that commands won't go through this server at all.
>
> Here, HA-proxy is responsible to stop sending API requests to the
> corresponding server with the help of this local agent check.
>
> In case you want to cancel the maintenance shutdown, you could write
> "up/ready" in the file and the different schedulers will be restarted.
>
> This is really more a change for operation around CS for people doing live
> upgrade on a regular basis, so I'm unsure if the community would want such
> a change in the code base. It goes a bit in the opposite direction of the
> change for removing the need of HA-proxy
> https://github.com/apache/cloudstack/pull/2309
>
> If there is enough positive feedback for such a change, I will port them
> to match with the upstream branch in a PR.
>
> Kind regards,
> Marc-Aurèle
>

Re: [README][Quarterly Call] - CloudStack Development, Blockers and Community Efforts

[ilya musayev]

Hi Folks

Apologies for dropping the ball on this initiative. I'm going to be away
for the next week or two dealing with personal issues.

Regards
Ilya

On Wed, Sep 6, 2017 at 11:21 AM ilya musayev <ilya.mailing.li...@gmail.com>
wrote:

> Hi All,
>
> We had a great discussion earlier today. I will update the meeting
> notes and start additional mail threads on behalf of pending parties
> for things we need to discuss.
>
> Meeting notes will be here:
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/2017-09-06+Meeting+notes
>
> The next call will be held 3 months from now. If you feel we need to
> do something sooner or can think of a way to make the communication
> better, please let me know.
>
> Thank you everyone for taking the time out of your busy schedule to
> attend this call.
>
> Regards,
> ilya
>

Re: [README][Quarterly Call] - CloudStack Development, Blockers and Community Efforts

[ilya musayev]

Hi All,

We had a great discussion earlier today. I will update the meeting
notes and start additional mail threads on behalf of pending parties
for things we need to discuss.

Meeting notes will be here:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/2017-09-06+Meeting+notes

The next call will be held 3 months from now. If you feel we need to
do something sooner or can think of a way to make the communication
better, please let me know.

Thank you everyone for taking the time out of your busy schedule to
attend this call.

Regards,
ilya


binxg9nzlrjxR.bin
Description: application/pgp-encrypted

[NOTICE] Meeting with Accelerite Leadership

[ilya musayev]

Dear CloudStackers,

Last week, Johh Kinsella and myself were suppose to meet with Accelerite
leadership team. Unfortunately John could not make it - so i was alone.

We discussed ways we can improve community collaboration and leverage
Accelerite"s resources to align and drive larger community agenda including
extendes roadmap.

Many topics have been mentioned, below is the summary of our discussion. I
will list things in the order i see being important.


---
1) Proposal was made to have a quarterly call (or more often as needed)
with all interested parties to discuss:
Upcoming Features you are working on developing (to avoid collision
and
maintain the roadmap)
Blockers that are impacting release and adoption
Other topics

The length of the call would be 90 minutes. Each party will get a
fair
amount of time. The agenda will be collated and presented prior to the
call with a link to FS on Confluence and time allotted for each topic.

Minutes will be taken and posted on dev list. If there are issues and or
suggestions, we will note it down in few sentences, identify interested
parties and have them do a "post" discussion on the mailing list.

The proposed date and time  - Thursday August 17th 9AM PT

--
2) Accelerite is considering funding a position for a person who will be
working within community - as community manager. Help organize and
facilitate discussions, make sure Confluence and JIRA are up to date,
help new users with answering basic questions or finding right
individual to assist with solution. While funded by Accelerite - it must
be clear that the person is working with/for Apache CloudStack project.

3) Marketing was mentioned, i suggested we do more press releases - and
possibly make use of interns

4) OpenStack VS CloudStack (unbiased technology comparison), there is a
common question - we need to come up something that can help justify
Apache CloudStack to clients leadership

5) Cinder integration with Cloudstack was mentioned - but no solid plans
yet.

6) Creating Appliances of CloudStack - that are ready to be consumed and
user can spin nested VMs to try CloudStack effortlessly

7) CoudStack Template Repository (plugin)- there is a code written for it by
Citrix and resides on ASF git - but for some reason it was dropped or
never completed. If we can give user a rich marketplace of appliances to
consume - we will certainly get a good edge. This can improve the adoption.

8) MeetUps - we need to re-kickstart this initiative within SF Bay Area
and stream it to other locations/meetups.

9) Demo environment of CloudStack.  David mentioned Citrix donated gear
is in one of ASF locations - but sitting idle. I proposed we make use of
it and let new CloudStack explorers try it out - without the hassle of
deploying it.

10) If we can get CloudStack into EPEL fedora and ubuntu upstream
repositories - it will help with adoption as well.

Please let me know if you would be interested in item #1, which is
quarterly meeting. The proposed time is 9am PST, August 17th.

I will help setting up the first few initial calls and be a moderator.

Looking forward to your comments

Regards
ilya

Re: failed to start CS 4.5

[ilya musayev]

Dan,

Thats not an issue, its just means java was unable to unzip a jar, but 
its really not impacting.


I've upgraded several test environments to 4.5 and while i see this 
information error, everything works. We should investigate why we see 
this, but as mentioned earlier - its non impacting.


Did you import the systemvm 4.5 template beforehand?

Regards,
ilya

On 3/13/15 9:16 AM, d...@soleks.com wrote:

Hi Devs,

I've downloaded 4.5-RC4, build RPM without issues (centos 6.6). 
Database conversion went smoothly, but eventually wasn't able to start 
cloudstack-management, only one error that i found in the log, not 
sure if it's road-blocker or easy-to-fix issue, however here is some log:



2015-03-12 17:35:57,110 DEBUG [o.a.c.f.s.OnwireClassRegistry] 
(main:null) Encountered IOException

java.io.IOException: Stream closed
at 
java.util.zip.ZipInputStream.ensureOpen(ZipInputStream.java:66)
at 
java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:115)
at 
java.util.jar.JarInputStream.getNextEntry(JarInputStream.java:142)
at 
java.util.jar.JarInputStream.getNextJarEntry(JarInputStream.java:179)
at 
org.apache.cloudstack.framework.serializer.OnwireClassRegistry.getFromJARFile(OnwireClassRegistry.java:162)
at 
org.apache.cloudstack.framework.serializer.OnwireClassRegistry.getClasses(OnwireClassRegistry.java:119)
at 
org.apache.cloudstack.framework.serializer.OnwireClassRegistry.getClasses(OnwireClassRegistry.java:95)
at 
org.apache.cloudstack.framework.serializer.OnwireClassRegistry.scan(OnwireClassRegistry.java:76)

at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1608)
at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1549)
at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1479)
at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
at 
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:295)
at 
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
at 
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:292)
at 
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at 
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:628)
at 
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932)
at 
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479)
at 
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
at 
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
at 
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
at 
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
at 
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
at 
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
at 
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
at 
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.load(DefaultModuleDefinitionSet.java:79)
at 
org.apache.cloudstack.spring.module.factory.ModuleBasedContextFactory.loadModules(ModuleBasedContextFactory.java:37)
at 
org.apache.cloudstack.spring.module.factory.CloudStackSpringContext.init(CloudStackSpringContext.java:70)
at 

Re: failed to start CS 4.5

[ilya musayev]

Post the complete log, we cant help with few lines..

On 3/13/15 1:35 PM, d...@soleks.com wrote:


Hi Ilya,

Yes, system VM were imported into CS. If that error with unzip can be 
ignored, then the last log entry in managment-server.log what i see is:


2015-03-13 13:18:26,792 INFO  [c.c.u.c.ComponentContext] (main:null) 
Starting 
com.cloud.bridge.service.core.ec2.EC2Engine_EnhancerByCloudStack_c995b789
2015-03-13 13:18:26,793 INFO  [c.c.u.c.ComponentContext] (main:null) 
Starting 
com.cloud.bridge.service.controller.s3.ServiceProvider_EnhancerByCloudStack_f4c651fe


after that nothing. getting 404 in webui

description The requested resource () is not available.



Dan,

Thats not an issue, its just means java was unable to unzip a jar, 
but its really not impacting.


I've upgraded several test environments to 4.5 and while i see this 
information error, everything works. We should investigate why we see 
this, but as mentioned earlier - its non impacting.


Did you import the systemvm 4.5 template beforehand?

Regards,
ilya

On 3/13/15 9:16 AM, d...@soleks.com wrote:

Hi Devs,

I've downloaded 4.5-RC4, build RPM without issues (centos 6.6). 
Database conversion went smoothly, but eventually wasn't able to 
start cloudstack-management, only one error that i found in the log, 
not sure if it's road-blocker or easy-to-fix issue, however here is 
some log:



2015-03-12 17:35:57,110 DEBUG [o.a.c.f.s.OnwireClassRegistry] 
(main:null) Encountered IOException

java.io.IOException: Stream closed
   at 
java.util.zip.ZipInputStream.ensureOpen(ZipInputStream.java:66)
   at 
java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:115)
   at 
java.util.jar.JarInputStream.getNextEntry(JarInputStream.java:142)
   at 
java.util.jar.JarInputStream.getNextJarEntry(JarInputStream.java:179)
   at 
org.apache.cloudstack.framework.serializer.OnwireClassRegistry.getFromJARFile(OnwireClassRegistry.java:162)
   at 
org.apache.cloudstack.framework.serializer.OnwireClassRegistry.getClasses(OnwireClassRegistry.java:119)
   at 
org.apache.cloudstack.framework.serializer.OnwireClassRegistry.getClasses(OnwireClassRegistry.java:95)
   at 
org.apache.cloudstack.framework.serializer.OnwireClassRegistry.scan(OnwireClassRegistry.java:76)

   at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
   at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
   at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

   at java.lang.reflect.Method.invoke(Method.java:606)
   at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1608)
   at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1549)
   at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1479)
   at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:521)
   at 
org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:458)
   at 
org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:295)
   at 
org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:223)
   at 
org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:292)
   at 
org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
   at 
org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:628)
   at 
org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:932)
   at 
org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:479)
   at 
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
   at 
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
   at 
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
   at 
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
   at 
org.apache.cloudstack.spring.module.model.impl.DefaultModuleDefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
   at 

Re: [HELP] VR sometimes fail to start

[ilya musayev]

.
LibvirtComputingResource]
(agentRequest-Handler-1:) Executing:
/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patchviasocket.pl
-n
r-2191-VM -p %vpccidr=
10.0.0.0/8%domain=cs2cloud.internal%dns1=8.8.8.8%dns2=%
template=domP%name=r-2191-VM%eth0ip=169.254.2.99%eth0mask=
255.255.0.0%type=vpcrouter%disable_rp_filter=true

2015-03-10 16:45:36,684{GMT} WARN  [kvm.resource.
LibvirtComputingResource]
(Script-5:) Interrupting script.
2015-03-10 16:45:36,686{GMT} WARN  [kvm.resource.
LibvirtComputingResource]
(agentRequest-Handler-1:) Timed out:
/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patchviasocket.pl
-n
r-2191-VM -p %vpccidr=
10.0.0.0/8%domain=cs2cloud.internal%dns1=8.8.8.8%dns2=%
template=domP%name=r-2191-VM%eth0ip=169.254.2.99%eth0mask=
255.255.0.0%type=vpcrouter%disable_rp_filter=true
.  Output is:
2015-03-10 16:45:36,686{GMT} DEBUG [kvm.resource.
LibvirtComputingResource]
(agentRequest-Handler-1:) passcmd failed:timeout
2015-03-10 16:45:36,686{GMT} DEBUG
[resource.virtualnetwork.VirtualRoutingResource]
(agentRequest-Handler-1:)
Trying to connect to 169.254.2.99
2015-03-10 16:45:37,436{GMT} DEBUG [cloud.agent.Agent]
(agentRequest-Handler-2:) Processing command:
com.cloud.agent.api.GetHostStatsCommand
2015-03-10 16:45:37,436{GMT} DEBUG [kvm.resource.
LibvirtComputingResource]
(agentRequest-Handler-2:) Executing: /bin/bash -c idle=$(top -b -n 1|grep
Cpu\(s\):|cut -d% -f4|cut -d, -f2);echo $idle
2015-03-10 16:45:37,967{GMT} DEBUG [kvm.resource.
LibvirtComputingResource]
(agentRequest-Handler-2:) Execution is successful.
2015-03-10 16:45:37,968{GMT} DEBUG [kvm.resource.
LibvirtComputingResource]
(agentRequest-Handler-2:) Executing: /bin/bash -c freeMem=$(free|grep
cache:|awk '{print $4}');echo $freeMem
2015-03-10 16:45:37,973{GMT} DEBUG [kvm.resource.
LibvirtComputingResource]
(agentRequest-Handler-2:) Execution is successful.
2015-03-10 16:45:37,973{GMT} DEBUG [kvm.resource.
LibvirtComputingResource]
(agentRequest-Handler-2:) Executing: /bin/bash -c free|grep Mem:|awk
'{print $2}'
2015-03-10 16:45:37,977{GMT} DEBUG [kvm.resource.
LibvirtComputingResource]
(agentRequest-Handler-2:) Execution is successful.
2015-03-10 16:45:39,692{GMT} DEBUG
[resource.virtualnetwork.VirtualRoutingResource]
(agentRequest-Handler-1:)
Could not connect to 169.254.2.99
2015-03-10 16:45:44,693{GMT} DEBUG
[resource.virtualnetwork.VirtualRoutingResource]
(agentRequest-Handler-1:)
Trying to connect to 169.254.2.99
2015-03-10 16:45:47,700{GMT} DEBUG
[resource.virtualnetwork.VirtualRoutingResource]
(agentRequest-Handler-1:)
Could not connect to 169.254.2.99
2015-03-10 16:45:51,807{GMT} DEBUG [cloud.agent.Agent]
(agentRequest-Handler-3:) Processing command:
com.cloud.agent.api.GetVmStatsCommand
2015-03-10 16:45:52,701{GMT} DEBUG
[resource.virtualnetwork.VirtualRoutingResource]
(agentRequest-Handler-1:)
Unable to logon to 169.254.2.99

On 10 March 2015 at 17:50, ilya musayev ilya.mailing.li...@gmail.com
wrote:

  What kernel are you running? Please include Libvirt and Qemu version.


On 3/10/15 8:21 AM, Andrija Panic wrote:

  Hi,

we are having issues from time to time with starting VR - while creating
VPC.

Here is the few lines from MGMT log:

2015-03-10 16:11:50,018 WARN  [c.c.n.v.VpcManagerImpl]
(Job-Executor-27:ctx-84f4fba4 ctx-970acb35) Failed to start vpc [VPC
[621-VPC-delete] due to
com.cloud.exception.AgentUnavailableException: Resource [Host:788] is
unreachable: Host 788: Unable to start instance due to Unable to start
VM[DomainRouter|r-2189-VM] due to error in finalizeStart, not retrying
   at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(
VirtualMachineManagerImpl.java:1072)


The interesting part is on the Agent itself:


2015-03-10 16:03:28,520{GMT} WARN  [kvm.resource.
LibvirtComputingResource]
(agentRequest-Handler-2:) Timed out:
/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patc
hviasocket.pl
-n
r-2189-VM -p %vpccidr=
10.0.0.0/24%domain=cs640cloud.internal%dns1=8.8.8.8%dns2=%
template=domP%name=r-2189-VM%eth0ip=169.254.2.134%eth0mask=
255.255.0.0%type=vpcrouter%disable_rp_filter=true
.  Output is:


Interesting stuff is - it's seems somewhat random, 3-4  out of 5 works.

Any suggestion are really appreciated, since this is live system :(
ACS 4.3.0, KVM, CEPH

Thanks,

sudo chmod - bad practice

[ilya musayev]

Is there any way we can avoid having chmod in /etc/sudoers file?

This line in sudoers file makes me very uncomfortable

https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a

723 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l723 

724 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l724 
class sudoersConfig(serviceCfgBase):
725 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l725 
def __init__(self, syscfg):
726 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l726 
super(sudoersConfig, self).__init__(syscfg)
727 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l727 
self.serviceName = sudoers
728 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l728 
def config(self):
729 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l729 
try:
730 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l730 
cfo = configFileOps(/etc/sudoers, self)
731 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l731 
cfo.addEntry(cloud ALL , NOPASSWD : /bin/chmod, /bin/cp, /bin/mkdir, /bin/mount, /bin/umount)
732 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l732 
cfo.rmEntry(Defaults, requiretty,  )
733 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l733 
cfo.save()
734 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l734 
return True
735 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l735 
except:
736 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l736 
raise
737 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l737 

738 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l738 
def restore(self):
739 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l739 
return True
740 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l740 




Can this be avoided?

Re: sudo chmod - bad practice

[ilya musayev]

I dont see where MS requires the use of chmod directly based on the 
quick look against ASF grep search, if someone finds it - please let me 
know.


https://git-wip-us.apache.org/repos/asf?p=cloudstack.gita=searchh=a10a9d522c56f707a9674f1486f58697e8c5206ast=greps=chmod



On 3/11/15 4:06 PM, ilya musayev wrote:

Is there any way we can avoid having chmod in /etc/sudoers file?

This line in sudoers file makes me very uncomfortable

https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a

723 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l723 

724 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l724 
class sudoersConfig(serviceCfgBase):
725 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l725 
def __init__(self, syscfg):
726 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l726 
super(sudoersConfig, self).__init__(syscfg)
727 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l727 
self.serviceName = sudoers
728 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l728 
def config(self):
729 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l729 
try:
730 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l730 
cfo = configFileOps(/etc/sudoers, self)
731 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l731 
cfo.addEntry(cloud ALL , NOPASSWD : /bin/chmod, /bin/cp, /bin/mkdir, /bin/mount, /bin/umount)
732 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l732 
cfo.rmEntry(Defaults, requiretty,  )
733 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l733 
cfo.save()
734 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l734 
return True
735 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l735 
except:
736 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l736 
raise
737 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l737 

738 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l738 
def restore(self):
739 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l739 
return True
740 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=python/lib/cloudutils/serviceConfig.py;hb=a10a9d522c56f707a9674f1486f58697e8c5206a#l740 




Can this be avoided?

Re: Overprovisioning of local storage

[ilya musayev]

fixed in 4.4, but i'd go with 4.5

https://issues.apache.org/jira/browse/CLOUDSTACK-7752



On 3/10/15 6:47 AM, Wido den Hollander wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

I've ran into the issue where it seems that local storage can't be
overprovisioned.

storage.overprovisioning.factor is set to 1.5 in both the global and
zone settings, but this is ignored.

In this case it's a 1TB SSD which currently has 68GB allocated on
disk, but 750GB of disk space has been allocated according to CloudStack.

Looking StorageManagerImpl there is this if-statement:

if (storagePool.getPoolType() == StoragePoolType.NetworkFilesystem ||
storagePool.getPoolType() == StoragePoolType.VMFS) {
   DO OVERPROVISION
} else {
   DO NOT OVERPROVISION
}

So why is local storage excluded here? Since QCOW2 is sparsely
allocated and templates deployments are just clones of a parent QCOW2
it's fair to say that with local storage you can also overprovision?

Any objections to adding this?

Wido
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQIcBAEBAgAGBQJU/vXdAAoJEAGbWC3bPspCSIEP/RW5Yzx5tap2YQxDMKjF5b/r
TewVIheKexlvOsr6BzVY4M++55YutZjXBGzRouy//d2T5ozqJg7EyrXZtzU3wbXM
T/nyMjyuvziGjB/poTHVDE+yEDQ79UbU2BXKdp5CYBZJHDSw1ilZajUqo26KhqKb
N2xatwcCAo2p0dMMze3QwYxkd3gtMwkREok3lvN153SIhi2pKQSoRMU0eA68q0wZ
30jprOBnjqzGcRkbkvM6/KN84OB3720BNVuc3JbJPdItG7UchbzIW8bJvHfhMViD
MXZObpD0aWSLbZZYSfgNH+bL1P2XXldOWeQ+HI5q+A7Sf8q0i9PlEpAR+gqROHGA
UtGk8MCi0OP7zjXciDOHrhGYfxQ+o0iIF0knt6fCXpma+KzRDxQo6Eo7D2rHGv2e
8xarZ4oamhAmytUxiL2i74cNS6d248P4fER0CUQ4tML/4jdXMRvALud2XYwK+e+C
HRnZ0/X2YPzQbhJeZne8cYkf3vz3emLbck0dcgvAQtDIlPekeNAhs6Ik5hxCP29G
Nsp8LILw69VPcSkDAHmLV6Gzd1jaIxtObM67CQLA0F3q/ehFwD29mncxe7kh1AhP
yuW8s+oUjyQxhaV/glMTj2odbV4MfLA8Hzw3/wGtj6vHYcMAvCrRGGUo7gO5RPgv
l1tt1vl1aPA0OR7BZEXF
=I9Y1
-END PGP SIGNATURE-

Re: Overprovisioning of local storage

[ilya musayev]

sorry, wrong solution.. but still relevant issue you may encounter

On 3/10/15 11:59 PM, ilya musayev wrote:

fixed in 4.4, but i'd go with 4.5

https://issues.apache.org/jira/browse/CLOUDSTACK-7752



On 3/10/15 6:47 AM, Wido den Hollander wrote:

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Hi,

I've ran into the issue where it seems that local storage can't be
overprovisioned.

storage.overprovisioning.factor is set to 1.5 in both the global and
zone settings, but this is ignored.

In this case it's a 1TB SSD which currently has 68GB allocated on
disk, but 750GB of disk space has been allocated according to 
CloudStack.


Looking StorageManagerImpl there is this if-statement:

if (storagePool.getPoolType() == StoragePoolType.NetworkFilesystem ||
storagePool.getPoolType() == StoragePoolType.VMFS) {
   DO OVERPROVISION
} else {
   DO NOT OVERPROVISION
}

So why is local storage excluded here? Since QCOW2 is sparsely
allocated and templates deployments are just clones of a parent QCOW2
it's fair to say that with local storage you can also overprovision?

Any objections to adding this?

Wido
-BEGIN PGP SIGNATURE-
Version: GnuPG v1

iQIcBAEBAgAGBQJU/vXdAAoJEAGbWC3bPspCSIEP/RW5Yzx5tap2YQxDMKjF5b/r
TewVIheKexlvOsr6BzVY4M++55YutZjXBGzRouy//d2T5ozqJg7EyrXZtzU3wbXM
T/nyMjyuvziGjB/poTHVDE+yEDQ79UbU2BXKdp5CYBZJHDSw1ilZajUqo26KhqKb
N2xatwcCAo2p0dMMze3QwYxkd3gtMwkREok3lvN153SIhi2pKQSoRMU0eA68q0wZ
30jprOBnjqzGcRkbkvM6/KN84OB3720BNVuc3JbJPdItG7UchbzIW8bJvHfhMViD
MXZObpD0aWSLbZZYSfgNH+bL1P2XXldOWeQ+HI5q+A7Sf8q0i9PlEpAR+gqROHGA
UtGk8MCi0OP7zjXciDOHrhGYfxQ+o0iIF0knt6fCXpma+KzRDxQo6Eo7D2rHGv2e
8xarZ4oamhAmytUxiL2i74cNS6d248P4fER0CUQ4tML/4jdXMRvALud2XYwK+e+C
HRnZ0/X2YPzQbhJeZne8cYkf3vz3emLbck0dcgvAQtDIlPekeNAhs6Ik5hxCP29G
Nsp8LILw69VPcSkDAHmLV6Gzd1jaIxtObM67CQLA0F3q/ehFwD29mncxe7kh1AhP
yuW8s+oUjyQxhaV/glMTj2odbV4MfLA8Hzw3/wGtj6vHYcMAvCrRGGUo7gO5RPgv
l1tt1vl1aPA0OR7BZEXF
=I9Y1
-END PGP SIGNATURE-

Re: New SSL vulnerability #FREAK

[ilya musayev]

This might be relevant, ways to disable weak ciphers for port 8250, i'm 
under impression same can be done for all java related processes. This 
came from Citrix security team sometime in August, when we reached and 
mentioned that 8250 exposes weak ciphers as per internal audit.




Who Should apply this workaround?

This workaround is to make SSL Ciphers stronger than 128 bits. This 
workaround is for customers running Citrix CloudPlatform version 3.x and 
above. Apply this workaround on all the CloudPlatform 3.x and above 
Management Servers.



Package Name

NA

Issues Resolved In This Hotfix

This workaround resolves the following issue:
CS-17504: Weak SSL ciphers supported by the management server


Installing the Hotfix

Use the following steps to for the workaround. As with any software 
update, please back up your data before applying this workaround.


1.  Stop the Management Server:# service cloudstack-management stop.

2.  First Backup and then Modify java.security file at following 
location (You may have different location based on your Java version), 
if you are unsure, do a “find / -name java.security” to get the exact 
location): #


cp 
/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.51.x86_64/jre/lib/security/java.security 
java.security.backup


/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.51.x86_64/jre/lib/security/java.security 



3.  Make following changes to java.security:#

#Start of Edit

#Following are the default settings, needs to be commented.
#jdk.certpath.disabledAlgorithms=MD2, RSA keySize  1024

#Following needs to be added
jdk.tls.disabledAlgorithms=DH keySize  128, RSA keySize  128, DES 
keySize  128, SHA1  keySize  128, MD5 keySize  128, RC4 keySize  128


  # End of edit

4.  Start the Management Server: # service cloudstack-management start.


Log in to the CloudPlatform UI as administrator, and check the status of 
the hosts. All hosts should come to Up state (except those that you know 
to be offline). You may need to wait 20 or 30 minutes, depending on the 
number of hosts.


Troubleshooting: If login fails, clear your browser cache and reload the 
page.





On 3/9/15 5:05 AM, Rohit Yadav wrote:

Hi Lucian,

I made a similar fix on 4.5/master today for apache2. I'm not sure how
to deal with this in Java. There are ways to get enabled cipher suites,
but should we be doing this or configuring the jre security settings?

On Monday 09 March 2015 04:21 PM, Nux! wrote:

Ok, so for Apache HTTPD something like this would do the job:

SSLProtocol all -SSLv2 -SSLv3 -TLSv1
SSLCipherSuite 
ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-DSS-AES128-GCM-SHA256:kEDH+AESGCM:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA384:ECDHE-RSA-AES256-SHA:ECDHE-ECDSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-DSS-AES128-SHA256:DHE-RSA-AES256-SHA256:DHE-DSS-AES256-SHA:DHE-RSA-AES256-SHA:!aNULL:!eNULL:!EXPORT:!DES:!RC4:!3DES:!MD5:!PSK

SSLHonorCipherOrder on

I do not know how to do this for the CPVM java thingy that runs 
there; I think it's based on Jetty.


--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -

From: Erik Weber terbol...@gmail.com
To: dev dev@cloudstack.apache.org
Sent: Monday, 9 March, 2015 09:34:08
Subject: Re: New SSL vulnerability #FREAK



On Mon, Mar 9, 2015 at 9:59 AM, Nux! n...@li.nux.ro wrote:


BTW, the command I used is:

nmap --script ssl-enum-ciphers $HOST

I'm not entirely sure which cipher is good or not.



Anyone with EXPORT in it is bad (in the FREAK case).

This is a scan of my 4.3.2 systemvm with nmap:

| ssl-enum-ciphers:
|   SSLv3:
| ciphers:
|   TLS_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA - weak
|   TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA - strong
|   TLS_DHE_RSA_WITH_AES_128_CBC_SHA - strong
|   TLS_DHE_RSA_WITH_AES_256_CBC_SHA - strong
|   TLS_DHE_RSA_WITH_DES_CBC_SHA - weak
|   TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA - strong
|   TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA - strong
|   TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA - strong
|   TLS_ECDHE_RSA_WITH_RC4_128_SHA - strong
|   TLS_RSA_EXPORT_WITH_DES40_CBC_SHA - weak
|   TLS_RSA_EXPORT_WITH_RC4_40_MD5 - weak
|   TLS_RSA_WITH_3DES_EDE_CBC_SHA - strong
|   TLS_RSA_WITH_AES_128_CBC_SHA - strong
|   TLS_RSA_WITH_AES_256_CBC_SHA - strong
|   TLS_RSA_WITH_DES_CBC_SHA - weak
|   TLS_RSA_WITH_RC4_128_MD5 - strong
|   TLS_RSA_WITH_RC4_128_SHA - strong
| compressors:
|   NULL
|   TLSv1.0:
| ciphers:
|   TLS_DHE_RSA_EXPORT_WITH_DES40_CBC_SHA - weak
|   TLS_DHE_RSA_WITH_3DES_EDE_CBC_SHA - strong
|   TLS_DHE_RSA_WITH_AES_128_CBC_SHA - strong
|   TLS_DHE_RSA_WITH_AES_256_CBC_SHA - strong
|   TLS_DHE_RSA_WITH_DES_CBC_SHA - weak
|   TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA 

Re: [HELP] VR sometimes fail to start

[ilya musayev]

What kernel are you running? Please include Libvirt and Qemu version.

On 3/10/15 8:21 AM, Andrija Panic wrote:

Hi,

we are having issues from time to time with starting VR - while creating
VPC.

Here is the few lines from MGMT log:

2015-03-10 16:11:50,018 WARN  [c.c.n.v.VpcManagerImpl]
(Job-Executor-27:ctx-84f4fba4 ctx-970acb35) Failed to start vpc [VPC
[621-VPC-delete] due to
com.cloud.exception.AgentUnavailableException: Resource [Host:788] is
unreachable: Host 788: Unable to start instance due to Unable to start
VM[DomainRouter|r-2189-VM] due to error in finalizeStart, not retrying
 at
com.cloud.vm.VirtualMachineManagerImpl.orchestrateStart(VirtualMachineManagerImpl.java:1072)


The interesting part is on the Agent itself:


2015-03-10 16:03:28,520{GMT} WARN  [kvm.resource.LibvirtComputingResource]
(agentRequest-Handler-2:) Timed out:
/usr/share/cloudstack-common/scripts/vm/hypervisor/kvm/patchviasocket.pl -n
r-2189-VM -p %vpccidr=
10.0.0.0/24%domain=cs640cloud.internal%dns1=8.8.8.8%dns2=%template=domP%name=r-2189-VM%eth0ip=169.254.2.134%eth0mask=255.255.0.0%type=vpcrouter%disable_rp_filter=true
.  Output is:


Interesting stuff is - it's seems somewhat random, 3-4  out of 5 works.

Any suggestion are really appreciated, since this is live system :(
ACS 4.3.0, KVM, CEPH

Thanks,

Re: Disable ciphers in Console VM's Java server #FREAK

[ilya musayev]

This might be relevant, ways to disable weak ciphers for port 8250, i'm 
under impression same can be done for all java related processes. This 
came from Citrix security team sometime in August, when we reached and 
mentioned that 8250 exposes weak ciphers as per internal audit.




Who Should apply this workaround?

This workaround is to make SSL Ciphers stronger than 128 bits. This 
workaround is for customers running Citrix CloudPlatform version 3.x and 
above. Apply this workaround on all the CloudPlatform 3.x and above 
Management Servers.



Package Name

NA

Issues Resolved In This Hotfix

This workaround resolves the following issue:
CS-17504: Weak SSL ciphers supported by the management server


Installing the Hotfix

Use the following steps to for the workaround. As with any software 
update, please back up your data before applying this workaround.


1.  Stop the Management Server:# service cloudstack-management stop.

2.  First Backup and then Modify java.security file at following 
location (You may have different location based on your Java version), 
if you are unsure, do a “find / -name java.security” to get the exact 
location): #


cp 
/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.51.x86_64/jre/lib/security/java.security 
java.security.backup


/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.51.x86_64/jre/lib/security/java.security 



3.  Make following changes to java.security:#

#Start of Edit

#Following are the default settings, needs to be commented.
#jdk.certpath.disabledAlgorithms=MD2, RSA keySize  1024

#Following needs to be added
jdk.tls.disabledAlgorithms=DH keySize  128, RSA keySize  128, DES 
keySize  128, SHA1  keySize  128, MD5 keySize  128, RC4 keySize  128


  # End of edit

4.  Start the Management Server: # service cloudstack-management start.


Log in to the CloudPlatform UI as administrator, and check the status of 
the hosts. All hosts should come to Up state (except those that you know 
to be offline). You may need to wait 20 or 30 minutes, depending on the 
number of hosts.


Troubleshooting: If login fails, clear your browser cache and reload the 
page.






On 3/9/15 11:23 AM, Nux! wrote:

Hello,

Can anyone advise how to disable SSLv2, SSLv3, TLSv1 and various ciphers in the 
Java process which serves CPVM:443 ?

The CPVM is currently affected by the FREAK issue, we'd like to fix that.

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

Re: Disable ciphers in Console VM's Java server #FREAK

[ilya musayev]

OpenJDK 1.7 is a requirement for this to work.

On 3/9/15 2:31 PM, ilya musayev wrote:
This might be relevant, ways to disable weak ciphers for port 8250, 
i'm under impression same can be done for all java related processes. 
This came from Citrix security team sometime in August, when we 
reached and mentioned that 8250 exposes weak ciphers as per internal 
audit.




Who Should apply this workaround?

This workaround is to make SSL Ciphers stronger than 128 bits. This 
workaround is for customers running Citrix CloudPlatform version 3.x 
and above. Apply this workaround on all the CloudPlatform 3.x and 
above Management Servers.



Package Name

NA

Issues Resolved In This Hotfix

This workaround resolves the following issue:
CS-17504: Weak SSL ciphers supported by the management server


Installing the Hotfix

Use the following steps to for the workaround. As with any software 
update, please back up your data before applying this workaround.


1.  Stop the Management Server:# service cloudstack-management stop.

2.  First Backup and then Modify java.security file at following 
location (You may have different location based on your Java version), 
if you are unsure, do a “find / -name java.security” to get the exact 
location): #


cp 
/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.51.x86_64/jre/lib/security/java.security 
java.security.backup


/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.51.x86_64/jre/lib/security/java.security 



3.  Make following changes to java.security:#

#Start of Edit

#Following are the default settings, needs to be commented.
#jdk.certpath.disabledAlgorithms=MD2, RSA keySize  1024

#Following needs to be added
jdk.tls.disabledAlgorithms=DH keySize  128, RSA keySize  128, DES 
keySize  128, SHA1  keySize  128, MD5 keySize  128, RC4 keySize  128


  # End of edit

4.  Start the Management Server: # service cloudstack-management start.


Log in to the CloudPlatform UI as administrator, and check the status 
of the hosts. All hosts should come to Up state (except those that you 
know to be offline). You may need to wait 20 or 30 minutes, depending 
on the number of hosts.


Troubleshooting: If login fails, clear your browser cache and reload 
the page.






On 3/9/15 11:23 AM, Nux! wrote:

Hello,

Can anyone advise how to disable SSLv2, SSLv3, TLSv1 and various 
ciphers in the Java process which serves CPVM:443 ?


The CPVM is currently affected by the FREAK issue, we'd like to fix 
that.


--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

Re: Russian Language Support for ACS UI

[ilya musayev]

Just checked 4.5, yep - its there..



On 3/6/15 2:46 PM, Nux! wrote:

Ilya,

Not sure about older versions, but in 4.5 this is what I see in the drop down 
menu.
http://img.nux.ro/f9J-Menu_053.png

I also spotted a bug, when switching to Russian, the string English also was 
translated in the drop down - and I could not find my way back to English. :-)
Luckily Italian was not translated and clicked that.

We should not translate the language names themselves, as they appear in their 
own native format.

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -

From: ilya musayev ilya.mailing.li...@gmail.com
To: dev@cloudstack.apache.org
Sent: Friday, 6 March, 2015 22:30:06
Subject: Re: Russian Language Support for ACS UI
Then the second part question,

How do i make it appear as drop down for language selection when i login?

On 3/5/15 5:37 PM, David Nalley wrote:

Are you sure it isn't already there?

https://www.transifex.com/projects/p/CloudStack_UI/language/ru_RU/

--David

On Thu, Mar 5, 2015 at 8:19 PM, ilya musayev
ilya.mailing.li...@gmail.com wrote:

Several folks on this list speak and write Russian fluently.

I'm curious what it would take to add a support for russian language in ACS
UI.

Any pointers on what needs to be changed/added would help.

Thanks
ilya

Re: Russian Language Support for ACS UI

[ilya musayev]

Then the second part question,

How do i make it appear as drop down for language selection when i login?

On 3/5/15 5:37 PM, David Nalley wrote:

Are you sure it isn't already there?

https://www.transifex.com/projects/p/CloudStack_UI/language/ru_RU/

--David

On Thu, Mar 5, 2015 at 8:19 PM, ilya musayev
ilya.mailing.li...@gmail.com wrote:

Several folks on this list speak and write Russian fluently.

I'm curious what it would take to add a support for russian language in ACS
UI.

Any pointers on what needs to be changed/added would help.

Thanks
ilya

Re: or website

[ilya musayev]

Suggestion: we should try reaching out to academia and see if anyone 
wants to do and internship project to help redesign the homepage.


On 3/5/15 6:33 AM, Sebastien Goasguen wrote:

On Mar 5, 2015, at 9:31 AM, Sebastien Goasguen run...@gmail.com wrote:


Hi,

The website has been bugging me greatly, it's not up to par with today's 
website and does not portray our project in a good way.

I decided to try something quick. I created a gh-pages branch in our repo, 
orphaned from the rest of the code based and pushed that.
It shows up on github due to the mirroring:

https://github.com/apache/cloudstack/tree/gh-pages

And it builds:

https://github.com/apache/cloudstack/tree/gh-pages

http://apache.github.io/cloudstack/


I believe that's nice, it allows us to keep the website under ASF root repo but 
the mirroring allows us to use github pages.
That means we can take pull requests to build our website instead of using svn 
or the apache CMS.

Any amazing web designer on these lists who wants to take a crack at a modern 
design for cloudstack ?

thanks,

-sebastien

Re: Cannot delete range as IP is being used for providing dhcp service in this subnet

[ilya musayev]

I assume you can make sense of these queries, install something like 
MySQL Workbench - or tool you like the most for sql work so you can make 
changes in place without knowing much of SQL.


Here are  snippets i wrote that may help

First find the network ID you need to clean up
SELECT * from cloud.networks

check what i used to check what network uses which IPs.
SELECT name,ip4_address FROM cloud.nics,cloud.vm_instance where 
broadcast_uri like %106 and nics.instance_id=vm_instance.id and 
nics.state not like Deallocating;


Check what IPs are used by specific network, you can alter them to Free 
once you find the disparity
SELECT * FROM cloud.user_ip_address where source_network_id=221 and 
state not like Free;


Same on NICs table, set them to Deallocating if they are set to Reserved 
or Allocated - and append the removed date
SELECT * FROM cloud.nics where network_id=212 and state not like 
Deallocating and removed is null;


On 3/3/15 5:17 PM, Nux! wrote:

I have disabled the zone hours ago and killed the VR. I still cannot remove 
that range, same error.
I'll let it do its thing overnight and try again tomorrow morning.

At this point the waiting times are kind of bad and I'd rather dive in the DB 
if there's a reasonable way of doing it that way.

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -

From: Nux! n...@li.nux.ro
To: dev@cloudstack.apache.org
Sent: Tuesday, 3 March, 2015 23:18:24
Subject: Re: Cannot delete range as IP is being used for providing dhcp service 
in this subnet
Hi Marcus,

Yes, shared guest networks.

I'll try your way.

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -

From: Marcus shadow...@gmail.com
To: dev@cloudstack.apache.org
Sent: Tuesday, 3 March, 2015 22:20:05
Subject: Re: Cannot delete range as IP is being used for providing dhcp service
in this subnet
We're not talking guest networks, right?  In the past I've been able
to remove these by disabling the zone, then deleting the router and/or
applicable system vms. Disabling the zone ensures the system vms don't
come back up. The vms do have to expunge though before you'll have any
luck.

On Tue, Mar 3, 2015 at 2:06 PM, Somesh Naidu somesh.na...@citrix.com wrote:

Lucian,

Not sure if you can get past that without destroying that network (or
manipulating the DB).

I am not sure if network restart with cleanup true releases the VR IP so you
probably could try that. If it does, then we'd have to figure how to avoid the
new IP getting chosen from the same IP range.

Regards,
Somesh


-Original Message-
From: Nux! [mailto:n...@li.nux.ro]
Sent: Tuesday, March 03, 2015 2:31 PM
To: dev
Subject: [4.5] Cannot delete range as IP is being used for providing dhcp
service in this subnet

Hi,

I'm having this problem, I've added a secondary subnet/range in my Adv+SG zone
and now I want to delete it.
All the VMs using it have been deleted (and I've been waiting for a while for
the explunge/cleanup jobs to run), yet I am getting this error:

Cannot delete vlan range 2 as 192.168.203.102 is being used for providing dhcp
service in this subnet. Delete all VMs in this subnet and try again

All the VMs are deleted, only the virtual router is using IPs from that range.

root@r-4-VM:~# ip a|grep 192.168
 inet 192.168.203.102/24 brd 192.168.203.255 scope global eth0:55
 inet 192.168.203.103/24 brd 192.168.203.255 scope global secondary eth0:56

How can I make the VR let go of those IPs?

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

Re: [DISCUSS] Allow configuring system VMs to use local storage at zone level

[ilya musayev]

Hi Koushik

Huge +1 from me

I've asked Citrix for this feature as well, wayback, its a pain when you 
have mixed zones - but the feature is global.


Thanks
ilya

On 2/27/15 3:25 AM, Koushik Das wrote:

Currently it is not possible to configure system VMs to use local storage on a 
per zone basis. Since it is a global config (system.vm.use.local.storage), 
either it is enabled for all zones or none. The idea is to make it a zone level 
config.
The current zone level setting of 'enabling local storage for user VMs' won't 
be impacted. Now there will be an additional setting for system VMs as well.

I will create a tracking bug and FS for this.

Comments?

Thanks,
Koushik

[LIVE STREAM] Docker, Kubernetes, CoreOS and Big Data in Apache CloudStack by Sebastien Goasguen hosted by Apple

[ilya musayev]

Hi all,

We are going to live stream Sebastien’s presentation on Docker, 
Kubernetes, CoreOS and Big Data in Apache CloudStack on Tuesday, 
February 24th. If you would like to watch it, you can register here: 
clds.co/1CURN2T


Presentation: Docker, Kubernetes, CoreOS and Big Data in Apache 
CloudStack by Sebastien Goasguen

Date: February 24th, 2015
Time: 6:30pm – 8:30pm Pacific
You must register in order to watch the webinar: clds.co/1CURN2T
If you are in the Silicon Valley and attending in-person, register here: 
http://www.meetup.com/CloudStack-Silicon-Valley-User-Group/events/220330313/


Help promote!

Click to tweet: http://ctt.ec/Zcd6B - The #Docker #Kubernetes #CoreOS  
#BigData in #CloudStack talk by @sebgoa on Feb 24 will be streamed live! 
Sign up: clds.co/1CURN2T


Thanks,
CloudStack SV Group

Re: Libvirt RBD caching

[ilya musayev]

Logan

Side note: it would help great if you can post your notes/guide on 
setting up Ceph as primary with CloudStack.

There arent any docs out there.

Thanks
ilya
On 2/18/15 12:00 PM, Logan Barfield wrote:

Our current deployment is KVM with Ceph RBD primary storage.  We have
rbd_cache enabled, and use cache=none in Qemu by default.

I've been running some tests to try to figure out why our write speeds
with FreeBSD are significantly lower than Linux.  I was testing both
RBD and local SSD storage, with various cache configurations.  Out of
all of them the only one that performed close to our standard Linux
images was local SSD, Qemu cache=writeback, FreeBSD gpt journal
enabled.

I've been reading on various lists the reasons and risks for
cache=none vs cache=writeback:
- cache=none: Safer for live migration
- cache=writeback: Ceph RBD docs claim that this is required for data
integrity when using rbd_cache

 From what I can tell performance is generally the same with both,
except in the case of FreeBSD.

What is the current line of thinking on this?  Should be using 'none'
or 'writeback' with RBD by default?  Is 'writeback' considered safe
for live migration?

[ACS450] Template Delete Button Missing in UI

[ilya musayev]

In ACS 4.5.0 Template Delete Button Missing in UI? is this by design?

This would be a major bug (for some folks who operate public clouds with 
cloudstack UI, blocker).

Re: [ACS450] Template Delete Button Missing in UI

[ilya musayev]

Mike,

Good catch, i did see the view change, but i was under impression that 
by rolling over the quick view (+) i'd see the delete button. Instead 
quickview shows nothing.


I pressed on the template and i see it.

Thanks
ilya

On 2/18/15 10:03 AM, Mike Tutkowski wrote:

I don't think it's missing.

You need to go into the details view for the template and select the
applicable zone.

You should then be able to delete the template.

On Wednesday, February 18, 2015, ilya musayev ilya.mailing.li...@gmail.com
wrote:


In ACS 4.5.0 Template Delete Button Missing in UI? is this by design?

This would be a major bug (for some folks who operate public clouds with
cloudstack UI, blocker).

Re: Ans: About Instance Storage Live Migration on VMware

[ilya musayev]

Star,

There is an API call MigrateVirtualMachineWithVolume,

http://cloudstack.apache.org/docs/api/apidocs-4.4/root_admin/migrateVirtualMachineWithVolume.html

It only works over NFS, Citrix promised to add support for VMFS in the 
near future.


Regards
ilya

On 2/13/15 4:15 AM, Erik Weber wrote:

According to https://issues.apache.org/jira/browse/CLOUDSTACK-2701 it
should be supported.
Haven't tried it myself though.

Re: [VOTE] Apache CloudStack 4.5.0 RC3

[ilya musayev]

My finding thus far - not a blocker, could be classified as known 
issues, but should consider fixing:


My environment:
OEL 6.3
CloudStack 4.3.2
vSphere 5.5 and ESXi5.5

Issues noticed:

-
CloudStack MS complains:
2015-02-11 00:55:58,545 DEBUG [c.c.u.s.Script] (main:null) Executing: 
sudo keytool -genkey -keystore 
/etc/cloudstack/management/cloudmanagementserver.keystore -storepass 
vmops.com -keypass vmops.co
m -keyalg RSA -validity 3650 -dname cn=Cloudstack 
User,ou=example.com,o=corp.apple.com,c=Unknown

2015-02-11 00:55:58,572 DEBUG [c.c.u.s.Script] (main:null) Exit value is 1
2015-02-11 00:55:58,573 DEBUG [c.c.u.s.Script] (main:null) sudo: no tty 
present and no askpass program specified
2015-02-11 00:55:58,574 WARN  [c.c.s.ConfigurationServerImpl] 
(main:null) Would use fail-safe keystore to continue.
java.io.IOException: Fail to generate certificate!: sudo: no tty present 
and no askpass program specified
at 
com.cloud.server.ConfigurationServerImpl.generateDefaultKeystore(ConfigurationServerImpl.java:604)
at 
com.cloud.server.ConfigurationServerImpl.updateSSLKeystore(ConfigurationServerImpl.java:632)


Solution:
Append following to sudoers
 Defaults:cloud  !requiretty

---
CloudStack MS complains:

2015-02-11 00:55:58,545 DEBUG [c.c.u.s.Script] (main:null) Executing: 
sudo keytool -genkey -keystore 
/etc/cloudstack/management/cloudmanagementserver.keystore -storepass 
vmops.com -keypass vmops.com -keyalg RSA -validity 3650 -dname 
cn=Cloudstack User,ou=example.com,o=corp.apple.com,c=Unknown

2015-02-11 00:55:58,572 DEBUG [c.c.u.s.Script] (main:null) Exit value is 1
2015-02-11 00:55:58,573 DEBUG [c.c.u.s.Script] (main:null) sudo: no tty 
present and no askpass program specified
2015-02-11 00:55:58,574 WARN  [c.c.s.ConfigurationServerImpl] 
(main:null) Would use fail-safe keystore to continue.
java.io.IOException: Fail to generate certificate!: sudo: no tty present 
and no askpass program specified
at 
com.cloud.server.ConfigurationServerImpl.generateDefaultKeystore(ConfigurationServerImpl.java:604)
at 
com.cloud.server.ConfigurationServerImpl.updateSSLKeystore(ConfigurationServerImpl.java:632)


Solution:
append /usr/bin/keytool to cloud user, example:
 cloud ALL =NOPASSWD : /bin/chmod, /bin/cp, /bin/mkdir, /bin/mount, 
/bin/umount, /usr/bin/keytool



alternatively, create a separate file in /etc/sudoers.d/cloudstack and 
append both lines

Defaults:cloud  !requiretty
cloud ALL =NOPASSWD : /bin/chmod, /bin/cp, /bin/mkdir, /bin/mount, 
/bin/umount, /usr/bin/keytool



Going to open another bug about OnwireClassRegistry, what it this error? 
It occurs when MS starts.


2015-02-11 01:06:59,582 DEBUG [o.a.c.f.c.i.ConfigDepotImpl] (main:null) 
Retrieving keys from ExtensionRegistry
2015-02-11 01:07:02,372 DEBUG [o.a.c.f.s.OnwireClassRegistry] 
(main:null) Encountered IOException

java.io.IOException: Stream closed
at java.util.zip.ZipInputStream.ensureOpen(ZipInputStream.java:66)
at java.util.zip.ZipInputStream.getNextEntry(ZipInputStream.java:115)
at java.util.jar.JarInputStream.getNextEntry(JarInputStream.java:142)
at 
java.util.jar.JarInputStream.getNextJarEntry(JarInputStream.java:179)
at 
org.apache.cloudstack.framework.serializer.OnwireClassRegistry.getFromJARFile(OnwireClassRegistry.java:162)
at 
org.apache.cloudstack.framework.serializer.OnwireClassRegistry.getClasses(OnwireClassRegistry.java:119)
at 
org.apache.cloudstack.framework.serializer.OnwireClassRegistry.getClasses(OnwireClassRegistry.java:95)
at 
org.apache.cloudstack.framework.serializer.OnwireClassRegistry.scan(OnwireClassRegistry.java:76)



There is something strange with db.properties and key file - i think 
mostly related to ownership and perms. I will rerun upgrade to confirm - 
but it seemed to be the case on my several attempts.




Successful tests:

Create VM with multiple disks
Start VM
Destroy VM
Snapshot VM
No noticed issues or errors while MS core has been running for few hours 
except for the ones above.



If we can get clarity on how to address the issues or note them in docs, 
I dont see any blockers that cause functional issues. So i'm +0 - 
pending my final tests,  i would like to know what can be done for the 
errors above.


I can submit a patch, but due to $work policies, it may take much longer 
for me to do.


Regards
ilya

On 2/7/15 11:01 PM, David Nalley wrote:

Hi All,

I've created yet another 4.5.0 release candidate, with the following
artifacts up for a vote:

Git Branch and Commit SH:
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/4.5-RC20150208T0646
Commit: 187935a0e3c68e01ed33a8f7f0e6d69e6cdb0aca

Source release (checksums and signatures are available at the same
location):
https://dist.apache.org/repos/dist/dev/cloudstack/4.5.0-rc3/

PGP release keys (signed using 0x6fe50f1c):

Re: [VOTE] Apache CloudStack 4.5.0 RC3

[ilya musayev]

I cant positively vote on this just yet, I've noticed that I have issues 
with encryption/decryption of passwords..


-0, until i make further progress...


On 2/7/15 11:01 PM, David Nalley wrote:

Hi All,

I've created yet another 4.5.0 release candidate, with the following
artifacts up for a vote:

Git Branch and Commit SH:
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/4.5-RC20150208T0646
Commit: 187935a0e3c68e01ed33a8f7f0e6d69e6cdb0aca

Source release (checksums and signatures are available at the same
location):
https://dist.apache.org/repos/dist/dev/cloudstack/4.5.0-rc3/

PGP release keys (signed using 0x6fe50f1c):
https://dist.apache.org/repos/dist/release/cloudstack/KEYS

Vote will be open for at least 72 hours.

For sanity in tallying the vote, can PMC members please be sure to
indicate (binding) with their vote?

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)


--David

Re: Serious (ajax) GUI issues sometimes

[ilya musayev]

Somehow we need to add this to documentation of common issues - so end 
users knows he needs to use newer version of tomcat.


This is not cloudstack related,  but more of a tomcat bug that was 
addressed post 6.0.38 with folks that have high latency or server takes 
a bit of time to respond.


On 2/6/15 10:59 AM, Marcus wrote:

See the comments on

https://issues.apache.org/jira/browse/CLOUDSTACK-7907

Regarding tomcat bug

On Fri, Nov 14, 2014 at 12:14 AM, Andrija Panic andrija.pa...@gmail.com wrote:

Yes, but I would not call it annoying  - I would say - my clients gets
confused and pissed off :) Not so funny, when you are in the middle of the
demo with potenital client - and it doesn't work. Because we are using ACS
for public cloud - for our own use, it's annoying but not critical.

I just feel bad, that I can not help directly, since I'm not developer...

Best

On 14 November 2014 02:39, ilya musayev ilya.mailing.li...@gmail.com
wrote:


Hi Andrija

I notice both of these issues as well. Truly annoying is the first one,
i've seen this in earlier versions as well, i.e. 3.0.6..

Seems to be browser independent and happens on all browsers.

Regards
ilya

On 11/13/14, 8:11 AM, Andrija Panic wrote:


Hi Mike,

thanks for responding. Here is the ticket:
https://issues.apache.org/jira/browse/CLOUDSTACK-7907

Not sure if it makes any sense to report other bugs/improvements - no
sorting, random sorting(seem to me) on TimeZones (completely
unusable/frustrating), etc...

Thanks again,

On 13 November 2014 16:55, Mike Tutkowski mike.tutkow...@solidfire.com
wrote:

  Hi Andrija,

Thanks for the feedback...very useful.

I wonder if you'd be willing to open up JIRA tickets for these and mark
them as UI related? That way the people who work on the UI regularly will
have easy access to your comments.


https://issues.apache.org/jira/browse/CLOUDSTACK/?
selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel

Thanks again!
Mike

On Wed, Nov 12, 2014 at 12:41 AM, Andrija Panic andrija.pa...@gmail.com
wrote:

  Hi team,

is anybody of you aware of the GUI issues when connected to some i.e.
wireless with somewhat big latencies or it sometimes happen even on more
than good internet connection:

i.e:
1)
(A serious one, that we encounter pretty often):

Issue: I start new VM deloyment wizard, choose template etc at the


end


when I click the finish, when the job should be sent to MGMT server -
simply nothing happens - so ajax does't get executed at all, no litle
circle spinning etc - no logs in mgmt server, simply ajax doesn't get
executed...Same behaviour sometimes happen when I click on Configure
on
the VPC.

I confirmed behaviour in acs 4.3.0 and I'm still checking in 4.4.1.

OR

2)
(not a big issue, however very annoying):
I filter instances by some account/domain, then click on some instance
(view it's properties or whatever), than in the breadcrumb I click back


on


instances, and instead of being show the page with all the filtered
instances, I get back to the home page of ACS...

So it doesn't really happens always, but randomly, with different


browsers,


clearing all cache etc...
The issue here is that nothing get's logged to MGMT log at all...

Please let me know if anybody get similar issues, or if this is
something
on my side...

Thanks,


--

Andrija Panić



--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
http://solidfire.com/solution/overview/?video=play*™*






--

Andrija Panić
--
   http://admintweets.com
--

MeetUp SFBay: Docker, Kubernetes, coreOS and Big Data in Apache CloudStack (Hosted at Apple)

[ilya musayev]

Apple is hosting CloudStack SFBay MeetUp on February 24th.

Details here:
http://www.meetup.com/CloudStack-Silicon-Valley-User-Group/events/220330313/

Re: Cross Zone Templates

[ilya musayev]

As you know, when template is imported as all zones template, it has 
an ID that is same on all zones. If you delete a cross zone template on 
1 zone, it will be removed from all.


The moment you break away from this model and load templates as unique 
entities in each zone, the ID will change from zone to zone.


You can also try clonning the template in the UI to a different zone, 
i'm not certain if ID is preserved - but you can give it a try.


If you abstract cloudstack with your own frontend. You can fetch the 
template ID by using name and zone id.


Regards,
ilya
On 1/26/15 1:48 PM, Logan Barfield wrote:

I'm setting up a test zone for a multi-site deployment, and I've thus
far been unable to deploy a VM from our existing templates.

We're using S3 for secondary storage, and we have set up an NFS
staging server in the remote zone.  The management server is able to
mount the NFS store over the site-to-site VPN.

When we attempt to deploy a VM in the new zone, we get Template id
is not available.

It appears that this is because our templates are not set up as Cross
Zone Templates.

We originally created these templates from existing volumes (e.g.,
installed via ISO, configured, shut down VM, created template from
volume).  I did not see any option in the UI or API to specify it as a
cross zone template, nor do I see a way to update it after the fact
(other than manually editing the database).  This functionality only
seems to be available when Registering a template via a URL, which
isn't any help here.

Is there a reason for this weirdness?


Thank You,

Logan Barfield
Tranquil Hosting

Re: [jira] [Commented] (CLOUDSTACK-7907) UI heavily broken

[ilya musayev]

Ok.. so I believe I figured out whats happening. This problem has been 
causing issues for me for 2 years.


I was using safari pointed against CloudStack 4.3.1 env and noticed a 
trend where my create vm would not work time to time. I initiated 
firebug and went thru the deployment wizard once more, this time i 
noticed the error:


GET 
/client/api?command=deployVirtualMachineresponse=...layname=kvm03name=kvm03group=kvm_=1422045760139 
505 HTTP Version not supported127m


Then i did some googling and it brought me to this page, which 
summarizes an issue with Tomcat 6.0.37 and 7.0.37. Versions in between 
or lower maybe affected.


https://issues.apache.org/bugzilla/show_bug.cgi?id=54947

Next is to try this out on version of tomcat that have this patched. 
What tomcat versions newer than 6.0.37 can be used?



Regards
ilya

PS: Interesting observation, but Chrome would not show this error in 
debug mode. So for now, i'm sticking to Safari with Firebug for 
debugging :)


On 12/17/14 5:55 PM, ilya musayev (JIRA) wrote:

 [ 
https://issues.apache.org/jira/browse/CLOUDSTACK-7907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14250989#comment-14250989
 ]

ilya musayev commented on CLOUDSTACK-7907:
--

Alex and Andrija

Besides management-server.log file, there should also be an api log file, can 
you see your call there?

The management server log file should also log this api request, would it be 
possible to include the log file and 200 lines above and 200 below the api call?

Regards
ilya


UI heavily broken
-

 Key: CLOUDSTACK-7907
 URL: https://issues.apache.org/jira/browse/CLOUDSTACK-7907
 Project: CloudStack
  Issue Type: Bug
  Security Level: Public(Anyone can view this level - this is the default.)
  Components: API
Affects Versions: 4.3.0, 4.4.1
 Environment: not relevant
Reporter: Andrija Panic
Priority: Critical

(A serious one, that we encounter pretty often):
Issue: I start new VM deloyment wizard, choose template etc at the end when I click 
the finish, when the job should be sent to MGMT server - simply nothing happens - so ajax 
does't get executed at all, no litle circle spinning etc - no logs in mgmt server, simply 
ajax doesn't get executed...Same behaviour sometimes happen when I click on 
Configure on the VPC.
I confirmed behaviour in acs 4.3.0 and I'm still checking in 4.4.1, but I doubt 
anything has changed
OR
2)
(not a big issue, however very annoying):
I filter instances by some account/domain, then click on some instance (view it's 
properties or whatever), than in the breadcrumb I click back on instances, 
and instead of being show the page with all the filtered instances, I get back to the 
home page of ACS...
So it doesn't really happens always, but randomly, with different browsers, 
clearing all cache etc...
The issue here is that nothing get's logged to MGMT log at all...



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

CloudStack UI Issue with latest and any RHEL tomcat 6.0.24

[ilya musayev]

It appears the CloudStack UI issues i (and others have been experiencing) 
relate to outdated tomcat version we are using.

Please reference CLOUDSTACK-7907

Tomcat issue
https://issues.apache.org/bugzilla/show_bug.cgi?id=54947
Bug 54947 - Http11NioProtocol generates HTTP 505 responses on valid requests if 
request is split across multiple packets


Surprisingly enough, redhat 6 and its latest at the moment tomcat 6.0.24.62 
does not have the fix backported.
http://rpm.pbone.net/index.php3/stat/22/idpl/25442959/com/changelog.html

If i was to use non redhat tomcat, could i use apache tomcat 6.0.38?

Thanks
ilya

Re: CloudStack UI Issue with latest and any RHEL tomcat 6.0.24

[ilya musayev]

Well, apparently latest is tomcat6-6.0.24-80.el6.x86_64 and that is 
still missing that patch.. it sad. We dont have too many options here.


I'm thinking of taking the source code of tomcat6-6.0.24-80.el6.x86_64 
and applying the patch on top, but we would need to ask RedHat to 
backport this.



On 1/23/15 2:48 PM, ilya musayev wrote:
It appears the CloudStack UI issues i (and others have been 
experiencing) relate to outdated tomcat version we are using.


Please reference CLOUDSTACK-7907

Tomcat issue
https://issues.apache.org/bugzilla/show_bug.cgi?id=54947
Bug 54947 - Http11NioProtocol generates HTTP 505 responses on valid 
requests if request is split across multiple packets



Surprisingly enough, redhat 6 and its latest at the moment tomcat 
6.0.24.62 does not have the fix backported.

http://rpm.pbone.net/index.php3/stat/22/idpl/25442959/com/changelog.html

If i was to use non redhat tomcat, could i use apache tomcat 6.0.38?

Thanks
ilya

[SOLUTION] CloudStack 4.3.1 some VMs would get stopped continuously

[ilya musayev]

I've seen an issue where cloudstack would stop some VMs after they have 
been started. Posting the solution for whoever comes across this problem.


I've noticed following pattern:

VM was trying to be migrated to another host while another thread would 
stop the VM within 10 minutes of the start time.


The solution was to look for a message in the log that has 
(981,VmWorkJobQueue, 2371) is reaching concurrency limit 1 continuously.


Delete the reference id of 2371 from cloud.sync_queue_item
Delete the reference id of 981 from cloud.sync_queue

Re: [VOTE] Release Apache CloudStack CloudMonkey 5.3.1

[ilya musayev]

+1 (binding), did basic testing all seems well..
On 1/2/15 3:13 AM, Rohit Yadav wrote:

Hi,

Since a lot of people in the community are returning from holidays, please 
consider testing and voting for 5.3.1.

As per our bylaws we will need at least 3 +1 PMC votes and no veteos, negative 
vote to pass this, so the PMC is requested to participate as well (so far we’ve 
one from Daan). Thanks.


On 19-Dec-2014, at 9:08 pm, Rohit Yadav rohit.ya...@shapeblue.com wrote:

Hi Daan,

On Friday 19 December 2014 08:50 PM, Daan Hoogland wrote:

On Fri, Dec 19, 2014 at 4:03 PM, Rohit Yadav rohit.ya...@shapeblue.com wrote:

Hi Daan,

On Friday 19 December 2014 07:23 PM, Daan Hoogland wrote:

Rohit, I am testing against 4.4 simulator, I have tested against
master simulator. seems like I will +1 this before the end of the day.

have now confirmed with 4.4^HEAD and some more monkey testing then on master

+1 (binding)

...

One question though, is there a known issue with the color coding? it
seems to change halfway through words in the output and not be
consistent in the help/error messages. Not a blocker and I may need to
read more but it looks a bit strange.


If you running the CLI as a tool on bash (single query like cloudmonkey
list users), no colors are output (say in a bash script) this is to
avoid printing escape chars (for color) which mess up in bash script
(when people try to grep/sed/awk stuff with the output).

I am running interactive


In case colors are enabled then errors are printed in red color if they
start with Error string. The color scheme is haphazard for anyone (I
set the colors such that they looked best when using Solarized dark
theme on terminal http://ethanschoonover.com/solarized)

Please advise on the color behaviour and any improvements we should have?

it is kind of christmassish and I like it but it seems like to have no
intention to make sense at all. I am using iTerm2 with no special
color configuration.
it seems to be all random coloring to me on regular output, see a
annotated fragment of output below:


id = 6afa0b72-8787-11e4-a3e5-964184d18d4f -- id in red-undelined, '='
in grey and value in yellow
aclid = 6af9e6f6-8787-11e4-a3e5-964184d18d4f -- same scheme
action = Deny -- green, light-blue, grey
cidrlist = 0.0.0.0/0 -- cid- in red-underline/-rlist in grey, '='
grey, '0.' in lightblue/'0.0.0/0' in grey
fordisplay = True
number = 1
protocol = all
state = Active -- 'Active' in purple
tags: -- red-underline
traffictype = Ingress

on error output:
(local)   add host
Missing arguments: password hypervisor podid url username zoneid

'password', 'hypervisor' and 'url' are grey, 'username' is green and
the rest of the line-elements are red-undelined

That's sort of expected. When I wrote the color output (using Pygments)
I thought some parts of the output (things like id, uuids, numbers,
names etc) were more important than others so I gave them different colors.

There are regex rules to decide what color the key or value should
get. It's sort of haphazard indeed (those who are not used to it):
https://github.com/apache/cloudstack-cloudmonkey/blob/master/cloudmonkey/printer.py

I've known this issue since the colors and regex rules are hardcoded in
above since the early days, I think one solution is to allow people plug
their color schemes and (regex) rules, perhaps for future versions.




On Fri, Dec 19, 2014 at 12:01 PM, Erik Weber terbol...@gmail.com wrote:

On Mon, Dec 15, 2014 at 7:10 AM, Rohit Yadav bhais...@apache.org wrote:


Hi All,

I've created a 5.3.1 release of CloudMonkey, with the following
artifacts
up for a vote:

Git Branch and Commit SHA:


https://git-wip-us.apache.org/repos/asf?p=cloudstack-cloudmonkey.git;a=shortlog;h=refs/heads/master
Commit: 387f0338f9715f2337a9e10c92d1c76fa5d2a381

List of changes:


https://git-wip-us.apache.org/repos/asf?p=cloudstack-cloudmonkey.git;a=blob_plain;f=CHANGES.md

Source release (checksums and signatures are available at the same
location):
https://dist.apache.org/repos/dist/dev/cloudstack/cloudmonkey-5.3.1/

PGP release keys (signed using key ID 0EE3D884):
https://dist.apache.org/repos/dist/release/cloudstack/KEYS

Vote will be open for 72 hours.

For sanity in tallying the vote, can PMC members please be sure to
indicate
(binding) with their vote?

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove (and reason why)

Regards.



+1

Testing done:
   + Successfully upgraded from 5.3.0 to 5.3.1 on Mac OS X
   + Sync works
   + Loading profile on the command line (with -p) works
   + Using param completion when not at the end of line/string works
   + Various daily list/reset/deploy tasks


--
Erik





--
Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +91 8826230892 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab
PS. If you see any footer below, I did not add it :)
Find out more about ShapeBlue and our range of CloudStack 

Re: CloudStack and Windows 2008 R2 SCSI Disk Controller issue

[ilya musayev]

As per dev team from citrix, to get this to work:

change rootDiskController to osdefault - when importing the template and 
bump up windows version to 2012 (while running 2008R2 guest).


cloudmonkey example:

 register template format=ova hypervisor=vmware 
url=http://somehost.com/windows/2008R2_Sysprep-v6.ova  ispublic=true 
isfeatured=true passwordenabled=false 
details[0].rootDiskController=osdefault details[0].nicAdapter=E1000 
details[0].keyboard=us details[0].keyboard.typematicMindelay=200 
ostypeid=168 zoneid=-1  displaytext=Windows-2008-R2-v6 
name=Windows-2008-R2-v6



On 12/10/14, 3:03 PM, Erik Weber wrote:

Sateesh had a session about it during CCCEU, he might be able to update the
dev list about his work.

http://sched.co/X3guL1

Re: Status of ACS and OpenDayLight

[ilya musayev]

We also would like to see where its going and what integration there 
might be.


Chiradeep is on the board of ODL, perhaps he can share.

On 12/11/14, 9:13 AM, Chip Childers wrote:

On Thu, Dec 11, 2014 at 11:36:43AM +, Mathias Mullins wrote:

Hi everyone,

Does anyone know the status of CloudStack and OpenDayLight connectivity? I 
would like to learn how to make them work together to do more work with SDN.

Thank you,
Matt

IIRC, Hugo was working on that for a bit. Not sure where it got to
though.

Re: [ANNOUNCE] New committer: Erik Weber (terbolous)

[ilya musayev]

Congrats and welcome!
On 12/13/14, 12:27 PM, Rohit Yadav wrote:

Congrats Erik!


On 14-Dec-2014, at 1:27 am, Nux! n...@li.nux.ro wrote:

Gratulerer :-)

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -

From: Pierre-Luc Dion pdion...@apache.org
To: dev@cloudstack.apache.org
Sent: Saturday, 13 December, 2014 17:03:32
Subject: [ANNOUNCE] New committer: Erik Weber (terbolous)
The Project Management Committee (PMC) for Apache CloudStack
has asked Erik Weber to become a committer and we are pleased to
announce that they have accepted.

Erik is doing a great job on helping the community thru the mailing list.
He helped a lot on testing and troubleshooting new releases of CloudStack
as well as contributing to the documentation. Thanks Erik and Congrat!


Being a committer allows many contributors to contribute more
autonomously. For developers, it makes it easier to submit changes and
eliminates the need to have contributions reviewed via the patch
submission process. Whether contributions are development-related or
otherwise, it is a recognition of a contributor's participation in the
project and commitment to the project and the Apache Way.

Please join me in congratulating Erik

Pierre-Luc
on behalf of the CloudStack PMC

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +91 88 262 30892 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab



Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design  Buildhttp://shapeblue.com/iaas-cloud-design-and-build//
CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Software 
Engineeringhttp://shapeblue.com/cloudstack-software-engineering/
CloudStack Infrastructure 
Supporthttp://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/

This email and any attachments to it may be confidential and are intended solely 
for the use of the individual to whom it is addressed. Any views or opinions 
expressed are solely those of the author and do not necessarily represent those of 
Shape Blue Ltd or related companies. If you are not the intended recipient of this 
email, you must neither take any action based upon its contents, nor copy or show 
it to anyone. Please contact the sender if you believe you have received this email 
in error. Shape Blue Ltd is a company incorporated in England  Wales. 
ShapeBlue Services India LLP is a company incorporated in India and is operated 
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company 
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue 
SA Pty Ltd is a company registered by The Republic of South Africa and is traded 
under license from Shape Blue Ltd. ShapeBlue is a registered trademark.

CloudStack and Windows 2008 R2 SCSI Disk Controller issue

[ilya musayev]

CloudStack/CloudPlatform 4.3 defaults to LSI Logic Parallel and we need 
to use LSI Logic SAS when windows VMs are created.


Would anyone know if there is a way to override the controller through 
vm_details table and preset the controller to use there?


Right now it takes the options of scsi and ide, i've tried changing it 
to osdefault (which should have addressed the issue, but it appears we 
are still defaulting to LSI Logic Parallel)


osdefault for Windows 2008 R2 ESXi 5.1+ should be set to LSI Logic Parallel

Any help is appreciated,

Thanks
ilya

Re: CloudStack and Windows 2008 R2 SCSI Disk Controller issue

[ilya musayev]

Erik

Would you know where in the code? Looked around, could not find the place.

Thanks
ilya
On 12/10/14, 2:13 PM, Erik Weber wrote:

On Wed, Dec 10, 2014 at 10:44 PM, ilya musayev ilya.mailing.li...@gmail.com

wrote:
CloudStack/CloudPlatform 4.3 defaults to LSI Logic Parallel and we need to
use LSI Logic SAS when windows VMs are created.

Would anyone know if there is a way to override the controller through
vm_details table and preset the controller to use there?

Right now it takes the options of scsi and ide, i've tried changing it to
osdefault (which should have addressed the issue, but it appears we are
still defaulting to LSI Logic Parallel)

osdefault for Windows 2008 R2 ESXi 5.1+ should be set to LSI Logic Parallel



Been a known problem for a long time, only known workaround that I know of
is to change the controller in the code but that would change it for all
your vms.

See
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Granular+SCSI+Controller+support+in+CloudStack+over+VMware+deployments
and
https://issues.apache.org/jira/browse/CLOUDSTACK-4787

Re: CloudStack and Windows 2008 R2 SCSI Disk Controller issue

[ilya musayev]

edit:

osdefault for Windows 2008 R2 ESXi 5.1+ should be set to LSI Logic SAS or 
lsisas1068


On 12/10/14, 2:04 PM, Somesh Naidu wrote:

Try registering the template as Windows Server 2008 (without the R2) instead 
and see if that fixes it.

-Original Message-
From: ilya musayev [mailto:ilya.mailing.li...@gmail.com]
Sent: Wednesday, December 10, 2014 4:45 PM
To: dev@cloudstack.apache.org; Sateesh Chodapuneedi; Marcus
Subject: CloudStack and Windows 2008 R2 SCSI Disk Controller issue

CloudStack/CloudPlatform 4.3 defaults to LSI Logic Parallel and we need
to use LSI Logic SAS when windows VMs are created.

Would anyone know if there is a way to override the controller through
vm_details table and preset the controller to use there?

Right now it takes the options of scsi and ide, i've tried changing it
to osdefault (which should have addressed the issue, but it appears we
are still defaulting to LSI Logic Parallel)

osdefault for Windows 2008 R2 ESXi 5.1+ should be set to LSI Logic Parallel

Any help is appreciated,

Thanks
ilya

Re: CloudStack Quality Process

[ilya musayev]

I'd like to attend,  i will be commuting during the meeting hours 
getting to work - my connection will be choppy but i can listen in.

On 12/5/14, 2:20 PM, Pierre-Luc Dion wrote:

GTM if voice required although, wouldn't make more sense to use
#cloudstack-meeting as it keep record of discussions and is the regular
channel? either way, I will be in on 10 Dec 16 UTC



On Fri, Dec 5, 2014 at 5:07 PM, Animesh Chaturvedi 
animesh.chaturv...@citrix.com wrote:


Agreed


-Original Message-
From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On
Behalf Of Will Stevens
Sent: Thursday, December 04, 2014 2:41 PM
To: dev@cloudstack.apache.org
Cc: Steve Wilson
Subject: Re: CloudStack Quality Process

I am speaking as a committer who has limited insight into the 'correct'

way to do

this via Apache (so be gentle).  :)

I like the idea of a wiki page to help get everyone on the same page and

to track

the consensus as we move forward...

I also agree that it is hard to come to a consensus on the list because

it is really

hard to have a constructive conversation on here in a timely manner

where the

different voices can be heard.

I think it would be interesting to schedule sessions/meetings on the

list so any

interested party can join.  These sessions/meetings would happen in a

format

like IRC where the transcript of the session can be later posted to the

list as well

as a summary of the transcript so it can be reviewed by any member who

could

not make the meeting.  This way we keep all of the actual conversation

in the

list, but we also make it easier to actually have a 'conversation' at

the same time.

It is hard to beat real time when working through this sort of stuff.

Does this make sense to others?  Thoughts?

Will


*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|* tw
@CloudOps_

On Thu, Dec 4, 2014 at 5:17 PM, Animesh Chaturvedi 
animesh.chaturv...@citrix.com wrote:


Wearing my PMC hat and with past experience on these discussions we
have not made much progress on mailing list despite agreeing on the
goals and have locked horns. One possibility after reading Chip's
email and concerns I see is that, we create a wiki outlining the
problem space, possible
solution(s) and their specific pros and cons and have people

collaborate.

Once a general consensus is there and wiki is stable we can bring it
back to the mailing list for final approval. This is open as well as
requires participant a higher degree of commitment to collaborate and
will be more structured.

Thanks
Animesh


On Thu, Dec 4, 2014 at 7:24 PM, Chip Childers
chipchild...@apache.org

wrote:

Steve,

(Speaking with my PMC hat on, but not as someone that has the time
to help with this process)

I love the idea of moving forward with resolving some of the
quality process / tooling / etc... challenges that we face as a
project and community. I also love the idea that companies getting
commercial value from this project are talking (as companies)
about how to best support the project through either directing
their employees to work on this problem, allowing those interested
the time to do so, and / or offering (as Citrix did) required
hardware/software resources to make improvements for the common
good.  Importantly, I like that the companies involved are
mutually agreeing that this is for the common good.

That said, I have a concern about the outline below, specifically
in how the definition of approach and eventual execution are

handled.

The proposal of taking this off-list until there is a proposal to

ratify

is what I'd like to see changed. I would fully expect that a
fleshed out proposal hitting the list would be met with more
discussion than you would like (and perhaps even met with

frustration).

What has worked well for us in the past, where there is a need to
have those interested in doing work to be able to focus on that
work, has been to start with a call for interested parties (as you
did). Then, using a combination of threads on this list and live
meetings, make progress on defining the requirements and approach

incrementally.

Execution of any work should similarly be open and shared on this

list.

Throughout that process, allowing comments and openings for
participants are critical.

One of the things we learned about using live meetings to speed
up the consensus process in the past is to make sure that while
they are fantastic at allowing the participants to understand each
other, it's critical to remember that (1) there are no project
decisions made outside of the mailing lists and (2) that it's
important to have minutes or notes from those live meetings shared
with the community as

a

whole.

Now a very real concern that some of us have is getting bogged
down in arguments based on opinion, especially the drive by
opinions. This issue (plus challenges with people violently
agreeing with each other, yet 

Re: Any more bugs we should fix in 4.3 branch for 4.3.2?

[ilya musayev]

Rohit,

If possible and not too late, please consider adding

https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=f43ffb9a
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commitdiff;h=4ff3130b 
(minor)
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=1716067dddf20e434cdbfac69e19da7b71fea89d 
(minor db change, but really helpful)

https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commit;h=03daae571b44354d31913c7e0ecb9e1f64cdcd55

I will help on regression testing to best of my abilities with vmware. 
Marcus and I  plan to build internal CI to automate regression testing - 
but thats in the future and wont be ready for 4.3.2 release.


Regards,
ilya

On 12/3/14, 7:34 AM, Rohit Yadav wrote:

Hi,

I'm planning to drive testing efforts on 4.3 branch from next week that
target towards 4.3.2 release, are there any more bugs especially
blockers/criticals bugs that we should backport or fix that affects
4.3.0 or 4.3.1 release? For your reference, here is a list of changes
that were backported or fixed on 4.3 branch since 4.3.1 release for your
reference:

https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob_plain;f=CHANGES.md;hb=refs/heads/4.3 



After regression testing, I'm planning to merge those backported changes
to 4.4 branch and with blessings from Daan we can work towards 4.4.3
release as well.

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +91 8826230892 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab
Find out more about ShapeBlue and our range of CloudStack related 
services


IaaS Cloud Design  
Buildhttp://shapeblue.com/iaas-cloud-design-and-build//

CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Software 
Engineeringhttp://shapeblue.com/cloudstack-software-engineering/
CloudStack Infrastructure 
Supporthttp://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training 
Courseshttp://shapeblue.com/cloudstack-training/


This email and any attachments to it may be confidential and are 
intended solely for the use of the individual to whom it is addressed. 
Any views or opinions expressed are solely those of the author and do 
not necessarily represent those of Shape Blue Ltd or related 
companies. If you are not the intended recipient of this email, you 
must neither take any action based upon its contents, nor copy or show 
it to anyone. Please contact the sender if you believe you have 
received this email in error. Shape Blue Ltd is a company incorporated 
in England  Wales. ShapeBlue Services India LLP is a company 
incorporated in India and is operated under license from Shape Blue 
Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in 
Brasil and is operated under license from Shape Blue Ltd. ShapeBlue SA 
Pty Ltd is a company registered by The Republic of South Africa and is 
traded under license from Shape Blue Ltd. ShapeBlue is a registered 
trademark.

Re: Status of Autoscale on 4.4

[ilya musayev]

Silvano

Please create or update functional specs for change you would like to 
propose

Create JIRA issue tracker and reference in FS
Generally merge goes to master branch as well as other branch that has 
not been frozen for feature development yet.


Unless its a bug fix you cannot commit features to active 4.4 branch

Regards
ilya
On 12/4/14, 1:36 PM, Silvano Nogueira Buback wrote:

Hi guys,

 At Globo.com we are working on LB to our network orchestration tool (
http://globonetworkapi.readthedocs.org/en/latest/about.html) and we would
like to help to develop autoscale feature on ACS 4.4.2 version. In 4.4.2 I
could see many autoscale commands. What is the situation of autoscale in
this version? There is any specific branch can I use to merge latest
development of autoscale code to 4.4.2 release?

There are any other relevant documentation different the links bellow?
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Autoscale+framework
http://docs.cloudstack.apache.org/en/latest/networking/autoscale_without_netscaler.html
http://www.slideshare.net/tuna20073882/autoscale-without-netscalerccceu13

Thanks

Silvano Buback

Re: Serious (ajax) GUI issues sometimes

[ilya musayev]

Hi Andrija

I notice both of these issues as well. Truly annoying is the first one, 
i've seen this in earlier versions as well, i.e. 3.0.6..


Seems to be browser independent and happens on all browsers.

Regards
ilya
On 11/13/14, 8:11 AM, Andrija Panic wrote:

Hi Mike,

thanks for responding. Here is the ticket:
https://issues.apache.org/jira/browse/CLOUDSTACK-7907

Not sure if it makes any sense to report other bugs/improvements - no
sorting, random sorting(seem to me) on TimeZones (completely
unusable/frustrating), etc...

Thanks again,

On 13 November 2014 16:55, Mike Tutkowski mike.tutkow...@solidfire.com
wrote:


Hi Andrija,

Thanks for the feedback...very useful.

I wonder if you'd be willing to open up JIRA tickets for these and mark
them as UI related? That way the people who work on the UI regularly will
have easy access to your comments.


https://issues.apache.org/jira/browse/CLOUDSTACK/?selectedTab=com.atlassian.jira.jira-projects-plugin:summary-panel

Thanks again!
Mike

On Wed, Nov 12, 2014 at 12:41 AM, Andrija Panic andrija.pa...@gmail.com
wrote:


Hi team,

is anybody of you aware of the GUI issues when connected to some i.e.
wireless with somewhat big latencies or it sometimes happen even on more
than good internet connection:

i.e:
1)
(A serious one, that we encounter pretty often):

Issue: I start new VM deloyment wizard, choose template etc at the

end

when I click the finish, when the job should be sent to MGMT server -
simply nothing happens - so ajax does't get executed at all, no litle
circle spinning etc - no logs in mgmt server, simply ajax doesn't get
executed...Same behaviour sometimes happen when I click on Configure on
the VPC.

I confirmed behaviour in acs 4.3.0 and I'm still checking in 4.4.1.

OR

2)
(not a big issue, however very annoying):
I filter instances by some account/domain, then click on some instance
(view it's properties or whatever), than in the breadcrumb I click back

on

instances, and instead of being show the page with all the filtered
instances, I get back to the home page of ACS...

So it doesn't really happens always, but randomly, with different

browsers,

clearing all cache etc...
The issue here is that nothing get's logged to MGMT log at all...

Please let me know if anybody get similar issues, or if this is something
on my side...

Thanks,


--

Andrija Panić




--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
http://solidfire.com/solution/overview/?video=play*™*

Re: UI: CPU (in MHz) doesn't make sense

[ilya musayev]

Similar to VmWare, however unless you enable cpu reservation flag in 
general settings, it does not actually allocate and reserve it on vmware 
side - it only keeps it in cloudstack for tracking purposes.





On 11/10/14, 10:02 AM, Marcus wrote:

I can't speak for all hypervisors, but with KVM, it takes the hypervisor
CPU MHz * cores and treats that as 'capacity' of the hypervisor. This
translates directly to the number used for cgroups cpu shares. So a 2GHz
quad core would have 8000 MHz worth of vms allocated to it  (ignoring
overprovisioning).

On Mon, Nov 10, 2014 at 9:58 AM, Nux! n...@li.nux.ro wrote:


Personally, I'd propose the defaults should be values proportional or
equal to the cores number (that's how openstack does it).

Anyway, all this doesn't matter as long as no developer is listening. :-)

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -

From: Logan Barfield lbarfi...@tqhosting.com
To: dev@cloudstack.apache.org
Sent: Monday, 10 November, 2014 17:49:15
Subject: Re: UI: CPU (in MHz) doesn't make sense
That was definitely only an assumption.  If each host handles it

different

it may be preferable to hard code a Default level for each hypervisor
type, as well as a few different levels (e.g., 'Max', 'High', 'Default',
'Low', 'Min'  'Custom').  This would operators  end-users clear options
to work with, while retaining the flexibility of a custom option.


Thank You,

Logan Barfield
Tranquil Hosting

On Mon, Nov 10, 2014 at 11:46 AM, Nux! n...@li.nux.ro wrote:


I am not entirely sure if 1 vs 2 = 1 vs 1000. It might be that the one
with 1000 will get 1000 more prio to CPU compared to the one with 1.

This

needs to be clarified per each hypervisor.

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -

From: Logan Barfield lbarfi...@tqhosting.com
To: dev@cloudstack.apache.org
Sent: Monday, 10 November, 2014 16:23:41
Subject: Re: UI: CPU (in MHz) doesn't make sense
I agree completely.  We've set all of our service offerings to equal
weights, and hard coded the same weight into the custom offering form.
It's a bit too confusing otherwise.

The way I understand the weights for (Xen/KVM at least) is that

they're

just relative, so 1 vs 2 is the same as 1 vs 1000.  That being the

case

I'd

suggest a solution that has worked for us in the past: set the weight

equal

to the memory amount (in MB).

Thoughts?


Thank You,

Logan Barfield
Tranquil Hosting

On Mon, Nov 10, 2014 at 11:17 AM, Nux! n...@li.nux.ro wrote:


Hi,

Basically I'm annoyed with the CPU (in MHz) usage in service

offerings

as they are a lie basically.
Opened https://issues.apache.org/jira/browse/CLOUDSTACK-7874 and

suggest

to have calculated automatically based on CPU cores number or at

least

having it renamed to something like cpu weight.
MHz means nothing.

Thoughts?

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

Re: [ACS431] upgrade require systemvm upgrade?

[ilya musayev]

Personal experience, i've used latest 4.3.1 and older 4.3.0 sysvm 
template, with no issues. I'm running vSphere setup.


To be on the safe side, look through all commits between 4.3.0 and 4.3.1 
and see if any patch references systemvm.


On 9/20/14, 8:14 AM, Pierre-Luc Dion wrote:

does 4.3.1 require different system vm then 4.3.0 ?

if someone upgrade from 4.x.x 4.3.1 can he use latest 4.3.0 system vm ?


*Pierre-Luc DION*
Architecte de Solution Cloud | Cloud Solutions Architect
t 855.652.5683

*CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

Re: [VMWARE SETUP ERROR] Error seeing in brining up System VMs with Vmware setup with CS server

[ilya musayev]

Ritu,

Open a ticket in JIRA as major/blocker for 4.5, so it gets noticed.

You can also try running you code on 4.4 or 4.3.1, but i know it maybe 
too much work to backport.


Mike thank you for spending time trying to figure this out.

Regards
ilya
On 9/19/14, 4:21 PM, Ritu Sabharwal wrote:

Hi Mike,

Thanks for trying out this and the investigation.

I hope the developer for system vm would pick up this issue and help to resolve.

Thanks  Regards,
Ritu S.

-Original Message-
From: Mike Tutkowski [mailto:mike.tutkow...@solidfire.com]
Sent: Friday, September 19, 2014 9:01 AM
To: dev@cloudstack.apache.org
Subject: Re: [VMWARE SETUP ERROR] Error seeing in brining up System VMs with 
Vmware setup with CS server

http://jenkins.buildacloud.org/job/build-systemvm64-master/lastSuccessfulBuild/artifact/tools/appliance/dist/systemvm64template-master-vmware.ova

On Fri, Sep 19, 2014 at 9:58 AM, Erik Weber terbol...@gmail.com wrote:


Which systemvm template are you using?

Erik
19. sep. 2014 03:47 skrev Mike Tutkowski
mike.tutkow...@solidfire.com
følgende:


Actually, the virtual router says it requires an upgrade.

I think I can just turn that check off and re-run my create-VM test.

On Thu, Sep 18, 2014 at 7:43 PM, Mike Tutkowski 
mike.tutkow...@solidfire.com wrote:


OK, I checked in a fix for this:




https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=commitdiff;
h=18e5aa116e99906f24a292bf7546809e05c72ec2;hp=7537c33bcdda65672fdd8f81
509e6885aa536932

I did notice that both before and after my fix that the virtual
router

is

NOT coming up.

The system VM being cloned and the starting of the VM looks fine.
It

gets

to the command prompt and then a little while later is shut down
and restarted. I get an error in the GUI indicating that my VM
deployment failed.

I don't personally know a whole lot about how the virtual router
code works, so we should pass that on to an appropriate developer.

On Thu, Sep 18, 2014 at 6:35 PM, Mike Tutkowski 
mike.tutkow...@solidfire.com wrote:


Just destroyed and re-created my env from scratch without
8bb4022f3729154f85036c4a7f55e2de783e4909 and it works fine.

I think I know what the author was intending with the code that
is causing the problem, so I'll go ahead and put in what I think
is a

fix,

re-create my env, and see if it works.

On Thu, Sep 18, 2014 at 6:26 PM, ilya musayev
ilya.musa...@gmail.com
wrote:


  Thank you Mike, please keep us posted.

Regards
ilya

On 9/18/14, 4:14 PM, Mike Tutkowski wrote:

Little clarification on this:

  The dsName is derived from the UUID of the datastore.

  I mean that the dsName is derived from the UUID primary storage
in

the

DB.

On Thu, Sep 18, 2014 at 5:12 PM, Mike Tutkowski 
mike.tutkow...@solidfire.com wrote:


Just a little update on this:

  The problem doesn't seem to have anything to do with managed
storage code that was introduced back in March.

  The dsName is derived from the UUID of the datastore. This
leads

to a

null being returned from

diskInfoBuilder.getDiskInfoByBackingFileBaseName,

which subsequently leads to a null being returned from this
method

(and a

NullPointerException being thrown).

  I plan to take a look at the history of this file in gitk.

 private VirtualMachineDiskInfo
getMatchingExistingDisk(VirtualMachineDiskInfoBuilder

diskInfoBuilder,

DiskTO vol) {

 if (diskInfoBuilder != null) {

 VolumeObjectTO volume =
(VolumeObjectTO)vol.getData();

 String dsName =

volume.getDataStore().getUuid().replace(-,

);


  MapString, String details = vol.getDetails();

 boolean isManaged = details != null 
Boolean.parseBoolean(details.get(DiskTO.MANAGED));


  VirtualMachineDiskInfo diskInfo =


diskInfoBuilder.getDiskInfoByBackingFileBaseName(isManaged ?
new
DatastoreFile(volume.getPath()).getFileBaseName() :

volume.getPath(),

dsName);

On Thu, Sep 18, 2014 at 4:34 PM, Mike Tutkowski 
mike.tutkow...@solidfire.com wrote:


I think I found the magic. :)

sudo mount -t nfs 192.168.129.46:/export/secondary /mnt/sec

If I mount /export/secondary to /mnt/sec (I did it in a bit of
a roundabout way here), then the management server is happy.

I think this is only VMware thing for the management server.

Does anyone know if we document this?

Thanks!

On Thu, Sep 18, 2014 at 4:18 PM, Mike Tutkowski 
mike.tutkow...@solidfire.com wrote:


The instructions even say the following:

  If your secondary storage mount point is not named

/mnt/secondary,

substitute your own mount point name.

On Thu, Sep 18, 2014 at 4:15 PM, Mike Tutkowski 
mike.tutkow...@solidfire.com wrote:


Yeah, I've been through the instructions a couple times and
don't see anything that indicates why it's having trouble
with /mnt/sec

(it says

/mnt/sec instead of /mnt/secondary in the log messages).

On Thu, Sep 18, 2014 at 4:01 PM, Ritu Sabharwal 
rsabh...@brocade.com wrote:


I have an external NFS server and it is mounted to

/mnt/secondary

on my CS

Re: [VMWARE SETUP ERROR] Error seeing in brining up System VMs with Vmware setup with CS server

[ilya musayev]

Thank you Mike, please keep us posted.

Regards
ilya
On 9/18/14, 4:14 PM, Mike Tutkowski wrote:

Little clarification on this:

The dsName is derived from the UUID of the datastore.

I mean that the dsName is derived from the UUID primary storage in the DB.

On Thu, Sep 18, 2014 at 5:12 PM, Mike Tutkowski 
mike.tutkow...@solidfire.com mailto:mike.tutkow...@solidfire.com 
wrote:


Just a little update on this:

The problem doesn't seem to have anything to do with managed
storage code that was introduced back in March.

The dsName is derived from the UUID of the datastore. This leads
to a null being returned from
diskInfoBuilder.getDiskInfoByBackingFileBaseName, which
subsequently leads to a null being returned from this method (and
a NullPointerException being thrown).

I plan to take a look at the history of this file in gitk.

private VirtualMachineDiskInfo
getMatchingExistingDisk(VirtualMachineDiskInfoBuilder
diskInfoBuilder, DiskTO vol) {

if (diskInfoBuilder != null) {

VolumeObjectTO volume = (VolumeObjectTO)vol.getData();

String dsName =
volume.getDataStore().getUuid().replace(-, );


MapString, String details = vol.getDetails();

boolean isManaged = details != null 
Boolean.parseBoolean(details.get(DiskTO.MANAGED));


VirtualMachineDiskInfo diskInfo =

diskInfoBuilder.getDiskInfoByBackingFileBaseName(isManaged ? new
DatastoreFile(volume.getPath()).getFileBaseName() :
volume.getPath(), dsName);


On Thu, Sep 18, 2014 at 4:34 PM, Mike Tutkowski
mike.tutkow...@solidfire.com
mailto:mike.tutkow...@solidfire.com wrote:

I think I found the magic. :)

sudo mount -t nfs 192.168.129.46:/export/secondary /mnt/sec

If I mount /export/secondary to /mnt/sec (I did it in a bit of
a roundabout way here), then the management server is happy.

I think this is only VMware thing for the management server.

Does anyone know if we document this?

Thanks!


On Thu, Sep 18, 2014 at 4:18 PM, Mike Tutkowski
mike.tutkow...@solidfire.com
mailto:mike.tutkow...@solidfire.com wrote:

The instructions even say the following:

If your secondary storage mount point is not named
/mnt/secondary, substitute your own mount point name.

On Thu, Sep 18, 2014 at 4:15 PM, Mike Tutkowski
mike.tutkow...@solidfire.com
mailto:mike.tutkow...@solidfire.com wrote:

Yeah, I've been through the instructions a couple
times and don't see anything that indicates why it's
having trouble with /mnt/sec (it says /mnt/sec instead
of /mnt/secondary in the log messages).

On Thu, Sep 18, 2014 at 4:01 PM, Ritu Sabharwal
rsabh...@brocade.com mailto:rsabh...@brocade.com
wrote:

I have an external NFS server and it is mounted to
/mnt/secondary on my CS server so seeding command
I used is:


/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
-m /mnt/secondary -u

http://cloudstack.apt-get.eu/systemvm/4.4/systemvm64template-4.4.0-6-vmware.ova
-h vmware -F

I guess, if the secondary storage is local, then
/export/secondary should be fine.

Regards,
Ritu S.
-Original Message-
From: Mike Tutkowski
[mailto:mike.tutkow...@solidfire.com
mailto:mike.tutkow...@solidfire.com]
Sent: Thursday, September 18, 2014 2:34 PM
To: dev@cloudstack.apache.org
mailto:dev@cloudstack.apache.org
Cc: ilya musayev; Koushik Das; Sateesh
Chodapuneedi; Adip Shetty
Subject: Re: [VMWARE SETUP ERROR] Error seeing in
brining up System VMs with Vmware setup with CS server

The instructions to seed secondary storage for
VMware look like this:


/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sys-tmplt
\ -m /mnt/secondary \ -u

http://cloudstack.apt-get.eu/systemvm/4.4/systemvm64template-4.4.0-6-vmware.ova
\
-h vmware \
-s optional-management-server-secret-key \ -F

For -m, I have /export/secondary (not
/mnt/secondary). I assume that's correct because
that is, in fact, where my secondary storage
system template is located for XenServer

Re: Fwd: Re: Upgrade to 4.3.1

[ilya musayev]

Has this worked for you in 4.3.0, before the upgrade to 4.3.1?

I've briefly looked through change log from 4.3.0 to 4.3.1, i dont see 
any changes related to console - but this check was very brief.


Regards
ilya
On 9/17/14, 1:42 AM, Mārtiņš Jakubovičs wrote:

Hello,

I installed fresh CS 4.3.1 and it also don't use 
consoleproxy.url.domain value.


This is in console proxy, source:

htmltitlev-2-VM/titleframesetframe 
src=https://.realhostip.com/ajax?token=1cttHTopOwqQbK_Q9gYmoYMOnzK9fcHUlUv3q6rIUsyVdY3hU7Kmwa7A9RxTz7ZuG4MherlIa0tllGr5XWez73GVyP19iBqxP-c5eCttLrUAotMhMu3GLqPDdEzB2JmVFLZp2DLMBiiAYcdtnNvRl6by-V065VBXyZ729Awd-dnGWE1dNebljz78M5RmNWJXy7HsmrWalWaJyUoI2vMYjBc7FEDksGmCGS_akmN_rPmKH6CdPEyHhNAcvEfb0EiaFFMSA2NUZ9sDfhmmapVxGc1_mHWNWyTqq0WxMfIt6y5CvjmOSUutcagll5sm_SbdUcp7NHB_xZmNAx7yWKnyiMm9XMnYTPnqfrJe8YSSNc481oCgPgPBiQOwbwaoOqtR6PZLDwTPIT6ULkDpy-nL9eVX-lPUJByWzAmgDOGN3ZxV7gwZbvPJGn8cF65Sa5Cb8WN97mYqNqgFZlMChx5ADxNaBYxvWKHfATQoo6utXRg;/frame/frameset/html 



When I set in url correct domain my-ip-add.domain.com, than console 
works, example like this:


https://11-111-111-11.domain.com/ajax?token=1cttHTopOwqQbK_Q9gYmoYMOnzK9fcHUlUv3q6rIUsyVdY3hU7Kmwa7A9RxTz7ZuG4MherlIa0tllGr5XWez73GVyP19iBqxP-c5eCttLrUAotMhMu3GLqPDdEzB2JmVFLZp2DLMBiiAYcdtnNvRl6by-V065VBXyZ729Awd-dnGWE1dNebljz78M5RmNWJXy7HsmrWalWaJyUoI2vMYjBc7FEDksGmCGS_akmN_rPmKH6CdPEyHhNAcvEfb0EiaFFMSA2NUZ9sDfhmmapVxGc1_mHWNWyTqq0WxMfIt6y5CvjmOSUutcagll5sm_SbdUcp7NHB_xZmNAx7yWKnyiMm9XMnYTPnqfrJe8YSSNc481oCgPgPBiQOwbwaoOqtR6PZLDwTPIT6ULkDpy-nL9eVX-lPUJByWzAmgDOGN3ZxV7gwZbvPJGn8cF65Sa5Cb8WN97mYqNqgFZlMChx5ADxNaBYxvWKHfATQoo6utXRg 




 Forwarded Message 
Subject: Re: Upgrade to 4.3.1
Date: Wed, 17 Sep 2014 09:05:21 +0300
From: Mārtiņš Jakubovičs mart...@vertigs.lv
To: us...@cloudstack.apache.org



Hello Ilya,

Thanks for response, I upgraded from 4.3.0 to 4.3.1.

On 2014.09.17. 03:34, ilya musayev wrote:

Mārtiņš

If you dont get a response here, please post on dev. I'll try this out
shortly in my env, what version did you upgrade from?

Regards
ilya
On 9/16/14, 6:07 AM, Mārtiņš Jakubovičs wrote:

Hello,

In CentOS 6.5 today I discovered that is available update for CS 4.3.
I installed it but after management service restart 
consoleproxy.url.domain value stopped to work. When I launch console
proxy, it connect to .realhostip.com domain in a way to connect my
real domain. When I clean this value, console proxy connects to IP.
After entering any other value it connects to .realhostip.com
anyway. How I can solve this?

Thanks.

Re: [DISCUSS] CloudStack Future

[ilya musayev]

Hi Outback Dingo,

Would you please elaborate on your network topology a little more.

Thanks
ilya
On 9/16/14, 3:55 PM, Outback Dingo wrote:

Some of us would love to contribute, yet don't feel the requirement to
sign-up for sites to simply post their feelings.
That being said... heres mine in public.. remove the dependency
on NFS as primary/secondary allow
for more configurable storage options. Its one of the reasons why we
dropped cloudstack. That and certain networking
configuration requirements didn't fit our network topology.

On Wed, Sep 17, 2014 at 2:51 AM, Mike Tutkowski 
mike.tutkow...@solidfire.com wrote:


Hi everyone,

First: Thanks to Rohit and Daan for working on this.

Next: Definitely feel free to e-mail ideas privately; however, I'd like to
especially encourage people to make their ideas known publicly, if you feel
comfortable doing this. Doing it publicly might make it easier for us as a
community to brainstorm the ideas and play around with taking them in
different directions.

Thanks!
Mike

On Tue, Sep 16, 2014 at 3:08 AM, Rohit Yadav rohit.ya...@shapeblue.com
wrote:


Hi everyone,

Some of us are in Amsterdam and discussing various things we want to do
for the project. I’ve aggregated some of them on a Trello board here:
https://trello.com/b/nj8dDBWl/apache-cloudstack-future

Please share your ideas, publicly or private to me; I’ll add them on the
board. Our main focus right now is testing, release quality and aligning
efforts.

We’re now able to run simulator tests on TravisCI for 4.4 and master
branches:
https://travis-ci.org/apache/cloudstack/builds

Some of us are also experimenting with Github pull requests and we

already

see that it’s encouraging to get TravisCI verify them.

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab

Find out more about ShapeBlue and our range of CloudStack related

services

IaaS Cloud Design  Build
http://shapeblue.com/iaas-cloud-design-and-build//
CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Infrastructure Support
http://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training Courses
http://shapeblue.com/cloudstack-training/

This email and any attachments to it may be confidential and are intended
solely for the use of the individual to whom it is addressed. Any views

or

opinions expressed are solely those of the author and do not necessarily
represent those of Shape Blue Ltd or related companies. If you are not

the

intended recipient of this email, you must neither take any action based
upon its contents, nor copy or show it to anyone. Please contact the

sender

if you believe you have received this email in error. Shape Blue Ltd is a
company incorporated in England  Wales. ShapeBlue Services India LLP is

a

company incorporated in India and is operated under license from Shape

Blue

Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in

Brasil

and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd

is

a company registered by The Republic of South Africa and is traded under
license from Shape Blue Ltd. ShapeBlue is a registered trademark.




--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
http://solidfire.com/solution/overview/?video=play*™*

CloudStack Docker and Mesos Support

[ilya musayev]

Hi all,

Would you know where we stand with Mesos and Docker?

Thanks
ilya

Re: [VMWARE SETUP ERROR] Error seeing in brining up System VMs with Vmware setup with CS server

[ilya musayev]

++ Mike as per commit

Repository: cloudstack
Updated Branches:
  refs/heads/master d68e13fcb - 0879ab812


CLOUDSTACK-6170 (VMware root-disk support for managed storage)



On 9/16/14, 4:22 PM, Ritu Sabharwal wrote:


Hi Sateesh, Koushik,

I am setting up Vmware Cluster with CS (master) and seeing errors. 
ilya was helping me for this setup and figured out this error.


The System VMs(Secondary Stogare VM and Console Proxy VM) are not 
coming up properly. The system vms are reconfiguring and destroying 
again and again in cycle.


I have used Vmware Vsphere 5.1 SDK for building CS.

The error logs give this error:

2014-09-16 16:11:01,619 WARN  [c.c.h.v.r.VmwareResource] 
(DirectAgent-8:ctx-8909215e 10.24.41.149, job-54/job-68, cmd: 
StartCommand) StartCommand failed due to Exception: 
java.lang.NullPointerException


Message: null

java.lang.NullPointerException

at 
com.cloud.hypervisor.vmware.resource.VmwareResource.postDiskConfigBeforeStart(VmwareResource.java:2094)


at 
com.cloud.hypervisor.vmware.resource.VmwareResource.execute(VmwareResource.java:1685)


at 
com.cloud.hypervisor.vmware.resource.VmwareResource.executeRequest(VmwareResource.java:448)


at 
com.cloud.agent.manager.DirectAgentAttache$Task.runInContext(DirectAgentAttache.java:294)


at 
org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(ManagedContextRunnable.java:49)


at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext$1.call(DefaultManagedContext.java:56)


at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.callWithContext(DefaultManagedContext.java:103)


at 
org.apache.cloudstack.managed.context.impl.DefaultManagedContext.runWithContext(DefaultManagedContext.java:53)


at 
org.apache.cloudstack.managed.context.ManagedContextRunnable.run(ManagedContextRunnable.java:46)


at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)


at java.util.concurrent.FutureTask.run(FutureTask.java:262)

at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)


at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)


at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)


at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)


at java.lang.Thread.run(Thread.java:744)

Please help to resolve this issue.

Thanks  Regards,

Ritu S.

Re: [Help] Help for setting up Vmware Cluster in CloudStack...

[ilya musayev]

Ritu,

I will email you privately to help with setup.

Regards
ilya
On 9/15/14, 10:50 AM, Ritu Sabharwal wrote:

Hi Rayees,

Can you please point me to the right system VM template for VMware. I am using 
the one from 
http://cloudstack.apt-get.eu/systemvm/systemvm64template-2014-01-14-master-vmware.ova
With this one, the import did not work correctly because the ova created under 
/mnt/secondary/template/tmpl/1/8/ is of size 0.

Thanks  Regards,
Ritu S.

-Original Message-
From: Amogh Vasekar [mailto:amogh.vase...@citrix.com]
Sent: Thursday, September 11, 2014 11:53 AM
To: dev@cloudstack.apache.org; ilya musayev; Rayees Namathponnan
Cc: Adip Shetty
Subject: Re: [Help] Help for setting up Vmware Cluster in CloudStack...

Hi,

The management server logs will help.
Templates for launching user Vms won't be available until SSVM comes up and 
downloads them to the correct location.
You may want to confirm if you are using the right systemVM template for the 
(4.5?) build.
Adding Rayees who may be able to point to the right systemVM template.

Thanks,
Amogh

On 9/11/14 11:25 AM, Ritu Sabharwal rsabh...@brocade.com wrote:


Hello CS developers,

I need help in setting up Vmware cluster in CloudStack.

Any help is appreciated!

Thanks  Regards,
Ritu S.

-Original Message-
From: Ritu Sabharwal [mailto:rsabh...@brocade.com]
Sent: Friday, September 05, 2014 4:51 PM
To: ilya musayev
Cc: dev@cloudstack.apache.org; Adip Shetty
Subject: RE: [Help] Help for setting up Vmware Cluster in CloudStack...

Hi Ilya,

I am trying to do this vmware integration with Cloudstack again but
seeing some issues. Need your help to move forward.

I followed the steps from the tutorial:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Advan
ced
+Network+Tutorial+-+Step+by+Step

I imported the system template from the location
http://cloudstack.apt-get.eu/systemvm/systemvm64template-2014-01-14-mas
ter
-vmware.ova by running the command

/usr/share/cloudstack-common/scripts/storage/secondary/cloud-install-sy
s-t mplt -m /mnt/secondary -u
systemvm64template-2014-01-14-master-vmware.ova
-h vmware -F

Looks like the import did not happen successfully because the ova file
created under /mnt/secondary/template/tmpl/1/8/ is of size 0.

I followed rest of the steps in the tutorial . All went fine, adding
zone, pod, cluster, primary storage, secondary storage but the system
vms (Secondary storage VM and Console Proxy VM)  are not deployed.

Now when I am trying to create a user VM, I don't see any templates
available to create it.

Please help to unblock me with this setup.

Thanks  Regards,
Ritu S.

From: ilya musayev [mailto:ilya.mailing.li...@gmail.com]
Sent: Wednesday, July 09, 2014 10:53 PM
To: Ritu Sabharwal; Hugo Trippaers
Cc: dev@cloudstack.apache.org
Subject: Re: [Help] Help for setting up Vmware Cluster in CloudStack...

I'm in Sunnyvale. I've been trying to do a meetup and perhaps demo the
whole vmware/cloudstack setup, but i never really got to do this.

Hopefully, we can put something together for July or early August
timeframe for the meetup.

Also, Router VMs have 1 Management and 1 Guest IP - just realized i
missed it.

If i can give you a suggestion in the meantime, try to get your setup
working with stable 4.3. Once you figure the process out - which is
really not that difficult in my opinion (should take 10-30 minutes at
most - granted you have everything pre-set), then try it with master
codebase, and only then master codebase and your patch.

You did not mention if you are using VLAN tagging btw. Not that it
matters greatly, but consider using newer vSphere, you can download
trials from VMware site, you can even do a nested setup and run many
vSphere CloudStack environments for dev/qa - which is what i've been
doing lately. You have 60 day trial, post that, you can reset the
environment to be compliant with vmware licensing model - needless to
say, this can be semi or completely automated, depends how much effort
you want to put into this. I'd double check this with vmware rep to be
on the safe side.


On 7/9/14, 5:36 PM, Ritu Sabharwal wrote:
Hi Ilya,

Thanks for your replies.

This is really helpful. We can surely get into some video conferencing
and get this setup ready. I am in San Jose, CA.

Thanks  Regards,
Ritu S.

From: ilya musayev [mailto:ilya.mailing.li...@gmail.com]
Sent: Wednesday, July 09, 2014 12:58 PM
To: Ritu Sabharwal; Hugo Trippaers
Cc: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [Help] Help for setting up Vmware Cluster in CloudStack...

Ritu,

That tutorial is a bit old.

Let me try to answer them, i do have an alternative solution, perhaps
we can get on the phone and do a webex session or something similar -
this week is bad for me, i'm traveling, but next week, Thursday and
beyond - my schedule is flexible. I assume you are on PST timezone
somewhere in bay area.

Please see response inline to your inquiries.



On 7/8/14, 6:29 PM, Ritu Sabharwal

Re: [VOTE] Release Apache CloudStack 4.3.1 round #4

[ilya musayev]

Upgraded one of my test environments, no issues found so far +1 binding.
On 9/11/14, 4:07 PM, Mike Tutkowski wrote:

Based on my regression testing, I am +1.

On Thu, Sep 11, 2014 at 1:50 PM, Sebastien Goasguen run...@gmail.com
wrote:


On Sep 11, 2014, at 3:47 PM, Mike Tutkowski mike.tutkow...@solidfire.com
wrote:


OK, I looked into this more.

It turns out the regression test that was failing is only valid in 4.4

and

beyond, so it is not a valid test for 4.3.


so can you vote ? we need your binding vote :)


On Thu, Sep 11, 2014 at 11:14 AM, Mike Tutkowski 
mike.tutkow...@solidfire.com wrote:


I have an integration test failing when trying to create a (XenServer)
storage repository.

Let me look into it for an hour or so and then I can respond back as to

if

it's a code issue.

I'm thinking now it's a problem with XAPI. At first the code wasn't
compiling due to XAPI changes and now I'm getting this XAPI issue when
trying to create a storage repository. Seems possibly related.

On Thu, Sep 11, 2014 at 10:37 AM, Erik Weber terbol...@gmail.com

wrote:

Except heartbleed fix.

I was merely checking if it's even poosible to do an automated upgrade.

The way i know the upgrade process we add some sql magic to check for

new

systemvm templates. If that's not made for 4.3.1 it won't do much good

to

add it as systemvm-hypervisor-version.

Erik
11. sep. 2014 15:45 skrev Wei ZHOU ustcweiz...@gmail.com følgende:


As far as I know, there is no systemvm change during upgrade from

4.3.0

to

4.3.1

2014-09-11 15:22 GMT+02:00 Erik Weber terbol...@gmail.com:


On Thu, Sep 11, 2014 at 3:01 PM, Nux! n...@li.nux.ro wrote:


Hello,

I've upgraded and it looks ok for now.
I have used these system VMs


http://jenkins.buildacloud.org/view/4.3/job/cloudstack-4.3-systemvm/

and

added them as systemvm-kvm-4.3 in the Templates.
I expected them to replace SystemVM Template (KVM), but they are

still

listed as systemvm-kvm-4.3 alongside the old ones.
Other than that everything seems ok for now.



Is there an upgrade path from 4.3.0 systemvm template to 4.3.1? Log

on

to a

newly deployed one and check /etc/cloudstack-version

--
Erik




--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
http://solidfire.com/solution/overview/?video=play*™*




--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
http://solidfire.com/solution/overview/?video=play*™*

Re: [ASK] Download volume failed for 4.3.0/ESX

[ilya musayev]

Rohit,

I've been using ACS 4.3.0+ with VmWare vSphere 5.1 and 5.5 for quite 
sometime, I do not see any issues with downloads of templates.


Perhaps its related to NFS mount or blocking firewall. You can see all 
the operations performed in vCenter Tasks. Make sure you can mount NFS 
secondary on hypervisors as well as SSVM.


Logs would help, take a look at SSVM logs.

Regards
ilya

On 9/4/14, 9:47 AM, Rohit Yadav wrote:

Hi,

Is anyone having issues around downloading volumes or templates from UI when 
using Apache CloudStack 4.3.0 with vmware/ESXi?

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab

Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design  Buildhttp://shapeblue.com/iaas-cloud-design-and-build//
CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Infrastructure 
Supporthttp://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/

This email and any attachments to it may be confidential and are intended solely 
for the use of the individual to whom it is addressed. Any views or opinions 
expressed are solely those of the author and do not necessarily represent those of 
Shape Blue Ltd or related companies. If you are not the intended recipient of this 
email, you must neither take any action based upon its contents, nor copy or show 
it to anyone. Please contact the sender if you believe you have received this email 
in error. Shape Blue Ltd is a company incorporated in England  Wales. 
ShapeBlue Services India LLP is a company incorporated in India and is operated 
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company 
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue 
SA Pty Ltd is a company registered by The Republic of South Africa and is traded 
under license from Shape Blue Ltd. ShapeBlue is a registered trademark.

Re: [ACS431] Help testing 4.3.1 vote candidate

[ilya musayev]

Ack, will be trying later tonight.

On 9/4/14, 1:22 PM, Rohit Yadav wrote:

Dear community,

Please help in testing 4.3.1 voting candidate that Sebastien has just shared 
today. 4.3.0 was a good release, this minor release tries to fix several issues 
but we need regression testing.

To get help from community folks who don’t want to build from source, I’ve 
built and put debs and rpm using the voting candidate 
(41c0eaa14d2f8cd5b58bca71f2254c2390cc6913).

For debian, put this in sources.list and install/upgrade:

deb http://packages.bhaisaab.org/cloudstack/main/debian ./

For centos/fedora/rhel, put this in a yum.repos.d/cloudstack.repo:

[apache-cloudstack]
name=Apache CloudStack
baseurl=http://packages.bhaisaab.org/cloudstack/main/centos/
enabled=1
gpgcheck=0

Feel free to use them, this location is temporary. This repo was created using:
http://cloudstack-installation.readthedocs.org/en/latest/building_from_source.html#building-deb-packages

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab

Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design  Buildhttp://shapeblue.com/iaas-cloud-design-and-build//
CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Infrastructure 
Supporthttp://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/

This email and any attachments to it may be confidential and are intended solely 
for the use of the individual to whom it is addressed. Any views or opinions 
expressed are solely those of the author and do not necessarily represent those of 
Shape Blue Ltd or related companies. If you are not the intended recipient of this 
email, you must neither take any action based upon its contents, nor copy or show 
it to anyone. Please contact the sender if you believe you have received this email 
in error. Shape Blue Ltd is a company incorporated in England  Wales. 
ShapeBlue Services India LLP is a company incorporated in India and is operated 
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company 
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue 
SA Pty Ltd is a company registered by The Republic of South Africa and is traded 
under license from Shape Blue Ltd. ShapeBlue is a registered trademark.

Re: [VOTE] Release Apache CloudStack 4.3.1

[ilya musayev]

I've been running acs 4.3.0+ patches for quite sometime in large scale 
environment with VmWare, but have not had a chance to try out this 
specific build. Looked at commits and dont see very major changes, but 
will try later today on smaller env or laptop build.


This is by far the most stable release, looks like I will be riding it 
for a while.


Regards
ilya
On 9/4/14, 8:07 AM, sebgoa wrote:

Hi All,

I've created a 4.3.1 release, with the following artifacts up for a vote:

Git Branch and Commit SH:
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;h=refs/heads/4.3
Commit: 41c0eaa14d2f8cd5b58bca71f2254c2390cc6913

List of changes:
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=CHANGES.md;hb=4.3

Source release (checksums and signatures are available at the same location):
https://dist.apache.org/repos/dist/dev/cloudstack/4.3.1/

PGP release keys (signed using 8F89EB84):
https://dist.apache.org/repos/dist/release/cloudstack/KEYS

Vote will be open for 72 hours.

For sanity in tallying the vote, can PMC members please be sure to indicate 
(binding) with their vote?

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Re: [DISCUSS] Prevent force push on release and master branches

[ilya musayev]

+1 on disable force push.

On 8/29/14, 8:39 AM, Nate Gordon wrote:

+1 Rewrite history on a public repo opens the door for very catastrophic
situations.


On Fri, Aug 29, 2014 at 7:23 AM, Ian Duffy i...@ianduffy.ie wrote:


So, I’m told that we need to discuss this issue and only after a

decision/voting they would implement such a change? What do you think, what
are the pros and cons of disabling force pushes and should we do it?

Definitely a +1 on disabling. forced pushing in my view should be never
used on a public repository. It can break somebodies pull and reset/rewrite
history in a repo.

There should be no reason for anybody to ever have a requirement to do a
force push. Git supplies us with a revert that will allow us to undo
commits without destroying history.


On 29 August 2014 12:57, Rohit Yadav rohit.ya...@shapeblue.com wrote:


Hi all,

I found by accident that you can force push on branches on our ACS/ASF
repo.

So, I’m concerned that anyone could do a force push, sneak in changes on
ACS/ASF branches; opensource projects have had issues with people

changing

histories, putting backdoors and whatnot, so I would like to avoid that.
For example, I would like us to disable force-push(es) on our git repos

to

avoid a situation like  Jenkins had:




https://groups.google.com/forum/#!searchin/jenkinsci-dev/force$20push/jenkinsci-dev/-myjRIPcVwU/mrwn8VkyXagJ

So, I requested the infra team [1] to disable force pushes on release and
master branches because that made sense and I was told that they don’t
allow force pushes on master (relief!) but not on release branches

because

it would be hard to track the release branch names. I asked them if we
should disable force pushed on all branches at all and if they people

want

to do it, they can do it in their own private/github repos/forks.

So, I’m told that we need to discuss this issue and only after a
decision/voting they would implement such a change? What do you think,

what

are the pros and cons of disabling force pushes and should we do it?

[1] https://issues.apache.org/jira/browse/INFRA-8228


Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab



Find out more about ShapeBlue and our range of CloudStack related

services

IaaS Cloud Design  Build
http://shapeblue.com/iaas-cloud-design-and-build//
CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Infrastructure Support
http://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training Courses
http://shapeblue.com/cloudstack-training/

This email and any attachments to it may be confidential and are intended
solely for the use of the individual to whom it is addressed. Any views

or

opinions expressed are solely those of the author and do not necessarily
represent those of Shape Blue Ltd or related companies. If you are not

the

intended recipient of this email, you must neither take any action based
upon its contents, nor copy or show it to anyone. Please contact the

sender

if you believe you have received this email in error. Shape Blue Ltd is a
company incorporated in England  Wales. ShapeBlue Services India LLP is

a

company incorporated in India and is operated under license from Shape

Blue

Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in

Brasil

and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd

is

a company registered by The Republic of South Africa and is traded under
license from Shape Blue Ltd. ShapeBlue is a registered trademark.

[4.3.1] - rpm build failed

[ilya musayev]

Sebastian

I've checked out latest 4.3.1 and try rebuilding the rpm, it failed.

[INFO] Scanning for projects...
[ERROR] The build could not read 1 project - [Help 1]
[ERROR]
[ERROR]   The project org.apache.cloudstack:xapi:5.6.100-1-SNAPSHOT 
(/build/cloudstack/deps/XenServerJava/pom.xml) has 1 error
[ERROR] Non-resolvable parent POM: Failure to find 
org.apache.cloudstack:cloudstack:pom:4.3.0 in 
http://repo.maven.apache.org/maven2 was cached in the local repository, 
resolution will not be reattempted until the update interval of central 
has elapsed or updates are forced and 'parent.relativePath' points at 
wrong local POM @ line 21, column 11 - [Help 2]

[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the 
-e switch.

[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, 
please read the following articles:
[ERROR] [Help 1] 
http://cwiki.apache.org/confluence/display/MAVEN/ProjectBuildingException
[ERROR] [Help 2] 
http://cwiki.apache.org/confluence/display/MAVEN/UnresolvableModelException


In summary, the script that you ran to bump up the version, missed to 
update a version properly. Here is what was done to fix the build.


@@ -21,7 +21,7 @@
   parent
 groupIdorg.apache.cloudstack/groupId
 artifactIdcloudstack/artifactId
-version4.3.0/version
+version4.3.1-SNAPSHOT/version
 relativePath../../pom.xml/relativePath
   /parent
   artifactIdxapi/artifactId

Regards
ilya

Re: 4.4 Logging

[ilya musayev]

Michael,

Edit /etc/cloudstack/management/log4j-cloud.xml

Change from INFO to DEBUG, ERROR or TRACE specific components to make 
logging more verbose.


Regards
ilya
On 8/25/14, 2:00 PM, Michael Phillips wrote:

Seems the logging level in 4.4 has been toned down, how do you crank it up to 
verbose levels?   

Re: VMs stuck in expunging

[ilya musayev]

Its apparent that expunge failed, because VM name ketchup-web was to 
good to be deleted :)


Try increasing the verbosity level on log4j so we can get a better picture.


On 8/26/14, 7:29 AM, Francois Gaudreault wrote:

Hi,

We have couple VMs stuck in Expunging state in our setup, and I am not 
sure why. This is a 4.3.1-snapshot. This is what the log says:


2014-08-25 23:30:31,789 DEBUG [c.c.v.VirtualMachineManagerImpl] 
(UserVm-Scavenger-1:ctx-bfdf6aa3) Stopped called on 
VM[User|ketchup-web] but the state is Expunging
2014-08-25 23:30:31,808 WARN  [c.c.u.n.Link] 
(AgentManager-Selector:null) SSL: Fail to find the generated keystore. 
Loading fail-safe one to continue.
2014-08-25 23:30:31,810 DEBUG [c.c.c.CapacityManagerImpl] 
(UserVm-Scavenger-1:ctx-bfdf6aa3) VM state transitted from :Expunging 
to Expunging with event: ExpungeOperationvm's original host id: 1 new 
host id: null host id before state transition: null
2014-08-25 23:30:31,811 DEBUG [c.c.v.VirtualMachineManagerImpl] 
(UserVm-Scavenger-1:ctx-bfdf6aa3) Destroying vm VM[User|ketchup-web]
2014-08-25 23:30:31,811 DEBUG [c.c.v.VirtualMachineManagerImpl] 
(UserVm-Scavenger-1:ctx-bfdf6aa3) Cleaning up NICS
2014-08-25 23:30:31,811 DEBUG [o.a.c.e.o.NetworkOrchestrator] 
(UserVm-Scavenger-1:ctx-bfdf6aa3) Cleaning network for vm: 61
2014-08-25 23:30:31,814 DEBUG [c.c.v.VirtualMachineManagerImpl] 
(UserVm-Scavenger-1:ctx-bfdf6aa3) Cleaning up hypervisor data 
structures (ex. SRs in XenServer) for managed storage
2014-08-25 23:30:31,821 DEBUG [o.a.c.e.o.VolumeOrchestrator] 
(UserVm-Scavenger-1:ctx-bfdf6aa3) Cleaning storage for vm: 61
2014-08-25 23:30:31,825 DEBUG [c.c.v.VirtualMachineManagerImpl] 
(UserVm-Scavenger-1:ctx-bfdf6aa3) Expunged VM[User|ketchup-web]
2014-08-25 23:30:31,825 DEBUG [c.c.v.UserVmManagerImpl] 
(UserVm-Scavenger-1:ctx-bfdf6aa3) Starting cleaning up vm 
VM[User|ketchup-web] resources...
2014-08-25 23:30:31,856 DEBUG [c.c.n.f.FirewallManagerImpl] 
(UserVm-Scavenger-1:ctx-bfdf6aa3) No firewall rules are found for vm 
id=61
2014-08-25 23:30:31,872 DEBUG [c.c.v.UserVmManagerImpl] 
(UserVm-Scavenger-1:ctx-bfdf6aa3) Firewall rules are removed 
successfully as a part of vm id=61 expunge
2014-08-25 23:30:31,884 DEBUG [c.c.u.d.T.Transaction] 
(UserVm-Scavenger-1:ctx-bfdf6aa3) Rolling back the transaction: Time = 
5 Name =  UserVm-Scavenger-1; called by 
-TransactionLegacy.rollback:896-TransactionLegacy.removeUpTo:839-TransactionLegacy.close:663-Transaction.execute:41-Transaction.execute:46-FirewallManagerImpl.revokeRule:732-GeneratedMethodAccessor278.invoke:-1-DelegatingMethodAccessorImpl.invoke:43-Method.invoke:606-AopUtils.invokeJoinpointUsingReflection:317-ReflectiveMethodInvocation.invokeJoinpoint:183-ReflectiveMethodInvocation.proceed:150
2014-08-25 23:30:31,887 WARN  [c.c.v.UserVmManagerImpl] 
(UserVm-Scavenger-1:ctx-bfdf6aa3) Unable to expunge VM[User|ketchup-web]


Any idea what's going on? Bug?

Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0 (take #3)

[ilya musayev]

+1 from, i did very basic tests so far - sorry i'm late with response.

On 8/25/14, 5:03 PM, Rohit Yadav wrote:

Hi Mike,

Thanks for participating and you’re welcome!

Will work on an announcement and some documentation; meanwhile can you help 
check svn access as I’m unable to move/publish cloudmonkey on the release 
location using svn?

Everyone, the new version is available via pypi;
pip install —upgrade cloudmonkey

Cheers.

On 26-Aug-2014, at 1:50 am, Mike Tutkowski mike.tutkow...@solidfire.com wrote:


Thanks again, Rohit!


On Mon, Aug 25, 2014 at 5:38 PM, Rohit Yadav rohit.ya...@shapeblue.com
wrote:


Hi all,

Thanks to everyone for participating.

We started the voting thread on 21st and it’s 26th now (in my tz). The
vote for CloudStack CloudMonkey 5.2.0 *passes* with

+1 (PMC / binding)
3 person

+1 (non binding)
2 person

0
none

-1
none

I will now prepare the release announcement to go out after 24 hours to
give the mirrors time to catch up, push the tag and publish the source
distribution on pypi.

Cheers,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab


Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design  Build
http://shapeblue.com/iaas-cloud-design-and-build//
CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Infrastructure Support
http://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training Courses
http://shapeblue.com/cloudstack-training/

This email and any attachments to it may be confidential and are intended
solely for the use of the individual to whom it is addressed. Any views or
opinions expressed are solely those of the author and do not necessarily
represent those of Shape Blue Ltd or related companies. If you are not the
intended recipient of this email, you must neither take any action based
upon its contents, nor copy or show it to anyone. Please contact the sender
if you believe you have received this email in error. Shape Blue Ltd is a
company incorporated in England  Wales. ShapeBlue Services India LLP is a
company incorporated in India and is operated under license from Shape Blue
Ltd. Shape Blue Brasil Consultoria Ltda is a company incorporated in Brasil
and is operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is
a company registered by The Republic of South Africa and is traded under
license from Shape Blue Ltd. ShapeBlue is a registered trademark.




--
*Mike Tutkowski*
*Senior CloudStack Developer, SolidFire Inc.*
e: mike.tutkow...@solidfire.com
o: 303.746.7302
Advancing the way the world uses the cloud
http://solidfire.com/solution/overview/?video=play*™*

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab



Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design  Buildhttp://shapeblue.com/iaas-cloud-design-and-build//
CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Infrastructure 
Supporthttp://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/

This email and any attachments to it may be confidential and are intended solely 
for the use of the individual to whom it is addressed. Any views or opinions 
expressed are solely those of the author and do not necessarily represent those of 
Shape Blue Ltd or related companies. If you are not the intended recipient of this 
email, you must neither take any action based upon its contents, nor copy or show 
it to anyone. Please contact the sender if you believe you have received this email 
in error. Shape Blue Ltd is a company incorporated in England  Wales. 
ShapeBlue Services India LLP is a company incorporated in India and is operated 
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company 
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue 
SA Pty Ltd is a company registered by The Republic of South Africa and is traded 
under license from Shape Blue Ltd. ShapeBlue is a registered trademark.

CloudStack UI - very slow or never loads content in large env

[ilya musayev]

I have an issue with 4.3.0.1 when i login as admin user.

CloudStack UI does not display the object unless i click around multiple 
times on different sections. This is evident in environment with greater 
than 500+ VMs.


Example: You login and press on Instances - that may or may not load on 
the first attempt. You press elsewhere and press Instances again, then 
the page loads. You pick the VM, press on it - it may or may not show 
the content unless i click around elsewhere. Sometimes i can do 2 - 3 
click drill down on the 4th, it wont load the content completely.


I've checked apiserver.log and i do see that api calls go thru. If i use 
cloudmonkey, i get the response rather quick, so it appears to be UI 
related only.


I've tried different browsers, the issue affects all browsers.

Any idea on how to troubleshoot this issue? i've given tomcat plenty of 
heap, but it did not help.


Thanks,
ilya

Re: [GSOC] [CLOUDSTACK-6045] Final update

[ilya musayev]

Wanted to add to Sebastian's comment your work is very much appreciated. 
We hope you stay with us in the future.


On 8/20/14, 12:54 AM, Sebastien Goasguen wrote:

Great work Seif, thanks for the slides.

I hope you stay involved with the community.

-sebastien

On Aug 19, 2014, at 10:26 PM, Seif Eddine Jemli seifeddineje...@gmail.com 
wrote:


Hi everyone,

As you probably know, I have worked on this project whose goal is making
the GUI support adding primary storage to CS that is based on storage
plug-ins.

https://cwiki.apache.org/confluence/display/CLOUDSTACK/Create+GUI+to+add+primary+storage+based+on+plug-ins

The pencils down firm deadline was Monday the 18th. I spent yesterday
preparing a presentation that summarizes my GSoC tasks from this summer.

Here is the link to the presentation:

http://fr.slideshare.net/seif_100/seif-mike-gsoc2014cloudstack
http://www.facebook.com/l.php?u=http%3A%2F%2Ffr.slideshare.net%2Fseif_100%2Fseif-mike-gsoc2014cloudstackh=QAQEcJBPc



I have worked mainly on two tasks:

* The first task I worked on was adding support into CloudStack's GUI to
enable admins to add Primary Storage that is based on a custom plug-in.
Prior to this, admins needed to perform this task either via CloudStack's
CLI, its API, or some other tool that leveraged CloudStack's CLI or its API.

* The other task was adding the following GUI control to CloudStack:

http://loopj.com/jquery-tokeninput/

This control replaces the need for admins to type in a comma-separated list
of what CloudStack calls storage tags (we were also able to apply this
control to what CloudStack calls host tags).

As part of this process, I needed to create two new API calls to CloudStack
and invoke them from the GUI. These API calls retrieve the current list of
storage tags or host tags and populate them into the GUI control. The admin
can then either select one or more existing tags and/or add new ones.
Overall, this feature will make it much easier for admins to avoid input
errors.

GUI forms that make use of this new control include the following:

-Add Primary Storage

-Add Host

-Add Compute Offering

-Add Disk Offering

Two detail views make use of this new control for editing tags:

-Edit mode for a primary storage

-Edit mode for a host

These two tasks were completely fulfilled, and I hope that they will be
useful for the CloudStack admins.

I would like to thank my mentor, Mike Tutkowski, for his help and guidance
during the project. I would also like to thank the Apache Cloudstack
Community members for their helpful answers.

The GSoC experience was very good overall, with especially a lot of
learning. That being the case, I'll probably apply next year for another
project.

Beyond GSoC, I am thinking of modifying the URL input on the Add Primary
Storage form to make it more user friendly by accepting key-value inputs.

Thanks everyone,

Seif

Re: Apache CloudStack 4.3.1 Release Help

[ilya musayev]

Francois

I agree, but many on this list had tough time getting to 4.3. Efforts on 
4.4.1 are definitely needed, but not all folks will benefit from it just 
yet. Many will purposely stay 1 major version behind, for the known reasons.


4.3.1 should be an easy one to get done vs 4.4.1 which needs much of QA.

Regards
iya

On 8/19/14, 7:15 PM, Francois Gaudreault wrote:

At that point, shouldn't we put all the efforts on 4.4.1 instead?

Francois

On 2014-08-19, 8:28 PM, ilya musayev wrote:

Sebastian,

I'd like to help you with Apache CloudStack 4.3.1 release that is if 
you need help.


If you do need help, lets sync up via call, GTM, IRC or whatever else 
works.


Regards
ilya

Re: CloudStack UI - very slow or never loads content in large env

[ilya musayev]

Ian,

If i'm not mistaken, was it you who helped with graphics optimization?

Thanks
ilya

On 8/20/14, 3:40 AM, Ian Duffy wrote:

Ilya,

I'm interesting in knowing if this is due to the UI or just slow responses
from the API.

Can you open the UI within Google Chrome and open the developer tools. If
you switch over to the networking tab you should be able to see how long
API calls are taking to respond.


On 20 August 2014 08:49, ilya musayev ilya.mailing.li...@gmail.com wrote:


I have an issue with 4.3.0.1 when i login as admin user.

CloudStack UI does not display the object unless i click around multiple
times on different sections. This is evident in environment with greater
than 500+ VMs.

Example: You login and press on Instances - that may or may not load on
the first attempt. You press elsewhere and press Instances again, then the
page loads. You pick the VM, press on it - it may or may not show the
content unless i click around elsewhere. Sometimes i can do 2 - 3 click
drill down on the 4th, it wont load the content completely.

I've checked apiserver.log and i do see that api calls go thru. If i use
cloudmonkey, i get the response rather quick, so it appears to be UI
related only.

I've tried different browsers, the issue affects all browsers.

Any idea on how to troubleshoot this issue? i've given tomcat plenty of
heap, but it did not help.

Thanks,
ilya

Re: [VOTE] Release Apache CloudStack CloudMonkey 5.2.0

[ilya musayev]

Rohit,

I was POCing the new release yesterday, did not notice this issue (also 
got side tracked and could not complete testing), but will retry tomorrow.


Thanks for doing great work,

CloudMonkey is life saver,

Regards
ilya
On 8/20/14, 4:45 PM, Rohit Yadav wrote:

Hi guys,

A bug was identified in newly introduce server profiles feature and the logic 
of reading/writing config files and was fixed on 5.2 branch. I’ve pushed new 
artifacts, please use them for voting/testing.

The updated Commit SHA is 264ddcecb1274ead2ca4879627e373ba78780117 for the same 
branch:
https://git-wip-us.apache.org/repos/asf?p=cloudstack-cloudmonkey.git;a=shortlog;h=refs/heads/5.2

Source release (checksums and signatures are available at the same location):
https://dist.apache.org/repos/dist/dev/cloudstack/cloudmonkey-5.2.0/

PGP release keys (signed using my key ID 0EE3D884):
https://dist.apache.org/repos/dist/release/cloudstack/KEYS

+1 from my side
Tested against latest master and 4.4 for key, password and non-authenticated 
auth methods.

Regards.

On 19-Aug-2014, at 3:31 pm, Rohit Yadav rohit.ya...@shapeblue.com wrote:


Hi All,

I've created a 5.2.0 release of CloudMonkey, with the following artifacts up 
for a vote:

Git Branch and Commit SH:
https://git-wip-us.apache.org/repos/asf?p=cloudstack-cloudmonkey.git;a=shortlog;h=refs/heads/5.2
Commit: b66b7dd62cebd77b0f1d74046170810e931fe852

List of changes:
https://git-wip-us.apache.org/repos/asf?p=cloudstack-cloudmonkey.git;a=blob_plain;f=CHANGES;hb=5.2

Source release (checksums and signatures are available at the same location):
https://dist.apache.org/repos/dist/dev/cloudstack/cloudmonkey-5.2.0/

PGP release keys (signed using my key ID 0EE3D884):
https://dist.apache.org/repos/dist/release/cloudstack/KEYS

Vote will be open for 72 hours.

For sanity in tallying the vote, can PMC members please be sure to indicate 
(binding) with their vote?

[ ] +1  approve
[ ] +0  no opinion
[ ] -1  disapprove (and reason why)

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab
Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design  Buildhttp://shapeblue.com/iaas-cloud-design-and-build//
CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Infrastructure 
Supporthttp://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/

This email and any attachments to it may be confidential and are intended solely 
for the use of the individual to whom it is addressed. Any views or opinions 
expressed are solely those of the author and do not necessarily represent those of 
Shape Blue Ltd or related companies. If you are not the intended recipient of this 
email, you must neither take any action based upon its contents, nor copy or show 
it to anyone. Please contact the sender if you believe you have received this email 
in error. Shape Blue Ltd is a company incorporated in England  Wales. 
ShapeBlue Services India LLP is a company incorporated in India and is operated 
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company 
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue 
SA Pty Ltd is a company registered by The Republic of South Africa and is traded 
under license from Shape Blue Ltd. ShapeBlue is a registered trademark.

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab



Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design  Buildhttp://shapeblue.com/iaas-cloud-design-and-build//
CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Infrastructure 
Supporthttp://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/

This email and any attachments to it may be confidential and are intended solely 
for the use of the individual to whom it is addressed. Any views or opinions 
expressed are solely those of the author and do not necessarily represent those of 
Shape Blue Ltd or related companies. If you are not the intended recipient of this 
email, you must neither take any action based upon its contents, nor copy or show 
it to anyone. Please contact the sender if you believe you have received this email 
in error. Shape Blue Ltd is a company incorporated in England  Wales. 
ShapeBlue Services India LLP is a company incorporated in India and is operated 
under license from Shape Blue Ltd. Shape Blue Brasil Consultoria Ltda is a company 
incorporated in Brasil and is operated under license from Shape Blue Ltd. ShapeBlue 
SA Pty Ltd is a company 

Re: 4.5 RM

[ilya musayev]

David

Since CI has been mentioned several times, do you know where we stand 
with donated hardware?


Thanks
ilya
On 8/19/14, 10:15 AM, David Nalley wrote:

IMHO we should not even release 4.5 until we have a agreed upon:

-what our issues are and why we released 4.4 and 4.3 late.
-taken action to resolve those issues
-guarantees that 4.5 will be on time

If we don't do that, I don't even know why we are putting ourselves through the 
pain of a release schedule.


So I've been trying to give this some thought. Here's my current line
of thinking.

The issues with late releases are not a function of our release
process per se; but are instead a function of our development process.
CloudStack is a relatively large codebase. It has a lots of points
that interact with each other, and it's moderately complex.
Development moves forward and at least happy-path testing is done for
new features, but the range of options is so large that testing
everything is a bit difficult. When someone makes a merge request; I
suspect few people do much looking. Understandable, it's a boring
task; and really looking doesn't tell us much except for style and
egregious errors. We've rarely done mandatory testing of feature
branches before they are merged in. If you want to ship on time, you
must ensure that we are vociferously guarding the quality of the
master and release branches; that we can verify programmatically that
a commit or merge doesn't break things. We must insist on automated
testing being added.

So I've said all of that to say that I think that ship has sailed for
4.5. We are well past feature freeze; and we didn't really have any
gating functionality. We frankly have very little idea of quality of
whats in master right now. It's certainly worse than 4.4. So now we'll
enter code freeze, we'll try and play catch up and fix all of the
things we discover that are broken. And invariably, we'll be late
again.

If you want to solve this problem; my personal belief is that its
really is tied to CI. Efforts around Travis are interesting and
perhaps are a piece of that puzzle. Discussions around running CI are
important as well, but I truly believe that we need a gating function
that prohibits commits that increase our level of untested code or
code that fails to pass testing. I've seen some other projects using
pull requests in github, and then using the github pull request
builder[1] for jenkins to verify that every PR works. I know we've
talked about gerrit previously, and perhaps that will work as well.

[1] https://wiki.cloudbees.com/bin/view/DEV/Github+Pull+Request+Validation

Apache CloudStack 4.3.1 Release Help

[ilya musayev]

Sebastian,

I'd like to help you with Apache CloudStack 4.3.1 release that is if you 
need help.


If you do need help, lets sync up via call, GTM, IRC or whatever else 
works.


Regards
ilya

Re: cannot create template

[ilya musayev]

Please check if convert to template job timed out.
Also, if you see an error in vCenter when template created, please post 
it here. You may need to pull out hostd.log from the hypervisor that the 
command was executed on.


Template generation maybe a slow process for vmware in general - 
depending on your storage setup.


Have you tried using 4.3?


On 8/14/14, 1:04 AM, fight300...@sina.com wrote:

The envirionment is cloudstack4.0.0+vmware5.0.There was an error when i
created a template.The following is my procedure:
1. stop the instance
2. click the create template button on the website of the root disk of the 
instance.
Then there was an error.The following is the errorlog of mangement-server:
2014-07-29 22:02:28,113 DEBUG [agent.transport.Request] (AgentManager-Handler-4:null) Seq 12-916361029: Processing:  { Ans: , MgmtId: 345049268249, 
via: 12, Ver: v1, Flags: 10, 
[{storage.CreatePrivateTemplateAnswer:{_virtualSize:0,_physicalSize:0,result:false,details:CreatePrivateTemplateFromVolumeCommand
 exception: 
java.lang.NullPointerException\ncom.cloud.hypervisor.vmware.mo.HostMO.getVmPropertiesOnHyperHost(HostMO.java:586)\ncom.cloud.hypervisor.vmware.mo.HostMO.loadVmCache(HostMO.java:485)\ncom.cloud.hypervisor.vmware.mo.HostMO.findVmOnHyperHost(HostMO.java:475)\ncom.cloud.hypervisor.vmware.manager.VmwareStorageManagerImpl.execute(VmwareStorageManagerImpl.java:264)\ncom.cloud.storage.resource.VmwareSecondaryStorageResourceHandler.execute(VmwareSecondaryStorageResourceHandler.java:121)\ncom.cloud.storage.resource.VmwareSecondaryStorageResourceHandler.executeRequest(VmwareSecondaryStorageResourceHandler.java:73)\ncom.cloud.storage.resource.PremiumSecondaryStorageResource.executeRequest(PremiumSecondaryStorageResource.java:54)\ncom.cloud.agent.Agent.processRequest(Agent.java:518)\ncom.cloud.agent.Agent$AgentRequestHandler.doTask(Agent.java:831)\ncom.cloud.utils.nio.Task.run(Task.java:83)\njava.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)\njava.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)\njava.lang.Thread.run(Thread.java:662)\n,wait:0}}]
 }
2014-07-29 22:02:28,114 DEBUG [agent.transport.Request] 
(Job-Executor-20:job-200) Seq 12-916361029: Received:  { Ans: , MgmtId: 
345049268249, via: 12, Ver: v1, Flags: 10, { CreatePrivateTemplateAnswer } }
2014-07-29 22:02:28,140 ERROR [cloud.api.ApiDispatcher] 
(Job-Executor-20:job-200) Exception while executing CreateTemplateCmd:
com.cloud.utils.exception.CloudRuntimeException: Failed to create a template
 at 
com.cloud.vm.UserVmManagerImpl.createPrivateTemplate(UserVmManagerImpl.java:1662)
 at 
com.cloud.utils.component.ComponentLocator$InterceptorDispatcher.intercept(ComponentLocator.java:1231)
 at 
com.cloud.vm.UserVmManagerImpl.createPrivateTemplate(UserVmManagerImpl.java:237)
 at 
com.cloud.api.commands.CreateTemplateCmd.execute(CreateTemplateCmd.java:265)
 at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:138)
 at 
com.cloud.async.AsyncJobManagerImpl$1.run(AsyncJobManagerImpl.java:432)
 at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
 at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
 at java.util.concurrent.FutureTask.run(FutureTask.java:166)
 at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
 at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
 at java.lang.Thread.run(Thread.java:679)
2014-07-29 22:02:28,141 DEBUG [cloud.async.AsyncJobManagerImpl] 
(Job-Executor-20:job-200) Complete async job-200, jobStatus: 2, resultCode: 
530, result: Error Code: 530 Error text: Failed to create a template

Re: 'Read timed out' issue

[ilya musayev]

Denis

You havent mentioned anything about your setup and CloudStack version.

Regards,
ilya
On 7/29/14, 1:42 AM, Denis Finko wrote:

Hello CloudStack Developers,

Could you please help me to investigate or understand following issue with time 
out:

When I am creating new VMs from CloudStack template with large storage size and 
if 'Clone virtual machine' in VMware takes more that 20 min CloudStack can't 
provision this VM and show following error:

Start cloning new VM:
2014-07-25 08:02:55,419 DEBUG [agent.manager.ClusteredAgentAttache] 
(Job-Executor-107:job-3742 = [ aca3b27c-3ef8-46ed-9542-3148dd055533 ]) Seq 
4-421602206: Forwarding Seq 4-421602206:  { Cmd , MgmtId: 345048632606, via: 4, 
Ver: v1, Flags: 100111,
[{org.apache.cloudstack.storage.command.CopyCommand:{srcTO:{org.apache.cloudstack.storage.to.TemplateObjectTO:{path:abc3a4f5f06634c890e8364f97b0380d,origUrl:http://mydomain.com/WS12R2-I-MY5MS12S-0.ova,uuid:c44d5426-4b07-4dc3-b925-4fc36dfcc175,id:632,format:OVA,accountId:2,checksum:78af69cd4c13694a36f605d035aa8a34,hvm:true,displayText:Microsoft
Windows Server 2012 R2 featuring Internet Information Services (IIS) 8.5, MSSQL 
2012 Standard Edition and MySQL 5.x without the overhead of a control
panel.,imageDataStore:{org.apache.cloudstack.storage.to.PrimaryDataStoreTO:{uuid:a419f8f4-6511-3e15-aabf-44578f6c1b8f,id:3,poolType:VMFS,host:VMFS
 datastore: /DS/AUTO1,path:/DS/
AUTO1,port:0}},name:632-2-cebb1554-b755-3197-b636-b43648f36554,hypervisorType:VMware}},destTO:{org.apache.cloudstack.storage.to.VolumeObjectTO:{uuid:5f5a9b6c-6e51-4837-91c9-9844a1e6f0cc,volumeType:ROOT,dataStore:{org.apache.cloudstack.storage.to.PrimaryDataStoreTO:{uuid:a419f8f4-6511-3e15-aabf-44578f6c1b8f,id:3,poolType:VMFS,host:VMFS
datastore: 
/DS/AUTO1,path:/DS/AUTO1,port:0}},name:ROOT-1265,size:42949672960,volumeId:1319,vmName:i-43-1265-VM,accountId:43,format:OVA,id:1319,hypervisorType:VMware}},executeInSequence:true,wait:0}}]
 } to 345048630655

After 20 minutes:
2014-07-25 08:22:55,965 DEBUG [agent.transport.Request] (AgentManager-Handler-13:null) Seq 4-421602206: 
Processing:  { Ans: , MgmtId: 345048632606, via: 4, Ver: v1, Flags: 110, 
[{org.apache.cloudstack.storage.command.CopyCmdAnswer:{result:false,details:javax.xml.ws.WebServiceException:
java.net.SocketTimeoutException: Read timed out,wait:0}}] }

2014-07-25 08:22:55,974 WARN  [storage.datastore.ObjectInDataStoreManagerImpl] 
(Job-Executor-107:job-3742 = [ aca3b27c-3ef8-46ed-9542-3148dd055533 ]) 
Unsupported data object (VOLUME, 
org.apache.cloudstack.storage.datastore.PrimaryDataStoreImpl@33a78d5c), no need 
to delete from object in store ref table
2014-07-25 08:22:55,975 INFO  [storage.volume.VolumeServiceImpl] 
(Job-Executor-107:job-3742 = [ aca3b27c-3ef8-46ed-9542-3148dd055533 ]) 
releasing lock for VMTemplateStoragePool 4102
2014-07-25 08:22:55,976 WARN  [utils.db.Merovingian2] 
(Job-Executor-107:job-3742 = [ aca3b27c-3ef8-46ed-9542-3148dd055533 ]) Was 
unable to find lock for the key template_spool_ref4102 and thread id 611898505
2014-07-25 08:22:55,976 DEBUG [cloud.storage.VolumeManagerImpl] 
(Job-Executor-107:job-3742 = [ aca3b27c-3ef8-46ed-9542-3148dd055533 ]) Unable 
to create Vol[1319|vm=1265|ROOT]:javax.xml.ws.WebServiceException: 
java.net.SocketTimeoutException: Read timed out
2014-07-25 08:22:55,976 INFO  [cloud.vm.VirtualMachineManagerImpl] 
(Job-Executor-107:job-3742 = [ aca3b27c-3ef8-46ed-9542-3148dd055533 ]) Unable 
to contact resource.
com.cloud.exception.StorageUnavailableException: Resource [StoragePool:3] is 
unreachable: Unable to create 
Vol[1319|vm=1265|ROOT]:javax.xml.ws.WebServiceException: 
java.net.SocketTimeoutException: Read timed out
 at 
com.cloud.storage.VolumeManagerImpl.recreateVolume(VolumeManagerImpl.java:2629)
 at 
com.cloud.storage.VolumeManagerImpl.prepare(VolumeManagerImpl.java:2680)
 at 
com.cloud.vm.VirtualMachineManagerImpl.advanceStart(VirtualMachineManagerImpl.java:958)
 at 
com.cloud.vm.VirtualMachineManagerImpl.start(VirtualMachineManagerImpl.java:647)
 at 
org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.deployVirtualMachine(VMEntityManagerImpl.java:227)
 at 
org.apache.cloudstack.engine.cloud.entity.api.VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:209)
 at 
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3588)
 at 
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3107)
 at 
com.cloud.vm.UserVmManagerImpl.startVirtualMachine(UserVmManagerImpl.java:3093)
 at 
com.cloud.utils.component.ComponentInstantiationPostProcessor$InterceptorDispatcher.intercept(ComponentInstantiationPostProcessor.java:125)
 at 
org.apache.cloudstack.api.command.user.vm.DeployVMCmd.execute(DeployVMCmd.java:437)
 at 
com.ds.cloud.api.DeployVMWithIpRangeCmd.execute(DeployVMWithIpRangeCmd.java:85)
 at 

Re: [DISCUSS] [PROPOSAL] SAML2 plugin for SSO/SLO in CloudStack

[ilya musayev]

Rohit,

definite +1

Thanks,
ilya

On 7/20/14, 8:35 AM, Rohit Yadav wrote:

Hi,

I'm assuming no one objects the proposal and the spec, I'll move forward
with the first implementation starting next week but will be mostly
offline till 28th July.

Regards.

Rohit Yadav wrote:

Hi guys,

There has been a lot of interest [4] around auth related problems in
CloudStach such as -- SSO/SLO (single sign on / log out), 2-factor
authentication, role based network/IP/CIDR checking etc.

A lot of challenge in implementing them in CloudStack is because of two
divergent authentication mechanisms (one that is
username/password/cookie based, other which is api/secret keys or
hmac/signature based).

This thread tries to kickstart a project in that direction which will in
short term try to implement a SAML2 plugin and in long term have a much
better authentication framework.

Let me start by briefly explaining what SAML2 [1] is -- it's an XML
based authentication and authorization protocol widely used to implement
single sign on service. Having a SAML plugin in ACS will give users and
organization a new mode of authentication who already have such an
infrastructure in place.

A SAML based SSO infrastructure consists of three entities - user-agent
(UA), service provider (SP) and identity provider (IdP). The UA is the
user/browser, the SP is the application that the UA is accessing (i.e.
Apache CloudStack UI) and the IdP is the identity service and does
authentication and authorization, management of users among other
things. IdP could be backed by LDAP, AD etc. For the scope of this
feature, we only need to implement SAML SP plugin in CloudStack and use
any free SAML 2.0 compliant IdP server [5] for testing.

For this I researched and explored ways of implementing that and have a
first draft which needs to be discussed and iterated in the ACS dev
community.

After comparing many opensource SAML 2.0 implementations, their
security and stability, we'll use OpenSAML [2] which is the most stable
and widely used Java implementation. Since within CloudStack, we've been
using Spring (for DI etc.) I explored and found Spring security SAML
extension [3] which fits perfectly and it too uses OpenSAML.

I also have a working proof-of-concept general implementation using the
above based on which I've put together a design document draft on this
feature for your review:

https://cwiki.apache.org/confluence/display/CLOUDSTACK/SAML+2.0+Plugin

There are some complex stories/cases around security and user management
in CloudStack, some of which are listed under 'open ended questions' in
the draft above most of which I'm not sure how to address.

After first round of discussion, I'll go ahead with a basic
implementation of this feature. The second phase will address broader
use cases.

Comments, questions, suggestions?

References:

[1] http://en.wikipedia.org/wiki/SAML_2.0
[2] https://wiki.shibboleth.net/confluence/display/OpenSAML/Home
[3] http://projects.spring.io/spring-security-saml
[4] John Burwell's talk on SSO in CloudStack:
https://www.youtube.com/watch?v=kCR0TzrfCOM
[5] https://idp.ssocircle.com/sso/UI/Login

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +91 88 262 30892 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab


Find out more about ShapeBlue and our range of CloudStack related 
services


IaaS Cloud Design 
Buildhttp://shapeblue.com/iaas-cloud-design-and-build//
CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Infrastructure
Supporthttp://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training
Courseshttp://shapeblue.com/cloudstack-training/

This email and any attachments to it may be confidential and are
intended solely for the use of the individual to whom it is addressed.
Any views or opinions expressed are solely those of the author and do
not necessarily represent those of Shape Blue Ltd or related companies.
If you are not the intended recipient of this email, you must neither
take any action based upon its contents, nor copy or show it to anyone.
Please contact the sender if you believe you have received this email in
error. Shape Blue Ltd is a company incorporated in England  Wales.
ShapeBlue Services India LLP is a company incorporated in India and is
operated under license from Shape Blue Ltd. Shape Blue Brasil
Consultoria Ltda is a company incorporated in Brasil and is operated
under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company
registered by The Republic of South Africa and is traded under license
from Shape Blue Ltd. ShapeBlue is a registered trademark.


--
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab


Find out more about ShapeBlue and our range of CloudStack related 
services


IaaS Cloud Design  
Buildhttp://shapeblue.com/iaas-cloud-design-and-build//

CSForge 

[QUESTION] Baremetal DHCP Server - Abstracting Router VM

[ilya musayev]

We are trying to abstract router vm completely from our environment as 
it has dual nics which is big no no in hardened security environments. 
This is for shared (non-vpc) advanced security zone.


CloudStack already has a support for Baremetal DHCP Server under 
Network Service Providers.


Would anyone provide the context on how one go about using it? I assume 
we would need to write a support of some sort on our end. Examples and 
documentation would be appreciated.


Thank you
ilya