Re: Congratulations to Sven - Apache Software Foundation Member

[Chiradeep Vittal]

Congratulations Sven!

On Wed, Mar 17, 2021 at 2:14 PM Paul Angus  wrote:

> Hi All,
>
>
>
> More great news.
>
>
>
> Please join me in congratulating Sven,  for being made a Member of the
> Apache Software Foundation.
>
>
>
> Congratulations Sven, keep up the good work!
>
>
>
> Kind regards
>
>
>
> Paul Angus
>
>
>
>

Re: [NOTICE] Impending shutdown of download.cloud.com - this may affect your CloudStack installation

[Chiradeep Vittal]

FINAL NOTICE
download.cloud.com will be deleted permanently on May 4 2018

On Tue, Apr 10, 2018 at 11:13 AM, Chiradeep Vittal <chirade...@gmail.com>
wrote:

> I have not heard any complaints, although from the access logs I see lots
> of attempts to download legacy templates.
> download.cloud.com will remain offline until April 30.
> On April 30 2018, download.cloud.com will be decommissioned permanently.
>
> On Fri, Mar 30, 2018 at 12:37 PM, Chiradeep Vittal <chirade...@gmail.com>
> wrote:
>
>> download.cloud.com will go offline temporarily for a few days. If you
>> need access to it, check the instructions below. If that doesn't work,
>> reply on this thread.
>>
>> On Tue, Mar 20, 2018 at 11:44 AM, Chiradeep Vittal <chirade...@gmail.com>
>> wrote:
>>
>>> On April 30, 2018, the domain download.cloud.com will be decommissioned.
>>>
>>>
>>> THIS MAY AFFECT YOUR CLOUDSTACK INSTALLATION. PLEASE READ FURTHER.
>>>
>>> BACKGROUND
>>> download.cloud.com was used to host the seed templates (images) for a
>>> CloudStack installation. This included the system vm templates for releases
>>> prior to 4.10. Around March 2017[1], a new site was established (
>>> download.cloudstack.org) and all the content (templates) was copied
>>> over to the new site. From release 4.10 onwards[2][3], the seed templates
>>> are fetched / downloaded from the new site during an installation of Apache
>>> CloudStack.
>>>
>>>
>>> To check if your CloudStack installation has a reference to the old site
>>> (download.cloud.com), check your vm_template table:
>>>
>>> SELECT url FROM vm_template;
>>>
>>> WHO MAY BE AFFECTED
>>> CloudStack operators installing a version of Apache CloudStack prior to
>>> version 4.10 after the shutdown. An already running/functioning CloudStack
>>> cloud will continue to operate with no disruption after the shutdown.
>>> However, operations such as: adding a zone, backup and restore, secondary
>>> storage backup/restore, etc may be affected.
>>>
>>> STEPS TO BE TAKEN TO AVOID IMPACT
>>> Update the vm_template table to point to the new location:
>>>
>>> UPDATE vm_template SET url = REPLACE(url, 'download.cloud.com', '
>>> download.cloudstack.org');
>>>
>>> TRIAL RUN OF THE SHUTDOWN
>>> During the last week of March 2018, download.cloud.com will be taken
>>> temporarily offline. This is to make sure that CloudStack operators who
>>> have not paid attention to this notice but are affected, will pay attention.
>>>
>>> This notice will be posted periodically until the shutdown.
>>> --
>>> Chiradeep
>>>
>>> [1] http://mail-archives.apache.org/mod_mbox/cloudstack-dev/
>>> 201703.mbox/%3c596136829.12220.1489142173...@ox.pcextreme.nl%3E
>>> [2]  https://github.com/apache/cloudstack/pull/1582
>>> [3] https://github.com/apache/cloudstack/commit/70ef0788c932
>>> f4de1060fd60025ce120f7da5be4
>>>
>>>
>>
>

Re: [NOTICE] Impending shutdown of download.cloud.com - this may affect your CloudStack installation

[Chiradeep Vittal]

I have not heard any complaints, although from the access logs I see lots
of attempts to download legacy templates.
download.cloud.com will remain offline until April 30.
On April 30 2018, download.cloud.com will be decommissioned permanently.

On Fri, Mar 30, 2018 at 12:37 PM, Chiradeep Vittal <chirade...@gmail.com>
wrote:

> download.cloud.com will go offline temporarily for a few days. If you
> need access to it, check the instructions below. If that doesn't work,
> reply on this thread.
>
> On Tue, Mar 20, 2018 at 11:44 AM, Chiradeep Vittal <chirade...@gmail.com>
> wrote:
>
>> On April 30, 2018, the domain download.cloud.com will be decommissioned.
>>
>> THIS MAY AFFECT YOUR CLOUDSTACK INSTALLATION. PLEASE READ FURTHER.
>>
>> BACKGROUND
>> download.cloud.com was used to host the seed templates (images) for a
>> CloudStack installation. This included the system vm templates for releases
>> prior to 4.10. Around March 2017[1], a new site was established (
>> download.cloudstack.org) and all the content (templates) was copied over
>> to the new site. From release 4.10 onwards[2][3], the seed templates are
>> fetched / downloaded from the new site during an installation of Apache
>> CloudStack.
>>
>>
>> To check if your CloudStack installation has a reference to the old site (
>> download.cloud.com), check your vm_template table:
>>
>> SELECT url FROM vm_template;
>>
>> WHO MAY BE AFFECTED
>> CloudStack operators installing a version of Apache CloudStack prior to
>> version 4.10 after the shutdown. An already running/functioning CloudStack
>> cloud will continue to operate with no disruption after the shutdown.
>> However, operations such as: adding a zone, backup and restore, secondary
>> storage backup/restore, etc may be affected.
>>
>> STEPS TO BE TAKEN TO AVOID IMPACT
>> Update the vm_template table to point to the new location:
>>
>> UPDATE vm_template SET url = REPLACE(url, 'download.cloud.com', '
>> download.cloudstack.org');
>>
>> TRIAL RUN OF THE SHUTDOWN
>> During the last week of March 2018, download.cloud.com will be taken
>> temporarily offline. This is to make sure that CloudStack operators who
>> have not paid attention to this notice but are affected, will pay attention.
>>
>> This notice will be posted periodically until the shutdown.
>> --
>> Chiradeep
>>
>> [1] http://mail-archives.apache.org/mod_mbox/cloudstack-dev/
>> 201703.mbox/%3c596136829.12220.1489142173...@ox.pcextreme.nl%3E
>> [2]  https://github.com/apache/cloudstack/pull/1582
>> [3] https://github.com/apache/cloudstack/commit/70ef0788c932
>> f4de1060fd60025ce120f7da5be4
>>
>>
>

Re: [NOTICE] Impending shutdown of download.cloud.com - this may affect your CloudStack installation

[Chiradeep Vittal]

download.cloud.com will go offline temporarily for a few days. If you need
access to it, check the instructions below. If that doesn't work, reply on
this thread.

On Tue, Mar 20, 2018 at 11:44 AM, Chiradeep Vittal <chirade...@gmail.com>
wrote:

> On April 30, 2018, the domain download.cloud.com will be decommissioned.
>
> THIS MAY AFFECT YOUR CLOUDSTACK INSTALLATION. PLEASE READ FURTHER.
>
> BACKGROUND
> download.cloud.com was used to host the seed templates (images) for a
> CloudStack installation. This included the system vm templates for releases
> prior to 4.10. Around March 2017[1], a new site was established (
> download.cloudstack.org) and all the content (templates) was copied over
> to the new site. From release 4.10 onwards[2][3], the seed templates are
> fetched / downloaded from the new site during an installation of Apache
> CloudStack.
>
>
> To check if your CloudStack installation has a reference to the old site (
> download.cloud.com), check your vm_template table:
>
> SELECT url FROM vm_template;
>
> WHO MAY BE AFFECTED
> CloudStack operators installing a version of Apache CloudStack prior to
> version 4.10 after the shutdown. An already running/functioning CloudStack
> cloud will continue to operate with no disruption after the shutdown.
> However, operations such as: adding a zone, backup and restore, secondary
> storage backup/restore, etc may be affected.
>
> STEPS TO BE TAKEN TO AVOID IMPACT
> Update the vm_template table to point to the new location:
>
> UPDATE vm_template SET url = REPLACE(url, 'download.cloud.com', '
> download.cloudstack.org');
>
> TRIAL RUN OF THE SHUTDOWN
> During the last week of March 2018, download.cloud.com will be taken
> temporarily offline. This is to make sure that CloudStack operators who
> have not paid attention to this notice but are affected, will pay attention.
>
> This notice will be posted periodically until the shutdown.
> --
> Chiradeep
>
> [1] http://mail-archives.apache.org/mod_mbox/cloudstack-dev/201703.mbox/%
> 3c596136829.12220.1489142173...@ox.pcextreme.nl%3E
> [2]  https://github.com/apache/cloudstack/pull/1582
> [3] https://github.com/apache/cloudstack/commit/70ef0788c932
> f4de1060fd60025ce120f7da5be4
>
>

[NOTICE] Impending shutdown of download.cloud.com - this may affect your CloudStack installation

[Chiradeep Vittal]

On April 30, 2018, the domain download.cloud.com will be decommissioned.

THIS MAY AFFECT YOUR CLOUDSTACK INSTALLATION. PLEASE READ FURTHER.

BACKGROUND
download.cloud.com was used to host the seed templates (images) for a
CloudStack installation. This included the system vm templates for releases
prior to 4.10. Around March 2017[1], a new site was established (
download.cloudstack.org) and all the content (templates) was copied over to
the new site. From release 4.10 onwards[2][3], the seed templates are
fetched / downloaded from the new site during an installation of Apache
CloudStack.


To check if your CloudStack installation has a reference to the old site (
download.cloud.com), check your vm_template table:

SELECT url FROM vm_template;

WHO MAY BE AFFECTED
CloudStack operators installing a version of Apache CloudStack prior to
version 4.10 after the shutdown. An already running/functioning CloudStack
cloud will continue to operate with no disruption after the shutdown.
However, operations such as: adding a zone, backup and restore, secondary
storage backup/restore, etc may be affected.

STEPS TO BE TAKEN TO AVOID IMPACT
Update the vm_template table to point to the new location:

UPDATE vm_template SET url = REPLACE(url, 'download.cloud.com', '
download.cloudstack.org');

TRIAL RUN OF THE SHUTDOWN
During the last week of March 2018, download.cloud.com will be taken
temporarily offline. This is to make sure that CloudStack operators who
have not paid attention to this notice but are affected, will pay attention.

This notice will be posted periodically until the shutdown.
--
Chiradeep

[1] http://mail-archives.apache.org/mod_mbox/cloudstack-dev/
201703.mbox/%3c596136829.12220.1489142173...@ox.pcextreme.nl%3E
[2]  https://github.com/apache/cloudstack/pull/1582
[3] https://github.com/apache/cloudstack/commit/
70ef0788c932f4de1060fd60025ce120f7da5be4

Re: [IMPENDING SHUTDOWN] Re: Replacing download.cloud.com by download.cloudstack.org

[Chiradeep Vittal]

You are right, found a few more locations:
http://docs.cloudstack.apache.org/en/latest/ansible.html
http://docs.cloudstack.apache.org/projects/cloudstack-release-notes/en/4.10/upgrade/upgrade-4.9.html


There's blogs and books (!) that still refer to download.cloud.com but
nothing we can do about it.

On Wed, Feb 28, 2018 at 2:55 PM, Sean Lair <sl...@ippathways.com> wrote:

> Looks like it is still referenced here:
>
> http://docs.cloudstack.apache.org/projects/cloudstack-
> administration/en/4.11/templates/_password.html
>
>
>
> -Original Message-
> From: Chiradeep Vittal [mailto:chirade...@gmail.com]
> Sent: Tuesday, February 27, 2018 3:59 PM
> To: dev <dev@cloudstack.apache.org>
> Subject: Re: [IMPENDING SHUTDOWN] Re: Replacing download.cloud.com by
> download.cloudstack.org
>
> For the last 6 days, here are the stats (first column is downloads)
>
>2381 templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2
>
>  89 templates/4.5.1/systemvm64template-2015-05-14-4.5.1-xen.vhd.bz2
>
>  77 releases/2.2.0/eec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2
>
>  51 templates/4.3/systemvm64template-2014-04-10-master-xen.vhd.bz2
>
>  50 templates/acton/acton-systemvm-02062012.vhd.bz2
>
>  50 templates/4.2/64bit/systemvmtemplate64-2013-07-15-
> master-xen.vhd.bz2
>
>  34 templates/4.5/systemvm64template-2015-02-03-4.5.0-xen.vhd.bz2
>
>  25 templates/4.5.1/systemvm64template-2015-08-20-4.5.1-xen.vhd.bz2
>
>  24 templates/4.7/systemvm64template-2016-03-24-4.7.0-xen.vhd.bz2
>
>  24 templates/4.5/systemvm64template-4.5-kvm.qcow2.bz2
>
>  22 templates/4.2/systemvmtemplate-2013-06-12-master-kvm.qcow2.bz2
>
>  20 templates/4.3/systemvm64template-2014-09-30-4.3-xen.vhd.bz2
>
>  20 templates/4.2/systemvmtemplate-2013-07-12-master-xen.vhd.bz2
>
>  16 templates/4.3/systemvm64template-2014-01-14-master-xen.vhd.bz2
>
>  15 templates/builtin/centos56-x86_64.vhd.bz2
>
>  15 templates/4.5/systemvm64template-2014-12-18-4.5.0.0-xen.vhd.bz2
>
>  13 templates/4.3/systemvm64template-2015-02-04-4.3-xen.vhd.bz2
>
>   5 templates/4.3/systemvm64template-2014-06-23-master-kvm.qcow2.bz2
>
>   4 templates/4.5.1/systemvm64template-2016-04-15-4.5.1-xen.vhd.bz2
>
>   3 templates/4.3/systemvm64template-2014-06-23-master-xen.vhd.bz2
>
>   2 templates/4.3/systemvm64template-2015-02-04-4.3-kvm.qcow2.bz2
>
>   1 templates/acton/acton-systemvm-02062012.qcow2.bz2
>
>   1 releases/4.3/centos6_4_64bit.vhd.bz2
>
>
> Here's a list of the top IPs downloading. If you recognize yourself here
> (the top one is from one location in Montreal), please fix:
>
>2370 216.113.73.34
>
> 220 175.107.195.22
>
>  60 66.165.176.60
>
>  42 180.222.191.150
>
>  38 82.192.93.187
>
>  16 60.27.95.237
>
>  16 200.124.137.20
>
>  10 216.55.171.4
>
>  10 130.185.128.25
>
>   9 81.142.101.129
>
>   9 211.125.79.4
>
>   8 193.144.82.46
>
>   7 217.192.89.130
>
>   7 199.115.112.53
>
>   6 5.152.164.8
>
>   6 121.15.182.138
>
>   5 91.223.182.11
>
>   5 69.26.35.120
>
>   5 219.163.55.73
>
>   5 194.19.236.162
>
>   5 178.16.163.70
>
>   5 14.139.116.2
>
>   4 84.33.37.2
>
>   4 74.84.196.150
>
>   4 185.53.31.146
>
>   4 12.129.245.254
>
>   4 119.31.171.20
>
>   2 61.50.103.158
>
>   2 58.210.242.134
>
>   2 58.140.89.62
>
>   2 41.77.158.254
>
>   2 222.80.81.132
>
>   2 218.104.96.139
>
>   2 178.170.92.5
>
>   2 177.47.20.58
>
>   2 118.70.146.225
>
>   2 115.249.104.17
>
>   2 112.217.184.196
>
>   2 112.217.184.195
>
>   2 111.198.74.125
>
>   2 103.4.132.9
>
> On Tue, Feb 27, 2018 at 11:22 AM, Wido den Hollander <w...@widodh.nl>
> wrote:
>
> > Yes! Sounds good to me
> >
> > > Op 27 feb. 2018 om 20:02 heeft Rohit Yadav
> > > <rohit.ya...@shapeblue.com>
> > het volgende geschreven:
> > >
> > > Sounds good to me.
> > >
> > >
> > > - Rohit
> > >
> > > 
> > > From: Chiradeep Vittal <chirade...@gmail.com>
> > > Sent: Tuesday, February 27, 2018 18:33
> > > To: dev
> > > Subject: [IMPENDING SHUTDOWN] Re: Replacing download.cloud.com by
> > download.cloudstack.org
> > >
> > > I would like to propose a shutdown window of 60 days starting March 1.
>

[NOTICE] Impending shutdown of download.cloud.com - this may affect your CloudStack installation

[Chiradeep Vittal]

On April 30, 2018, the domain download.cloud.com will be decommissioned.

THIS MAY AFFECT YOUR CLOUDSTACK INSTALLATION. PLEASE READ FURTHER.

BACKGROUND
download.cloud.com was used to host the seed templates (images) for a
CloudStack installation. This included the system vm templates for releases
prior to 4.10. Around March 2017[1], a new site was established (
download.cloudstack.org) and all the content (templates) was copied over to
the new site. From release 4.10 onwards[2][3], the seed templates are
fetched / downloaded from the new site during an installation of Apache
CloudStack.


To check if your CloudStack installation has a reference to the old site (
download.cloud.com), check your vm_template table:

SELECT url FROM vm_template;

WHO MAY BE AFFECTED
CloudStack operators installing a version of Apache CloudStack prior to
version 4.10 after the shutdown. An already running/functioning CloudStack
cloud will continue to operate with no disruption after the shutdown.
However, operations such as: adding a zone, backup and restore, secondary
storage backup/restore, etc may be affected.

STEPS TO BE TAKEN TO AVOID IMPACT
Update the vm_template table to point to the new location:

UPDATE vm_template SET url = REPLACE(url, 'download.cloud.com', '
download.cloudstack.org');

TRIAL RUN OF THE SHUTDOWN
During the last week of March 2018, download.cloud.com will be taken
temporarily offline. This is to make sure that CloudStack operators who
have not paid attention to this notice but are affected, will pay attention.

This notice will be posted periodically until the shutdown.
--
Chiradeep

[1]
http://mail-archives.apache.org/mod_mbox/cloudstack-dev/201703.mbox/%3c596136829.12220.1489142173...@ox.pcextreme.nl%3E
[2]  https://github.com/apache/cloudstack/pull/1582
[3]
https://github.com/apache/cloudstack/commit/70ef0788c932f4de1060fd60025ce120f7da5be4

Re: [IMPENDING SHUTDOWN] Re: Replacing download.cloud.com by download.cloudstack.org

[Chiradeep Vittal]

For the last 6 days, here are the stats (first column is downloads)

   2381 templates/4.3/systemvm64template-2014-01-14-master-kvm.qcow2.bz2

 89 templates/4.5.1/systemvm64template-2015-05-14-4.5.1-xen.vhd.bz2

 77 releases/2.2.0/eec2209b-9875-3c8d-92be-c001bd8a0faf.qcow2.bz2

 51 templates/4.3/systemvm64template-2014-04-10-master-xen.vhd.bz2

 50 templates/acton/acton-systemvm-02062012.vhd.bz2

 50 templates/4.2/64bit/systemvmtemplate64-2013-07-15-master-xen.vhd.bz2

 34 templates/4.5/systemvm64template-2015-02-03-4.5.0-xen.vhd.bz2

 25 templates/4.5.1/systemvm64template-2015-08-20-4.5.1-xen.vhd.bz2

 24 templates/4.7/systemvm64template-2016-03-24-4.7.0-xen.vhd.bz2

 24 templates/4.5/systemvm64template-4.5-kvm.qcow2.bz2

 22 templates/4.2/systemvmtemplate-2013-06-12-master-kvm.qcow2.bz2

 20 templates/4.3/systemvm64template-2014-09-30-4.3-xen.vhd.bz2

 20 templates/4.2/systemvmtemplate-2013-07-12-master-xen.vhd.bz2

 16 templates/4.3/systemvm64template-2014-01-14-master-xen.vhd.bz2

 15 templates/builtin/centos56-x86_64.vhd.bz2

 15 templates/4.5/systemvm64template-2014-12-18-4.5.0.0-xen.vhd.bz2

 13 templates/4.3/systemvm64template-2015-02-04-4.3-xen.vhd.bz2

  5 templates/4.3/systemvm64template-2014-06-23-master-kvm.qcow2.bz2

  4 templates/4.5.1/systemvm64template-2016-04-15-4.5.1-xen.vhd.bz2

  3 templates/4.3/systemvm64template-2014-06-23-master-xen.vhd.bz2

  2 templates/4.3/systemvm64template-2015-02-04-4.3-kvm.qcow2.bz2

  1 templates/acton/acton-systemvm-02062012.qcow2.bz2

  1 releases/4.3/centos6_4_64bit.vhd.bz2


Here's a list of the top IPs downloading. If you recognize yourself here
(the top one is from one location in Montreal), please fix:

   2370 216.113.73.34

220 175.107.195.22

 60 66.165.176.60

 42 180.222.191.150

 38 82.192.93.187

 16 60.27.95.237

 16 200.124.137.20

 10 216.55.171.4

 10 130.185.128.25

  9 81.142.101.129

  9 211.125.79.4

  8 193.144.82.46

  7 217.192.89.130

  7 199.115.112.53

  6 5.152.164.8

  6 121.15.182.138

  5 91.223.182.11

  5 69.26.35.120

  5 219.163.55.73

  5 194.19.236.162

  5 178.16.163.70

  5 14.139.116.2

  4 84.33.37.2

  4 74.84.196.150

  4 185.53.31.146

  4 12.129.245.254

  4 119.31.171.20

  2 61.50.103.158

  2 58.210.242.134

  2 58.140.89.62

  2 41.77.158.254

  2 222.80.81.132

  2 218.104.96.139

  2 178.170.92.5

  2 177.47.20.58

  2 118.70.146.225

  2 115.249.104.17

  2 112.217.184.196

  2 112.217.184.195

  2 111.198.74.125

  2 103.4.132.9

On Tue, Feb 27, 2018 at 11:22 AM, Wido den Hollander <w...@widodh.nl> wrote:

> Yes! Sounds good to me
>
> > Op 27 feb. 2018 om 20:02 heeft Rohit Yadav <rohit.ya...@shapeblue.com>
> het volgende geschreven:
> >
> > Sounds good to me.
> >
> >
> > - Rohit
> >
> > 
> > From: Chiradeep Vittal <chirade...@gmail.com>
> > Sent: Tuesday, February 27, 2018 18:33
> > To: dev
> > Subject: [IMPENDING SHUTDOWN] Re: Replacing download.cloud.com by
> download.cloudstack.org
> >
> > I would like to propose a shutdown window of 60 days starting March 1.
> > Every few days (3?) we can announce the impending shutdown on users@
> > and dev@.
> > At day 30 we can temporarily disable (using S3 permissions)
> > download.cloud.com for 72 hours to see if this any impact.
> >
> > Thoughts?
> >
> >
> > rohit.ya...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >> On Mon, Feb 26, 2018 at 4:26 AM, Wido den Hollander <w...@widodh.nl>
> wrote:
> >>
> >>
> >>> On 02/25/2018 12:11 AM, Chiradeep Vittal wrote:
> >>>
> >>> Which releases still refer to them in the setup SQL?
> >>>
> >>
> >> I think it was 4.9? Does anybody know this exactly?
> >>
> >> Wido
> >>
> >>
> >>> Sent from my iPhone
> >>>
> >>>> On Feb 24, 2018, at 12:31 PM, Wido den Hollander <w...@widodh.nl>
> wrote:
> >>>>
> >>>>
> >>>>
> >>>>> On 02/24/2018 12:14 AM, Chiradeep Vittal wrote:
> >>>>> Citrix is wondering if they can shut down download.cloud.com at this
> >>>>> point.
> >>>>
> >>>>
> >>>> Grepping through the source I see no references to download.cloud.com
> >>>> anymore.
> >>>>
> >>>> I *think* it can be shut

[IMPENDING SHUTDOWN] Re: Replacing download.cloud.com by download.cloudstack.org

[Chiradeep Vittal]

I would like to propose a shutdown window of 60 days starting March 1.
Every few days (3?) we can announce the impending shutdown on users@
and dev@.
At day 30 we can temporarily disable (using S3 permissions)
download.cloud.com for 72 hours to see if this any impact.

Thoughts?

On Mon, Feb 26, 2018 at 4:26 AM, Wido den Hollander <w...@widodh.nl> wrote:
>
>
> On 02/25/2018 12:11 AM, Chiradeep Vittal wrote:
>>
>> Which releases still refer to them in the setup SQL?
>>
>
> I think it was 4.9? Does anybody know this exactly?
>
> Wido
>
>
>> Sent from my iPhone
>>
>>> On Feb 24, 2018, at 12:31 PM, Wido den Hollander <w...@widodh.nl> wrote:
>>>
>>>
>>>
>>>> On 02/24/2018 12:14 AM, Chiradeep Vittal wrote:
>>>> Citrix is wondering if they can shut down download.cloud.com at this
>>>> point.
>>>
>>>
>>> Grepping through the source I see no references to download.cloud.com
>>> anymore.
>>>
>>> I *think* it can be shut down. Any other opinions on this?
>>>
>>> Wido
>>>
>>>> Thanks!
>>>>>
>>>>> On Fri, Mar 10, 2017 at 2:39 AM, Wido den Hollander <w...@widodh.nl>
>>>>> wrote:
>>>>>
>>>>>> Op 10 maart 2017 om 11:36 schreef Wido den Hollander <w...@widodh.nl>:
>>>>>>
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Will Stevens and I have been working on a replacement for
>>>>>> download.cloud.com.
>>>>>>
>>>>>> This resulted in download.cloudstack.org which is also available over
>>>>>> SSL!
>>>>>>
>>>>>> download.cloudstack.org is a CNAME to cloudstack.apt-get.eu and is
>>>>>> hosted by me in Amsterdam (on CloudStack and Ceph).
>>>>>>
>>>>>> In the future I'd like to scale this out to more systems on different
>>>>>> providers by using RR-DNS, but for now let's not make it very difficult.
>>>>>>
>>>>>> There is a PR open [0] to start to do this, but we also have
>>>>>> documentation which might need to be fixed.
>>>>>>
>>>>>> RPM and DEB packages are also available on download.cloudstack.org:
>>>>>>
>>>>>> - RPM: https://download.cloudstack.org/centos
>>>>>> - DEB: https://download.cloudstack.org/ubuntu
>>>>>>
>>>>>> SystemVMs are also available:
>>>>>> https://download.cloudstack.org/systemvm/
>>>>>>
>>>>>> In total there is 126GB of data on the server:
>>>>>>
>>>>>> 0 centos7
>>>>>> 4.0K  README
>>>>>> 4.0K  release.asc
>>>>>> 4.0K  RPM-GPG-KEY
>>>>>> 12K   foss
>>>>>> 316K  tools
>>>>>> 629M  tmp
>>>>>> 2.9G  archive
>>>>>> 3.7G  rhel
>>>>>> 5.3G  releases
>>>>>> 8.3G  centos
>>>>>> 19G   ubuntu
>>>>>> 24G   systemvm
>>>>>> 64G   templates
>>>>>>
>>>>>> If you want to be able to add files to the system, send me your SSH
>>>>>> key and we can fix this. A few PMC members already have access and can 
>>>>>> add
>>>>>> files.
>>>>>>
>>>>>> Any things which we need to fix in addition?
>>>>>>
>>>>>
>>>>> Forgot to add, the system already has rsync available, so you can sync
>>>>> it if you want:
>>>>>
>>>>> $ rsync -avr --stats --progress download.cloudstack.org::cloudstack .
>>>>>
>>>>> Might want to do that already so that we can quickly move to a
>>>>> different server should the one in Amsterdam be down for a long time.
>>>>>
>>>>>> Wido
>>>>>>
>>>>>> [0]: https://github.com/apache/cloudstack/pull/1582

Re: Replacing download.cloud.com by download.cloudstack.org

[Chiradeep Vittal]

Which releases still refer to them in the setup SQL?

Sent from my iPhone

> On Feb 24, 2018, at 12:31 PM, Wido den Hollander <w...@widodh.nl> wrote:
> 
> 
> 
>> On 02/24/2018 12:14 AM, Chiradeep Vittal wrote:
>> Citrix is wondering if they can shut down download.cloud.com at this point.
> 
> Grepping through the source I see no references to download.cloud.com anymore.
> 
> I *think* it can be shut down. Any other opinions on this?
> 
> Wido
> 
>> Thanks!
>>> On Fri, Mar 10, 2017 at 2:39 AM, Wido den Hollander <w...@widodh.nl> wrote:
>>> 
>>>> Op 10 maart 2017 om 11:36 schreef Wido den Hollander <w...@widodh.nl>:
>>>> 
>>>> 
>>>> Hi,
>>>> 
>>>> Will Stevens and I have been working on a replacement for 
>>>> download.cloud.com.
>>>> 
>>>> This resulted in download.cloudstack.org which is also available over SSL!
>>>> 
>>>> download.cloudstack.org is a CNAME to cloudstack.apt-get.eu and is hosted 
>>>> by me in Amsterdam (on CloudStack and Ceph).
>>>> 
>>>> In the future I'd like to scale this out to more systems on different 
>>>> providers by using RR-DNS, but for now let's not make it very difficult.
>>>> 
>>>> There is a PR open [0] to start to do this, but we also have documentation 
>>>> which might need to be fixed.
>>>> 
>>>> RPM and DEB packages are also available on download.cloudstack.org:
>>>> 
>>>> - RPM: https://download.cloudstack.org/centos
>>>> - DEB: https://download.cloudstack.org/ubuntu
>>>> 
>>>> SystemVMs are also available: https://download.cloudstack.org/systemvm/
>>>> 
>>>> In total there is 126GB of data on the server:
>>>> 
>>>> 0 centos7
>>>> 4.0K  README
>>>> 4.0K  release.asc
>>>> 4.0K  RPM-GPG-KEY
>>>> 12K   foss
>>>> 316K  tools
>>>> 629M  tmp
>>>> 2.9G  archive
>>>> 3.7G  rhel
>>>> 5.3G  releases
>>>> 8.3G  centos
>>>> 19G   ubuntu
>>>> 24G   systemvm
>>>> 64G   templates
>>>> 
>>>> If you want to be able to add files to the system, send me your SSH key 
>>>> and we can fix this. A few PMC members already have access and can add 
>>>> files.
>>>> 
>>>> Any things which we need to fix in addition?
>>>> 
>>> 
>>> Forgot to add, the system already has rsync available, so you can sync it 
>>> if you want:
>>> 
>>> $ rsync -avr --stats --progress download.cloudstack.org::cloudstack .
>>> 
>>> Might want to do that already so that we can quickly move to a different 
>>> server should the one in Amsterdam be down for a long time.
>>> 
>>>> Wido
>>>> 
>>>> [0]: https://github.com/apache/cloudstack/pull/1582

Re: Replacing download.cloud.com by download.cloudstack.org

[Chiradeep Vittal]

Citrix is wondering if they can shut down download.cloud.com at this point.

Thanks!

On Fri, Mar 10, 2017 at 2:39 AM, Wido den Hollander  wrote:
>
>> Op 10 maart 2017 om 11:36 schreef Wido den Hollander :
>>
>>
>> Hi,
>>
>> Will Stevens and I have been working on a replacement for download.cloud.com.
>>
>> This resulted in download.cloudstack.org which is also available over SSL!
>>
>> download.cloudstack.org is a CNAME to cloudstack.apt-get.eu and is hosted by 
>> me in Amsterdam (on CloudStack and Ceph).
>>
>> In the future I'd like to scale this out to more systems on different 
>> providers by using RR-DNS, but for now let's not make it very difficult.
>>
>> There is a PR open [0] to start to do this, but we also have documentation 
>> which might need to be fixed.
>>
>> RPM and DEB packages are also available on download.cloudstack.org:
>>
>> - RPM: https://download.cloudstack.org/centos
>> - DEB: https://download.cloudstack.org/ubuntu
>>
>> SystemVMs are also available: https://download.cloudstack.org/systemvm/
>>
>> In total there is 126GB of data on the server:
>>
>> 0 centos7
>> 4.0K  README
>> 4.0K  release.asc
>> 4.0K  RPM-GPG-KEY
>> 12K   foss
>> 316K  tools
>> 629M  tmp
>> 2.9G  archive
>> 3.7G  rhel
>> 5.3G  releases
>> 8.3G  centos
>> 19G   ubuntu
>> 24G   systemvm
>> 64G   templates
>>
>> If you want to be able to add files to the system, send me your SSH key and 
>> we can fix this. A few PMC members already have access and can add files.
>>
>> Any things which we need to fix in addition?
>>
>
> Forgot to add, the system already has rsync available, so you can sync it if 
> you want:
>
> $ rsync -avr --stats --progress download.cloudstack.org::cloudstack .
>
> Might want to do that already so that we can quickly move to a different 
> server should the one in Amsterdam be down for a long time.
>
>> Wido
>>
>> [0]: https://github.com/apache/cloudstack/pull/1582

Re: Fail with vpn customer gateway creation through terraform

[Chiradeep Vittal]

Check
https://github.com/apache/cloudstack/blob/77864992fe8f80dbabd1240f6373d2ba3e98713c/utils/src/main/java/com/cloud/utils/net/NetUtils.java#L1221

On Tue, Nov 21, 2017 at 10:11 AM, Nux!  wrote:

> Hi,
>
> I'm trying out terraform and had success so far, except for the vpn
> customer gateway feature.
> For some reason, terraform fails to create it, though I use the same
> options as in UI/cloudmonkey where it works just fine.
>
> The snippet for it is:
>
> resource "cloudstack_vpn_customer_gateway" "default" {
>   name   = "test-vpc"
>   cidr   = "10.0.0.0/24"
>   esp_policy = "aes256-sha1"
>   gateway= "1.2.3.4"
>   ike_policy = "sha1-aes256;modp3072"
>   ipsec_psk  = "terraformxyz7"
> }
>
> It always complains about the ike_policy:
> * cloudstack_vpn_customer_gateway.default: Error creating VPN Customer
> Gateway test-vpc: Undefined error: {"errorcode":431,"errortext":"The
> customer gateway IKE policy sha1-aes256;modp3072 is invalid!  Verify the
> required Diffie Hellman (DH) group is specified."}
>
> I tried all sorts of ways to write the ike_policy, escaped, web
> encoded/decoded, nothing worked. What am I missing?
> The example terraform docs provide suffers the same fate.
>
> Lucian
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>

Re: Modern template hosting

[Chiradeep Vittal]

To me, that is like saying that a customer won't buy a car because he can't
figure out how to roll down his window, so let's build a voice-controlled
window controller. Can be done, sounds great, but what happens when that
voice controller thing in the cloud goes down?


On Fri, Mar 3, 2017 at 3:09 PM, Will Stevens <williamstev...@gmail.com>
wrote:

> I agree with Paul. Look at the list of things they have to
> learn/master/care about. We don't want to add to that list, we want to
> remove from that list. Make the project/product more accessible. Reduce the
> barrier to entry...
>
> On Mar 3, 2017 5:50 PM, "Chiradeep Vittal" <chirade...@gmail.com> wrote:
>
> > Seriously?
> > apt-get install apache2
> >
> > To install CloudStack, they need to know
> > - DB installation, perhaps some SQL
> > - VLAN configuration on their switches
> > - Ins-and-outs between port forwarding, static ips,
> > - NFS
> > - package management
> > - VHDs, qcow2. vmdk
> > - hyperviosrs
> > - and on and on.
> >
> > If they can figure all that out and not figure out a web server, they can
> > always come to the mailing list.
> >
> >
> >
> >
> > On Fri, Mar 3, 2017 at 1:45 PM, Paul Angus <paul.an...@shapeblue.com>
> > wrote:
> >
> > > The issue is not with supporting highly experienced cloud operators. It
> > is
> > > an issue for new users and other 'relatively' inexperienced operators.
> > >
> > > I have helped enough newbies and cloud operators who have been running
> > > their cloud for a while to know that the barriers to entry are too high
> > as
> > > they are. And telling anyone that they need to create a web server so
> > that
> > > they can add their initial template to get started or in order to
> create
> > a
> > > new zone, just isn't going to fly.
> > >
> > > I'm a huge advocate of the 'download.cloudstack.org' endpoint which
> the
> > > community can add/remove mirrors to or from for system VMs or built-in
> > > templates.  Can the same system be used for binary repos ? although
> there
> > > is an added complication of redist vs no-redist there...
> > >
> > >
> > >
> > >
> > >
> > >
> > > paul.an...@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > > @shapeblue
> > >
> > >
> > >
> > >
> > > -Original Message-
> > > From: Chiradeep Vittal [mailto:chirade...@gmail.com]
> > > Sent: 03 March 2017 18:28
> > > To: dev <dev@cloudstack.apache.org>
> > > Subject: Re: Modern template hosting
> > >
> > > I do feel like this is early optimization. Mirrors rarely fail. I'd
> > expect
> > > a single web server hosted on Apache Infra without any monitors to fail
> > > more often than a mirror. We already expect Wido's systemvm repository
> to
> > > be up all the time. And it has been. Similarly, I don't believe Nux's
> > > repository has ever been down. And if Accelerite wants to host on S3,
> > that
> > > one is pretty solid as well.
> > >
> > > This is an infrequent operation in a cloud. After the cloud is
> installed,
> > > the download servers are only needed for a new zone. If we trust the
> user
> > > to run a cloud, surely he/she can run a web server to serve some
> built-in
> > > templates. And if her cloud is successful, she needs to figure out how
> to
> > > host her templates anyway and not rely on 3rd parties.
> > >
> > >
> > > On Fri, Mar 3, 2017 at 9:31 AM, Will Stevens <wstev...@cloudops.com>
> > > wrote:
> > >
> > > > 1) If the legacy implementations do not support redirects, that does
> > > > cause a problem.  A potential solution in that case is to have the
> web
> > > > server actually proxy the download, but that is not ideal and I would
> > > > like to avoid it if possible.  Thanks for bringing that up Chiradeep.
> > > >
> > > > 2) I think we need to have a single URL which people can target.
> Once
> > > > they make the switch to the new URL, we want the implementation to be
> > > > able to handle mirror failures without affecting the end client.  We
> > > > want to avoid the situation where an ACS user will ever have to
> change
> > > > this URL more than once.  Mirror failures SHOULD NOT affect the ACS
> > > > users assuming

Re: Modern template hosting

[Chiradeep Vittal]

Seriously?
apt-get install apache2

To install CloudStack, they need to know
- DB installation, perhaps some SQL
- VLAN configuration on their switches
- Ins-and-outs between port forwarding, static ips,
- NFS
- package management
- VHDs, qcow2. vmdk
- hyperviosrs
- and on and on.

If they can figure all that out and not figure out a web server, they can
always come to the mailing list.




On Fri, Mar 3, 2017 at 1:45 PM, Paul Angus <paul.an...@shapeblue.com> wrote:

> The issue is not with supporting highly experienced cloud operators. It is
> an issue for new users and other 'relatively' inexperienced operators.
>
> I have helped enough newbies and cloud operators who have been running
> their cloud for a while to know that the barriers to entry are too high as
> they are. And telling anyone that they need to create a web server so that
> they can add their initial template to get started or in order to create a
> new zone, just isn't going to fly.
>
> I'm a huge advocate of the 'download.cloudstack.org' endpoint which the
> community can add/remove mirrors to or from for system VMs or built-in
> templates.  Can the same system be used for binary repos ? although there
> is an added complication of redist vs no-redist there...
>
>
>
>
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Chiradeep Vittal [mailto:chirade...@gmail.com]
> Sent: 03 March 2017 18:28
> To: dev <dev@cloudstack.apache.org>
> Subject: Re: Modern template hosting
>
> I do feel like this is early optimization. Mirrors rarely fail. I'd expect
> a single web server hosted on Apache Infra without any monitors to fail
> more often than a mirror. We already expect Wido's systemvm repository to
> be up all the time. And it has been. Similarly, I don't believe Nux's
> repository has ever been down. And if Accelerite wants to host on S3, that
> one is pretty solid as well.
>
> This is an infrequent operation in a cloud. After the cloud is installed,
> the download servers are only needed for a new zone. If we trust the user
> to run a cloud, surely he/she can run a web server to serve some built-in
> templates. And if her cloud is successful, she needs to figure out how to
> host her templates anyway and not rely on 3rd parties.
>
>
> On Fri, Mar 3, 2017 at 9:31 AM, Will Stevens <wstev...@cloudops.com>
> wrote:
>
> > 1) If the legacy implementations do not support redirects, that does
> > cause a problem.  A potential solution in that case is to have the web
> > server actually proxy the download, but that is not ideal and I would
> > like to avoid it if possible.  Thanks for bringing that up Chiradeep.
> >
> > 2) I think we need to have a single URL which people can target.  Once
> > they make the switch to the new URL, we want the implementation to be
> > able to handle mirror failures without affecting the end client.  We
> > want to avoid the situation where an ACS user will ever have to change
> > this URL more than once.  Mirror failures SHOULD NOT affect the ACS
> > users assuming there is still at least one mirror who can serve the
> requested resource.
> >
> > These are obviously my personal opinions and others will probably have
> > differing opinions.
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > <https://goo.gl/NYZ8KK>
> >
> > On Fri, Mar 3, 2017 at 12:23 PM, Chiradeep Vittal
> > <chirade...@gmail.com>
> > wrote:
> >
> > > 1. If you are targeting legacy installations, they are not able to
> > > follow redirects. The line of code that added this capability was
> > > added on
> > 11/16.
> > > 2. If you trust the users to edit the database to change the URL,
> > > you can trust them to change it to anything. Just document a known
> > > list of good template locations.
> > >
> > > On Fri, Mar 3, 2017 at 8:13 AM, Will Stevens <wstev...@cloudops.com>
> > > wrote:
> > >
> > > > So the main issue I see with this is the following.
> > > >
> > > > This implementation is designed to target legacy installations
> > > > which
> > will
> > > > be affected when download.cloud.com disappears.  These people will
> > need
> > > to
> > > > make a DB change to replace the 'download.cloud.com' with some
> > > > other
> > url
> > > > (maybe 'download.cloudstack.org').  Once the DB has been updated,
> > > > we
> > can
> > > > not expect anything else of the client, th

Re: Modern template hosting

[Chiradeep Vittal]

I do feel like this is early optimization. Mirrors rarely fail. I'd expect
a single web server hosted on Apache Infra without any monitors to fail
more often than a mirror. We already expect Wido's systemvm repository to
be up all the time. And it has been. Similarly, I don't believe Nux's
repository has ever been down. And if Accelerite wants to host on S3, that
one is pretty solid as well.

This is an infrequent operation in a cloud. After the cloud is installed,
the download servers are only needed for a new zone. If we trust the user
to run a cloud, surely he/she can run a web server to serve some built-in
templates. And if her cloud is successful, she needs to figure out how to
host her templates anyway and not rely on 3rd parties.


On Fri, Mar 3, 2017 at 9:31 AM, Will Stevens <wstev...@cloudops.com> wrote:

> 1) If the legacy implementations do not support redirects, that does cause
> a problem.  A potential solution in that case is to have the web server
> actually proxy the download, but that is not ideal and I would like to
> avoid it if possible.  Thanks for bringing that up Chiradeep.
>
> 2) I think we need to have a single URL which people can target.  Once they
> make the switch to the new URL, we want the implementation to be able to
> handle mirror failures without affecting the end client.  We want to avoid
> the situation where an ACS user will ever have to change this URL more than
> once.  Mirror failures SHOULD NOT affect the ACS users assuming there is
> still at least one mirror who can serve the requested resource.
>
> These are obviously my personal opinions and others will probably have
> differing opinions.
>
> *Will STEVENS*
> Lead Developer
>
> <https://goo.gl/NYZ8KK>
>
> On Fri, Mar 3, 2017 at 12:23 PM, Chiradeep Vittal <chirade...@gmail.com>
> wrote:
>
> > 1. If you are targeting legacy installations, they are not able to follow
> > redirects. The line of code that added this capability was added on
> 11/16.
> > 2. If you trust the users to edit the database to change the URL, you can
> > trust them to change it to anything. Just document a known list of good
> > template locations.
> >
> > On Fri, Mar 3, 2017 at 8:13 AM, Will Stevens <wstev...@cloudops.com>
> > wrote:
> >
> > > So the main issue I see with this is the following.
> > >
> > > This implementation is designed to target legacy installations which
> will
> > > be affected when download.cloud.com disappears.  These people will
> need
> > to
> > > make a DB change to replace the 'download.cloud.com' with some other
> url
> > > (maybe 'download.cloudstack.org').  Once the DB has been updated, we
> can
> > > not expect anything else of the client, they need to be able to just
> > > continue operation as they were without any need to rebuild or upgrade
> > > their ACS.
> > >
> > > If we try to force the decision to the client, then we break the legacy
> > > implementations.  Since the SSVM is likely going to be the client in
> some
> > > cases, and since it does not already have the logic to handle the 300
> > > approach correctly, I feel like it is not a viable solution for the
> > legacy
> > > deployments.
> > >
> > > In addition to that.  I don't think the client is in any better
> position
> > to
> > > make the mirror decision than I am.  I am likely able to make a more
> > > 'educated' decision than the client would be able to because I can do
> > > different tests on the endpoint(s) before making a final decision.
> > >
> > > Does this make sense or am I missing something obvious?
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > <https://goo.gl/NYZ8KK>
> > >
> > > On Fri, Mar 3, 2017 at 10:48 AM, Rafael Weingärtner <
> > > rafaelweingart...@gmail.com> wrote:
> > >
> > > > I think I understood you, but I did not understand you.
> > > >
> > > > Let me see if I can create an example to illustrate how we could do
> > using
> > > > the HTTP 300 code.
> > > >
> > > > The HTTP 300 code indicates clients that he/she/it has multiple
> choices
> > > to
> > > > access the requested resource. If the request is a HEAD, this is the
> > only
> > > > thing that the server returns. If the request if a GET, then the
> server
> > > > returns the choices for the client. The response type is defined on
> the
> > > > server based on the Content-Type, user agent and others.
> > > >
> > >

Re: Modern template hosting

[Chiradeep Vittal]

t; > > >> Coming to urls hard coded in db, the popular suggestion is to
> > > point
> > > > them
> > > > >> to mirrors. This looks good but this is going to take some
> > effort
> > > > that has
> > > > >> not materialised yet. Once this change is made it will be
> going
> > in
> > > > some
> > > > >> future release like 4.11; the problem still remains for the
> > > > releases that
> > > > >> are already out there. In case access to download.cloud.com
> is
> > > > dropped
> > > > >> at some point we will have to document the procedure such that
> > > > people know
> > > > >> how to make these stable releases work. That is where the
> > > procedure
> > > > >> documented here https://shankerbalan.net/blog/
> > > > >> seed-cloudstack-templates-offline/ should be officially
> adapted
> > > > with a
> > > > >> marketplace to pick up templates of choosing. Maybe we can
> have
> > a
> > > > pre-setup
> > > > >> script to make choices instead of written wiki. Note that this
> > > will
> > > > be
> > > > >> anyway needed by the older releases.
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> On 01/03/17, 4:24 PM, "Paul Angus" <paul.an...@shapeblue.com>
> > > > wrote:
> > > > >>
> > > > >> >-1
> > > > >> >
> > > > >> >We are trying to get MORE people to use CloudStack.  And
> > raising
> > > > the
> > > > >> barrier to entry does nothing to help that.
> > > > >> >
> > > > >> >
> > > > >> >Kind regards,
> > > > >> >
> > > > >> >Paul Angus
> > > > >> >
> > > > >> >paul.an...@shapeblue.com
> > > > >> >www.shapeblue.com
> > > > >> >53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > > > >> >@shapeblue
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> >-Original Message-
> > > > >> >From: Abhinandan Prateek [mailto:abhinandan.prateek@
> > > shapeblue.com]
> > > > >> >Sent: 01 March 2017 09:39
> > > > >> >To: dev@cloudstack.apache.org
> > > > >> >Subject: Re: Modern template hosting
> > > > >> >
> > > > >> >+1
> > > > >> >
> > > > >> >After seeing this discussion running in circles several
> times,
> > I
> > > > think
> > > > >> we should at least get started with the simplest option.
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> >
> > > > >> >On 27/02/17, 11:54 PM, "Chiradeep Vittal" <
> > chirade...@gmail.com>
> > > > wrote:
> > > > >> >
> > > > >> >>The alternative foolproof way is to simply not provide the
> > > > automatic
> > > > >> >>download.
> > > > >> >>
> > > > >> >>Just document the mirror list URL where the templates can be
> > > > found. I
> > > > >> >>don't know why people reject this option.
> > > > >> >>
> > > > >> >>https://shankerbalan.net/blog/seed-cloudstack-
> > > templates-offline/
> > > > >> >>
> > > > >> >>On Mon, Feb 27, 2017 at 10:15 AM, Will Stevens
> > > > >> >><williamstev...@gmail.com>
> > > > >> >>wrote:
> > > > >> >>
> > > > >> >>> OK. Thanks for the heads up.
> > > > >> >>>
> > > > >> >>> On Feb 27, 2017 1:08 PM, "Chiradeep Vittal" <
> > > > chirade...@gmail.com>
> > > >

Re: Modern template hosting

[Chiradeep Vittal]

Yes, that's my preferred solution. But there's a couple of scenarios to be
dealt with:
 1. standard isolated networks where the VR is the default router
 2. basic zone where the VR is not the default router
 3. isolated networks where the VR is not the default router

For (2) and (3) you may have to send down a static route in the DHCP
response to point 169.254.169.254 to the VR (or do it in all cases,
shouldn't be a big deal).


On Tue, Feb 28, 2017 at 9:57 AM, Wido den Hollander <w...@widodh.nl> wrote:

>
> > Op 28 februari 2017 om 18:55 schreef Chiradeep Vittal <
> chirade...@gmail.com>:
> >
> >
> > To repeat what was discussed earlier in the thread: the cloud templates
> > provided by Ubuntu and CentOS do not work because they expect the
> metadata
> > URL to be at http://169.254.269.254/. Well, they work, but they take
> > awfully long to time out and after they time out, you still wouldn't  be
> > able to login since they do not install the ssh keys.
>
> But that is a 'simple fix' in the VR. Like you mentioned, add a DNAT rule
> in the VR and have the VR handle the traffic.
>
> Wido
>
> >
> > On Tue, Feb 28, 2017 at 9:34 AM, Wido den Hollander <w...@widodh.nl>
> wrote:
> >
> > >
> > > > Op 28 februari 2017 om 2:40 schreef Pierre-Luc Dion <
> pdion...@apache.org
> > > >:
> > > >
> > > >
> > > > hi!
> > > >
> > > > I think we should work with distro provider to have their cloud
> builds
> > > work
> > > > with cloudstack. Good example is CoreOS, it work out of the box from
> > > their
> > > > channel builds.
> > > > it shouldn't be too complicated to have centos, ubuntu and debian,
> unless
> > > > ...
> > >
> > > I fully agree. I think that we shouldn't bother about VM templates.
> CentOS
> > > and Ubuntu both have a proper template which we can include.
> > >
> > > The less we host as a project, the better.
> > >
> > > Keep in mind that we host a Open Source project. the RPM and DEB
> packages
> > > on cloudstack.apt-get.eu are a service, but we release source as a
> > > project.
> > >
> > > Wido
> > >
> > > >
> > > > For our systemvm templates, can we just change the URL for somthing
> like
> > > > cloudstack.apache.org/systemvm/... ?
> > > > and for old systemvm that are depricated but that we might want to
> keep,
> > > > could we archive them into a github repo in
> > > > https://github.com/apachecloudstack ?
> > > >
> > > >
> > > >
> > > >
> > > > On Mon, Feb 27, 2017 at 6:28 PM, Chiradeep Vittal <
> chirade...@gmail.com>
> > > > wrote:
> > > >
> > > > > My stance is that the current workflow does a disservice to the
> user
> > > > > community by letting them install / use outdated and insecure
> > > templates.
> > > > > Now, let's assume download.cloud.com is gone forever. What do we
> tell
> > > ACS
> > > > > users pre-4.11 as far as *built-in templates* go?
> > > > > 1. Direct them to update templates.sql with some new URL, but with
> the
> > > same
> > > > > dirty old templates
> > > > > 2. Direct them to update templates.sql with some new URL, but with
> nice
> > > > > templates (e.g., open.vm.eu)
> > > > > 3. Same as (2), but document more choices.
> > > > >
> > > > > Now, why should things be different for 4.11 and later?
> Documenting the
> > > > > steps to install templates offline is trivial (and can be scripted
> to a
> > > > > large part, like cloud-install-sys-tmplt)
> > > > >
> > > > > For pre-4.11 users, for *systemvms*, anyway we tell them to use
> > > > > http://cloudstack.apt-get.eu which is not controlled by ACS.
> > > > >
> > > > >
> > > > > On Mon, Feb 27, 2017 at 2:50 PM, Rafael Weingärtner <
> > > > > rafaelweingart...@gmail.com> wrote:
> > > > >
> > > > > > Agree with you.
> > > > > > We need to support the current working flow. And then, define the
> > > first
> > > > > > version that will start using the new approach.
> > > > > >
> > > > > > On Mon, Feb 27, 2017 at 5:36 PM, Will Stevens <
> wstev...@cloudops.com
> > > >
> > > 

Re: Modern template hosting

[Chiradeep Vittal]

To repeat what was discussed earlier in the thread: the cloud templates
provided by Ubuntu and CentOS do not work because they expect the metadata
URL to be at http://169.254.269.254/. Well, they work, but they take
awfully long to time out and after they time out, you still wouldn't  be
able to login since they do not install the ssh keys.

On Tue, Feb 28, 2017 at 9:34 AM, Wido den Hollander <w...@widodh.nl> wrote:

>
> > Op 28 februari 2017 om 2:40 schreef Pierre-Luc Dion <pdion...@apache.org
> >:
> >
> >
> > hi!
> >
> > I think we should work with distro provider to have their cloud builds
> work
> > with cloudstack. Good example is CoreOS, it work out of the box from
> their
> > channel builds.
> > it shouldn't be too complicated to have centos, ubuntu and debian, unless
> > ...
>
> I fully agree. I think that we shouldn't bother about VM templates. CentOS
> and Ubuntu both have a proper template which we can include.
>
> The less we host as a project, the better.
>
> Keep in mind that we host a Open Source project. the RPM and DEB packages
> on cloudstack.apt-get.eu are a service, but we release source as a
> project.
>
> Wido
>
> >
> > For our systemvm templates, can we just change the URL for somthing like
> > cloudstack.apache.org/systemvm/... ?
> > and for old systemvm that are depricated but that we might want to keep,
> > could we archive them into a github repo in
> > https://github.com/apachecloudstack ?
> >
> >
> >
> >
> > On Mon, Feb 27, 2017 at 6:28 PM, Chiradeep Vittal <chirade...@gmail.com>
> > wrote:
> >
> > > My stance is that the current workflow does a disservice to the user
> > > community by letting them install / use outdated and insecure
> templates.
> > > Now, let's assume download.cloud.com is gone forever. What do we tell
> ACS
> > > users pre-4.11 as far as *built-in templates* go?
> > > 1. Direct them to update templates.sql with some new URL, but with the
> same
> > > dirty old templates
> > > 2. Direct them to update templates.sql with some new URL, but with nice
> > > templates (e.g., open.vm.eu)
> > > 3. Same as (2), but document more choices.
> > >
> > > Now, why should things be different for 4.11 and later? Documenting the
> > > steps to install templates offline is trivial (and can be scripted to a
> > > large part, like cloud-install-sys-tmplt)
> > >
> > > For pre-4.11 users, for *systemvms*, anyway we tell them to use
> > > http://cloudstack.apt-get.eu which is not controlled by ACS.
> > >
> > >
> > > On Mon, Feb 27, 2017 at 2:50 PM, Rafael Weingärtner <
> > > rafaelweingart...@gmail.com> wrote:
> > >
> > > > Agree with you.
> > > > We need to support the current working flow. And then, define the
> first
> > > > version that will start using the new approach.
> > > >
> > > > On Mon, Feb 27, 2017 at 5:36 PM, Will Stevens <wstev...@cloudops.com
> >
> > > > wrote:
> > > >
> > > > > I think we almost need a two pronged approach.
> > > > >
> > > > > 1) Get a solution in place which will enable us to document and
> serve
> > > > > templates for legacy systems.  I will work on this.
> > > > > 2) Discuss and understand how we SHOULD be handling this problem
> in the
> > > > > future and in what release we can expect it.
> > > > >
> > > > > I think we need to do both.  I think we should start to try to
> really
> > > > > understand what we want to deliver in (2) going forward.
> > > > >
> > > > > *Will STEVENS*
> > > > > Lead Developer
> > > > >
> > > > > <https://goo.gl/NYZ8KK>
> > > > >
> > > > > On Mon, Feb 27, 2017 at 4:53 PM, Rafael Weingärtner <
> > > > > rafaelweingart...@gmail.com> wrote:
> > > > >
> > > > > > My worry is exactly with system VMs templates.
> > > > > >
> > > > > > Currently, we indicate administrators to download them from
> > > > > > http://cloudstack.apt-get.eu/systemvm/4.6/ [1]. However, the
> > > > > installation
> > > > > > docs do not mention the expected hashes for the file that is
> going to
> > > > be
> > > > > > downloaded.
> > > > > > Also, I do not know the code that downloads system VMs templates
> 

Re: Modern template hosting

[Chiradeep Vittal]

The cloud templates provided by Ubuntu and CentOS (Debian doesn't provide
one) can work out of the box if we support http://1692.254.169.254/ as a
metadata URL. I believe this is as simple as adding this IP to the VR (+ a
DNAT rule in the VR?) for advanced zone. For Basic Zone the default router
is not the VR, so additional DNAT rule will have to be added on the xenbr0
bridge.

On Mon, Feb 27, 2017 at 5:40 PM, Pierre-Luc Dion <pdion...@apache.org>
wrote:

> hi!
>
> I think we should work with distro provider to have their cloud builds work
> with cloudstack. Good example is CoreOS, it work out of the box from their
> channel builds.
> it shouldn't be too complicated to have centos, ubuntu and debian, unless
> ...
>
> For our systemvm templates, can we just change the URL for somthing like
> cloudstack.apache.org/systemvm/... ?
> and for old systemvm that are depricated but that we might want to keep,
> could we archive them into a github repo in
> https://github.com/apachecloudstack ?
>
>
>
>
> On Mon, Feb 27, 2017 at 6:28 PM, Chiradeep Vittal <chirade...@gmail.com>
> wrote:
>
> > My stance is that the current workflow does a disservice to the user
> > community by letting them install / use outdated and insecure templates.
> > Now, let's assume download.cloud.com is gone forever. What do we tell
> ACS
> > users pre-4.11 as far as *built-in templates* go?
> > 1. Direct them to update templates.sql with some new URL, but with the
> same
> > dirty old templates
> > 2. Direct them to update templates.sql with some new URL, but with nice
> > templates (e.g., open.vm.eu)
> > 3. Same as (2), but document more choices.
> >
> > Now, why should things be different for 4.11 and later? Documenting the
> > steps to install templates offline is trivial (and can be scripted to a
> > large part, like cloud-install-sys-tmplt)
> >
> > For pre-4.11 users, for *systemvms*, anyway we tell them to use
> > http://cloudstack.apt-get.eu which is not controlled by ACS.
> >
> >
> > On Mon, Feb 27, 2017 at 2:50 PM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > Agree with you.
> > > We need to support the current working flow. And then, define the first
> > > version that will start using the new approach.
> > >
> > > On Mon, Feb 27, 2017 at 5:36 PM, Will Stevens <wstev...@cloudops.com>
> > > wrote:
> > >
> > > > I think we almost need a two pronged approach.
> > > >
> > > > 1) Get a solution in place which will enable us to document and serve
> > > > templates for legacy systems.  I will work on this.
> > > > 2) Discuss and understand how we SHOULD be handling this problem in
> the
> > > > future and in what release we can expect it.
> > > >
> > > > I think we need to do both.  I think we should start to try to really
> > > > understand what we want to deliver in (2) going forward.
> > > >
> > > > *Will STEVENS*
> > > > Lead Developer
> > > >
> > > > <https://goo.gl/NYZ8KK>
> > > >
> > > > On Mon, Feb 27, 2017 at 4:53 PM, Rafael Weingärtner <
> > > > rafaelweingart...@gmail.com> wrote:
> > > >
> > > > > My worry is exactly with system VMs templates.
> > > > >
> > > > > Currently, we indicate administrators to download them from
> > > > > http://cloudstack.apt-get.eu/systemvm/4.6/ [1]. However, the
> > > > installation
> > > > > docs do not mention the expected hashes for the file that is going
> to
> > > be
> > > > > downloaded.
> > > > > Also, I do not know the code that downloads system VMs templates
> > (when
> > > > > upgrading), but if the hash being checked is taken from the mirror
> > used
> > > > to
> > > > > download the file; the only thing it checks is that if the download
> > > > > finished successfully (no transmission errors). If we want to check
> > > > > integrity, check that the template we created is untampered; we
> need
> > to
> > > > > host and serve the hash in a secure manner.
> > > > >
> > > > > [1]
> > > > > http://docs.cloudstack.apache.org/projects/cloudstack-
> > > > installation/en/4.9/
> > > > > management-server/index.html#prepare-the-system-vm-template
> > > > >
> > > > >
> > > > > On Mon, Feb 27, 2017 at 4:36 PM, Chiradeep 

Re: Modern template hosting

[Chiradeep Vittal]

My stance is that the current workflow does a disservice to the user
community by letting them install / use outdated and insecure templates.
Now, let's assume download.cloud.com is gone forever. What do we tell ACS
users pre-4.11 as far as *built-in templates* go?
1. Direct them to update templates.sql with some new URL, but with the same
dirty old templates
2. Direct them to update templates.sql with some new URL, but with nice
templates (e.g., open.vm.eu)
3. Same as (2), but document more choices.

Now, why should things be different for 4.11 and later? Documenting the
steps to install templates offline is trivial (and can be scripted to a
large part, like cloud-install-sys-tmplt)

For pre-4.11 users, for *systemvms*, anyway we tell them to use
http://cloudstack.apt-get.eu which is not controlled by ACS.


On Mon, Feb 27, 2017 at 2:50 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Agree with you.
> We need to support the current working flow. And then, define the first
> version that will start using the new approach.
>
> On Mon, Feb 27, 2017 at 5:36 PM, Will Stevens <wstev...@cloudops.com>
> wrote:
>
> > I think we almost need a two pronged approach.
> >
> > 1) Get a solution in place which will enable us to document and serve
> > templates for legacy systems.  I will work on this.
> > 2) Discuss and understand how we SHOULD be handling this problem in the
> > future and in what release we can expect it.
> >
> > I think we need to do both.  I think we should start to try to really
> > understand what we want to deliver in (2) going forward.
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > <https://goo.gl/NYZ8KK>
> >
> > On Mon, Feb 27, 2017 at 4:53 PM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > My worry is exactly with system VMs templates.
> > >
> > > Currently, we indicate administrators to download them from
> > > http://cloudstack.apt-get.eu/systemvm/4.6/ [1]. However, the
> > installation
> > > docs do not mention the expected hashes for the file that is going to
> be
> > > downloaded.
> > > Also, I do not know the code that downloads system VMs templates (when
> > > upgrading), but if the hash being checked is taken from the mirror used
> > to
> > > download the file; the only thing it checks is that if the download
> > > finished successfully (no transmission errors). If we want to check
> > > integrity, check that the template we created is untampered; we need to
> > > host and serve the hash in a secure manner.
> > >
> > > [1]
> > > http://docs.cloudstack.apache.org/projects/cloudstack-
> > installation/en/4.9/
> > > management-server/index.html#prepare-the-system-vm-template
> > >
> > >
> > > On Mon, Feb 27, 2017 at 4:36 PM, Chiradeep Vittal <
> chirade...@gmail.com>
> > > wrote:
> > >
> > > > Hashes are checked (md5 IIRC) today.
> > > > But given the issues, I think the project should steer away from
> > hosting
> > > > templates except the systemvm template.
> > > >
> > > > On Mon, Feb 27, 2017 at 1:31 PM, Rafael Weingärtner <
> > > > rafaelweingart...@gmail.com> wrote:
> > > >
> > > > > Will, I think we could support different path structures. This can
> > > > > facilitate different deployment of mirrors based on the structure
> the
> > > > host
> > > > > has.
> > > > >
> > > > > Could I add something else to the discussion? Have we discussed the
> > > > > security impacts of setting up this mirrors approach?
> > > > > I mean, if any of the mirrors gets corrupted (let`s say by a
> hacker),
> > > and
> > > > > the templates are injected with malicious code, an attacker could
> > > > > potentially get un-monitored and unlimited access to a cloud
> > > environment.
> > > > >
> > > > > If we assume that the mirror may get malicious (it is not that I do
> > not
> > > > > trust you guys, but bad things happen), we cannot host hashes
> there.
> > > > Where
> > > > > do you think we could store Sha512 or another hash type for these
> > > > > templates? Could we host in the newly proposed Github repo or maybe
> > > some
> > > > > place in the ACS website?
> > > > >
> > > > > This would have an impact on clients (needing clear documentation)
> > and
> > > &g

Re: Modern template hosting

[Chiradeep Vittal]

Hashes are checked (md5 IIRC) today.
But given the issues, I think the project should steer away from hosting
templates except the systemvm template.

On Mon, Feb 27, 2017 at 1:31 PM, Rafael Weingärtner <
rafaelweingart...@gmail.com> wrote:

> Will, I think we could support different path structures. This can
> facilitate different deployment of mirrors based on the structure the host
> has.
>
> Could I add something else to the discussion? Have we discussed the
> security impacts of setting up this mirrors approach?
> I mean, if any of the mirrors gets corrupted (let`s say by a hacker), and
> the templates are injected with malicious code, an attacker could
> potentially get un-monitored and unlimited access to a cloud environment.
>
> If we assume that the mirror may get malicious (it is not that I do not
> trust you guys, but bad things happen), we cannot host hashes there. Where
> do you think we could store Sha512 or another hash type for these
> templates? Could we host in the newly proposed Github repo or maybe some
> place in the ACS website?
>
> This would have an impact on clients (needing clear documentation) and our
> code that automatically downloads System VM templates (does it check hashes
> when automatically installing templates today? It may require
> implementation changes).
>
> On Mon, Feb 27, 2017 at 3:48 PM, Will Stevens <wstev...@cloudops.com>
> wrote:
>
> > so this is what I am looking to do.  Please let me know if you have
> > suggestions for me or think I should be solving the problem a different
> > way.
> >
> > - We request a new Github repository from the ASF at:
> > 'apache/cloudstack-mirror-list'
> > - In this repository we track a text file in the 'gh-pages' branch with a
> > list of valid download mirrors.
> > - I build a binary to be hosted by the ASF (or at least with the ASF
> > pointing a domain at the binary and I could potentially host it).  We
> will
> > see how they want to handle the hosting of the binary.
> >
> > The binary would expose a web server which would behave as follows:
> > - When the 'client' requests a download url the following flow is kicked
> > off:
> > -- The mirror list is queried from github (or from a static site hosted
> on
> > asf, as we see fit).
> > -- The Lat/Lon of the 'client' is determined based on their IP.
> > -- The Lat/Lon for each of the 'mirror's is determined based on an IP
> > lookup of the hostname.
> > -- The closest geographical mirror is determined, the target is validated
> > to be available and the user is redirected.
> >
> > Some questions I have right now:
> > - Will every mirror have the same path structure to access the equivalent
> > resources?
> > - Should we support adding a path to the mirror url to specify the path
> to
> > the base common path?
> > -- Example: lets say the binary is hosted on 'dl.acs.com' and there are
> > three mirrors 'abc.com', 'pqr.com/files' and 'xyx.com/downloads'.
> > -- If the path being requested is '
> > dl.acs.com/templates/systemvm-4.6.xen.vhd.bz2', it would result in the
> > following potential paths for the mirrors:
> > -- 'abc.com/templates/systemvm-4.6.xen.vhd.bz2'
> > -- 'pqr.com/files/templates/systemvm-4.6.xen.vhd.bz2'
> > -- 'xyz.com/downloads/templates/systemvm-4.6.xen.vhd.bz2'
> >
> > Does this all make sense?
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > <https://goo.gl/NYZ8KK>
> >
> > On Mon, Feb 27, 2017 at 1:31 PM, Chiradeep Vittal <chirade...@gmail.com>
> > wrote:
> >
> > > My bad. A few lines down, this has been added recently:
> > >
> > > this.request.setFollowRedirects(true);
> > >
> > > On Mon, Feb 27, 2017 at 10:15 AM, Will Stevens <
> williamstev...@gmail.com
> > >
> > > wrote:
> > >
> > > > OK. Thanks for the heads up.
> > > >
> > > > On Feb 27, 2017 1:08 PM, "Chiradeep Vittal" <chirade...@gmail.com>
> > > wrote:
> > > >
> > > > > Sounds workable. The downloader code in the SSVM won't follow
> > > redirects I
> > > > > think.
> > > > > https://github.com/apache/cloudstack/blob/
> > > 5511065fc20787619d9cd0444a65a3
> > > > > 155fc9c921/core/src/com/cloud/storage/template/
> > > > > HttpTemplateDownloader.java#L93
> > > > > https://goo.gl/dSi0r5
> > > > >  Might need to add
> > > > > client.setRedirectStrategy(new LaxRedirectStrategy());
> > > > >
> > > > > O

Re: Modern template hosting

[Chiradeep Vittal]

Infra supports Ubuntu VMs apparently so you could do what you want there
https://www.apache.org/dev/services#virtual-servers

Another idea is to distribute the http server binary with ACS - the
CloudStack admin installs the http server and gives it a reachable fqdn /
ip and updates the templates table.


On Mon, Feb 27, 2017 at 12:48 PM, Will Stevens <wstev...@cloudops.com>
wrote:

> so this is what I am looking to do.  Please let me know if you have
> suggestions for me or think I should be solving the problem a different
> way.
>
> - We request a new Github repository from the ASF at:
> 'apache/cloudstack-mirror-list'
> - In this repository we track a text file in the 'gh-pages' branch with a
> list of valid download mirrors.
> - I build a binary to be hosted by the ASF (or at least with the ASF
> pointing a domain at the binary and I could potentially host it).  We will
> see how they want to handle the hosting of the binary.
>
> The binary would expose a web server which would behave as follows:
> - When the 'client' requests a download url the following flow is kicked
> off:
> -- The mirror list is queried from github (or from a static site hosted on
> asf, as we see fit).
> -- The Lat/Lon of the 'client' is determined based on their IP.
> -- The Lat/Lon for each of the 'mirror's is determined based on an IP
> lookup of the hostname.
> -- The closest geographical mirror is determined, the target is validated
> to be available and the user is redirected.
>
> Some questions I have right now:
> - Will every mirror have the same path structure to access the equivalent
> resources?
> - Should we support adding a path to the mirror url to specify the path to
> the base common path?
> -- Example: lets say the binary is hosted on 'dl.acs.com' and there are
> three mirrors 'abc.com', 'pqr.com/files' and 'xyx.com/downloads'.
> -- If the path being requested is '
> dl.acs.com/templates/systemvm-4.6.xen.vhd.bz2', it would result in the
> following potential paths for the mirrors:
> -- 'abc.com/templates/systemvm-4.6.xen.vhd.bz2'
> -- 'pqr.com/files/templates/systemvm-4.6.xen.vhd.bz2'
> -- 'xyz.com/downloads/templates/systemvm-4.6.xen.vhd.bz2'
>
> Does this all make sense?
>
> *Will STEVENS*
> Lead Developer
>
> <https://goo.gl/NYZ8KK>
>
> On Mon, Feb 27, 2017 at 1:31 PM, Chiradeep Vittal <chirade...@gmail.com>
> wrote:
>
> > My bad. A few lines down, this has been added recently:
> >
> > this.request.setFollowRedirects(true);
> >
> > On Mon, Feb 27, 2017 at 10:15 AM, Will Stevens <williamstev...@gmail.com
> >
> > wrote:
> >
> > > OK. Thanks for the heads up.
> > >
> > > On Feb 27, 2017 1:08 PM, "Chiradeep Vittal" <chirade...@gmail.com>
> > wrote:
> > >
> > > > Sounds workable. The downloader code in the SSVM won't follow
> > redirects I
> > > > think.
> > > > https://github.com/apache/cloudstack/blob/
> > 5511065fc20787619d9cd0444a65a3
> > > > 155fc9c921/core/src/com/cloud/storage/template/
> > > > HttpTemplateDownloader.java#L93
> > > > https://goo.gl/dSi0r5
> > > >  Might need to add
> > > > client.setRedirectStrategy(new LaxRedirectStrategy());
> > > >
> > > > On Mon, Feb 27, 2017 at 9:57 AM, Will Stevens <wstev...@cloudops.com
> >
> > > > wrote:
> > > >
> > > > > We haven't opened a ticket yet because we don't have a strategy
> yet.
> > > > >
> > > > > What do you guys think of this:
> > > > > - We setup a new github repo in the 'apache' org which consists of
> a
> > > > single
> > > > > file with a list of active/supported mirrors.
> > > > > - I write a small web server, distributed as a binary, which can be
> > > > hosted
> > > > > by ASF Infra.  This web server will query the current list of
> mirrors
> > > and
> > > > > will select one and then do a 302 redirect to that mirror.
> > > > >
> > > > > The act of 'choosing' a mirror could be done in a number of ways.
> > > > > - If we want to define an order, then it could just try from the
> top
> > of
> > > > the
> > > > > list and work its way down.  It would curl the target to make sure
> it
> > > > gets
> > > > > a 200 and if it does, it would do a 302 redirect.
> > > > > - Or, if we want to distribute the load across the mirrors, we
> could
> > > pick
> > > > > from the list randomly.  Again, doing a curl 

Re: Modern template hosting

[Chiradeep Vittal]

My bad. A few lines down, this has been added recently:

this.request.setFollowRedirects(true);

On Mon, Feb 27, 2017 at 10:15 AM, Will Stevens <williamstev...@gmail.com>
wrote:

> OK. Thanks for the heads up.
>
> On Feb 27, 2017 1:08 PM, "Chiradeep Vittal" <chirade...@gmail.com> wrote:
>
> > Sounds workable. The downloader code in the SSVM won't follow redirects I
> > think.
> > https://github.com/apache/cloudstack/blob/5511065fc20787619d9cd0444a65a3
> > 155fc9c921/core/src/com/cloud/storage/template/
> > HttpTemplateDownloader.java#L93
> > https://goo.gl/dSi0r5
> >  Might need to add
> > client.setRedirectStrategy(new LaxRedirectStrategy());
> >
> > On Mon, Feb 27, 2017 at 9:57 AM, Will Stevens <wstev...@cloudops.com>
> > wrote:
> >
> > > We haven't opened a ticket yet because we don't have a strategy yet.
> > >
> > > What do you guys think of this:
> > > - We setup a new github repo in the 'apache' org which consists of a
> > single
> > > file with a list of active/supported mirrors.
> > > - I write a small web server, distributed as a binary, which can be
> > hosted
> > > by ASF Infra.  This web server will query the current list of mirrors
> and
> > > will select one and then do a 302 redirect to that mirror.
> > >
> > > The act of 'choosing' a mirror could be done in a number of ways.
> > > - If we want to define an order, then it could just try from the top of
> > the
> > > list and work its way down.  It would curl the target to make sure it
> > gets
> > > a 200 and if it does, it would do a 302 redirect.
> > > - Or, if we want to distribute the load across the mirrors, we could
> pick
> > > from the list randomly.  Again, doing a curl to verify the mirror is up
> > and
> > > then doing a redirect.
> > > - If we want to get fancy, we could do a reverse IP lookup and try to
> > match
> > > the requester with their closest geographical mirror.
> > >
> > > Thoughts?
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > <https://goo.gl/NYZ8KK>
> > >
> > > On Mon, Feb 27, 2017 at 12:46 PM, Chiradeep Vittal <
> chirade...@gmail.com
> > >
> > > wrote:
> > >
> > > > What steps are needed to set up a mirror? What does Infra need to do?
> > Has
> > > > anybody filed a ticket with Infra?
> > > >
> > > > On Sun, Feb 26, 2017 at 10:17 PM, Raja Pullela <
> > > > raja.pull...@accelerite.com>
> > > > wrote:
> > > >
> > > > > Hi will,
> > > > >
> > > > > I believe, we didn’t get to close ‘getting a mirror on Apache’
> > because
> > > we
> > > > > needed someone on the Apache Infra side to close this. BTW,
> > > > > cloudstack-apt.get.eu (I think Nux manages this?) has all/most of
> > the
> > > > > content.  Once we can close on the Apache mirror for hosting the
> > > > content, I
> > > > > can help assist getting the content there.
> > > > >
> > > > > For now, we have replicated the download.cloud.com content to ‘
> > > > > s3.download.accelerite.com’.
> > > > > Also, we are working on a set of steps/procedure to help with this
> > > > > change.  I will update everyone in about a week’s time on the
> > details.
> > > > >
> > > > > Best,
> > > > > Raja Pullela
> > > > > Engineering Team,
> > > > > Accelerite, 2055 Laurelwood Road,
> > > > > Santa Clara, CA, 95054
> > > > >
> > > > > On 2/24/17, 11:23 PM, "williamstev...@gmail.com on behalf of Will
> > > > > Stevens" <williamstev...@gmail.com on behalf of
> > wstev...@cloudops.com>
> > > > > wrote:
> > > > >
> > > > > unfortunately the template mirror conversation got caught up in
> > details
> > > > and
> > > > > nobody took the lead on implementing a solution.
> > > > >
> > > > > citrix has been pinging me every couple months to say 'dude, we
> need
> > to
> > > > > remove the dependency on download.citrix.com', but i have not had
> > the
> > > > > cycles to get in and solve the problem.  the shutdown of that is
> > > imminent
> > > > > right now, so we need to solve it asap.
> > > >

Re: Modern template hosting

[Chiradeep Vittal]

The alternative foolproof way is to simply not provide the automatic
download.

Just document the mirror list URL where the templates can be found. I don't
know why people reject this option.

https://shankerbalan.net/blog/seed-cloudstack-templates-offline/

On Mon, Feb 27, 2017 at 10:15 AM, Will Stevens <williamstev...@gmail.com>
wrote:

> OK. Thanks for the heads up.
>
> On Feb 27, 2017 1:08 PM, "Chiradeep Vittal" <chirade...@gmail.com> wrote:
>
> > Sounds workable. The downloader code in the SSVM won't follow redirects I
> > think.
> > https://github.com/apache/cloudstack/blob/5511065fc20787619d9cd0444a65a3
> > 155fc9c921/core/src/com/cloud/storage/template/
> > HttpTemplateDownloader.java#L93
> > https://goo.gl/dSi0r5
> >  Might need to add
> > client.setRedirectStrategy(new LaxRedirectStrategy());
> >
> > On Mon, Feb 27, 2017 at 9:57 AM, Will Stevens <wstev...@cloudops.com>
> > wrote:
> >
> > > We haven't opened a ticket yet because we don't have a strategy yet.
> > >
> > > What do you guys think of this:
> > > - We setup a new github repo in the 'apache' org which consists of a
> > single
> > > file with a list of active/supported mirrors.
> > > - I write a small web server, distributed as a binary, which can be
> > hosted
> > > by ASF Infra.  This web server will query the current list of mirrors
> and
> > > will select one and then do a 302 redirect to that mirror.
> > >
> > > The act of 'choosing' a mirror could be done in a number of ways.
> > > - If we want to define an order, then it could just try from the top of
> > the
> > > list and work its way down.  It would curl the target to make sure it
> > gets
> > > a 200 and if it does, it would do a 302 redirect.
> > > - Or, if we want to distribute the load across the mirrors, we could
> pick
> > > from the list randomly.  Again, doing a curl to verify the mirror is up
> > and
> > > then doing a redirect.
> > > - If we want to get fancy, we could do a reverse IP lookup and try to
> > match
> > > the requester with their closest geographical mirror.
> > >
> > > Thoughts?
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > <https://goo.gl/NYZ8KK>
> > >
> > > On Mon, Feb 27, 2017 at 12:46 PM, Chiradeep Vittal <
> chirade...@gmail.com
> > >
> > > wrote:
> > >
> > > > What steps are needed to set up a mirror? What does Infra need to do?
> > Has
> > > > anybody filed a ticket with Infra?
> > > >
> > > > On Sun, Feb 26, 2017 at 10:17 PM, Raja Pullela <
> > > > raja.pull...@accelerite.com>
> > > > wrote:
> > > >
> > > > > Hi will,
> > > > >
> > > > > I believe, we didn’t get to close ‘getting a mirror on Apache’
> > because
> > > we
> > > > > needed someone on the Apache Infra side to close this. BTW,
> > > > > cloudstack-apt.get.eu (I think Nux manages this?) has all/most of
> > the
> > > > > content.  Once we can close on the Apache mirror for hosting the
> > > > content, I
> > > > > can help assist getting the content there.
> > > > >
> > > > > For now, we have replicated the download.cloud.com content to ‘
> > > > > s3.download.accelerite.com’.
> > > > > Also, we are working on a set of steps/procedure to help with this
> > > > > change.  I will update everyone in about a week’s time on the
> > details.
> > > > >
> > > > > Best,
> > > > > Raja Pullela
> > > > > Engineering Team,
> > > > > Accelerite, 2055 Laurelwood Road,
> > > > > Santa Clara, CA, 95054
> > > > >
> > > > > On 2/24/17, 11:23 PM, "williamstev...@gmail.com on behalf of Will
> > > > > Stevens" <williamstev...@gmail.com on behalf of
> > wstev...@cloudops.com>
> > > > > wrote:
> > > > >
> > > > > unfortunately the template mirror conversation got caught up in
> > details
> > > > and
> > > > > nobody took the lead on implementing a solution.
> > > > >
> > > > > citrix has been pinging me every couple months to say 'dude, we
> need
> > to
> > > > > remove the dependency on download.citrix.com', but i have not had
> > the
> &

Re: Modern template hosting

[Chiradeep Vittal]

Sounds workable. The downloader code in the SSVM won't follow redirects I
think.
https://github.com/apache/cloudstack/blob/5511065fc20787619d9cd0444a65a3155fc9c921/core/src/com/cloud/storage/template/HttpTemplateDownloader.java#L93
https://goo.gl/dSi0r5
 Might need to add
client.setRedirectStrategy(new LaxRedirectStrategy());

On Mon, Feb 27, 2017 at 9:57 AM, Will Stevens <wstev...@cloudops.com> wrote:

> We haven't opened a ticket yet because we don't have a strategy yet.
>
> What do you guys think of this:
> - We setup a new github repo in the 'apache' org which consists of a single
> file with a list of active/supported mirrors.
> - I write a small web server, distributed as a binary, which can be hosted
> by ASF Infra.  This web server will query the current list of mirrors and
> will select one and then do a 302 redirect to that mirror.
>
> The act of 'choosing' a mirror could be done in a number of ways.
> - If we want to define an order, then it could just try from the top of the
> list and work its way down.  It would curl the target to make sure it gets
> a 200 and if it does, it would do a 302 redirect.
> - Or, if we want to distribute the load across the mirrors, we could pick
> from the list randomly.  Again, doing a curl to verify the mirror is up and
> then doing a redirect.
> - If we want to get fancy, we could do a reverse IP lookup and try to match
> the requester with their closest geographical mirror.
>
> Thoughts?
>
> *Will STEVENS*
> Lead Developer
>
> <https://goo.gl/NYZ8KK>
>
> On Mon, Feb 27, 2017 at 12:46 PM, Chiradeep Vittal <chirade...@gmail.com>
> wrote:
>
> > What steps are needed to set up a mirror? What does Infra need to do? Has
> > anybody filed a ticket with Infra?
> >
> > On Sun, Feb 26, 2017 at 10:17 PM, Raja Pullela <
> > raja.pull...@accelerite.com>
> > wrote:
> >
> > > Hi will,
> > >
> > > I believe, we didn’t get to close ‘getting a mirror on Apache’ because
> we
> > > needed someone on the Apache Infra side to close this. BTW,
> > > cloudstack-apt.get.eu (I think Nux manages this?) has all/most of the
> > > content.  Once we can close on the Apache mirror for hosting the
> > content, I
> > > can help assist getting the content there.
> > >
> > > For now, we have replicated the download.cloud.com content to ‘
> > > s3.download.accelerite.com’.
> > > Also, we are working on a set of steps/procedure to help with this
> > > change.  I will update everyone in about a week’s time on the details.
> > >
> > > Best,
> > > Raja Pullela
> > > Engineering Team,
> > > Accelerite, 2055 Laurelwood Road,
> > > Santa Clara, CA, 95054
> > >
> > > On 2/24/17, 11:23 PM, "williamstev...@gmail.com on behalf of Will
> > > Stevens" <williamstev...@gmail.com on behalf of wstev...@cloudops.com>
> > > wrote:
> > >
> > > unfortunately the template mirror conversation got caught up in details
> > and
> > > nobody took the lead on implementing a solution.
> > >
> > > citrix has been pinging me every couple months to say 'dude, we need to
> > > remove the dependency on download.citrix.com', but i have not had the
> > > cycles to get in and solve the problem.  the shutdown of that is
> imminent
> > > right now, so we need to solve it asap.
> > >
> > >
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > <https://goo.gl/NYZ8KK>
> > >
> > > On Fri, Feb 24, 2017 at 12:38 PM, Paul Angus <paul.an...@shapeblue.com
> >
> > > wrote:
> > >
> > > > Hi Nathan,
> > > >
> > > > Ideally, if you put the template location in (or use a template
> defined
> > > > in)  test_data.py then the actual location can be overridden by
> anyone
> > > > testing.
> > > >
> > > > For Trillian, we've copied all of the templates that people have
> define
> > > to
> > > > a local repo and then replace the URLs in test_data.py to reduce
> > > bandwidth
> > > > use and download times.
> > > >
> > > > Ie:
> > > >
> > > > "bootableIso":
> > > > {
> > > > "displaytext": "Test Bootable ISO",
> > > > "name": "testISO",
> > > > "bootable": True,
> > > > "ispu

Re: Modern template hosting

[Chiradeep Vittal]

What steps are needed to set up a mirror? What does Infra need to do? Has
anybody filed a ticket with Infra?

On Sun, Feb 26, 2017 at 10:17 PM, Raja Pullela 
wrote:

> Hi will,
>
> I believe, we didn’t get to close ‘getting a mirror on Apache’ because we
> needed someone on the Apache Infra side to close this. BTW,
> cloudstack-apt.get.eu (I think Nux manages this?) has all/most of the
> content.  Once we can close on the Apache mirror for hosting the content, I
> can help assist getting the content there.
>
> For now, we have replicated the download.cloud.com content to ‘
> s3.download.accelerite.com’.
> Also, we are working on a set of steps/procedure to help with this
> change.  I will update everyone in about a week’s time on the details.
>
> Best,
> Raja Pullela
> Engineering Team,
> Accelerite, 2055 Laurelwood Road,
> Santa Clara, CA, 95054
>
> On 2/24/17, 11:23 PM, "williamstev...@gmail.com on behalf of Will
> Stevens" 
> wrote:
>
> unfortunately the template mirror conversation got caught up in details and
> nobody took the lead on implementing a solution.
>
> citrix has been pinging me every couple months to say 'dude, we need to
> remove the dependency on download.citrix.com', but i have not had the
> cycles to get in and solve the problem.  the shutdown of that is imminent
> right now, so we need to solve it asap.
>
>
>
> *Will STEVENS*
> Lead Developer
>
> 
>
> On Fri, Feb 24, 2017 at 12:38 PM, Paul Angus 
> wrote:
>
> > Hi Nathan,
> >
> > Ideally, if you put the template location in (or use a template defined
> > in)  test_data.py then the actual location can be overridden by anyone
> > testing.
> >
> > For Trillian, we've copied all of the templates that people have define
> to
> > a local repo and then replace the URLs in test_data.py to reduce
> bandwidth
> > use and download times.
> >
> > Ie:
> >
> > "bootableIso":
> > {
> > "displaytext": "Test Bootable ISO",
> > "name": "testISO",
> > "bootable": True,
> > "ispublic": False,
> > "url": "{{ marvin_images_location
> > }}/TinyCore-current.iso",
> > "ostype": 'Other Linux (64-bit)',
> > "mode": 'HTTP_DOWNLOAD'
> > },
> >
> >
> >
> > I thought that we had come up with a solution for download.cloud.com, by
> > having a mirrorlist hosted in Community Apache 'space' with anyone able
> to
> > out themselves forward as a mirror.
> > But I must admit I lost track of whether anyone made the requisite
> changes
> > in code
> >
> >
> >
> >
> > Kind regards,
> >
> > Paul Angus
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On
> > Behalf Of Will Stevens
> > Sent: 24 February 2017 16:30
> > To: dev@cloudstack.apache.org
> > Subject: Re: Modern template hosting
> >
> > this is a hard questions.  in general, we should be setting up a mirror
> on
> > some cloudstack/apache domain and then mirror to other provided
> templates.
> >
> > we MUST come up with a solution to deprecate 'download.cloud.com', that
> > is going to be going away any day now.
> >
> > i don't know the right way to solve this to be honest, but if you have
> > ideas, i am willing to help.
> >
> >
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > 
> >
> > On Fri, Feb 24, 2017 at 11:25 AM, Nathan Johnson 
> wrote:
> >
> > > So not to re-open a can of worms, but I’m in a situation where I need
> > > to come up with a Marvin component test that depends on a template
> > > based on a kernel that’s relatively new, i.e., newer than Centos 5.3 /
> > Ubuntu 10.04 .
> > > I see openvm.eu has a suitable template (Ubuntu 16.0.4 for KVM), but
> > > from looking at the thread "Migrating CloudStack content from
> > > download.cloud.com” it looks like there is resistance to using this at
> > > least for hosting system vm templates over concerns of neutrality.
> > > Would this be suitable for a component test?  If not, what is a
> “blessed”
> > > template location?
> > >
> > > Thanks in advance!
> > >
> > > Nathan Johnson
> > > R Engineer
> > >
> > >
> > >
> > > 618 Grassmere Park Drive, Suite 12
> > > Nashville, TN 37211
> > > General Office: 615-312-6000
> > >
> > > website | blog | support
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
>
>
>
>
>
>
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which is
> the property of Accelerite, a Persistent Systems business. It is intended
> only for the use of the individual or entity to which it is addressed. If
> you are not the intended recipient, you are not authorized to 

Re: How would you like your Cloudstack templates?

[Chiradeep Vittal]

Use the distro default 

> On Feb 25, 2017, at 3:40 AM, Nux!  wrote:
> 
> Hello,
> 
> Since templates are being discussed, I wanted to chip in and let you know I 
> am about to revamp the templates at OpenVM.eu which I have neglected a bit.
> The roadmap is short and should look something like this:
> 
> - we build everything from scratch (as opposed to trying to reuse and modify 
> upstream images - I'm doing this for ubu/deb now) - except coreos which is 
> weird and provide compatible images anyway
> 
> - all the kickstarts and preseeds will be on github, everyone is welcome to 
> chip in with improvements or build their own
> 
> - separate per hypervisor type template builds, xenserver builds will ship 
> xs-tools, vmware builds will ship openvm-tools, kvm will ship qemu-ga, hyperv 
> will include their tools
> 
> 
> The hw is sponsored, I am not paying for it. I'm happy to allow responsible 
> PMC members access to it.
> 
> Also happy to host systemvm templates at OpenVM as a "neutral third party".
> 
> 
> 
> Now a few questions to which I'd really like some feedback.
> 
> 1. Cloud-init is rather stubborn in using a default unprivileged user instead 
> of root.
> Should I use per distro users - ubuntu for ubuntu, centos for centos, freebsd 
> for freebsd - or should I use something like "cloud" or "admin" for all of 
> them?
> 
> 2. I currently disable IPv6 in the templates I build from scratch as it is a 
> pain point with Cloudstack at the moment - alas. Should I just leave the 
> defaults on (ie enabled)?
> 
> 
> Thanks!
> 
> 
> 
> --
> Sent from the Delta quadrant using Borg technology!
> 
> Nux!
> www.nux.ro

Re: Modern template hosting

[Chiradeep Vittal]

Is the issue that ACS doesn't support 169.254.169.254 as the metadata URL?

For those hypervisors that use the Linux bridge and iptables, it should be easy 
to add a DNAT rule to translate it to the VR IP? I'm talking about code change 
here. 
Or send down a static route in the DHCP response that points 169.254.269.254 to 
the VR IP? 

Sent from my iPhone

> On Feb 25, 2017, at 3:18 AM, Nux! <n...@li.nux.ro> wrote:
> 
> Hello,
> 
> The upstream Ubuntu and CentOS images will not work out of the box with 
> Cloudstack. I wanted to reuse them for my OpenVM project and failed.
> 
> The default metadata source is EC2 or Openstack, don't remember, cloud-init 
> will take ages to time-out and there is zero support for the root password 
> feature - in fact the cloud world is moving away from direct root access and 
> so should we - at least for compatibility reasons if not something else.
> 
> I am happy to put some more elbow grease into openvm.eu and make some 
> more/better templates. 
> 
> I can also host other templates such as the cloud.com ones.
> 
> Open to comments, questions etc.
> 
> Lucian
> 
> --
> Sent from the Delta quadrant using Borg technology!
> 
> Nux!
> www.nux.ro
> 
> - Original Message -
>> From: "Chiradeep Vittal" <chirade...@gmail.com>
>> To: "dev" <dev@cloudstack.apache.org>
>> Sent: Saturday, 25 February, 2017 00:18:48
>> Subject: Re: Modern template hosting
> 
>> Also CentOS KVM images can be obtained here:
>> http://cloud.centos.org/centos/7/images/
>> 
>> If these work, then the only question is where to host the systemvm images.
>> 
>> On Fri, Feb 24, 2017 at 4:05 PM, Chiradeep Vittal <chirade...@gmail.com>
>> wrote:
>> 
>>> I noticed that Ubuntu provides daily builds, including VHDs. Perhaps we
>>> could point the default templates to their URL? Anyone willing to quickly
>>> try out their images to see if they work on ACS?
>>> 
>>> https://cloud-images.ubuntu.com/xenial/current/
>>> 
>>> On Fri, Feb 24, 2017 at 9:53 AM, Will Stevens <wstev...@cloudops.com>
>>> wrote:
>>> 
>>>> unfortunately the template mirror conversation got caught up in details
>>>> and
>>>> nobody took the lead on implementing a solution.
>>>> 
>>>> citrix has been pinging me every couple months to say 'dude, we need to
>>>> remove the dependency on download.citrix.com', but i have not had the
>>>> cycles to get in and solve the problem.  the shutdown of that is imminent
>>>> right now, so we need to solve it asap.
>>>> 
>>>> 
>>>> 
>>>> *Will STEVENS*
>>>> Lead Developer
>>>> 
>>>> <https://goo.gl/NYZ8KK>
>>>> 
>>>> On Fri, Feb 24, 2017 at 12:38 PM, Paul Angus <paul.an...@shapeblue.com>
>>>> wrote:
>>>> 
>>>>> Hi Nathan,
>>>>> 
>>>>> Ideally, if you put the template location in (or use a template defined
>>>>> in)  test_data.py then the actual location can be overridden by anyone
>>>>> testing.
>>>>> 
>>>>> For Trillian, we've copied all of the templates that people have define
>>>> to
>>>>> a local repo and then replace the URLs in test_data.py to reduce
>>>> bandwidth
>>>>> use and download times.
>>>>> 
>>>>> Ie:
>>>>> 
>>>>>"bootableIso":
>>>>>{
>>>>>"displaytext": "Test Bootable ISO",
>>>>>"name": "testISO",
>>>>>"bootable": True,
>>>>>"ispublic": False,
>>>>>"url": "{{ marvin_images_location
>>>>> }}/TinyCore-current.iso",
>>>>>"ostype": 'Other Linux (64-bit)',
>>>>>"mode": 'HTTP_DOWNLOAD'
>>>>>},
>>>>> 
>>>>> 
>>>>> 
>>>>> I thought that we had come up with a solution for download.cloud.com,
>>>> by
>>>>> having a mirrorlist hosted in Community Apache 'space' with anyone able
>>>> to
>>>>> out themselves forward as a mirror.
>>>>> But I must admit I lost track of whether anyone made the requisite

Re: Modern template hosting

[Chiradeep Vittal]

Also CentOS KVM images can be obtained here:
http://cloud.centos.org/centos/7/images/

If these work, then the only question is where to host the systemvm images.

On Fri, Feb 24, 2017 at 4:05 PM, Chiradeep Vittal <chirade...@gmail.com>
wrote:

> I noticed that Ubuntu provides daily builds, including VHDs. Perhaps we
> could point the default templates to their URL? Anyone willing to quickly
> try out their images to see if they work on ACS?
>
> https://cloud-images.ubuntu.com/xenial/current/
>
> On Fri, Feb 24, 2017 at 9:53 AM, Will Stevens <wstev...@cloudops.com>
> wrote:
>
>> unfortunately the template mirror conversation got caught up in details
>> and
>> nobody took the lead on implementing a solution.
>>
>> citrix has been pinging me every couple months to say 'dude, we need to
>> remove the dependency on download.citrix.com', but i have not had the
>> cycles to get in and solve the problem.  the shutdown of that is imminent
>> right now, so we need to solve it asap.
>>
>>
>>
>> *Will STEVENS*
>> Lead Developer
>>
>> <https://goo.gl/NYZ8KK>
>>
>> On Fri, Feb 24, 2017 at 12:38 PM, Paul Angus <paul.an...@shapeblue.com>
>> wrote:
>>
>> > Hi Nathan,
>> >
>> > Ideally, if you put the template location in (or use a template defined
>> > in)  test_data.py then the actual location can be overridden by anyone
>> > testing.
>> >
>> > For Trillian, we've copied all of the templates that people have define
>> to
>> > a local repo and then replace the URLs in test_data.py to reduce
>> bandwidth
>> > use and download times.
>> >
>> > Ie:
>> >
>> > "bootableIso":
>> > {
>> > "displaytext": "Test Bootable ISO",
>> > "name": "testISO",
>> > "bootable": True,
>> > "ispublic": False,
>> > "url": "{{ marvin_images_location
>> > }}/TinyCore-current.iso",
>> > "ostype": 'Other Linux (64-bit)',
>> > "mode": 'HTTP_DOWNLOAD'
>> > },
>> >
>> >
>> >
>> > I thought that we had come up with a solution for download.cloud.com,
>> by
>> > having a mirrorlist hosted in Community Apache 'space' with anyone able
>> to
>> > out themselves forward as a mirror.
>> > But I must admit I lost track of whether anyone made the requisite
>> changes
>> > in code
>> >
>> >
>> >
>> >
>> > Kind regards,
>> >
>> > Paul Angus
>> >
>> > paul.an...@shapeblue.com
>> > www.shapeblue.com
>> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> > @shapeblue
>> >
>> >
>> >
>> >
>> > -Original Message-
>> > From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On
>> > Behalf Of Will Stevens
>> > Sent: 24 February 2017 16:30
>> > To: dev@cloudstack.apache.org
>> > Subject: Re: Modern template hosting
>> >
>> > this is a hard questions.  in general, we should be setting up a mirror
>> on
>> > some cloudstack/apache domain and then mirror to other provided
>> templates.
>> >
>> > we MUST come up with a solution to deprecate 'download.cloud.com', that
>> > is going to be going away any day now.
>> >
>> > i don't know the right way to solve this to be honest, but if you have
>> > ideas, i am willing to help.
>> >
>> >
>> >
>> > *Will STEVENS*
>> > Lead Developer
>> >
>> > <https://goo.gl/NYZ8KK>
>> >
>> > On Fri, Feb 24, 2017 at 11:25 AM, Nathan Johnson <njohn...@ena.com>
>> wrote:
>> >
>> > > So not to re-open a can of worms, but I’m in a situation where I need
>> > > to come up with a Marvin component test that depends on a template
>> > > based on a kernel that’s relatively new, i.e., newer than Centos 5.3 /
>> > Ubuntu 10.04 .
>> > > I see openvm.eu has a suitable template (Ubuntu 16.0.4 for KVM), but
>> > > from looking at the thread "Migrating CloudStack content from
>> > > download.cloud.com” it looks like there is resistance to using this
>> at
>> > > least for hosting system vm templates over concerns of neutrality.
>> > > Would this be suitable for a component test?  If not, what is a
>> “blessed”
>> > > template location?
>> > >
>> > > Thanks in advance!
>> > >
>> > > Nathan Johnson
>> > > R Engineer
>> > >
>> > >
>> > >
>> > > 618 Grassmere Park Drive, Suite 12
>> > > Nashville, TN 37211
>> > > General Office: 615-312-6000
>> > >
>> > > website | blog | support
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> > >
>> >
>>
>
>

Re: Modern template hosting

[Chiradeep Vittal]

I noticed that Ubuntu provides daily builds, including VHDs. Perhaps we
could point the default templates to their URL? Anyone willing to quickly
try out their images to see if they work on ACS?

https://cloud-images.ubuntu.com/xenial/current/

On Fri, Feb 24, 2017 at 9:53 AM, Will Stevens  wrote:

> unfortunately the template mirror conversation got caught up in details and
> nobody took the lead on implementing a solution.
>
> citrix has been pinging me every couple months to say 'dude, we need to
> remove the dependency on download.citrix.com', but i have not had the
> cycles to get in and solve the problem.  the shutdown of that is imminent
> right now, so we need to solve it asap.
>
>
>
> *Will STEVENS*
> Lead Developer
>
> 
>
> On Fri, Feb 24, 2017 at 12:38 PM, Paul Angus 
> wrote:
>
> > Hi Nathan,
> >
> > Ideally, if you put the template location in (or use a template defined
> > in)  test_data.py then the actual location can be overridden by anyone
> > testing.
> >
> > For Trillian, we've copied all of the templates that people have define
> to
> > a local repo and then replace the URLs in test_data.py to reduce
> bandwidth
> > use and download times.
> >
> > Ie:
> >
> > "bootableIso":
> > {
> > "displaytext": "Test Bootable ISO",
> > "name": "testISO",
> > "bootable": True,
> > "ispublic": False,
> > "url": "{{ marvin_images_location
> > }}/TinyCore-current.iso",
> > "ostype": 'Other Linux (64-bit)',
> > "mode": 'HTTP_DOWNLOAD'
> > },
> >
> >
> >
> > I thought that we had come up with a solution for download.cloud.com, by
> > having a mirrorlist hosted in Community Apache 'space' with anyone able
> to
> > out themselves forward as a mirror.
> > But I must admit I lost track of whether anyone made the requisite
> changes
> > in code
> >
> >
> >
> >
> > Kind regards,
> >
> > Paul Angus
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On
> > Behalf Of Will Stevens
> > Sent: 24 February 2017 16:30
> > To: dev@cloudstack.apache.org
> > Subject: Re: Modern template hosting
> >
> > this is a hard questions.  in general, we should be setting up a mirror
> on
> > some cloudstack/apache domain and then mirror to other provided
> templates.
> >
> > we MUST come up with a solution to deprecate 'download.cloud.com', that
> > is going to be going away any day now.
> >
> > i don't know the right way to solve this to be honest, but if you have
> > ideas, i am willing to help.
> >
> >
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > 
> >
> > On Fri, Feb 24, 2017 at 11:25 AM, Nathan Johnson 
> wrote:
> >
> > > So not to re-open a can of worms, but I’m in a situation where I need
> > > to come up with a Marvin component test that depends on a template
> > > based on a kernel that’s relatively new, i.e., newer than Centos 5.3 /
> > Ubuntu 10.04 .
> > > I see openvm.eu has a suitable template (Ubuntu 16.0.4 for KVM), but
> > > from looking at the thread "Migrating CloudStack content from
> > > download.cloud.com” it looks like there is resistance to using this at
> > > least for hosting system vm templates over concerns of neutrality.
> > > Would this be suitable for a component test?  If not, what is a
> “blessed”
> > > template location?
> > >
> > > Thanks in advance!
> > >
> > > Nathan Johnson
> > > R Engineer
> > >
> > >
> > >
> > > 618 Grassmere Park Drive, Suite 12
> > > Nashville, TN 37211
> > > General Office: 615-312-6000
> > >
> > > website | blog | support
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> >
>

Re: [QUESTION] Upgrade path to JDK8

[Chiradeep Vittal]

You can build the system vm with  -source 1.7 -target 1.7 
Also unless you are using Java8 features (lambda) the classfiles produced by 
javac 8 should work in a 1.7 JVM

Sent from my iPhone

> On Feb 20, 2017, at 11:51 AM, Will Stevens  wrote:
> 
> yes, that is what I was expecting.  which is why I was asking about Wei's
> setup because he seems to have worked around that problem.  Or he has a
> custom SystemVM template running with both JDK7 and JDK8.
> 
> *Will STEVENS*
> Lead Developer
> 
> 
> 
>> On Mon, Feb 20, 2017 at 2:20 PM, Syed Ahmed  wrote:
>> 
>> The problem is that systemvm.iso is built with java 8 whereas java on the
>> VR is java 7
>>> On Mon, Feb 20, 2017 at 13:20 Will Stevens  wrote:
>>> 
>>> Did it work after resetting a VPC or when blowing away the SSVM or
>> CPVM?  I
>>> would not expect the SSVM or the CPVM to come up if the management server
>>> was built with JDK8 and the system vm template is only using JDK7.  Can
>> you
>>> confirm?​
>>> 
>>> *Will STEVENS*
>>> Lead Developer
>>> 
>>> 
>>> 
 On Mon, Feb 20, 2017 at 1:15 PM, Wei ZHOU  wrote:
 
 We've tested management server 4.7.1 with ubuntu 16.04/openjdk8 and
 systemvm 4.6 with debian7/openjdk7.
 The systemvms (ssvm, cpvm) work fine.
 
 I agree we need consider the openjdk upgrade in systemvm template.
 
 -Wei
 
 2017-02-20 18:15 GMT+01:00 Will Stevens :
 
> Regarding my question. Is it because of the version of Java that the
> systemvm.iso is built on?
> 
> On Feb 20, 2017 11:58 AM, "Will Stevens" 
>>> wrote:
> 
>> A question that is hidden in here is:
>> - Why does the JDK version on the management server have to match
>> the
 JDK
>> version of the System VM?
>> 
>> *Will STEVENS*
>> Lead Developer
>> 
>> 
>> 
>> On Mon, Feb 20, 2017 at 11:50 AM, Pierre-Luc Dion <
>>> pd...@cloudops.com>
>> wrote:
>> 
>>> Hi,
>>> 
>>> In the context of deployment of CloudStack with VPCs,
>>> What will happen to a cloud when upgrading to 4.10 that now use
>>> jdk8?
>>> 
>>> Does upgrading the management-server to 4.10 jdk8 and then keep
>> the
 old
>>> VRs
>>> run for a while that run on JDK7 will still work ?
>>> 
>>> Because If we upgrade the management-server to jdk8, we need to
>> keep
 VR
> to
>>> work until they get upgraded but we can't force an upgrade of VR
>>> just
>>> because the management-server is now using JDK8.
>>> 
>>> Thanks,
>>> 
>> 
>> 
> 
 
>>> 
>> 

Re: ACS - Some VMs unable to get DHCP IP from VR

[Chiradeep Vittal]

I wonder if there are more VMs than the range allowed in the guest network.
The excess VMs may not be getting IPs.
Check
/etc/dnsmasq.d/multiple_ranges.conf
/etc/dnsmasq.conf

Check this
http://lists.thekelleys.org.uk/pipermail/dnsmasq-discuss/2008q3/002301.html

You can also turn on logging on dnsmasq by adding log-dhcp to
/etc/dnsmasq.conf (and restarting dnsmasq)




On Mon, Nov 7, 2016 at 11:27 AM, Cloud List  wrote:

> Hi Wei,
>
> In addition,
>
> The VR is serving a shared not isolated network, meaning the IP it serves
> is 'guest' not 'public' IP. Will that make a difference on the iptables
> command we need to execute?
>
> Looking forward to your reply, thank you.
>
> Cheers.
>
>
> On Tue, Nov 8, 2016 at 3:19 AM, Cloud List  wrote:
>
> > Hi Wei and Ozhan,
> >
> > Thanks for your reply.
> >
> > The problem doesn't affect only Debian-based guest VMs, but also affected
> > some Windows and Ubuntu-based VMs as well. I have executed the command on
> > the VR and reset the NIC of the guest VM, but unfortunately the issue
> still
> > persists.
> >
> > iptables -t mangle -A POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM
> > --checksum-fill
> >
> > After issuing the above command on VR and reset the NIC on guest vm
> > (ifdown eth0, ifup eth0):
> >
> > On VR's /var/log/dnsmasq.log:
> >
> > Nov  7 19:09:22 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:b1:22:01:13:17
> > ignored
> > Nov  7 19:09:25 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:b1:22:01:13:17
> > ignored
> > Nov  7 19:09:30 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:b1:22:01:13:17
> > ignored
> > Nov  7 19:09:36 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0) 06:b1:22:01:13:17
> > ignored
> >
> > On the guest VM:
> >
> > root@vm:~# ifup eth0
> > Internet Systems Consortium DHCP Client 4.2.2
> > Copyright 2004-2011 Internet Systems Consortium.
> > All rights reserved.
> > For info, please visit https://www.isc.org/software/dhcp
> >
> > Listening on LPF/eth0/06:b1:22:01:13:17
> > Sending on LPF/eth0/06:b1:22:01:13:17
> > Sending on Socket/fallback
> > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 4
> > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 5
> > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 64DHCPDISCOVER
> on
> > eth0 to 255.255.255.255 port 67 interval 14
> > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 9
> > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 10
> > DHCPDISCOVER on eth0 to 255.255.255.255 port 67 interval 13
> > No DHCOFFERS received.
> > No working leases in persistent database - sleeping.
> >
> > I also tried to execute similar hotfix as mentioned on the bug report:
> >
> > iptables -A POSTROUTING -t mangle -p udp --dport bootpc -j CHECKSUM
> > --checksum-fill
> >
> > The problem still persists. Any further advice on this is highly
> > appreciated.
> >
> > Looking forward to your reply, thank you.
> >
> > Cheers.
> >
> >
> > On Tue, Nov 8, 2016 at 2:41 AM, Wei ZHOU  wrote:
> >
> >> GOOD point.
> >>
> >> @CloudList, can you try again after executing the following command in
> VR
> >> ?
> >>
> >> iptables -t mangle -A POSTROUTING -p udp -m udp --dport 68 -j CHECKSUM
> >> --checksum-fill
> >>
> >> -Wei
> >>
> >> 2016-11-07 14:46 GMT+01:00 Özhan Rüzgar Karaman <
> oruzgarkara...@gmail.com
> >> >:
> >>
> >> > Hi;
> >> > Whats your problematic vm's type is it Debian? Maybe you are affected
> >> from
> >> > the bug CLOUDSTACK-8326? I do not know if this bug has effected on ACS
> >> 4.2
> >> > release but i know that it effects release 4.8.x 4.9.x
> >> >
> >> > Thanks
> >> > Özhan
> >> >
> >> > On Mon, Nov 7, 2016 at 4:36 PM, Cloud List 
> wrote:
> >> >
> >> > > Hi Wei,
> >> > >
> >> > > Thanks for your reply.
> >> > >
> >> > > I checked and I can confirm that the mac address is listed on
> >> > > /etc/dhcphosts.txt in VR.
> >> > >
> >> > > For example:
> >> > >
> >> > > Nov  7 13:30:19 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
> >> 06:31:ac:01:13:YY
> >> > > ignored
> >> > > Nov  7 13:30:24 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
> >> 06:31:ac:01:13:YY
> >> > > ignored
> >> > > Nov  7 13:30:33 dnsmasq-dhcp[18986]: DHCPDISCOVER(eth0)
> >> 06:31:ac:01:13:YY
> >> > > ignored
> >> > >
> >> > > root@r-4155-VM:/var/log# grep 06:31:ac:01:13:f0 /etc/dhcphosts.txt
> >> > > 06:31:ac:01:13:YY,X.X.X.X,vm-name,infinite
> >> > >
> >> > > YY - last two digits of the mac address
> >> > > X.X.X.X - ip address which is supposed to be allocated to the VM
> >> > >
> >> > > Any advice how can I troubleshoot this further?
> >> > >
> >> > > Looking forward to your reply, thank you.
> >> > >
> >> > > Cheers.
> >> > >
> >> > >
> >> > > On Mon, Nov 7, 2016 at 9:22 PM, Wei ZHOU 
> >> wrote:
> >> > >
> >> > > > If the mac address is not listed in /etc/dhcphosts.txt in VR, the
> >> > request
> >> > > > will be ignored.
> >> > > >
> >> > > > Can you give more details so we can reproduce it and fix it ?
> >> > 

Re: Migrating CloudStack content from download.cloud.com

[Chiradeep Vittal]

Also, the default templates are bad defaults. For example, "CentOS
5.3(64-bit) no GUI (XenServer)" is severely out of date and is probably
riddled with hundreds of critical vulnerabilities. We should treat our
community as informed choosers and point them to
 - well known / frequently updated builds (e.g.,
http://dl.openvm.eu/cloudstack/)
 - instructions on how to build default templates (e.g., using Packer
https://www.packer.io/docs/builders/qemu.html )


On Fri, Nov 4, 2016 at 9:48 AM, Chiradeep Vittal <chirade...@gmail.com>
wrote:

> I am all for the change, but urge you to consider this:
>  - past releases are still in use and will be affected
>  - it would be good to decouple the release of the management server from
> the release of the templates. So, let's say there is a new 'SHELLACK' bug
> in bash which needs an update to the systemvm template -- today that
> requires a new CloudStack release to address it. Instead, it would be nice
> if you could just update the systemvm and builtin templates and ask people
> to re-run some scripts.
>
> A solution that would automate something like this: https://shankerbalan.
> net/blog/seed-cloudstack-templates-offline/ would solve for most cases.
>
> Instead of download.cloud.com in the blog post, you would have  list>.
>
>
> On Fri, Nov 4, 2016 at 5:56 AM, John Burwell <john.burw...@shapeblue.com>
> wrote:
>
>> Chiradeep,
>>
>> I am +1 to using a mirror list downloaded from
>> downloads.cloudstack.apache.org via https only (i.e.
>> https://downloads.cloudstack.apache.org/mirror.lst).  This approach
>> seems to be a common approach employed by other Apache projects that need
>> to provided downloadable assets.  Therefore, I would imagine it would be
>> something ASF Infra would be able to support.
>>
>> It also seems like we should have a hashes of a assets hosted on
>> downloads.cloudstack.apache.org in order to allow users to verify that
>> mirror has not been corrupted.
>>
>> Anyone opposed to targeting this change for the next round of releases --
>> 4.9.2.0, 4.10.1.0, and 4.11.0.0?
>>
>> Thanks,
>> -John
>>
>>
>> john.burw...@shapeblue.com
>> www.shapeblue.com
>> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
>> @shapeblue
>>
>>
>>
>> > On Nov 1, 2016, at 2:25 PM, Chiradeep Vittal <chirade...@gmail.com>
>> wrote:
>> >
>> > I believe that there is already a new site, but the software still
>> points
>> > to download.cloud.com.  There was also discussion about
>> > - building mirror sites instead of a SPOF site
>> > - eliminating the automatic download of templates and updating the
>> > documentation on use of offline templates.
>> > Older releases may not be changed, but users need to be notified and
>> > documentation needs to be updated and publicized.
>> >
>> > Thread here:
>> > http://mail-archives.apache.org/mod_mbox/cloudstack-dev/2016
>> 05.mbox/thread?4
>> >
>> > We should ask for the download.cloud.com site to be taken offline for
>> 24
>> > hours to see how many people scream, perhaps then the community will
>> swing
>> > into action?
>> >
>> >
>> >
>> > On Mon, Oct 31, 2016 at 1:49 PM, Rajesh Ramchandani <
>> > rajesh.ramchand...@accelerite.com> wrote:
>> >
>> >> Are we looking for new server to host this site?
>> >>
>> >>
>> >>
>> >> On Mon, Oct 31, 2016 at 12:46 PM -0700, "Chiradeep Vittal" <
>> >> chirade...@gmail.com<mailto:chirade...@gmail.com>> wrote:
>> >>
>> >> Bumping this. Any progress on this? download.cloud.com is not an
>> Apache
>> >> property and is subject to the whims of its owner (Citrix).
>> >>
>> >> On Mon, Jun 6, 2016 at 9:44 AM, Chiradeep Vittal <chirade...@gmail.com
>> >
>> >> wrote:
>> >>
>> >>> Any progress on this? There's nearly a 100K downloads of systemvm
>> >>> templates from download.cloud.com per month. Would be a shame to
>> leave
>> >>> these folks stranded.
>> >>>
>> >>> On Tue, May 31, 2016 at 4:49 AM, Paul Angus <paul.an...@shapeblue.com
>> >
>> >>> wrote:
>> >>>
>> >>>> +1.  we need to figure out the way forward, but this needs to be done
>> >>>> 'right' not just 'fast'.
>> >>>>
>> >>>>
>> >>>> paul.an...@shapeblue.com
>>

Re: Migrating CloudStack content from download.cloud.com

[Chiradeep Vittal]

I am all for the change, but urge you to consider this:
 - past releases are still in use and will be affected
 - it would be good to decouple the release of the management server from
the release of the templates. So, let's say there is a new 'SHELLACK' bug
in bash which needs an update to the systemvm template -- today that
requires a new CloudStack release to address it. Instead, it would be nice
if you could just update the systemvm and builtin templates and ask people
to re-run some scripts.

A solution that would automate something like this:
https://shankerbalan.net/blog/seed-cloudstack-templates-offline/ would
solve for most cases.

Instead of download.cloud.com in the blog post, you would have .


On Fri, Nov 4, 2016 at 5:56 AM, John Burwell <john.burw...@shapeblue.com>
wrote:

> Chiradeep,
>
> I am +1 to using a mirror list downloaded from
> downloads.cloudstack.apache.org via https only (i.e.
> https://downloads.cloudstack.apache.org/mirror.lst).  This approach seems
> to be a common approach employed by other Apache projects that need to
> provided downloadable assets.  Therefore, I would imagine it would be
> something ASF Infra would be able to support.
>
> It also seems like we should have a hashes of a assets hosted on
> downloads.cloudstack.apache.org in order to allow users to verify that
> mirror has not been corrupted.
>
> Anyone opposed to targeting this change for the next round of releases --
> 4.9.2.0, 4.10.1.0, and 4.11.0.0?
>
> Thanks,
> -John
>
>
> john.burw...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
> @shapeblue
>
>
>
> > On Nov 1, 2016, at 2:25 PM, Chiradeep Vittal <chirade...@gmail.com>
> wrote:
> >
> > I believe that there is already a new site, but the software still points
> > to download.cloud.com.  There was also discussion about
> > - building mirror sites instead of a SPOF site
> > - eliminating the automatic download of templates and updating the
> > documentation on use of offline templates.
> > Older releases may not be changed, but users need to be notified and
> > documentation needs to be updated and publicized.
> >
> > Thread here:
> > http://mail-archives.apache.org/mod_mbox/cloudstack-dev/
> 201605.mbox/thread?4
> >
> > We should ask for the download.cloud.com site to be taken offline for 24
> > hours to see how many people scream, perhaps then the community will
> swing
> > into action?
> >
> >
> >
> > On Mon, Oct 31, 2016 at 1:49 PM, Rajesh Ramchandani <
> > rajesh.ramchand...@accelerite.com> wrote:
> >
> >> Are we looking for new server to host this site?
> >>
> >>
> >>
> >> On Mon, Oct 31, 2016 at 12:46 PM -0700, "Chiradeep Vittal" <
> >> chirade...@gmail.com<mailto:chirade...@gmail.com>> wrote:
> >>
> >> Bumping this. Any progress on this? download.cloud.com is not an Apache
> >> property and is subject to the whims of its owner (Citrix).
> >>
> >> On Mon, Jun 6, 2016 at 9:44 AM, Chiradeep Vittal <chirade...@gmail.com>
> >> wrote:
> >>
> >>> Any progress on this? There's nearly a 100K downloads of systemvm
> >>> templates from download.cloud.com per month. Would be a shame to leave
> >>> these folks stranded.
> >>>
> >>> On Tue, May 31, 2016 at 4:49 AM, Paul Angus <paul.an...@shapeblue.com>
> >>> wrote:
> >>>
> >>>> +1.  we need to figure out the way forward, but this needs to be done
> >>>> 'right' not just 'fast'.
> >>>>
> >>>>
> >>>> paul.an...@shapeblue.com
> >>>> www.shapeblue.com<http://www.shapeblue.com>
> >>>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> >>>> @shapeblue
> >>>>
> >>>>
> >>>>
> >>>> -Original Message-----
> >>>> From: Raja Pullela [mailto:raja.pull...@accelerite.com]
> >>>> Sent: 31 May 2016 07:33
> >>>> To: CloudStack Dev <dev@cloudstack.apache.org>
> >>>> Subject: Re: Migrating CloudStack content from download.cloud.com
> >>>>
> >>>> +1, hope we will be able to discuss this at the Collab this week and
> >>>> decide on the next steps.
> >>>>
> >>>> best,
> >>>> Raja Pullela
> >>>> Senior Manager, Product Development
> >>>> Accelerate,
> >>>> 2055,Laurelwood Road,  Santa Clara, CA 95054, USA
> >>>> Phone: 1-

Re: Migrating CloudStack content from download.cloud.com

[Chiradeep Vittal]

I believe that there is already a new site, but the software still points
to download.cloud.com.  There was also discussion about
 - building mirror sites instead of a SPOF site
 - eliminating the automatic download of templates and updating the
documentation on use of offline templates.
Older releases may not be changed, but users need to be notified and
documentation needs to be updated and publicized.

Thread here:
http://mail-archives.apache.org/mod_mbox/cloudstack-dev/201605.mbox/thread?4

We should ask for the download.cloud.com site to be taken offline for 24
hours to see how many people scream, perhaps then the community will swing
into action?



On Mon, Oct 31, 2016 at 1:49 PM, Rajesh Ramchandani <
rajesh.ramchand...@accelerite.com> wrote:

> Are we looking for new server to host this site?
>
>
>
> On Mon, Oct 31, 2016 at 12:46 PM -0700, "Chiradeep Vittal" <
> chirade...@gmail.com<mailto:chirade...@gmail.com>> wrote:
>
> Bumping this. Any progress on this? download.cloud.com is not an Apache
> property and is subject to the whims of its owner (Citrix).
>
> On Mon, Jun 6, 2016 at 9:44 AM, Chiradeep Vittal <chirade...@gmail.com>
> wrote:
>
> > Any progress on this? There's nearly a 100K downloads of systemvm
> > templates from download.cloud.com per month. Would be a shame to leave
> > these folks stranded.
> >
> > On Tue, May 31, 2016 at 4:49 AM, Paul Angus <paul.an...@shapeblue.com>
> > wrote:
> >
> >> +1.  we need to figure out the way forward, but this needs to be done
> >> 'right' not just 'fast'.
> >>
> >>
> >> paul.an...@shapeblue.com
> >> www.shapeblue.com<http://www.shapeblue.com>
> >> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> >> @shapeblue
> >>
> >>
> >>
> >> -Original Message-
> >> From: Raja Pullela [mailto:raja.pull...@accelerite.com]
> >> Sent: 31 May 2016 07:33
> >> To: CloudStack Dev <dev@cloudstack.apache.org>
> >> Subject: Re: Migrating CloudStack content from download.cloud.com
> >>
> >> +1, hope we will be able to discuss this at the Collab this week and
> >> decide on the next steps.
> >>
> >> best,
> >> Raja Pullela
> >> Senior Manager, Product Development
> >> Accelerate,
> >> 2055,Laurelwood Road,  Santa Clara, CA 95054, USA
> >> Phone: 1-408-216-7010,  www.accelerite.com,@accelerite<http://www.
> accelerite.com,@accelerite>
> >>
> >> > On May 31, 2016, at 10:23 AM, Chiradeep Vittal <chirade...@gmail.com>
> >> wrote:
> >> >
> >> > I hope this gets discussed during the CloudStack Collab over the next
> >> > few days. Again, I'd urge everybody to consider: "what if
> >> > download.cloud.com went away next week". Waiting till some
> >> > hypothetical last date only means that we will scramble a week before
> >> that last date.
> >> >
> >> > On Fri, May 20, 2016 at 5:42 PM, Chiradeep Vittal
> >> > <chirade...@gmail.com>
> >> > wrote:
> >> >
> >> >> Yes, the mirror site would be on github or apache.org
> >> >>
> >> >> Step 6 in the install guide would have instructions like:
> >> >> a. Install System VM Templates:
> >> >> <here, cloud-install-sys-tmplt has been modified to use the mirror
> >> >> list> b. Installing other templates cloud-install-tmplt
> >> >>> enter OS (linux only)
> >> >>> Ubuntu 16.04
> >> >>   Installing...
> >> >>
> >> >> Or, step (b) could generate a cloudmonkey script.
> >> >>
> >> >>
> >> >> On Fri, May 20, 2016 at 1:57 PM, Will Stevens <wstev...@cloudops.com
> >
> >> >> wrote:
> >> >>
> >> >>> Cant we just host the mirror list in apache.org and then actually
> >> >>> host the mirrors in different places around the world?  A company
> >> >>> could sponsor the few bucks a month for AWS and have one of the
> >> >>> mirrors be in AWS and the mirror list in apache.org would just be
> >> >>> updated to add the AWS mirror.
> >> >>>
> >> >>> Isn't that basically what we are proposing here?  The ability to
> >> >>> have different mirrors with the 'official' endpoint being in an
> >> >>> apache.org endpoint to list the mirrors?
> >> >>>
> >> >>>

Re: Migrating CloudStack content from download.cloud.com

[Chiradeep Vittal]

Bumping this. Any progress on this? download.cloud.com is not an Apache
property and is subject to the whims of its owner (Citrix).

On Mon, Jun 6, 2016 at 9:44 AM, Chiradeep Vittal <chirade...@gmail.com>
wrote:

> Any progress on this? There's nearly a 100K downloads of systemvm
> templates from download.cloud.com per month. Would be a shame to leave
> these folks stranded.
>
> On Tue, May 31, 2016 at 4:49 AM, Paul Angus <paul.an...@shapeblue.com>
> wrote:
>
>> +1.  we need to figure out the way forward, but this needs to be done
>> 'right' not just 'fast'.
>>
>>
>> paul.an...@shapeblue.com
>> www.shapeblue.com
>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> @shapeblue
>>
>>
>>
>> -Original Message-
>> From: Raja Pullela [mailto:raja.pull...@accelerite.com]
>> Sent: 31 May 2016 07:33
>> To: CloudStack Dev <dev@cloudstack.apache.org>
>> Subject: Re: Migrating CloudStack content from download.cloud.com
>>
>> +1, hope we will be able to discuss this at the Collab this week and
>> decide on the next steps.
>>
>> best,
>> Raja Pullela
>> Senior Manager, Product Development
>> Accelerate,
>> 2055,Laurelwood Road,  Santa Clara, CA 95054, USA
>> Phone: 1-408-216-7010,  www.accelerite.com,@accelerite
>>
>> > On May 31, 2016, at 10:23 AM, Chiradeep Vittal <chirade...@gmail.com>
>> wrote:
>> >
>> > I hope this gets discussed during the CloudStack Collab over the next
>> > few days. Again, I'd urge everybody to consider: "what if
>> > download.cloud.com went away next week". Waiting till some
>> > hypothetical last date only means that we will scramble a week before
>> that last date.
>> >
>> > On Fri, May 20, 2016 at 5:42 PM, Chiradeep Vittal
>> > <chirade...@gmail.com>
>> > wrote:
>> >
>> >> Yes, the mirror site would be on github or apache.org
>> >>
>> >> Step 6 in the install guide would have instructions like:
>> >> a. Install System VM Templates:
>> >> <here, cloud-install-sys-tmplt has been modified to use the mirror
>> >> list> b. Installing other templates cloud-install-tmplt
>> >>> enter OS (linux only)
>> >>> Ubuntu 16.04
>> >>   Installing...
>> >>
>> >> Or, step (b) could generate a cloudmonkey script.
>> >>
>> >>
>> >> On Fri, May 20, 2016 at 1:57 PM, Will Stevens <wstev...@cloudops.com>
>> >> wrote:
>> >>
>> >>> Cant we just host the mirror list in apache.org and then actually
>> >>> host the mirrors in different places around the world?  A company
>> >>> could sponsor the few bucks a month for AWS and have one of the
>> >>> mirrors be in AWS and the mirror list in apache.org would just be
>> >>> updated to add the AWS mirror.
>> >>>
>> >>> Isn't that basically what we are proposing here?  The ability to
>> >>> have different mirrors with the 'official' endpoint being in an
>> >>> apache.org endpoint to list the mirrors?
>> >>>
>> >>> *Will STEVENS*
>> >>> Lead Developer
>> >>>
>> >>> *CloudOps* *| *Cloud Solutions Experts
>> >>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>> >>> tw @CloudOps_
>> >>>
>> >>> On Fri, May 20, 2016 at 1:17 PM, Daan Hoogland
>> >>> <daan.hoogl...@gmail.com>
>> >>> wrote:
>> >>>
>> >>>> is there no way, within foundation bylaws, to host this somewhere
>> >>>> on apache.org? we are using peoples wibesites for tests as well,
>> >>>> aren't
>> >>> we.
>> >>>>
>> >>>> On Fri, May 20, 2016 at 7:04 PM, Rafael Weingärtner <
>> >>>> rafaelweingart...@gmail.com> wrote:
>> >>>>
>> >>>>> We could host a mirror here. We are in South America, Brazil.
>> >>>>>
>> >>>>> On Fri, May 20, 2016 at 2:01 PM, Chiradeep Vittal <
>> >>> chirade...@gmail.com>
>> >>>>> wrote:
>> >>>>>
>> >>>>>> I was proposing a new S3 bucket earlier
>> >>>>>> (templates.cloudstack.org),
>> >>>> but
>> >>>>> I
>> >>>>>>

Re: [DISCUSS] Replacing the VR

[Chiradeep Vittal]

I rather doubt that the Cloudrouter will fit the needs of the CloudStack
project
 - it is AGPL licensed. Many enterprises will not touch anything that has
AGPL
 - the github repo shows rather infrequent updates. Quite likely they
aren't considering the use cases of the CloudStack community

I'd back John B's comments on disaggregating the VR. Split it into many
docker containers
 - password server
 - userdata server
 - DHCP / DNS
 - s2s VPN
 - RA VPN
 - intra-VPC routing and ACL
 - Port forwarding + NAT
 - FW
 - LB (public)
 - LB (internal),
 - secondary storage
 - agent
Glue them together with  docker compose files (one per use case - basic
zone, isolated, VPC, SSVM, etc).

The VR image then becomes a JeOS + docker. You can test each of the
components independently and fixing one bug in the field (say DHCP) is
hitless to the other components. You don't need to build per-hypervisor
VRs. You could even run on baremetal.

Along the way you need to figure out how to
 - make the traffic traverse the containers that are needed to be traversed
(in most cases just 1)
 - bootstrap the router (how does it find its compose file? where is the
registry?)
 - rethink the command and control of the VR functions. SSH works, but
something more declarative, idempotent should be explored.

As you do this, it becomes clearer which of the functions can be
substituted by for example CloudRouter. Command and Control of the docker
containers can be moved out to another container. Etc.







On Wed, Sep 14, 2016 at 12:59 AM, Marty Godsey  wrote:

> This one does look nice. My biggest concern is the lack of VXLANs. It
> seems that any of the ones we mentioned do not have an API so we may be
> stuck at the SSH method.
>
> Regards,
> Marty Godsey
> nSource Solutions
>
> -Original Message-
> From: Abhinandan Prateek [mailto:abhinandan.prat...@shapeblue.com]
> Sent: Wednesday, September 14, 2016 2:26 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Replacing the VR
>
> Cloudrouter looks promising. These have potential to save future
> engineering effort for example on ipv6 routing, OSPF etc.
> And the best part is they come with test automation framework.
>
>
>
>
>
> On 13/09/16, 4:22 PM, "Jayapal Uradi" 
> wrote:
>
> >Hi,
> >
> >Instead of replacing the VR in first place we should add VyOS/cloudrouter
> as provider. Once it is stable, network offerings (on upgrade) can be
> updated to use it and we can drop the VR if we want at that release onwards.
> >
> >VR is stabilized over a period of time and some of them are running
> without issues.  When we replicate the ACS VR features in new solution it
> takes some to find the missing pieces (hidden bugs).
> >
> >Thanks,
> >Jayapal
> >
> >> On Sep 13, 2016, at 2:52 PM, Nux! <
> >
> >> n...@li.nux.ro> wrote:
> >>
> >> Hi,
> >>
> >> I like the idea.
> >>
> >> Cloudrouter looks really promising, I'm not too keen on VyOS (it
> doesn't have a proper http api etc).
> >>
> >> --
> >> Sent from the Delta quadrant using Borg technology!
> >>
> >> Nux!
> >> www.nux.ro
> >>
> >>
> abhinandan.prat...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>
>
>
> - Original Message -
> >>> From: "Will Stevens" 
> >>> To: dev@cloudstack.apache.org
> >>> Sent: Monday, 12 September, 2016 21:20:11
> >>> Subject: [DISCUSS] Replacing the VR
> >>
> >>> *Disclaimer:* This is a thought experiment and should be treated as
> such.
> >>> Please weigh in with the good and bad of this idea...
> >>>
> >>> A couple of us have been discussing the idea of potentially
> >>> replacing the ACS VR with the VyOS [1] (Open Source Vyatta VM).
> >>> There may be a license issue because I think it is licensed under
> >>> GPL, but for the sake of discussion, let's assume we can overcome any
> license issues.
> >>>
> >>> I have spent some time recently with the VyOS and I have to admit, I
> >>> was pretty impressed.  It is simple and intuitive and it gives you a
> >>> lot more options for auditing the configuration etc...
> >>>
> >>> Items of potential interest:
> >>> - Clean up our current VR script spaghetti to a simpler more
> >>> auditable configuration workflow.
> >>> - Gives a cleaner path for IPv6 support.
> >>> - Handles VPN configuration via the same configuration interface.
> >>> - Support for OSPF & BGP.
> >>> - VPN support through OpenVPN & StrongSwan.
> >>> - Easily supports HA (redundant routers) through VRRP.
> >>> - VXLAN support.
> >>> - Transaction based changes to the VR with rollback on error.
> >>>
> >>> Items that could be difficult to solve:
> >>> - Userdata password reset workflow and implementation.
> >>> - Upgrade process.
> >>>
> >>> The VyOS is not the only option if we were to consider this approach.
> >>> Another option, which I don't know as well, would be CloudRouter
> >>> (AGPL
> >>> license) [2] which is purely API driven.
> >>>
> >>> Anyway, would 

Re: [VOTE] Apache Cloudstack 4.8.2 RC1

[Chiradeep Vittal]

With this release, is ACS still dependent on downloads.cloud.com in any way?

On Mon, Aug 8, 2016 at 7:31 AM, Will Stevens 
wrote:

> I have pushed a new vote for RC2.  Sorry for the mix up.  The release
> should have been 4.8.1 and I made a mistake and put it as 4.8.2 for RC1.
>
> Sorry for the confusion...
>
> On Fri, Aug 5, 2016 at 9:44 PM, Will Stevens 
> wrote:
>
> > Yes. I think I should stop the vote and fix this. Thx...
> >
> > On Aug 5, 2016 7:56 PM, "Milamber"  wrote:
> >
> >> if I change the version in ./tools/checkstyle/pom.xml to 4.8.2, I can
> >> build the debian packages successfully.
> >> Note I use the git tag 4.8.2RC1 to build, but the tar.bz2 archives have
> >> the 4.8.1 version for checkstyle pom.xml file too.
> >>
> >> Perhaps, Will, you must stop the vote, fixes, and launch a RC2?
> >>
> >> One question: this issue exist with the CentOS build?
> >>
> >>
> >>
> >> On 05/08/2016 20:50, Milamber wrote:
> >>
> >>
> >> On 05/08/2016 22:45, Will Stevens wrote:
> >>
> >>> I will look into this.  Thanks...
> >>>
> >>> *Will STEVENS*
> >>> Lead Developer
> >>>
> >>> *CloudOps* *| *Cloud Solutions Experts
> >>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> >>> w cloudops.com *|* tw @CloudOps_
> >>>
> >>> On Fri, Aug 5, 2016 at 3:50 PM, Milamber  wrote:
> >>>
> >>> Hello,
> 
>  I have this error when I try to build the debian packages (on ubuntu
>  14.04)
> 
>  [INFO] --- maven-remote-resources-plugin:1.3:process (default) @
>  cloudstack-framework ---
>  [INFO] Building jar: /root/CS/cloudstack-dev/tools/
>  checkstyle/target/checkstyle-4.8.1.jar
> 
>  [...]
> 
>  [ERROR] Failed to execute goal org.apache.maven.plugins:maven
>  -checkstyle-plugin:2.11:check
>  (cloudstack-checkstyle) on project cloud-maven-standard: Execution
>  cloudstack-checkstyle of goal org.apache.maven.plugins:maven
>  -checkstyle-plugin:2.11:check
>  failed: Plugin org.apache.maven.plugins:maven-checkstyle-plugin:2.11
> or
>  one of its dependencies could not be resolved: Failure to find
>  org.apache.cloudstack:checkstyle:jar:4.8.2 in
>  http://repo.maven.apache.org/maven2 was cached in the local
> repository,
>  resolution will not be reattempted until the update interval of
> central
>  has
>  elapsed or updates are forced -> [Help 1]
> 
> 
>  The version number 4.8.1 still in ./tools/checkstyle/pom.xml
> 
>  Probably the same issue with :
>  ./services/iam/server/pom.xml
>  ./services/iam/plugin/pom.xml
>  ./services/console-proxy/plugin/pom.xml
> 
> 
>  Milamber
> 
> 
> 
>  On 05/08/2016 18:51, Will Stevens wrote:
> 
>  Bump...
> >
> > On Thu, Aug 4, 2016 at 11:38 AM, Will Stevens <
> > williamstev...@gmail.com>
> > wrote:
> >
> > Link didn't get auto-created correctly, so here it is again for
> > everyone
> >
> >> who is using something like gmail as a client:
> >>
> >> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=s
> >> hortlog;h=refs/heads/4.8.2-RC20160804T1130
> >>
> >> On Thu, Aug 4, 2016 at 11:37 AM, Will Stevens <
> >> williamstev...@gmail.com>
> >> wrote:
> >>
> >> Hi All,
> >>
> >>> I've created a 4.8.2 release, with the following artifacts up for a
> >>> vote:
> >>>
> >>> Git Branch and Commit SH:
> >>> https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=s
> >>> hortlog;h=refs/heads/4.8.2-RC20160804T1130
> >>> Commit: d6fdde04357625e4d2d8b5ef09aafd2a0741dc17
> >>>
> >>> List of changes will be produced with the release notes once the RC
> >>> passes.
> >>>
> >>> Source release (checksums and signatures are available at the same
> >>> location):
> >>> https://dist.apache.org/repos/dist/dev/cloudstack/4.8.2/
> >>>
> >>> PGP release keys (signed using CB818F64):
> >>> https://dist.apache.org/repos/dist/release/cloudstack/KEYS
> >>>
> >>> Vote will be open for 72 hours.
> >>>
> >>> For sanity in tallying the vote, can PMC members please be sure to
> >>> indicate "(binding)" with their vote?
> >>>
> >>> [ ] +1  approve
> >>> [ ] +0  no opinion
> >>> [ ] -1  disapprove (and reason why)
> >>>
> >>>
> >>>
> >>
>

Re: cloud.com references

[Chiradeep Vittal]

RE: CNAME from download.cloudstack.org to cloudstack.apt-get.eu
 - Should this be a request to Apache Infra?

On Tue, Jun 7, 2016 at 11:49 AM, Wido den Hollander  wrote:

>
> > Op 7 juni 2016 om 19:29 schreef Daan Hoogland :
> >
> >
> > Paul, I have no access to any dns record under cloudstack.org or
> apache.org
> > so if you do, please give me the relevant name. I will use
> > cloudstack.apt-get.eu untill someone guarantees me access to something
> else
> > for which I can create a cname record to point to whatever. I will ignore
> > this -1 if it will not be acompanied with such an alternative.
> >
>
> I am all in favor of a CNAME towards apt-get.eu btw :) Just offering it
> as a service, not because I want my domain name in there.
>
> Wido
>
> > On Tue, Jun 7, 2016 at 5:54 PM, Paul Angus 
> wrote:
> >
> > > -1
> > >
> > > Using apt-get is still an external dependency.  If the hardcoded value
> is
> > > an Apache/CloudStack domain we can always repoint as required.
> > >
> > >
> > >
> > > Kind regards,
> > >
> > > Paul Angus
> > >
> > > paul.an...@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > > @shapeblue
> > >
> > >
> > >
> > > -Original Message-
> > > From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
> > > Sent: 07 June 2016 16:15
> > > To: dev 
> > > Subject: Re: cloud.com references
> > >
> > > I grepped the source for all occurances of download.cloud.com and did
> $
> > > wget -P `dirname $file | cut -d '/' -f 4-` $file making the best
> mirror of
> > > download.com that I could. I think having the domain and pointing it
> to
> > > our repo is easiest but a possible next step is also changing all to
> our
> > > new domain. I did all did on cloudstack.apt-get.eu so for now I will
> add
> > > a commit to my PR to change the urls to apt-get.
> > >
> > > gotcha's:
> > > 1: I don't feel sure this makes for everything
> > > 2: apt-get is only a slightly bigger ensurance for continuity so we
> would
> > > want this to be a replacement string (e.g. templated)
> > > 3: problably as well
> > >
> > >
> > > so far, more to follow
> > >
> > >
> > > On Tue, Jun 7, 2016 at 4:09 PM, Paul Angus 
> > > wrote:
> > >
> > > > I *think* that we got to a general consensus that the ideal way
> > > > forward would be to host a mirror list at download.cloudstack.org on
> > > Apache infra.
> > > >
> > > > For existing systemVMs and templates I guess we could use
> > > > archive.cloudstack.org pointed at apt-get..?
> > > >
> > > >
> > > > Kind regards,
> > > >
> > > > Paul Angus
> > > >
> > > > paul.an...@shapeblue.com
> > > > www.shapeblue.com
> > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> > > >
> > > >
> > > >
> > > > -Original Message-
> > > > From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
> > > > Sent: 07 June 2016 15:00
> > > > To: dev 
> > > > Subject: Re: cloud.com references
> > > >
> > > > I would say of course if I didn't feel I don't see your concern,
> Will.
> > > Can
> > > > you explain?
> > > >
> > > > Biligual auto correct use.  Read at your own risico Op 7 jun. 2016
> 3:33
> > > PM
> > > > schreef "Will Stevens" :
> > > >
> > > > > I am not sure, but I would make sure we have a strategy to solve
> for
> > > > > all of the different URLs we are deprecating...
> > > > >
> > > > > *Will STEVENS*
> > > > > Lead Developer
> > > > >
> > > > > *CloudOps* *| *Cloud Solutions Experts
> > > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com
> *|* tw
> > > > > @CloudOps_
> > > > >
> > > > > On Tue, Jun 7, 2016 at 4:12 AM, Daan Hoogland
> > > > > 
> > > > > wrote:
> > > > >
> > > > > > during a search on refs to cloud.com i found
> > > > > > http://download.cloud.com.s3.amazonaws.com/ a couple of times
> in the
> > > > > > devcloud and devcloud4 projects dirs.
> > > > > > Ar people still using those or could we remove them? Or maybe
> > > > > > extract and put in a sepparate project?
> > > > > >
> > > > > > --
> > > > > > Daan
> > > > > >
> > > > >
> > > >
> > >
> > >
> > >
> > > --
> > > Daan
> > >
> >
> >
> >
> > --
> > Daan
>

Re: Migrating CloudStack content from download.cloud.com

[Chiradeep Vittal]

Any progress on this? There's nearly a 100K downloads of systemvm templates
from download.cloud.com per month. Would be a shame to leave these folks
stranded.

On Tue, May 31, 2016 at 4:49 AM, Paul Angus <paul.an...@shapeblue.com>
wrote:

> +1.  we need to figure out the way forward, but this needs to be done
> 'right' not just 'fast'.
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> -Original Message-
> From: Raja Pullela [mailto:raja.pull...@accelerite.com]
> Sent: 31 May 2016 07:33
> To: CloudStack Dev <dev@cloudstack.apache.org>
> Subject: Re: Migrating CloudStack content from download.cloud.com
>
> +1, hope we will be able to discuss this at the Collab this week and
> decide on the next steps.
>
> best,
> Raja Pullela
> Senior Manager, Product Development
> Accelerate,
> 2055,Laurelwood Road,  Santa Clara, CA 95054, USA
> Phone: 1-408-216-7010,  www.accelerite.com,@accelerite
>
> > On May 31, 2016, at 10:23 AM, Chiradeep Vittal <chirade...@gmail.com>
> wrote:
> >
> > I hope this gets discussed during the CloudStack Collab over the next
> > few days. Again, I'd urge everybody to consider: "what if
> > download.cloud.com went away next week". Waiting till some
> > hypothetical last date only means that we will scramble a week before
> that last date.
> >
> > On Fri, May 20, 2016 at 5:42 PM, Chiradeep Vittal
> > <chirade...@gmail.com>
> > wrote:
> >
> >> Yes, the mirror site would be on github or apache.org
> >>
> >> Step 6 in the install guide would have instructions like:
> >> a. Install System VM Templates:
> >> <here, cloud-install-sys-tmplt has been modified to use the mirror
> >> list> b. Installing other templates cloud-install-tmplt
> >>> enter OS (linux only)
> >>> Ubuntu 16.04
> >>   Installing...
> >>
> >> Or, step (b) could generate a cloudmonkey script.
> >>
> >>
> >> On Fri, May 20, 2016 at 1:57 PM, Will Stevens <wstev...@cloudops.com>
> >> wrote:
> >>
> >>> Cant we just host the mirror list in apache.org and then actually
> >>> host the mirrors in different places around the world?  A company
> >>> could sponsor the few bucks a month for AWS and have one of the
> >>> mirrors be in AWS and the mirror list in apache.org would just be
> >>> updated to add the AWS mirror.
> >>>
> >>> Isn't that basically what we are proposing here?  The ability to
> >>> have different mirrors with the 'official' endpoint being in an
> >>> apache.org endpoint to list the mirrors?
> >>>
> >>> *Will STEVENS*
> >>> Lead Developer
> >>>
> >>> *CloudOps* *| *Cloud Solutions Experts
> >>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> >>> tw @CloudOps_
> >>>
> >>> On Fri, May 20, 2016 at 1:17 PM, Daan Hoogland
> >>> <daan.hoogl...@gmail.com>
> >>> wrote:
> >>>
> >>>> is there no way, within foundation bylaws, to host this somewhere
> >>>> on apache.org? we are using peoples wibesites for tests as well,
> >>>> aren't
> >>> we.
> >>>>
> >>>> On Fri, May 20, 2016 at 7:04 PM, Rafael Weingärtner <
> >>>> rafaelweingart...@gmail.com> wrote:
> >>>>
> >>>>> We could host a mirror here. We are in South America, Brazil.
> >>>>>
> >>>>> On Fri, May 20, 2016 at 2:01 PM, Chiradeep Vittal <
> >>> chirade...@gmail.com>
> >>>>> wrote:
> >>>>>
> >>>>>> I was proposing a new S3 bucket earlier
> >>>>>> (templates.cloudstack.org),
> >>>> but
> >>>>> I
> >>>>>> realized that someone needs to own the account. There is no legal
> >>>> entity
> >>>>>> for Apache CloudStack (except the ASF) that can enter into a
> >>> contract
> >>>>> with
> >>>>>> AWS (even if it is for a few bucks a month).  To make this work,
> >>>>>> alternatives:
> >>>>>> (1) ASF Infra creates this bucket and IAM keys to the PMC to
> >>> operate
> >>>>> this
> >>>>>> bucket. ACS users might want to make targeted donations to
> >>>>>> support
>

Re: Migrating CloudStack content from download.cloud.com

[Chiradeep Vittal]

I hope this gets discussed during the CloudStack Collab over the next few
days. Again, I'd urge everybody to consider: "what if download.cloud.com
went away next week". Waiting till some hypothetical last date only means
that we will scramble a week before that last date.

On Fri, May 20, 2016 at 5:42 PM, Chiradeep Vittal <chirade...@gmail.com>
wrote:

> Yes, the mirror site would be on github or apache.org
>
> Step 6 in the install guide would have instructions like:
> a. Install System VM Templates:
> <here, cloud-install-sys-tmplt has been modified to use the mirror list>
> b. Installing other templates
> cloud-install-tmplt
> > enter OS (linux only)
>> Ubuntu 16.04
>Installing...
>
> Or, step (b) could generate a cloudmonkey script.
>
>
> On Fri, May 20, 2016 at 1:57 PM, Will Stevens <wstev...@cloudops.com>
> wrote:
>
>> Cant we just host the mirror list in apache.org and then actually host
>> the
>> mirrors in different places around the world?  A company could sponsor the
>> few bucks a month for AWS and have one of the mirrors be in AWS and the
>> mirror list in apache.org would just be updated to add the AWS mirror.
>>
>> Isn't that basically what we are proposing here?  The ability to have
>> different mirrors with the 'official' endpoint being in an apache.org
>> endpoint to list the mirrors?
>>
>> *Will STEVENS*
>> Lead Developer
>>
>> *CloudOps* *| *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> w cloudops.com *|* tw @CloudOps_
>>
>> On Fri, May 20, 2016 at 1:17 PM, Daan Hoogland <daan.hoogl...@gmail.com>
>> wrote:
>>
>> > is there no way, within foundation bylaws, to host this somewhere on
>> > apache.org? we are using peoples wibesites for tests as well, aren't
>> we.
>> >
>> > On Fri, May 20, 2016 at 7:04 PM, Rafael Weingärtner <
>> > rafaelweingart...@gmail.com> wrote:
>> >
>> > > We could host a mirror here. We are in South America, Brazil.
>> > >
>> > > On Fri, May 20, 2016 at 2:01 PM, Chiradeep Vittal <
>> chirade...@gmail.com>
>> > > wrote:
>> > >
>> > > > I was proposing a new S3 bucket earlier (templates.cloudstack.org),
>> > but
>> > > I
>> > > > realized that someone needs to own the account. There is no legal
>> > entity
>> > > > for Apache CloudStack (except the ASF) that can enter into a
>> contract
>> > > with
>> > > > AWS (even if it is for a few bucks a month).  To make this work,
>> > > > alternatives:
>> > > >  (1) ASF Infra creates this bucket and IAM keys to the PMC to
>> operate
>> > > this
>> > > > bucket. ACS users might want to make targeted donations to support
>> this
>> > > > bucket, but I don't believe that it is possible.
>> > > >
>> > > > (2) Cloudstack.org gets registered as a (non-profit?) entity etc and
>> > ACS
>> > > > users make donations to this entity. PMC has control over the
>> bucket.
>> > > >
>> > > > A mirror list and a python script (e.g., [1]) sounds like a much
>> better
>> > > > idea :)
>> > > >
>> > > > Raise of hands:
>> > > > Who is willing to host a mirror of systemvm templates and default
>> > > > templates. I figure we need at least 5, preferably all over the
>> world,
>> > > > especially in East Asia.
>> > > >
>> > > > [1]
>> > > >
>> > > >
>> > >
>> >
>> http://code.activestate.com/recipes/284631-a-python-script-to-test-download-mirrors/
>> > > >
>> > > >
>> > > >
>> > > >
>> > > >
>> > > > On Fri, May 20, 2016 at 9:52 AM, Chiradeep Vittal <
>> > chirade...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Agree with Daan : we shouldn't be pushing these ancient insecure
>> > > > > templates!
>> > > > >
>> > > > >
>> > > > > On Fri, May 20, 2016 at 9:51 AM, Chiradeep Vittal <
>> > > chirade...@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > >> :) According to Raja, there is some time before
>> download.cloud.com
>> > is
>> > > > >> doomed. Let's pretend that date i

Re: Migrating CloudStack content from download.cloud.com

[Chiradeep Vittal]

Yes, the mirror site would be on github or apache.org

Step 6 in the install guide would have instructions like:
a. Install System VM Templates:
<here, cloud-install-sys-tmplt has been modified to use the mirror list>
b. Installing other templates
cloud-install-tmplt
> enter OS (linux only)
   > Ubuntu 16.04
   Installing...

Or, step (b) could generate a cloudmonkey script.


On Fri, May 20, 2016 at 1:57 PM, Will Stevens <wstev...@cloudops.com> wrote:

> Cant we just host the mirror list in apache.org and then actually host the
> mirrors in different places around the world?  A company could sponsor the
> few bucks a month for AWS and have one of the mirrors be in AWS and the
> mirror list in apache.org would just be updated to add the AWS mirror.
>
> Isn't that basically what we are proposing here?  The ability to have
> different mirrors with the 'official' endpoint being in an apache.org
> endpoint to list the mirrors?
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Fri, May 20, 2016 at 1:17 PM, Daan Hoogland <daan.hoogl...@gmail.com>
> wrote:
>
> > is there no way, within foundation bylaws, to host this somewhere on
> > apache.org? we are using peoples wibesites for tests as well, aren't we.
> >
> > On Fri, May 20, 2016 at 7:04 PM, Rafael Weingärtner <
> > rafaelweingart...@gmail.com> wrote:
> >
> > > We could host a mirror here. We are in South America, Brazil.
> > >
> > > On Fri, May 20, 2016 at 2:01 PM, Chiradeep Vittal <
> chirade...@gmail.com>
> > > wrote:
> > >
> > > > I was proposing a new S3 bucket earlier (templates.cloudstack.org),
> > but
> > > I
> > > > realized that someone needs to own the account. There is no legal
> > entity
> > > > for Apache CloudStack (except the ASF) that can enter into a contract
> > > with
> > > > AWS (even if it is for a few bucks a month).  To make this work,
> > > > alternatives:
> > > >  (1) ASF Infra creates this bucket and IAM keys to the PMC to operate
> > > this
> > > > bucket. ACS users might want to make targeted donations to support
> this
> > > > bucket, but I don't believe that it is possible.
> > > >
> > > > (2) Cloudstack.org gets registered as a (non-profit?) entity etc and
> > ACS
> > > > users make donations to this entity. PMC has control over the bucket.
> > > >
> > > > A mirror list and a python script (e.g., [1]) sounds like a much
> better
> > > > idea :)
> > > >
> > > > Raise of hands:
> > > > Who is willing to host a mirror of systemvm templates and default
> > > > templates. I figure we need at least 5, preferably all over the
> world,
> > > > especially in East Asia.
> > > >
> > > > [1]
> > > >
> > > >
> > >
> >
> http://code.activestate.com/recipes/284631-a-python-script-to-test-download-mirrors/
> > > >
> > > >
> > > >
> > > >
> > > >
> > > > On Fri, May 20, 2016 at 9:52 AM, Chiradeep Vittal <
> > chirade...@gmail.com>
> > > > wrote:
> > > >
> > > > > Agree with Daan : we shouldn't be pushing these ancient insecure
> > > > > templates!
> > > > >
> > > > >
> > > > > On Fri, May 20, 2016 at 9:51 AM, Chiradeep Vittal <
> > > chirade...@gmail.com>
> > > > > wrote:
> > > > >
> > > > >> :) According to Raja, there is some time before
> download.cloud.com
> > is
> > > > >> doomed. Let's pretend that date is Jan 1 2017. Now pretend that it
> > is
> > > > Dec
> > > > >> 25, 2016 and we're still debating mirror list vs steps and what
> not.
> > > > Now,
> > > > >> just change the 2017 to 2016 and re-consider. We need this done
> > > > yesterday!
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >>
> > > > >> On Fri, May 20, 2016 at 1:53 AM, Daan Hoogland <
> > > daan.hoogl...@gmail.com
> > > > >
> > > > >> wrote:
> > > > >>
> > > > >>> send by tablet, no warranties in existence
> > > &g

Re: Migrating CloudStack content from download.cloud.com

[Chiradeep Vittal]

I was proposing a new S3 bucket earlier (templates.cloudstack.org), but I
realized that someone needs to own the account. There is no legal entity
for Apache CloudStack (except the ASF) that can enter into a contract with
AWS (even if it is for a few bucks a month).  To make this work,
alternatives:
 (1) ASF Infra creates this bucket and IAM keys to the PMC to operate this
bucket. ACS users might want to make targeted donations to support this
bucket, but I don't believe that it is possible.

(2) Cloudstack.org gets registered as a (non-profit?) entity etc and ACS
users make donations to this entity. PMC has control over the bucket.

A mirror list and a python script (e.g., [1]) sounds like a much better
idea :)

Raise of hands:
Who is willing to host a mirror of systemvm templates and default
templates. I figure we need at least 5, preferably all over the world,
especially in East Asia.

[1]
http://code.activestate.com/recipes/284631-a-python-script-to-test-download-mirrors/





On Fri, May 20, 2016 at 9:52 AM, Chiradeep Vittal <chirade...@gmail.com>
wrote:

> Agree with Daan : we shouldn't be pushing these ancient insecure
> templates!
>
>
> On Fri, May 20, 2016 at 9:51 AM, Chiradeep Vittal <chirade...@gmail.com>
> wrote:
>
>> :) According to Raja, there is some time before download.cloud.com is
>> doomed. Let's pretend that date is Jan 1 2017. Now pretend that it is Dec
>> 25, 2016 and we're still debating mirror list vs steps and what not. Now,
>> just change the 2017 to 2016 and re-consider. We need this done yesterday!
>>
>>
>>
>>
>>
>>
>>
>> On Fri, May 20, 2016 at 1:53 AM, Daan Hoogland <daan.hoogl...@gmail.com>
>> wrote:
>>
>>> send by tablet, no warranties in existence
>>> On 20 May 2016 09:29, "Paul Angus" <paul.an...@shapeblue.com> wrote:
>>> >
>>> > I'm a -1 on adding another step to the installation process.
>>> >
>>> > I agree with Will, it will only be another barrier to platform
>>> adoption.
>>> >
>>> > The SSVM going and downloading the built-in template is a good
>>> milestone
>>> in a CloudStack build.
>>> >
>>> > To be honest the mgmt. server should be able to go get the template
>>> itself if required rather than making it a user installation step.
>>> great idea, looking forward to your PR.
>>>
>>> >
>>> > Kind regards,
>>> >
>>> > Paul Angus
>>> >
>>> > paul.an...@shapeblue.com
>>> > www.shapeblue.com
>>> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>>> > @shapeblue
>>> >
>>> >
>>> >
>>> > -Original Message-
>>> > From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
>>> > Sent: 19 May 2016 23:06
>>> > To: dev <dev@cloudstack.apache.org>
>>> > Subject: Re: Migrating CloudStack content from download.cloud.com
>>> >
>>> > sounds good, so if we put both sections together the worry is over. acs
>>> is a complex system and requires a lot of reading so let''s not put it
>>> in a
>>> different section. would be duplicate code anyhow.
>>> >
>>> > On Thu, May 19, 2016 at 11:56 PM, Chiradeep Vittal <
>>> chirade...@gmail.com>
>>> > wrote:
>>> >
>>> > > Well, you have to define "worry" :)
>>> > >
>>> > > I'm proposing an almost identical step to
>>> > >
>>> > >
>>> http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/
>>> > > 4.6/management-server/#prepare-the-system-vm-template
>>> > >
>>> > > If they can do that, then it is not a stretch to assume they can
>>> > > download the default template in a similar manner. The
>>> > > "cloud-install-default-template" (proposed) script can do the mirror
>>> > > checking etc.
>>> > >
>>> > >
>>> > >
>>> > >
>>> > > On Thu, May 19, 2016 at 2:44 PM, Will Stevens <wstev...@cloudops.com
>>> >
>>> > > wrote:
>>> > >
>>> > > > I definitely agree with this.  I don't think people doing a PoC
>>> > > > should
>>> > > have
>>> > > > to worry about getting a template installed on first setup.  I
>>> think
>>> > > > that is a basic feature if we want platform adoption.  Getting the
>>> >

Re: Migrating CloudStack content from download.cloud.com

[Chiradeep Vittal]

Agree with Daan : we shouldn't be pushing these ancient insecure templates!


On Fri, May 20, 2016 at 9:51 AM, Chiradeep Vittal <chirade...@gmail.com>
wrote:

> :) According to Raja, there is some time before download.cloud.com is
> doomed. Let's pretend that date is Jan 1 2017. Now pretend that it is Dec
> 25, 2016 and we're still debating mirror list vs steps and what not. Now,
> just change the 2017 to 2016 and re-consider. We need this done yesterday!
>
>
>
>
>
>
>
> On Fri, May 20, 2016 at 1:53 AM, Daan Hoogland <daan.hoogl...@gmail.com>
> wrote:
>
>> send by tablet, no warranties in existence
>> On 20 May 2016 09:29, "Paul Angus" <paul.an...@shapeblue.com> wrote:
>> >
>> > I'm a -1 on adding another step to the installation process.
>> >
>> > I agree with Will, it will only be another barrier to platform adoption.
>> >
>> > The SSVM going and downloading the built-in template is a good milestone
>> in a CloudStack build.
>> >
>> > To be honest the mgmt. server should be able to go get the template
>> itself if required rather than making it a user installation step.
>> great idea, looking forward to your PR.
>>
>> >
>> > Kind regards,
>> >
>> > Paul Angus
>> >
>> > paul.an...@shapeblue.com
>> > www.shapeblue.com
>> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> > @shapeblue
>> >
>> >
>> >
>> > -Original Message-
>> > From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
>> > Sent: 19 May 2016 23:06
>> > To: dev <dev@cloudstack.apache.org>
>> > Subject: Re: Migrating CloudStack content from download.cloud.com
>> >
>> > sounds good, so if we put both sections together the worry is over. acs
>> is a complex system and requires a lot of reading so let''s not put it in
>> a
>> different section. would be duplicate code anyhow.
>> >
>> > On Thu, May 19, 2016 at 11:56 PM, Chiradeep Vittal <
>> chirade...@gmail.com>
>> > wrote:
>> >
>> > > Well, you have to define "worry" :)
>> > >
>> > > I'm proposing an almost identical step to
>> > >
>> > >
>> http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/
>> > > 4.6/management-server/#prepare-the-system-vm-template
>> > >
>> > > If they can do that, then it is not a stretch to assume they can
>> > > download the default template in a similar manner. The
>> > > "cloud-install-default-template" (proposed) script can do the mirror
>> > > checking etc.
>> > >
>> > >
>> > >
>> > >
>> > > On Thu, May 19, 2016 at 2:44 PM, Will Stevens <wstev...@cloudops.com>
>> > > wrote:
>> > >
>> > > > I definitely agree with this.  I don't think people doing a PoC
>> > > > should
>> > > have
>> > > > to worry about getting a template installed on first setup.  I think
>> > > > that is a basic feature if we want platform adoption.  Getting the
>> > > > first boot process from download to first VM launched should be as
>> > > > smooth as
>> > > possible.
>> > > >
>> > > > *Will STEVENS*
>> > > > Lead Developer
>> > > >
>> > > > *CloudOps* *| *Cloud Solutions Experts
>> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
>> > > > tw @CloudOps_
>> > > >
>> > > > On Thu, May 19, 2016 at 5:37 PM, Daan Hoogland
>> > > > <daan.hoogl...@gmail.com>
>> > > > wrote:
>> > > >
>> > > > > Chiradeep,
>> > > > >
>> > > > > It makes sense not to have this old default very insecure guest
>> > > template
>> > > > > still be downloaded but it would be nice to have some kind of
>> > > > > meta-data pointing to the latest version of an acceptable template
>> > > > > (symlink) for trial installs so PoCcers are swiftly on their way.
>> Would you agree?
>> > > > >
>> > > > > On Thu, May 19, 2016 at 8:15 PM, Chiradeep Vittal <
>> > > chirade...@gmail.com>
>> > > > > wrote:
>> > > > >
>> > > > > > Pierre,
>> > > > > > I meant the auto-download 

Re: Migrating CloudStack content from download.cloud.com

[Chiradeep Vittal]

:) According to Raja, there is some time before download.cloud.com is
doomed. Let's pretend that date is Jan 1 2017. Now pretend that it is Dec
25, 2016 and we're still debating mirror list vs steps and what not. Now,
just change the 2017 to 2016 and re-consider. We need this done yesterday!







On Fri, May 20, 2016 at 1:53 AM, Daan Hoogland <daan.hoogl...@gmail.com>
wrote:

> send by tablet, no warranties in existence
> On 20 May 2016 09:29, "Paul Angus" <paul.an...@shapeblue.com> wrote:
> >
> > I'm a -1 on adding another step to the installation process.
> >
> > I agree with Will, it will only be another barrier to platform adoption.
> >
> > The SSVM going and downloading the built-in template is a good milestone
> in a CloudStack build.
> >
> > To be honest the mgmt. server should be able to go get the template
> itself if required rather than making it a user installation step.
> great idea, looking forward to your PR.
>
> >
> > Kind regards,
> >
> > Paul Angus
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> > -Original Message-
> > From: Daan Hoogland [mailto:daan.hoogl...@gmail.com]
> > Sent: 19 May 2016 23:06
> > To: dev <dev@cloudstack.apache.org>
> > Subject: Re: Migrating CloudStack content from download.cloud.com
> >
> > sounds good, so if we put both sections together the worry is over. acs
> is a complex system and requires a lot of reading so let''s not put it in a
> different section. would be duplicate code anyhow.
> >
> > On Thu, May 19, 2016 at 11:56 PM, Chiradeep Vittal <chirade...@gmail.com
> >
> > wrote:
> >
> > > Well, you have to define "worry" :)
> > >
> > > I'm proposing an almost identical step to
> > >
> > > http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/
> > > 4.6/management-server/#prepare-the-system-vm-template
> > >
> > > If they can do that, then it is not a stretch to assume they can
> > > download the default template in a similar manner. The
> > > "cloud-install-default-template" (proposed) script can do the mirror
> > > checking etc.
> > >
> > >
> > >
> > >
> > > On Thu, May 19, 2016 at 2:44 PM, Will Stevens <wstev...@cloudops.com>
> > > wrote:
> > >
> > > > I definitely agree with this.  I don't think people doing a PoC
> > > > should
> > > have
> > > > to worry about getting a template installed on first setup.  I think
> > > > that is a basic feature if we want platform adoption.  Getting the
> > > > first boot process from download to first VM launched should be as
> > > > smooth as
> > > possible.
> > > >
> > > > *Will STEVENS*
> > > > Lead Developer
> > > >
> > > > *CloudOps* *| *Cloud Solutions Experts
> > > > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6 w cloudops.com *|*
> > > > tw @CloudOps_
> > > >
> > > > On Thu, May 19, 2016 at 5:37 PM, Daan Hoogland
> > > > <daan.hoogl...@gmail.com>
> > > > wrote:
> > > >
> > > > > Chiradeep,
> > > > >
> > > > > It makes sense not to have this old default very insecure guest
> > > template
> > > > > still be downloaded but it would be nice to have some kind of
> > > > > meta-data pointing to the latest version of an acceptable template
> > > > > (symlink) for trial installs so PoCcers are swiftly on their way.
> Would you agree?
> > > > >
> > > > > On Thu, May 19, 2016 at 8:15 PM, Chiradeep Vittal <
> > > chirade...@gmail.com>
> > > > > wrote:
> > > > >
> > > > > > Pierre,
> > > > > > I meant the auto-download of default templates, not systemvm.
> > > > > > Note that the initial systemvm template has to be
> > > > > > hand-downloaded
> > > > anyway.
> > > > > > If you are a single hypervisor cloud, then that is all that is
> > > needed.
> > > > > >
> > > > > > On Thu, May 19, 2016 at 11:08 AM, Pierre-Luc Dion <
> > > pd...@cloudops.com>
> > > > > > wrote:
> > > > > >
> > > > > > > I would be -1 about disabling the autodo

Re: Migrating CloudStack content from download.cloud.com

[Chiradeep Vittal]

Well, you have to define "worry" :)

I'm proposing an almost identical step to
http://docs.cloudstack.apache.org/projects/cloudstack-installation/en/4.6/management-server/#prepare-the-system-vm-template

If they can do that, then it is not a stretch to assume they can download
the default template in a similar manner. The
"cloud-install-default-template" (proposed) script can do the mirror
checking etc.




On Thu, May 19, 2016 at 2:44 PM, Will Stevens <wstev...@cloudops.com> wrote:

> I definitely agree with this.  I don't think people doing a PoC should have
> to worry about getting a template installed on first setup.  I think that
> is a basic feature if we want platform adoption.  Getting the first boot
> process from download to first VM launched should be as smooth as possible.
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Thu, May 19, 2016 at 5:37 PM, Daan Hoogland <daan.hoogl...@gmail.com>
> wrote:
>
> > Chiradeep,
> >
> > It makes sense not to have this old default very insecure guest template
> > still be downloaded but it would be nice to have some kind of meta-data
> > pointing to the latest version of an acceptable template (symlink) for
> > trial installs so PoCcers are swiftly on their way. Would you agree?
> >
> > On Thu, May 19, 2016 at 8:15 PM, Chiradeep Vittal <chirade...@gmail.com>
> > wrote:
> >
> > > Pierre,
> > > I meant the auto-download of default templates, not systemvm.
> > > Note that the initial systemvm template has to be hand-downloaded
> anyway.
> > > If you are a single hypervisor cloud, then that is all that is needed.
> > >
> > > On Thu, May 19, 2016 at 11:08 AM, Pierre-Luc Dion <pd...@cloudops.com>
> > > wrote:
> > >
> > > > I would be -1 about disabling the autodownload of the systemvm
> > template,
> > > > has this should be simplify deployment of a cloudstack system.
> > > > although, the download URL should be a configurable value, maybe in a
> > > > property file or thru the API somehow or via a pre-install command.
> > > >
> > > > I really like  download.cloudstack.org + mirror. I'm looking at
> > getting
> > > > the
> > > > DNS entry: builds.cloudstack.org for the new Jenkins and I got no
> > > > objections on using the cloudstack.org domain for community services
> > so
> > > > I'm
> > > > pretty sure we would be ok to use download.cloudstack.org, where
> would
> > > we
> > > > setting this up? on the current webserver of our website? could we ?
> > > >
> > > >
> > > > Cheers,
> > > >
> > > > PL
> > > >
> > > > On Thu, May 19, 2016 at 11:00 AM, Giles Sirett <
> > > giles.sir...@shapeblue.com
> > > > >
> > > > wrote:
> > > >
> > > > > Erik
> > > > > I agree on the existing users - not the end of the world. We could
> > even
> > > > > release a tiny patch to update the DB records
> > > > >
> > > > > The piece that we need to be really careful with is the "I've just
> > > > > downloaded cloudstack  thing for the first time" scenario
> > > > >
> > > > > But - there seems to be plenty of good ideas flowing on that
> > > > >
> > > > >
> > > > > Kind Regards
> > > > > Giles
> > > > >
> > > > >
> > > > > giles.sir...@shapeblue.com
> > > > > www.shapeblue.com
> > > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > > > > @shapeblue
> > > > >
> > > > >
> > > > >
> > > > > -Original Message-
> > > > > From: Erik Weber [mailto:terbol...@gmail.com]
> > > > > Sent: 19 May 2016 09:32
> > > > > To: dev <dev@cloudstack.apache.org>
> > > > > Subject: Re: Migrating CloudStack content from download.cloud.com
> > > > >
> > > > > As a user (with old ACS installations) I'd say that this isn't all
> > that
> > > > > troublesome, this is easily fixed by updating the appropriate db
> > > records.
> > > > >
> > > > > We should ensure that we have some good posts discovered by Google
> > and
>

Re: Migrating CloudStack content from download.cloud.com

[Chiradeep Vittal]

Pierre,
I meant the auto-download of default templates, not systemvm.
Note that the initial systemvm template has to be hand-downloaded anyway.
If you are a single hypervisor cloud, then that is all that is needed.

On Thu, May 19, 2016 at 11:08 AM, Pierre-Luc Dion 
wrote:

> I would be -1 about disabling the autodownload of the systemvm template,
> has this should be simplify deployment of a cloudstack system.
> although, the download URL should be a configurable value, maybe in a
> property file or thru the API somehow or via a pre-install command.
>
> I really like  download.cloudstack.org + mirror. I'm looking at getting
> the
> DNS entry: builds.cloudstack.org for the new Jenkins and I got no
> objections on using the cloudstack.org domain for community services so
> I'm
> pretty sure we would be ok to use download.cloudstack.org, where would we
> setting this up? on the current webserver of our website? could we ?
>
>
> Cheers,
>
> PL
>
> On Thu, May 19, 2016 at 11:00 AM, Giles Sirett  >
> wrote:
>
> > Erik
> > I agree on the existing users - not the end of the world. We could even
> > release a tiny patch to update the DB records
> >
> > The piece that we need to be really careful with is the "I've just
> > downloaded cloudstack  thing for the first time" scenario
> >
> > But - there seems to be plenty of good ideas flowing on that
> >
> >
> > Kind Regards
> > Giles
> >
> >
> > giles.sir...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> > -Original Message-
> > From: Erik Weber [mailto:terbol...@gmail.com]
> > Sent: 19 May 2016 09:32
> > To: dev 
> > Subject: Re: Migrating CloudStack content from download.cloud.com
> >
> > As a user (with old ACS installations) I'd say that this isn't all that
> > troublesome, this is easily fixed by updating the appropriate db records.
> >
> > We should ensure that we have some good posts discovered by Google and
> > other search engines, so that when somebody running old installations hit
> > this issue they find the workaround.
> > There's a lot of time to reach out to users and inform of the fix
> > necessary.
> >
> > --
> > Erik
> >
> >
> > On Tue, May 17, 2016 at 10:34 AM, Giles Sirett <
> giles.sir...@shapeblue.com
> > >
> > wrote:
> >
> > > This is a MAJOR problem and needs to be figured out before the current
> > > download site is removed - @Raja - have Citrix committed to it being
> > there
> > > another year ?
> > >
> > > It is my understanding that Cloudstack relies on this download at
> initial
> > > build/startup and then periodically if people do things like build a
> new
> > > zone
> > >
> > > So, users environments aren't going to stop if its pulled, but nobody
> > > would be a able to build/install cloudstack and it may cause issues as
> > and
> > > when users make config changes
> > >
> > > The problem is, for new builds, its hard coded : you start the  Sec
> > > Storage VM, CloudStack will try to download the built in template from
> > > download.cloud.com
> > >
> > > My understanding is that its simple db  change to make once an
> > environment
> > > is running (exisiting users) but *may* require a code patch to fix the
> > > initial install
> > >
> > > #notideal
> > >
> > > Really, this should be maintained by ASF/ the project as its key to the
> > > software working, however my understanding is it involves binaries
> which
> > > ASF is never keen on (and may be some 3rd party binaries) - so that
> just
> > > wont happen
> > >
> > >
> > >
> > > Apt-get is controlled by Wido et al (for which we all owe him a massive
> > > thanks)  - but you are right, it is a 3rd party dependency and
> arguably a
> > > SPOF.  But, if ASF wont host binaries, we're always going to have this
> > > issue.
> > >
> > > So, we could patch the code to point somewhere else - but where ? - by
> > > definition its got to be something maintained by a 3rd party
> > >
> > >
> > > We had this debate some time ago when talking about the repositories.
> We
> > > (ShapeBlue) maintain a repo for our customers, which wev'e always
> opened
> > up
> > > as public. However, there were concerns about having a company domain
> > name
> > > (which I understand) - so apt-get became our preferred repo.
> > >
> > >
> > > Thoughts on how to fix this:
> > >
> > > 1. if ASF would allow us to create the host entry
> > download.cloudstack.org.
> > > Its under their domain, but with the site being with a 3rd party it
> could
> > > be allowed
> > >
> > > 2. maybe Citrix could commit to pointing the current DNS entry to a new
> > > location (would be simplest but may be legally complex and relies on
> > > ongoing goodwill)
> > >
> > > 3. make a mod to allow users to choose the download location on
> install.
> > > We can then maintain apt-get. To remove the SPOF, we could then get the
> > > templates copied to a number of 3rd party locations 

Re: Migrating CloudStack content from download.cloud.com

[Chiradeep Vittal]

Changing the download code to use the mirror is quite involved. Should we
just disable (remove from templates.sql) download of default templates and
advise people to use the method outlined here.
http://shankerbalan.net/blog/seed-cloudstack-templates-offline/

(actually, we could write a script similar to cloud-install-sys-tmplt to
install default templates)

On Wed, May 18, 2016 at 12:14 PM, Chiradeep Vittal <chirade...@gmail.com>
wrote:

> Mirrors still require a "seed" site, right? E.g., msync.centos.org is the
> seed for CentOS. Depending on the number of mirrors, this seed site would
> still require a non-trivial bandwidth limit and cap.
> How soon can the mirror be set up? This week? :)
>
>
> On Wed, May 18, 2016 at 10:12 AM, Will Stevens <wstev...@cloudops.com>
> wrote:
>
>> Inline...
>>
>> *Will STEVENS*
>> Lead Developer
>>
>> *CloudOps* *| *Cloud Solutions Experts
>> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
>> w cloudops.com *|* tw @CloudOps_
>>
>> On Wed, May 18, 2016 at 1:08 PM, Chiradeep Vittal <chirade...@gmail.com>
>> wrote:
>>
>> > The mirrolist is a great idea. There's 2 issues though:
>> >
>> > 1. Already released ACS releases in the wild will continue to attempt to
>> > download from download.cloud.com whenever a zone is created. These old
>> > releases cannot understand mirror lists. But users can be advised to
>> patch
>> > templates.sql to point to an alternate location. The question is what is
>> > this alternate location? I propose "templates.cloudstack.org/old"
>> >
>>
>> ​If they patch, maybe they can pull one of the URLs from the mirror list?
>> That way we don't have to hardcode a specific mirror into the patch
>> procedure.​
>>
>>
>> >
>> > 2. New releases of ACS. The next release of ACS has to support the
>> mirror
>> > list, otherwise we're kicking the can too far down the road.
>> >
>>
>> ​I agree, we will need to get this feature into ACS ASAP.​
>>
>>
>> >
>> > For (1)
>> > * templates.cloudstack.org/old could redirect (A record) to
>> > cloudstack.apt-get.eu . Or the project could use AWS S3 (as
>> > download.cloud.com is currently hosted in S3). Current bill is $40-50
>> per
>> > month (between 7GB-40GB downloaded per day). I like the S3 approach
>> myself
>> > but it needs the project to cough up some money or a benevolent sponsor.
>> > There's quite a few organizations making money off of ACS, so this
>> > shouldn't be a problem?
>> > ​
>> >
>>
>>
>> > For (2)
>> > * Assuming somebody steps up to write the new code, we need to find
>> > sponsors that can host the mirrors.
>> >
>> > As somebody said earlier, this needs to happen much sooner than later.
>> >
>> > On Wed, May 18, 2016 at 1:06 AM, Paul Angus <paul.an...@shapeblue.com>
>> > wrote:
>> >
>> > > Pulling a few threads together - if the ASF would create a
>> > > download.cloudstack.org page on ASF infra with the mirrorlist json
>> in it
>> > > as suggested by Nux, we can maintain just that page and add/remove
>> > mirrors
>> > > at will.
>> > >
>> > >
>> > > Kind regards,
>> > >
>> > > Paul Angus
>> > >
>> > > paul.an...@shapeblue.com
>> > > www.shapeblue.com
>> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> > > @shapeblue
>> > >
>> > >
>> > >
>> > > -Original Message-
>> > > From: Wido den Hollander [mailto:w...@widodh.nl]
>> > > Sent: 18 May 2016 08:37
>> > > To: Giles Sirett <giles.sir...@shapeblue.com>;
>> dev@cloudstack.apache.org
>> > > Subject: RE: Migrating CloudStack content from download.cloud.com
>> > >
>> > >
>> > > > Op 17 mei 2016 om 10:34 schreef Giles Sirett <
>> > giles.sir...@shapeblue.com
>> > > >:
>> > > >
>> > > >
>> > > > This is a MAJOR problem and needs to be figured out before the
>> current
>> > > download site is removed - @Raja - have Citrix committed to it being
>> > there
>> > > another year ?
>> > > >
>> > > > It is my understanding that Cloudstack relies on this download at
>> > > > initial build/startup and then periodically if people do things like
>

Re: Migrating CloudStack content from download.cloud.com

[Chiradeep Vittal]

Mirrors still require a "seed" site, right? E.g., msync.centos.org is the
seed for CentOS. Depending on the number of mirrors, this seed site would
still require a non-trivial bandwidth limit and cap.
How soon can the mirror be set up? This week? :)


On Wed, May 18, 2016 at 10:12 AM, Will Stevens <wstev...@cloudops.com>
wrote:

> Inline...
>
> *Will STEVENS*
> Lead Developer
>
> *CloudOps* *| *Cloud Solutions Experts
> 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> w cloudops.com *|* tw @CloudOps_
>
> On Wed, May 18, 2016 at 1:08 PM, Chiradeep Vittal <chirade...@gmail.com>
> wrote:
>
> > The mirrolist is a great idea. There's 2 issues though:
> >
> > 1. Already released ACS releases in the wild will continue to attempt to
> > download from download.cloud.com whenever a zone is created. These old
> > releases cannot understand mirror lists. But users can be advised to
> patch
> > templates.sql to point to an alternate location. The question is what is
> > this alternate location? I propose "templates.cloudstack.org/old"
> >
>
> ​If they patch, maybe they can pull one of the URLs from the mirror list?
> That way we don't have to hardcode a specific mirror into the patch
> procedure.​
>
>
> >
> > 2. New releases of ACS. The next release of ACS has to support the mirror
> > list, otherwise we're kicking the can too far down the road.
> >
>
> ​I agree, we will need to get this feature into ACS ASAP.​
>
>
> >
> > For (1)
> > * templates.cloudstack.org/old could redirect (A record) to
> > cloudstack.apt-get.eu . Or the project could use AWS S3 (as
> > download.cloud.com is currently hosted in S3). Current bill is $40-50
> per
> > month (between 7GB-40GB downloaded per day). I like the S3 approach
> myself
> > but it needs the project to cough up some money or a benevolent sponsor.
> > There's quite a few organizations making money off of ACS, so this
> > shouldn't be a problem?
> > ​
> >
>
>
> > For (2)
> > * Assuming somebody steps up to write the new code, we need to find
> > sponsors that can host the mirrors.
> >
> > As somebody said earlier, this needs to happen much sooner than later.
> >
> > On Wed, May 18, 2016 at 1:06 AM, Paul Angus <paul.an...@shapeblue.com>
> > wrote:
> >
> > > Pulling a few threads together - if the ASF would create a
> > > download.cloudstack.org page on ASF infra with the mirrorlist json in
> it
> > > as suggested by Nux, we can maintain just that page and add/remove
> > mirrors
> > > at will.
> > >
> > >
> > > Kind regards,
> > >
> > > Paul Angus
> > >
> > > paul.an...@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > > @shapeblue
> > >
> > >
> > >
> > > -Original Message-
> > > From: Wido den Hollander [mailto:w...@widodh.nl]
> > > Sent: 18 May 2016 08:37
> > > To: Giles Sirett <giles.sir...@shapeblue.com>;
> dev@cloudstack.apache.org
> > > Subject: RE: Migrating CloudStack content from download.cloud.com
> > >
> > >
> > > > Op 17 mei 2016 om 10:34 schreef Giles Sirett <
> > giles.sir...@shapeblue.com
> > > >:
> > > >
> > > >
> > > > This is a MAJOR problem and needs to be figured out before the
> current
> > > download site is removed - @Raja - have Citrix committed to it being
> > there
> > > another year ?
> > > >
> > > > It is my understanding that Cloudstack relies on this download at
> > > > initial build/startup and then periodically if people do things like
> > > > build a new zone
> > > >
> > > > So, users environments aren't going to stop if its pulled, but nobody
> > > > would be a able to build/install cloudstack and it may cause issues
> as
> > > > and when users make config changes
> > > >
> > > > The problem is, for new builds, its hard coded : you start the  Sec
> > > > Storage VM, CloudStack will try to download the built in template
> from
> > > > download.cloud.com
> > > >
> > > > My understanding is that its simple db  change to make once an
> > > > environment is running (exisiting users) but *may* require a code
> > > > patch to fix the initial install
> > > >
> > > > #notideal
> > > >
> > > > Really, this should be maintaine

Re: Migrating CloudStack content from download.cloud.com

[Chiradeep Vittal]

The mirrolist is a great idea. There's 2 issues though:

1. Already released ACS releases in the wild will continue to attempt to
download from download.cloud.com whenever a zone is created. These old
releases cannot understand mirror lists. But users can be advised to patch
templates.sql to point to an alternate location. The question is what is
this alternate location? I propose "templates.cloudstack.org/old"

2. New releases of ACS. The next release of ACS has to support the mirror
list, otherwise we're kicking the can too far down the road.

For (1)
* templates.cloudstack.org/old could redirect (A record) to
cloudstack.apt-get.eu . Or the project could use AWS S3 (as
download.cloud.com is currently hosted in S3). Current bill is $40-50 per
month (between 7GB-40GB downloaded per day). I like the S3 approach myself
but it needs the project to cough up some money or a benevolent sponsor.
There's quite a few organizations making money off of ACS, so this
shouldn't be a problem?

For (2)
* Assuming somebody steps up to write the new code, we need to find
sponsors that can host the mirrors.

As somebody said earlier, this needs to happen much sooner than later.

On Wed, May 18, 2016 at 1:06 AM, Paul Angus 
wrote:

> Pulling a few threads together - if the ASF would create a
> download.cloudstack.org page on ASF infra with the mirrorlist json in it
> as suggested by Nux, we can maintain just that page and add/remove mirrors
> at will.
>
>
> Kind regards,
>
> Paul Angus
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> -Original Message-
> From: Wido den Hollander [mailto:w...@widodh.nl]
> Sent: 18 May 2016 08:37
> To: Giles Sirett ; dev@cloudstack.apache.org
> Subject: RE: Migrating CloudStack content from download.cloud.com
>
>
> > Op 17 mei 2016 om 10:34 schreef Giles Sirett  >:
> >
> >
> > This is a MAJOR problem and needs to be figured out before the current
> download site is removed - @Raja - have Citrix committed to it being there
> another year ?
> >
> > It is my understanding that Cloudstack relies on this download at
> > initial build/startup and then periodically if people do things like
> > build a new zone
> >
> > So, users environments aren't going to stop if its pulled, but nobody
> > would be a able to build/install cloudstack and it may cause issues as
> > and when users make config changes
> >
> > The problem is, for new builds, its hard coded : you start the  Sec
> > Storage VM, CloudStack will try to download the built in template from
> > download.cloud.com
> >
> > My understanding is that its simple db  change to make once an
> > environment is running (exisiting users) but *may* require a code
> > patch to fix the initial install
> >
> > #notideal
> >
> > Really, this should be maintained by ASF/ the project as its key to
> > the software working, however my understanding is it involves binaries
> > which ASF is never keen on (and may be some 3rd party binaries) - so
> > that just wont happen
> >
> >
> >
> > Apt-get is controlled by Wido et al (for which we all owe him a massive
> thanks)  - but you are right, it is a 3rd party dependency and arguably a
> SPOF.  But, if ASF wont host binaries, we're always going to have this
> issue.
> >
>
> Fully agree with you. I'am happy to host this, but for the project it is a
> danger. The domain name should be controlled by the project.
>
> > So, we could patch the code to point somewhere else - but where ? - by
> > definition its got to be something maintained by a 3rd party
> >
> >
> > We had this debate some time ago when talking about the repositories. We
> (ShapeBlue) maintain a repo for our customers, which wev'e always opened up
> as public. However, there were concerns about having a company domain name
> (which I understand) - so apt-get became our preferred repo.
> >
> >
> > Thoughts on how to fix this:
> >
> > 1. if ASF would allow us to create the host entry
> > download.cloudstack.org. Its under their domain, but with the site
> > being with a 3rd party it could be allowed
> >
>
> I think that would be the best. A CNAME pointing to cloudstack.apt-get.eu
> for now might be enough.
>
> > 2. maybe Citrix could commit to pointing the current DNS entry to a
> > new location (would be simplest but may be legally complex and relies
> > on ongoing goodwill)
> >
> > 3. make a mod to allow users to choose the download location on
> > install. We can then maintain apt-get. To remove the SPOF, we could
> > then get the templates copied to a number of 3rd party locations and
> > allow people to choose which one they use
> >
> >
> >
> >
> > Kind Regards
> > Giles
> >
> >
> > giles.sir...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> >
> >
> >
> > -Original Message-
> > From: williamstev...@gmail.com 

Re: SDN solutions in use today with CloudStack

[Chiradeep Vittal]

Nuage seems to be in production in quite a few CloudStack clouds

From: Simon Weller swel...@ena.commailto:swel...@ena.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Thursday, August 20, 2015 at 11:41 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: SDN solutions in use today with CloudStack

Hi all,

We  are currently exploring options for improving our networking setup (read 
SDN) within CS for our advanced zones.

There are a few plugins currently available for various third party SDN 
providers (Nicira, Big Switch VNS, MidoNet, Stratosphere SSP, Nuage VSP  et 
al), as well as some pre-SDN functionality using native VXLAN, OVS GRE and of 
course vlans.

Most of the organizations mentioned above appear to have drunken the OpenStack 
kool-aid and don't even mention support for CloudStack on their commercial 
websites.

It seems that some of these plugins are getting rather old now and I'd be 
interesting to know (if you're willing to share) what is in use in the 
community today.

- Si

Re: [DISCUSS][PROPOSAL] Support for standard metadata/userdata URL

[Chiradeep Vittal]

The Apache web server listens only on the eth0 IP.
I added the 169.* address to eth0 and changed the Apache config to listen on 
that IP as well.
While the Apache Webserver now receives the request, it returns a 404. Fiddling 
with the .htaccess file could fix that, but now that’s a bigger change than the 
one-liner I proposed.


From: Erik Weber terbol...@gmail.commailto:terbol...@gmail.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Thursday, August 6, 2015 at 9:31 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [DISCUSS][PROPOSAL] Support for standard metadata/userdata URL

Would it not work to assign the address 169.254.169.254 on the VR?

Erik

Den torsdag 6. august 2015 skrev Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com følgende:

Today the metadata / userdata server is retrieved by the guest from
http://address of DHCP server/latest/meta-data and
http://addresshttp://address of DHCP server/latest/user-data

This of course has been inspired by the EC2 standard[1]
http://169.254.169.254/latest/

While this works and has been incorporated into cloud-init [2], there are
other platforms that do not use cloud-init and consume the
userdata/metadata directly. For example, CloudFoundry [3] does not use
cloud-init, rather it tries to have adapters for different cloud types.
Adding support for CloudStack in this case can take a long time.
Another problem is that cloud-init tries the ec2 way first, times out and
then tries the CloudStack way.

I found a easy way to get the EC2-style metadata addressing working in
Advanced Zone / Isolated / VPC

By adding the following iptables rule in the VR, the guest VM can safely
address the 169.254.169.254 URL
iptables -t nat -A PREROUTING  -i eth0 -p tcp -m tcp --dport 80 -d
169.254.169.254/32 -j DNAT --to-destination eth0 ip:80


The current addressing scheme continues to work while supporting the EC2
scheme.
Since the VR is not on the default router in Basic Zone and other cases,
this is not universally applicable.
For KVM  XenServer in Basic Zone we could program the iptables rules in
dom0 (host) to NAT 169.254.169.254 queries to the DHCP server.

For XenServer in Advanced Zone where OVS is the default, there is no
stateful NAT possible.
For other hypervisors  (Hyper-V, VMWare) and Baremetal, there is no
comparable solution for Basic Zone and advanced networks where the VR is
not the default router.


However, note that this is not a breaking change, rather, an enhancement.

Thoughts?
—
Chiradeep
[1]
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
[2] https://cloudinit.readthedocs.org/en/latest/
[3]
https://github.com/cloudfoundry/bosh-agent/blob/master/settings/settings.go

Re: [DISCUSS][PROPOSAL] Support for standard metadata/userdata URL

[Chiradeep Vittal]

Excellent tip! Didn’t notice this in the docs before:
http://docs.cloudstack.apache.org/projects/cloudstack-administration/en/4.5/virtual_machines/user-data.html

From: Carlos Reategui car...@reategui.commailto:car...@reategui.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Thursday, August 6, 2015 at 9:17 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [DISCUSS][PROPOSAL] Support for standard metadata/userdata URL

BTW you can speed up cloud-init to look for CloudStack first/only by
overriding the datasource_list.  The way I do it is by adding a file in my
templates to the /etc/cloud/cloud.cfg.d directory with only CloudStack in
it:

echo datasource_list: [ CloudStack ] 
etc/cloud/cloud.cfg.d/90_datasource.cfg


Start to finish on a first time boot of an instance cloud-init takes ~12
seconds on my system.  A reboot of that same system, cloud-init only takes
5 seconds.  This is with a default/stock cloud-init config on an ubuntu
14.04 system with just that one file added.




On Wed, Aug 5, 2015 at 10:46 PM, Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com wrote:

Today the metadata / userdata server is retrieved by the guest from
http://address of DHCP server/latest/meta-data and
http://addresshttp://address of DHCP server/latest/user-data

This of course has been inspired by the EC2 standard[1]
http://169.254.169.254/latest/

While this works and has been incorporated into cloud-init [2], there are
other platforms that do not use cloud-init and consume the
userdata/metadata directly. For example, CloudFoundry [3] does not use
cloud-init, rather it tries to have adapters for different cloud types.
Adding support for CloudStack in this case can take a long time.
Another problem is that cloud-init tries the ec2 way first, times out and
then tries the CloudStack way.

I found a easy way to get the EC2-style metadata addressing working in
Advanced Zone / Isolated / VPC

By adding the following iptables rule in the VR, the guest VM can safely
address the 169.254.169.254 URL
iptables -t nat -A PREROUTING  -i eth0 -p tcp -m tcp --dport 80 -d
169.254.169.254/32 -j DNAT --to-destination eth0 ip:80


The current addressing scheme continues to work while supporting the EC2
scheme.
Since the VR is not on the default router in Basic Zone and other cases,
this is not universally applicable.
For KVM  XenServer in Basic Zone we could program the iptables rules in
dom0 (host) to NAT 169.254.169.254 queries to the DHCP server.

For XenServer in Advanced Zone where OVS is the default, there is no
stateful NAT possible.
For other hypervisors  (Hyper-V, VMWare) and Baremetal, there is no
comparable solution for Basic Zone and advanced networks where the VR is
not the default router.


However, note that this is not a breaking change, rather, an enhancement.

Thoughts?
—
Chiradeep
[1]
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
[2] https://cloudinit.readthedocs.org/en/latest/
[3]
https://github.com/cloudfoundry/bosh-agent/blob/master/settings/settings.go

[DISCUSS][PROPOSAL] Support for standard metadata/userdata URL

[Chiradeep Vittal]

Today the metadata / userdata server is retrieved by the guest from
http://address of DHCP server/latest/meta-data and
http://addresshttp://address of DHCP server/latest/user-data

This of course has been inspired by the EC2 standard[1]
http://169.254.169.254/latest/

While this works and has been incorporated into cloud-init [2], there are other 
platforms that do not use cloud-init and consume the userdata/metadata 
directly. For example, CloudFoundry [3] does not use cloud-init, rather it 
tries to have adapters for different cloud types. Adding support for CloudStack 
in this case can take a long time.
Another problem is that cloud-init tries the ec2 way first, times out and then 
tries the CloudStack way.

I found a easy way to get the EC2-style metadata addressing working in Advanced 
Zone / Isolated / VPC

By adding the following iptables rule in the VR, the guest VM can safely 
address the 169.254.169.254 URL
iptables -t nat -A PREROUTING  -i eth0 -p tcp -m tcp --dport 80 -d 
169.254.169.254/32 -j DNAT --to-destination eth0 ip:80


The current addressing scheme continues to work while supporting the EC2 scheme.
Since the VR is not on the default router in Basic Zone and other cases, this 
is not universally applicable.
For KVM  XenServer in Basic Zone we could program the iptables rules in dom0 
(host) to NAT 169.254.169.254 queries to the DHCP server.

For XenServer in Advanced Zone where OVS is the default, there is no stateful 
NAT possible.
For other hypervisors  (Hyper-V, VMWare) and Baremetal, there is no comparable 
solution for Basic Zone and advanced networks where the VR is not the default 
router.


However, note that this is not a breaking change, rather, an enhancement.

Thoughts?
—
Chiradeep
[1] 
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html
[2] https://cloudinit.readthedocs.org/en/latest/
[3] https://github.com/cloudfoundry/bosh-agent/blob/master/settings/settings.go

Re: OpenDaylight status

[Chiradeep Vittal]

I’m not sure. I saw that somebody from Inocybe (Mathieu Lemay) was interested 
in supporting the plugin. As to whether it works with the latest release of 
ODL, not sure.

From: ilya
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Monday, April 20, 2015 at 2:59 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org, Chiradeep 
Vittal
Subject: Re: OpenDaylight status

Chiradeep was on OpenDayLight board, perhaps he knows where we are with
cloudstack..

On 4/19/15 1:41 AM, Erik Weber wrote:
Does anyone know what the status of the ODL plugin is?

Erik

Re: OpenDaylight status

[Chiradeep Vittal]

http://permalink.gmane.org/gmane.comp.apache.cloudstack.devel/35692


From: Marcus
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Monday, April 20, 2015 at 4:30 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: OpenDaylight status

What state was it in when it was merged? Functioning?

On Mon, Apr 20, 2015 at 4:19 PM, Chiradeep Vittal
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com wrote:
I’m not sure. I saw that somebody from Inocybe (Mathieu Lemay) was interested 
in supporting the plugin. As to whether it works with the latest release of 
ODL, not sure.

From: ilya
Reply-To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Monday, April 20, 2015 at 2:59 PM
To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org,
 Chiradeep Vittal
Subject: Re: OpenDaylight status

Chiradeep was on OpenDayLight board, perhaps he knows where we are with
cloudstack..

On 4/19/15 1:41 AM, Erik Weber wrote:
Does anyone know what the status of the ODL plugin is?

Erik

Re: [PROPOSAL] DHCP/DNS offload and config drive support for adv zone shared network

[Chiradeep Vittal]

Ilya, there is already a external service.

I think this is proposing a solution where that external service is not desired.
Several “NFV” - type solutions use config drive to configure themselves instead 
using a http-based service.
CoreOS also uses config drive.

That is, this is not a replacement, but an additional option.


From: ilya ilya.mailing.li...@gmail.commailto:ilya.mailing.li...@gmail.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Friday, March 20, 2015 at 6:12 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [PROPOSAL] DHCP/DNS offload and config drive support for adv zone 
shared network

I think config drive is not the best design choice.

You are relying on external ISO to deliver content private to VM. The
ISO is stored in secondary or primary storage, if exposed - I get the
private data of all the VMs. There maybe issues with storage migration
and general vmotion if ISO is attached.

If we are following this model because openstack has chosen this path -
i think its just wasted effort and wrong.

In my opinion, an external service is much better solution, i.e. AWS -
not OpenStack.

Also, how does this work CS retrieves the ip from the VM and update in
the DB nics table. ?


On 3/20/15 4:56 PM, Marcus wrote:
I agree, that's generally the model, right? The network offering
describes where the services come from.

On Fri, Mar 20, 2015 at 12:16 PM, Alena Prokharchyk 
alena1...@gmail.commailto:alena1...@gmail.com wrote:
  From the FS:

Create empty network offering with no service selected. Only DHCP, DNS
services are provided by external servers.
  Metadata - information is included in the config drive
  Userdata, vm password, ssh key - If these are passed then included in
the config drive with user data service.
Retrieving IP assigned by external DHCP server to userVM. Store it in CS
DB.


Why not just introduce the notion of the external provider for the
DHCP/DNS/UserData service? Not specifying the services on the offering and
implementing the service and storing the service data - UserData/MetaData
and IP  - in the CloudStack DB, is confusing. Unless all the
metadata/userdata is stored/managed on/by the external provider side.

On Fri, Mar 20, 2015 at 6:20 AM, Adrian Lewis 
adr...@alsiconsulting.co.ukmailto:adr...@alsiconsulting.co.uk
wrote:

Can't see the wiki at the moment as it's down for maintenance but on a
slightly different but related note, would it be feasible to use DHCP relay
functionality in dnsmasq on a VR and still get the IP address assigned by
an
external DHCP server registered into the ACS MS? Not quite sure if under
normal circumstances ACS picks up the IP from dnsmasq or if ACS manages the
pool and sends dnsmasq static leases. If it's picking up what dnsmasq
decides to lease out, what is this mechanism and does/would it also work
for
DHCP relay?

This doesn’t solve the issue of a DHCP server on the same network however
and would still require a VR on the network with upstream connectivity to
the DHCP server.

I'm definitely definitely up for the concept of simple networks with no VR
if we can provision some of the essentials without one. Big +1


-Original Message-
From: Nux! [mailto:n...@li.nux.ro]
Sent: 20 March 2015 09:34
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [PROPOSAL] DHCP/DNS offload and config drive support for adv
zone shared network

+1, good idea

One thing though:  let's make the config drive available for all types of
zones, many people use the basic or adsg zones.

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro

- Original Message -
From: Jayapal Reddy Uradi 
jayapalreddy.ur...@citrix.commailto:jayapalreddy.ur...@citrix.com
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Sent: Friday, 20 March, 2015 09:12:19
Subject: [PROPOSAL]  DHCP/DNS offload and config drive support for adv
zone shared network
In advanced zone shared network if someone wants to use DHCP server
outside the cloudstack, currently it can be done by not selecting the
DHCP service But the problem here is that the VM actual ip is
different from what cloudstack showing.

If there are no services selected for the network offering there is no
need of the VR.
In the absense of VR there should be way to provide password,
userdata/metadata, ssh keys to user vm.

With this feature we can do the following.
1. Create network without VR.
2. Retrive the IP from the VM and update it in the cloudstack DB.
3. Add config drive support for the VMs in this network.

Please provide your comments for the below FS.

ACS ticket: https://issues.apache.org/jira/browse/CLOUDSTACK-8324
FS:
https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=53740
797


Thanks,
Jayapal


--
Alena Prokharchyk
https://twitter.com/Lemonjet

[CROSSPOST] [ovsdb-dev] CloudStack OVSDB?

[Chiradeep Vittal]

-Original Message-
From: 
ovsdb-dev-boun...@lists.opendaylight.orgmailto:ovsdb-dev-boun...@lists.opendaylight.org
 [mailto:ovsdb-dev-boun...@lists.opendaylight.org] On Behalf Of Dave Neary
Sent: Tuesday, January 13, 2015 12:03 PM
To: ovsdb-...@lists.opendaylight.orgmailto:ovsdb-...@lists.opendaylight.org
Subject: [ovsdb-dev] CloudStack  OVSDB?

Hi all,

I saw Hugo Trippaers' presentation from last year's FOSDEM about using OVSDB 
and CloudStack.

As some of you may have noticed, I was looking for someone to document using 
OpenDaylight with CloudStack for the new Using OpenDaylight
section of the wiki:
https://wiki.opendaylight.org/view/CloudStack_and_OpenDaylight

Is there anyone here who's familiar with this who can help document Helium with 
CloudStack 4.4, please?

Thanks,
Dave.

--
Dave Neary - NFV/SDN Community Strategy
Open Source and Standards, Red Hat - 
http://secure-web.cisco.com/1dDDhuQflPzd19tLV5kjckvanMMnd62cqqA-4LBPmf4LYnM7d72a73uz4zzIjskvgu9VA1tB69c0BNKJoGwVu-P8RS6kuLNJA048jvJNq7u1ryaYXO8zdDm56fj92K91KFxKw5zU-76QMYZhT7su0OXf35sS1wJeey0hn2iR4bDE/http%3A%2F%2Fcommunity.redhat.com
Ph: +1-978-399-2182 / Cell: +1-978-799-3338 
___
ovsdb-dev mailing list
ovsdb-...@lists.opendaylight.orgmailto:ovsdb-...@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/ovsdb-dev

Re: A secure way to reset VMs password

[Chiradeep Vittal]

You would need client-side certs as well since the password server needs to be 
able to validate WHO is asking for the password. Currently it is based on the 
client's IP address.
Also the current scheme is a single-use password — as soon as the password is 
retrieved, it is not available to anybody else (of course a MITM could sniff 
the first exchange).

You could eliminate a lot of MITM-style attacks by running the password server 
locally on each hypervisor (hard for VMW), or by attaching an ISO (containing 
the password) to the VM.

From: John Kinsella j...@stratosec.comailto:j...@stratosec.co
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Tuesday, December 2, 2014 at 1:32 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: A secure way to reset VMs password

That password reset infrastructure has bigger issues than just SSL. The server 
side works, but that’s about all I can say for it. This topic comes up every 
6-12 months. :)

I thought there was a Jira entry but I can’t find it…personally I’d love to see 
the client and server sides both rewritten from scratch.

John

On Nov 28, 2014, at 11:33 AM, Nux! n...@li.nux.romailto:n...@li.nux.ro 
wrote:
Jayapal,
Not necesarily, one could run stunnel or nginx as SSL proxy on some other port 
(8443?), this way SSL and non-SSL connections will still work and give you 
plenty of time to update your templates, if you so wish.
Am I missing any important bits here?
Lucian
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
- Original Message -
From: Jayapal Reddy Uradi 
jayapalreddy.ur...@citrix.commailto:jayapalreddy.ur...@citrix.com
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Cc: Alireza Eskandari 
astro.alir...@yahoo.commailto:astro.alir...@yahoo.com
Sent: Friday, 28 November, 2014 09:34:02
Subject: Re: A secure way to reset VMs password
Another point to note is all the vms in production has to update
with the new cloud-set-guest-password scripts because of the new password reset
method.
Thanks,
Jayapal
On 28-Nov-2014, at 2:28 PM, Erik Weber 
terbol...@gmail.commailto:terbol...@gmail.com
wrote:
On Thu, Nov 27, 2014 at 3:54 PM, Alireza Eskandari 
astro.alir...@yahoo.com.invalidmailto:astro.alir...@yahoo.com.invalid wrote:
HiI viewed the bash script that resets Linux password (
http://download.cloud.com/templates/4.2/bindir/cloud-set-guest-password.in)It
seems that it doesn't use a secure way for transferring password string to
instance.Instances on a shared network can sniff password requests and
export requested password of other instances.I suggest to use SSL (https)
instead of plan text.Regards
I like the idea, but there's a couple of obstacles to overcome, namely
which SSL certificates to use.
- certificates need a subject name, ie. IP or hostname for web pages, you
could solve this by making the mgmt server a CA and have each VR get a
signed certificate by it, but it's complicated
- if the community bundle a pre generated certificate it is commonly known
and not to be trusted, also not sure how to handle subject name
- assuming everyone to supply a valid certificate is quite complicated (CA
must be on VR etc), and makes it considerably harder to get a working setup
- using self signed causes issues with validation
Don't get me wrong, I love the idea, but it's not just to flip a switch and
have (proper) SSL in place.
--
Erik

Re: [DISCUSS] Major business logic refactoring: Move from Account to UserAccount

[Chiradeep Vittal]

Instead of adding yet another parameter, could we look into adding a generic 
filter as in:
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/ApiReference-query-DescribeInstances.html

From: Rohit Yadav rohit.ya...@shapeblue.commailto:rohit.ya...@shapeblue.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Tuesday, November 25, 2014 at 1:27 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Major business logic refactoring: Move from Account to 
UserAccount

Good ideas, I’ll use them. So I think no one disagrees with this;

- list VMs still has user_id, but deployVM won’t
- We’ll use first user in the account if someone’s impersonating; else use 
logged in user to get user_id

On 25-Nov-2014, at 12:17 am, Prachi Damle 
prachi.da...@citrix.commailto:prachi.da...@citrix.com wrote:

Hi Rohit,

I see your point: when deploy VM is called by an admin impersonating another 
account, the user_id value will be set to logged in user, which will be the 
admin. And this will break your usecase.
Correct?

Do you think your functionality needs this usecase i.e an admin impersonating 
deployVm for another user? If you won't hit this scenario primarily, we can 
just set the user_id to first user in the account being impersonated to cover 
this case - just as your upgrade code for existing Vms.

What do you think?

Thanks,
Prachi

-Original Message-
From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
Sent: Friday, November 21, 2014 11:13 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Major business logic refactoring: Move from Account to 
UserAccount

Hi Min, Prachi,

Thanks for your comments. I see your point, the use case is to list VMs for a 
user_id (uuid, not name). I'm going to add the arg/option the listVM api to 
accept user_id and return the list of VMs for that user, and add option in the 
UI to do the same. Note, this is not for auditing purposes (for that we have 
events).

But, since we allow impersonation of account while deploying a VM by the same 
logic we should allow impersonation at the user_id as well which we only accept 
in the deploy VM API if an account/domain is mentioned along with the user_id. 
If I only use logged-in user ID, it makes implementation very simple but at the 
same time but sort of breaks impersonation semantics. Note: the fix will be 
simple, won't change IAM and this is just to add capability to list VMs for a 
user ID.

On 21-Nov-2014, at 11:57 pm, Prachi Damle 
prachi.da...@citrix.commailto:prachi.da...@citrix.com wrote:

Hi Rohit,
The accountId in deployVm API is serving the purpose of impersonation and can 
be passed typically by admin accounts to deploy VM on behalf of other User.
So Ideally with IAM, this parameter should be removed from the API and 
impersonation should be handled separately.
Keeping this goal, I think let's not add userID parameter in the API.

We should default the value to the logged in user - this will prevent usecases 
around cross-account/cross-user scenarios.
Thanks,
Prachi


-Original Message-
From: Min Chen [mailto:min.c...@citrix.com]
Sent: Friday, November 21, 2014 8:16 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Major business logic refactoring: Move from
Account to UserAccount

If I understood correctly, (account, domainId) passed into deployVMCmd is used 
for impersonation-like behavior, that is, caller is deploying a VM on behalf of 
an account. Personally I don't like this kind of putting so many parameters in 
one API to perform several different functionalities, impersonation should be 
done through IAM separately. Too many parameters will just make our API 
semantics very hard to understand and maintain.
Along this line, I will not like to see this user_id added here.

Thanks
-min

On 11/21/14 5:20 AM, Rohit Yadav 
rohit.ya...@shapeblue.commailto:rohit.ya...@shapeblue.com wrote:

Hi Prachi,

Since we¹re already allowing users to specific account and list VMs
by account, following the same pattern I added the case so as to
allow users to specify user_id in both list/deploy VM commands. In
case the userid is not specified, in that case the logged in user¹s ID will be 
used.

It¹s open for discussion of course, let me know if it¹s a good idea
to follow the same pattern or strictly use the logged-in user¹s ID?

On 21-Nov-2014, at 1:41 am, Prachi Damle 
prachi.da...@citrix.commailto:prachi.da...@citrix.com
wrote:

Rohit,

I checked the code here
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=shortlog;
h= ref s/heads/useraccount-refactoring and I don't understand why we
need to expose the userId parameter in the deployVm API.
I think we should be using the userId of the logged in user always.
Exposing the parameter at the API allows it to be set by 

Re: Moving ec2stack and gstack to the cloudstack repos.

[Chiradeep Vittal]

I think unit tests are great for type checking and the like, but are there any 
integration tests? If there is a change made to the CloudStack API (usually a 
parameter is added to the response, rarely a semantic change), will some 
automated test find the breakage if any?
Any plans to add any?

From: Ian Duffy i...@ianduffy.iemailto:i...@ianduffy.ie
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Monday, November 24, 2014 at 2:43 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: Moving ec2stack and gstack to the cloudstack repos.

Chiradeep,

Of course, check out the git repository at
https://github.com/brogand1993/ec2stack the code is pretty clean and there
is 99% test coverage. TravisCI is setup to run on every commit and execute
the tests along with pylint for static analysis.

There's a screencast here of a rough overview
https://www.youtube.com/watch?v=xvu-gc8h4Qglist=UUoGs2iiOIGrfXofp-3g-Qqg
It looks at installation, configuration and basic usage against exoscale
using awscli.


Regards,

Ian


On 24 Nov 2014 19:02, Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com
wrote:

Good to know, but are there any automated tests?

From: Ian Duffy 
i...@ianduffy.iemailto:i...@ianduffy.iemailto:i...@ianduffy.ie
Reply-To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Monday, November 24, 2014 at 10:56 AM
To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: Moving ec2stack and gstack to the cloudstack repos.

Chiradeep,

During the development of ec2stack we tested it using boto, awscli,
vagrant-aws and eutester.

Hope this satisfy your concerns.

On 24 November 2014 at 18:19, Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com
wrote:

Seems legit, but from (bitter) experience, there is no point in a
compatible API layer unless somebody puts in the elbow grease to test the
compatibility. Since the actual EC2 API as implemented by AWS changes
frequently and has undocumented semantics and  behavior that varies from
the WSDL, this takes some work. So, my question would be how would this
benefit the community (unless someone has tested out the compatibility with
various tools such as boto, ec2-* CLI).

From: Sebastien Goasguen 
run...@gmail.commailto:run...@gmail.commailto:run...@gmail.com
mailto:run...@gmail.com
Reply-To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
mailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Saturday, November 22, 2014 at 12:41 PM
To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Moving ec2stack and gstack to the cloudstack repos.

Folks,

Some of you may know of the existence of:

https://github.com/BroganD1993/ec2stack
https://github.com/NOPping/gstack

These represent a EC2 and a GCE interface to cloudstack.
Flask applications that map the requests to the cloudstack API.

There was only 3 contributors, myself, Ian (PMC and committer on CS) and
Darren Brogan.
Darren worked on this during his GSoC 2014 summer project.

Both projects are on Apache V2 license.

The three of us (Ian, Darren and myself) agree that we would like to move
them under the umbrella of cloudstack and manage separate releases like we
do cloud monkey.

Any objections ?

-Sebastien

Re: Moving ec2stack and gstack to the cloudstack repos.

[Chiradeep Vittal]

Seems legit, but from (bitter) experience, there is no point in a compatible 
API layer unless somebody puts in the elbow grease to test the compatibility. 
Since the actual EC2 API as implemented by AWS changes frequently and has 
undocumented semantics and  behavior that varies from the WSDL, this takes some 
work. So, my question would be how would this benefit the community (unless 
someone has tested out the compatibility with various tools such as boto, ec2-* 
CLI).

From: Sebastien Goasguen run...@gmail.commailto:run...@gmail.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Saturday, November 22, 2014 at 12:41 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Moving ec2stack and gstack to the cloudstack repos.

Folks,

Some of you may know of the existence of:

https://github.com/BroganD1993/ec2stack
https://github.com/NOPping/gstack

These represent a EC2 and a GCE interface to cloudstack.
Flask applications that map the requests to the cloudstack API.

There was only 3 contributors, myself, Ian (PMC and committer on CS) and Darren 
Brogan.
Darren worked on this during his GSoC 2014 summer project.

Both projects are on Apache V2 license.

The three of us (Ian, Darren and myself) agree that we would like to move them 
under the umbrella of cloudstack and manage separate releases like we do cloud 
monkey.

Any objections ?

-Sebastien

Re: Moving ec2stack and gstack to the cloudstack repos.

[Chiradeep Vittal]

Good to know, but are there any automated tests?

From: Ian Duffy i...@ianduffy.iemailto:i...@ianduffy.ie
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Monday, November 24, 2014 at 10:56 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: Moving ec2stack and gstack to the cloudstack repos.

Chiradeep,

During the development of ec2stack we tested it using boto, awscli,
vagrant-aws and eutester.

Hope this satisfy your concerns.

On 24 November 2014 at 18:19, Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com
wrote:

Seems legit, but from (bitter) experience, there is no point in a
compatible API layer unless somebody puts in the elbow grease to test the
compatibility. Since the actual EC2 API as implemented by AWS changes
frequently and has undocumented semantics and  behavior that varies from
the WSDL, this takes some work. So, my question would be how would this
benefit the community (unless someone has tested out the compatibility with
various tools such as boto, ec2-* CLI).

From: Sebastien Goasguen 
run...@gmail.commailto:run...@gmail.commailto:run...@gmail.com
Reply-To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Saturday, November 22, 2014 at 12:41 PM
To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Moving ec2stack and gstack to the cloudstack repos.

Folks,

Some of you may know of the existence of:

https://github.com/BroganD1993/ec2stack
https://github.com/NOPping/gstack

These represent a EC2 and a GCE interface to cloudstack.
Flask applications that map the requests to the cloudstack API.

There was only 3 contributors, myself, Ian (PMC and committer on CS) and
Darren Brogan.
Darren worked on this during his GSoC 2014 summer project.

Both projects are on Apache V2 license.

The three of us (Ian, Darren and myself) agree that we would like to move
them under the umbrella of cloudstack and manage separate releases like we
do cloud monkey.

Any objections ?

-Sebastien

Re: Moving ec2stack and gstack to the cloudstack repos.

[Chiradeep Vittal]

“..nobody in the community (aside from you, Likitha and Prachi) have actually 
touched that code in the last two years. So if we don't maintain that code..
That’s false equivalence. Clearly it has been maintained since there are bug 
fixes.

But we’re looking to make things better. I am not sure HOW bringing in another 
compatibility layer brings benefits, UNLESS WE propose to commit time to 
provide a suite of integration tests (say, via eutester)

Thanks
—
Chiradeep

From: sebgoa run...@gmail.commailto:run...@gmail.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Monday, November 24, 2014 at 11:39 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: Moving ec2stack and gstack to the cloudstack repos.


On Nov 24, 2014, at 7:19 PM, Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com wrote:

Seems legit, but from (bitter) experience, there is no point in a compatible 
API layer unless somebody puts in the elbow grease to test the compatibility. 
Since the actual EC2 API as implemented by AWS changes frequently and has 
undocumented semantics and  behavior that varies from the WSDL, this takes some 
work. So, my question would be how would this benefit the community (unless 
someone has tested out the compatibility with various tools such as boto, ec2-* 
CLI).

I think the main issue is the on-going maintenance of such an interface. That's 
also one of the main reason why I advocate to remove awsapi, nobody in the 
community (aside from you, Likitha and Prachi) have actually touched that code 
in the last two years. So if we don't maintain that code and indeed run CI 
against this interface, advertising that we have it gives a false hope to 
users.

On the other side of the coin, I think most cloud tools out there now have 
native cloudstack API support (vagrant, cfg mgmt , libcloud etc…), so the need 
for a pure ec2 interface has diminished greatly.

-sebastien

From: Sebastien Goasguen 
run...@gmail.commailto:run...@gmail.commailto:run...@gmail.com
Reply-To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Saturday, November 22, 2014 at 12:41 PM
To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Moving ec2stack and gstack to the cloudstack repos.
Folks,
Some of you may know of the existence of:
https://github.com/BroganD1993/ec2stack
https://github.com/NOPping/gstack
These represent a EC2 and a GCE interface to cloudstack.
Flask applications that map the requests to the cloudstack API.
There was only 3 contributors, myself, Ian (PMC and committer on CS) and Darren 
Brogan.
Darren worked on this during his GSoC 2014 summer project.
Both projects are on Apache V2 license.
The three of us (Ian, Darren and myself) agree that we would like to move them 
under the umbrella of cloudstack and manage separate releases like we do cloud 
monkey.
Any objections ?
-Sebastien

Re: Moving ec2stack and gstack to the cloudstack repos.

[Chiradeep Vittal]

I do see a bug fix this year from Likitha  and the fact that Hugo etc are 
making fixes is positive as well.
But, the end state we desire is (a) good AWSAPI implementation with automated 
tests, not (b) 2 AWSAPI implementations with no tests!

—
Chiradeep

From: Sebastien Goasguen run...@gmail.commailto:run...@gmail.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Monday, November 24, 2014 at 1:36 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: Moving ec2stack and gstack to the cloudstack repos.


On Nov 24, 2014, at 3:44 PM, Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com wrote:

“..nobody in the community (aside from you, Likitha and Prachi) have actually 
touched that code in the last two years. So if we don't maintain that code..
That’s false equivalence. Clearly it has been maintained since there are bug 
fixes.

I don't know…I look at:

https://github.com/apache/cloudstack/tree/master/awsapi

I see Hugo has fixed some coverity issues

I applied a review 8 months ago

the rest is older. but maybe I am not looking at this the right way.

there is one review still pending:

https://reviews.apache.org/r/21776/

So from looking at it this way it does not look actively maintained. No ?

But we’re looking to make things better. I am not sure HOW bringing in another 
compatibility layer brings benefits, UNLESS WE propose to commit time to 
provide a suite of integration tests (say, via eutester)

Do we have a suite of integration tests for awsapi that is running right now ? 
where ?

I did play with eutester and actually patched it to work with cloudstack when I 
worked on ec2stack:

http://sebgoa.blogspot.de/2014/06/eutester-interesting-tool-based-on-boto.html

-sebastien

Thanks
—
Chiradeep
From: sebgoa 
run...@gmail.commailto:run...@gmail.commailto:run...@gmail.com
Reply-To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Monday, November 24, 2014 at 11:39 AM
To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: Moving ec2stack and gstack to the cloudstack repos.
On Nov 24, 2014, at 7:19 PM, Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com
 wrote:
Seems legit, but from (bitter) experience, there is no point in a compatible 
API layer unless somebody puts in the elbow grease to test the compatibility. 
Since the actual EC2 API as implemented by AWS changes frequently and has 
undocumented semantics and  behavior that varies from the WSDL, this takes some 
work. So, my question would be how would this benefit the community (unless 
someone has tested out the compatibility with various tools such as boto, ec2-* 
CLI).
I think the main issue is the on-going maintenance of such an interface. That's 
also one of the main reason why I advocate to remove awsapi, nobody in the 
community (aside from you, Likitha and Prachi) have actually touched that code 
in the last two years. So if we don't maintain that code and indeed run CI 
against this interface, advertising that we have it gives a false hope to 
users.
On the other side of the coin, I think most cloud tools out there now have 
native cloudstack API support (vagrant, cfg mgmt , libcloud etc…), so the need 
for a pure ec2 interface has diminished greatly.
-sebastien
From: Sebastien Goasguen 
run...@gmail.commailto:run...@gmail.commailto:run...@gmail.commailto:run...@gmail.com
Reply-To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Saturday, November 22, 2014 at 12:41 PM
To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Moving ec2stack and gstack to the cloudstack repos.
Folks,
Some of you may know of the existence of:
https://github.com/BroganD1993/ec2stack
https://github.com/NOPping/gstack
These represent a EC2 and a GCE interface to cloudstack.
Flask applications that map the requests to the cloudstack API.
There was only 3 contributors, myself, Ian (PMC and committer on CS) and Darren 
Brogan.
Darren worked on this during his GSoC 2014 summer project.
Both projects are on Apache V2 license.
The three of us (Ian, Darren and myself) agree that we would like to move them 
under the umbrella

Re: NetScaler 10.5

[Chiradeep Vittal]

Looks like it was fixed
https://issues.apache.org/jira/browse/CLOUDSTACK-6261

From: Francois Gaudreault 
fgaudrea...@cloudops.commailto:fgaudrea...@cloudops.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org, 
fgaudrea...@cloudops.commailto:fgaudrea...@cloudops.com 
fgaudrea...@cloudops.commailto:fgaudrea...@cloudops.com
Date: Thursday, November 20, 2014 at 6:29 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: NetScaler 10.5

Guys,

Is NetScaler 10.5 supposed to work on the 4.4.x branch? I was under to
impression there was a bug with the way ACS uses the API of 10.5,
correct? Is that bug fixed in 4.5? If its the case, can we merge back
that commit in 4.4?

Thanks!

--
Francois Gaudreault
Gestionnaire de Produit | Product Manager - Cloud Platform  Services
t:514-629-6775

CloudOps Votre partenaire infonuagique | Cloud Solutions Experts
420 rue Guy | Montreal | Quebec | H3J 1S6
w: cloudops.com | tw: @CloudOps_

Re: [PROPOSAL] Move to github PR only during moratorium on commit

[Chiradeep Vittal]

Won’t this proposal make GitHub the canonical repository? I don’t see ASF infra 
being too happy with that.

From: sebgoa run...@gmail.commailto:run...@gmail.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Saturday, October 18, 2014 at 2:00 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: [PROPOSAL] Move to github PR only during moratorium on commit

After [1] I would like to officially bring up the following proposal.

[Proposal]

All commits come through github PR, *even* for committers. We declare a 
moratorium period (agreed suspension of activity) during which direct commit to 
master is forbidden.
Only the master RM is allowed to merge PR in master (we define a master RM). If 
direct commit to master is done, master RM reverts without warning. Same for 
4.5 and 4.4. branches.


This is drastic and I am sure some folks will not like it, but here is my 
justification for such a measure:

[Reasons]:

Our commit and release processes have so far been based on the idea that 
development happens on master and that a release branch is cut from master 
(unstable development branch). Then a different set of community members harden 
the release branch, QA and bring it to GA level. During that time development 
keeps on going in master.

This is an OK process if we have the luxury of having a QA team and can cope 
with split personality of being developers and release managers.

My point of view is that as a community we cannot afford such a split brain 
organization and our experience overt the last year proves my point (delayed 
release date, broken builds, features merged without warning…)

We can avoid this by cutting a release branch from a stable one (from the 
start), then as you (Daan) have mentioned several times, fix bugs in the 
release branch and merge them back in the stable source of the release (be it 
master).

Feature development need to be done outside master, period. Not only for 
non-committers but also for committers. And merge request need to be called. 
This will help review and avoid surprises.

New git workflow were proposed and shutdown, mostly calling for better CI to 
solve quality issues. CI will not solve our quality issues alone. We need to 
better police ourselves.

To avoid long discussions, I propose this simple but drastic measure. We move 
all our commits to github PR until 4.5 is out, this stands for committers and 
non-committers, direct commits (especially to master) would be reverted 
immediately.


Our development and release process is broken, we cannot continue like this, 
let's fix it.

[1] http://markmail.org/thread/xeliefp3oleq3g54

-sebastien

Re: [PROPOSAL] Move to github PR only during moratorium on commit

[Chiradeep Vittal]

Thanks. Question for Sebastien then. The argument is that this new proposal 
will avoid problems such as
 - broken builds. Presumably this is  test failures (don’t recall a 
compile-time failure). How exactly would this be achieved WITHOUT a CI process?
 - delayed releases. Not sure how this fixes it. Seems like it is just 
introducing a bottleneck for the sake of slowing things down?
 - features merged without warning. Warnings are good. But what does one do 
with (let’s say a 3-day) warning? One must trust the developer or trust the 
developer and the CI system.

The argument it seems is for a stable master vs a stable release branch.  Is 
that the correct summary? Why not switch to that model on ACS git instead of 
GitHub?

From: Daan Hoogland daan.hoogl...@gmail.commailto:daan.hoogl...@gmail.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Monday, October 20, 2014 at 10:34 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [PROPOSAL] Move to github PR only during moratorium on commit

No Chiradeep, Pull of the request will still be on the local committer repo
and pushed to ASF infra (wip)

On Mon, Oct 20, 2014 at 7:26 PM, Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com wrote:

Won’t this proposal make GitHub the canonical repository? I don’t see ASF
infra being too happy with that.

From: sebgoa 
run...@gmail.commailto:run...@gmail.commailto:run...@gmail.com
Reply-To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Saturday, October 18, 2014 at 2:00 AM
To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: [PROPOSAL] Move to github PR only during moratorium on commit

After [1] I would like to officially bring up the following proposal.

[Proposal]

All commits come through github PR, *even* for committers. We declare a
moratorium period (agreed suspension of activity) during which direct
commit to master is forbidden.
Only the master RM is allowed to merge PR in master (we define a master
RM). If direct commit to master is done, master RM reverts without warning.
Same for 4.5 and 4.4. branches.


This is drastic and I am sure some folks will not like it, but here is my
justification for such a measure:

[Reasons]:

Our commit and release processes have so far been based on the idea that
development happens on master and that a release branch is cut from master
(unstable development branch). Then a different set of community members
harden the release branch, QA and bring it to GA level. During that time
development keeps on going in master.

This is an OK process if we have the luxury of having a QA team and can
cope with split personality of being developers and release managers.

My point of view is that as a community we cannot afford such a split
brain organization and our experience overt the last year proves my point
(delayed release date, broken builds, features merged without warning…)

We can avoid this by cutting a release branch from a stable one (from the
start), then as you (Daan) have mentioned several times, fix bugs in the
release branch and merge them back in the stable source of the release (be
it master).

Feature development need to be done outside master, period. Not only for
non-committers but also for committers. And merge request need to be
called. This will help review and avoid surprises.

New git workflow were proposed and shutdown, mostly calling for better CI
to solve quality issues. CI will not solve our quality issues alone. We
need to better police ourselves.

To avoid long discussions, I propose this simple but drastic measure. We
move all our commits to github PR until 4.5 is out, this stands for
committers and non-committers, direct commits (especially to master) would
be reverted immediately.


Our development and release process is broken, we cannot continue like
this, let's fix it.

[1] http://markmail.org/thread/xeliefp3oleq3g54

-sebastien




--
Daan

Re: What IPsec VPN could we use to replace OpenSwan?

[Chiradeep Vittal]

+1 to StrongSwan since it seems to be compatible with OpenSwan as well strong 
routing support. Also superior documentation.


From: Demetrius Tsitrelis 
demetrius.tsitre...@citrix.commailto:demetrius.tsitre...@citrix.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Friday, October 3, 2014 at 5:00 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: What IPsec VPN could we use to replace OpenSwan?

It doesn't seem that OpenSwan is very actively maintained if there is an issue 
with the OS X client.  Is there another IPsec VPN we could use instead 
(strongSwan, Libreswan, etc.)?

-Original Message-
From: Harikrishna Patnala [mailto:nore...@reviews.apache.org] On Behalf Of 
Harikrishna Patnala
Sent: Tuesday, September 30, 2014 4:42 AM
To: Kishan Kavala; Jayapal Reddy Uradi
Cc: ASF Subversion and Git Services; Harikrishna Patnala; cloudstack
Subject: Re: Review Request 23837: CLOUDSTACK-7087: Downgrade openswan to 
previous version for VPN services to fix OSX client


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/23837/
---

(Updated Sept. 30, 2014, 11:42 a.m.)


Review request for cloudstack, Jayapal Reddy and Kishan Kavala.


Bugs: CLOUDSTACK-7087
https://issues.apache.org/jira/browse/CLOUDSTACK-7087


Repository: cloudstack-git


Description
---

CLOUDSTACK-7087: Downgrade openswan to previous version for VPN services to fix 
OSX client Downgrading openswan version to 1:2.6.37-3


Diffs
-

  tools/appliance/definitions/systemvm64template/postinstall.sh 8763a9f
  tools/appliance/definitions/systemvmtemplate/postinstall.sh 587d44d

Diff: https://reviews.apache.org/r/23837/diff/


Testing
---


Thanks,

Harikrishna Patnala

ACTION Required from Cloud Operators [realhostip]: Your users may lose console access to their CloudStack VMs soon!

[Chiradeep Vittal]

Hi

The realhostip.com DNS resolver will be turned off imminently. This is the
default resolver used by the Console in releases 4.2 and below to resolve
the IP of the console proxy VM. Once the DNS resolver is turned off, IF
YOU HAVE NOT taken action to not replace the default, console access will
not work.


See these blog posts:
https://blogs.apache.org/cloudstack/entry/realhostip_service_is_being_retir
ed
https://blogs.apache.org/cloudstack/entry/cloudstack_s_realhostip_service_t
o

http://shapeblue.com/cloudstack/retirement-of-the-realhostip-com-service/


Please avoid inconvenience to your users and make the required changes
immediately.

‹
Chiradeep

Re: Can't launch VMs

[Chiradeep Vittal]

Not sure what is a “basic zone no security groups”. Check your vlan table to 
see if there has been any allocation of vlans?

From: Carlos Reategui create...@gmail.commailto:create...@gmail.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org, 
car...@reategui.commailto:car...@reategui.com 
car...@reategui.commailto:car...@reategui.com
Date: Thursday, October 2, 2014 at 3:48 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Fwd: Can't launch VMs

Hi devs,
Daan suggested I check with you all regarding this email I sent to the
users list.

He said the line of code that breaks expects a uri for a vlan, However I
am using basic networking and I don't know where to add a vlan in that setup.
He also said this was a known issue.  Hopefully one of you all knows a fix.

This deployment had been working fine.  Originally a 4.1, upgraded to 4.2.1
and more recently to 4.3 and 4.3.1.  It was definitely working under 4.2.1
and I though it had been working under 4.3 as well but I am not sure now.
There is no problem with existing instances.  The problem is when trying to
launch new ones.

Let me know what additional info would be useful.

thank you,
Carlos



-- Forwarded message --
From: Carlos Reategui create...@gmail.commailto:create...@gmail.com
Date: Mon, Sep 29, 2014 at 4:53 PM
Subject: Can't launch VMs
To: us...@cloudstack.apache.orgmailto:us...@cloudstack.apache.org 
us...@cloudstack.apache.orgmailto:us...@cloudstack.apache.org


Following up on my earlier email regarding errors in my logs it appears
things are not as great as I thought.  Trying to launch instances is not
working.  Please help.
thanks,
Carlos

ACS: 4.3.1
Hosts: XenServer 6.2
Network: Basic Shared Network no SG

Things appear ok up until it tries to create the VIF.  Here are the logs:

2014-09-29 16:26:24,816 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-235:ctx-7f8db7ac) Seq 2-868943178: Executing request

2014-09-29 16:26:25,003 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-235:ctx-7f8db7ac) 1. The VM i-3-98-VM is in Starting state.

2014-09-29 16:26:25,174 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-235:ctx-7f8db7ac) Created VM
71314961-a1bf-689e-eaec-73b9e3639db8 for i-3-98-VM

2014-09-29 16:26:25,405 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-235:ctx-7f8db7ac) VBD 9297b527-7b1f-600c-0d55-8486a8bd35d7
created for com.cloud.agent.api.to.DiskTO@53d6031b

2014-09-29 16:26:25,509 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-399:ctx-73a60525) Seq 2-868943177: Response Received:

2014-09-29 16:26:25,509 DEBUG [c.c.a.t.Request]
(StatsCollector-3:ctx-13616221) Seq 2-868943177: Received:  { Ans: ,
MgmtId: 233845174730255, via: 2, Ver: v1, Flags: 10, {
GetStorageStatsAnswer } }

2014-09-29 16:26:25,512 DEBUG [c.c.a.m.DirectAgentAttache]
(DirectAgent-392:ctx-401c432d) Seq 2-868943179: Executing request

2014-09-29 16:26:25,618 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-235:ctx-7f8db7ac) VBD 6609b989-ec81-10c1-8b00-b1fb1eb3885f
created for com.cloud.agent.api.to.DiskTO@7fbb227a

2014-09-29 16:26:25,658 DEBUG [c.c.a.ApiServlet]
(catalina-exec-12:ctx-a4bf4221) ===START===  172.30.36.159 -- GET
command=queryAsyncJobResultjobId=f38a8e56-9a85-48aa-9da2-d498d3179634response=jsonsessionkey=UWhJaGdVTH3zXZ9WFdv4EwAlqA4%3D_=1412033185658

2014-09-29 16:26:25,679 DEBUG [c.c.a.ApiServlet]
(catalina-exec-12:ctx-a4bf4221 ctx-d566be7f) ===END===  172.30.36.159 --
GET
command=queryAsyncJobResultjobId=f38a8e56-9a85-48aa-9da2-d498d3179634response=jsonsessionkey=UWhJaGdVTH3zXZ9WFdv4EwAlqA4%3D_=1412033185658

2014-09-29 16:26:25,735 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-235:ctx-7f8db7ac) VBD d47f3673-7dc8-68ff-7eb7-fcd6ba8b0c41
created for com.cloud.agent.api.to.DiskTO@194e84ba

2014-09-29 16:26:25,735 DEBUG [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-235:ctx-7f8db7ac) Creating VIF for i-3-98-VM on nic
[Nic:Guest-172.30.45.143-null]

2014-09-29 16:26:25,834 WARN  [c.c.h.x.r.CitrixResourceBase]
(DirectAgent-235:ctx-7f8db7ac) Catch Exception: class
java.lang.NullPointerException due to java.lang.NullPointerException

java.lang.NullPointerException

at
com.cloud.network.Networks$BroadcastDomainType.getSchemeValue(Networks.java:173)

at
com.cloud.network.Networks$BroadcastDomainType.getValue(Networks.java:228)

at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.getNetwork(CitrixResourceBase.java:1035)

at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.createVif(CitrixResourceBase.java:1088)

at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.execute(CitrixResourceBase.java:1718)

at
com.cloud.hypervisor.xen.resource.CitrixResourceBase.executeRequest(CitrixResourceBase.java:545)

at
com.cloud.hypervisor.xen.resource.XenServer56Resource.executeRequest(XenServer56Resource.java:59)

at

Re: Can't launch VMs

[Chiradeep Vittal]

Wonder if Daan is talking about
https://issues.apache.org/jira/browse/CLOUDSTACK-4346


As you can see from
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=plugins/h
ypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resource/CitrixReso
urceBase.java;h=9313e5673f30e6a70f915258ee78c7e5832399d4;hb=HEAD#l1019 ,
the first check for ³untagged² should¹ve gone through, except that it goes
through and either hits line 1025 instead.
So, check your Œnetworks¹ table and see if the URI is correct.


From:  Carlos Reátegui create...@gmail.com
Reply-To:  dev@cloudstack.apache.org dev@cloudstack.apache.org
Date:  Friday, October 3, 2014 at 12:15 PM
To:  dev@cloudstack.apache.org dev@cloudstack.apache.org
Subject:  Re: Can't launch VMs


Thanks for having a look.  Inline...

On Oct 3, 2014, at 11:38 AM, Chiradeep Vittal
chiradeep.vit...@citrix.com wrote:

 Not sure what is a ³basic zone no security groups².
Basic Zone type with the following Network Offering:
DefaultSharedNetworkOffering as opposed to the default/first option
DefaultSharedNetworkOfferingWithSGService.

 Check your vlan table to see if there has been any allocation of vlans?
mysql select * from vlan;
++--+-+
--+---+-++-
---++-+-+--+---
+
| id | uuid | vlan_id |
vlan_gateway | vlan_netmask  | description | vlan_type
 | data_center_id | network_id | physical_network_id | ip6_gateway |
ip6_cidr | ip6_range |
++--+-+
--+---+-++-
---++-+-+--+---
+
|  1 | 5ebc3075-2946-4be4-9e7a-f32a86e7edfc | vlan://untagged |
172.30.45.1  | 255.255.255.0 | 172.30.45.100-172.30.45.174 |
DirectAttached |  1 |204 | 200 | NULL
  | NULL | NULL  |
++--+-+
--+---+-++-
---++-+-+--+---
+
1 row in set (0.00 sec)



 
 From: Carlos Reategui create...@gmail.commailto:create...@gmail.com
 Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org,
car...@reategui.commailto:car...@reategui.com
car...@reategui.commailto:car...@reategui.com
 Date: Thursday, October 2, 2014 at 3:48 PM
 To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 Subject: Fwd: Can't launch VMs
 
 Hi devs,
 Daan suggested I check with you all regarding this email I sent to the
 users list.
 
 He said the line of code that breaks expects a uri for a vlan, However
I
 am using basic networking and I don't know where to add a vlan in that
setup.
 He also said this was a known issue.  Hopefully one of you all knows a
fix.
 
 This deployment had been working fine.  Originally a 4.1, upgraded to
4.2.1
 and more recently to 4.3 and 4.3.1.  It was definitely working under
4.2.1
 and I though it had been working under 4.3 as well but I am not sure now.
 There is no problem with existing instances.  The problem is when trying
to
 launch new ones.
 
 Let me know what additional info would be useful.
 
 thank you,
 Carlos
 
 
 
 -- Forwarded message --
 From: Carlos Reategui create...@gmail.commailto:create...@gmail.com
 Date: Mon, Sep 29, 2014 at 4:53 PM
 Subject: Can't launch VMs
 To: us...@cloudstack.apache.orgmailto:us...@cloudstack.apache.org
us...@cloudstack.apache.orgmailto:us...@cloudstack.apache.org
 
 
 Following up on my earlier email regarding errors in my logs it appears
 things are not as great as I thought.  Trying to launch instances is not
 working.  Please help.
 thanks,
 Carlos
 
 ACS: 4.3.1
 Hosts: XenServer 6.2
 Network: Basic Shared Network no SG
 
 Things appear ok up until it tries to create the VIF.  Here are the logs:
 
 2014-09-29 16:26:24,816 DEBUG [c.c.a.m.DirectAgentAttache]
 (DirectAgent-235:ctx-7f8db7ac) Seq 2-868943178: Executing request
 
 2014-09-29 16:26:25,003 DEBUG [c.c.h.x.r.CitrixResourceBase]
 (DirectAgent-235:ctx-7f8db7ac) 1. The VM i-3-98-VM is in Starting state.
 
 2014-09-29 16:26:25,174 DEBUG [c.c.h.x.r.CitrixResourceBase]
 (DirectAgent-235:ctx-7f8db7ac) Created VM
 71314961-a1bf-689e-eaec-73b9e3639db8 for i-3-98-VM
 
 2014-09-29 16:26:25,405 DEBUG [c.c.h.x.r.CitrixResourceBase]
 (DirectAgent-235:ctx-7f8db7ac) VBD 9297b527-7b1f-600c-0d55-8486a8bd35d7
 created for com.cloud.agent.api.to.DiskTO@53d6031b
 
 2014-09-29 16:26:25,509 DEBUG [c.c.a.m.DirectAgentAttache]
 (DirectAgent-399:ctx-73a60525) Seq 2-868943177: Response Received:
 
 2014

Re: Can't launch VMs

[Chiradeep Vittal]

 | 1 |  1 | NULL | NULL | NULL  |  0 |
Domain   | cs1cloud.internal | NULL   | Shared |
 0 | 2013-04-30 21:25:58 | NULL| 1 |   NULL | NULL
   | NULL | NULL |   1 |   NULL |
+-+--+--+--
+--+---+---
+-+++-+
-++---+---+
-+---++--+--+---++-
-+---+++--+
-+-+---++-+
--+--+-++
5 rows in set (0.00 sec)




On Oct 3, 2014, at 12:39 PM, Chiradeep Vittal
chiradeep.vit...@citrix.com wrote:

 Wonder if Daan is talking about
 https://issues.apache.org/jira/browse/CLOUDSTACK-4346
 
 
 As you can see from
 
https://git-wip-us.apache.org/repos/asf?p=cloudstack.git;a=blob;f=plugins/
h
 
ypervisors/xenserver/src/com/cloud/hypervisor/xenserver/resource/CitrixRes
o
 urceBase.java;h=9313e5673f30e6a70f915258ee78c7e5832399d4;hb=HEAD#l1019 ,
 the first check for ³untagged² should¹ve gone through, except that it
goes
 through and either hits line 1025 instead.
 So, check your Œnetworks¹ table and see if the URI is correct.
 
 
 From:  Carlos Reátegui create...@gmail.com
 Reply-To:  dev@cloudstack.apache.org dev@cloudstack.apache.org
 Date:  Friday, October 3, 2014 at 12:15 PM
 To:  dev@cloudstack.apache.org dev@cloudstack.apache.org
 Subject:  Re: Can't launch VMs
 
 
 Thanks for having a look.  Inline...
 
 On Oct 3, 2014, at 11:38 AM, Chiradeep Vittal
 chiradeep.vit...@citrix.com wrote:
 
 Not sure what is a ³basic zone no security groups².
 Basic Zone type with the following Network Offering:
 DefaultSharedNetworkOffering as opposed to the default/first option
 DefaultSharedNetworkOfferingWithSGService.
 
 Check your vlan table to see if there has been any allocation of vlans?
 mysql select * from vlan;
 
++--+-+---
-
 
--+---+-++
-
 
---++-+-+--+--
-
 +
 | id | uuid | vlan_id |
 vlan_gateway | vlan_netmask  | description | vlan_type
 | data_center_id | network_id | physical_network_id | ip6_gateway |
 ip6_cidr | ip6_range |
 
++--+-+---
-
 
--+---+-++
-
 
---++-+-+--+--
-
 +
 |  1 | 5ebc3075-2946-4be4-9e7a-f32a86e7edfc | vlan://untagged |
 172.30.45.1  | 255.255.255.0 | 172.30.45.100-172.30.45.174 |
 DirectAttached |  1 |204 | 200 | NULL
  | NULL | NULL  |
 
++--+-+---
-
 
--+---+-++
-
 
---++-+-+--+--
-
 +
 1 row in set (0.00 sec)
 
 
 
 
 From: Carlos Reategui create...@gmail.commailto:create...@gmail.com
 Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org,
 car...@reategui.commailto:car...@reategui.com
 car...@reategui.commailto:car...@reategui.com
 Date: Thursday, October 2, 2014 at 3:48 PM
 To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 Subject: Fwd: Can't launch VMs
 
 Hi devs,
 Daan suggested I check with you all regarding this email I sent to the
 users list.
 
 He said the line of code that breaks expects a uri for a vlan, However
 I
 am using basic networking and I don't know where to add a vlan in that
 setup.
 He also said this was a known issue.  Hopefully one of you all knows a
 fix.
 
 This deployment had been working fine.  Originally a 4.1, upgraded to
 4.2.1
 and more recently to 4.3 and 4.3.1.  It was definitely working under
 4.2.1
 and I though it had been working under 4.3 as well but I am not sure
now.
 There is no problem with existing instances.  The problem is when trying
 to
 launch new ones.
 
 Let me know what additional info would be useful.
 
 thank you,
 Carlos
 
 
 
 -- Forwarded message --
 From: Carlos Reategui create...@gmail.commailto:create...@gmail.com
 Date: Mon, Sep 29, 2014 at 4:53 PM
 Subject: Can't launch VMs
 To: us...@cloudstack.apache.orgmailto:us...@cloudstack.apache.org
 us...@cloudstack.apache.orgmailto:us...@cloudstack.apache.org
 
 
 Following up on my earlier email regarding

Re: Realhostip service extended till Sep 30th

[Chiradeep Vittal]

I checked the DNS logs and still see quite a few requests for 
xx-yy-zz-aa.realhostip.com
Please work on removing the realhostip.com domain in your installation ASAP.

https://cwiki.apache.org/confluence/x/rAZ_Ag

From: Mihaela Stoica 
mihaela.sto...@citrix.commailto:mihaela.sto...@citrix.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Wednesday, September 3, 2014 at 4:04 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org, 
us...@cloudstack.apache.orgmailto:us...@cloudstack.apache.org 
us...@cloudstack.apache.orgmailto:us...@cloudstack.apache.org, 
users...@cloudstack.apache.orgmailto:users...@cloudstack.apache.org 
users...@cloudstack.apache.orgmailto:users...@cloudstack.apache.org
Subject: RE: Realhostip service extended till Sep 30th

If you are using the latest cloudstack build from master or 4.4 branch, you can 
upload custom certificates via the UI, instead of invoking the 
UploadCustomCertificate API directly.

We changed the SSL Certificate dialog to allow uploading chained SSL 
certificates, see 
CLOUDSTACK-6695https://issues.apache.org/jira/browse/CLOUDSTACK-6695.



Thanks,

Mihaela



-Original Message-
From: Nitin Mehta [mailto:nitin.me...@citrix.com]
Sent: 02 September 2014 22:26
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org; Animesh 
Chaturvedi; us...@cloudstack.apache.orgmailto:us...@cloudstack.apache.org; 
users...@cloudstack.apache.orgmailto:users...@cloudstack.apache.org
Cc: Amogh Vasekar
Subject: Re: Realhostip service extended till Sep 30th



Created another wiki [1] for troubleshooting and information on the 
implementation details.

[1]

https://cwiki.apache.org/confluence/display/CLOUDSTACK/Troubleshooting+-+up

loading+custom+domain+certificate+instead+of+using+realhostip.com







Thanks,

-Nitin



On 05/06/14 7:03 PM, Nitin Mehta 
nitin.me...@citrix.commailto:nitin.me...@citrix.commailto:nitin.me...@citrix.com
 wrote:



Please find wiki [1] containing the procedure to replace realhostip.com

with Your own custom domain name.



[1]

https://cwiki.apache.org/confluence/display/CLOUDSTACK/Procedure+to+Replac

e

+realhostip.com+with+Your+Own+Domain+Name





Thanks,

-Nitin



On 05/06/14 4:23 PM, Animesh Chaturvedi 
animesh.chaturv...@citrix.commailto:animesh.chaturv...@citrix.commailto:animesh.chaturv...@citrix.com

wrote:





Folks



I wanted to provide an update on shutting down of RealhostIp service.

Citrix has decided to move the date by one quarter and the new date will

be Sep 30th.  The new dates allow users of CloudStack additional time for

updating their infrastructure. While testing Realhostip fixes additional

product issues were found which are tracked in following three defects

CLOUDSTACK-6499 [1], CLOUDSTACK-6599 [2], CLOUDSTACK-6824 [3] . These

defects are fixed in 4.4 and master branch. Please verify in your setups.

If you are on older version of CloudStack do plan on upgrading to 4.4 .

Additional details on fixes are available in JIRA.  Updated wiki with

steps will be published shortly.



[1] https://issues.apache.org/jira/browse/CLOUDSTACK-6499

[2] https://issues.apache.org/jira/browse/CLOUDSTACK-6599

[3] https://issues.apache.org/jira/browse/CLOUDSTACK-6824





Thanks

Animesh









-Original Message-

From: John Kinsella [mailto:j...@stratosec.co]

Sent: Thursday, April 17, 2014 8:35 AM

To: 
us...@cloudstack.apache.orgmailto:us...@cloudstack.apache.orgmailto:us...@cloudstack.apache.org;
 
users...@cloudstack.apache.orgmailto:users...@cloudstack.apache.orgmailto:users...@cloudstack.apache.org;

dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org

Subject: REMINDER RealhostIP going away



Reminder, folks - please migrate off RealhostIP.com or you¹re going to

get a

nasty surprise this summer. More info at link below.





https://blogs.apache.org/cloudstack/entry/RealhostIP_service_is_being_re

t

ired

Re: [VOTE] Release Apache CloudStack 4.3.1 round #4

[Chiradeep Vittal]

+1 (binding)
Tested RHEL 6.3 MS with XS 6.2 (fresh install). Followed test procedure 
(s/4.3.0/4.3.1) to verify bits and hash. 

--
Chiradeep

 On Sep 11, 2014, at 1:02 PM, Sebastien Goasguen run...@gmail.com wrote:
 
 
 On Sep 11, 2014, at 3:56 PM, Wei ZHOU ustcweiz...@gmail.com wrote:
 
 Sorry I just notice Rohit made a systemvm template change (commit
 3a0f1550289a8a2a50b0f786821cec2e5c914e62) on 9th, September.
 The systemvm template will be built based on debian 7.5.0 instead of 7.4.0
 
 Wei, can you test and vote ? that would be great
 
 
 
 ==
 
 commit 3a0f1550289a8a2a50b0f786821cec2e5c914e62
 Author: Rohit Yadav rohit.ya...@shapeblue.com
 Date:   Tue Sep 9 18:37:15 2014 +0200
 
   appliance: fix newer debian iso links and checksums for debian
 
   Signed-off-by: Rohit Yadav rohit.ya...@shapeblue.com
   (cherry picked from commit ac5309b8b47ddb1e5588a2d5090f2bcce0e7c2a4)
 
   Conflicts:
   tools/appliance/definitions/systemvm64template/definition.rb
   tools/appliance/definitions/systemvmtemplate/definition.rb
 
 diff --git a/tools/appliance/definitions/systemvm64template/definition.rb
 b/tools/appliance/definitions/systemvm64template/definition.rb
 index f9308a0..bd59025 100644
 --- a/tools/appliance/definitions/systemvm64template/definition.rb
 +++ b/tools/appliance/definitions/systemvm64template/definition.rb
 @@ -3,9 +3,9 @@ Veewee::Definition.declare({
  :memory_size= '256',
  :disk_size = '2500', :disk_format = 'VDI', :hostiocache = 'off',
  :os_type_id = 'Debian_64',
 -  :iso_file = debian-7.4.0-amd64-netinst.iso,
 -  :iso_src = 
 http://cdimage.debian.org/mirror/cdimage/archive/7.4.0/amd64/iso-cd/debian-7.4.0-amd64-netinst.iso
 ,
 -  :iso_md5 = e7e9433973f082a297793c3c5010b2c5,
 +  :iso_file = debian-7.5.0-amd64-netinst.iso,
 +  :iso_src = 
 http://cdimage.debian.org/mirror/cdimage/archive/7.5.0/amd64/iso-cd/debian-7.5.0-amd64-netinst.iso
 ,
 +  :iso_md5 = 8fdb6715228ea90faba58cb84644d296,
  :iso_download_timeout = 1000,
  :boot_wait = 10, :boot_cmd_sequence = [
 'Esc',
 diff --git a/tools/appliance/definitions/systemvmtemplate/definition.rb
 b/tools/appliance/definitions/systemvmtemplate/definition.rb
 index fcaab4d..9b57379 100644
 --- a/tools/appliance/definitions/systemvmtemplate/definition.rb
 +++ b/tools/appliance/definitions/systemvmtemplate/definition.rb
 @@ -3,9 +3,9 @@ Veewee::Definition.declare({
  :memory_size= '256',
  :disk_size = '2500', :disk_format = 'VDI', :hostiocache = 'off',
  :os_type_id = 'Debian',
 -  :iso_file = debian-7.4.0-i386-netinst.iso,
 -  :iso_src = 
 http://cdimage.debian.org/mirror/cdimage/archive/7.4.0/i386/iso-cd/debian-7.4.0-i386-netinst.iso
 ,
 -  :iso_md5 = 7339b668a81b417ac023d73739dc6a03,
 +  :iso_file = debian-7.5.0-i386-netinst.iso,
 +  :iso_src = 
 http://cdimage.debian.org/mirror/cdimage/archive/7.5.0/i386/iso-cd/debian-7.5.0-i386-netinst.iso
 ,
 +  :iso_md5 = 0e6eaacb5a5828473afe90f6df9c8f16,
  :iso_download_timeout = 1000,
  :boot_wait = 10, :boot_cmd_sequence = [
 'Esc',
 
 
 2014-09-11 21:50 GMT+02:00 Sebastien Goasguen run...@gmail.com:
 
 
 On Sep 11, 2014, at 3:47 PM, Mike Tutkowski mike.tutkow...@solidfire.com
 wrote:
 
 OK, I looked into this more.
 
 It turns out the regression test that was failing is only valid in 4.4
 and
 beyond, so it is not a valid test for 4.3.
 
 so can you vote ? we need your binding vote :)
 
 On Thu, Sep 11, 2014 at 11:14 AM, Mike Tutkowski 
 mike.tutkow...@solidfire.com wrote:
 
 I have an integration test failing when trying to create a (XenServer)
 storage repository.
 
 Let me look into it for an hour or so and then I can respond back as to
 if
 it's a code issue.
 
 I'm thinking now it's a problem with XAPI. At first the code wasn't
 compiling due to XAPI changes and now I'm getting this XAPI issue when
 trying to create a storage repository. Seems possibly related.
 
 On Thu, Sep 11, 2014 at 10:37 AM, Erik Weber terbol...@gmail.com
 wrote:
 
 Except heartbleed fix.
 
 I was merely checking if it's even poosible to do an automated upgrade.
 
 The way i know the upgrade process we add some sql magic to check for
 new
 systemvm templates. If that's not made for 4.3.1 it won't do much good
 to
 add it as systemvm-hypervisor-version.
 
 Erik
 11. sep. 2014 15:45 skrev Wei ZHOU ustcweiz...@gmail.com følgende:
 
 As far as I know, there is no systemvm change during upgrade from
 4.3.0
 to
 4.3.1
 
 2014-09-11 15:22 GMT+02:00 Erik Weber terbol...@gmail.com:
 
 On Thu, Sep 11, 2014 at 3:01 PM, Nux! n...@li.nux.ro wrote:
 
 Hello,
 
 I've upgraded and it looks ok for now.
 I have used these system VMs
 http://jenkins.buildacloud.org/view/4.3/job/cloudstack-4.3-systemvm/
 and
 added them as systemvm-kvm-4.3 in the Templates.
 I expected them to replace SystemVM Template (KVM), but they are
 still
 listed as systemvm-kvm-4.3 alongside the old ones.
 Other than that everything seems ok for now.
 Is there an upgrade path from 4.3.0 systemvm template to 4.3.1? Log
 on
 to a
 newly deployed 

Re: Should CloudStack support forced password reset?

[Chiradeep Vittal]

The cloud operator can call the updateUser API themselves? Then they can send 
an email to their users telling them their new password. There is no ‘password 
change’ protocol at the moment. It is assumed that user provisioning and user 
lifecycle is best left to a different system.

From: Demetrius Tsitrelis 
demetrius.tsitre...@citrix.commailto:demetrius.tsitre...@citrix.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Thursday, August 21, 2014 at 11:28 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Should CloudStack support forced password reset?

For legacy reasons the MD5 and plaintext plugins are included in the list of 
authenticators.  If a company has been using CloudStack for awhile they may 
want to move all their users to a stronger plugin such as SHA256SALTED (which 
is now the default).

Is there a mechanism to do that?  It doesn't appear that there is so I propose 
modify the API as follows:


1)  Include a result in the response to the login API which indicates 
whether a user must change his password.

2)  If a user is in this state have him call a new API called 
changeMyPassword.  That API would require his old password and a new password.  
If the calls succeeds then the user can retry the login API with his new 
password.

3)  Add a new parameter named forceUserToChangePassword to the UpdateUser 
API.  An admin would set that parameter value to indicate that a user is 
required to change his password.

Thoughts?

Re: How to re-use CloudStack's keystore?

[Chiradeep Vittal]

I think the keystoreManager provides a generic interface. I’d think that 
certificate re-use is probably not a good idea, might be worth using a 
different cert for different purposes. One compromise will affect only one part 
of the system?

From: Rohit Yadav rohit.ya...@shapeblue.commailto:rohit.ya...@shapeblue.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Friday, August 22, 2014 at 2:35 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: How to re-use CloudStack's keystore?

Hello!

On 18-Aug-2014, at 11:48 am, Rohit Yadav 
rohit.ya...@shapeblue.commailto:rohit.ya...@shapeblue.com wrote:

Is there any documentation on how one can reuse CloudStack’s default java 
keystore keys (private and public) and any recommendation on security 
consideration? If not, can anyone share something on this, how it is 
created/updated, who uses it currently and how one can (re)use the keys for 
x509 based authentication, encryption and decryption.

Ping?

Anyone has any idea how to re-use as I don’t want to duplicate x509 crypto 
usage within the codebase?

Regards,
Rohit Yadav
Software Architect, ShapeBlue
M. +41 779015219 | rohit.ya...@shapeblue.commailto:rohit.ya...@shapeblue.com
Blog: bhaisaab.org | Twitter: @_bhaisaab



Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design  Buildhttp://shapeblue.com/iaas-cloud-design-and-build//
CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Infrastructure 
Supporthttp://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/

This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England  Wales. ShapeBlue Services India LLP is a company 
incorporated in India and is operated under license from Shape Blue Ltd. Shape 
Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is 
operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company 
registered by The Republic of South Africa and is traded under license from 
Shape Blue Ltd. ShapeBlue is a registered trademark.

Re: Developing for Project Support

[Chiradeep Vittal]

IMO, the existing project implementation isn’t a good example (looks like the 
requirements were incomplete).

From: Will Stevens wstev...@cloudops.commailto:wstev...@cloudops.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Monday, August 25, 2014 at 11:20 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Developing for Project Support

Hey All,
I have had a quick look around at some of the different implementations and
I am seeing that projects in CS seem to be handled as 'special' accounts.
The API calls have fields like 'projectid' and such, but those details are
not saved into the DB, the only thing that happens in the DB is a new
project account ID is added to the db for that element (think, networks,
instances, etc...)

If I want to extend the functionality of an existing piece of code which
does not support projects to also support projects, do you know of any
specific examples in the code that are good implementations which I should
be using as a good example to follow for adding project support?

Thanks,

*Will STEVENS*
Lead Developer

*CloudOps* *| *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

Re: TomCat 404

[Chiradeep Vittal]

So says https://cwiki.apache.org/confluence/x/hQLVAQ
I believe folks have had success with Tomcat 7 though (anybody can confirm?)

From: Carlos Reategui car...@reategui.commailto:car...@reategui.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Tuesday, August 26, 2014 at 5:49 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: TomCat 404

Also make sure the default tomcat service is set to NOT start.  Cloudstack
should be the one starting it.


On Tue, Aug 26, 2014 at 5:47 PM, Michael Phillips 
mphilli7...@hotmail.commailto:mphilli7...@hotmail.com
wrote:

Pretty sure they say 6.0.33 is the recommended version...any devs want to
chime in on that?

 Date: Tue, 26 Aug 2014 20:42:11 -0400
 From: m...@daoenix.commailto:m...@daoenix.com
 To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 Subject: TomCat 404

 Greetings,

 I am having issues with tomcat 404 error, I presently have
 Apache Tomcat/6.0.39

 I believe that may be the issue? I am also using KVM / CentoOS 6.5.

 Anyone else have any idea?

 - Mo

Re: rfc: newsystemvm

[Chiradeep Vittal]

Thanks Leo

From: Leo Simons lsim...@schubergphilis.commailto:lsim...@schubergphilis.com
Date: Monday, July 21, 2014 at 7:24 AM
To: Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com, 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org, int-toolkit 
int-tool...@schubergphilis.commailto:int-tool...@schubergphilis.com
Subject: Re: rfc: newsystemvm

Hey Chiradeep,

Thanks for taking a look. I’ve now re-done this work, but carefully and cleanly 
and on top of current master, in 37 small commits instead of 2 scary ones.

Please take a look at

  https://github.com/schubergphilis/cloudstack/compare/systemvm-refactor

Summarizing this kind of thing is always hard...it’s many little things...the 
interesting stuff is at the end/bottom, in particular the two main improvements

  
https://github.com/schubergphilis/cloudstack/commit/142d087f6a97f6ac70a858a35d2fe8b638c58cbb
When working on the systemvm in isolation, or using vagrant or similar tools,
it can be useful to inject a custom SSH key before merging a management server
systemvm.iso into it. This option allows that. It should _not_ have effect
on management-server-managed vms which always get their SSH keys injected.

  
https://github.com/schubergphilis/cloudstack/commit/e2240eaed18000d4d94dbf6a6e40612db1aeda34
The current build downloads its script from master by fetching a cloudstack
tarball. Besides being an unneeded load on the apache git server, this is a
problem when working on a branch and wanting to inject a different set of
scripts. It also makes it pretty likely that the injected copy of the script
will not match what a production release wants, so there is very little
chance of not needing to overwrite the scripts.

Ideally we would just rsync over some files. However, veewee does not provide
an option to do that. In order to keep a 'cleanly veewee-only' build possible,
and work with any recent veewee version, in this change we restor to using
shar (http://en.wikipedia.org/wiki/Shar) to produce an archive which can
execute as a script, which we feed to veewee to execute.

In order to avoid having to re-do this cleanup twice, I also ended up merging 
the systemvm and systemvm64 template definitions, factoring out their small 
differences by inspecting the os architecture.

  
https://github.com/schubergphilis/cloudstack/commit/f570b3921cd52672f841fc5f99cdd96f9737d629
  
https://github.com/schubergphilis/cloudstack/commit/50e91217f90fc952182dccac02a5af06ac33fb45

Everything else…well it pretty much falls into two categories:
  * general code cleanup without functional changes
  * general code defensiveness to survive various jenkins build scenarios

All in all it should help with ongoing maintenance, I think.

Note I still have some work to do (testing, merging this version back into our 
redundant vpc branch, moar testing, ...) before submitting a merge-able 
patchset. But since it’s such a big change and since the testing is a bit slow 
(what with creating and destroying VMs) any early comments would be quite 
useful so I don’t have to re-re-do lots of testing.


Thanks!


Leo

On Jul 18, 2014, at 7:35 PM, Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com wrote:

Thanks Leo. Can you summarize the changes (it is a lot of changes)
From: Leo Simons lsim...@schubergphilis.commailto:lsim...@schubergphilis.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Friday, July 18, 2014 at 7:42 AM
To: int-toolkit 
int-tool...@schubergphilis.commailto:int-tool...@schubergphilis.com, 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: rfc: newsystemvm
Hey folks,
https://github.com/schubergphilis/cloudstack/commit/f125f1564e8921def00dc0235ecca51470a2a22e
https://github.com/schubergphilis/cloudstack/tree/f125f1564e8921def00dc0235ecca51470a2a22e/tools/appliance
This started out as wanting the systemvm build to take 
systemvm/patches/debian/{debian,vpn} from the local machine/branch, rather than 
downloading from the apache git master [1]. In working out how on earth to get 
veewee to do that cleanly (hint: you can’t, hence resorting to shar usage) I 
got quite frustrated with the image rebuild times.
It so happens that veewee has a --skip-to-postinstall instruction which is 
_quite_ useful while debugging these scripts. To get that working requires the 
post install steps to be retryable/convergent. Of course, our existing scripts 
weren’t set up for that. So I had to add a bunch of tests whether changes had 
applied already. Which implied a pretty significant refactor.
I think I was careful enough and I expect this new template will work just as 
well as the old one. This is a change that we can (and probably should?) merge 
to master independently of the redundant VPC work (though

Re: [DISCUSS] Acquire New Ip from a different range on shared networks

[Chiradeep Vittal]

Do you want to acquire IPs for the VIP (front-end)?

From: Silvano Nogueira Buback 
silv...@corp.globo.commailto:silv...@corp.globo.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Monday, July 14, 2014 at 2:59 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: [DISCUSS] Acquire New Ip from a different range on shared networks

Hi guys,

At Globo.com we are working in a load balancer plugin for Cloudstack
with a network api developed internally. This api manages shared networks
and is working with cloudstack 4.3 (as a network guru implementation). Our
load balancers are in a different network, so to implement a network
element of load balancer, first I need to acquire an IP from the load
balancers network. What is the best way to do this?

I looked at portable IPs and that makes sense to me, but I would prefer
a solution where my guru can give this IP to the network. Is there any
other way?

Thanks in advance,

Silvano Buback

Re: [QUESTION] Baremetal DHCP Server - Abstracting Router VM

[Chiradeep Vittal]

Are you trying to get rid of the virtual router? Have you looked at the DNSAPI 
implementation discussed earlier?

From: ilya musayev 
ilya.mailing.li...@gmail.commailto:ilya.mailing.li...@gmail.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Monday, July 21, 2014 at 11:35 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: [QUESTION] Baremetal DHCP Server - Abstracting Router VM

We are trying to abstract router vm completely from our environment as
it has dual nics which is big no no in hardened security environments.
This is for shared (non-vpc) advanced security zone.

CloudStack already has a support for Baremetal DHCP Server under
Network Service Providers.

Would anyone provide the context on how one go about using it? I assume
we would need to write a support of some sort on our end. Examples and
documentation would be appreciated.

Thank you
ilya

Re: rfc: newsystemvm

[Chiradeep Vittal]

Thanks Leo. Can you summarize the changes (it is a lot of changes)

From: Leo Simons lsim...@schubergphilis.commailto:lsim...@schubergphilis.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Friday, July 18, 2014 at 7:42 AM
To: int-toolkit 
int-tool...@schubergphilis.commailto:int-tool...@schubergphilis.com, 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: rfc: newsystemvm

Hey folks,

https://github.com/schubergphilis/cloudstack/commit/f125f1564e8921def00dc0235ecca51470a2a22e
https://github.com/schubergphilis/cloudstack/tree/f125f1564e8921def00dc0235ecca51470a2a22e/tools/appliance

This started out as wanting the systemvm build to take 
systemvm/patches/debian/{debian,vpn} from the local machine/branch, rather than 
downloading from the apache git master [1]. In working out how on earth to get 
veewee to do that cleanly (hint: you can’t, hence resorting to shar usage) I 
got quite frustrated with the image rebuild times.

It so happens that veewee has a --skip-to-postinstall instruction which is 
_quite_ useful while debugging these scripts. To get that working requires the 
post install steps to be retryable/convergent. Of course, our existing scripts 
weren’t set up for that. So I had to add a bunch of tests whether changes had 
applied already. Which implied a pretty significant refactor.

I think I was careful enough and I expect this new template will work just as 
well as the old one. This is a change that we can (and probably should?) merge 
to master independently of the redundant VPC work (though the `apt-get install 
chef` would need to be taken out). But, given how big of a chunk of code has 
changed here, before upstreaming (a version of) this to apache we (I) need to 
do more testing. So for now I’ve put this change next to the existing 
definitions rather than replace ‘em, to not block anything else.

Comments/thoughts?


cheers,


Leo


[1] 
https://github.com/schubergphilis/cloudstack/blob/master/tools/appliance/definitions/systemvmtemplate/postinstall.sh#L228

Begin forwarded message:
...
M tools/appliance/build.sh
...
A tools/appliance/definitions/newsystemvm/apt_upgrade.sh
A tools/appliance/definitions/newsystemvm/authorized_keys.sh
A tools/appliance/definitions/newsystemvm/build_time.sh
A tools/appliance/definitions/newsystemvm/cleanup.sh
A tools/appliance/definitions/newsystemvm/configure_acpid.sh
A tools/appliance/definitions/newsystemvm/configure_conntrack.sh
A tools/appliance/definitions/newsystemvm/configure_grub.sh
A tools/appliance/definitions/newsystemvm/configure_locale.sh
A tools/appliance/definitions/newsystemvm/configure_login.sh
A tools/appliance/definitions/newsystemvm/configure_networking.sh
A tools/appliance/definitions/newsystemvm/configure_systemvm_services.sh
A tools/appliance/definitions/newsystemvm/definition.rb
A tools/appliance/definitions/newsystemvm/install_systemvm_packages.sh
A tools/appliance/definitions/newsystemvm/preseed.cfg
A tools/appliance/definitions/newsystemvm/zerodisk.sh
A tools/appliance/shar_cloud_scripts.sh
A tools/appliance/test.sh
...
Work in progress to rewrite systemvm box definition in clean code.
* Refactor build.sh to use functions and pretty logging.
* Add a new test.sh which tries several different build.sh invocations.
* Add a new 'debianbase' definition which is just the basic debian box (no 
systemvm functionality). This is used for testing.
* Add a new 'newsystemvm' definition which is a work-in-progress replacement of 
the 'systemvmtemplate' definition:
** reduce duplication between preseed.cfg and postinstall.sh
** remove duplication between postinstall.sh and cloud-scripts
** install cloud-scripts from current working copy instead of tying to a 
download of the apache master branch
** split up postinstall.sh into small utility scripts and clean up that code
** clean up preseed.cfg to match current veewee/debian best practices and 
remove cruft

Re: Infiniband support

[Chiradeep Vittal]

Is this for storage (presumably)? You can use a “pre-setup” storage

From: Dan Crowe dan.cr...@shapeblue.commailto:dan.cr...@shapeblue.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Wednesday, July 16, 2014 at 3:49 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Infiniband support

Hi all

I have a query re support for Infiniband on ACS - are there any plans to 
support moving forward?  I did see some mails on the list, but am unsure of 
current status?

Thanks

Dan

Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design  Buildhttp://shapeblue.com/iaas-cloud-design-and-build//
CSForge - rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Infrastructure 
Supporthttp://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/

This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England  Wales. ShapeBlue Services India LLP is a company 
incorporated in India and is operated under license from Shape Blue Ltd. Shape 
Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is 
operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company 
registered by The Republic of South Africa and is traded under license from 
Shape Blue Ltd. ShapeBlue is a registered trademark.

Re: vms stopped while restarted by user

[Chiradeep Vittal]

Agree. Not sure why your system is so slow, but these parameters should help

From: Daan Hoogland daan.hoogl...@gmail.commailto:daan.hoogl...@gmail.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Tuesday, July 15, 2014 at 6:29 AM
To: Tomasz Zięba t.a.zi...@gmail.commailto:t.a.zi...@gmail.com
Cc: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org, Marcus Sorensen 
shadow...@gmail.commailto:shadow...@gmail.com, Damoder Reddy 
damoder.re...@citrix.commailto:damoder.re...@citrix.com
Subject: vms stopped while restarted by user

Tomasz,

I can only fantasize on the full rationale of the implementation of
the retry but in general it makes sense to me. A job has a time to try
and a times tried field. the worker manager has time to sleep and max
retries. As you can see below these are read from the configuration:

value = params.get(time.to.sleep);
_timeToSleep = NumbersUtil.parseInt(value, 60) * 1000;

value = params.get(max.retries);
_maxRetries = NumbersUtil.parseInt(value, 5);

there is also

value = params.get(stop.retry.interval);
_stopRetryInterval = NumbersUtil.parseInt(value, 10 * 60);


The time.to.sleep and stop.retry.interval seem to jointly explain the
ten minute scenario you described in the bug report. They don't do
completely as some of the handling of the values is based on
bitshifting and not on datetime calculus (using mixed factors of
1000,60,60,24 and 365.25)
You can try and play with those to tune your setting. In any case
looking at the vm to decide to restart the vm is not usefull as
Cloudstack will do some cleanup after stopping the instance. You
should really wait untill cloudstack reports on the job with either
succes or error.

On Tue, Jul 15, 2014 at 3:12 PM, Tomasz Zięba 
t.a.zi...@gmail.commailto:t.a.zi...@gmail.com wrote:
Hello,

The user does not receive confirmation of the operation.
From the point of view of user input it looks like the machine itself
stopped.

As you can see in the logs, the ACS explicitly sends stop command, as if
they press the Stop button from the GUI, so it is aware of the action from
the perspective of the ACS / MS.

I can not point out which component may be responsible for it.
We have tried to analyze the code to understand what is happening,
but the part of the code related to HAWorker is not very clear.
Unfortunately we could not find online any assumptions on the level of
architecture / design of HAWorker.

Maybe method of small steps help find a solution.
First a small question: why HAWorker performs reschedule. What was the idea
for such action.




2014-07-15 14:26 GMT+02:00 Daan Hoogland 
daan.hoogl...@gmail.commailto:daan.hoogl...@gmail.com:

Tomasz,

As I understand the issue this is what happens:

The user stops the vm from the UI
The MS sends the stop command to the machine
The machine stops and takes a long time for it
The MS reschedules the stop
Then machine stops
the user starts the machine
the MS get by stopping the machine

Did the user ever get a confirmation that the machine was stopped or
that stopping failed? If so, this is the bug, as it seems the MS works
as designed.

Don't get me wrong; I am trying to figure out a path to a solution for
you. I am not convinced there is a bug in the management server
though. That doesn't mean it can be in cloudstack over all. Either at
a design level or for instance in some inter-process communication.

kind regards,
Daan Hoogland


On Fri, Jul 11, 2014 at 2:45 PM, Tomasz Zięba 
t.a.zi...@gmail.commailto:t.a.zi...@gmail.com wrote:
 Hello,

 We are waiting for the patch with longingly.

 Error associated with self-closing of machines causes very serious
 complications, both from the technical (users need to wait for 10
 minutes
 and check if the machine is not closed automatically) as well as the
 business side (this problem does not look very professional from the
 user
 side)

 Given that:
 - An error has been detected in February so 5 months ago,
 - in earlier versions  (3.0.2) error does not exists,
 - there is a procedure to reproduce this error,

 we would be very grateful if this issue will be resolved in ACS4.4.


 --
 Regards,
 Tomasz Zięba
 Twitter: @TZieba
 LinkedIn: pl.linkedin.com/pub/tomasz-zięba-ph-d/3b/7a8/ab6/



--
Daan

Re: [DISCUSS] [PROPOSAL] Implementation of DNS Provider for Bind (for 4.5)

[Chiradeep Vittal]

I think the question is relevant to network creation as well. If I provide a 
domain that already exists, what is the result?

A couple of other comments:
 - are we going to handle the case of secondary IP addresses?
 - DNSAPI sounds generic, but it actually refers to one specific API 
architected by Globo. To avoid confusion, would it make sense to rename it 
GloboDNSAPI? Alternately, give the DNSAPI project a less generic name (e.g., 
vincular)

From: David Nalley da...@gnsa.usmailto:da...@gnsa.us
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Friday, July 11, 2014 at 10:06 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [DISCUSS] [PROPOSAL] Implementation of DNS Provider for Bind (for 
4.5)

I tend to agree with Erik, flexibility solves the problem for more
people, while solution 1 is likely the easiest to implement. I am not
sure that it makes sense for most people though - and would really
only work for greenfield deployments or clouds that had all of the DNS
entries relating to instances in the cloud.

First question; how are you intending on using it? Which of those
solutions works for you?

--David



On Thu, Jul 3, 2014 at 3:49 PM, Erik Weber 
terbol...@gmail.commailto:terbol...@gmail.com wrote:
To push that choice over to the operator you could add it as a
global/zone/network option.

As an operator i would prefer to have my own logic to handle cleanup, but
this varies for everyone hence the option :-)

Erik
3. juli 2014 21:45 skrev Silvano Nogueira Buback 
silv...@corp.globo.commailto:silv...@corp.globo.com
følgende:

Hi guys,

 I think you are busy because 4.4 release tasks, but I'm worried about
the time to 4.5 feature freeze. I put the documentation of feature in wiki
as requested and I hoped people read there and make some comments here.

To help, I will put design issues that are in document, one by one, and we
can discuss in this thread. After each discussion I will change the
document.

I have one question about removing DNS domain when network has been
deleted. In my current implementation I remove DNS domain when network is
removed. But if the DNS domain is shared with another network or maybe is a
dns domain used outside ACS this can be a problem. What I can do with DNS
domain when network is removed:

1. Keep the current implementation. Always deleted DNS domain when
network is removed (works well if the ACS is the only manager for the
DNS
(one network domain per network).
2. Remove DNS domain only if the domain was created by ACS. This can be
a problem if someone put records after ACS creation.
3. Remove DNS domain only if there is no more records there. Maybe DNS
domain can stay forever there because an inconsistency that keep only
one
record.


Which one is the best?

[]'s,

Silvano Buback



On Thu, Jun 26, 2014 at 11:34 AM, Silvano Nogueira Buback 
silv...@corp.globo.commailto:silv...@corp.globo.com wrote:

 Thank you David.

 I put design documents on wiki:

https://cwiki.apache.org/confluence/display/CLOUDSTACK/Bind+and+PowerDNS+integration+by+Globo+DNSAPI
.
 I create an issue https://issues.apache.org/jira/browse/CLOUDSTACK-6998
 too.

 I look forward to hearing your feedbacks.

 []'s,

 Silvano Buback


 On Wed, Jun 25, 2014 at 5:50 PM, David Nalley 
 da...@gnsa.usmailto:da...@gnsa.us wrote:

 On Wed, Jun 25, 2014 at 4:38 PM, Silvano Nogueira Buback
 silv...@corp.globo.commailto:silv...@corp.globo.com wrote:
  Hi guys,
 
 I finish the first version of design document:
 

https://docs.google.com/document/d/1kbPQJrBC87ZtR-t7LwHFDzAmT436ShtjwKE84FVfByM/pub
  .
 
 Someone could give me access to put design documents in wiki?
Bellow
 the
  username of people work with Cloudstack in Globo.com and need access.
 
  snbuback silv...@corp.globo.commailto:silv...@corp.globo.com
  daniel.simoes 
  daniel.sim...@corp.globo.commailto:daniel.sim...@corp.globo.com
  lokama - lok...@gmail.commailto:lok...@gmail.com
 
  Regards,
 
  Silvano Buback
 
 
 
  On Thu, Jun 19, 2014 at 11:29 AM, Silvano Buback 
  snbub...@gmail.commailto:snbub...@gmail.com
 wrote:
 
  Of course, I forgotten my account info:
  snbuback / silv...@corp.globo.commailto:silv...@corp.globo.com
 


 Done.

 --David

Re: KVM with OVS GRE has been supported on 4.3 release ?

[Chiradeep Vittal]

I believe there is a bug in 4.3 (but fixed in 4.4)

From: lifuhui gdblessu99...@hotmail.commailto:gdblessu99...@hotmail.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Sunday, July 13, 2014 at 8:02 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: KVM with OVS GRE has been supported on 4.3 release ?

Hello, everyone,
Has anyone used OVS GRE with KVM on 4.3 release for isolation guest network ?
Why there is no code support such as in

private String getGreEndpointIP(Host host, Network nw)
...
switch (hvType) {
case XenServer:
String label = physNetTT.getXenNetworkLabel();
if ((label != null)  (!label.equals())) {
physNetLabel = label;
}
break;
default:
throw new CloudRuntimeException(Hypervisor  +
hvType.toString() +
 unsupported by OVS Tunnel Manager);
...

PS: Using 4.3 release code

If not support, why a release doc has mentioned as below:

3.4 The OVS Plugin
3.4.1 Introduction to the OVS Plugin
The OVS plugin is the native SDN implementations in CloudStack, using GRE 
isolation method. The plugin can be
used by CloudStack to implement isolated guest networks and to provide 
additional services like NAT, port forwarding
and load balancing.
Features of the OVS Plugin
The following table lists the CloudStack network services provided by the OVS 
Plugin.
Network Service
Virtual Networking
Static NAT
Port Forwarding
Load Balancing
CloudStack version
= 4.0
= 4.3
= 4.3
= 4.3
Table: Supported Services
Note: The Virtual Networking service was originally called ‘Connectivity’ in 
CloudStack 4.0
The following hypervisors are supported by the OVS Plugin.
Hypervisor
XenServer
KVM
CloudStack version
= 4.0
= 4.3
Table: Supported Hypervisors

Thanks

Re: Network usage in Basic Zone

[Chiradeep Vittal]

It depends on what you want to charge for. Do you want to charge for VM-to-VM 
traffic? Or only VM-to-Internet traffic?
Some folks use sFlow on their routers and then use this integration with 
CloudStack:
https://cwiki.apache.org/confluence/x/cQj8AQ


From: Wei ZHOU ustcweiz...@gmail.commailto:ustcweiz...@gmail.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Tuesday, July 1, 2014 at 2:15 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: Network usage in Basic Zone

Nux,
I will change and commit it to 4.3 branch in the next days.

-Wei


2014-07-01 10:28 GMT+02:00 Nux! n...@li.nux.romailto:n...@li.nux.ro:

Thanks Sebastian and Wei, looking forward to having this working.

Wei, if you have any patches for 4.3 I'd be more than willing to test.

Lucian

--
Sent from the Delta quadrant using Borg technology!

Nux!
www.nux.ro


- Original Message -
 From: Wei ZHOU ustcweiz...@gmail.commailto:ustcweiz...@gmail.com
 To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 Sent: Tuesday, 1 July, 2014 9:10:24 AM
 Subject: Re: Network usage in Basic Zone

 The network usage for individual vms can be fetched from hypervisor.
 I have changed for KVM (only), but not committed yet.



 2014-07-01 9:59 GMT+02:00 sebgoa run...@gmail.commailto:run...@gmail.com:

  Morning,
 
  Nux opened https://issues.apache.org/jira/browse/CLOUDSTACK-6993
 
  Talking to Fabrice it seems to be expected behavior. In Basic zones,
  traffic does not go through the VR and the network is shared.
 
  Fabrice pointed me to:
 
 
https://github.com/apache/cloudstack/blob/master/server/src/com/cloud/network/router/VirtualNetworkApplianceManagerImpl.java#L910
 
  Does everyone agree with this ?
 
  How could we record network usage in Basic zones ?
 
  -Sebastien

Re: openvswitch version with cloudstack

[Chiradeep Vittal]

We do not know. But with OVS moving so rapidly, I would generally expect 
problems with older versions of OVS.

From: Sebastien Goasguen run...@gmail.commailto:run...@gmail.com
Date: Wednesday, July 2, 2014 at 2:22 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org, Wido den 
Hollander w...@widodh.nlmailto:w...@widodh.nl, Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com
Cc: Kuang-Ching Wang 
kuangching.w...@gmail.commailto:kuangching.w...@gmail.com
Subject: Re: openvswitch version with cloudstack


On Jul 1, 2014, at 1:03 PM, Kuang-Ching Wang 
kuangching.w...@gmail.commailto:kuangching.w...@gmail.com wrote:

With ubuntu apt-get, openvswitch 1.4.6 is installed.  Is there any reason that 
cloudstack cannot work with 1.4.6, and have to work with 1.9.0 as documented 
here 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/KVM+with+OpenVSwitch?
Thanks,
KC

let me copy Wido our Ubuntu packaging guru and Chiradeep who wrote the native 
GRE controller.

-sebastien

Re: What to modify to increase instance password length?

[Chiradeep Vittal]

Also, to the original question, the password is meant to be temporary — you are 
supposed to login to the vm and change it to whatever standards you have.

From: Harikrishna Patnala 
harikrishna.patn...@citrix.commailto:harikrishna.patn...@citrix.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Wednesday, June 25, 2014 at 10:44 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: What to modify to increase instance password length?

It is better to use ConfigKey class to define configuration parameters, this 
handles default values in case of NULL or db entry missing and no need to put 
in upgrade script.

static final ConfigKeyLong CustomDiskOfferingMaxSize = new 
ConfigKeyLong(Advanced,
Long.class,
custom.diskoffering.size.max,
1024,
Maximum size in GB for custom disk offering.,
true
);

More info@ https://cwiki.apache.org/confluence/display/CLOUDSTACK/Configuration

Thanks
Harikrishna

On 26-Jun-2014, at 10:39 am, Koushik Das 
koushik@citrix.commailto:koushik@citrix.com wrote:

Shouldn't the code handle the scenario if vm.password.length is missing from 
db? In this case the upgrade script will put it but if someone accidentally 
removes it.
-Original Message-
From: Nux! [mailto:n...@li.nux.ro]
Sent: Thursday, 26 June 2014 2:55 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: What to modify to increase instance password length?
Wunderbar! Thanks guys!
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
- Original Message -
From: Daan Hoogland daan.hoogl...@gmail.commailto:daan.hoogl...@gmail.com
To: dev dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Sent: Wednesday, 25 June, 2014 8:35:00 PM
Subject: Re: What to modify to increase instance password length?
pulled both
On Wed, Jun 25, 2014 at 9:33 PM, Ian Duffy 
i...@ianduffy.iemailto:i...@ianduffy.ie wrote:
Yeah sorry, just noticed and fixed it myself.
On 25 June 2014 20:32, Daan Hoogland 
daan.hoogl...@gmail.commailto:daan.hoogl...@gmail.com wrote:
yeah, so I should probably get that one in as well?
On Wed, Jun 25, 2014 at 9:28 PM, Ian Duffy 
i...@ianduffy.iemailto:i...@ianduffy.ie wrote:
:( Get them solved?
Fixed a styecheck issue in 2bb12f2a85c8497a15cd7b17099aefbd7c486f9b
needed
to remove unused imports.
On 25 June 2014 20:24, Daan Hoogland 
daan.hoogl...@gmail.commailto:daan.hoogl...@gmail.com wrote:
c70cadff0e3d70f625e7160a3aee600a3198cc67 in 4.4 it had conflicts
in the sql:(
On Wed, Jun 25, 2014 at 8:53 PM, Ian Duffy 
i...@ianduffy.iemailto:i...@ianduffy.ie wrote:
Just pushed a change for this to the 4.4-forward branch.
Daan, will you review / cherrypick?
96412e3e58fd1ced9d269e4552aaa6410bedf556
Testing done:
Brought up simulator.
Changed password flag for the builtin template.
Brought up VM, password was displayed at length of 6. Stopped
the VM,
reset
the password, new password was displayed at length of 6.
Went into global settings, modified the value for
vm.password.length
to
20.
Restarted the management server.
Created a new VM, password was displayed at length of 20.
Stopped the
VM,
reset the password, new password was displayed at length of 20.
Thanks,
Ian
On 25 June 2014 18:50, Nux! n...@li.nux.romailto:n...@li.nux.ro wrote:
Volunteer to do it in time for 4.4?
Lucian
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
- Original Message -
From: ilya musayev 
ilya.mailing.li...@gmail.commailto:ilya.mailing.li...@gmail.com
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Sent: Wednesday, 25 June, 2014 6:30:25 PM
Subject: Re: What to modify to increase instance password length?
You should ask if this can be done as global setting variable -
not
hard
coded.
This should be an easy one.
On 6/25/14, 10:14 AM, Nux! wrote:
I should submit a bug report to rewrite ACS in a scripting
language.
Cheers :)
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
- Original Message -
From: Ian Duffy i...@ianduffy.iemailto:i...@ianduffy.ie
To: CloudStack Dev 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Sent: Wednesday, 25 June, 2014 6:11:23 PM
Subject: Re: What to modify to increase instance password length?
Afaik yes. (Will to be corrected on this but it appears to be
hard
coded)
On 25 June 2014 18:06, Nux! n...@li.nux.romailto:n...@li.nux.ro wrote:
Thanks, Ian,
This means I need to modify the source, rebuild the RPMs and
update,
right? (ie it's not something that I can just modify on the
mgmt
server
right now).
Lucian
--
Sent from the Delta quadrant using Borg technology!
Nux!
www.nux.ro
- Original Message -
From: Ian Duffy i...@ianduffy.iemailto:i...@ianduffy.ie
To: CloudStack Dev 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Sent: Wednesday, 25 June, 2014 6:02:12 

Re: NetworkOrchestrator selects 2 NetworkGurus at one time....

[Chiradeep Vittal]

For 4.3/4.4, I’m guessing this is the same solution.

For 4.5, here’s a couple of options we could implement:

  1.  New isolation provider (“BrocadeVLAN” or “JuniperEXVLAN”)
  2.  When creating the network offering, the administrator gets to select the 
guru
  3.  New VLAN provider mechanism.

From: Pradeep Cloudstack 
pradeepcloudst...@yahoo.commailto:pradeepcloudst...@yahoo.com
Reply-To: Pradeep Cloudstack 
pradeepcloudst...@yahoo.commailto:pradeepcloudst...@yahoo.com
Date: Wednesday, June 25, 2014 at 3:06 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org, Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com
Cc: Sheng Yang sheng.y...@citrix.commailto:sheng.y...@citrix.com, Jayapal 
Reddy Uradi 
jayapalreddy.ur...@citrix.commailto:jayapalreddy.ur...@citrix.com, Alena 
Prokharchyk alena.prokharc...@citrix.commailto:alena.prokharc...@citrix.com
Subject: Re: NetworkOrchestrator selects 2 NetworkGurus at one time

We have a use-case where we will patch an existing 4.3 installation with our 
plugin.
We are facing the same issue .
In 4.2, we used to disable the entry for ExternalNetworkGuru in
componentContext.xml as part of installing the patch.

How do we do this in 4.3 (on an existing installation) ?

-Pradeep


From: Ritu Sabharwal rsabh...@brocade.commailto:rsabh...@brocade.com
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org; Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com
Cc: Sheng Yang sheng.y...@citrix.commailto:sheng.y...@citrix.com; Jayapal 
Reddy Uradi 
jayapalreddy.ur...@citrix.commailto:jayapalreddy.ur...@citrix.com; Alena 
Prokharchyk alena.prokharc...@citrix.commailto:alena.prokharc...@citrix.com
Sent: Thursday, June 12, 2014 12:13 AM
Subject: RE: NetworkOrchestrator selects 2 NetworkGurus at one time

Thanks Chiradeep and Murali for the reply!

I am thinking of explicitly telling ExternalNetworkGuru to skip design when 
Brocade plugin is designing the network. I don't want to disable 
ExternalNetworkGuru from default build when Brocade plugin is not present so 
won't exclude it from the spring class loader.

Thanks  Regards,
Ritu S.



-Original Message-
From: Murali Reddy 
[mailto:murali.re...@citrix.commailto:murali.re...@citrix.com]
Sent: Tuesday, June 10, 2014 10:54 PM
To: Chiradeep Vittal; 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Cc: Sheng Yang; Jayapal Reddy Uradi; Alena Prokharchyk
Subject: Re: NetworkOrchestrator selects 2 NetworkGurus at one time

This is know design issue. Unlike service orchestration (which has prescriptive 
way to tell which network elements to be called for with network offerings ) 
there is no such logic for network design. Orchestrator just loops through all 
the network guru's asking to design the network which can results in one or 
more networks. Hugo did a cleanup [1] but I believe it was not merged as there 
was no consensus. There is 1-1 mapping between isolation type and Guru but In 
this case both Brocade Guru and ExternalNetworkGuru will attempt to design the 
VLAN isolated networks.

One in-elegent solution is to hard code ExternalGuestNetworu guru to skip 
network deign when Brocade plug-in is supposed to do design the network. Other 
option could be exclude ExternalNetworkGuru bean from spring class loader.

[1] https://www.mail-archive.com/dev@cloudstack.apache.org/msg17344.html

From: Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com
Date: Wednesday, 11 June 2014 6:24 AM
To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Cc: Sheng Yang 
sheng.y...@citrix.commailto:sheng.y...@citrix.commailto:sheng.y...@citrix.commailto:sheng.y...@citrix.com,
 Murali Reddy 
murali.re...@citrix.commailto:murali.re...@citrix.commailto:murali.re...@citrix.commailto:murali.re...@citrix.com,
 Jayapal Reddy Uradi 
jayapalreddy.ur...@citrix.commailto:jayapalreddy.ur...@citrix.commailto:jayapalreddy.ur...@citrix.commailto:jayapalreddy.ur...@citrix.com,
 Alena Prokharchyk 
alena.prokharc...@citrix.commailto:alena.prokharc...@citrix.commailto:alena.prokharc...@citrix.commailto:alena.prokharc...@citrix.com
Subject: Re: NetworkOrchestrator selects 2 NetworkGurus at one time

That is strange. Looks like a bug to me. That is because the 
ExternalGuestNetworkGuru returns 'true' for canHandle.

From: Ritu Sabharwal 
rsabh...@brocade.commailto:rsabh...@brocade.commailto:rsabh...@brocade.commailto:rsabh...@brocade.com
Reply-To: 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.orgmailto:dev

Re: [DISCUSS] Global Settings - Inheritance Rules

[Chiradeep Vittal]

Sounds reasonable, but I think it would take a certain amount of refactoring.

From: ilya musayev 
ilya.mailing.li...@gmail.commailto:ilya.mailing.li...@gmail.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Monday, June 23, 2014 at 2:37 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: [DISCUSS] Global Settings - Inheritance Rules

Not certain if this mechanism exists, if it does, please let me know.

Some settings in ACS that are defined as global setting - can be used on
much lower level, for example
vm.full.clone - can be used on global, zone, pod, cluster and possibly
even VM itself.
systemvm.use.local.disk - should be a global and zone
vm.instancename.flag - can be used on zone level, pod and cluster (or
even VM)

Instead of porting one call at a time based on customer request, would
it be possible to define Global Setting inheritance rule set. For
example if inheritance rule is set to global, it can be overriden on
zone leve, and even cluster or host. We would need a settings
inheritance tab - that shows what configs have been inherited from
upstream and ability to override the config on lower level.

I'd propose for inheritance to go from Global Setting, Zone, Pod,
Cluster, Host and VM. Needless to say, not all Global Settings can be in
each category.

Any thoughts on this?

Re: [API]How to check how many IPs are available in guest network

[Chiradeep Vittal]

Not sure I understand. Would this be a single network encompassing several 
vlans and ip ranges?

From: ilya musayev 
ilya.mailing.li...@gmail.commailto:ilya.mailing.li...@gmail.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Tuesday, June 17, 2014 at 6:39 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [API]How to check how many IPs are available in guest network


Chiradeep,

So i did a little more digging in your example, I also realized you code
has assumption that i have /24 entirely dedicated to CloudStack :)

Welcome to enterprise madness where IP space, while being internal and
free - is close guarded and restricted.

Do we have a concept of IP Pools in future versions? That is an
aggregate of smaller ip ranges that end user can group into 1 entity. So
if range x runs out of IP space, we just go to the next range and vlan
in that pool, without returning and error - you ran out of IP space.

Thanks,
ilya



On 6/16/14, 12:56 PM, Chiradeep Vittal wrote:
used=$(cloudmonkey list virtualmachines
networkid=f5119c7a-a5b8-47aa-bf26-d0f41dfa2d6b | grep ipaddress | wc -l)
space=$(cloudmonkey list networks id=f5119c7a-a5b8-47aa-bf26-d0f41dfa2d6b
| grep cidr | awk -F: '{print $2}' | awk -F / '{print $2}')

space=${space%\,}
let space=32 - $space
let space=³2**$space - 1²



let avail=$space - $used -1²
echo ³Available IPs = $avail




From:  ilya musayev 
ilya.mailing.li...@gmail.commailto:ilya.mailing.li...@gmail.com
Reply-To:  dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date:  Monday, June 16, 2014 at 3:21 PM
To:  dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject:  [API]How to check how many IPs are available in guest network


Is there a way to check how many guest IPs are available in specific
guest network?

Current cloudstack reporting is not granular enough, i could not find a
command that says listAvailableIPs or something similar.

Any help is appreciated.

Thanks,
ilya

Re: [API]How to check how many IPs are available in guest network

[Chiradeep Vittal]

Or is it that the client doesn’t care about the network (since all networks are 
reachable from one-another) and just wants to run a VM.
This is called a basic zone :)
Some higher level portal could achieve this logic for an advanced zone, I guess.


From: Chiradeep Vittal 
chiradeep.vit...@citrix.commailto:chiradeep.vit...@citrix.com
Date: Tuesday, June 17, 2014 at 10:49 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [API]How to check how many IPs are available in guest network

Not sure I understand. Would this be a single network encompassing several 
vlans and ip ranges?

From: ilya musayev 
ilya.mailing.li...@gmail.commailto:ilya.mailing.li...@gmail.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Tuesday, June 17, 2014 at 6:39 PM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [API]How to check how many IPs are available in guest network


Chiradeep,

So i did a little more digging in your example, I also realized you code
has assumption that i have /24 entirely dedicated to CloudStack :)

Welcome to enterprise madness where IP space, while being internal and
free - is close guarded and restricted.

Do we have a concept of IP Pools in future versions? That is an
aggregate of smaller ip ranges that end user can group into 1 entity. So
if range x runs out of IP space, we just go to the next range and vlan
in that pool, without returning and error - you ran out of IP space.

Thanks,
ilya



On 6/16/14, 12:56 PM, Chiradeep Vittal wrote:
used=$(cloudmonkey list virtualmachines
networkid=f5119c7a-a5b8-47aa-bf26-d0f41dfa2d6b | grep ipaddress | wc -l)
space=$(cloudmonkey list networks id=f5119c7a-a5b8-47aa-bf26-d0f41dfa2d6b
| grep cidr | awk -F: '{print $2}' | awk -F / '{print $2}')

space=${space%\,}
let space=32 - $space
let space=³2**$space - 1²



let avail=$space - $used -1²
echo ³Available IPs = $avail




From:  ilya musayev 
ilya.mailing.li...@gmail.commailto:ilya.mailing.li...@gmail.com
Reply-To:  dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date:  Monday, June 16, 2014 at 3:21 PM
To:  dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject:  [API]How to check how many IPs are available in guest network


Is there a way to check how many guest IPs are available in specific
guest network?

Current cloudstack reporting is not granular enough, i could not find a
command that says listAvailableIPs or something similar.

Any help is appreciated.

Thanks,
ilya

Re: [API]How to check how many IPs are available in guest network

[Chiradeep Vittal]

used=$(cloudmonkey list virtualmachines
networkid=f5119c7a-a5b8-47aa-bf26-d0f41dfa2d6b | grep ipaddress | wc -l)
space=$(cloudmonkey list networks id=f5119c7a-a5b8-47aa-bf26-d0f41dfa2d6b
| grep cidr | awk -F: '{print $2}' | awk -F / '{print $2}')

space=${space%\,}
let space=32 - $space
let space=³2**$space - 1²



let avail=$space - $used -1²
echo ³Available IPs = $avail




From:  ilya musayev ilya.mailing.li...@gmail.com
Reply-To:  dev@cloudstack.apache.org dev@cloudstack.apache.org
Date:  Monday, June 16, 2014 at 3:21 PM
To:  dev@cloudstack.apache.org dev@cloudstack.apache.org
Subject:  [API]How to check how many IPs are available in guest network


Is there a way to check how many guest IPs are available in specific
guest network?

Current cloudstack reporting is not granular enough, i could not find a
command that says listAvailableIPs or something similar.

Any help is appreciated.

Thanks,
ilya

Re: [API]How to check how many IPs are available in guest network

[Chiradeep Vittal]

Can you explain does not work

--
Chiradeep

 On Jun 16, 2014, at 10:02 PM, ilya musayev ilya.mailing.li...@gmail.com 
 wrote:
 
 Thanks Chiradeep and Wei
 
 Unfortunately neither gave me what i needed, so i wrote this SQL script:
 
 SELECT b.name, count(*) FreeIPCount
 FROM cloud.user_ip_address a, cloud.networks b
 WHERE b.id=a.source_network_id
 AND a.state = Free
 and b.name is not null
 group by b.name;
 
 Chiradeep, for some reason list virtualmachines calls in cloudmonkey does not 
 always work, especially if you have a very large environment with several 
 thousand VMs. I'd think it has something to do with how cloudmonkey buffers 
 the output, but i dont know enough about its inner working.
 
 On 6/16/14, 12:56 PM, Chiradeep Vittal wrote:
 used=$(cloudmonkey list virtualmachines
 networkid=f5119c7a-a5b8-47aa-bf26-d0f41dfa2d6b | grep ipaddress | wc -l)
 space=$(cloudmonkey list networks id=f5119c7a-a5b8-47aa-bf26-d0f41dfa2d6b
 | grep cidr | awk -F: '{print $2}' | awk -F / '{print $2}')
 
 space=${space%\,}
 let space=32 - $space
 let space=³2**$space - 1²
 
 
 
 let avail=$space - $used -1²
 echo ³Available IPs = $avail
 
 
 
 
 From:  ilya musayev ilya.mailing.li...@gmail.com
 Reply-To:  dev@cloudstack.apache.org dev@cloudstack.apache.org
 Date:  Monday, June 16, 2014 at 3:21 PM
 To:  dev@cloudstack.apache.org dev@cloudstack.apache.org
 Subject:  [API]How to check how many IPs are available in guest network
 
 
 Is there a way to check how many guest IPs are available in specific
 guest network?
 
 Current cloudstack reporting is not granular enough, i could not find a
 command that says listAvailableIPs or something similar.
 
 Any help is appreciated.
 
 Thanks,
 ilya
 

Re: [DISCUSS] [PROPOSAL] Implementation of DNS Provider for Bind (for 4.5)

[Chiradeep Vittal]

It looks like the DnsProvider calls the REST API of the RoR-based DNSAPI.

+1, but as Rohit said, I’d love to see the design details on the Wiki. This 
will make it easier for folks like Erik to integrate PowerDns.
Does the VR use the Bind server for name resolution? That is, is the Bind 
server the same as the zone DNS? Is this configured on a region level or a zone 
level? Or is it strictly per network offering?

From: Rohit Yadav bhais...@apache.orgmailto:bhais...@apache.org
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Friday, June 13, 2014 at 6:53 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: Re: [DISCUSS] [PROPOSAL] Implementation of DNS Provider for Bind (for 
4.5)

Hi Silvano,

On Fri, Jun 13, 2014 at 10:51 AM, Silvano Nogueira Buback 
silv...@corp.globo.commailto:silv...@corp.globo.com wrote:

Hi there,


I work at Globo.com, a media company in Brazil. Here we use a cloudstack
private network with an advanced zone setup (isolated vlans).

For some couple of reasons, the name of virtual machine needs to be
available not only on virtual router network context, but on our internal
DNS servers.

Our proposal is integrate cloudstack (v 4.5) with DNS server (Bind server)
thru an open source API written by globo.com called DNSAPI. More info at
https://github.com/globocom/Dns-Api.


Thanks for the proposal.

I recommend that you document your design goals in 4.5 or above design docs
wiki:
https://cwiki.apache.org/confluence/display/CLOUDSTACK/4.5+Design+Documents

I saw the video you shared, it looked seamless but I could not figure out
how the ACS plugin interacts with the DNS provider. The API library you
mentioned is written in Ruby, so how does it integrate or work with the dns
plugin in ACS, is it over HTTP or RPC/Thrift?

Regards.



To make this implementation of DNS provider, we based our plugin on
dns-notifier, but we had to add more classes for our implementation.

* DnsAPINetworkDAO to manage the networkDomain for each network.
* DnsAPIVirtualMachineDAO to manage DNS records for vms.
* DnsAPIElement, this class implements the provider itself.
* DnsAPIResource, implements all communications with DNSAPI
(ServerResource).

Besides this classes, another one was necessary to the call to
DnsAPIResource and return the answer, and one API command was created to
configure the provider in Zone.

Above a video that show you how everything was integrated.

https://www.youtube.com/watch?v=fAB53T_NZMI

We really appreciate all your comments about our implementation,

thanks in advance
PS: Sorry about duplicated e-mail in mailing list, but I forget to use
DISCUSS and send using company e-mail)

Re: [DISCUSS] continuous integration for plugins requiring proprietary / hardware infra

[Chiradeep Vittal]

If there is a simulator or emulator that would work too

--
Chiradeep

 On Jun 12, 2014, at 2:20 AM, ilya musayev ilya.mailing.li...@gmail.com 
 wrote:
 
 Chiradeep,
 
 Perhaps we need to provide a little more background information.
 
 We are in process of drafting a request to ASF for lab/dev for Apache 
 CloudStack Continuous Integration environment. In order for us to make sure 
 that future releases of Apache CloudStack wont break functionality with 
 existing plugins vendors like Nuage, Brocade, NetApp, USC, VmWare NVP and 
 other provide. It would be ideal if there is sample hardware - be it 
 physical, virtual or some other method that can confirm existing 
 functionality wont break with new releases.
 
 With that in mind, this is question mostly to developers who are representing 
 vendors, is there anything on your end you can help facilitate? If there are 
 other alternatives you can propose, we would like to hear them.
 
 Thank you for your participation and making Apache CloudStack a better cloud 
 platform,
 
 Regards
 ilya
 
 On 6/10/14, 5:32 PM, Chiradeep Vittal wrote:
 Hi,
 
 Since the Nuage VSP and Brocade plugins have been proposed recently, I was
 wondering what is the mechanism to ensure that CI will test their plugins?
 
 Thanks
 ‹
 Chiradeep
 

Re: [PROPOSAL] Brocade network plugin to orchestrate Brocade VDX Switches for CloudStack 4.5

[Chiradeep Vittal]

Well done.
Question: How do you figure out the VLAN –to- Brocade switch port mapping? By 
using the mac address of the VM nic?
Comment: It looks like you are creating a table (BrocadeNetworkHostMapping). 
Could you add details of this table (schema). Make sure this table appears in 
any upgrade scripts from previous releases.

From: Ritu Sabharwal rsabh...@brocade.commailto:rsabh...@brocade.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Tuesday, June 10, 2014 at 11:51 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Cc: Prakash Kaligotla pkali...@brocade.commailto:pkali...@brocade.com, 
Nagendra Jaladanki njala...@brocade.commailto:njala...@brocade.com
Subject: [PROPOSAL] Brocade network plugin to orchestrate Brocade VDX Switches 
for CloudStack 4.5

Hi CS Developers,

I have added the Design document for the Plugin in the wiki.  Here is the link 
: 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Brocade+Network+Plugin+to+Orchestrate+Brocade+VDX+Switches

Please review it and provide your comments.

Thanks  Regards,
Ritu S.

Re: [CS 4.3 | XenServer] Unable to start Virtual Router (com.cloud.exception.AgentUnavailableException)

[Chiradeep Vittal]

Found this after some googling.
http://comments.gmane.org/gmane.comp.apache.cloudstack.devel/22230

From: Florin Dumitrascu 
florin.dumitra...@intunenetworks.commailto:florin.dumitra...@intunenetworks.com
Reply-To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Date: Tuesday, June 10, 2014 at 6:23 AM
To: dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org 
dev@cloudstack.apache.orgmailto:dev@cloudstack.apache.org
Subject: [CS 4.3 | XenServer] Unable to start Virtual Router 
(com.cloud.exception.AgentUnavailableException)

due to: getDomRVersionCmd failed