Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Rohit Yadav]

Hi René,


Thanks for your comments, this will be a strictly maintenance project at the 
moment I think we should consider expanding its scope to include any new 
feature/integration. The current aim and scope of this work are to migrate the 
base image to Debian9 and ensure no regressions. However, feel free to 
discuss/work on the proposed features/integrations separately.


Regards.



From: Rene Moser <m...@renemoser.net>
Sent: Tuesday, November 21, 2017 8:37:49 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate

Hi Rohit

First, thank you very much for the effort!

On 11/21/2017 01:07 PM, Rohit Yadav wrote:

> Please advise if you can collaborate with me on this, especially around 
> testing. Thanks!

I have some 2 "nice to have"s, AFAICS these haven't been addressed yet:

- SNMP readonly listen on "linklocalip"
- HAProxy stats (or even exporter
https://github.com/prometheus/haproxy_exporter) listen on linklocalip

Regards
René


rohit.ya...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Rene Moser]

Hi Rohit

First, thank you very much for the effort!

On 11/21/2017 01:07 PM, Rohit Yadav wrote:

> Please advise if you can collaborate with me on this, especially around 
> testing. Thanks!

I have some 2 "nice to have"s, AFAICS these haven't been addressed yet:

- SNMP readonly listen on "linklocalip"
- HAProxy stats (or even exporter
https://github.com/prometheus/haproxy_exporter) listen on linklocalip

Regards
René

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Rohit Yadav]

All,


I wanted to revive this thread and bring recent progress to your attention:


- We're now able to build debian9 based systemvmtemplates successfully that 
work okay-ish as CPVM/SSVM host and some initial networking support (dns, dhcp, 
basic zone, advanced zone: firewall, pf, snat etc.) for VirtualRouter

- Several optimization improvements around disk size, services load time, a lot 
of work still to be done

- Several services migrated to use systemd, pending optimizations

- PR: https://github.com/apache/cloudstack/pull/2211


I could finally deploy a test KVM environment (advanced zone) and run some 
smoke tests on it, they are available in the above-mentioned PR. Most failures 
are around networking and services.


Given Debian7 has EOL-ed, and Java7 too, I think it's about time to work on 
this and aim to get this done for ACS 4.11 (end of Q1/2018). Please share your 
feedback and comments.


Using Trillian we can test the "migration to a debian9/java8 based template" PR 
against KVM, XenServer and VMware, however, what other hypervisors should we be 
testing/supporting - HyperV, baremetal, OVM etc? What are the status of these 
hypervisor plugins - baremetal, hyperv, lxc, ovm/ovm3, ucs?


Please advise if you can collaborate with me on this, especially around 
testing. Thanks!


Regards.



From: Rohit Yadav <rohit.ya...@shapeblue.com>
Sent: Monday, July 31, 2017 3:05:30 PM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate

Moving to a packer based build system can still reuse the same scripts and 
recipes to build a systemvmtemplate which is agnostic of who/how you build it. 
With lack of time and resources, moving to debian9 based systemvmtemplate is a 
much needed effort and moving to a new build system can be a next step.

The current build system may not be perfect, may be tricky to setup at first. 
Most recently updated docs are at tools/appliance/README.md, if anyone has 
issues you may ping me and I may be able to help.

Now back to the topic of supporting debian9 as systemvmtemplate base, I was 
able to get something up and running this weekend and get serial console. I 
could also verify some patching done by the systemvm.iso file however I'm 
facing issues with running cloud-early-config, postinit and cloud services in a 
certain order and I need help around the systemd scripts.

Here's the PR:
https://github.com/apache/cloudstack/pull/2211

The PR branch is pushed on ASF remote so any committer can collaborate with me 
by pushing changes/fixes as a separate commit on that branch, feel free to do 
so. Thanks.

- Rohit

From: Paul Angus <paul.an...@shapeblue.com>
Sent: Monday, July 31, 2017 11:11:53 AM
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] Move to Debian9 systemvmtemplate

Depending on the timescales that we looking at, if we can get an agreement to 
use Packer going forward, there is an argument to say that spending time 
getting the Debian 9 template to work on VeeWee and then on Packer is wasted 
effort and that we should just use this as the opportunity to move over to 
Packer/Debian9.

Having spent the weekend fighting with RVM/VeeWee/Ruby.  And finding that we've 
hard linked rvm to Ruby 2.1.1 when it's now on 2.4, Veewee hasn't been updated 
for years  and other mismatches.  I'm very interested to see other options 
explored.

Veewee doesn't do the disk conversions at them moment, so we can still keep 
that a separate process for corner cases that Packer (or something else) can't 
manage..

Kind regards,

Paul Angus

paul.an...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue




rohit.ya...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue




rohit.ya...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
Sent: 28 July 2017 20:30
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate

I think we can move to packer once we can get the Debian9 based 
systemvmtemplate to work.

I think we should focus on doing this first and then focus on migration to a 
new build system as a next step.


I spent some time today and with some help from veewee authors, I could get a 
base template up and running:

https://github.com/apache/cloudstack/pull/2211


The above PR branch is pushed on ASF remote and allows for cross-collaboration 
with all ACS committers. Please collaborate with me on this and feel free to 
push changes on the branch as separate commits and/or make changes to the PR. 
Thanks.


- Rohit


From: Tim Mackey <tmac...@gmail.com>
Sent: Friday, July 28, 2017 3:59:36 AM
To: dev@cloudstac

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Rohit Yadav]

Moving to a packer based build system can still reuse the same scripts and 
recipes to build a systemvmtemplate which is agnostic of who/how you build it. 
With lack of time and resources, moving to debian9 based systemvmtemplate is a 
much needed effort and moving to a new build system can be a next step.

The current build system may not be perfect, may be tricky to setup at first. 
Most recently updated docs are at tools/appliance/README.md, if anyone has 
issues you may ping me and I may be able to help.

Now back to the topic of supporting debian9 as systemvmtemplate base, I was 
able to get something up and running this weekend and get serial console. I 
could also verify some patching done by the systemvm.iso file however I'm 
facing issues with running cloud-early-config, postinit and cloud services in a 
certain order and I need help around the systemd scripts.

Here's the PR:
https://github.com/apache/cloudstack/pull/2211

The PR branch is pushed on ASF remote so any committer can collaborate with me 
by pushing changes/fixes as a separate commit on that branch, feel free to do 
so. Thanks.

- Rohit

From: Paul Angus <paul.an...@shapeblue.com>
Sent: Monday, July 31, 2017 11:11:53 AM
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] Move to Debian9 systemvmtemplate

Depending on the timescales that we looking at, if we can get an agreement to 
use Packer going forward, there is an argument to say that spending time 
getting the Debian 9 template to work on VeeWee and then on Packer is wasted 
effort and that we should just use this as the opportunity to move over to 
Packer/Debian9.

Having spent the weekend fighting with RVM/VeeWee/Ruby.  And finding that we've 
hard linked rvm to Ruby 2.1.1 when it's now on 2.4, Veewee hasn't been updated 
for years  and other mismatches.  I'm very interested to see other options 
explored.

Veewee doesn't do the disk conversions at them moment, so we can still keep 
that a separate process for corner cases that Packer (or something else) can't 
manage..

Kind regards,

Paul Angus

paul.an...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue




rohit.ya...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com]
Sent: 28 July 2017 20:30
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate

I think we can move to packer once we can get the Debian9 based 
systemvmtemplate to work.

I think we should focus on doing this first and then focus on migration to a 
new build system as a next step.


I spent some time today and with some help from veewee authors, I could get a 
base template up and running:

https://github.com/apache/cloudstack/pull/2211


The above PR branch is pushed on ASF remote and allows for cross-collaboration 
with all ACS committers. Please collaborate with me on this and feel free to 
push changes on the branch as separate commits and/or make changes to the PR. 
Thanks.


- Rohit


From: Tim Mackey <tmac...@gmail.com>
Sent: Friday, July 28, 2017 3:59:36 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate

Syed,

I did a bunch of work on XenServer with Packer [1] before leaving Citrix.
My stuff works rather well and was tested with XS 6.2, 6.5 and 7. It shouldn't 
be hard to validate with newest XS and updated Packer - I just lack the infra 
to do the testing.

[1] https://github.com/xenserverarmy/packer

-tim


rohit.ya...@shapeblue.com
www.shapeblue.com<http://www.shapeblue.com>
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue



On Thu, Jul 27, 2017 at 11:19 AM, Syed Ahmed <sah...@cloudops.com> wrote:

> -1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO.
> I've used Packer before and I really like it, the only downside that I
> see is that Packer lacks support for XenServer VHD images. There is
> some work on a XenServer plugin but I haven't tested that. If the
> community decides to use Packer, I can do some initial validation of it on 
> XenServer.
>
> Thanks,
> -Syed
>
> On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander <w...@widodh.nl>
> wrote:
>
> >
> > > Op 24 juli 2017 om 19:07 schreef Rene Moser <m...@renemoser.net>:
> > >
> > >
> > > Hi Rohit
> > >
> > >
> > > On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > > > All,
> > > >
> > > >
> > > > Just want to kick an initial discussion around migration to
> > > > Debian9
> > based systemvmtemplate, and get your feedback on the same.
> > > >
> > > > Here's a work-in-progress PR: https://github.co

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Satki Badal]

The whole ruby platform is running out of steam, high time to move to Packer 
while moving system VMS to Debian9.

- Satki


> On 31-Jul-2017, at 2:41 PM, Paul Angus <paul.an...@shapeblue.com> wrote:
> 
> Depending on the timescales that we looking at, if we can get an agreement to 
> use Packer going forward, there is an argument to say that spending time 
> getting the Debian 9 template to work on VeeWee and then on Packer is wasted 
> effort and that we should just use this as the opportunity to move over to 
> Packer/Debian9.
> 
> Having spent the weekend fighting with RVM/VeeWee/Ruby.  And finding that 
> we've hard linked rvm to Ruby 2.1.1 when it's now on 2.4, Veewee hasn't been 
> updated for years  and other mismatches.  I'm very interested to see other 
> options explored.
> 
> Veewee doesn't do the disk conversions at them moment, so we can still keep 
> that a separate process for corner cases that Packer (or something else) 
> can't manage..
> 
> Kind regards,
> 
> Paul Angus
> 
> paul.an...@shapeblue.com 
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
> 
> 
> 
> 
> -Original Message-
> From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com] 
> Sent: 28 July 2017 20:30
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
> 
> I think we can move to packer once we can get the Debian9 based 
> systemvmtemplate to work.
> 
> I think we should focus on doing this first and then focus on migration to a 
> new build system as a next step.
> 
> 
> I spent some time today and with some help from veewee authors, I could get a 
> base template up and running:
> 
> https://github.com/apache/cloudstack/pull/2211
> 
> 
> The above PR branch is pushed on ASF remote and allows for 
> cross-collaboration with all ACS committers. Please collaborate with me on 
> this and feel free to push changes on the branch as separate commits and/or 
> make changes to the PR. Thanks.
> 
> 
> - Rohit
> 
> ________
> From: Tim Mackey <tmac...@gmail.com>
> Sent: Friday, July 28, 2017 3:59:36 AM
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate
> 
> Syed,
> 
> I did a bunch of work on XenServer with Packer [1] before leaving Citrix.
> My stuff works rather well and was tested with XS 6.2, 6.5 and 7. It 
> shouldn't be hard to validate with newest XS and updated Packer - I just lack 
> the infra to do the testing.
> 
> [1] https://github.com/xenserverarmy/packer
> 
> -tim
> 
> 
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> 
> 
> 
> On Thu, Jul 27, 2017 at 11:19 AM, Syed Ahmed <sah...@cloudops.com> wrote:
> 
>> -1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO. 
>> I've used Packer before and I really like it, the only downside that I 
>> see is that Packer lacks support for XenServer VHD images. There is 
>> some work on a XenServer plugin but I haven't tested that. If the 
>> community decides to use Packer, I can do some initial validation of it on 
>> XenServer.
>> 
>> Thanks,
>> -Syed
>> 
>> On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander <w...@widodh.nl>
>> wrote:
>> 
>>> 
>>>> Op 24 juli 2017 om 19:07 schreef Rene Moser <m...@renemoser.net>:
>>>> 
>>>> 
>>>> Hi Rohit
>>>> 
>>>> 
>>>> On 07/23/2017 06:08 PM, Rohit Yadav wrote:
>>>>> All,
>>>>> 
>>>>> 
>>>>> Just want to kick an initial discussion around migration to 
>>>>> Debian9
>>> based systemvmtemplate, and get your feedback on the same.
>>>>> 
>>>>> Here's a work-in-progress PR: https://github.com/apache/
>>> cloudstack/pull/2198
>>>> 
>>>> Have you considered to replace veewee by packer?
>>>> 
>>> 
>>> Packer is really nice indeed. We use it to build our templates [0] 
>>> which we use on CloudStack.
>>> 
>>> Building the SSVM using Packer should be rather easy I think.
>>> 
>>> [0]: https://github.com/pcextreme/packer-templates
>>> 
>>>> Our friends from schubergphilis have already done some work here 
>>>> https://github.com/MissionCriticalCloud/systemvm-packer.
>>>> 
>>>> However there would be also an official way to convert the 
>>>> definitions https://www.packer.io/guides/veewee-to-packer.html
>>>> 
>>>> Regards René
>>> 
>> 
> 

RE: [DISCUSS] Move to Debian9 systemvmtemplate

[Paul Angus]

Depending on the timescales that we looking at, if we can get an agreement to 
use Packer going forward, there is an argument to say that spending time 
getting the Debian 9 template to work on VeeWee and then on Packer is wasted 
effort and that we should just use this as the opportunity to move over to 
Packer/Debian9.

Having spent the weekend fighting with RVM/VeeWee/Ruby.  And finding that we've 
hard linked rvm to Ruby 2.1.1 when it's now on 2.4, Veewee hasn't been updated 
for years  and other mismatches.  I'm very interested to see other options 
explored.

Veewee doesn't do the disk conversions at them moment, so we can still keep 
that a separate process for corner cases that Packer (or something else) can't 
manage..

Kind regards,

Paul Angus

paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com] 
Sent: 28 July 2017 20:30
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate

I think we can move to packer once we can get the Debian9 based 
systemvmtemplate to work.

I think we should focus on doing this first and then focus on migration to a 
new build system as a next step.


I spent some time today and with some help from veewee authors, I could get a 
base template up and running:

https://github.com/apache/cloudstack/pull/2211


The above PR branch is pushed on ASF remote and allows for cross-collaboration 
with all ACS committers. Please collaborate with me on this and feel free to 
push changes on the branch as separate commits and/or make changes to the PR. 
Thanks.


- Rohit


From: Tim Mackey <tmac...@gmail.com>
Sent: Friday, July 28, 2017 3:59:36 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate

Syed,

I did a bunch of work on XenServer with Packer [1] before leaving Citrix.
My stuff works rather well and was tested with XS 6.2, 6.5 and 7. It shouldn't 
be hard to validate with newest XS and updated Packer - I just lack the infra 
to do the testing.

[1] https://github.com/xenserverarmy/packer

-tim


rohit.ya...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
  
 

On Thu, Jul 27, 2017 at 11:19 AM, Syed Ahmed <sah...@cloudops.com> wrote:

> -1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO. 
> I've used Packer before and I really like it, the only downside that I 
> see is that Packer lacks support for XenServer VHD images. There is 
> some work on a XenServer plugin but I haven't tested that. If the 
> community decides to use Packer, I can do some initial validation of it on 
> XenServer.
>
> Thanks,
> -Syed
>
> On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander <w...@widodh.nl>
> wrote:
>
> >
> > > Op 24 juli 2017 om 19:07 schreef Rene Moser <m...@renemoser.net>:
> > >
> > >
> > > Hi Rohit
> > >
> > >
> > > On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > > > All,
> > > >
> > > >
> > > > Just want to kick an initial discussion around migration to 
> > > > Debian9
> > based systemvmtemplate, and get your feedback on the same.
> > > >
> > > > Here's a work-in-progress PR: https://github.com/apache/
> > cloudstack/pull/2198
> > >
> > > Have you considered to replace veewee by packer?
> > >
> >
> > Packer is really nice indeed. We use it to build our templates [0] 
> > which we use on CloudStack.
> >
> > Building the SSVM using Packer should be rather easy I think.
> >
> > [0]: https://github.com/pcextreme/packer-templates
> >
> > > Our friends from schubergphilis have already done some work here 
> > > https://github.com/MissionCriticalCloud/systemvm-packer.
> > >
> > > However there would be also an official way to convert the 
> > > definitions https://www.packer.io/guides/veewee-to-packer.html
> > >
> > > Regards René
> >
>

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Syed Ahmed]

This is awesome Tim. Any chance we merge it upstream with Packer?



On Thu, Jul 27, 2017 at 9:59 PM, Tim Mackey  wrote:

> Syed,
>
> I did a bunch of work on XenServer with Packer [1] before leaving Citrix.
> My stuff works rather well and was tested with XS 6.2, 6.5 and 7. It
> shouldn't be hard to validate with newest XS and updated Packer - I just
> lack the infra to do the testing.
>
> [1] https://github.com/xenserverarmy/packer
>
> -tim
>
> On Thu, Jul 27, 2017 at 11:19 AM, Syed Ahmed  wrote:
>
> > -1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO. I've
> > used Packer before and I really like it, the only downside that I see is
> > that Packer lacks support for XenServer VHD images. There is some work
> on a
> > XenServer plugin but I haven't tested that. If the community decides to
> use
> > Packer, I can do some initial validation of it on XenServer.
> >
> > Thanks,
> > -Syed
> >
> > On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander 
> > wrote:
> >
> > >
> > > > Op 24 juli 2017 om 19:07 schreef Rene Moser :
> > > >
> > > >
> > > > Hi Rohit
> > > >
> > > >
> > > > On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > > > > All,
> > > > >
> > > > >
> > > > > Just want to kick an initial discussion around migration to Debian9
> > > based systemvmtemplate, and get your feedback on the same.
> > > > >
> > > > > Here's a work-in-progress PR: https://github.com/apache/
> > > cloudstack/pull/2198
> > > >
> > > > Have you considered to replace veewee by packer?
> > > >
> > >
> > > Packer is really nice indeed. We use it to build our templates [0]
> which
> > > we use on CloudStack.
> > >
> > > Building the SSVM using Packer should be rather easy I think.
> > >
> > > [0]: https://github.com/pcextreme/packer-templates
> > >
> > > > Our friends from schubergphilis have already done some work here
> > > > https://github.com/MissionCriticalCloud/systemvm-packer.
> > > >
> > > > However there would be also an official way to convert the
> definitions
> > > > https://www.packer.io/guides/veewee-to-packer.html
> > > >
> > > > Regards René
> > >
> >
>

RE: [DISCUSS] Move to Debian9 systemvmtemplate

[Paul Angus]

Can you update the documentation in confluence
 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/Build+Your+Own+SystemVM+Templates
to reflect what's required now.
Its woefully short on information, and links don't work anymore.




Kind regards,

Paul Angus

paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com] 
Sent: 28 July 2017 20:30
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate

I think we can move to packer once we can get the Debian9 based 
systemvmtemplate to work.

I think we should focus on doing this first and then focus on migration to a 
new build system as a next step.


I spent some time today and with some help from veewee authors, I could get a 
base template up and running:

https://github.com/apache/cloudstack/pull/2211


The above PR branch is pushed on ASF remote and allows for cross-collaboration 
with all ACS committers. Please collaborate with me on this and feel free to 
push changes on the branch as separate commits and/or make changes to the PR. 
Thanks.


- Rohit


From: Tim Mackey <tmac...@gmail.com>
Sent: Friday, July 28, 2017 3:59:36 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate

Syed,

I did a bunch of work on XenServer with Packer [1] before leaving Citrix.
My stuff works rather well and was tested with XS 6.2, 6.5 and 7. It shouldn't 
be hard to validate with newest XS and updated Packer - I just lack the infra 
to do the testing.

[1] https://github.com/xenserverarmy/packer

-tim


rohit.ya...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
  
 

On Thu, Jul 27, 2017 at 11:19 AM, Syed Ahmed <sah...@cloudops.com> wrote:

> -1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO. 
> I've used Packer before and I really like it, the only downside that I 
> see is that Packer lacks support for XenServer VHD images. There is 
> some work on a XenServer plugin but I haven't tested that. If the 
> community decides to use Packer, I can do some initial validation of it on 
> XenServer.
>
> Thanks,
> -Syed
>
> On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander <w...@widodh.nl>
> wrote:
>
> >
> > > Op 24 juli 2017 om 19:07 schreef Rene Moser <m...@renemoser.net>:
> > >
> > >
> > > Hi Rohit
> > >
> > >
> > > On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > > > All,
> > > >
> > > >
> > > > Just want to kick an initial discussion around migration to 
> > > > Debian9
> > based systemvmtemplate, and get your feedback on the same.
> > > >
> > > > Here's a work-in-progress PR: https://github.com/apache/
> > cloudstack/pull/2198
> > >
> > > Have you considered to replace veewee by packer?
> > >
> >
> > Packer is really nice indeed. We use it to build our templates [0] 
> > which we use on CloudStack.
> >
> > Building the SSVM using Packer should be rather easy I think.
> >
> > [0]: https://github.com/pcextreme/packer-templates
> >
> > > Our friends from schubergphilis have already done some work here 
> > > https://github.com/MissionCriticalCloud/systemvm-packer.
> > >
> > > However there would be also an official way to convert the 
> > > definitions https://www.packer.io/guides/veewee-to-packer.html
> > >
> > > Regards René
> >
>

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Rohit Yadav]

I think we can move to packer once we can get the Debian9 based 
systemvmtemplate to work.

I think we should focus on doing this first and then focus on migration to a 
new build system as a next step.


I spent some time today and with some help from veewee authors, I could get a 
base template up and running:

https://github.com/apache/cloudstack/pull/2211


The above PR branch is pushed on ASF remote and allows for cross-collaboration 
with all ACS committers. Please collaborate with me on this and feel free to 
push changes on the branch as separate commits and/or make changes to the PR. 
Thanks.


- Rohit


From: Tim Mackey <tmac...@gmail.com>
Sent: Friday, July 28, 2017 3:59:36 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate

Syed,

I did a bunch of work on XenServer with Packer [1] before leaving Citrix.
My stuff works rather well and was tested with XS 6.2, 6.5 and 7. It
shouldn't be hard to validate with newest XS and updated Packer - I just
lack the infra to do the testing.

[1] https://github.com/xenserverarmy/packer

-tim


rohit.ya...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 

On Thu, Jul 27, 2017 at 11:19 AM, Syed Ahmed <sah...@cloudops.com> wrote:

> -1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO. I've
> used Packer before and I really like it, the only downside that I see is
> that Packer lacks support for XenServer VHD images. There is some work on a
> XenServer plugin but I haven't tested that. If the community decides to use
> Packer, I can do some initial validation of it on XenServer.
>
> Thanks,
> -Syed
>
> On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander <w...@widodh.nl>
> wrote:
>
> >
> > > Op 24 juli 2017 om 19:07 schreef Rene Moser <m...@renemoser.net>:
> > >
> > >
> > > Hi Rohit
> > >
> > >
> > > On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > > > All,
> > > >
> > > >
> > > > Just want to kick an initial discussion around migration to Debian9
> > based systemvmtemplate, and get your feedback on the same.
> > > >
> > > > Here's a work-in-progress PR: https://github.com/apache/
> > cloudstack/pull/2198
> > >
> > > Have you considered to replace veewee by packer?
> > >
> >
> > Packer is really nice indeed. We use it to build our templates [0] which
> > we use on CloudStack.
> >
> > Building the SSVM using Packer should be rather easy I think.
> >
> > [0]: https://github.com/pcextreme/packer-templates
> >
> > > Our friends from schubergphilis have already done some work here
> > > https://github.com/MissionCriticalCloud/systemvm-packer.
> > >
> > > However there would be also an official way to convert the definitions
> > > https://www.packer.io/guides/veewee-to-packer.html
> > >
> > > Regards René
> >
>

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Tim Mackey]

Syed,

I did a bunch of work on XenServer with Packer [1] before leaving Citrix.
My stuff works rather well and was tested with XS 6.2, 6.5 and 7. It
shouldn't be hard to validate with newest XS and updated Packer - I just
lack the infra to do the testing.

[1] https://github.com/xenserverarmy/packer

-tim

On Thu, Jul 27, 2017 at 11:19 AM, Syed Ahmed  wrote:

> -1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO. I've
> used Packer before and I really like it, the only downside that I see is
> that Packer lacks support for XenServer VHD images. There is some work on a
> XenServer plugin but I haven't tested that. If the community decides to use
> Packer, I can do some initial validation of it on XenServer.
>
> Thanks,
> -Syed
>
> On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander 
> wrote:
>
> >
> > > Op 24 juli 2017 om 19:07 schreef Rene Moser :
> > >
> > >
> > > Hi Rohit
> > >
> > >
> > > On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > > > All,
> > > >
> > > >
> > > > Just want to kick an initial discussion around migration to Debian9
> > based systemvmtemplate, and get your feedback on the same.
> > > >
> > > > Here's a work-in-progress PR: https://github.com/apache/
> > cloudstack/pull/2198
> > >
> > > Have you considered to replace veewee by packer?
> > >
> >
> > Packer is really nice indeed. We use it to build our templates [0] which
> > we use on CloudStack.
> >
> > Building the SSVM using Packer should be rather easy I think.
> >
> > [0]: https://github.com/pcextreme/packer-templates
> >
> > > Our friends from schubergphilis have already done some work here
> > > https://github.com/MissionCriticalCloud/systemvm-packer.
> > >
> > > However there would be also an official way to convert the definitions
> > > https://www.packer.io/guides/veewee-to-packer.html
> > >
> > > Regards René
> >
>

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Syed Ahmed]

-1 on Arch as well. Moving to Debian 9 seems the wiser choice IMO. I've
used Packer before and I really like it, the only downside that I see is
that Packer lacks support for XenServer VHD images. There is some work on a
XenServer plugin but I haven't tested that. If the community decides to use
Packer, I can do some initial validation of it on XenServer.

Thanks,
-Syed

On Tue, Jul 25, 2017 at 3:19 AM, Wido den Hollander  wrote:

>
> > Op 24 juli 2017 om 19:07 schreef Rene Moser :
> >
> >
> > Hi Rohit
> >
> >
> > On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > > All,
> > >
> > >
> > > Just want to kick an initial discussion around migration to Debian9
> based systemvmtemplate, and get your feedback on the same.
> > >
> > > Here's a work-in-progress PR: https://github.com/apache/
> cloudstack/pull/2198
> >
> > Have you considered to replace veewee by packer?
> >
>
> Packer is really nice indeed. We use it to build our templates [0] which
> we use on CloudStack.
>
> Building the SSVM using Packer should be rather easy I think.
>
> [0]: https://github.com/pcextreme/packer-templates
>
> > Our friends from schubergphilis have already done some work here
> > https://github.com/MissionCriticalCloud/systemvm-packer.
> >
> > However there would be also an official way to convert the definitions
> > https://www.packer.io/guides/veewee-to-packer.html
> >
> > Regards René
>

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Wido den Hollander]

> Op 24 juli 2017 om 19:07 schreef Rene Moser :
> 
> 
> Hi Rohit
> 
> 
> On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> > All,
> > 
> > 
> > Just want to kick an initial discussion around migration to Debian9 based 
> > systemvmtemplate, and get your feedback on the same.
> > 
> > Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198
> 
> Have you considered to replace veewee by packer?
> 

Packer is really nice indeed. We use it to build our templates [0] which we use 
on CloudStack.

Building the SSVM using Packer should be rather easy I think.

[0]: https://github.com/pcextreme/packer-templates

> Our friends from schubergphilis have already done some work here
> https://github.com/MissionCriticalCloud/systemvm-packer.
> 
> However there would be also an official way to convert the definitions
> https://www.packer.io/guides/veewee-to-packer.html
> 
> Regards René

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Rene Moser]

Hi Rohit


On 07/23/2017 06:08 PM, Rohit Yadav wrote:
> All,
> 
> 
> Just want to kick an initial discussion around migration to Debian9 based 
> systemvmtemplate, and get your feedback on the same.
> 
> Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198

Have you considered to replace veewee by packer?

Our friends from schubergphilis have already done some work here
https://github.com/MissionCriticalCloud/systemvm-packer.

However there would be also an official way to convert the definitions
https://www.packer.io/guides/veewee-to-packer.html

Regards René

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Rene Moser]

Hi

On 07/23/2017 06:21 PM, Paul Angus wrote:
> 
> I think that we should look at lighter Distros like Arch Linux in order to 
> get the boot times of the system VMs down.
> That in conjunction with improving the configuration will gives as a much 
> leaner, meaner System VM.

I tend to -1 for Arch.

Debian is a proven mature Distro. It is used as base for many critical
systems e.g. VyOS or also cumulus OS, not to mention Ubuntu.

Arch linux was not designed as a server OS. It has a rolling release
policy and it is great if you want to use the latest releases, you have
to update often.

Because we usually do not update VR OS that often, Arch is IMHO not a
good candidate. On the other hand Debian is a good one.

I would also doubt that Arch "boots" faster than Debian. IMHO the ways
_we_ configure the VR is still the performance killer.

Let's stick with Debian.

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Rohit Yadav]

Only though testing we can know if there are any regressions, with Debian7 to 
be unsupported somewhere in 2018 we need to start some work around this.

Distros such as Arch Linux are not solid enough or work out of the box and have 
bleeding edge unstable software. Debian 9 "stretch" finally have 
openjdk-8-jre-headless (which surprising was not in debian8, though was in 
debian8's backports) so we can move away from using Azul's distro as well, 
along with having updated stable dependencies/versions.


Debian distributions are usually gold standard for security with other distros 
such as Tails and Kali Linux build on top of it, maybe a reason why systemvm 
templates were originally build using Debian as base. Moving to a new 
distribution will be a lot more work given highly assumed and dependent code 
(bash/python scripts) that targets Debian based systemvmtemplate. Perhaps a 
more modular and modern approach would be to split systemvm services and run 
them in containers, using a light base such as Alpine Linux, however that's a 
different discussion and a much bigger project to undertake.


- Rohit


From: Jayapal Uradi <jayapal.ur...@accelerite.com>
Sent: Monday, July 24, 2017 6:34:08 AM
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] Move to Debian9 systemvmtemplate


First one come to my mind is VPN feature.
But other general issues will depend on what are changed in debain9.

Thanks,
Jayapal




rohit.ya...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 

> On Jul 24, 2017, at 9:54 AM, Satki Badal <satki.ba...@gmail.com> wrote:
>
> Hi Jayapal,
>
> Do we know what all features re covered by Marvin and what all will require 
> manual testing ?
> I guess without this list we can never be sure is all the features got tested 
> or not.
>
> Thanks,
> Satki
>
>> On 24-Jul-2017, at 9:48 AM, Jayapal Uradi <jayapal.ur...@accelerite.com> 
>> wrote:
>>
>> Hi,
>>
>> We should throughly check all the VR features to see feature breakage 
>> because of  Debian9.
>>
>> Thanks,
>> Jayapal
>>
>>
>>> On Jul 23, 2017, at 9:38 PM, Rohit Yadav <rohit.ya...@shapeblue.com> wrote:
>>>
>>> All,
>>>
>>>
>>> Just want to kick an initial discussion around migration to Debian9 based 
>>> systemvmtemplate, and get your feedback on the same.
>>>
>>> Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198
>>>
>>>
>>> Regards.
>>>
>>> rohit.ya...@shapeblue.com
>>> www.shapeblue.com<http://www.shapeblue.com>
>>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>>> @shapeblue
>>>
>>>
>>>
>>
>> DISCLAIMER
>> ==
>> This e-mail may contain privileged and confidential information which is the 
>> property of Accelerite, a Persistent Systems business. It is intended only 
>> for the use of the individual or entity to which it is addressed. If you are 
>> not the intended recipient, you are not authorized to read, retain, copy, 
>> print, distribute or use this message. If you have received this 
>> communication in error, please notify the sender and delete all copies of 
>> this message. Accelerite, a Persistent Systems business does not accept any 
>> liability for virus infected mails.
>>
>

RE: [DISCUSS] Move to Debian9 systemvmtemplate

[Wido den Hollander]

> Op 23 juli 2017 om 18:21 schreef Paul Angus <paul.an...@shapeblue.com>:
> 
> 
> 
> I think that we should look at lighter Distros like Arch Linux in order to 
> get the boot times of the system VMs down.

Debian 9 uses systemd and that makes it boot a lot faster.

> That in conjunction with improving the configuration will gives as a much 
> leaner, meaner System VM.

A lot of things can be done, but going to Debian 9 isn't that difficult.

Yes, we will need to fix a few things around systemd and such, but Debian is a 
proven and stable distribution.

Let's not try to do two things at the same time.

Our current Debian 7 is dangerously old and needs to be replaced. Debian 9 
seems like a very good candidate to me.

Wido

> 
> Kind regards,
> 
> Paul Angus
> 
> paul.an...@shapeblue.com 
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>   
>  
> 
> 
> -Original Message-
> From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com] 
> Sent: 23 July 2017 17:08
> To: dev@cloudstack.apache.org
> Subject: [DISCUSS] Move to Debian9 systemvmtemplate
> 
> All,
> 
> 
> Just want to kick an initial discussion around migration to Debian9 based 
> systemvmtemplate, and get your feedback on the same.
> 
> Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198
> 
> 
> Regards.
> 
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>   
>  
>

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Jayapal Uradi]

First one come to my mind is VPN feature. 
But other general issues will depend on what are changed in debain9.

Thanks,
Jayapal



> On Jul 24, 2017, at 9:54 AM, Satki Badal  wrote:
> 
> Hi Jayapal,
> 
> Do we know what all features re covered by Marvin and what all will require 
> manual testing ?
> I guess without this list we can never be sure is all the features got tested 
> or not.
> 
> Thanks,
> Satki
> 
>> On 24-Jul-2017, at 9:48 AM, Jayapal Uradi  
>> wrote:
>> 
>> Hi,
>> 
>> We should throughly check all the VR features to see feature breakage 
>> because of  Debian9.
>> 
>> Thanks,
>> Jayapal
>> 
>> 
>>> On Jul 23, 2017, at 9:38 PM, Rohit Yadav  wrote:
>>> 
>>> All,
>>> 
>>> 
>>> Just want to kick an initial discussion around migration to Debian9 based 
>>> systemvmtemplate, and get your feedback on the same.
>>> 
>>> Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198
>>> 
>>> 
>>> Regards.
>>> 
>>> rohit.ya...@shapeblue.com 
>>> www.shapeblue.com
>>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>>> @shapeblue
>>> 
>>> 
>>> 
>> 
>> DISCLAIMER
>> ==
>> This e-mail may contain privileged and confidential information which is the 
>> property of Accelerite, a Persistent Systems business. It is intended only 
>> for the use of the individual or entity to which it is addressed. If you are 
>> not the intended recipient, you are not authorized to read, retain, copy, 
>> print, distribute or use this message. If you have received this 
>> communication in error, please notify the sender and delete all copies of 
>> this message. Accelerite, a Persistent Systems business does not accept any 
>> liability for virus infected mails.
>> 
> 

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Satki Badal]

Hi Jayapal,

Do we know what all features re covered by Marvin and what all will require 
manual testing ?
I guess without this list we can never be sure is all the features got tested 
or not.

Thanks,
Satki

> On 24-Jul-2017, at 9:48 AM, Jayapal Uradi  
> wrote:
> 
> Hi,
> 
> We should throughly check all the VR features to see feature breakage because 
> of  Debian9.
> 
> Thanks,
> Jayapal
> 
> 
>> On Jul 23, 2017, at 9:38 PM, Rohit Yadav  wrote:
>> 
>> All,
>> 
>> 
>> Just want to kick an initial discussion around migration to Debian9 based 
>> systemvmtemplate, and get your feedback on the same.
>> 
>> Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198
>> 
>> 
>> Regards.
>> 
>> rohit.ya...@shapeblue.com 
>> www.shapeblue.com
>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> @shapeblue
>> 
>> 
>> 
> 
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which is the 
> property of Accelerite, a Persistent Systems business. It is intended only 
> for the use of the individual or entity to which it is addressed. If you are 
> not the intended recipient, you are not authorized to read, retain, copy, 
> print, distribute or use this message. If you have received this 
> communication in error, please notify the sender and delete all copies of 
> this message. Accelerite, a Persistent Systems business does not accept any 
> liability for virus infected mails.
> 

Re: [DISCUSS] Move to Debian9 systemvmtemplate

[Jayapal Uradi]

Hi,

We should throughly check all the VR features to see feature breakage because 
of  Debian9.

Thanks,
Jayapal


> On Jul 23, 2017, at 9:38 PM, Rohit Yadav  wrote:
> 
> All,
> 
> 
> Just want to kick an initial discussion around migration to Debian9 based 
> systemvmtemplate, and get your feedback on the same.
> 
> Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198
> 
> 
> Regards.
> 
> rohit.ya...@shapeblue.com 
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
> 
> 
> 

DISCLAIMER
==
This e-mail may contain privileged and confidential information which is the 
property of Accelerite, a Persistent Systems business. It is intended only for 
the use of the individual or entity to which it is addressed. If you are not 
the intended recipient, you are not authorized to read, retain, copy, print, 
distribute or use this message. If you have received this communication in 
error, please notify the sender and delete all copies of this message. 
Accelerite, a Persistent Systems business does not accept any liability for 
virus infected mails.

RE: [DISCUSS] Move to Debian9 systemvmtemplate

[Paul Angus]

I think that we should look at lighter Distros like Arch Linux in order to get 
the boot times of the system VMs down.
That in conjunction with improving the configuration will gives as a much 
leaner, meaner System VM.

Kind regards,

Paul Angus

paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com] 
Sent: 23 July 2017 17:08
To: dev@cloudstack.apache.org
Subject: [DISCUSS] Move to Debian9 systemvmtemplate

All,


Just want to kick an initial discussion around migration to Debian9 based 
systemvmtemplate, and get your feedback on the same.

Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198


Regards.

rohit.ya...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
  
 

[DISCUSS] Move to Debian9 systemvmtemplate

[Rohit Yadav]

All,


Just want to kick an initial discussion around migration to Debian9 based 
systemvmtemplate, and get your feedback on the same.

Here's a work-in-progress PR: https://github.com/apache/cloudstack/pull/2198


Regards.

rohit.ya...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue