Re: [cloudstack-go] sdk releasing

[Pierre-Luc Dion]

Hi Rohit,

I've tried the approach when I did the release of cloudstack-go SDK
v4.15.1. I had to remove it and create v2.10.0; The reason is captured here
[1]
Basically, because we call our go module cloudstack/v2 if we want to have
release v4.x we would need to change our module to cloudstack/v4.

I don't know how big of a deal this is to move from /v2 to /v4, on the
other hand, why not get rid of the version in the module definition instead
?

[1] https://github.com/apache/cloudstack-go/pull/7#issuecomment-918225329
[2] https://go.dev/blog/publishing-go-modules

Regards,

On Tue, Oct 19, 2021 at 3:57 AM Rohit Yadav 
wrote:

> Hi All, cc PL
>
> I thought to check with the community for any objections, following PL's
> approach for ACS 4.15.2 we've created:
> https://github.com/apache/cloudstack-go/releases/tag/v2.11.0
>
> Explicit testing/voting may not be necessary as it's largely autogenerated
> against API of an ACS release and further it doesn't serve purpose in
> itself but currently used by the CloudStack k8s provider and terrraform
> provider which will be tested/voted against (indirectly testing/voting
> these projects will also validate/test the go-sdk).
>
> Unless there are any objections, do we agree to tag the cloudstack-go repo
> with tags against ACS releases using the approach PL started?
>
> Regards.
>
>
>
>
>
> --
> *From:* Rohit Yadav 
> *Sent:* Wednesday, September 8, 2021 13:49
> *To:* Pierre-Luc Dion ; dev <
> dev@cloudstack.apache.org>
> *Subject:* Re: [cloudstack-go] sdk releasing
>
> +1
>
> That makes sense. In the go-sdk we've a generator that consumes the
> listApis output of a ACS release and generates the library -
> https://github.com/apache/cloudstack-go/tree/main/generate
>
> I suppose for every ACS release, we can update go-sdk with
> release-specific API list, test it, release and tag it. Even automate this?
>
> I would say - no need to vote it unless the SDK is manually changed. Since
> it is used with the k8s provider or the terraform provider so tags on
> go-sdk may go in-line with tags/release of these users.
>
> Regards.
>
> 
> From: Pierre-Luc Dion 
> Sent: Friday, August 27, 2021 17:57
> To: Rohit Yadav ; dev <
> dev@cloudstack.apache.org>
> Subject: [cloudstack-go] sdk releasing
>
> I've messing around with cloudstack-go
>
> Did a fix that rohit merged yesturday for hostsservices, but this fix will
> only work for acs4.15, I'd like to fix it for previous acs version too, but
> look like the version of the SDK depend on acs version;
>
> Example; for the hostservices, the host attribute managementserverid is a
> UUID in 4.15, but an integer in an older version of xenserver. This breaks
> the structs,or map, we must have some other similar issue.
>
> so I'd like to help on creating a tag or version of the SDK so they would
> match acs version target,
> ie: SDK version = v4.15-0; where the last digit would define the sdk
> version or increase with fixes.
> Current versioning in use =
> https://github.com/apache/cloudstack-go/releases
>
> The issue I'm expecting to face is if we create a release , let's say
> v4.15-0 from main branch today, if we want to create v4.14.0, it will not
> be possible from the main branch because we need to revert the last commit
> but also fix hostservices.
>
> Here are a bunch of questions I have:
> 1. Should we create a branch for older ACS versions  and keep main for
> latest fixes and future releases ?
> 2. Do we need to vote for such changes?
> 3. Does such releases could impact other Go projects that use this one,
> such as terraform and kubernetes drivers?
> 4. Should we provide similar versioning on our kubernetes driver?
>
>
>
>
>

Re: [VOTE] Release Apache CloudStack CloudMonkey 6.2.0

[Pierre-Luc Dion]

epos/dist/dev/cloudstack/cloudmonkey-6.2.0/
>
> PGP release keys (signed using 986611B4A5B7090D0145B230E7DB9FC18F16C6AE)
> https://dist.apache.org/repos/dist/release/cloudstack/KEYS
>
> The vote will be open until October 1st, 2021.
>
> For sanity in tallying the vote, can PMC members please be sure to
> indicate "(binding)" with their vote?
> [ ] +1 approve
> [ ] +0 no opinion
> [ ] -1 disapprove (and the reason why)
>
> Regards,
> Pearl Dsilva
>
>
>
>

-- 

*Pierre-Luc Dion*Lead Cloud Architect | Architecte infonuagique principal
t 888.796.8364 ext. 1403

c 514.880.7765

[image: LOGO-CloudDotCa-RGB (1) (1).png]

Re: [DISCUSS] Export Virtual Router Statistics to Users

[Pierre-Luc Dion]

We've been looking at something similar on our side where we would have
installed telegraf on the VR template and telegraf would have been sending
data to a port forwarding on the hypervisor host.
I don't know how  we could have the VR exposting Prometheus data securely;
outside of the customer network without going through the internet
otherwise.


On Tue, Aug 31, 2021 at 9:57 AM Sina Kashipazha
 wrote:

> Hey there,
>
> We want to improve virtual router statistic visibility. One option that
> pops into my mind is to export some statistics in router VM in Prometheus
> format and let our customers do what they like with those data.
> It enabled our customers to find packet loss, memory usage, etc. Based on
> these data, they can create their alert and dashboard.
>
> If we implement this feature, is this going to merge into upstream?
>
> Another option is to have these data visible in cloudstack dashboard in
> some graph directly. Please let me know if you have a better solution to
> address this issue?
>
>
> Kind regards,
> Sina
>

Re: [Discussion] Release Cycle

[Pierre-Luc Dion]

I like the notion of LTS every 2 years and having  1 or 2 regular releases
per year, like the Ubuntu model.

Typically, upgrading our cloud from a major release is a big level of
effort, especially around QA to make sure the upgrade does not affect
customers.
So, having to jump between LTS every year or so could be challenging.

I'd be curious to know about other users' upgrade cycles.


On Wed, Sep 1, 2021 at 1:11 AM David Jumani 
wrote:

> Hi Gabriel,
>
>
>   1.  I'm a +1 on having regular releases between LTS ones and the
> reasoning behind it. While stability is great, it will also be nice to have
> a "pilot" as you mentioned which the community can test and issues are
> resolved in the following LTS, rather than waiting for 2 - 3 releases to
> get something allegedly stable in, which could still have issues reported
> by users.
>   2.  I'm for having 1 Regular release (Mar-Apr) followed by an LTS one
> (~6 months down the line) each year. Wrt the LTS releases, they should be
> supported for 1.5 to 2 years (at least 6 months after the following LTS is
> out). If that might be too much then a single alternating release annually
> around the same dates
>   3.  No strong opinion on this but it does seem like a good idea, but not
> sure how religiously people will update the new page with every feature
> they intend to add and follow up on it
>
> 
> From: Gabriel Bräscher 
> Sent: Tuesday, August 31, 2021 11:14 PM
> To: dev 
> Subject: [Discussion] Release Cycle
>
> Hello,
>
> I would like to open a discussion regarding the project release cycle. More
> specifically on the following topics:
>
> 1. LTS and Regular releases
>
> 2. Releases period
>
> 3. Enhance roadmap and Release cycle for users
>
>  1 LTS and Regular releases
>
> It has been a while since the last regular release. Nowadays there are only
> LTS releases; maybe we should get back to having regular versions in
> between LTS.
>
> With a “Regular” release users would be able to trade stability for new
> features. Additionally, developers and users would have a “pilot” of the
> next LTS which could anticipate issues and result in a stable long-term
> release.
>
> Please, let me know what you think of this. Should we get back to releasing
> Regular releases in between LTS releases?
>
> For reference, here follow the past releases:
>
> +-+-+--+-+
> | Release | Type| Release date | EOL |
> +-+-+--+-+
> | 4.15| LTS | 19 Jan 2021  | 1 July 2022 |
> +-+-+--+-+
> | 4.14| LTS | 26 May 2020  | 1 Jan 2022  |
> +-+-+--+-+
> | 4.13| LTS | 24 Sep. 2019 | 1 May 2021  |
> +-+-+--+-+
> | 4.12| Regular | 4 April 2019 | N/A |
> +-+-+--+-+
> | 4.11| LTS | 12 Feb. 2018 | 1 July 2019 |
> +-+-+--+-+
> | 4.10| Regular | 6 July 2017  | N/A |
> +-+-+--+-+
> | 4.9 | LTS | 25 July 2016 | 1 July 2018 |
> +-+-+--+-+
>
>  2 Releases period
>
>
> We had in the past a new LTS per year. Then, we got into two new LTS per
> year. This led from having 2 LTS maintained at the same time to 3.
> With that said, I think this is neither documented nor has it been
> discussed in the ML.
>
> We have the LTS minimum and maximum support dates well defined, but so far
> there is no definition/guidelines towards the release period.
> I would like to open this discussion so we can update the CloudStack wiki
> page [https://cwiki.apache.org/confluence/display/CLOUDSTACK/LTS] and have
> a clear definition of when the community should expect each release.
>
>  3 Enhance roadmap and Release cycle for users
>
> This topic is an extension of Topic 2. Once we have “Topic 2” well defined
> we will be able to present a draft of future releases.
>
> The main idea of this email is to look for project stability and
> predictability with a release cycle/roadmap similar to what is done by
> Ubuntu [https://ubuntu.com/about/release-cycle].
> We would then be able to give users and developers a roadmap to look after.
> I would also suggest such a release cycle to be presented on the website,
> in addition to the “cwiki” page [
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/LTS].
>
> Please let me know what you think.
>
> Best regards,
> Gabriel.
>
>
>
>

Re: [DISCUSS] hackathon at ccc

[Pierre-Luc Dion]

I'm aware of the major upgrade you did to get marvin moved to python3.
On our side we need to ramp up on how to use Marvin so we can probably help
a bit on fixing issues.

On Fri, Aug 27, 2021 at 10:57 AM Daan Hoogland 
wrote:

> :D that is merged PL, please have at 5082 where I try to fix some failing
> component tests. test frameworks and such is a good subject though!
>
> On Fri, Aug 27, 2021 at 4:09 PM Pierre-Luc Dion 
> wrote:
>
> > +1 : marving ports to python3
> >
> >
> > On Tue, Aug 24, 2021 at 8:37 AM Gabriel Bräscher 
> > wrote:
> >
> > > +1 Sounds good to me.
> > >
> > > I would propose someone organizing the hackathon by raising the topics
> > and
> > > adding at a Kanban during the Hackathon, then we can split into teams
> if
> > > there are too many tasks/attendees.
> > > The Kanban would be similar to what was done in the ApacheCon Montréal
> > 2018
> > > Hackathon [1,2].
> > > I would suggest making it at the CloudStack Github projects, similar to
> > > this example: [3].
> > >
> > > I know that not everyone will be able to attend, but that happens in
> the
> > > "In loco"/"not-online" events as well :-(
> > > With an online Hackathon, we can share a video on the YT channel or do
> > > something as Rohit mentioned, telecast on twitch or something.
> > >
> > > Regards,
> > > Gabriel.
> > >
> > > [1]
> > >
> > >
> >
> https://trello.com/invite/b/ztgaumtx/fd51a88ce97b2de02201ed52e99384b9/cloudstack-hackathon-18
> > > [2]
> > >
> > >
> >
> https://lists.apache.org/thread.html/7b11d18f97f4e4f1c69aa3861670f77dd21c9f2d55f23bbe79d39ca6%40%3Cdev.cloudstack.apache.org%3E
> > > [3] https://github.com/GabrielBrascher/cloudstack/projects/1
> > >
> > > Em ter., 24 de ago. de 2021 às 08:53, Sina Kashipazha
> > >  escreveu:
> > >
> > > > Nothing particular in my mind right now. But I thought it would be
> nice
> > > to
> > > > extend one of my recent works. We have had an issue recently that
> needs
> > > > network monitoring. I created a small python app using Jupyterlab and
> > > Scapy
> > > > to monitor network packets. I thought it would be nice to extend it
> to
> > > > cover more use cases and add it to the virtual router.
> > > >
> > > > I have no time limit.
> > > >
> > > >
> > > > ‐‐‐ Original Message ‐‐‐
> > > >
> > > > On Tuesday, August 24th, 2021 at 11:36, Daan Hoogland <
> > > > daan.hoogl...@gmail.com> wrote:
> > > >
> > > > > thanks Sina,
> > > > >
> > > >
> > > > > Any subjects you would like to work on, and what length of time
> would
> > > you
> > > > >
> > > >
> > > > > like to participate?
> > > > >
> > > >
> > > > > On Tue, Aug 24, 2021 at 11:22 AM Sina Kashipazha
> > > > >
> > > >
> > > > > s.kashipa...@protonmail.com.invalid wrote:
> > > > >
> > > >
> > > > > > +1
> > > > > >
> > > >
> > > > > > I love to participate :)
> > > > > >
> > > >
> > > > > > ‐‐‐ Original Message ‐‐‐
> > > > > >
> > > >
> > > > > > On Tuesday, August 24th, 2021 at 10:30, Daan Hoogland <
> > > > > >
> > > >
> > > > > > daan.hoogl...@gmail.com> wrote:
> > > > > >
> > > >
> > > > > > > Devs,
> > > > > >
> > > >
> > > > > > > At our next ccc [1] we will be convening virtually. Is there
> > > > interest in
> > > > > >
> > > >
> > > > > > > doing a hackathon virtually as well? My current subject of
> > eternal
> > > > > > >
> > > >
> > > > > > > torture
> > > > > >
> > > >
> > > > > > > is test frameworks (getting half way through porting tests to
> > > python3
> > > > > > >
> > > >
> > > > > > > now)
> > > > > >
> > > >
> > > > > > > but I can imagine way more interesting subjects to hack on (ui,
> > the
> > > > new
> > > > > > >
> > > >
> > > > > > > SDN
> > > > > >
> > > >
> > > > > > > tungsten, any logging facility improvements, reimplementing
> > > > cloudstack in
> > > > > >
> > > >
> > > > > > > rust) We do not have to be very formal on subjects to start
> with
> > > but
> > > > we
> > > > > > >
> > > >
> > > > > > > do
> > > > > >
> > > >
> > > > > > > need to have enough people saying jeey and a time frame. Will
> we
> > > take
> > > > > > >
> > > >
> > > > > > > half
> > > > > >
> > > >
> > > > > > > a day, two days or something in between. If there is enough
> > > > enthusiasm we
> > > > > >
> > > >
> > > > > > > can do preemptive work on the wiki somewhere below [2]
> > > > > >
> > > >
> > > > > > > [1] http://cloudstackcollab.org/
> > > > > >
> > > >
> > > > > > > [2]
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/Events
> > > > > >
> > > >
> > > > > > > regards
> > > > > >
> > > >
> > > > > > > Daan
> > > > >
> > > >
> > > > > --
> > > > >
> > > >
> > > > > Daan
> > >
> >
>
>
> --
> Daan
>

Re: [DISCUSS] hackathon at ccc

[Pierre-Luc Dion]

+1 : marving ports to python3


On Tue, Aug 24, 2021 at 8:37 AM Gabriel Bräscher 
wrote:

> +1 Sounds good to me.
>
> I would propose someone organizing the hackathon by raising the topics and
> adding at a Kanban during the Hackathon, then we can split into teams if
> there are too many tasks/attendees.
> The Kanban would be similar to what was done in the ApacheCon Montréal 2018
> Hackathon [1,2].
> I would suggest making it at the CloudStack Github projects, similar to
> this example: [3].
>
> I know that not everyone will be able to attend, but that happens in the
> "In loco"/"not-online" events as well :-(
> With an online Hackathon, we can share a video on the YT channel or do
> something as Rohit mentioned, telecast on twitch or something.
>
> Regards,
> Gabriel.
>
> [1]
>
> https://trello.com/invite/b/ztgaumtx/fd51a88ce97b2de02201ed52e99384b9/cloudstack-hackathon-18
> [2]
>
> https://lists.apache.org/thread.html/7b11d18f97f4e4f1c69aa3861670f77dd21c9f2d55f23bbe79d39ca6%40%3Cdev.cloudstack.apache.org%3E
> [3] https://github.com/GabrielBrascher/cloudstack/projects/1
>
> Em ter., 24 de ago. de 2021 às 08:53, Sina Kashipazha
>  escreveu:
>
> > Nothing particular in my mind right now. But I thought it would be nice
> to
> > extend one of my recent works. We have had an issue recently that needs
> > network monitoring. I created a small python app using Jupyterlab and
> Scapy
> > to monitor network packets. I thought it would be nice to extend it to
> > cover more use cases and add it to the virtual router.
> >
> > I have no time limit.
> >
> >
> > ‐‐‐ Original Message ‐‐‐
> >
> > On Tuesday, August 24th, 2021 at 11:36, Daan Hoogland <
> > daan.hoogl...@gmail.com> wrote:
> >
> > > thanks Sina,
> > >
> >
> > > Any subjects you would like to work on, and what length of time would
> you
> > >
> >
> > > like to participate?
> > >
> >
> > > On Tue, Aug 24, 2021 at 11:22 AM Sina Kashipazha
> > >
> >
> > > s.kashipa...@protonmail.com.invalid wrote:
> > >
> >
> > > > +1
> > > >
> >
> > > > I love to participate :)
> > > >
> >
> > > > ‐‐‐ Original Message ‐‐‐
> > > >
> >
> > > > On Tuesday, August 24th, 2021 at 10:30, Daan Hoogland <
> > > >
> >
> > > > daan.hoogl...@gmail.com> wrote:
> > > >
> >
> > > > > Devs,
> > > >
> >
> > > > > At our next ccc [1] we will be convening virtually. Is there
> > interest in
> > > >
> >
> > > > > doing a hackathon virtually as well? My current subject of eternal
> > > > >
> >
> > > > > torture
> > > >
> >
> > > > > is test frameworks (getting half way through porting tests to
> python3
> > > > >
> >
> > > > > now)
> > > >
> >
> > > > > but I can imagine way more interesting subjects to hack on (ui, the
> > new
> > > > >
> >
> > > > > SDN
> > > >
> >
> > > > > tungsten, any logging facility improvements, reimplementing
> > cloudstack in
> > > >
> >
> > > > > rust) We do not have to be very formal on subjects to start with
> but
> > we
> > > > >
> >
> > > > > do
> > > >
> >
> > > > > need to have enough people saying jeey and a time frame. Will we
> take
> > > > >
> >
> > > > > half
> > > >
> >
> > > > > a day, two days or something in between. If there is enough
> > enthusiasm we
> > > >
> >
> > > > > can do preemptive work on the wiki somewhere below [2]
> > > >
> >
> > > > > [1] http://cloudstackcollab.org/
> > > >
> >
> > > > > [2] https://cwiki.apache.org/confluence/display/CLOUDSTACK/Events
> > > >
> >
> > > > > regards
> > > >
> >
> > > > > Daan
> > >
> >
> > > --
> > >
> >
> > > Daan
>

[cloudstack-go] sdk releasing

[Pierre-Luc Dion]

I've messing around with cloudstack-go

Did a fix that rohit merged yesturday for hostsservices, but this fix will
only work for acs4.15, I'd like to fix it for previous acs version too, but
look like the version of the SDK depend on acs version;

Example; for the hostservices, the host attribute managementserverid is a
UUID in 4.15, but an integer in an older version of xenserver. This breaks
the structs,or map, we must have some other similar issue.

so I'd like to help on creating a tag or version of the SDK so they would
match acs version target,
ie: SDK version = v4.15-0; where the last digit would define the sdk
version or increase with fixes.
Current versioning in use = https://github.com/apache/cloudstack-go/releases

The issue I'm expecting to face is if we create a release , let's say
v4.15-0 from main branch today, if we want to create v4.14.0, it will not
be possible from the main branch because we need to revert the last commit
but also fix hostservices.

Here are a bunch of questions I have:
1. Should we create a branch for older ACS versions  and keep main for
latest fixes and future releases ?
2. Do we need to vote for such changes?
3. Does such releases could impact other Go projects that use this one,
such as terraform and kubernetes drivers?
4. Should we provide similar versioning on our kubernetes driver?

golang lib for cloudstack-api

[Pierre-Luc Dion]

Hello,

I've just found that we have a go library for cloudstack [1],
I'm curious to know if we are using this lib in cloudmonkey and our
terraform driver?
Is it considered as a stable lib ?

Cheers,

[1] https://github.com/apache/cloudstack-go

Re: [DISCUSS] NextGen VR?

[Pierre-Luc Dion]

Hi Rohit,

So look like OpenWRT could be a good alternative then, but how would you
handle other services such as load-balancing, vpn,...? would that be from
another Service VMs, kind of doing service mesh ?

Cheers,



On Mon, Aug 16, 2021 at 5:37 AM Rohit Yadav 
wrote:

> Hi PL,
>
> I tested OpenWRT on KVM, and I don't see any performance loss in terms of
> bandwidth/IO and memory, in fact it seems to be faster than the Debian
> based systemvmtemplate due to:
>
>   *   Smaller kernel size (the Debian kernel includes all sorts of device
> drivers for sound and what not which is not necessary for VR/systemvm)
>   *   Faster ssh (the dropbear ssh kernel is on-par with openssh-server
> but has smaller footprint and missing features such as sftp that we don't
> need in VR/systemvm; the ssh and programming seemed really fast compared to
> Debian's openssh-server)
>   *   Really low memory consumption (Debian10+ eats up some RAM due to
> non-optimised services, esp systemd compared to OpenWRT)
>   *   Lua based really-fast APIs and bus (openwrt has
> https://openwrt.org/docs/techref/ubus, I'm looking into how we can build
> something similar for our Debian based systemvmtemplate if migrating to
> OpenWRT is going to be a pain or unacceptable by the community)
>
> Based on your comment I found that VyOS rolling release has API support (
> https://blog.vyos.io/vyos-rolling-release-has-got-an-http-api) however
> that is probably not supported for their lts/legacy release, we'll also
> face issue with installing/configuring custom packages (java etc) that we
> need for systemvm/VR.
>
>
> Regards.
>
> 
> From: Pierre-Luc Dion 
> Sent: Friday, August 13, 2021 21:51
> To: dev 
> Subject: Re: [DISCUSS] NextGen VR?
>
> From a certain point of view, OpenWRT is most likely not suitable for our
> needs, this project is highly oriented for router appliance platforms and
> kernel is most likely highly optimised for a small memory footprint and
> might not have performance and virtualization in mind.
>
> It's a shame that there is not some sort of API driven open source VyOS :-S
>
> Anything useful that would come out of the CloudNative projects?
>
>
> On Fri, Aug 13, 2021 at 5:48 AM Rohit Yadav 
> wrote:
>
> > Hi Pierre-Luc,
> >
> > Thanks for replying.
> >
> > We've another thread on-going to discuss IPv6 support where we've indeed
> > discussed idea of introducing dynamic routing capabilities in the VR in a
> > future phase with FRR in the VR.
> >
> > For the VR agent idea, I'm indeed thinking of API driven way (the agent
> > exposes an API for CloudStack and a custom CLI perhaps) for it to manage
> > iptables, software packages/services such as dnsmasq etc. Right now it's
> > possible to configure some of the services in network offering (for ex.
> if
> > you don't want LB etc).
> >
> > I've indeed looked at some alternatives:
> >
> >   *   Alpine: I experimented a new systemvmtemplate based on Alpine (
> > https://github.com/shapeblue/alpine-systemvm) but we saw many cons with
> > Alpine and the gains weren't significant so we decided to drop the idea
> of
> > moving from Debian to Alpine.
> >   *   VyOS: I've concerns about project and community (I could be wrong)
> > to have our VR depend on it as the "default" provider; though I see an
> old
> > PR
> >   *   OpenWRT: Among available router distributions, I really liked it -
> > it has (a) a good health and active community and commit activity (
> > https://github.com/openwrt/openwrt/graphs/contributors), (b) multi-years
> > (10+yrs) of releases (https://openwrt.org/about/history), (c) a
> > configuration system (UCI) and UI (LUCI) which can be used to manage
> > services, interfaces, firewall, routes etc with a Lua based API, (d) opkg
> > packaging system, most software packages or equivalent are available that
> > our Debian based systemvmtemplate needs/uses (java jre workaround from
> > Alpine though), (e) good documentation on extending features/writing
> > plugins etc, (f) they tend to do regular releases and use Linux/LTS
> kernel.
> >   *   pfSense/opnSense: the major cons are FreeBSD-based kernel and
> skills
> > that CloudStack community may not have in terms of debugging,
> investigation
> > etc like they have for Linux
> >
> > Among the above, OpenWRT is probably worth exploring in my opinion; other
> > some firewall/iptables and iproute2 (interfaces and routes) API/library
> > that can be used by the VR agent on Debian to program the firewall (acls,
> > firewall, pf), IP/interface management and routing features

Re: [DISCUSS] NextGen VR?

[Pierre-Luc Dion]

>From a certain point of view, OpenWRT is most likely not suitable for our
needs, this project is highly oriented for router appliance platforms and
kernel is most likely highly optimised for a small memory footprint and
might not have performance and virtualization in mind.

It's a shame that there is not some sort of API driven open source VyOS :-S

Anything useful that would come out of the CloudNative projects?


On Fri, Aug 13, 2021 at 5:48 AM Rohit Yadav 
wrote:

> Hi Pierre-Luc,
>
> Thanks for replying.
>
> We've another thread on-going to discuss IPv6 support where we've indeed
> discussed idea of introducing dynamic routing capabilities in the VR in a
> future phase with FRR in the VR.
>
> For the VR agent idea, I'm indeed thinking of API driven way (the agent
> exposes an API for CloudStack and a custom CLI perhaps) for it to manage
> iptables, software packages/services such as dnsmasq etc. Right now it's
> possible to configure some of the services in network offering (for ex. if
> you don't want LB etc).
>
> I've indeed looked at some alternatives:
>
>   *   Alpine: I experimented a new systemvmtemplate based on Alpine (
> https://github.com/shapeblue/alpine-systemvm) but we saw many cons with
> Alpine and the gains weren't significant so we decided to drop the idea of
> moving from Debian to Alpine.
>   *   VyOS: I've concerns about project and community (I could be wrong)
> to have our VR depend on it as the "default" provider; though I see an old
> PR
>   *   OpenWRT: Among available router distributions, I really liked it -
> it has (a) a good health and active community and commit activity (
> https://github.com/openwrt/openwrt/graphs/contributors), (b) multi-years
> (10+yrs) of releases (https://openwrt.org/about/history), (c) a
> configuration system (UCI) and UI (LUCI) which can be used to manage
> services, interfaces, firewall, routes etc with a Lua based API, (d) opkg
> packaging system, most software packages or equivalent are available that
> our Debian based systemvmtemplate needs/uses (java jre workaround from
> Alpine though), (e) good documentation on extending features/writing
> plugins etc, (f) they tend to do regular releases and use Linux/LTS kernel.
>   *   pfSense/opnSense: the major cons are FreeBSD-based kernel and skills
> that CloudStack community may not have in terms of debugging, investigation
> etc like they have for Linux
>
> Among the above, OpenWRT is probably worth exploring in my opinion; other
> some firewall/iptables and iproute2 (interfaces and routes) API/library
> that can be used by the VR agent on Debian to program the firewall (acls,
> firewall, pf), IP/interface management and routing features in the VR
> (currently it's all done in a manual sense which is untested and often
> source of bugs and issues).
>
> Lastly, we already have this PR which aims to do automatic
> systemvmtemplate registeration/installation/seeding and handle upgrades
> (targetted for 4.16): https://github.com/apache/cloudstack/pull/4329 The
> PR also unifies the systemvm template use to be extended for CKS use-case
> (i.e. no separate template installation required for CKS).
>
>
> Regards.
>
> 
> From: Pierre-Luc Dion 
> Sent: Thursday, August 12, 2021 16:45
> To: dev 
> Subject: Re: [DISCUSS] NextGen VR?
>
> Hi Rohit,
>
> Like it! Our VR system is due for some rethinking!
> I don't have much points to add to issues you highlight, it seems pretty
> complete.
>
> Here are some more features or ideas that would be interesting to see in a
> new VR system:
>  - Use or support for routing protocol, such as BGP or OSBP so we could
> provide a more dynamic PrivateGateway concept. using FRR[1]?
>  - have an API driven way to configure IPtables and other network services.
>  - could we decouple network services such as LB, VPNserver, gateways from
> the VR ?
>
> Debian has been  a pain for building VR because the iso defined in our
> config need constant update, but on the other hand it's been proved to be a
> reliable OS, we saw some of our VR with uptime over 1000 days.
>
> As an alternative OS, I'd be curious to look at VyOS[2] or Alpine Linux.
>
> From a certain perspective, us providing the systemvm template make sure
> that systemVM will deploy. work reliably and make it a single template to
> test. Compared to a system that would just  provide RPM/DEB packages and
> mechanism to push configs, this could require to test all kinds of template
> scenarios, since users could use any version of distro to deploy their
> systemVM/VRs.
>
> "Users forget to register the right systemvmtemplate for a new ACS version
> and upgrades fail": Maybe we could automatically register

Re: [DISCUSS] NextGen VR?

[Pierre-Luc Dion]

Hi Rohit,

Like it! Our VR system is due for some rethinking!
I don't have much points to add to issues you highlight, it seems pretty
complete.

Here are some more features or ideas that would be interesting to see in a
new VR system:
 - Use or support for routing protocol, such as BGP or OSBP so we could
provide a more dynamic PrivateGateway concept. using FRR[1]?
 - have an API driven way to configure IPtables and other network services.
 - could we decouple network services such as LB, VPNserver, gateways from
the VR ?

Debian has been  a pain for building VR because the iso defined in our
config need constant update, but on the other hand it's been proved to be a
reliable OS, we saw some of our VR with uptime over 1000 days.

As an alternative OS, I'd be curious to look at VyOS[2] or Alpine Linux.

>From a certain perspective, us providing the systemvm template make sure
that systemVM will deploy. work reliably and make it a single template to
test. Compared to a system that would just  provide RPM/DEB packages and
mechanism to push configs, this could require to test all kinds of template
scenarios, since users could use any version of distro to deploy their
systemVM/VRs.

"Users forget to register the right systemvmtemplate for a new ACS version
and upgrades fail": Maybe we could automatically register the new template
during the upgrade process? This feature used to exist in the Citrix
CloudPlatform.


[1] https://frrouting.org/
[2] https://github.com/vyos/vyos-1x

On Wed, Aug 11, 2021 at 12:37 PM Rohit Yadav 
wrote:

> All,
>
> We've over the years create a VR that largely is stable but we've
> discussed the pain of maintaining, extending, and upgrading VRs both in
> lists and in user-groups and CCC conferences.
>
> On a high-level the pain points are:
>
>   *   It's difficult to debug, investigate VR for operators and support
> team
>   *   Maintaining the VR code, fixing bugs, implementing features is a
> pain for developers; further the xml databags based programming model
> is confusing
>   *   Any fix or changes requires a new systemvmtemplate or VR codebase
> whose patching requires restarting the VR or destroying an old VR
>   *   No uniform VR programming API (current approach is SSH+databags and
> execute a script), this makes testing VR difficult and QA in isolation is
> not possible
>   *   Users forget to register the right systemvmtemplate for a new ACS
> version and upgrades fail
>   *   Others (please share yours)?
>
> Among these pain points my colleagues have proposed a PR targeting 4.16
> [1] that aims to unify systemvmtemplate as a building block that is bundled
> as part of ACS rpm/deb/* packages which CloudStack will automatically
> seed/register/use with which upgrades will be as simple as a yum update or
> apt-get upgrade. Further, my colleagues and I are exploring a live patch
> API which in near future that can patch a running systemvm/VR using
> systemvm.iso (or deprecate systemvm.iso and use ssh/scp to patch?) without
> requiring to reboot/recreate it. Hopefully, this addresses some of those
> pain points. We request the community for your feedback and
> review/participation in the PR.
>
> Open questions, topics to discuss and gather feedback:
>
>   *   VR programming: Should we explore a new light-weight VR agent that
> provides an API (restful/grpc, or CLI?), some mechanism of live patching VR
> code, packages, and kernel?
>   *   Refactoring isolated network and VPC codebases into a unified
> codebase and feature sets (assumption that isolated network are largely a
> VPC with single tier), does it benefit the community, users, and developers?
>   *   Underlying OS:
>  *   should we consider something other than Debian, any suggestions?
>  *   or explore stable/widely used and maintained opensource router
> distributions such as OpenWRT [2] which ships with a UI and
> CLI/configuration system UCI [3]? The cons of the approach are a new
> dependency and some likely missing packages.
>  *   In current VR codebase, most of the effort is spent in
> implementing/maintaining router packages/configure codebase which we can
> get rid of by depending on a stable Linux router distro which ships with
> some API/config system. Not choosing an existing router distribution means
> we continue to DIY router programming+config management codebase.
>  *   Any other ideas?
>
> Thoughts, feedback?
>
> [1] https://github.com/apache/cloudstack/pull/4329
> [2] https://github.com/openwrt/openwrt/graphs/contributors
> [3] https://openwrt.org/docs/guide-user/base-system/uci
>
>
> Regards.
>
>
>
>

Re: [DISCUSS] Moving to OpenVPN as the remote access VPN provider

[Pierre-Luc Dion]

btw, I like the idea of CloudStack offering OpenVPN as a solution !

On Fri, Jun 11, 2021 at 10:40 AM Pierre-Luc Dion 
wrote:

> Just to be sure, what CloudStack > v4.15 uses Strongswan/l2tp or
> strongswan/ikev2 ?
>
> Because l2tp became complicated to configure on native vpn clients on some
> OSes, kind of deprecated remote management VPN, compared to IKEv2.
> I'm a bit concerned about OpenVPN for the clients, what if binaries become
> subscription based availability or become proprietary ?
>
> For sure we need the option to select what type of VPN solution to offer
> when deploying a cloud.
>
> From my perspective I cannot use/offer OpenVPN as a solution to my
> customers because it involves forcing them to download third party software
> on their workstations and I don't want to be responsible for
> a security breach on their workstation because of a requirement for 3rd
> party software that we don't control.
>
>
>
> On Fri, Jun 11, 2021 at 10:14 AM Rohit Yadav 
> wrote:
>
>> Thanks all for the feedback so far, looks like the majority of people on
>> the thread would prefer OpenVPN but for s2s they may continue to prefer
>> strongswan/ipsec for site-to-site VPC feature. If we're unable to reach
>> consensus then a general-purpose provider-framework may be more flexible to
>> the end-user or admin (to select which VPN provider they want for their
>> network, we heard in this thread - openvpn, strongswan/l2tp, wireguard, and
>> maybe other providers in future).
>>
>> Btw, ikev2 is supported now with strongswan with this -
>> https://github.com/apache/cloudstack/pull/4953
>>
>> My personal opinion: As user of most of these VPN providers, I personally
>> like OpenVPN which I found to be easier to use both on desktop/laptop and
>> on phone. With openvpn as the default I imagine in CloudStack I could
>> enable VPN for a network and CloudStack gives me an option to download a
>> .ovpn file which I can import in my openvpn client (desktop, phone, cli...)
>> click connect to connect to the VPN. For certificate generation/storage,
>> the CA framework could be used so the openvpn server certs are the same
>> across network restarts (with cleanup). I think a process like this could
>> be simpler than what we've right now, and the ovpn download+import workflow
>> would be easier than what we'll get from either strongswan/current or
>> wireguard. While I like the simplicity of wireguard, which is more like SSH
>> setup I wouldn't mind doing setup on individual VMs (much like setting up
>> ssh key) or use something like TailScale.
>>
>>
>> Regards.
>>
>> 
>> From: Gabriel Bräscher 
>> Sent: Friday, June 11, 2021 19:28
>> To: dev 
>> Cc: users 
>> Subject: Re: [DISCUSS] Moving to OpenVPN as the remote access VPN provider
>>
>> I understand that OpenVPN is a great option and far adopted.
>> I am  ++1 in allowing Users/Admins to choose which VPN provider suits them
>> best; creating an offering (or global settings) that would allow setting
>> which VPN provider will be used would be awesome.
>>
>> I understand that OpenVPN is a great option and far adopted; however, I
>> would be -1 if this would impact on removing support for Strongswan --
>> which from what I understood is not the proposal, but saying anyway to be
>> sure.
>>
>> Thanks for raising this proposal/discussion, Rohit.
>>
>> Cheers,
>> Gabriel.
>>
>>
>> Em sex., 11 de jun. de 2021 às 08:46, Pierre-Luc Dion <
>> pdion...@apache.org>
>> escreveu:
>>
>> > Hello,
>> >
>> > Daan, I agree we should provide capability to select the vpn solution to
>> > use, the question would be,  should it be a global setting generic for
>> the
>> > whole region or per VPC?
>> > I think it should be a global setting to reduce the requirement
>> complexity
>> > of a region, but per VPC or customer(account or domain) would be ideal.
>> >
>> > Hean, the current implementation from PR:2850
>> > <https://github.com/apache/cloudstack/pull/2850> that use strongswan
>> does
>> > support multiple users behind the same public IPs, but I don't recall
>> for
>> > Windows generic clients.
>> > With OpenVPN, can you be connected to multiple VPN tunnels at the same
>> time
>> > ? We had the challenge a few times where we needed to be connected to 2
>> > VPCs at the same time.
>> >
>> > I think adding support to OpenVPN is a good idea, the more options
>> > ava

Re: [DISCUSS] Moving to OpenVPN as the remote access VPN provider

[Pierre-Luc Dion]

Just to be sure, what CloudStack > v4.15 uses Strongswan/l2tp or
strongswan/ikev2 ?

Because l2tp became complicated to configure on native vpn clients on some
OSes, kind of deprecated remote management VPN, compared to IKEv2.
I'm a bit concerned about OpenVPN for the clients, what if binaries become
subscription based availability or become proprietary ?

For sure we need the option to select what type of VPN solution to offer
when deploying a cloud.

>From my perspective I cannot use/offer OpenVPN as a solution to my
customers because it involves forcing them to download third party software
on their workstations and I don't want to be responsible for
a security breach on their workstation because of a requirement for 3rd
party software that we don't control.



On Fri, Jun 11, 2021 at 10:14 AM Rohit Yadav 
wrote:

> Thanks all for the feedback so far, looks like the majority of people on
> the thread would prefer OpenVPN but for s2s they may continue to prefer
> strongswan/ipsec for site-to-site VPC feature. If we're unable to reach
> consensus then a general-purpose provider-framework may be more flexible to
> the end-user or admin (to select which VPN provider they want for their
> network, we heard in this thread - openvpn, strongswan/l2tp, wireguard, and
> maybe other providers in future).
>
> Btw, ikev2 is supported now with strongswan with this -
> https://github.com/apache/cloudstack/pull/4953
>
> My personal opinion: As user of most of these VPN providers, I personally
> like OpenVPN which I found to be easier to use both on desktop/laptop and
> on phone. With openvpn as the default I imagine in CloudStack I could
> enable VPN for a network and CloudStack gives me an option to download a
> .ovpn file which I can import in my openvpn client (desktop, phone, cli...)
> click connect to connect to the VPN. For certificate generation/storage,
> the CA framework could be used so the openvpn server certs are the same
> across network restarts (with cleanup). I think a process like this could
> be simpler than what we've right now, and the ovpn download+import workflow
> would be easier than what we'll get from either strongswan/current or
> wireguard. While I like the simplicity of wireguard, which is more like SSH
> setup I wouldn't mind doing setup on individual VMs (much like setting up
> ssh key) or use something like TailScale.
>
>
> Regards.
>
> 
> From: Gabriel Bräscher 
> Sent: Friday, June 11, 2021 19:28
> To: dev 
> Cc: users 
> Subject: Re: [DISCUSS] Moving to OpenVPN as the remote access VPN provider
>
> I understand that OpenVPN is a great option and far adopted.
> I am  ++1 in allowing Users/Admins to choose which VPN provider suits them
> best; creating an offering (or global settings) that would allow setting
> which VPN provider will be used would be awesome.
>
> I understand that OpenVPN is a great option and far adopted; however, I
> would be -1 if this would impact on removing support for Strongswan --
> which from what I understood is not the proposal, but saying anyway to be
> sure.
>
> Thanks for raising this proposal/discussion, Rohit.
>
> Cheers,
> Gabriel.
>
>
> Em sex., 11 de jun. de 2021 às 08:46, Pierre-Luc Dion  >
> escreveu:
>
> > Hello,
> >
> > Daan, I agree we should provide capability to select the vpn solution to
> > use, the question would be,  should it be a global setting generic for
> the
> > whole region or per VPC?
> > I think it should be a global setting to reduce the requirement
> complexity
> > of a region, but per VPC or customer(account or domain) would be ideal.
> >
> > Hean, the current implementation from PR:2850
> > <https://github.com/apache/cloudstack/pull/2850> that use strongswan
> does
> > support multiple users behind the same public IPs, but I don't recall for
> > Windows generic clients.
> > With OpenVPN, can you be connected to multiple VPN tunnels at the same
> time
> > ? We had the challenge a few times where we needed to be connected to 2
> > VPCs at the same time.
> >
> > I think adding support to OpenVPN is a good idea, the more options
> > available the better Cloudstack will be.
> >
> > I don't know if 4.15 still uses L2TP from strongswan but we've moved away
> > from it a while ago because it was not reliable, connection kept
> > dropping, support only one windows client at a time, issue configuring
> > clients, no helpful connection logs..
> >
> > An interesting improvement is made to remote access VPN, would be to
> > optionally use dns resolution of the VR from VPN clients so a user
> > connected to the VPN could use hostname to access VMs. I think iptabl

Re: [DISCUSS] Moving to OpenVPN as the remote access VPN provider

[Pierre-Luc Dion]

Hello,

Daan, I agree we should provide capability to select the vpn solution to
use, the question would be,  should it be a global setting generic for the
whole region or per VPC?
I think it should be a global setting to reduce the requirement complexity
of a region, but per VPC or customer(account or domain) would be ideal.

Hean, the current implementation from PR:2850
 that use strongswan does
support multiple users behind the same public IPs, but I don't recall for
Windows generic clients.
With OpenVPN, can you be connected to multiple VPN tunnels at the same time
? We had the challenge a few times where we needed to be connected to 2
VPCs at the same time.

I think adding support to OpenVPN is a good idea, the more options
available the better Cloudstack will be.

I don't know if 4.15 still uses L2TP from strongswan but we've moved away
from it a while ago because it was not reliable, connection kept
dropping, support only one windows client at a time, issue configuring
clients, no helpful connection logs..

An interesting improvement is made to remote access VPN, would be to
optionally use dns resolution of the VR from VPN clients so a user
connected to the VPN could use hostname to access VMs. I think iptable
currently blocks dns query from the vpn.

Cheers,

On Fri, Jun 11, 2021 at 5:58 AM Hean Seng  wrote:

> If thinking of only Site-to-Site VPN , then OpenVPN and WireGuard is  no
> much different , or even current one is gpod.  Only only time setup at
> router.  However if considering of Mobile Client, OpenVPN is more
> complicated.
>
> The only concern now is multiple people in the same public IP need to
> access the VPN.  And this consideration will be OpenVPN or Wireguard to
> handle this requirement.   And for this purpose of multiple people in same
> public ip need to access to VPN, then  we will have  think of usability and
> easy installation of VPN client.
>
> We are using OpenVPN for more then 5 years, but always  there is new PC
> need to configure VPN Client, windows , android, ios, it is painful ( we
> are not using access server) .
>
> Currently we test on WireGuard, just forgot about performance or
> whatsoever, just the conveniences of implementation,  that is very great
> and easy for client installation ,  even mobile client on phone or tablet.
>
>
>
>
> On Fri, Jun 11, 2021 at 5:04 PM Daan Hoogland 
> wrote:
>
> > This is a potential religious debate, I think it makes the most sense to
> > try and make the provider optional and let the operator or even the
> > end-user decide. I see how this is an extra challenge, but does it make
> > sense?
> >
> > On Thu, Jun 10, 2021 at 10:24 AM Rohit Yadav 
> > wrote:
> >
> > > All,
> > >
> > > We've historically supported openswan and nowadays strongswan as the
> VPN
> > > provider in VR for both site-to-site and remote access modes. After
> > > discussing the situation with a few users and colleagues I learnt that
> > > OpenVPN is generally far easier to use, have clients for most OS and
> > > platforms (desktop, laptop, tablet, phones...)  and allows multiple
> > clients
> > > in the same public IP (for example, multiple people in the office
> > sharing a
> > > client-side public IP/nat while trying to connect to a VPC or an
> isolated
> > > network) and for these reasons many users actually deploy pfSense or
> > setup
> > > a OpenVPN server in their isolated network or VPC and use that instead.
> > >
> > > Therefore for the point-to-point VPN use-case of remote access [1] does
> > it
> > > make sense to switch to OpenVPN? Or, are there users using
> > > strongswan/ipsec/l2tpd for remote access VPN?
> > >
> > > A general-purpose VPN-framework/provider where an account or admin (via
> > > offering) can specify which VPN provider they want in the network
> > > (strongswan/ipsec, OpenVPN, Wireguard...). However, it may be more
> > complex
> > > to implement and maintain. Any other thoughts in general about VPN
> > > implementation and support in CloudStack? Thanks.
> > >
> > > [1]
> > >
> >
> http://docs.cloudstack.apache.org/en/latest/adminguide/networking_and_traffic.html#remote-access-vpn
> > >
> > >
> > >
> > > Regards.
> > >
> > >
> > >
> > >
> >
> > --
> > Daan
> >
>
>
> --
> Regards,
> Hean Seng
>

Re: [DISCUSS] Moving to OpenVPN as the remote access VPN provider

[Pierre-Luc Dion]

Hello,

We've provided a PR for the remote management VPC to support IKEv2 using
SSL cert to auth the server and username/password for users [1].
The problem with OpenVPN is that it requires a custom client for some OSes
such as windows,  compared to IKEv2; it's supported out of the box on
Windows, OSx, Linux.
Client configuration is rather simple too.

This PR uses hashicorp Vault to generate an SSL cert and CA for
each customer, on a per domain basis if I remember correctly, that is push
to the VR as the cert for the IKEv2 vpn server.
then the UI and API allow you to download the CA so the client can trust
the vpn server to proceed with the vpn connection. that way to get a
connection you need to trust the CA and need a user/pass.
this was done before Cloudstack had internal CA generation features so the
PR would need some rework to support it I guess.

This solution is working great so far, connection time is almost immediate,
the tunnel is stable and reliable with a good performance compared to the
old L2TP version.

Another benefit with Stronswan + IKEv2 is user connection logs, we can get
username and connections state from logs on the VRs, valuable for
connections auditing.

[1] https://github.com/apache/cloudstack/pull/2850

Cheers,

On Thu, Jun 10, 2021 at 3:26 PM Wei ZHOU  wrote:

> Yes, OpenVPN is proposed to implement the remote access vpn feature (it is
> currently an IPSec/L2TP vpn server using Strongswan).
> site-to-site vpn in vpcs (also using strongswan) will not be changed.
>
> -Wei
> On Thu, 10 Jun 2021 at 18:51, Kristaps Cudars 
> wrote:
>
> > OpenVPN is SSL/TLS VPN and it has no support for IPSec. OpenVPN should
> > coexist with Strongswan. OpenVPN is ment for vpn client connective many
> to
> > one. Strongswan is meant for P2P connectivity.
> >
> > On 2021/06/10 08:39:14, Rudraksh MK 
> wrote:
> > > Hey!
> > >
> > > I’m personally a strong proponent of Wireguard. A couple years back,
> > implementing a S2S or remote-access VPN with WG was complicated and it
> > still is - but there’s definitely more tooling available these days.
> There
> > are clients for just about every major platform - desktop and mobile.
> > >
> > > In the long term though, I think a general-purpose VPN provider like
> the
> > one you outlined is far better - and I’d definitely like to take a stab
> at
> > it, although I’ll admit my Java skills are basically..zero. But even so
> - a
> > framework that allows users to select what platform they want -
> Strongswan
> > vs OpenVPN vs Wireguard - would be awesome.
> > >
> > >
> > > Best!
> > >
> > > Rudraksh Mukta Kulshreshtha
> > > Vice-President - DevOps & R
> > > IndiQus Technologies
> > > O +91 11 4055 1411 | M +91 99589 54879
> > > indiqus.com
> > >
> > > This message is intended only for the use of the individual or entity
> to
> > which it is addressed and may contain information that is confidential
> > and/or privileged. If you are not the intended recipient please delete
> the
> > original message and any copy of it from your computer system. You are
> > hereby notified that any dissemination, distribution or copying of this
> > communication is strictly prohibited unless proper authorization has been
> > obtained for such action. If you have received this communication in
> error,
> > please notify the sender immediately. Although IndiQus attempts to sweep
> > e-mail and attachments for viruses, it does not guarantee that both are
> > virus-free and accepts no liability for any damage sustained as a result
> of
> > viruses.
> > > On 10 Jun 2021, 1:55 PM +0530, Rohit Yadav  >,
> > wrote:
> > > > All,
> > > >
> > > > We've historically supported openswan and nowadays strongswan as the
> > VPN provider in VR for both site-to-site and remote access modes. After
> > discussing the situation with a few users and colleagues I learnt that
> > OpenVPN is generally far easier to use, have clients for most OS and
> > platforms (desktop, laptop, tablet, phones...) and allows multiple
> clients
> > in the same public IP (for example, multiple people in the office
> sharing a
> > client-side public IP/nat while trying to connect to a VPC or an isolated
> > network) and for these reasons many users actually deploy pfSense or
> setup
> > a OpenVPN server in their isolated network or VPC and use that instead.
> > > >
> > > > Therefore for the point-to-point VPN use-case of remote access [1]
> > does it make sense to switch to OpenVPN? Or, are there users using
> > strongswan/ipsec/l2tpd for remote access VPN?
> > > >
> > > > A general-purpose VPN-framework/provider where an account or admin
> > (via offering) can specify which VPN provider they want in the network
> > (strongswan/ipsec, OpenVPN, Wireguard...). However, it may be more
> complex
> > to implement and maintain. Any other thoughts in general about VPN
> > implementation and support in CloudStack? Thanks.
> > > >
> > > > [1]
> >
> 

Re: [VOTE] New life to Terraform Provider CloudStack with Apache CloudStack project

[Pierre-Luc Dion]

+1

On Tue, Apr 20, 2021 at 2:06 AM Suresh Anaparti <
suresh.anapa...@shapeblue.com> wrote:

> +1 , on this new repo under Apache. Hope the community takes it forward,
> with further improvements and maintenance of this provider.
>
> Regards,
> Suresh
>
> On 15/04/21, 2:35 PM, "Rohit Yadav"  wrote:
>
> Hi All,
>
> Following the discussion thread on Terraform [1], I would like to
> start a vote to gather consensus on the following actions:
>
>   1.  Create a new "cloudstack-terraform-provider" repository based on
> Apache Licence v2.0 using re-licensed codebase of the archived/former
> terraform cloudstack provider repository:
> https://github.com/hashicorp/terraform-provider-cloudstack (note:
> re-licensing from MPL to AL will be done by Hashicorp)
>   2.  Request ASF infra to enable issues, PR, and wiki features on the
> repository
>   3.  Work with the community towards any further maintenance,
> development, and releases of the provider
>   4.  Publish official releases on the official registry [2] if/after
> Apache CloudStack project gets a verified account (published by PMC members
> with access to the registry, or following guidelines from ASF infra if
> they've any)
>
> The vote will be open for 120 hours, until Wed 21 April 2021.
> For sanity in tallying the vote, can PMC members please be sure to
> indicate "(binding)" with their vote?
>
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)
>
> [1] https://markmail.org/message/iuggxin7kj6ri4hb
> [2] https://registry.terraform.io/browse/providers
>
>
> Regards.
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 3 London Bridge Street,  3rd floor, News Building, London  SE1 9SGUK
> @shapeblue
>
>
>
>
>
> suresh.anapa...@shapeblue.com
> www.shapeblue.com
> 3 London Bridge Street,  3rd floor, News Building, London  SE1 9SGUK
> @shapeblue
>
>
>
>

Re: Cloudstack developer training

[Pierre-Luc Dion]

This is a very good idea ! Thanks Giles, and Rohit, for putting this
together!



On Fri, Feb 26, 2021 at 10:43 AM Giles Sirett 
wrote:

> Hi all
>
>
>
> One of the biggest challenges with Cloudstack is learning its architecture
> and codebase  - its big and its complicated. Onboarding new software
> engineers can be a daunting process.
>
> For the last 2 years, we at ShapeBlue have built up a set of resources to
> help us with onboarding on new engineers who will be working on Cloudstack.
>
>
>
> This has evolved into a self-study course that we call “hackerbook”- the
> logic being that it’s a training course that gets engineers hands-on
> hacking in the code ASAP.  It’s a mix of videos, exercises and other
> resources.
>
>
>
> Today, we’ve opensourced this resource in order to make it available to
> anybody who may want to learn to develop on Cloudstack.
>
>
>
> Feedback and improvement PRs will be warmly accepted
>
>
>
> Its currently sitting in a shapeblue repo, happy to move under ASF if
> anybody thinks that’s important
>
>
>
> https://github.com/shapeblue/hackerbook
>
>
>
> Happy Hacking
>
>
>
> Kind regards
>
> Giles
>
>
>
> giles.sir...@shapeblue.com
> www.shapeblue.com
> @shapeblue
>
>
>
>

Re: [VOTE] Merge apache/cloudstack-primate with apache/cloudstack

[Pierre-Luc Dion]

+1

On Tue, Dec 29, 2020 at 3:57 AM Rohit Yadav 
wrote:

> Dear all,
>
> As discussed in a recent related thread and no objections were received on
> the proposal [1], I've started this voting thread on following:
>
>   *   Request ASF infra to archive the apache/cloudstack-primate
> (read-only)
> repository and remove all references to the name 'primate'
>   *   Move the UI code from apache/cloudstack-primate to
> apache/cloudstack's ui/
> directory (old code has been already moved to ui/legacy directory already
> in latest master,
> which will be removed from master after 4.15 branch is cut)
>   *   Fix packaging to build to bundle the ui/ codebase using npm and mvn;
> developers will
> need to run it separately while working on a feature, enhancement, and
> bugfix
>   *   The cloudstack-management rpm/deb pkg will include built UI (at the
> current of
> /usr/share/cloudstack-management/webapps) and a new package cloudstack-ui
> will be added
> for advanced users who only want to install/use the UI (installed at a new
> path,
>  for example, /usr/share/cloudstack-ui)
>
> The vote will be open for 72 hours and then extended until lazy consensus
> (3 PMC +1 votes) is reached.
> For sanity in tallying the vote, can PMC members please be sure to
> indicate "(binding)" with their vote?
>
> [ ] +1  approve
> [ ] +0  no opinion
> [ ] -1  disapprove (and reason why)
>
> [1] https://markmail.org/message/mdgeahlabpbx3v22
>
>
> Regards.
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 3 London Bridge Street,  3rd floor, News Building, London  SE1 9SGUK
> @shapeblue
>
>
>
>

-- 

*Pierre-Luc Dion*Lead Cloud Architect | Architecte infonuagique principal
t 888.796.8364 ext. 1403

c 514.880.7765

[image: LOGO-CloudDotCa-RGB (1) (1).png]

Re: [DISCUSS] python test framework

[Pierre-Luc Dion]

It would be interesting if we could come up with a  catalogue of api tests,
as serverspec [1] did for infrastructure tests.
if we would have some kind of library of tests, then the only  things we
would define in our tests suite would be the order of
unit tests and pretty much no code.

I don't know if there is an equivalent of serverspec framework on Python,
serverspec is the predecessor to Chef-inspec, unit tests for infra.
This could be a reference starting point.

Via nose2 seems to make sense.

[1] https://serverspec.org/


On Mon, Nov 16, 2020 at 5:34 AM  wrote:

> Hi there Dan,
>
> we work to setup the test environment and will share relevant output
> fragments with the limitation we can't share complete logs publicly.
>
> kind regards
> Peter
> 
> Von: Daan Hoogland 
> Gesendet: Montag, 16. November 2020 11:12:55
> An: dev
> Betreff: Re: [DISCUSS] python test framework
>
> sounds like I am on my own. Any body cares to chime in?
>
> On Thu, Nov 12, 2020 at 10:21 PM Nicolas Vazquez <
> nicolas.vazq...@shapeblue.com> wrote:
>
> > LGTM
> >
> >
> > Regards,
> >
> > Nicolas Vazquez
> >
> > 
> > From: Daan Hoogland 
> > Sent: Thursday, November 12, 2020 1:05 PM
> > To: dev 
> > Subject: Re: [DISCUSS] python test framework
> >
> > I checked the source code and the last commit is from the 3rd of March.
> > nose2 might require some effort from the community @here as well.
> >
> > On Thu, Nov 12, 2020 at 12:58 PM David Jumani <
> david.jum...@shapeblue.com>
> > wrote:
> >
> > > ++1 and like Boris said, need to look into backward compatibility
> > > 
> > > From: Boris Stoyanov 
> > > Sent: Thursday, November 12, 2020 2:26 PM
> > > To: dev@cloudstack.apache.org 
> > > Subject: Re: [DISCUSS] python test framework
> > >
> > > +1 on nose2, as the marvin plugin is built on nose, it seems the
> natural
> > > choice to upgrade to. We need to investigate backwards compatibility
> > > though..
> > >
> > > Bobby.
> > >
> > > On 11.11.20, 21:28, "Daan Hoogland" 
> > wrote:
> > >
> > > devs,
> > > Investigating an new organization of test categories , I found that
> > > our framework in use, nose, is no longer in maintenance. Its maintainer
> > has
> > > stopped his support in 2017. As it is quite stable it has slipped our
> > > attention, but it is urgent that we discuss and choose a way forward.
> > There
> > > is a successor, nose2, which as the documentation tells me, has less
> > > support for underlying test types but is also extensible. This is my
> goto
> > > choice to investigate further the coming days, but I’d like input from
> of
> > > you as many as possible.
> > > Regards,
> > >
> > > daan.hoogl...@shapeblue.com
> > > www.shapeblue.com
> > > 3 London Bridge Street,  3rd floor, News Building, London  SE1
> 9SGUK
> > > @shapeblue
> > >
> > >
> > >
> > >
> > >
> > > boris.stoya...@shapeblue.com
> > > www.shapeblue.com
> > > 3 London Bridge Street,  3rd floor, News Building, London  SE1 9SGUK
> > > @shapeblue
> > >
> > >
> > >
> > >
> > > david.jum...@shapeblue.com
> > > www.shapeblue.com
> > > 3 London Bridge Street,  3rd floor, News Building, London  SE1 9SGUK
> > > @shapeblue
> > >
> > >
> > >
> > >
> >
> > --
> > Daan
> >
> > nicolas.vazq...@shapeblue.com
> > www.shapeblue.com
> > 3 London Bridge Street,  3rd floor, News Building, London  SE1 9SGUK
> > @shapeblue
> >
> >
> >
> >
>
> --
> Daan
>

Re: Cloudstack

[Pierre-Luc Dion]

Hi Evgeniy,

They are supported with XenServer in case it can help you.

On Tue, Oct 27, 2020 at 7:21 AM Sven Vogel  wrote:

> Not at the moment.
>
>
> __
>
> Sven Vogel
> Lead Cloud Solution Architect
>
> EWERK DIGITAL GmbH
> Br?hl 24, D-04109 Leipzig
> P +49 341 42649 - 99
> F +49 341 42649 - 98
> s.vo...@ewerk.com
> www.ewerk.com
>
> Gesch?ftsf?hrer:
> Dr. Erik Wende, Hendrik Schubert, Tassilo M?schke
> Registergericht: Leipzig HRB 9065
>
> Support:
> +49 341 42649 555
>
> Zertifiziert nach:
> ISO/IEC 27001:2013
> DIN EN ISO 9001:2015
> DIN ISO/IEC 2-1:2011
>
> ISAE 3402 Typ II Assessed
>
> EWERK-Blog | LinkedIn<
> https://www.linkedin.com/company/ewerk-group> | Xing<
> https://www.xing.com/company/ewerk> | Twitter<
> https://twitter.com/EWERK_Group> | Facebook<
> https://de-de.facebook.com/EWERK.IT/>
>
>
> Ausk?nfte und Angebote per Mail sind freibleibend und unverbindlich.
>
> Disclaimer Privacy:
> Der Inhalt dieser E-Mail (einschlie?lich etwaiger beigef?gter Dateien) ist
> vertraulich und nur f?r den Empf?nger bestimmt. Sollten Sie nicht der
> bestimmungsgem??e Empf?nger sein, ist Ihnen jegliche Offenlegung,
> Vervielf?ltigung, Weitergabe oder Nutzung des Inhalts untersagt. Bitte
> informieren Sie in diesem Fall unverz?glich den Absender und l?schen Sie
> die E-Mail (einschlie?lich etwaiger beigef?gter Dateien) von Ihrem System.
> Vielen Dank.
>
> The contents of this e-mail (including any attachments) are confidential
> and may be legally privileged. If you are not the intended recipient of
> this e-mail, any disclosure, copying, distribution or use of its contents
> is strictly prohibited, and you should please notify the sender immediately
> and then delete it (including any attachments) from your system. Thank you.
>
> 
> Von: Evgeniy Tochilin 
> Gesendet: Tuesday, October 27, 2020 11:30:00 AM
> An: Rohit Yadav ; dev@cloudstack.apache.org <
> dev@cloudstack.apache.org>
> Betreff: Cloudstack
>
> Hi,
> In Cloudstack 4.14, GPU has supported on kvm hypervisor?
>
> --
>
> Best Regards,
> Evgeniy Tochilin
>

Re: [PROPOSAL] boot from network template on XenServer

[Pierre-Luc Dion]

Hi Rohit,

It is actually not quite the same.  the boot from network we are working on
is for a use case where the boot volume is on a streaming server basically,
so we need instance to boot from network to load the OS from that streaming
server basically. The template that would be created in CloudStack would
just refer to which FILENAME on the TFTP server would be used to boot. the
TFTP server IP would be defined at the VPC level.


the feature you describe for KVM is basically a way to register a regular
full template but send it directly to the PrimaryStorage instead of having
it in the Secondary Storage first, correct ?

Cheers,

On Sun, Aug 9, 2020 at 11:33 AM Rohit Yadav 
wrote:

> Hi PL,
>
> Good to hear from you!
>
> If you've a use-case where templates are created/hosted separately and may
> change often, could your use-case work by extending the direct-download
> feature for XenServer?
> Here are some links for the KVM support:
>
>
> http://docs.cloudstack.apache.org/en/latest/adminguide/templates.html#bypassing-secondary-storage-for-kvm-templates
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Bypass+Secondary+Storage+%28Direct+Download%29+on+KVM
>
> https://www.shapeblue.com/cloudstack-feature-first-look-direct-download-agnostic-of-the-storage-provider/
>
>
>
> Regards.
>
> 
> From: Pierre-Luc Dion 
> Sent: Saturday, August 8, 2020 20:06
> To: dev@cloudstack.apache.org ; Serge Nassar <
> snas...@cloud.ca>; Siddhartha Kattoju 
> Subject: [PROPOSAL] boot from network template on XenServer
>
> Hi guys, it's been a while!
>
> I'd like to know what would be the path to follow within the community to
> submit a feature into cloudstack, but also to review our proposal on
> offering a template that boots from the network.
>
> Basic idea, is that there is some workload, specially for XenDesktop where
> desktop images are hosted on a PVS server, to ease support of that workload
> in cloudstack, it kind of make sense that a Instance would boot from the
> network and load is root volume image from the PVS server. So what we
> recently tried in a VPC model, is to add a configuration at the VPC level
> to define the IP address of a TFTP server so the DHCP of the VR push the
> next_server ip defined in the VPC config, then we create a template where
> the URL is to boot file path on the tftp server that can reside outside of
> a vpc. At the VM creation using this template, the boot order of the
> instance on XenServer is to boot from the network and the root volume
> created is an empty volume that can be used as ephemeral or storage for
> caching.
>
> Cheers,
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 3 London Bridge Street,  3rd floor, News Building, London  SE1 9SGUK
> @shapeblue
>
>
>
>

[PROPOSAL] boot from network template on XenServer

[Pierre-Luc Dion]

Hi guys, it's been a while!

I'd like to know what would be the path to follow within the community to
submit a feature into cloudstack, but also to review our proposal on
offering a template that boots from the network.

Basic idea, is that there is some workload, specially for XenDesktop where
desktop images are hosted on a PVS server, to ease support of that workload
in cloudstack, it kind of make sense that a Instance would boot from the
network and load is root volume image from the PVS server. So what we
recently tried in a VPC model, is to add a configuration at the VPC level
to define the IP address of a TFTP server so the DHCP of the VR push the
next_server ip defined in the VPC config, then we create a template where
the URL is to boot file path on the tftp server that can reside outside of
a vpc. At the VM creation using this template, the boot order of the
instance on XenServer is to boot from the network and the root volume
created is an empty volume that can be used as ephemeral or storage for
caching.

Cheers,

Re: Centralised logging capability.

[Pierre-Luc Dion]

Hi,

This is a nice idea and initiative, we have a similar project in our
backlog, we cloud help to improve this PR.

There is some issues to send logs to management-server:
* this current feature impose use of rsyslog
* support  a single management server , would not work for a redundant
deployment of a management-server
* would require large amount of disk space if log rotate not managed
properly.
* a very good value added for a dev environment

Like Daan is suggesting, if the log destination could be a configurable
parameter, we could change the destination ip and port where to send logs
to a Logstash receiver potentially so we can centralize log in
another system such as an ELK cluster.

On our side we want to implement log forwarding for Virtual-Routers, the
idea we had in mind would be to deploy filebeat in the VR and sent their
log to the hypervisor internal management network, to the hypervisor host,
which would have a Port-Forwarding configure to the log destination IP.  So
far it's the simplest way we found it could work relatively safely without
adding a new NIC to VR or interacting with customers side of the VR. We
could also skip filebeat and just forward rsyslog but would need to be UDP
traffic for sure to avoid impacting perf or the VR is the destination is
down or experience high latency.

Cheers,

PL

On Wed, May 27, 2020 at 12:20 PM Sven Vogel  wrote:

> Hi Sina,
>
> First. Cool feature!
>
> Agree with Daan. This would be absolutely nice to have the possibility to
> send the log files to separate log server. So in the end of the day we have
> an Heavy management server.
>
> Cheers
>
> Sven
>
>
> __
>
> Sven Vogel
> Lead Cloud Solution Architect
>
> EWERK DIGITAL GmbH
> Brühl 24, D-04109 Leipzig
> P +49 341 42649 - 99
> F +49 341 42649 - 98
> s.vo...@ewerk.com
> www.ewerk.com
>
> Geschäftsführer:
> Dr. Erik Wende, Hendrik Schubert, Tassilo Möschke
> Registergericht: Leipzig HRB 9065
>
> Zertifiziert nach:
> ISO/IEC 27001:2013
> DIN EN ISO 9001:2015
> DIN ISO/IEC 2-1:2011
>
> EWERK-Blog | LinkedIn | Xing | Twitter | Facebook
>
> Auskünfte und Angebote per Mail sind freibleibend und unverbindlich.
>
> Disclaimer Privacy:
> Der Inhalt dieser E-Mail (einschließlich etwaiger beigefügter Dateien) ist
> vertraulich und nur für den Empfänger bestimmt. Sollten Sie nicht der
> bestimmungsgemäße Empfänger sein, ist Ihnen jegliche Offenlegung,
> Vervielfältigung, Weitergabe oder Nutzung des Inhalts untersagt. Bitte
> informieren Sie in diesem Fall unverzüglich den Absender und löschen Sie
> die E-Mail (einschließlich etwaiger beigefügter Dateien) von Ihrem System.
> Vielen Dank.
>
> The contents of this e-mail (including any attachments) are confidential
> and may be legally privileged. If you are not the intended recipient of
> this e-mail, any disclosure, copying, distribution or use of its contents
> is strictly prohibited, and you should please notify the sender immediately
> and then delete it (including any attachments) from your system. Thank you.
> > Am 27.05.2020 um 17:43 schrieb Daan Hoogland :
> >
> > great initiative Sina, I did leave a comment on the PR, about
> > configurability.
> > in short two worries:
> > 1. an operator uses a different log host than the MS (i.e an ip/hostname
> > config)
> > 2. an operator wants to not use the feature (i.e. a boolean flag)
> > I'm not sure how much this would require, but it seems minimal. Of
> course,
> > not opening the port is blocking the feature as well.
> >
> >
> >> On Wed, May 27, 2020 at 3:55 PM Sina Kashipazha
> >>  wrote:
> >>
> >>
> >> Centralised logging capability.
> >>
> >> Our improvements to systemvm allow Cloudstack administrator to access
> >> systemvms logs inside the management server. It removes the difficulty
> of
> >> downloading logs from systemvms. They are forwarded to the management
> >> server automatically, and administrators can access them in
> >> "/var/log/rsyslog/%HOSTNAME%/syslog”.
> >>
> >>
> >> It doesn’t require additional work, rsyslog setups on Cloudstack
> already.
> >> The only thing we have to do is open rsyslog port to receive these logs
> and
> >> tell hypervisors and systemvms to forward them to the management server.
> >>
> >> Any comments would be highly appreciated.
> >>
> >> For more information take a look at below issue and pull request:
> >> Issue: https://github.com/apache/cloudstack/issues/4093
> >> Pull request: https://github.com/apache/cloudstack/pull/4108 <
> >> https://github.com/apache/cloudstack/pull/4108>
> >>
> >> Kind Regards,
> >> Sina
> >
> >
> >
> > --
> > Daan
>

Re: [DISCUSS] SIG for SDN Tungsten Fabric Network Plugin

[Pierre-Luc Dion]

That's a nice initiative and good timing, we will be interesting and
willing to contribute as well, if we go with tungsten Fabric.

We recently looked at using Contrail, the commercial version of Tungsten,
not very impressed for managing network switches to be honest.
Overcomplicated software with limited configuration schema for switches.
Although we haven't looked at the service orchestration part.
It seems to be somewhat  useful when it comes to orchestrate services such
as firewall, LB, vxlan routers,...
Will also be challenging to deal with their documentation which lacked a
bit and also their high integration to OpenStack :-(
We are expecting to have a close eye on Tungsten for our network services
evolution.

Perso, I would look for another SDN, but I'm not knowledgeable in this
topic at the moment, it might be the best open source option available at
the moment.

Cheers,


On Mon, Feb 3, 2020 at 11:51 AM Simon Weller 
wrote:

> All,
>
> During the 2019 CCC @ Apachecon North America, a few of us discussed the
> need for a new Software Defined Networking (SDN) integration for
> CloudStack, now that Nuage has chosen to depreciate their SDN product
> portfolio.
> I've been working closely with Sven Vogel on outlining how we might be
> able to start a Special Interest Group (SIG) to design and build an ACS
> network plugin into the Linux Foundation project Tungsten Fabric (Formally
> known as Open Contrail).
>
> Both Sven's company, EWERK and my company ENA are willing to contribute
> developers to this effort, as we feel it's important for Apache CloudStack
> to have a robust SDN option that utilizes a well known and stable open
> source SDN project and one that is community supported.
> Although there is an existing plugin for Contrail that was originally
> contributed by Juniper, it has been orphaned in the ACS code base for many
> years and to my knowledge, it’s unusable.
> Over the years, we've had a number of SDN integrations come and go. This
> has left users in the lurch and discouraged other potential companies from
> considering these options, as one has to be confident in the longevity of
> the plugin.
>
> Why Tungsten Fabric?
> Tungsten Fabric has been around for quite a while and it is now officially
> a Linux Foundation project, so it has a considerable amount of support
> behind it. It's scalable, multi-tenant, supports a number of advanced
> security features, as well a large chunk of built in components we
> currently need a Virtual Router to provide.
> It's dual stack IPV4 and IPV6 and heavily utilizes BGP and MPLS. This
> makes it ideal for those of us that maintain our own networks, as it will
> provide tight integration options and eliminate the need for complicated
> Private Gateway (PG) setups for VPCs.
> Additionally, with service stitching and EVPN capabilities, it will make
> it a lot easier for operators to support other platforms without having to
> build a dedicated plugin, or figure out how to support those network or
> security features through other L2/L3 hacks.
>
> Sven and I would like to gauge feedback from the community on this
> proposal and see whether other organizations are interested in
> participating.
>
> Thanks,
>
> Sven and Simon
>

Re: Issue with newest mysql-connector-java

[Pierre-Luc Dion]

Hi,

We got similar issue earlier this week, initial install on CentOS 7 does
seams to work, but downgrading mysql connector to latest 5.1.x version
worked.

On Tue, Jan 21, 2020 at 1:21 PM Sean Lair  wrote:

> Opened Issue:
> https://github.com/apache/cloudstack/issues/3826
>
> We noticed that on mysql-connector-java version 8.0.19 (not sure about
> other 8.0.x versions) we have errors such as the following:
>
> Caused by: java.lang.IllegalArgumentException: Can not set long field
> com.cloud.upgrade.dao.VersionVO.id to java.math.BigInteger
> at
> sun.reflect.UnsafeFieldAccessorImpl.throwSetIllegalArgumentException(UnsafeFieldAccessorImpl.java:167)
> at
> sun.reflect.UnsafeFieldAccessorImpl.throwSetIllegalArgumentException(UnsafeFieldAccessorImpl.java:171)
> at
> sun.reflect.UnsafeLongFieldAccessorImpl.set(UnsafeLongFieldAccessorImpl.java:102)
>
> Looks like in code we are using Long with Auto Increment fields, but the
> DB columns are actually BigInt.  Downgrading to the EPEL release of
> mysql-connector-java (5.1.25-3) fixed the issue.  However, I expect lots of
> people would hit this, because in the upgrade guides we specify to add the
> mysql-community repo - which uses newer mysql-connectors:
>
>
> http://docs.cloudstack.apache.org/projects/archived-cloudstack-release-notes/en/4.11/upgrade/upgrade-4.9.html
>
> Thanks
> Sean
>

Re: CloudStack Kubernetes provider containers on DockerHub

[Pierre-Luc Dion]

I remember why we did create our own org in dockerhub, it was to be able to
create multiple images for various parts of CloudStack such as marvin,
cloudmonkey, management-server, ...
Would we be able to decouple cloudstack easily inside the apache org ?

Going to wait before releasing the current orgs  "cloudstack" and
"apachecloudstack"(empty), there is no way to hide them but I don't want
someone outside of our community take those names.

Cheers,

On Thu, Jan 16, 2020 at 1:53 AM Rohit Yadav 
wrote:

> Hi Paul,
>
> I think we added the Dockerfile, I did not actually do anything else.
> Maybe there is a hook that automatically kicks to build and publish
> container images on the apache dockerhub org (
> https://hub.docker.com/r/apache/cloudstack-cloudmonkey). I'm also not
> part of the apache org on dockerhub.
>
> Maybe try adding a Dockerfile if an automatic integration already exists
> or ask asf-dev community?
>
>
> Regards,
>
> Rohit Yadav
>
> Software Architect, ShapeBlue
>
> https://www.shapeblue.com
>
> 
> From: Paul Angus 
> Sent: Wednesday, January 15, 2020 22:30
> To: dev@cloudstack.apache.org ; Rohit Yadav <
> rohit.ya...@shapeblue.com>; Sven Vogel 
> Cc: Pierre-Luc Dion ; Will Stevens <
> wstev...@cloudops.com>; gabrasc...@gmail.com ; Wido
> den Hollander 
> Subject: RE: CloudStack Kubernetes provider containers on DockerHub
>
> Sorry folks, its been hectic...
>
> Yes Pierre-Luc, we should remove the non-Apache accounts.
>
> There is an /apache/cloudstack-cloudmonkey repository so it seems @Rohit
> Yadav knows how to get to upload to there @Sven Vogel
>
> I've been working on and off on docker files for building current
> simulator, as well as rpm/deb packaging images.  I was loosely planning to
> upload a current 'empty' simulator and a 'populated' simulator with
> multiple zones, accounts, VMs Networks, Firewall rules etc. It was with new
> UI development in mind.
>
> I was also thinking of a flow where we'd have automation where if there
> were any updates to the dockerfile we could press a button that would
> create and upload new build images.  An RM would be able to use the
> 'community' image to get a consistent outcome.
>
> Probably Feb before I can make them a polished reality.
>
> @Rohit Yadav can you share the process to get docker images pushed to
>
>
> Ever the optimist
>
> Paul.
>
>
>
> CTO
> paul.an...@shapeblue.com
> www.shapeblue.com<http://www.shapeblue.com>
>
>
>
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London  WC2E 9DPUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Sven Vogel 
> Sent: 14 January 2020 15:47
> To: dev 
> Cc: Pierre-Luc Dion ; Will Stevens <
> wstev...@cloudops.com>; gabrasc...@gmail.com; Wido den Hollander <
> w...@widodh.nl>
> Subject: Re: CloudStack Kubernetes provider containers on DockerHub
>
> Hi Pierre, Hi Paul,
>
> @Paul do you know how we proceed. How we can get the space from Apache and
> align to the policy?
>
> Cheers
>
> Sven
>
>
> __
>
> Sven Vogel
> Teamlead Platform
>
> EWERK DIGITAL GmbH
> Brühl 24, D-04109 Leipzig
> P +49 341 42649 - 99
> F +49 341 42649 - 98
> s.vo...@ewerk.com
> www.ewerk.com<http://www.ewerk.com>
>
> Geschäftsführer:
> Dr. Erik Wende, Hendrik Schubert, Frank Richter
> Registergericht: Leipzig HRB 9065
>
> Zertifiziert nach:
> ISO/IEC 27001:2013
> DIN EN ISO 9001:2015
> DIN ISO/IEC 2-1:2011
>
> EWERK-Blog | LinkedIn | Xing | Twitter | Facebook
>
> Auskünfte und Angebote per Mail sind freibleibend und unverbindlich.
>
> Disclaimer Privacy:
> Der Inhalt dieser E-Mail (einschließlich etwaiger beigefügter Dateien) ist
> vertraulich und nur für den Empfänger bestimmt. Sollten Sie nicht der
> bestimmungsgemäße Empfänger sein, ist Ihnen jegliche Offenlegung,
> Vervielfältigung, Weitergabe oder Nutzung des Inhalts untersagt. Bitte
> informieren Sie in diesem Fall unverzüglich den Absender und löschen Sie
> die E-Mail (einschließlich etwaiger beigefügter Dateien) von Ihrem System.
> Vielen Dank.
>
> The contents of this e-mail (including any attachments) are confidential
> and may be legally privileged. If you are not the intended recipient of
> this e-mail, any disclosure, copying, distribution or use of its contents
> is strictly prohibited, and you should please notify the sender immediately
> and then delete it (including any attachments) from your system. Thank you.
> > Am 13.01.2020 um 15:06 schrieb Pierre-Luc Dion :
> >
> > Hi Paul,
> >
> > So we should not

Re: CloudStack Kubernetes provider containers on DockerHub

[Pierre-Luc Dion]

Hi Paul,

So we should not use our 2 account but use under apache one instead,
correct?  if so, I will remove all owner and projects from the 2 I've
created a while back. they are outdated anyway so better hiding them then
keep old stuff that can cause confusion.


On Sun, Jan 12, 2020 at 4:10 PM Sven Vogel  wrote:

> Hi Paul,
>
> I agree! I would do the same If it’s possible.
>
> Cheers
>
>
> __
>
> Sven Vogel
> Teamlead Platform
>
> EWERK DIGITAL GmbH
> Brühl 24, D-04109 Leipzig
> P +49 341 42649 - 99
> F +49 341 42649 - 98
> s.vo...@ewerk.com
> www.ewerk.com
>
> Geschäftsführer:
> Dr. Erik Wende, Hendrik Schubert, Frank Richter
> Registergericht: Leipzig HRB 9065
>
> Zertifiziert nach:
> ISO/IEC 27001:2013
> DIN EN ISO 9001:2015
> DIN ISO/IEC 2-1:2011
>
> EWERK-Blog | LinkedIn | Xing | Twitter | Facebook
>
> Auskünfte und Angebote per Mail sind freibleibend und unverbindlich.
>
> Disclaimer Privacy:
> Der Inhalt dieser E-Mail (einschließlich etwaiger beigefügter Dateien) ist
> vertraulich und nur für den Empfänger bestimmt. Sollten Sie nicht der
> bestimmungsgemäße Empfänger sein, ist Ihnen jegliche Offenlegung,
> Vervielfältigung, Weitergabe oder Nutzung des Inhalts untersagt. Bitte
> informieren Sie in diesem Fall unverzüglich den Absender und löschen Sie
> die E-Mail (einschließlich etwaiger beigefügter Dateien) von Ihrem System.
> Vielen Dank.
>
> The contents of this e-mail (including any attachments) are confidential
> and may be legally privileged. If you are not the intended recipient of
> this e-mail, any disclosure, copying, distribution or use of its contents
> is strictly prohibited, and you should please notify the sender immediately
> and then delete it (including any attachments) from your system. Thank you.
> > Am 12.01.2020 um 21:40 schrieb Paul Angus :
> >
> > Hey Guys,
> >
> > Actually I've been meaning to get round to sorting this. We need to
> remove 'our' docker hub account and use the Apache run one
> > https://hub.docker.com/u/apache
> >
> > This came up on the Board mailing list with another project. We'll need
> to align ourselves with the policy also.
> >
> >
> >
> > Kind regards
> >
> >
> > Paul.
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > Amadeus House, Floral Street, London  WC2E 9DPUK
> > @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: Sven Vogel 
> > Sent: 12 January 2020 14:07
> > To: dev 
> > Cc: Pierre-Luc Dion ; Pierre-Luc Dion <
> pd...@cloudops.com>; Will Stevens ;
> gabrasc...@gmail.com; Wido den Hollander 
> > Subject: Re: CloudStack Kubernetes provider containers on DockerHub
> >
> > Hi Guys,
> >
> > Should we also find a ways to put the passwords accessible for all
> committers or PMC like Youtube Account?
> >
> > Make that sense?
> >
> > Cheers
> >
> > Sven
> >
> >
> > __
> >
> > Sven Vogel
> > Teamlead Platform
> >
> > EWERK DIGITAL GmbH
> > Brühl 24, D-04109 Leipzig
> > P +49 341 42649 - 99
> > F +49 341 42649 - 98
> > s.vo...@ewerk.com
> > www.ewerk.com
> >
> > Geschäftsführer:
> > Dr. Erik Wende, Hendrik Schubert, Frank Richter
> > Registergericht: Leipzig HRB 9065
> >
> > Zertifiziert nach:
> > ISO/IEC 27001:2013
> > DIN EN ISO 9001:2015
> > DIN ISO/IEC 2-1:2011
> >
> > EWERK-Blog | LinkedIn | Xing | Twitter | Facebook
> >
> > Auskünfte und Angebote per Mail sind freibleibend und unverbindlich.
> >
> > Disclaimer Privacy:
> > Der Inhalt dieser E-Mail (einschließlich etwaiger beigefügter Dateien)
> ist vertraulich und nur für den Empfänger bestimmt. Sollten Sie nicht der
> bestimmungsgemäße Empfänger sein, ist Ihnen jegliche Offenlegung,
> Vervielfältigung, Weitergabe oder Nutzung des Inhalts untersagt. Bitte
> informieren Sie in diesem Fall unverzüglich den Absender und löschen Sie
> die E-Mail (einschließlich etwaiger beigefügter Dateien) von Ihrem System.
> Vielen Dank.
> >
> > The contents of this e-mail (including any attachments) are confidential
> and may be legally privileged. If you are not the intended recipient of
> this e-mail, any disclosure, copying, distribution or use of its contents
> is strictly prohibited, and you should please notify the sender immediately
> and then delete it (including any attachments) from your system. Thank you.
> >> Am 12.01.2020 um 07:17 schrieb Rohit Yadav :
> >>
> >> Hi PL,
> >>
> >> Thanks for replying. My dock

Re: CloudStack Kubernetes provider containers on DockerHub

[Pierre-Luc Dion]

hi, sorry for the late response,

I still have admin access to or cloudstack org on docker hub, [1]  if you
want admin access to it, provide me your account name on docker hub and
I'll add you to the admin group.
Quickly looking at it, it's more than overdue, unfortunately all our
containers there are outdated :-(

We also have apachecloudstack org name if it's preferable.

These 2 orgs are not subprojects of Apache orgs in docker hub, and probably
it would be better to use the apache orgs  ? I don't have any rights on
apache docker hub org.

Cheers,


[1] https://hub.docker.com/orgs/cloudstack

On Fri, Jan 10, 2020 at 3:07 AM Riepl, Gregor (SWISS TXT) <
gregor.ri...@swisstxt.ch> wrote:

> Hello and happy new year to everybody!
>
> The k8s provider[1] is still lacking built containers so people can use it
> directly.
> Were you able to figure out how to access the mentioned Docker hub
> accounts in the meantime?
>
> We can also create a new one if necessary...
>
> Thanks,
> Greg
>
>
> [1] https://github.com/apache/cloudstack-kubernetes-provider
>
> 
> From: Rohit Yadav 
> Sent: 01 October 2019 12:18
> To: dev@cloudstack.apache.org ; Pierre-Luc
> Dion ; Will Stevens ;
> gabrasc...@gmail.com ; Wido den Hollander <
> w...@widodh.nl>
> Subject: Re: CloudStack Kubernetes provider containers on DockerHub
>
> Thanks Gregor.
>
> Let me tag a few people that may have access to one of the following
> docker hub organizations:
>
> apache
> apachecloudstack
> cloudstack
>
>
> Can you grant access to me (my docker username is 'bhaisaab') and Gregor?
> Thanks.
>
>
> Regards,
>
> Rohit Yadav
>
> Software Architect, ShapeBlue
>
> https://www.shapeblue.com
>
> 
> From: Riepl, Gregor (SWISS TXT) 
> Sent: Tuesday, October 1, 2019 14:30
> To: dev@cloudstack.apache.org 
> Subject: CloudStack Kubernetes provider containers on DockerHub
>
> Hi everyone,
>
> We still need to publish the new
> https://github.com/apache/cloudstack-kubernetes-provider on a public
> container registry.
> It looks like the ASF and/or CloudStack project already have accounts on
> DockerHub:
> https://hub.docker.com/u/apache
> https://hub.docker.com/u/apachecloudstack
>
> Who has access to these organisations and can publish the provider there?
> The Github repository contains a suitable Dockerfile that we used with
> DockerHub before and shouldn't require any complicated repository setup.
>
> If possible, please give access to @rhtyd so he can take care of things.
>
> Thanks!
> Gregor
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com<http://www.shapeblue.com>
> Amadeus House, Floral Street, London  WC2E 9DPUK
> @shapeblue
>
>
>
>

Re: vcpu socket configuration

[Pierre-Luc Dion]

splaytext": "Include zabbix agent",
>   "templateid": "6802f4cb-2529-48f4-bba9-ff7d82db3ab9",
>   "templatename": "Centos 7 EN Zabbix",
>   "userid": "d608a7ba-e184-11e8-9d95-4ab2a88c1305",
>   "username": "admin",
>   "zoneid": "2bc73a3d-4c92-49da-b323-1142a45dca8c",
>   "zonename": "zone1"
> }
>   ]
> }
>
> -邮件原件-
> 发件人: Nii Apleh Lartey mailto:niiap...@live.com>>
> 发送时间: 2019年11月25日 21:55
> 收件人: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>
> 主题: Re: vcpu socket configuration
>
> You have a "p" and the end of "details"
>
> 
> From: li jerry mailto:div...@hotmail.com>>
> Sent: Monday, November 25, 2019 1:52 PM
> To: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org> <
> dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>>
> Subject: 回复: vcpu socket configuration
>
> Thank you,
>
>
>
> Socket doesn't work after I deploy VM with the following command
>
>
>
> cmk deploy virtualmachine zoneid=2bc73a3d-4c92-49da-b323-1142a45dca8c
> templateid=6802f4cb-2529-48f4-bba9-ff7d82db3ab9
> serviceofferingid=b8850bca-88b4-4378-ba70-e87546db58fd
> name=test-cpu-socket-02 networkids=e28d44cf-9179-4526-81b6-990ef31ec9be
> detailsp[0].cpu.corespersocket=2
>
>
>
>
>
> ps: serviceoffering
>
>
>
> (localcloud)  > list serviceofferings
> id=b8850bca-88b4-4378-ba70-e87546db58fd
>
> {
>
> "Count": 1,
>
> "serviceoffering": [
>
> {
>
> "cpunumber": 20,
>
> "cpuspeed": 2000,
>
> "created": "2019-08-09T11:11:20+0800",
>
> "defaultuse": false,
>
> "deploymentplanner": "FirstFitPlanner",
>
> "displaytext": "SP01-20C4G",
>
> "domain": "ROOT",
>
> "domainid": "602d9221-e184-11e8-9d95-4ab2a88c1305",
>
> "hosttags": "ha",
>
> "id": "b8850bca-88b4-4378-ba70-e87546db58fd",
>
> "iscustomized": false,
>
> "issystem": false,
>
> "isvolatile": false,
>
> "limitcpuuse": false,
>
> "memory": 4096,
>
> "name": "SP01-20C4G",
>
> "offerha": true,
>
> "provisioningtype": "thin",
>
> "serviceofferingdetails": {
>
> "domainid": "1"
>
> }
>
> "storagetype": "shared",
>
> "tags": "rbd01"
>
> }
>
> ]
>
> }
> -邮件原件-
> 发件人: Pierre-Luc Dion mailto:pdion...@apache.org>>
> 发送时间: 2019年11月25日 21:36
> 收件人: dev@cloudstack.apache.org<mailto:dev@cloudstack.apache.org>
> 主题: Re: vcpu socket configuration
>
> Using cloudmonkey or cmk, you can specify the number of socket at the vm
> create using : "details[0].cpu.corespersocket=6"
> so if you define 8 vcpu with details[0].cpu.corespersocket=4  the vm will
> have 4 sockets @2cores.
>
> On Sun, Nov 24, 2019 at 8:41 AM li jerry  div...@hotmail.com>> wrote:
>
> > Hello All
> >
> > We found in the process of use, when the vcpu in service_offering is a
> > multiple of 6, vm cpu socket =vcpu/6, when vcpu is a multiple of 4, vm
> > cpu
> > socket=vcpu/4
> >
> >
> > Excuse me, why do we design this way?
> >
> > In my application scenario, there are restrictions on cpu socket, so I
> > want to lock the socket to cpu socket=vcpu/2, what are the drawbacks?
> > (Of course, I will exclude the singular vcpu)
> >
> >
> >
> >
> > -Jerry
> >
>

Re: vcpu socket configuration

[Pierre-Luc Dion]

When I used to use this command it was with XenServer has hypervisor, I've
never tested it with KVM. Also, make sure you use matching multiple of
vpcu/socket.
ie,  deploying 6 vcpu on 4 socket will not work.

On Mon, Nov 25, 2019 at 9:02 AM li jerry  wrote:

>
> After deploying the VM with the following command, the cpu socket is still
> invalid, but I saw cpu.corespersocket = 2 in the "Settings" of vm
>
>
> Ps :this is KVM Host
> cmk
> deploy virtualmachine zoneid=2bc73a3d-4c92-49da-b323-1142a45dca8c
> templateid=6802f4cb-2529-48f4-bba9-ff7d82db3ab9
> serviceofferingid=b8850bca-88b4-4378-ba70-e87546db58fd
> name=test-cpu-socket-02 networkids=e28d44cf-9179-4526-81b6-990ef31ec9be
> details[0].cpu.corespersocket=2
>
>
> cmk list vm
>
> "virtualmachine": [...
> {
>   "account": "admin",
>   "affinitygroup": [],
>   "cpunumber": 20,
>   "cpuspeed": 2000,
>   "cpuused": "0.05%",
>   "created": "2019-11-25T21:53:39+0800",
>   "details": {
> "Message.ReservedCapacityFreed.Flag": "false",
> "cpu.corespersocket": "2",
> "cpuOvercommitRatio": "4.0",
> "keyboard": "us",
> "memoryOvercommitRatio": "2.0"
>   },
>   "diskioread": 1,
>   "diskiowrite": 213,
>   "diskkbsread": 16,
>   "diskkbswrite": 4610,
>   "displayname": "test-cpu-socket-02",
>   "displayvm": true,
>   "domain": "ROOT",
>   "domainid": "602d9221-e184-11e8-9d95-4ab2a88c1305",
>   "guestosid": "c7a858b6-e184-11e8-9d95-4ab2a88c1305",
>   "haenable": true,
>   "hostid": "f0613074-9b79-4858-9dda-2453bca9c3e6",
>   "hostname": "hcicn03",
>   "hypervisor": "KVM",
>   "id": "f9e32a8c-1d27-4cff-b835-305b11c93fa5",
>   "instancename": "i-2-1133-VM",
>   "isdynamicallyscalable": false,
>   "memory": 4096,
>   "memoryintfreekbs": 755744,
>   "memorykbs": 4194304,
>   "memorytargetkbs": 4194304,
>   "name": "test-cpu-socket-02",
>   "networkkbsread": 0,
>   "networkkbswrite": 0,
>   "nic": [
> {
>   "broadcasturi": "vlan://835",
>   "extradhcpoption": [],
>   "gateway": "10.1.1.1",
>   "id": "d756de24-ac07-4ad4-8238-40f6d6f9b10c",
>   "ipaddress": "10.1.1.30",
>   "isdefault": true,
>   "isolationuri": "vlan://835",
>   "macaddress": "02:00:52:aa:00:08",
>   "netmask": "255.255.255.0",
>   "networkid": "e28d44cf-9179-4526-81b6-990ef31ec9be",
>   "networkname": "test-assistanz-mon",
>   "secondaryip": [],
>   "traffictype": "Guest",
>   "type": "Isolated"
> }
>   ],
>   "ostypeid": "c7a858b6-e184-11e8-9d95-4ab2a88c1305",
>   "passwordenabled": true,
>   "rootdeviceid": 0,
>   "rootdevicetype": "ROOT",
>   "securitygroup": [],
>   "serviceofferingid": "b8850bca-88b4-4378-ba70-e87546db58fd",
>   "serviceofferingname": "SP01-20C4G",
>   "state": "Running",
>   "tags": [],
>   "templatedisplaytext": "Include zabbix agent",
>   "templateid": "6802f4cb-2529-48f4-bba9-ff7d82db3ab9",
>   "templatename": "Centos 7 EN Zabbix",
>   "userid": "d608a7ba-e184-11e8-9d95-4ab2a88c1305",
>   "username": "admin",
>   "zoneid": "2bc73a3d-4c92-49da-b323-1142a45dca8c",
>   "zonename": "zone1"
> }
>   ]
> }
>
> -邮件原件-
> 发件人: Nii Apleh Lartey 
> 发送时间: 2019年11月25日 21:55
> 收件人: dev@cloudstack.apache.org
> 主题: Re: vcpu socket configuration
>
> You have a "p" and the end of "details"
>
> ________
> From: l

Re: vcpu socket configuration

[Pierre-Luc Dion]

Using cloudmonkey or cmk, you can specify the number of socket at the vm
create using : "details[0].cpu.corespersocket=6"
so if you define 8 vcpu with details[0].cpu.corespersocket=4  the vm will
have 4 sockets @2cores.

On Sun, Nov 24, 2019 at 8:41 AM li jerry  wrote:

> Hello All
>
> We found in the process of use, when the vcpu in service_offering is a
> multiple of 6, vm cpu socket =vcpu/6, when vcpu is a multiple of 4, vm cpu
> socket=vcpu/4
>
>
> Excuse me, why do we design this way?
>
> In my application scenario, there are restrictions on cpu socket, so I
> want to lock the socket to cpu socket=vcpu/2, what are the drawbacks? (Of
> course, I will exclude the singular vcpu)
>
>
>
>
> -Jerry
>

Re: [DISCUSS] CloudStack Kubernetes Service plugin

[Pierre-Luc Dion]

Make sense for the proposed implementation, would it handle redundant
master?
How would the k8s cluster would be created, using Rancher tools, kubectl or
other?

so far, the small part I understand from MaaS, it could be very interesting
to integrate it to cloudstack in a way where it could be use to  scale
Hypervisor host, specially KVM nodes.


On Wed, Sep 25, 2019 at 10:47 AM Paul Angus 
wrote:

> The proposed implementation will create a master and n worker nodes.
> It will also support (graceful) cluster resizing, the next step would be
> to enable the CloudStack plugin for Kubernetes to allow Kubernetes to drive
> that scaling, so that you can scale with demand rather than needing to
> oversize you environment to begin with.
>
> I've been keeping MaaS in mind as way of doing baremetal Kubernetes along
> side VM based Kubernetes clusters.  Interestingly a few people that I have
> spoken to have said that they prefer the use of VMs, because whole servers
> as the unit of scale is often very wasteful, unless you 'share' them which
> has all sorts of security implications...
>
>
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London  WC2E 9DPUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Pierre-Luc Dion 
> Sent: 25 September 2019 15:31
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] CloudStack Kubernetes Service plugin
>
> Hi Paul,
>
> Yeah, was bad timing for the CCCNA this year unfortunately :-(,  I'm not
> sure I'm curious to see how cloudstack could become more "other Apache
> products friendly" but I don't have particular use case compared to k8s
> integration. Has you are suggesting, would probably make sense to use Helm
> to deploy any other application stack.
>
> btw, we are still working on the Canonical MaaS integration, a bit more
> challenging than anticipated...
>
>
> To get back to a *Kubernetes Service plugin*:
> To me, as a user of cloudstack at the moment, If I deploy a k8s cluster, I
> need to deploy monstrous instances for worker nodes.
> which doesn't make sense if I'm a cloud consumer. So I think we need to
> solve something challenging: a k8s service that would scale has needed
> while keeping in mind redundancy of worker nodes without sacrifice on
> security. Is the worker node is part of the ongoing work or it's more about
> offering a k8s master and api infrastructure to a user ?
>
> An easy path would be some kind of shared worker nodes pool but that
> involve possible security risk unless you would trust users that consume
> those workers.
>
>
> On Wed, Sep 25, 2019 at 10:15 AM Paul Angus 
> wrote:
>
> > Hi Pierre-Luc,
> >
> > (we missed you at CCCNA!) How are you seeing CloudStack being more
> > deployment friendly?  What you do think that we could do on top of
> > creating the Kubenetes Cluster to begin with?
> > [thinking out loud - we could pre-package Tiller to make it easier to
> > deploy openWhisk via Helm charts ? ]
> >
> > Kind regards
> >
> >
> > Paul.
> >
> >
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > Amadeus House, Floral Street, London  WC2E 9DPUK @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: Pierre-Luc Dion 
> > Sent: 25 September 2019 13:37
> > To: dev@cloudstack.apache.org
> > Subject: Re: [DISCUSS] CloudStack Kubernetes Service plugin
> >
> > Hi Rohit, Nux,
> >
> > Thanks Rohit for cloudstack-provider, that's exactly it ! :-D Nux, I
> agree
> > with your opinion, but there is a lot of interest for k8s and seams like
> a
> > lot of organisations are moving to container based infrastructures to
> > standardized their deployment.
> >
> > if we want to extent the discussion to function as a service, would you
> > guys see a possibility for us to be more aligned or more deployment
> > friendly for Openwhisk ?
> >
> > Cheers,
> >
> >
> > On Wed, Sep 25, 2019 at 6:54 AM Will Stevens 
> > wrote:
> >
> > > We see huge demand for K8s in our customer base. Just a note...
> > >
> > > On Wed, Sep 25, 2019, 4:03 AM Nux!  wrote:
> > >
> > > > Do you guys see high demand for K8s?
> > > >  From where I'm looking it seems to be going the way of Openstack,
> > > > loads of hype, overcomplicated, near-impossible to upgrade.
> > > > Not sure if it's worth investing resources for this.
> > > >
> > > > Lucian
> > > >
> > > > ---
> > > > Sent from the Delta quadrant using Borg technology!
> &

Re: [DISCUSS] CloudStack Kubernetes Service plugin

[Pierre-Luc Dion]

Hi Paul,

Yeah, was bad timing for the CCCNA this year unfortunately :-(,  I'm not
sure I'm curious to see how cloudstack could become more
"other Apache products friendly" but I don't have particular use case
compared to k8s integration. Has you are suggesting,
would probably make sense to use Helm to deploy any other application stack.

btw, we are still working on the Canonical MaaS integration, a bit more
challenging than anticipated...


To get back to a *Kubernetes Service plugin*:
To me, as a user of cloudstack at the moment, If I deploy a k8s cluster, I
need to deploy monstrous instances for worker nodes.
which doesn't make sense if I'm a cloud consumer. So I think we need to
solve something challenging: a k8s service that would scale has needed
while keeping in mind redundancy of worker nodes without sacrifice on
security. Is the worker node is part of the ongoing work or it's more about
offering a k8s master and api infrastructure to a user ?

An easy path would be some kind of shared worker nodes pool but that
involve possible security risk unless you would trust users that consume
those workers.


On Wed, Sep 25, 2019 at 10:15 AM Paul Angus 
wrote:

> Hi Pierre-Luc,
>
> (we missed you at CCCNA!) How are you seeing CloudStack being more
> deployment friendly?  What you do think that we could do on top of creating
> the Kubenetes Cluster to begin with?
> [thinking out loud - we could pre-package Tiller to make it easier to
> deploy openWhisk via Helm charts ? ]
>
> Kind regards
>
>
> Paul.
>
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London  WC2E 9DPUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Pierre-Luc Dion 
> Sent: 25 September 2019 13:37
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] CloudStack Kubernetes Service plugin
>
> Hi Rohit, Nux,
>
> Thanks Rohit for cloudstack-provider, that's exactly it ! :-D Nux, I agree
> with your opinion, but there is a lot of interest for k8s and seams like a
> lot of organisations are moving to container based infrastructures to
> standardized their deployment.
>
> if we want to extent the discussion to function as a service, would you
> guys see a possibility for us to be more aligned or more deployment
> friendly for Openwhisk ?
>
> Cheers,
>
>
> On Wed, Sep 25, 2019 at 6:54 AM Will Stevens 
> wrote:
>
> > We see huge demand for K8s in our customer base. Just a note...
> >
> > On Wed, Sep 25, 2019, 4:03 AM Nux!  wrote:
> >
> > > Do you guys see high demand for K8s?
> > >  From where I'm looking it seems to be going the way of Openstack,
> > > loads of hype, overcomplicated, near-impossible to upgrade.
> > > Not sure if it's worth investing resources for this.
> > >
> > > Lucian
> > >
> > > ---
> > > Sent from the Delta quadrant using Borg technology!
> > >
> > > On 2019-09-24 07:41, Abhishek Kumar wrote:
> > > > Hi all,
> > > >
> > > > I would like to propose developing a plugin for Kubernetes
> > > > integration in CloudStack, can be named CloudStack Kubernetes
> Service plugin.
> > > > I've written down an initial design document for it here,
> > > >
> > >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Kube
> > rnetes+Service
> > > > Please review and provide your thoughts and suggestions.
> > > >
> > > > Regards,
> > > >
> > > >
> > > > Abhishek Kumar
> > > >
> > > > Software Engineer
> > > >
> > > > ShapeBlue
> > > >
> > > > abhishek.ku...@shapeblue.com
> > > >
> > > > www.shapeblue.com<http://www.shapeblue.com>
> > > >
> > > > abhishek.ku...@shapeblue.com
> > > > www.shapeblue.com
> > > > Amadeus House, Floral Street, London  WC2E 9DPUK @shapeblue
> > >
> >
>
>
> --
>
> *Pierre-Luc Dion*Lead Cloud Architect | Architecte infonuagique principal
> t 1.888.796.8364 ext. 1403
>
>
> <
> https://cloud.ca/?utm_source=email_medium=signature_content=cloud-ca-logo-1_campaign=general_email
> >
>


--

Re: [DISCUSS] CloudStack Kubernetes Service plugin

[Pierre-Luc Dion]

Hi Rohit, Nux,

Thanks Rohit for cloudstack-provider, that's exactly it ! :-D
Nux, I agree with your opinion, but there is a lot of interest for k8s and
seams like
a lot of organisations are moving to container based infrastructures to
standardized their deployment.

if we want to extent the discussion to function as a service, would you
guys see a possibility for us to
be more aligned or more deployment friendly for Openwhisk ?

Cheers,


On Wed, Sep 25, 2019 at 6:54 AM Will Stevens  wrote:

> We see huge demand for K8s in our customer base. Just a note...
>
> On Wed, Sep 25, 2019, 4:03 AM Nux!  wrote:
>
> > Do you guys see high demand for K8s?
> >  From where I'm looking it seems to be going the way of Openstack, loads
> > of hype, overcomplicated, near-impossible to upgrade.
> > Not sure if it's worth investing resources for this.
> >
> > Lucian
> >
> > ---
> > Sent from the Delta quadrant using Borg technology!
> >
> > On 2019-09-24 07:41, Abhishek Kumar wrote:
> > > Hi all,
> > >
> > > I would like to propose developing a plugin for Kubernetes integration
> > > in CloudStack, can be named CloudStack Kubernetes Service plugin.
> > > I've written down an initial design document for it here,
> > >
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Kubernetes+Service
> > > Please review and provide your thoughts and suggestions.
> > >
> > > Regards,
> > >
> > >
> > > Abhishek Kumar
> > >
> > > Software Engineer
> > >
> > > ShapeBlue
> > >
> > > abhishek.ku...@shapeblue.com
> > >
> > > www.shapeblue.com<http://www.shapeblue.com>
> > >
> > > abhishek.ku...@shapeblue.com
> > > www.shapeblue.com
> > > Amadeus House, Floral Street, London  WC2E 9DPUK
> > > @shapeblue
> >
>


-- 

*Pierre-Luc Dion*Lead Cloud Architect | Architecte infonuagique principal
t 1.888.796.8364 ext. 1403


<https://cloud.ca/?utm_source=email_medium=signature_content=cloud-ca-logo-1_campaign=general_email>

Re: [DISCUSS] CloudStack Kubernetes Service plugin

[Pierre-Luc Dion]

hello,

That look interesting, but any plan on doing the other way around too?
Like having k8s consume CloudStack resources so a k8s cluster could update
load-balancing service of a VPC or other network service from cloudstack,
attach Data-volumes and such..



On Tue, Sep 24, 2019 at 2:42 AM Abhishek Kumar 
wrote:

> Hi all,
>
> I would like to propose developing a plugin for Kubernetes integration in
> CloudStack, can be named CloudStack Kubernetes Service plugin.
> I've written down an initial design document for it here,
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/CloudStack+Kubernetes+Service
> Please review and provide your thoughts and suggestions.
>
> Regards,
>
>
> Abhishek Kumar
>
> Software Engineer
>
> ShapeBlue
>
> abhishek.ku...@shapeblue.com
>
> www.shapeblue.com
>
> abhishek.ku...@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London  WC2E 9DPUK
> @shapeblue
>
>
>
>

-- 

*Pierre-Luc *

Re: Secondary Storage VM timeout issue every hour

[Pierre-Luc Dion]

Do you have a load balancer in front of cloudstack? Did you set the global
settings "host" to the ip of the mgmt server?


Le jeu. 25 juill. 2019 03 h 24, Rakesh Venkatesh 
a écrit :

> Hello People
>
>
> I have a strange issue where mgt server times out to send a command to
> secondary storage VM every hour and because of this UI won't be accessible
> for a short duration of time. Sometimes I have to restart mgt server to get
> it back to working state and sometimes I don't need to restart it. I also
> see some exceptions while fetching the storage stats.
>
>
> The log says secondary storage VM is lagging behind mgt server in ping and
> it sends a disconnect message to other components. Can you let me know how
> to troubleshoot this issue? I destroyed the secondary storage VM but the
> issue still persists. I checked the date/time on the mgt server and SSVM
> and they are same. This is happening for quite a few days now. Below are
> the logs
>
>
>
> 2019-07-25 04:01:22,769 INFO  [c.c.a.m.AgentManagerImpl]
> (AgentMonitor-1:ctx-c33dbe74) (logid:5442158c) Found the following agents
> behind on ping: [183]
> 2019-07-25 04:01:22,775 WARN  [c.c.a.m.AgentManagerImpl]
> (AgentMonitor-1:ctx-c33dbe74) (logid:5442158c) Disconnect agent for
> CPVM/SSVM due to physical connection close. host: 183
> 2019-07-25 04:01:22,778 INFO  [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-1:ctx-66de2057) (logid:841d2a63) Host 183 is disconnecting
> with event ShutdownRequested
> 2019-07-25 04:01:22,781 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-1:ctx-66de2057) (logid:841d2a63) The next status of agent
> 183is Disconnected, current status is Up
> 2019-07-25 04:01:22,781 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-1:ctx-66de2057) (logid:841d2a63) Deregistering link for 183
> with state Disconnected
> 2019-07-25 04:01:22,781 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-1:ctx-66de2057) (logid:841d2a63) Remove Agent : 183
> 2019-07-25 04:01:22,781 DEBUG [c.c.a.m.ConnectedAgentAttache]
> (AgentTaskPool-1:ctx-66de2057) (logid:841d2a63) Processing Disconnect.
> 2019-07-25 04:01:22,782 DEBUG [c.c.a.m.AgentAttache]
> (AgentTaskPool-1:ctx-66de2057) (logid:841d2a63) Seq
> 183-7541559051008607242: Sending disconnect to class
> com.cloud.agent.manager.SynchronousListener
> 2019-07-25 04:01:22,782 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-1:ctx-66de2057) (logid:841d2a63) Sending Disconnect to
> listener: com.cloud.hypervisor.xenserver.discoverer.XcpServerDiscoverer
> 2019-07-25 04:01:22,782 DEBUG [c.c.u.n.NioConnection]
> (pool-2-thread-1:null) (logid:) Closing socket Socket[addr=/172.30.32.16
> ,port=38250,localport=8250]
> 2019-07-25 04:01:22,782 DEBUG [c.c.a.m.AgentAttache]
> (StatsCollector-2:ctx-b55657a9) (logid:dafc4881) Seq
> 183-7541559051008607242: Waiting some more time because this is the current
> command
> 2019-07-25 04:01:22,782 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-1:ctx-66de2057) (logid:841d2a63) Sending Disconnect to
> listener: com.cloud.hypervisor.hyperv.discoverer.HypervServerDiscoverer
> 2019-07-25 04:01:22,783 DEBUG [c.c.a.m.AgentAttache]
> (StatsCollector-2:ctx-b55657a9) (logid:dafc4881) Seq
> 183-7541559051008607242: Waiting some more time because this is the current
> command
> 2019-07-25 04:01:22,783 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-1:ctx-66de2057) (logid:841d2a63) Sending Disconnect to
> listener: com.cloud.deploy.DeploymentPlanningManagerImpl
> 2019-07-25 04:01:22,783 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-1:ctx-66de2057) (logid:841d2a63) Sending Disconnect to
> listener: com.cloud.network.security.SecurityGroupListener
> 2019-07-25 04:01:22,783 INFO  [c.c.u.e.CSExceptionErrorCode]
> (StatsCollector-2:ctx-b55657a9) (logid:dafc4881) Could not find exception:
> com.cloud.exception.OperationTimedoutException in error code list for
> exceptions
> 2019-07-25 04:01:22,783 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-1:ctx-66de2057) (logid:841d2a63) Sending Disconnect to
> listener: org.apache.cloudstack.engine.orchestration.NetworkOrchestrator
> 2019-07-25 04:01:22,783 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-1:ctx-66de2057) (logid:841d2a63) Sending Disconnect to
> listener: com.cloud.vm.ClusteredVirtualMachineManagerImpl
> 2019-07-25 04:01:22,783 WARN  [c.c.a.m.AgentAttache]
> (StatsCollector-2:ctx-b55657a9) (logid:dafc4881) Seq
> 183-7541559051008607242: Timed out on null
> 2019-07-25 04:01:22,783 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-1:ctx-66de2057) (logid:841d2a63) Sending Disconnect to
> listener: com.cloud.storage.listener.StoragePoolMonitor
> 2019-07-25 04:01:22,784 DEBUG [c.c.a.m.AgentAttache]
> (StatsCollector-2:ctx-b55657a9) (logid:dafc4881) Seq
> 183-7541559051008607242: Cancelling.
> 2019-07-25 04:01:22,784 DEBUG [c.c.a.m.AgentManagerImpl]
> (AgentTaskPool-1:ctx-66de2057) (logid:841d2a63) Sending Disconnect to
> listener: com.cloud.storage.secondary.SecondaryStorageListener
> 2019-07-25 04:01:22,784 DEBUG 

Re: [VOTE] Remove el6 support in future CloudStack versions (was Re: [DISCUSS] Remove support for el6 packaging in 4.13/4.14)

[Pierre-Luc Dion]

+1

On Tue, May 21, 2019 at 8:18 AM Wido den Hollander  wrote:

> +1
>
> On 5/21/19 11:40 AM, Rohit Yadav wrote:
> > All,
> >
> >
> > Thank you for your feedback and discussions. From what we've discussed
> so far, we've lazy consensus that nobody wants to use el6 or are limited to
> upgrade to el7/el8 due to potential risks.
> >
> >
> > Moving forward I put forth the following for voting:
> >
> >
> > - Next minor/major releases (such as 4.11.3.0, 4.13.0.0) will be last
> ones to support el6 packaging both for the management server and KVM host,
> but users are discouraged from using them
> >
> > - Next major release (4.13.0.0) will document in its release notes that
> we'll stop supporting centos6/rhel6 packaging in future versions, i.e. 4.14
> and onwards
> >
> > - After 4.13.0.0 is released, we will remove el6 related specs,
> packaging scripts etc. from the codebase in the master branch
> >
> >
> > [ ] +1 approve
> > [ ] +0 no opinion
> > [ ] -1 disapprove (and reason why)
> >
> >
> > ** PMCs kindly add binding to your votes, thanks.
> >
> >
> > Thanks.
> >
> >
> >
> > Regards,
> >
> > Rohit Yadav
> >
> > Software Architect, ShapeBlue
> >
> > https://www.shapeblue.com
> >
> >
> > 
> > From: Erik Weber 
> > Sent: Wednesday, April 24, 2019 19:32
> > To: dev
> > Subject: Re: [DISCUSS] Remove support for el6 packaging in 4.13/4.14
> >
> > CentOS7 was released 5 years ago, upgrading is long overdue anyway.
> > Realistically the next CloudStack release won't be out the door for
> > another ~4-6 months either.
> >
> > --
> > Erik
> >
> > On Wed, Apr 24, 2019 at 3:27 PM Ron Wheeler
> >  wrote:
> >>
> >> According to https://en.wikipedia.org/wiki/CentOS
> >>
> >> CentOS 6 EOL is 2020
> >> CentOS 7 EOL is 2024
> >>
> >>
> >> +1 for removing support for CentOS 6.
> >>
> >> As Erik pointed out the sites running CentOS6 will have to move soon in
> >> any event and it is probably better to do it now when there is still a
> >> lot of current expertise and information available about how to do it
> >> and how to make any changes to applications.
> >>
> >> Upgrading in a project that is under your control is usually easier than
> >> one forced on you by a security issue or an operational failure.
> >>
> >> Ron
> >>
> >> On 4/24/19 3:24 AM, Erik Weber wrote:
> >>> As an operations guy I can understand the want for future updates and
> >>> not upgrading, but with the release plan of RHEL/CentOS I don't find
> >>> it feasible.
> >>>
> >>> RHEL6 is 8 years old (and is still running kernel 2.6!) and isn't
> >>> scheduled to be fully EOL until 2024.
> >>>
> >>> It is true that upgrading requires some effort (and risk) from
> >>> operators, but this is work they eventually have to do anyway, so it's
> >>> not a matter of /if/ they have to do it, but rather when.
> >>>
> >>> It is also true that current CloudStack releases should continue to
> >>> work, it's also possible that someone might back port future fixes to
> >>> a RHEL6 compatible fork (you're more than welcome to).
> >>>
> >>> I'd vote +1 to remove support for el6 packaging.
> >>>
> >
> > rohit.ya...@shapeblue.com
> > www.shapeblue.com
> > Amadeus House, Floral Street, London  WC2E 9DPUK
> > @shapeblue
> >
> >
> >
> >
>

https://builds.cloudstack.org/

[Pierre-Luc Dion]

Hello community,

I was wondering if we still use jenkins service from
https://builds.cloudstack.org/ to create builds?
I have not maintain that service for quite some time so if it's not being
used I would decommission it. It was going to be a replacement of
jenkins.b.a.c.com a while back, but since then we moved into github and
since then, I kind of lost track.

I think this is still creating docker images:
* https://hub.docker.com/r/cloudstack/simulator
* https://hub.docker.com/r/cloudstack/marvin

Based on the number of pull, marvin is clearly unpopular but simulator have
~10k pulls...

Thanks,

Re: [DISCUSS] Remove support for el6 packaging in 4.13/4.14

[Pierre-Luc Dion]

I agree, it should simplify builds too.


On Mon, Apr 15, 2019 at 7:17 AM Andrija Panic 
wrote:

> +1
>
> CentOS 6 was a pain to make any newer features works, in regards to ancient
> qemu/libvirt and so on.
>
> On Mon, 15 Apr 2019 at 10:05, Wido den Hollander  wrote:
>
> >
> >
> > On 4/15/19 9:44 AM, Rohit Yadav wrote:
> > > All,
> > >
> > >
> > > With CentOS8 around the corner to be released sometime around the
> > summer, I would like to propose to deprecate CentOS6 as support
> management
> > server host distro and KVM host distro. Non-systemd enabled Ubuntu
> releases
> > have been already deprecated [1].
> > >
> > >
> > > The older CentOS6 version would hold us back as we try to adapt, use
> and
> > support newer JRE version, kvm/libvirt version, the Linux kernel, and
> > several other older dependencies. Both CentOS6 and RHEL6 have reached EOL
> > on May 10th, 2017 wrt full updates [1].
> > >
> > >
> > > If we don't have any disagreements, I propose we remove el6 packaging
> > support in the next major release - 4.13. But, if there are users and
> > organisations that will be badly impacted, let 4.13 be the last of
> releases
> > to support el6 and we definitely remove el6 support in 4.14.
> > >
> > > What are your thoughts?
> >
> > I agree! EL6 is just no longer suited for development. Good suggestion.
> >
> > Wido
> >
> > >
> > >
> > > [1] EOL date wiki reference:
> >
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Hypervisor+and+Management+Server+OS+EOL+Dates
> > >
> > >
> > >
> > > Regards,
> > >
> > > Rohit Yadav
> > >
> > > Software Architect, ShapeBlue
> > >
> > > https://www.shapeblue.com
> > >
> > > rohit.ya...@shapeblue.com
> > > www.shapeblue.com
> > > Amadeus House, Floral Street, London  WC2E 9DPUK
> > > @shapeblue
> > >
> > >
> > >
> > >
> >
>
>
> --
>
> Andrija Panić
>

Re: new contributor guide

[Pierre-Luc Dion]

Thanks Marco,

I'll look at those!

On Fri, Mar 22, 2019 at 9:31 AM Marc-Aurèle Brothier  wrote:

> Hi Pierre-Luc,
>
> You can have a look at those links for the install and development setup:
> - https://github.com/apache/cloudstack/blob/master/INSTALL.md
>
> - https://github.com/apache/cloudstack/blob/master/CONTRIBUTING.md
>
> Those should get you started. If those aren’t enough and/or you get stuck,
> come back to the community so we can update those with the missing stuff.
>
> A good first contribution candidate could be changes to those docs as it’s
> easier to get those extra missing details as a newbie to a project.
>
> Marco
>
> > On 22 Mar 2019, at 13:48, Pierre-Luc Dion  wrote:
> >
> > Hi team,
> >
> > I was wondering if someone would have an idea of which doc a new
> > contributor should look at when someone want to start contributing to
> > cloudstack? I haven't contribute in code for some time and I'm lost, and
> > while working with a university to help us on contributing to cloudstack
> I
> > realise we have no easy doc as a new dev member.
> >
> > things like,
> > * how to setup a dev station, (devcloud? which version?)
> > * how to contribute bug fix
> > * how to contribute new feature.
> >
> > If we have such info sweet, otherwise, I'm willing to update this part.
>

new contributor guide

[Pierre-Luc Dion]

Hi team,

I was wondering if someone would have an idea of which doc a new
contributor should look at when someone want to start contributing to
cloudstack? I haven't contribute in code for some time and I'm lost, and
while working with a university to help us on contributing to cloudstack I
realise we have no easy doc as a new dev member.

things like,
* how to setup a dev station, (devcloud? which version?)
* how to contribute bug fix
* how to contribute new feature.

If we have such info sweet, otherwise, I'm willing to update this part.

Re: [VOTE] Release Apache CloudStack CloudMonkey 6.0.0

[Pierre-Luc Dion]

+1

basic test on 6.0.0 but using the go based cmk for few weeks already.

On Mon, Mar 11, 2019 at 6:49 AM Andrija Panic 
wrote:

> +1
>
> Somewhat limited testing, but played with previous beta builds - and all
> minor issues are fixed in this one - no new problems found.
>
> andrija.pa...@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London  WC2E 9DPUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Dag Sonstebo 
> Sent: 11 March 2019 11:28
> To: dev@cloudstack.apache.org
> Subject: Re: [VOTE] Release Apache CloudStack CloudMonkey 6.0.0
>
> +1
>
> Tested and works well, no issues found. Navigation is greatly improved
> with options menus now more intuitive and selections with enter key allowed.
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
>
> On 11/03/2019, 09:51, "Boris Stoyanov" 
> wrote:
>
> +1
> Played with it locally, didn’t had any issues. All controls seems
> working.
>
>
> boris.stoya...@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London  WC2E 9DPUK
> @shapeblue
>
>
>
>
> dag.sonst...@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London  WC2E 9DPUK @shapeblue
>
>
>
> > On 11 Mar 2019, at 11:25, Wido den Hollander  wrote:
> >
> > +1
> >
> > Tested it locally on my laptop and does the things I do with it.
> >
> > I couldn't find anything which didn't work.
> >
> > Wido
> >
> > On 3/11/19 8:07 AM, Rohit Yadav wrote:
> >> All,
> >>
> >>
> >> Due to the lack of any testing/voting, the RC1 voting window will
> stay open indefinitely until lazy consensus/majority is met or a bug is
> reported requiring RC2. Thanks.
> >>
> >>
> >>
> >> Regards,
> >>
> >> Rohit Yadav
> >>
> >> Software Architect, ShapeBlue
> >>
> >> https://www.shapeblue.com
> >>
> >> 
> >> From: Rohit Yadav 
> >> Sent: Tuesday, March 5, 2019 5:45:01 PM
> >> To: dev@cloudstack.apache.org; us...@cloudstack.apache.org
> >> Subject: [VOTE] Release Apache CloudStack CloudMonkey 6.0.0
> >>
> >> Hi All,
> >>
> >> I've created a 6.0.0 release of CloudMonkey, with the following
> artifacts
> >> up for a vote:
> >>
> >> Git Branch and Commit SHA:
> >>
> https://github.com/apache/cloudstack-cloudmonkey/commit/74ff37cffc1d8bb3652f6887faa770d933ffe768
> >>
> >> Commit: 74ff37cffc1d8bb3652f6887faa770d933ffe768
> >>
> >> Github pre-release (for testing, contains changelog,
> artifacts/binaries to
> >> test, checksums/usage details):
> >> https://github.com/apache/cloudstack-cloudmonkey/releases/tag/6.0.0
> >>
> >> Source release (checksums and signatures are available at the same
> >> location):
> >> https://dist.apache.org/repos/dist/dev/cloudstack/cloudmonkey-6.0.0
> >>
> >> PGP release keys (signed using
> 5ED1E1122DC5E8A4A45112C2484248210EE3D884):
> >> https://dist.apache.org/repos/dist/release/cloudstack/KEYS
> >>
> >> The vote will be open for 72 hours.
> >>
> >> For sanity in tallying the vote, can PMC members please be sure to
> indicate
> >> "(binding)" with their vote?
> >> [ ] +1 approve
> >> [ ] +0 no opinion
> >> ] -1 disapprove (and the reason why)
> >>
> >> Regards,
> >> Rohit Yadav
> >>
> >> rohit.ya...@shapeblue.com
> >> www.shapeblue.com
> >> Amadeus House, Floral Street, London  WC2E 9DPUK
> >> @shapeblue
> >>
> >>
> >>
>
>
>
>

Re: Release schedule for CloudStack CloudMonkey v6.0.0

[Pierre-Luc Dion]

+1 !

https://www.shapeblue.com/whats-coming-in-the-new-cloudmonkey-6-0/


On Tue, Feb 12, 2019 at 4:33 AM Rohit Yadav 
wrote:

> All,
>
>
> I would like to invite everyone for a final round of testing before
> cloudmonkey v6.0.0 RC1 can be cut for voting.
>
> Proposed timeline:
>
>
> - Test, fix bugs, update documentation and stabilize: Until 24 Feb 2019
>
> - Start RC1: Monday 25 Feb 2019
>
>
> Kindly test the latest cloudmonkey (testing) v6.0.0-beta3:
> https://github.com/apache/cloudstack-cloudmonkey/releases
>
> And help report issues:
> https://github.com/apache/cloudstack-cloudmonkey/issues
>
>
> Recent blog: https://www.shapeblue.com/whats-coming-in-cloudmonkey-6-0/
>
>
> Thanks and regards,
>
> Rohit Yadav
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London  WC2E 9DPUK
> @shapeblue
>
>
>
>

Re: Dropping Nuage Networks support

[Pierre-Luc Dion]

Hi Kris,

Thanks a lot for your contributions, I hope you will remain close to our
community!




On Fri, Jan 25, 2019 at 3:33 PM Wei ZHOU  wrote:

> Hi Kris,
>
> It was nice to meet you some times and see your contributions to the
> project. Thank you and your colleagues for all your hard work.
>
> All the best in the future !
>
> -Wei
>
>
> Kris Sterckx  于2019年1月25日周五 下午7:14写道：
>
> > Folks,
> >
> >
> >
> > A management decision within Nuage Networks / Nokia has been made for
> > dropping CloudStack support in the upcoming release.
> >
> > With that, we have been working at a clean cut, taking out Nuage SDN
> > support from the code, as the last thing we want would be leaving
> > unsupported/broken code in the repo. Obviously, all generically
> applicable
> > contributions remain – only the Nuage specifics are taken out. Some of
> > these generic contributions include per-NIC extra DHCP options support,
> > extended Config Drive support (both discussed at the Miami CCC) and
> > Physical Network Migration (presented at the last Montreal CCC).
> >
> > The following PR has been uploaded, pending your review:
> >
> >
> > https://github.com/apache/cloudstack/pull/3146
> >
> >
> >
> > Together with Frank and Raf, I would like to thank everyone for the great
> > collaboration, the time we spent at conferences/meetups and the overall
> joy
> > we had. And I hope our ways to further cross.
> >
> > Keep up the great work.
> >
> >
> >
> > Kris
> >
>

Re: Introduction

[Pierre-Luc Dion]

Re-welcome Andrija !

On Fri, Jan 11, 2019 at 6:22 AM Dag Sonstebo 
wrote:

> Welcome again Andrija - great to have you onboard!
>
> Regards,
> Dag Sonstebo
> Cloud Architect
> ShapeBlue
>
>
> On 11/01/2019, 10:49, "Andrija Panic" 
> wrote:
>
> Hi all,
>
> I would like to take this opportunity to (re)introduce myself - some
> of you already know me from mailing list as Andrija Panic from HIAG/Safe
> Swiss Cloud.
>
> I have moved forward and joined a great team in ShapeBlue as a Cloud
> Architect and looking forward to further endeavors with CloudStack.
> FTR - I'm based in Belgrade, Serbia and been playing with CloudStack
> for last 5 years in production.
>
> Cheers,
> Andrija Panić
>
> andrija.pa...@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London  WC2E 9DPUK
> @shapeblue
>
>
>
>
>
>
> dag.sonst...@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London  WC2E 9DPUK
> @shapeblue
>
>
>
>

-- 

*Pierre-Luc Dion*Lead Cloud Architect | Architecte infonuagique principal
t 1.888.796.8364 ext. 1403


<https://cloud.ca/?utm_source=email_medium=signature_content=cloud-ca-logo-1_campaign=general_email>

Re: Quetion about API 'revertSnapshot'

[Pierre-Luc Dion]

Hi,

I think it work on XenServer too the revert if a VM snapshot, at least, it
used to work. but I'm not sure if this API is for revert disk snapshot or
vm snapshot.


On Fri, Dec 21, 2018 at 2:52 AM Haijiao <18602198...@163.com> wrote:

> Hi，All
>
> Maybe it’s a stupid question, but from the CloudStack API docs,  the API
> ‘revertSnapshot’ is ONLY supported with KVM, but not others, e.g. XenServer
> or VMware .
>
> Is there any special reason for this ?  Shall we expect this feature
> included in coming release ACS ?
>
> http://cloudstack.apache.org/api/apidocs-4.11/apis/revertSnapshot.html
>
> Thanks!



-- 

*Pierre-Luc Dion*Lead Cloud Architect | Architecte infonuagique principal
t 1.888.796.8364 ext. 1403


<https://cloud.ca/?utm_source=email_medium=signature_content=cloud-ca-logo-1_campaign=general_email>

Re: SSL offload in the VR

[Pierre-Luc Dion]

Hi Paul, ok I was not aware of that new feature. Is it wanted to support lb
on sourceNAT IP has in your screenshot?


Le mer. 21 nov. 2018 03 h 27, Paul Angus  a
écrit :

> Thanks @Pierre-Luc Dion and @jaya...@apache.org,
>
> My 'problem' is that being able to add an SSL cert to a VR loadbalancer
> has appeared from nowhere in the 4.11 branch. (easiest explained with a
> screen snip):
>
> https://imgur.com/a/r7zxA0b
>
> ... which led me to finding the assignCertToLoadBalancer API.
>
> I can't find a specific PR for it, or anything in the cwiki, and nothing
> in our read-the-docs...
> So I'm trying to find the author, so we can back fit some documentation,
> rather than trying to reverse engineer it out of the code/trial and error.
>
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London  WC2E 9DPUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Jayapal Uradi 
> Sent: 21 November 2018 04:31
> To: dev@cloudstack.apache.org
> Subject: Re: SSL offload in the VR
>
> What I remember is ssl offload feature  added for netscaler  and
> assignCertToLoadBalancer API added during that time.
>
> -Jayapal
>
> > On 21-Nov-2018, at 5:06 AM, Pierre-Luc Dion  wrote:
> >
> > Hey Paul, I'm not aware the vr support ssl offload. What you are
> > trying to achieve, there might be a work around.
> >
> > Le mar. 20 nov. 2018 07 h 08, Paul Angus  a
> > écrit :
> >
> >> Apologies if my GitHub foo has abandoned me (it wouldn’t me the first
> >> time);
> >>
> >> I searched for files which contained the API command
> >> ‘assignCertToLoadBalancer’ and found ‘AssignCertToLoadBalancerCmd.java’
> >>
> >> According to the history it was first committed as part of PR #2283
> >> “CLOUDSTACK-10105: Use maven standard project structure in all
> >> projects” [1]
> >>
> >>
> >> https://github.com/apache/cloudstack/commits/1d05fead49f5c856257a741b
> >> 07122f5633d2e359/api/src/main/java/org/apache/cloudstack/api/command/
> >> user/loadbalancer/AssignCertToLoadBalancerCmd.java
> >>
> >> again, sorry if I’ve misread or misinterpreted the history…
> >>
> >> Kind regards,
> >>
> >> Paul Angus
> >>
> >> From: Marc-Aurèle Brothier 
> >> Sent: 20 November 2018 11:23
> >> To: Paul Angus 
> >> Cc: dev@cloudstack.apache.org
> >> Subject: Re: SSL offload in the VR
> >>
> >> Hi Paul,
> >>
> >> What made you think so? I haven’t pushed any change related to the
> >> VR. At Exoscale, we removed the VR entirely instead and moved the
> >> services down at the HV level.
> >>
> >> Kind regards,
> >> Marc-Aurèle
> >>
> >>
> >> paul.an...@shapeblue.com
> >> www.shapeblue.com
> >> Amadeus House, Floral Street, London  WC2E 9DPUK @shapeblue
> >>
> >>
> >>
> >> On 20 Nov 2018, at 09:04, Paul Angus  >> paul.an...@shapeblue.com>> wrote:
> >> Hi Marc-Aurèle,
> >>
> >> As far as I can tell, I believe that you added the ability to upload
> >> an SSL certificate to the VR in 4.11 I can’t find any user
> >> documentation for it in our Wiki or the in read-the-docs.
> >> I guess it’s something that you use at Exoscale, could you do a pull
> >> request to cloudstack-documentation or forward me some documentation
> >> which I can then add to the users documentation.
> >>
> >>
> >> Kind regards,
> >>
> >> Paul Angus
> >>
> >>
> >> paul.an...@shapeblue.com<mailto:paul.an...@shapeblue.com>
> >> www.shapeblue.com<http://www.shapeblue.com>
> >> @shapeblue
> >>
> >>
> >>
> >>
>
> DISCLAIMER
> ==
> This e-mail may contain privileged and confidential information which is
> the property of Accelerite, a Persistent Systems business. It is intended
> only for the use of the individual or entity to which it is addressed. If
> you are not the intended recipient, you are not authorized to read, retain,
> copy, print, distribute or use this message. If you have received this
> communication in error, please notify the sender and delete all copies of
> this message. Accelerite, a Persistent Systems business does not accept any
> liability for virus infected mails.
>

Re: SSL offload in the VR

[Pierre-Luc Dion]

Hey Paul, I'm not aware the vr support ssl offload. What you are trying to
achieve, there might be a work around.

Le mar. 20 nov. 2018 07 h 08, Paul Angus  a
écrit :

> Apologies if my GitHub foo has abandoned me (it wouldn’t me the first
> time);
>
> I searched for files which contained the API command
> ‘assignCertToLoadBalancer’ and found ‘AssignCertToLoadBalancerCmd.java’
>
> According to the history it was first committed as part of PR #2283
> “CLOUDSTACK-10105: Use maven standard project structure in all projects” [1]
>
>
> https://github.com/apache/cloudstack/commits/1d05fead49f5c856257a741b07122f5633d2e359/api/src/main/java/org/apache/cloudstack/api/command/user/loadbalancer/AssignCertToLoadBalancerCmd.java
>
> again, sorry if I’ve misread or misinterpreted the history…
>
> Kind regards,
>
> Paul Angus
>
> From: Marc-Aurèle Brothier 
> Sent: 20 November 2018 11:23
> To: Paul Angus 
> Cc: dev@cloudstack.apache.org
> Subject: Re: SSL offload in the VR
>
> Hi Paul,
>
> What made you think so? I haven’t pushed any change related to the VR. At
> Exoscale, we removed the VR entirely instead and moved the services down at
> the HV level.
>
> Kind regards,
> Marc-Aurèle
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> Amadeus House, Floral Street, London  WC2E 9DPUK
> @shapeblue
>
>
>
> On 20 Nov 2018, at 09:04, Paul Angus  paul.an...@shapeblue.com>> wrote:
> Hi Marc-Aurèle,
>
> As far as I can tell, I believe that you added the ability to upload an
> SSL certificate to the VR in 4.11
> I can’t find any user documentation for it in our Wiki or the in
> read-the-docs.
> I guess it’s something that you use at Exoscale, could you do a pull
> request to cloudstack-documentation or forward me some documentation which
> I can then add to the users documentation.
>
>
> Kind regards,
>
> Paul Angus
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> @shapeblue
>
>
>
>

Re: Deploying VM with custom CPU and RAM

[Pierre-Luc Dion]

with cloudmonkey, it look like this:
details[0].cpuNumber=2 details[0].cpuSpeed=1000 details[0].memory=1024
rootdisksize=15

PL


On Sun, Sep 9, 2018 at 8:10 AM Fariborz Navidan 
wrote:

> Thank you, I just cheked apilog.log and found the parameter names.
>
> On Sun, Sep 9, 2018 at 2:41 PM Ivan Kudryavtsev 
> wrote:
>
> > Hi, why don't you just track RESTful API call sent thru API in Cloudstack
> > UI. Just click F12 in Chrome and trace the call.
> >
> > вс, 9 сент. 2018 г., 16:57 Fariborz Navidan :
> >
> > > Hello,
> > >
> > > I have created a customizable service offering in cloudstack and want
> to
> > > deploy VM via deployVirtualMachine() API call. If I specify id of
> custom
> > > service offering, I will need to specify cpu cores and memory, It is
> not
> > > documented how should I fill those parameters I have tried
> > > details[0].cpunumber=[0].cpuspeed=1000 still I get Invalid cpu
> > > cores value. How do I specify cpu cores?
> > >
> > > Thanks in advance.
> > >
> >
>

Re: [DISCUSS] deployment planner improvement

[Pierre-Luc Dion]

#scopecreep Paul ;)

But I think the problem you identify is related to selection of the
deployment planner strategy, globally define or at the compute offering.
You can select how cloudstack choose the host to deploy a new vm.

But even then, like marcus stated, if you add a node to a full cluster, all
new vm will be created on that node.

So if nobody have WIP around post deployment orchestration, I'll work on
the feature spec with the university, with objective in mind to easy
hypervisor maintenances, better distribution of workload.

I would not expect PR before ~6months, but will have some actions around it
very soon I hope.



Le ven. 7 sept. 2018 09 h 05, Marc-Andre Jutras  a
écrit :

> I agree, it is affecting all hypervisor... I basically had to migrate a
> bunch of vm manually to re-balance a cluster after an upgrade or even
> after re-adding new host to a cluster.
>
> Personally, I think Cloudstack should be able to handles this  balancing> of resource, example: having a piece of code somewhere that
> can run every hours or on demand to re-calculate and re-balance
> resources across hosts within a cluster...
>
> Even the deployment planner is not really relevant here: this process
> will basically balance new VM creation through different clusters of a
> POD, not between hosts within a cluster and it's also becoming a
> nightmare when you start to do cross-cluster migration...
>
> Sum of all : The deployment strategies planner should be re-worked a bit...
>
> +1 on #scoopcreep ;)
>
> Marcus ( mjut...@cloudops.com )
>
> On 2018-09-07 6:01 AM, Paul Angus wrote:
> > I think that this affects all hypervisors as CloudStack's deployment
> strategies are generally sub-optimal to say the least.
> >  From what our devs have told me, a large part of the problem is that
> capacity/usage and suitability due to tags is calculated by multiple parts
> of the code independently, there is no central method, which will give a
> consistent answer.
> >
> > In Trillian we take a micro-management approach and have a custom module
> which will return the least used cluster, the least used host or the least
> used host in a given cluster.  With that info we place VMs on a specific
> hosts - keeping virtualised hypervisors in the same cluster (least used) so
> that processor types match, and all other VMs on the least used hosts.
> >
> > For cross-cluster migrations (VMs and/or storage) I think that most
> times people want to move from cluster A to the least used
> (cluster/storage) in cluster B - making them choose which host/pool is
> actually unhelpful.
> >
> > #scopecreep - sorry Pierre-Luc
> >
> > Kind regards,
> >
> > Paul Angus
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > Amadeus House, Floral Street, London  WC2E 9DPUK
> > @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: Will Stevens 
> > Sent: 06 September 2018 19:45
> > To: dev@cloudstack.apache.org; Marc-Andre Jutras 
> > Subject: Re: [DISCUSS] deployment planner improvement
> >
> > If I remember correctly, we see similar issues on VMware.  Marcus, have
> you seen similar behavior on VMware?  I think I remember us having to
> manually vMotion a lot of VMs very often...
> >
> > *Will Stevens*
> > Chief Technology Officer
> > c 514.826.0190
> >
> > <https://goo.gl/NYZ8KK>
> >
> >
> > On Thu, Sep 6, 2018 at 2:34 PM Pierre-Luc Dion 
> wrote:
> >
> >> Hi,
> >>
> >> I'm working with a University in Montreal and we are looking at
> >> working together to improve the deployment planner. Mainly for post
> >> VM.CREATE tasks.
> >> Because  what we observed with cloudstack, in our case with XenServer,
> >> overtime, a cluster will become unbalanced in therm of workload, vm HA
> >> will move VMs all over the the cluster which cause hotspot inside a
> cluster.
> >> Also, when performing maintenance  xenmotion of VM spread them in the
> >> cluster but does not consider host usage and at the end of a
> >> maintenance it require manual operation to repopulate VMs on the last
> >> host updated.  OS preference not taken into account  except for
> VM.CREATE.
> >>
> >> So,
> >> I'd like to work on improving VMs dispersion during and post outage
> >> and maintenances. when a cluster resources are added or removed.
> >>
> >> Would you have any more requirement, we will document a feature spec
> >> in the wiki which I believe it's still a requirement ?
> >>
> >> Does using KVM have similar issues over time?
> >>
> >> I don't think it would make sense to cloudstack to automatically take
> >> decision on moving VMs but for now create report of recommended action
> >> to do and provide steps to do them. tbd.
> >>
> >> Cheers,
> >>
> >> PL
> >>
>
>

Re: [DISCUSS] Removing IAM services

[Pierre-Luc Dion]

No chances it's in used with Roles stuff ?




On Wed, Aug 22, 2018 at 8:19 AM Gabriel Beims Bräscher 
wrote:

> I am +1 on removing it as well.
>
> Em qua, 22 de ago de 2018 às 07:49, Rohit Yadav  >
> escreveu:
>
> > +1 it's about time
> >
> >
> >
> > - Rohit
> >
> > 
> >
> >
> >
> > 
> > From: Daan Hoogland 
> > Sent: Tuesday, August 21, 2018 9:02:58 PM
> > To: dev
> > Cc: users
> > Subject: [DISCUSS] Removing IAM services
> >
> > i'm +1 on this it has not been developed for the longest while. (changed
> > the title to indicate _all_ people are supposed to have an opinion.)
> >
> > On Tue, Aug 21, 2018 at 4:32 PM, Khosrow Moossavi <
> kmooss...@cloudops.com>
> > wrote:
> >
> > > Hello Community
> > >
> > > Following up after PR#2613[1] to cleanup POMs across the whole
> > repository,
> > > now in a new
> > > PR #2817 we are going to remove services/iam projects which seems to be
> > > disabled and
> > > not maintained since May 2014.
> > >
> > > Moving forward with the PR, corresponding tables will be deleted from
> > > database too.
> > >
> > > Please let us know if you have any questions, comments, concerns about
> > this
> > > removal or if
> > > you are willing to revive the project in some shape or form.
> > >
> > > [1]: https://github.com/apache/cloudstack/pull/2613
> > > [2]: https://github.com/apache/cloudstack/pull/2817
> > >
> > > Khosrow Moossavi
> > >
> > > Cloud Infrastructure Developer
> > >
> > > 
> > >
> >
> >
> >
> > --
> > Daan
> >
> > rohit.ya...@shapeblue.com
> > www.shapeblue.com
> > Amadeus House, Floral Street, London  WC2E 9DPUK
> > @shapeblue
> >
> >
> >
> >
>

[DISCUSS] deployment planner improvement

[Pierre-Luc Dion]

Hi,

I'm working with a University in Montreal and we are looking at working
together to improve the deployment planner. Mainly for post VM.CREATE tasks.
Because  what we observed with cloudstack, in our case with XenServer,
overtime, a cluster will become unbalanced in therm of workload, vm HA will
move VMs all over the the cluster which cause hotspot inside a cluster.
Also, when performing maintenance  xenmotion of VM spread them in the
cluster but does not consider host usage and at the end of a maintenance it
require manual operation to repopulate VMs on the last host updated.  OS
preference not taken into account  except for VM.CREATE.

So,
I'd like to work on improving VMs dispersion during and post outage and
maintenances. when a cluster resources are added or removed.

Would you have any more requirement, we will document a feature spec in the
wiki which I believe it's still a requirement ?

Does using KVM have similar issues over time?

I don't think it would make sense to cloudstack to automatically take
decision on moving VMs but for now create report of recommended action to
do and provide steps to do them. tbd.

Cheers,

PL

Re: Infra Shutdown (Jenkins?)

[Pierre-Luc Dion]

Hi Rohit, Will,

I have a backup of the jenkins.xml but it's from a while back, it
shouldn't  have changed much since recent work was done on
builds.cloudstack.org and github if not mistaking.
We also provide an alternative: https://builds.cloudstack.org/ hosted on
cloud.ca



On Wed, Jun 27, 2018 at 11:35 AM Rohit Yadav 
wrote:

> Will, you may take the xml jenkings config backup of the jenkins box if
> possible.
>
> Regards.
>
> Get Outlook for Android
>
> 
> From: Will Stevens 
> Sent: Wednesday, June 27, 2018 7:38:48 PM
> To: dev@cloudstack.apache.org
> Subject: Re: Infra Shutdown (Jenkins?)
>
> Hey All,
> I am guessing since I have not heard anyone scream that this infra was not
> used.  I will be blowing these resources away by the end of the week if no
> one is using these resources.
>
> Cheers,
>
> *Will Stevens*
> Chief Technology Officer
> c 514.826.0190
>
> 
>
>
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> On Fri, Jun 22, 2018 at 8:40 AM Will Stevens 
> wrote:
>
> > Hey Everyone,
> > So CloudOps took over paying for a Citrix AWS account a couple years ago
> > which apparently has VMs which is supporting the ACS community.  This
> > account has been costing us about $500/month, so we want to make sure the
> > VMs are actually used.
> >
> > I am shutting down the following VMs and if things stop working, please
> > let me know...
> >
> > - jenkins.cloudstack.org | IP: 75.101.146.23 | KeyPair: davidnalley
> > - deb build large | IP: 54.242.16.44 | KeyPair: edison
> > -  | IP: 54.90.178.215 | KeyPair: davidnalley
> > - PEOPLE AND FPASTE | IP: 107.21.237.59 | KeyPair: davidnalley
> >
> > If any of these VMs mean anything to you, please reach out to me.  I
> can't
> > justify paying for them if I don't know they are used.
> >
> > Thanks,
> >
> > *Will Stevens*
> > Chief Technology Officer
> > c 514.826.0190
> >
> > 
> >
>

I'd like to introduce you to Khosrow

[Pierre-Luc Dion]

Hi fellow colleagues,

I might be a bit late with this email...

I'd like to introduce Khosrow Moossavi, who recently join our team and his
focus is currently exclusively on dev for Cloudstack with cloud.ca.

Our 2 current priorities are:
-fixing VRs,SVMs to run has HVM VMs in xenserver.
- redesign, or rewrite, the remote management vpn for vpc, poc in progress
for IKEv2...



Some of you might have interact with him already.


Also, we are going to be more active for the upcomming 4.12 release.


Cheers!

Re: Copy Volume Failed in CloudStack 4.5 (XenServer 6.5)

[Pierre-Luc Dion]

I think there is a timout global settings you could change so the copy task
will take longer before it timeout and fail in cloudstack. This will not
improve your performance but might reduce failure.

On updating the database content, it could work, but only if the vhd
successfully copy, and mappings remain valid.

I hope this can help...



Le 6 févr. 2018 13 h 28, "anillakieni"  a
écrit :

Dear All,

Is somebody available here to assist me on fixing my issue.

Thanks,
Anil.

On Tue, Feb 6, 2018 at 9:00 PM, anillakieni  wrote:

> Hi All,
>
> I'm facing issue when copying  larger size volumes. i.e., Secondary
> Storage to Primary Storage (I mean attaching DATA volume to VM), after
> certain time around 37670 seconds.
>
> Version of:
> - CloudStack is 4.5.0
> - XenServer 6.5.0
> - MySQL 5.1.73
>
>
> The error and log is provided below, Could someone please assist me here
> which steps i have to take to fix this issue. Also, can we have a chance
to
> update the failed status to success through database tables because i have
> to upload the whole disk again to secondary storage and then later attach
> it to VM, which is consuming more time. My environment has very slow
> network transfers (I have only 1 Gig switch). Please let me know if we can
> tweak the DB to update the status of the disk or do we have any settings
to
> be changed to accept more time (wait time) for updating the status.
> "
>
> 2018-02-06 03:20:42,385 DEBUG [c.c.a.t.Request] (Work-Job-Executor-31:ctx-
c1c78a5a
> job-106186/job-106187 ctx-ea1ef3e6) (logid:c59b2359) Seq
> 38-367887794560851961: Received:  { Ans: , MgmtId: 47019105324719, via:
38,
> Ver: v1, Flags: 110, { CopyCmdAnswer } }
> 2018-02-06 03:20:42,389 DEBUG [o.a.c.s.v.VolumeObject]
> (Work-Job-Executor-31:ctx-c1c78a5a job-106186/job-106187 ctx-ea1ef3e6)
> (logid:c59b2359) *Failed to update state*
> *com.cloud.utils.exception.CloudRuntimeException: DB Exception on:
> com.mysql.jdbc.JDBC4PreparedStatement@54bd3a25: SELECT volume_store_ref.id
> , volume_store_ref.store_id,
> volume_store_ref.volume_id, volume_store_ref.zone_id,
> volume_store_ref.created, volume_store_ref.last_updated,
> volume_store_ref.download_pct, volume_store_ref.size,
> volume_store_ref.physical_size, volume_store_ref.download_state,
> volume_store_ref.checksum, volume_store_ref.local_path,
> volume_store_ref.error_str, volume_store_ref.job_id,
> volume_store_ref.install_path, volume_store_ref.url,
> volume_store_ref.download_url, volume_store_ref.download_url_created,
> volume_store_ref.destroyed, volume_store_ref.update_count,
> volume_store_ref.updated, volume_store_ref.state, volume_store_ref.ref_cnt
> FROM volume_store_ref WHERE volume_store_ref.store_id = 1  AND
> volume_store_ref.volume_id = 1178  AND volume_store_ref.destroyed = 0
> ORDER BY RAND() LIMIT 1*
> at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(
> GenericDaoBase.java:425)
> at com.cloud.utils.db.GenericDaoBase.searchIncludingRemoved(
> GenericDaoBase.java:361)
> at com.cloud.utils.db.GenericDaoBase.findOneIncludingRemovedBy(
> GenericDaoBase.java:889)
> at com.cloud.utils.db.GenericDaoBase.findOneBy(
> GenericDaoBase.java:900)
> at org.apache.cloudstack.storage.image.db.VolumeDataStoreDaoImpl.
> findByStoreVolume(VolumeDataStoreDaoImpl.java:209)
> at sun.reflect.GeneratedMethodAccessor306.invoke(Unknown Source)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at org.springframework.aop.support.AopUtils.
> invokeJoinpointUsingReflection(AopUtils.java:317)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.
> invokeJoinpoint(ReflectiveMethodInvocation.java:183)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.
> proceed(ReflectiveMethodInvocation.java:150)
> at com.cloud.utils.db.TransactionContextInterceptor.invoke(
> TransactionContextInterceptor.java:34)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.
> proceed(ReflectiveMethodInvocation.java:161)
> at org.springframework.aop.interceptor.
> ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:91)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.
> proceed(ReflectiveMethodInvocation.java:172)
> at org.springframework.aop.framework.JdkDynamicAopProxy.
> invoke(JdkDynamicAopProxy.java:204)
> at com.sun.proxy.$Proxy173.findByStoreVolume(Unknown Source)
> at org.apache.cloudstack.storage.datastore.
> ObjectInDataStoreManagerImpl.findObject(ObjectInDataStoreManagerImpl.
> java:353)
> at org.apache.cloudstack.storage.datastore.
> ObjectInDataStoreManagerImpl.findObject(ObjectInDataStoreManagerImpl.
> java:338)
> at 

Re: [DISCUSS] running sVM and VR as HVM on XenServer

[Pierre-Luc Dion]

Hi Tim,

As long as it work, since it's only used to for the initial instruction set
at the VR boot so eth0 can be configure, I think xenstore would work just
fine.
unless you are saying we could just not rely on xenstore in terms of
reliability?


*Pierre-Luc DION*
Architecte de Solution Cloud | Cloud Solutions Architect
t 855.652.5683

*CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Fri, Jan 12, 2018 at 7:34 PM, Tim Mackey <tmac...@gmail.com> wrote:

> > We found that we can use xenstore-read / xenstore-write to send data from
> dom0 to domU which are in our case  VRs or SVMs. Any reason not using this
> approach ?
>
> xenstore has had some issues in the past. The most notable of which were
> limitations on the number of event channels in use, followed by overall
> performance impact. iirc, the event channel stuff was fully resolved with
> XenServer 6.5, but they do speak to a need to test if there are any changes
> to the maximum number of VMs which can be reliably supported. It also
> limits legacy support (in case that matters).
>
> Architecturally I think this is a reasonable approach to the problem. One
> other thing to note is that xapi replicates xenstore information to all
> members of a pool. That might impact RVRs.
>
> -tim
>
> [1] "xenstore is not a high-performance facility and should beused only for
> small amounts of control plane data."
> https://xenbits.xen.org/docs/4.6-testing/misc/xenstore.txt
>
> On Fri, Jan 12, 2018 at 4:56 PM, Pierre-Luc Dion <pd...@cloudops.com>
> wrote:
>
> > After some verification with Syed and Khosrow,
> >
> > We found that we can use xenstore-read / xenstore-write to send data from
> > dom0 to domU which are in our case  VRs or SVMs. Any reason not using
> this
> > approach ?  that way we would not need a architectural change for
> XenServer
> > pods, and this would support HVM and PV virtual-router. more test
> required,
> > for sure, VR would need to have xentools pre-installed.
> >
> >
> > *Pierre-Luc DION*
> > Architecte de Solution Cloud | Cloud Solutions Architect
> > t 855.652.5683
> >
> > *CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
> > 420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
> > w cloudops.com *|* tw @CloudOps_
> >
> > On Fri, Jan 12, 2018 at 4:07 PM, Syed Ahmed <sah...@cloudops.com> wrote:
> >
> > > KVM uses a VirtIO channel to send information about the IP address and
> > > other params to the SystemVMs. We could use a similar strategy in
> > XenServer
> > > using XenStore. This would involve minimal changes to the code while
> > > keeping backward compatibility.
> > >
> > >
> > >
> > > On Fri, Jan 12, 2018 at 3:07 PM, Simon Weller <swel...@ena.com.invalid
> >
> > > wrote:
> > >
> > > > They do not. They receive a link-local ip address that is used for
> host
> > > > agent to VR communication. All VR commands are proxied through the
> host
> > > > agent. Host agent to VR communication is over SSH.
> > > >
> > > >
> > > > 
> > > > From: Rafael Weingärtner <rafaelweingart...@gmail.com>
> > > > Sent: Friday, January 12, 2018 1:42 PM
> > > > To: dev
> > > > Subject: Re: [DISCUSS] running sVM and VR as HVM on XenServer
> > > >
> > > > but we are already using this design in vmware deployments (not sure
> > > about
> > > > KVM). The management network is already an isolated network only used
> > by
> > > > system vms and ACS. Unless we are attacked by some internal agent, we
> > are
> > > > safe from customer attack through management networks. Also, we can
> (if
> > > we
> > > > don't do yet) restrict access only via these management interfaces in
> > > > system VMs(VRs, SSVM, console proxy and others to come).
> > > >
> > > >
> > > >
> > > > Can someone confirm if VRs receive management IPs in KVM deployments?
> > > >
> > > > On Fri, Jan 12, 2018 at 5:36 PM, Syed Ahmed <sah...@cloudops.com>
> > wrote:
> > > >
> > > > > The reason why we used link local in the first place was to isolate
> > the
> > > > VR
> > > > > from directly accessing the management network. This provides
> another
> > > > layer
> > > > > of security in case of a VR e

Re: [DISCUSS] running sVM and VR as HVM on XenServer

[Pierre-Luc Dion]

After some verification with Syed and Khosrow,

We found that we can use xenstore-read / xenstore-write to send data from
dom0 to domU which are in our case  VRs or SVMs. Any reason not using this
approach ?  that way we would not need a architectural change for XenServer
pods, and this would support HVM and PV virtual-router. more test required,
for sure, VR would need to have xentools pre-installed.


*Pierre-Luc DION*
Architecte de Solution Cloud | Cloud Solutions Architect
t 855.652.5683

*CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Fri, Jan 12, 2018 at 4:07 PM, Syed Ahmed <sah...@cloudops.com> wrote:

> KVM uses a VirtIO channel to send information about the IP address and
> other params to the SystemVMs. We could use a similar strategy in XenServer
> using XenStore. This would involve minimal changes to the code while
> keeping backward compatibility.
>
>
>
> On Fri, Jan 12, 2018 at 3:07 PM, Simon Weller <swel...@ena.com.invalid>
> wrote:
>
> > They do not. They receive a link-local ip address that is used for host
> > agent to VR communication. All VR commands are proxied through the host
> > agent. Host agent to VR communication is over SSH.
> >
> >
> > 
> > From: Rafael Weingärtner <rafaelweingart...@gmail.com>
> > Sent: Friday, January 12, 2018 1:42 PM
> > To: dev
> > Subject: Re: [DISCUSS] running sVM and VR as HVM on XenServer
> >
> > but we are already using this design in vmware deployments (not sure
> about
> > KVM). The management network is already an isolated network only used by
> > system vms and ACS. Unless we are attacked by some internal agent, we are
> > safe from customer attack through management networks. Also, we can (if
> we
> > don't do yet) restrict access only via these management interfaces in
> > system VMs(VRs, SSVM, console proxy and others to come).
> >
> >
> >
> > Can someone confirm if VRs receive management IPs in KVM deployments?
> >
> > On Fri, Jan 12, 2018 at 5:36 PM, Syed Ahmed <sah...@cloudops.com> wrote:
> >
> > > The reason why we used link local in the first place was to isolate the
> > VR
> > > from directly accessing the management network. This provides another
> > layer
> > > of security in case of a VR exploit. This will also have a side effect
> of
> > > making all VRs visible to each other. Are we okay accepting this?
> > >
> > > Thanks,
> > > -Syed
> > >
> > > On Fri, Jan 12, 2018 at 11:37 AM, Tim Mackey <tmac...@gmail.com>
> wrote:
> > >
> > > > dom0 already has a DHCP server listening for requests on internal
> > > > management networks. I'd be wary trying to manage it from an external
> > > > service like cloudstack lest it get reset upon XenServer patch. This
> > > alone
> > > > makes me favor option #2. I also think option #2 simplifies network
> > > design
> > > > for users.
> > > >
> > > > Agreed on making this as consistent across flows as possible.
> > > >
> > > >
> > > >
> > > > On Fri, Jan 12, 2018 at 9:44 AM, Rafael Weingärtner <
> > > > rafaelweingart...@gmail.com> wrote:
> > > >
> > > > > It looks reasonable to manage VRs via management IP network. We
> > should
> > > > > focus on using the same work flow for different deployment
> scenarios.
> > > > >
> > > > >
> > > > > On Fri, Jan 12, 2018 at 12:13 PM, Pierre-Luc Dion <
> > pd...@cloudops.com>
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > We need to start a architecture discussion about running SystemVM
> > and
> > > > > > Virtual-Router as HVM instances in XenServer. With recent
> > > > > Meltdown-Spectre,
> > > > > > one of the mitigation step is currently to run VMs as HVM on
> > > XenServer
> > > > to
> > > > > > self contain a user space attack from a guest OS.
> > > > > >
> > > > > > Recent hotfix from Citrix XenServer (XS71ECU1009) enforce VMs to
> > > start
> > > > > has
> > > > > > HVM. This is currently problematic for Virtual Routers and
> SystemVM
> > > > > because
> > > > > > CloudStack use PV "OS boot Options" to preconfigure t

[DISCUSS] running sVM and VR as HVM on XenServer

[Pierre-Luc Dion]

Hi,

We need to start a architecture discussion about running SystemVM and
Virtual-Router as HVM instances in XenServer. With recent Meltdown-Spectre,
one of the mitigation step is currently to run VMs as HVM on XenServer to
self contain a user space attack from a guest OS.

Recent hotfix from Citrix XenServer (XS71ECU1009) enforce VMs to start has
HVM. This is currently problematic for Virtual Routers and SystemVM because
CloudStack use PV "OS boot Options" to preconfigure the VR eth0:
cloud_link_local. While using HVM the "OS boot Options" is not accessible
to the VM so the VR fail to be properly configured.

I currently see 2 potential approaches for this:
1. Run a dhcpserver in dom0 managed by cloudstack so VR eth0 would receive
is network configuration at boot.
2. Change the current way of managing VR, SVMs on XenServer, potentiall do
same has with VMware: use pod management networks and assign a POD IP to
each VR.

I don't know how it's implemented in KVM, maybe cloning KVM approach would
work too, could someone explain how it work on this thread?

I'd a bit fan of a potential #2 aproach because it could facilitate VR
monitoring and logging, although a migration path for an existing cloud
could be complex.

Cheers,


Pierre-Luc

Re: [PROPOSE] EOL for supported OSes & Hypervisors

[Pierre-Luc Dion]

+1!

Do you think it would be the right page to also have debian version used by
the ssvm?

For the management-server section the cloudstack column would list the last
acs version tested on that OS?


Le 11 janv. 2018 12 h 53, "Will Stevens"  a écrit :

> I like this initiative.  I think this would be valuable to set an
> expectation around supportability.
>
> Cheers,
>
> *Will Stevens*
> CTO
>
> 
>
> On Thu, Jan 11, 2018 at 12:11 PM, Paul Angus 
> wrote:
>
> > I've cross-posted this as it ultimately effects users more than
> developers.
> >
> > I've created a wiki page with the EOL dates from the respective 'vendors'
> > of our main supported hypervisors and mgmt. server OSes.
> > I've taken End Of Life to be the end of 'mainstream' support i.e. the
> > point at which updates to packages will no longer be available.  And part
> > of the discussion should be whether this EOL date should be moved out to
> > consider end of security patching instead.
> >
> > https://cwiki.apache.org/confluence/display/CLOUDSTACK/
> > WIP+-+UNOFFICIAL+-+PROPOSAL+-+EOL+Dates
> >
> > I would like to propose, that as part of the release notes for the
> > forthcoming 4.11 release and as a general announcement, that we declare
> > that:
> >
> >
> >   *   For any OSes/Hypervisors that are already EOL - we will no longer
> > test/support them from the first release after June 2018 (6 months from
> > now). And they will be removed from codebase (mainly the database) in the
> > first release after Sept 2018 (9 months from now).
> >   *   We set End Of Support dates and Removal from Code dates for the
> > remaining OSes/Hypervisors.  I propose that End Of Support should be the
> > first release after EOL from the vendor, with code removal taking place
> in
> > the first release which occurs after 6 months from 'vendor' EOL date.
> >
> > Thoughts please
> >
> >
> > Kind regards,
> >
> > Paul Angus
> >
> >
> > paul.an...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
>

Re: [XenServer] meltdown-spectre

[Pierre-Luc Dion]

Hi,

Another problem,  VR are configured via eth0 (cloudlinklocal) that is
create via PV params of the VM on XenServer. So Creation HVM VR will not
work. We will have to think about an alternative way to setup eth0 on VR on
XenServer.
We will file a jira issue about this. This will be problematic with XS
7,1cu1, 7.2, 7.3

Tim, yes fro XS template, the maping, at least on Xs7.1 seams to work just
fine. They were pretty much all as HVM exept Centos 6, Debian,.. until the
Meltdown hotfix.




On Mon, Jan 8, 2018 at 10:47 PM, Tim Mackey <tmac...@gmail.com> wrote:

> PLD,
>
> One thing to add to your testing is template management. When I was doing
> all the Packer stuff with XS 6.5 and 7, ACS needed to know if the template
> was PV or HVM to provision properly. No idea if the ACS template logic has
> changed since then, but something to be aware of.
>
> From a performance perspective with XS 6.5 started having newer XS
> templates follow an HVM model. e.g. CentOS 7 on XS 6.5 was HVM not PV.
> iirc, the performance difference was negligible on reasonably new CPUs
> (Sandy Bridge+ I think).
>
> -tim
>
> On Mon, Jan 8, 2018 at 9:39 PM, Ivan Kudryavtsev <kudryavtsev...@bw-sw.com
> >
> wrote:
>
> > Hi. Every kind of virtualization is affected according to qemu
> developers.
> >
> > 9 янв. 2018 г. 9:32 пользователь "Pierre-Luc Dion" <pd...@cloudops.com>
> > написал:
> >
> > >  Hi,
> > >
> > > From recent blog post, I've read that system using full virtualization
> > such
> > > as KVM, VMware or Xen-HVM are not affected?  Anyhow, from the latest
> > hotfix
> > > of XenServer 7.1cu1 hf8, it look like they systematically convert VM
> from
> > > PV to HVM, so in the case of a VM stop/start by CloudStack, a PV vm
> would
> > > be restarted as HVM.
> > >
> > > Look like this could be problematic if your VM kernel does not support
> > > both, we've just starting tested and so far look like our Debian
> systemvm
> > > template work fine, it can be created as HVM.
> > >
> > > Another point is that Citrix released an hotfix for xs7.2, 7.3 but not
> > for
> > > 7.1, you need to cumulative update to remain on 7.1 which is LTS.
> > >
> > > And last, does anyone did some benchmark before and after the kernel
> fix
> > > for Meltdown ?  Some report state 30-35% cpu usage increase (not
> > hypervisor
> > > specific) and  Lucian [1] might indicate it would depend on the cpu
> > model.
> > > Any metrics to share ?  We are doing some tests on our side we should
> be
> > > able to share some stuff soon...
> > >
> > > Regards,
> > >
> > > [1] http://markmail.org/thread/wkzze3n24mns274x
> > >
> >
>

[XenServer] meltdown-spectre

[Pierre-Luc Dion]

 Hi,

>From recent blog post, I've read that system using full virtualization such
as KVM, VMware or Xen-HVM are not affected?  Anyhow, from the latest hotfix
of XenServer 7.1cu1 hf8, it look like they systematically convert VM from
PV to HVM, so in the case of a VM stop/start by CloudStack, a PV vm would
be restarted as HVM.

Look like this could be problematic if your VM kernel does not support
both, we've just starting tested and so far look like our Debian systemvm
template work fine, it can be created as HVM.

Another point is that Citrix released an hotfix for xs7.2, 7.3 but not for
7.1, you need to cumulative update to remain on 7.1 which is LTS.

And last, does anyone did some benchmark before and after the kernel fix
for Meltdown ?  Some report state 30-35% cpu usage increase (not hypervisor
specific) and  Lucian [1] might indicate it would depend on the cpu model.
Any metrics to share ?  We are doing some tests on our side we should be
able to share some stuff soon...

Regards,

[1] http://markmail.org/thread/wkzze3n24mns274x

Re: Performance considerations related to Intel Meltdown on KVM CPU types

[Pierre-Luc Dion]

Thanks for sharing this Nux!

PL

On Mon, Jan 8, 2018 at 10:11 AM, Nux!  wrote:

> Hello,
>
> Just stumbled upon this
> https://twitter.com/berrange/status/950209752486817792
>
> "ensure KVM guest CPU model you choose has the "pcid" feature, otherwise
> guests will suffer terrible performance from the Meltdown fixes. This means
> using a named Haswell, Broadwell or Skylake based model or host passthrough"
>
>
> This means whoever is running with the KVM default CPU (like I do) as
> opposed to specific ones or host passthrough needs to change this in order
> to avoid bad performance once the new mitigating kernel is installed.
>
>
> Bad news is older Xeons do not support this, check if "invpcid" flag shows
> up in /proc/cpuinfo (you might see "pcid", that one is not enough).
>
>
>
>
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>

Re: Release Notes

[Pierre-Luc Dion]

The way I was doing RN and generating  fixed issues and known issues list,
was by doing JIRA filters for each of them, then you use "./utils/jira.py"
from cloudstack-docs-rn
to generate a markdown formated list that you cut/paste in relevant RN
page.  This require Jira issues to be updated on Open/close  state, also
the Fix Version and Affected to be fully up to date. With recent use of
Gitbhub, I'm not sure how Jira is up to date now.  Will did use something
else to extract data from commit comments, but I don't know how he did it.

Hope this help a little...


Cheers,




On Thu, Dec 21, 2017 at 12:38 PM, Paul Angus 
wrote:

>
> Hi guys I'm hoping to get a head start on release notes over Xmas. I
> believe some guys have come up with a tool which gets human readable what's
> new and known issues...
>
> Could someone point me in the right direction please.
>
> (And merry Christmas and a happy new year to everyone)
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>

Re: [VR] dnsmasq reload instead of restart

[Pierre-Luc Dion]

good point !

Based on what I understand from dnsmasq, reload will not update conf for
the interface of a new network tier.



On Fri, Nov 24, 2017 at 6:44 PM, Erik Weber <terbol...@gmail.com> wrote:

> I'm not sure if this applies to dnsmasq or not, but some services does
> not bind to new interfaces during a reload.
> Meaning you'll have to be sure the reason you're reloading isn't
> because of something that added a device (say adding a new tier or
> similar).
>
> --
> Erik
>
> On Fri, Nov 24, 2017 at 9:05 PM, Pierre-Luc Dion <pd...@cloudops.com>
> wrote:
> > Hi,
> >
> > We recently found a way to reload dnsmasq process instead of doing a
> > restart.
> > Does anyone see a problem with that?  Basically we have to change where
> the
> > file /etc/dhcphosts.txt is define in the config so it would be define via
> > "--dhcp-hostsfile=/etc/dhcphosts.txt" when the process is start so it
> would
> > be re-read in a kill -HUP.
> >
> > The problem to solve here is that during VM.CREATE in a VPC, the dns
> > service is restarted and cause DNS service interruption to VM inside the
> > VPC.
> >
> > We are preparing a PR for this.
> >
> > is this could be problematic for rVR ?
> >
> > Thanks,
> >
> > PL
>

[VR] dnsmasq reload instead of restart

[Pierre-Luc Dion]

Hi,

We recently found a way to reload dnsmasq process instead of doing a
restart.
Does anyone see a problem with that?  Basically we have to change where the
file /etc/dhcphosts.txt is define in the config so it would be define via
"--dhcp-hostsfile=/etc/dhcphosts.txt" when the process is start so it would
be re-read in a kill -HUP.

The problem to solve here is that during VM.CREATE in a VPC, the dns
service is restarted and cause DNS service interruption to VM inside the
VPC.

We are preparing a PR for this.

is this could be problematic for rVR ?

Thanks,

PL

Re: Fail with vpn customer gateway creation through terraform

[Pierre-Luc Dion]

Hi Nux,

Could it be your cloudstack version ?  modp3072 is recent I think in
CloudStack so if you run a older version maybe it's not there?



On Tue, Nov 21, 2017 at 6:55 PM, Nux!  wrote:

> Thanks Chiradeep,
>
> Checked but brain says no. What should I have learned from there?
>
> AFAIK this is a terraform fail.
>
> Lucian
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> - Original Message -
> > From: "Chiradeep Vittal" 
> > To: "dev" 
> > Sent: Tuesday, 21 November, 2017 19:14:16
> > Subject: Re: Fail with vpn customer gateway creation through terraform
>
> > Check
> > https://github.com/apache/cloudstack/blob/77864992fe8f80dbabd1240f6373d2
> ba3e98713c/utils/src/main/java/com/cloud/utils/net/NetUtils.java#L1221
> >
> > On Tue, Nov 21, 2017 at 10:11 AM, Nux!  wrote:
> >
> >> Hi,
> >>
> >> I'm trying out terraform and had success so far, except for the vpn
> >> customer gateway feature.
> >> For some reason, terraform fails to create it, though I use the same
> >> options as in UI/cloudmonkey where it works just fine.
> >>
> >> The snippet for it is:
> >>
> >> resource "cloudstack_vpn_customer_gateway" "default" {
> >>   name   = "test-vpc"
> >>   cidr   = "10.0.0.0/24"
> >>   esp_policy = "aes256-sha1"
> >>   gateway= "1.2.3.4"
> >>   ike_policy = "sha1-aes256;modp3072"
> >>   ipsec_psk  = "terraformxyz7"
> >> }
> >>
> >> It always complains about the ike_policy:
> >> * cloudstack_vpn_customer_gateway.default: Error creating VPN Customer
> >> Gateway test-vpc: Undefined error: {"errorcode":431,"errortext":"The
> >> customer gateway IKE policy sha1-aes256;modp3072 is invalid!  Verify the
> >> required Diffie Hellman (DH) group is specified."}
> >>
> >> I tried all sorts of ways to write the ike_policy, escaped, web
> >> encoded/decoded, nothing worked. What am I missing?
> >> The example terraform docs provide suffers the same fate.
> >>
> >> Lucian
> >>
> >> --
> >> Sent from the Delta quadrant using Borg technology!
> >>
> >> Nux!
> >> www.nux.ro
>

Re: HTTPS LB and x-forwarded-for

[Pierre-Luc Dion]

That's funny!

Cloudstack ui does not provide lb protocol options, but the api does and
cloudstack already support proxy proto v1!!!

So that's cool!

Le 13 nov. 2017 09 h 18, "Wido den Hollander" <w...@widodh.nl> a écrit :

>
> > Op 13 november 2017 om 15:14 schreef Pierre-Luc Dion <pd...@cloudops.com
> >:
> >
> >
> > Hi,
> >
> > This is looking quite promising, I have a colleague that tested that last
> > Friday, look like the proxy proto v1 work out of the box with Nginx, but
> > would need an extra package for Apache 2.4 ?
>
> It depends. You need HTTPd 2.4.28, see: https://httpd.apache.org/docs/
> trunk/mod/mod_remoteip.html#remoteipproxyprotocol
>
> It's there upstream, but not in all packages.
>
> It can from this module:
>
> - https://github.com/roadrunner2/mod-proxy-protocol
> - https://roadrunner2.github.io/mod-proxy-protocol/mod_proxy_protocol.html
>
> They donated the code to go upstream and went into mod_remoteip but landed
> in 2.4.28
>
> It will probably make it into Ubuntu 18.04 and CentOS 7.4.
>
> Wido
>
> > Otherwise, it seems to be a good way to do  https LB without complicated
> > configuration and huge changes in CloudStack.
> >
> >
> >
> > On Fri, Nov 10, 2017 at 10:36 AM, Nux! <n...@li.nux.ro> wrote:
> >
> > > Pierre-Luc,
> > >
> > > Haproxy docs say it should work for any kind of traffic as long as both
> > > ends are PROXY-aware and it look like a majority of software is.
> > > So, in short, yes.
> > >
> > > --
> > > Sent from the Delta quadrant using Borg technology!
> > >
> > > Nux!
> > > www.nux.ro
> > >
> > > - Original Message -
> > > > From: "Pierre-Luc Dion" <pd...@cloudops.com>
> > > > To: "Wido den Hollander" <w...@widodh.nl>
> > > > Cc: "dev" <dev@cloudstack.apache.org>, "Khosrow Moossavi" <
> > > kmooss...@cloudops.com>, "Will Stevens"
> > > > <wstev...@cloudops.com>, "Nux!" <n...@li.nux.ro>, "users" <
> > > us...@cloudstack.apache.org>
> > > > Sent: Friday, 10 November, 2017 15:32:38
> > > > Subject: Re: HTTPS LB and x-forwarded-for
> > >
> > > > Hi Wido, do you know if this would work for https traffic too?
> > > >
> > > > Le 10 nov. 2017 09 h 35, "Wido den Hollander" <w...@widodh.nl> a
> écrit :
> > > >
> > > >>
> > > >> > Op 10 november 2017 om 14:27 schreef Pierre-Luc Dion <
> > > pd...@cloudops.com
> > > >> >:
> > > >> >
> > > >> >
> > > >> > I kind of like the proxy backend type, ill check on our end if
> that
> > > would
> > > >> > work but definitely a simple and efficient approach!
> > > >> >
> > > >>
> > > >> See: https://www.haproxy.com/blog/haproxy/proxy-protocol/
> > > >>
> > > >> Apache HTTPd supports PROXY since 2.4.28:
> > > https://httpd.apache.org/docs/
> > > >> trunk/mod/mod_remoteip.html#remoteipproxyprotocol
> > > >>
> > > >> "RemoteIPProxyProtocol is only available in httpd 2.4.28 and newer"
> > > >>
> > > >> Wido
> > > >>
> > > >> >
> > > >> >
> > > >> > Le 10 nov. 2017 01 h 44, "Wido den Hollander" <w...@widodh.nl> a
> > > écrit :
> > > >> >
> > > >> > >
> > > >> > > > Op 9 november 2017 om 19:59 schreef Nux! <n...@li.nux.ro>:
> > > >> > > >
> > > >> > > >
> > > >> > > > Wido,
> > > >> > > >
> > > >> > > > Excellent suggestion with the "transparent proxy", I was not
> > > aware of
> > > >> > > that.
> > > >> > > > I think that would be a great idea and wouldn't require too
> many
> > > >> > > modifications, especially as Haproxy comes already with the VR.
> > > >> > > >
> > > >> > >
> > > >> > > It's indeed just a matter of a HAProxy config setting. We could
> > > make it
> > > >> > > configurable per backend in HAProxy. Regular HTTP, TCP or PROXY
> for
> > > >

Re: HTTPS LB and x-forwarded-for

[Pierre-Luc Dion]

Hi,

This is looking quite promising, I have a colleague that tested that last
Friday, look like the proxy proto v1 work out of the box with Nginx, but
would need an extra package for Apache 2.4 ?
Otherwise, it seems to be a good way to do  https LB without complicated
configuration and huge changes in CloudStack.



On Fri, Nov 10, 2017 at 10:36 AM, Nux! <n...@li.nux.ro> wrote:

> Pierre-Luc,
>
> Haproxy docs say it should work for any kind of traffic as long as both
> ends are PROXY-aware and it look like a majority of software is.
> So, in short, yes.
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> ----- Original Message -
> > From: "Pierre-Luc Dion" <pd...@cloudops.com>
> > To: "Wido den Hollander" <w...@widodh.nl>
> > Cc: "dev" <dev@cloudstack.apache.org>, "Khosrow Moossavi" <
> kmooss...@cloudops.com>, "Will Stevens"
> > <wstev...@cloudops.com>, "Nux!" <n...@li.nux.ro>, "users" <
> us...@cloudstack.apache.org>
> > Sent: Friday, 10 November, 2017 15:32:38
> > Subject: Re: HTTPS LB and x-forwarded-for
>
> > Hi Wido, do you know if this would work for https traffic too?
> >
> > Le 10 nov. 2017 09 h 35, "Wido den Hollander" <w...@widodh.nl> a écrit :
> >
> >>
> >> > Op 10 november 2017 om 14:27 schreef Pierre-Luc Dion <
> pd...@cloudops.com
> >> >:
> >> >
> >> >
> >> > I kind of like the proxy backend type, ill check on our end if that
> would
> >> > work but definitely a simple and efficient approach!
> >> >
> >>
> >> See: https://www.haproxy.com/blog/haproxy/proxy-protocol/
> >>
> >> Apache HTTPd supports PROXY since 2.4.28:
> https://httpd.apache.org/docs/
> >> trunk/mod/mod_remoteip.html#remoteipproxyprotocol
> >>
> >> "RemoteIPProxyProtocol is only available in httpd 2.4.28 and newer"
> >>
> >> Wido
> >>
> >> >
> >> >
> >> > Le 10 nov. 2017 01 h 44, "Wido den Hollander" <w...@widodh.nl> a
> écrit :
> >> >
> >> > >
> >> > > > Op 9 november 2017 om 19:59 schreef Nux! <n...@li.nux.ro>:
> >> > > >
> >> > > >
> >> > > > Wido,
> >> > > >
> >> > > > Excellent suggestion with the "transparent proxy", I was not
> aware of
> >> > > that.
> >> > > > I think that would be a great idea and wouldn't require too many
> >> > > modifications, especially as Haproxy comes already with the VR.
> >> > > >
> >> > >
> >> > > It's indeed just a matter of a HAProxy config setting. We could
> make it
> >> > > configurable per backend in HAProxy. Regular HTTP, TCP or PROXY for
> >> example.
> >> > >
> >> > > That way your problem would be solved.
> >> > >
> >> > > Wido
> >> > >
> >> > > > To Paul:
> >> > > > - imho the LB solution ACS ships now is a bit handicaped since
> you do
> >> > > not know the remote host ip. You're flying blind unless you use
> google
> >> > > analytics (and these things have gotten more and more aggressively
> >> filtered
> >> > > by adblocks).
> >> > > > Enhancing Haproxy as Wido suggested would go a long way, it
> wouldn't
> >> > > break existing functionality and would also keep SSL processing off
> >> the VR.
> >> > > >
> >> > > > --
> >> > > > Sent from the Delta quadrant using Borg technology!
> >> > > >
> >> > > > Nux!
> >> > > > www.nux.ro
> >> > > >
> >> > > > - Original Message -
> >> > > > > From: "Andrija Panic" <andrija.pa...@gmail.com>
> >> > > > > To: "users" <us...@cloudstack.apache.org>
> >> > > > > Cc: "Khosrow Moossavi" <kmooss...@cloudops.com>, "Will
> Stevens" <
> >> > > wstev...@cloudops.com>, "dev"
> >> > > > > <dev@cloudstack.apache.org>, "Pierre-Luc Dion" <
> pd...@cloudops.com
> >> >
> >> > > > > Sent: Thursday, 9 November, 2017 13:10:58
> >> > > > > 

Re: HTTPS LB and x-forwarded-for

[Pierre-Luc Dion]

Hi Wido, do you know if this would work for https traffic too?

Le 10 nov. 2017 09 h 35, "Wido den Hollander" <w...@widodh.nl> a écrit :

>
> > Op 10 november 2017 om 14:27 schreef Pierre-Luc Dion <pd...@cloudops.com
> >:
> >
> >
> > I kind of like the proxy backend type, ill check on our end if that would
> > work but definitely a simple and efficient approach!
> >
>
> See: https://www.haproxy.com/blog/haproxy/proxy-protocol/
>
> Apache HTTPd supports PROXY since 2.4.28: https://httpd.apache.org/docs/
> trunk/mod/mod_remoteip.html#remoteipproxyprotocol
>
> "RemoteIPProxyProtocol is only available in httpd 2.4.28 and newer"
>
> Wido
>
> >
> >
> > Le 10 nov. 2017 01 h 44, "Wido den Hollander" <w...@widodh.nl> a écrit :
> >
> > >
> > > > Op 9 november 2017 om 19:59 schreef Nux! <n...@li.nux.ro>:
> > > >
> > > >
> > > > Wido,
> > > >
> > > > Excellent suggestion with the "transparent proxy", I was not aware of
> > > that.
> > > > I think that would be a great idea and wouldn't require too many
> > > modifications, especially as Haproxy comes already with the VR.
> > > >
> > >
> > > It's indeed just a matter of a HAProxy config setting. We could make it
> > > configurable per backend in HAProxy. Regular HTTP, TCP or PROXY for
> example.
> > >
> > > That way your problem would be solved.
> > >
> > > Wido
> > >
> > > > To Paul:
> > > > - imho the LB solution ACS ships now is a bit handicaped since you do
> > > not know the remote host ip. You're flying blind unless you use google
> > > analytics (and these things have gotten more and more aggressively
> filtered
> > > by adblocks).
> > > > Enhancing Haproxy as Wido suggested would go a long way, it wouldn't
> > > break existing functionality and would also keep SSL processing off
> the VR.
> > > >
> > > > --
> > > > Sent from the Delta quadrant using Borg technology!
> > > >
> > > > Nux!
> > > > www.nux.ro
> > > >
> > > > - Original Message -
> > > > > From: "Andrija Panic" <andrija.pa...@gmail.com>
> > > > > To: "users" <us...@cloudstack.apache.org>
> > > > > Cc: "Khosrow Moossavi" <kmooss...@cloudops.com>, "Will Stevens" <
> > > wstev...@cloudops.com>, "dev"
> > > > > <dev@cloudstack.apache.org>, "Pierre-Luc Dion" <pd...@cloudops.com
> >
> > > > > Sent: Thursday, 9 November, 2017 13:10:58
> > > > > Subject: Re: HTTPS LB and x-forwarded-for
> > > >
> > > > > Wido,
> > > > >
> > > > > backend servers are not Linux only, for example we have a ton of
> > > Windows
> > > > > customers, some WEB solutions / IIS etc...
> > > > >
> > > > > @all - If we try to please/solve everyone's proxying
> > > solution/requirement -
> > > > > this is impossible IMHO - I'm thinking more about some "do it as
> you
> > > like"
> > > > > solution, to let customer write his own haproxy config and upoad it
> > > (for
> > > > > example, or something better?).
> > > > >
> > > > > We can support newer version of haproxy (1.5+) which also implement
> > > > > "transarent proxy" (integrate with kernel so to speak)  to allow
> > > TCP-level
> > > > > connections to backend (TCP mode, not HTTP mode) but to still
> > > "preserve"
> > > > > remote IP by faking it (fake soruce IP = transarent proxy).
> > > > >
> > > > > For the rest of configuration options,  I would leave it to the
> > > customer
> > > > > how he/she wants to configure rest of haproxy configuration,
> inlcuding
> > > > > custom checks, etc. Haproxy configuration is never-ending story,
> and we
> > > > > probably should allow custom sripts/configuration instead of
> trying to
> > > > > provide GUI/API way to configure everything (which is
> impossible...)
> > > > >
> > > > > Just my 2 cents...
> > > > >
> > > > > On 9 November 2017 at 08:13, Wido den Hollander <w...@widodh.nl>
> > > wrote:
> > > > >
> 

Re: HTTPS LB and x-forwarded-for

[Pierre-Luc Dion]

I kind of like the proxy backend type, ill check on our end if that would
work but definitely a simple and efficient approach!



Le 10 nov. 2017 01 h 44, "Wido den Hollander" <w...@widodh.nl> a écrit :

>
> > Op 9 november 2017 om 19:59 schreef Nux! <n...@li.nux.ro>:
> >
> >
> > Wido,
> >
> > Excellent suggestion with the "transparent proxy", I was not aware of
> that.
> > I think that would be a great idea and wouldn't require too many
> modifications, especially as Haproxy comes already with the VR.
> >
>
> It's indeed just a matter of a HAProxy config setting. We could make it
> configurable per backend in HAProxy. Regular HTTP, TCP or PROXY for example.
>
> That way your problem would be solved.
>
> Wido
>
> > To Paul:
> > - imho the LB solution ACS ships now is a bit handicaped since you do
> not know the remote host ip. You're flying blind unless you use google
> analytics (and these things have gotten more and more aggressively filtered
> by adblocks).
> > Enhancing Haproxy as Wido suggested would go a long way, it wouldn't
> break existing functionality and would also keep SSL processing off the VR.
> >
> > --
> > Sent from the Delta quadrant using Borg technology!
> >
> > Nux!
> > www.nux.ro
> >
> > - Original Message -
> > > From: "Andrija Panic" <andrija.pa...@gmail.com>
> > > To: "users" <us...@cloudstack.apache.org>
> > > Cc: "Khosrow Moossavi" <kmooss...@cloudops.com>, "Will Stevens" <
> wstev...@cloudops.com>, "dev"
> > > <dev@cloudstack.apache.org>, "Pierre-Luc Dion" <pd...@cloudops.com>
> > > Sent: Thursday, 9 November, 2017 13:10:58
> > > Subject: Re: HTTPS LB and x-forwarded-for
> >
> > > Wido,
> > >
> > > backend servers are not Linux only, for example we have a ton of
> Windows
> > > customers, some WEB solutions / IIS etc...
> > >
> > > @all - If we try to please/solve everyone's proxying
> solution/requirement -
> > > this is impossible IMHO - I'm thinking more about some "do it as you
> like"
> > > solution, to let customer write his own haproxy config and upoad it
> (for
> > > example, or something better?).
> > >
> > > We can support newer version of haproxy (1.5+) which also implement
> > > "transarent proxy" (integrate with kernel so to speak)  to allow
> TCP-level
> > > connections to backend (TCP mode, not HTTP mode) but to still
> "preserve"
> > > remote IP by faking it (fake soruce IP = transarent proxy).
> > >
> > > For the rest of configuration options,  I would leave it to the
> customer
> > > how he/she wants to configure rest of haproxy configuration, inlcuding
> > > custom checks, etc. Haproxy configuration is never-ending story, and we
> > > probably should allow custom sripts/configuration instead of trying to
> > > provide GUI/API way to configure everything (which is impossible...)
> > >
> > > Just my 2 cents...
> > >
> > > On 9 November 2017 at 08:13, Wido den Hollander <w...@widodh.nl>
> wrote:
> > >
> > >>
> > >> > Op 8 november 2017 om 14:59 schreef Pierre-Luc Dion <
> pd...@cloudops.com
> > >> >:
> > >> >
> > >> >
> > >> > Same challenge here too!
> > >> >
> > >> > Let's look at improving Load-balancing offering from cloudstack, I
> guest
> > >> we
> > >> > should do a feature spec draft soon..,  from my perspective, doing
> SSL
> > >> > offload on the VR could be problematic if the VR spec if too small,
> and
> > >> the
> > >> > default spec of the VR being 1vcpu@256MB, considering it can be the
> > >> router
> > >> > of a VPC, doing VPN termination, adding HTTPS  is a bit ish... What
> would
> > >> > be your thought about this ?
> > >> >
> > >> > I'd be curious to have a LB offering in ACS where it would deploy a
> > >> > redundant traefik[1] beside the VR for doing http and https
> > >> Load-balancing.
> > >> > I think it would also be useful if the API of that traefik instance
> would
> > >> > be available from within the VPC or LBnetwork so is API would be
> > >> accessible
> > >> > to other apps orchestration tools such as  kubernetes or rancher.
> > >&g

Re: HTTPS LB and x-forwarded-for

[Pierre-Luc Dion]

Same challenge here too!

Let's look at improving Load-balancing offering from cloudstack, I guest we
should do a feature spec draft soon..,  from my perspective, doing SSL
offload on the VR could be problematic if the VR spec if too small, and the
default spec of the VR being 1vcpu@256MB, considering it can be the router
of a VPC, doing VPN termination, adding HTTPS  is a bit ish... What would
be your thought about this ?

I'd be curious to have a LB offering in ACS where it would deploy a
redundant traefik[1] beside the VR for doing http and https Load-balancing.
I think it would also be useful if the API of that traefik instance would
be available from within the VPC or LBnetwork so is API would be accessible
to other apps orchestration tools such as  kubernetes or rancher.

traefik or not, here is what I think is needed by cloudstack in the LB
improvement:

- support http, https (X-Forwarded-For)
- basic persistence tuning (API already exist)
- better backend monitoring, currently only a tcp connect validate if the
webserver is up.
- ssl offload
- metric collection, more stats, maybe just export the tool status page to
the private network.
- Container world support, right now if you have Rancher or kubernetes
cluster, you need to deploy your own LB solution behing mostlikely a static
nat., If cloudstack would deploy a traefik instance, Kub or Rancher could
reuse this instance and managed it to properly do LB between containers.


What would be your prefered LB tool:
haproxy, traefik or nginx?

CloudStack already have to code to handle SSL certs per projects and
accounts if not mistaking because that code was added to support NetScaler
as Load-balancer in the past. so one less thing to think about :-)


[1] https://traefik.io/


PL,



On Mon, Nov 6, 2017 at 7:10 AM, Nux!  wrote:

> Thanks Andrija,
>
> LB outside of the VR sounds like a good idea. An appliance based on, say
> cloud-init + ansible and so on could do the trick; alas it'd need to be
> outside ACS.
> I guess as users we could maybe come up with a spec for an improvement, at
> least we'd have something the devs could look at whenever it is possible.
>
> Regards,
> Lucian
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> - Original Message -
> > From: "Andrija Panic" 
> > To: "dev" 
> > Cc: "users" 
> > Sent: Thursday, 2 November, 2017 23:21:37
> > Subject: Re: HTTPS LB and x-forwarded-for
>
> > We used to make some special stuff for one of the clients, where all LB
> > configuration work is done from outside of the ACS, i.e. python script to
> > feed/configure VR - install latest haproxy 1.5.x for transparent proxy,
> > since client insisted on SSL termination done on backend web SSL
> servers
> > Not good idea, that is all I can say (custom configuration thing) - but
> the
> > LB setup is actually good - transparent mode haproxy, works on TCP level,
> > so you can see "real client IP" on the backend servers (which must use VR
> > as the default gtw, as per default, so the whole setup works properly).
> >
> > I'm still looking forward to see some special support of LB inside VR via
> > ACS - proper LB setup inside VR via GUI/API -  i.e. to enable LB
> > provisioning SCRIPT (bash, or whatever),  where all needed
> > install+configure can be done from client side  - otherwise covering all
> > user cases, with proper HTTP checks and similaris impossible to do
> > IMHO.
> >
> > Some other clients, actually have internal FW appliance (i.e. multihomed
> > VM, acting as gtw for all VMs in all networks), and haproxy instaled on
> > this device (with NAT configured from VR to this internal FW/VM, so
> remote
> > IP can be seen properly) - this setup is fully under customer control,
> and
> > can provide any kind of special haproxy config...
> >
> >
> >
> >
> >
> >
> > On 31 October 2017 at 19:54, Nux!  wrote:
> >
> >> Hello,
> >>
> >> Of the people running an LB (VR) with https backends, how do you deal
> with
> >> the lack of x-forwarded-for since for port 443 there's just simple TCP
> >> balancing?
> >>
> >> Has anyone thought of terminating SSL in the VR instead? Ideas?
> >>
> >> Cheers
> >>
> >> --
> >> Sent from the Delta quadrant using Borg technology!
> >>
> >> Nux!
> >> www.nux.ro
> >>
> >
> >
> >
> > --
> >
> > Andrija Panić
>

Re: Release packages for 4.9.3.0

[Pierre-Luc Dion]

I've updated jenkins and built packages for centos6 and Ubuntu. Packages
have been copied on cloudstack.apt-get.eu automatically. I haven't tested
them yet.


Le 28 sept. 2017 20 h 19, "Pierre-Luc Dion" <pdion...@apache.org> a écrit :

I'll work something up this weekend...

we need some work on 4.10 I think...

On Thu, Sep 28, 2017 at 3:14 AM, Wido den Hollander <w...@widodh.nl> wrote:

>
> > Op 27 september 2017 om 8:30 schreef Özhan Rüzgar Karaman <
> oruzgarkara...@gmail.com>:
> >
> >
> > Hi Wido;
> > I checked http://cloudstack.apt-get.eu/ web site and 4.9.3 packages are
> > only under /ubuntu/dists/xenial/4.9/pool/ directory, for trusty there are
> > no packages available for 4.9.3 .
> >
> > This directory(/ubuntu/dists/xenial/4.9/pool/) also have 4.10 packages
> as
> > well, which i think they should not be there.
> >
> > When you have suitable time could you check the packages and its script.
> >
>
> Let me check that! Something must have gone wrong with the build and
> upload system. I'll check.
>
> Wido
>
> > Thanks
> > Özhan
> >
> > On Mon, Sep 25, 2017 at 7:24 AM, Rohit Yadav <rohit.ya...@shapeblue.com>
> > wrote:
> >
> > > Thanks Wido, can you help building and uploading of rpms as well. Maybe
> > > Pierre-Luc can help?
> > >
> > >
> > > - Rohit
> > >
> > > 
> > > From: Wido den Hollander <w...@widodh.nl>
> > > Sent: Thursday, September 21, 2017 1:09:49 PM
> > > To: Rohit Yadav; dev@cloudstack.apache.org; Pierre-Luc Dion
> > > Subject: Re: Release packages for 4.9.3.0
> > >
> > > Ah, sorry! The DEB packages should have been uploaded already :-)
> > >
> > > Wido
> > >
> > > > Op 21 september 2017 om 7:33 schreef Rohit Yadav <
> > > rohit.ya...@shapeblue.com>:
> > > >
> > > >
> > > > Ping - Wido/PL?
> > > >
> > > >
> > > > - Rohit
> > > >
> > > > 
> > > > From: Rohit Yadav <rohit.ya...@shapeblue.com>
> > > > Sent: Tuesday, September 12, 2017 5:44:36 PM
> > > > To: Wido den Hollander; Pierre-Luc Dion
> > > > Cc: dev@cloudstack.apache.org
> > > > Subject: Release packages for 4.9.3.0
> > > >
> > > > Wido/PL/others,
> > > >
> > > >
> > > > Can you please help with building and publishing of 4.9.3.0 rpms/deb
> > > packages on the download.cloudstack.org repository? I've built and
> > > published the repos on packages.shapeblue.com now (
> shapeblue.com/packages
> > > for details).
> > > >
> > > >
> > > > Regards.
> > > >
> > > >
> > > > rohit.ya...@shapeblue.com
> > > > www.shapeblue.com<http://www.shapeblue.com>
> > > > 53 Chandos Place, Covent Garden, London  WC2N
> <https://maps.google.com/?q=53+Chandos+Place,+Covent+Garden,+London%C2%A0+WC2N=gmail=g>
> 4HSUK
> > > > @shapeblue
> > > >
> > > >
> > > >
> > > >
> > > > rohit.ya...@shapeblue.com
> > > > www.shapeblue.com<http://www.shapeblue.com>
> > > > 53 Chandos Place, Covent Garden, London  WC2N
> <https://maps.google.com/?q=53+Chandos+Place,+Covent+Garden,+London%C2%A0+WC2N=gmail=g>
> 4HSUK
> > > > @shapeblue
> > > >
> > > >
> > > >
> > >
> > > rohit.ya...@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London  WC2N
> <https://maps.google.com/?q=53+Chandos+Place,+Covent+Garden,+London%C2%A0+WC2N=gmail=g>
> 4HSUK
> > > @shapeblue
> > >
> > >
> > >
> > >
>

Re: Release packages for 4.9.3.0

[Pierre-Luc Dion]

I'll work something up this weekend...

we need some work on 4.10 I think...

On Thu, Sep 28, 2017 at 3:14 AM, Wido den Hollander <w...@widodh.nl> wrote:

>
> > Op 27 september 2017 om 8:30 schreef Özhan Rüzgar Karaman <
> oruzgarkara...@gmail.com>:
> >
> >
> > Hi Wido;
> > I checked http://cloudstack.apt-get.eu/ web site and 4.9.3 packages are
> > only under /ubuntu/dists/xenial/4.9/pool/ directory, for trusty there are
> > no packages available for 4.9.3 .
> >
> > This directory(/ubuntu/dists/xenial/4.9/pool/) also have 4.10 packages
> as
> > well, which i think they should not be there.
> >
> > When you have suitable time could you check the packages and its script.
> >
>
> Let me check that! Something must have gone wrong with the build and
> upload system. I'll check.
>
> Wido
>
> > Thanks
> > Özhan
> >
> > On Mon, Sep 25, 2017 at 7:24 AM, Rohit Yadav <rohit.ya...@shapeblue.com>
> > wrote:
> >
> > > Thanks Wido, can you help building and uploading of rpms as well. Maybe
> > > Pierre-Luc can help?
> > >
> > >
> > > - Rohit
> > >
> > > 
> > > From: Wido den Hollander <w...@widodh.nl>
> > > Sent: Thursday, September 21, 2017 1:09:49 PM
> > > To: Rohit Yadav; dev@cloudstack.apache.org; Pierre-Luc Dion
> > > Subject: Re: Release packages for 4.9.3.0
> > >
> > > Ah, sorry! The DEB packages should have been uploaded already :-)
> > >
> > > Wido
> > >
> > > > Op 21 september 2017 om 7:33 schreef Rohit Yadav <
> > > rohit.ya...@shapeblue.com>:
> > > >
> > > >
> > > > Ping - Wido/PL?
> > > >
> > > >
> > > > - Rohit
> > > >
> > > > 
> > > > From: Rohit Yadav <rohit.ya...@shapeblue.com>
> > > > Sent: Tuesday, September 12, 2017 5:44:36 PM
> > > > To: Wido den Hollander; Pierre-Luc Dion
> > > > Cc: dev@cloudstack.apache.org
> > > > Subject: Release packages for 4.9.3.0
> > > >
> > > > Wido/PL/others,
> > > >
> > > >
> > > > Can you please help with building and publishing of 4.9.3.0 rpms/deb
> > > packages on the download.cloudstack.org repository? I've built and
> > > published the repos on packages.shapeblue.com now (
> shapeblue.com/packages
> > > for details).
> > > >
> > > >
> > > > Regards.
> > > >
> > > >
> > > > rohit.ya...@shapeblue.com
> > > > www.shapeblue.com<http://www.shapeblue.com>
> > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > > > @shapeblue
> > > >
> > > >
> > > >
> > > >
> > > > rohit.ya...@shapeblue.com
> > > > www.shapeblue.com<http://www.shapeblue.com>
> > > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > > > @shapeblue
> > > >
> > > >
> > > >
> > >
> > > rohit.ya...@shapeblue.com
> > > www.shapeblue.com
> > > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > > @shapeblue
> > >
> > >
> > >
> > >
>

Re: Cluster anti-affinity

[Pierre-Luc Dion]

Hi Ivan,
I don't think cloudstack offer cluster anti affinity and i'm sure i would
be in favor of introducing another anti affinity level, because it would
expose cluster notion to your cloud user.

Although, look at the vm provisionning strategy config, their should be a
deployment strategy that would spread account vms across pods or clusters,
or prefer pod/cluster proximity. I think this could help you.

Regards,

Le 19 sept. 2017 01 h 57, "Ivan Kudryavtsev"  a
écrit :

> Hello, community. Right now cloudstack has affinity implementation for host
> anti-affinity and it's great and useful, but since the storage is often
> defined for a cluster (unless it's local or clustered like Ceph), it
> defined a failure domain. Does anybody experienced "cluster anti-affinity"
> implementation. Is it useful or were declined in the past by dev team? Any
> thoughts?
>
> How you tackle with VMs which should be completely independent and fault
> tolerant with shared storage (not Ceph)? I see that zone-level approach
> works for sure, but if the requirement is for intra-zone, I don't see the
> way to implement it, any thoughts?
>
> --
> With best regards, Ivan Kudryavtsev
> Bitworks Software, Ltd.
> Cell: +7-923-414-1515
> WWW: http://bitworks.software/ 
>

Re: cloud-set-guest-password working with systemctl

[Pierre-Luc Dion]

Hi,

On my side we use cloud-init in all our Linux templates. We got it to work
for password, password-reset, sshkey and sshkey-reset. It also support
user-data, and provide an easy way to manage a different user than root.

I think our community documentation need an update on this topic...

Cheers,

Le 28 août 2017 5:29 PM, "Sebastian Gomez" <tioc...@gmail.com> a écrit :

I had the same problem like Marc and I could not resolve it on Ubuntu 16.04.
I followed the cloudstack steps to set it up, but the script is not running
well on our environment.

If I execute the script from command line, it works fine. During the boot
process, there is any problem on systemd and DHCP conjunction, and VR is
not reachable to reply this request.

On the other hand, I found many different versions of this OLD script, and
I remember that we had to use other community version also for Ubuntu
14.04. I noticed that this is a very old script, and has not been modified
for years.
As you proposed, I'm trying to do it via cloud-init. Just landed from
holidays I will finish this integration.

To do a so simply task like setting up a simple template is becoming a
nightmare, so *I would like to know how people *is setting Ubuntu templates
up?
- SSL key pairs? would be a good way, but Cloudstack 4.9.2 does not allow
to manage SSL key pairs using the GUI, only via API, so this is not a way
for not advanced users.
- Default user/password?


I can't believe that we are the only two buddies with this problem.



Regards.




Atentamente,
Sebastián Gómez

On Mon, Aug 7, 2017 at 3:36 PM, Pierre-Luc Dion <pd...@cloudops.com> wrote:

> Has Syed is saying its most likely because of systemd. Have you try to use
> cloud-init? It work well but need some tuning for password and sshkey
> reset...
>
>
>
> Le 7 août 2017 09:33, "Syed Ahmed" <sah...@cloudops.com> a écrit :
>
>> Hi,
>>
>> Is the password server mentioned in the logs 192.168.155.1 reachable? The
>> password server is hosted on the VR. Sometimes, restarting the network
>> will
>> reboot the VR and fix this. If not let us know, we'll help you out.
>>
>> Thanks,
>> -Syed
>>
>> On Fri, Aug 4, 2017 at 9:33 AM, Marc Poll Garcia <
>> marc.poll.gar...@upcnet.es
>> > wrote:
>>
>> > Hello everyone,
>> >
>> > last days we are experiencing some issues on our "Ubuntu 16.04"
>> template,
>> > created step by step following this tutorial:
>> >
>> > http://docs.cloudstack.apache.org/projects/cloudstack-
>> > administration/en/4.8/templates/_create_linux.html
>> >
>> > It is happening on "Ubuntu 16.04.2 LTS \n \l" system and  "CloudStack
>> > 4.9.2.0" version.
>> >
>> >
>> > Unfortunately we detected that some of the features we had, no longer
>> work,
>> > such as the changing password one.
>> >
>> > After several test and reviewing it more deeply, i see that it works
>> > "intermittently" and the service sometimes does not start on boot:
>> >
>> > /etc/init.d/cloud-set-guest-password status
>> > ● cloud-set-guest-password.service - LSB: Init file for Password
>> Download
>> > Client
>> >Loaded: loaded (/etc/init.d/cloud-set-guest-password; bad; vendor
>> > preset: enabled)
>> >Active: *failed* (Result: exit-code) since Thu 2017-08-03 16:09:09
>> CEST;
>> > 39min ago
>> >  Docs: man:systemd-sysv-generator(8)
>> >
>> > Aug 03 16:08:58 Ubuntu16PassKO systemd[1]: Starting LSB: Init file for
>> > Password Download Client...
>> > Aug 03 16:08:58 Ubuntu16PassKO cloud-set-guest-password[1062]:  *
>> Starting
>> > cloud cloud-set-guest-password
>> > Aug 03 16:09:09 Ubuntu16PassKO cloud[1252]: DHCP file
>> > (/var/lib/dhcp/dhclient-ens160.leases) exists. No need to restart
>> > dhclient.
>> > Aug 03 16:09:09 Ubuntu16PassKO cloud[1256]: Using DHCP lease from
>> > /var/lib/dhcp/dhclient-ens160.leases
>> > Aug 03 16:09:09 Ubuntu16PassKO cloud[1263]: Found password server IP
>> > 192.168.155.1 in /var/lib/dhcp/dhclient-ens160.leases
>> > Aug 03 16:09:09 Ubuntu16PassKO cloud[1264]: Sending request to password
>> > server at 192.168.155.1
>> > Aug 03 16:09:09 Ubuntu16PassKO systemd[1]:
>> > cloud-set-guest-password.service: *Control process exited, code=exited
>> > status=4*
>> > Aug 03 16:09:09 Ubuntu16PassKO systemd[1]: *Failed to start LSB: *Init
>> file
>> > for Password Download Client.
>> > Aug 03 16:09:09 Ubuntu16PassKO systemd[1]:
>> > cloud-set-guest-pa

Re: cloud-set-guest-password working with systemctl

[Pierre-Luc Dion]

Has Syed is saying its most likely because of systemd. Have you try to use
cloud-init? It work well but need some tuning for password and sshkey
reset...



Le 7 août 2017 09:33, "Syed Ahmed"  a écrit :

> Hi,
>
> Is the password server mentioned in the logs 192.168.155.1 reachable? The
> password server is hosted on the VR. Sometimes, restarting the network will
> reboot the VR and fix this. If not let us know, we'll help you out.
>
> Thanks,
> -Syed
>
> On Fri, Aug 4, 2017 at 9:33 AM, Marc Poll Garcia <
> marc.poll.gar...@upcnet.es
> > wrote:
>
> > Hello everyone,
> >
> > last days we are experiencing some issues on our "Ubuntu 16.04" template,
> > created step by step following this tutorial:
> >
> > http://docs.cloudstack.apache.org/projects/cloudstack-
> > administration/en/4.8/templates/_create_linux.html
> >
> > It is happening on "Ubuntu 16.04.2 LTS \n \l" system and  "CloudStack
> > 4.9.2.0" version.
> >
> >
> > Unfortunately we detected that some of the features we had, no longer
> work,
> > such as the changing password one.
> >
> > After several test and reviewing it more deeply, i see that it works
> > "intermittently" and the service sometimes does not start on boot:
> >
> > /etc/init.d/cloud-set-guest-password status
> > ● cloud-set-guest-password.service - LSB: Init file for Password
> Download
> > Client
> >Loaded: loaded (/etc/init.d/cloud-set-guest-password; bad; vendor
> > preset: enabled)
> >Active: *failed* (Result: exit-code) since Thu 2017-08-03 16:09:09
> CEST;
> > 39min ago
> >  Docs: man:systemd-sysv-generator(8)
> >
> > Aug 03 16:08:58 Ubuntu16PassKO systemd[1]: Starting LSB: Init file for
> > Password Download Client...
> > Aug 03 16:08:58 Ubuntu16PassKO cloud-set-guest-password[1062]:  *
> Starting
> > cloud cloud-set-guest-password
> > Aug 03 16:09:09 Ubuntu16PassKO cloud[1252]: DHCP file
> > (/var/lib/dhcp/dhclient-ens160.leases) exists. No need to restart
> > dhclient.
> > Aug 03 16:09:09 Ubuntu16PassKO cloud[1256]: Using DHCP lease from
> > /var/lib/dhcp/dhclient-ens160.leases
> > Aug 03 16:09:09 Ubuntu16PassKO cloud[1263]: Found password server IP
> > 192.168.155.1 in /var/lib/dhcp/dhclient-ens160.leases
> > Aug 03 16:09:09 Ubuntu16PassKO cloud[1264]: Sending request to password
> > server at 192.168.155.1
> > Aug 03 16:09:09 Ubuntu16PassKO systemd[1]:
> > cloud-set-guest-password.service: *Control process exited, code=exited
> > status=4*
> > Aug 03 16:09:09 Ubuntu16PassKO systemd[1]: *Failed to start LSB: *Init
> file
> > for Password Download Client.
> > Aug 03 16:09:09 Ubuntu16PassKO systemd[1]:
> > cloud-set-guest-password.service: Unit entered failed state.
> > Aug 03 16:09:09 Ubuntu16PassKO systemd[1]:
> > cloud-set-guest-password.service: Failed with result 'exit-code'.
> >
> >
> > LSB part from the script:
> >
> > root@Uprovav1:~# head -n20 /etc/init.d/cloud-set-guest-password
> >
> > #!/bin/bash
> >
> > #
> >
> > # Init file for Password Download Client
> >
> > #
> >
> > ### BEGIN INIT INFO
> >
> > # Provides: cloud-set-guest-password
> >
> > # Required-Start:   $local_fs $syslog $network
> >
> > # Required-Stop:$local_fs $syslog $network
> >
> > # Default-Start:2 3 4 5
> >
> > # Default-Stop: 0 1 6
> >
> > # Short-Description:Init file for Password Download Client
> >
> > ### END INIT INFO
> >
> >
> > Is it happening to anyone else? Is there another compatible version from
> > this script?
> >
> > Or maybe we have to adapt it to the new "systemctl"
> >
> > Thanks in advance.
> >
> > --
> > Marc Poll Garcia
> > Technology Infrastructure . Àrea de Serveis TIC
> > Telèfon:  93.405.43.57
> >
> > [image: UPCnet]
> >
> > --
> > Aquest correu electrònic pot contenir informació confidencial o legalment
> > protegida i està exclusivament dirigit a la persona o entitat
> destinatària.
> > Si vostè no és el destinatari final o persona encarregada de recollir-lo,
> > no està autoritzat a llegir-lo, retenir-lo, modificar-lo, distribuir-lo,
> > copiar-lo ni a revelar el seu contingut. Si ha rebut aquest correu
> > electrònic per error, li preguem que informi al remitent i elimini del
> seu
> > sistema el missatge i el material annex que pugui contenir.
> > Gràcies per la seva col.laboració.
> > --
> >
> > *** Si us plau, no m'imprimeixis. Vull seguir sent digital ***
> > *** Por favor, no me imprimas. Quiero seguir siendo digital ***
> > *** Please, don't print me. I want to remain digital ***
> > --
> >
>

Re: access to github

[Pierre-Luc Dion]

Hi Will,

Not sure what  was visible  with the Pony mail link, doesn't seams to work.
but anyway , I'll check for the gitbox projet,
There is some PRs I'd like to merge into the documentation.

Thanks !



*Pierre-Luc DION*
Architecte de Solution Cloud | Cloud Solutions Architect
t 855.652.5683

*CloudOps* Votre partenaire infonuagique* | *Cloud Solutions Experts
420 rue Guy *|* Montreal *|* Quebec *|* H3J 1S6
w cloudops.com *|* tw @CloudOps_

On Wed, Aug 2, 2017 at 8:45 PM, Will Stevens <williamstev...@gmail.com>
wrote:

> Here you go.
>
> https://lists.apache.org/thread.html/9fb5894da34e2cc383b2f9b01ab239
> fd7a5131b90f67c282560ffc5f@
> 
>
> On Aug 2, 2017 8:42 PM, "Will Stevens" <williamstev...@gmail.com> wrote:
>
> > You have to 2FA enabled in your GitHub account and register for the
> gitbox
> > project to be able to commit to that. But you can create a PR for now
> and I
> > can work with you to make sure you get your committee access setup.
> >
> > On Aug 2, 2017 8:39 PM, "Will Stevens" <williamstev...@gmail.com> wrote:
> >
> >> I think your looking for this. No?
> >>
> >> https://github.com/apache/cloudstack-docs-admin
> >>
> >> On Aug 2, 2017 8:37 PM, "Pierre-Luc Dion" <pd...@cloudops.com> wrote:
> >>
> >>> Hi,
> >>>
> >>> His there a particular process to follow to have access to github for
> >>> documentation projects?
> >>> Could it be possible Github is not a mirror anymore since the apache
> repo
> >>> [1] doesn't seams to exist ?
> >>>
> >>> [1] git://git.apache.org/cloudstack-docs-admin.git
> >>>
> >>> Thanks,
> >>>
> >>> Pierre-Luc
> >>>
> >>
>

access to github

[Pierre-Luc Dion]

Hi,

His there a particular process to follow to have access to github for
documentation projects?
Could it be possible Github is not a mirror anymore since the apache repo
[1] doesn't seams to exist ?

[1] git://git.apache.org/cloudstack-docs-admin.git

Thanks,

Pierre-Luc

[4.10] repos

[Pierre-Luc Dion]

Hi,

I've created new jenkins jobs [1] to create package for our release, The
jobs pushed rpm's but for systemvm, look like there is a wget job somewhere
that download systemvm template from jenkins from build of master branch,
I've created a set of systemvm templates that I would like to published on
download.cloudstack.org but they are most likely going to be overwritten,
could we disable that job ?

[1] https://builds.cloudstack.org/view/release/

PL

Re: [DISCUSS] drop support for CentOS6 with 4.10

[Pierre-Luc Dion]

In this case, if we say 4.10.0.0 can be install on centos6 it mean we need
to provide some instruction on how to install it, and is there anyone that
tested 4.10 packages on CentOS6 ?

Cheers,

On Mon, Jul 10, 2017 at 3:03 AM, Paul Angus <paul.an...@shapeblue.com>
wrote:

> I would suggest that this would justify a 5.0 release.. might be a good
> time to get in any other major changes
>
>
>
> Kind regards,
>
> Paul Angus
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>
> -Original Message-
> From: Pierre-Luc Dion [mailto:pdion...@apache.org]
> Sent: 09 July 2017 13:46
> To: dev@cloudstack.apache.org
> Subject: [DISCUSS] drop support for CentOS6 with 4.10
>
> Since 4.10 require JDK8, should we drop support for CentOS 6 as OS for
> management-server and usage ?
>
> all the Upgrade path tests I've made was by creating new management server
> on CentOS 7 that have updated tomcat and openJDK 8.  our packages (RPMs)
> install well on centos7 and I'm not away somebody test them on centos6.
>
> Should we call a vote on this?  I'm writing the upgrade part of the
> release notes and I would focus on centos7 only. we should also update our
> Quick Install Guide...
>
>
> Thanks,
>
> PL
>

[DISCUSS] drop support for CentOS6 with 4.10

[Pierre-Luc Dion]

Since 4.10 require JDK8, should we drop support for CentOS 6 as OS for
management-server and usage ?

all the Upgrade path tests I've made was by creating new management server
on CentOS 7 that have updated tomcat and openJDK 8.  our packages (RPMs)
install well on centos7 and I'm not away somebody test them on centos6.

Should we call a vote on this?  I'm writing the upgrade part of the release
notes and I would focus on centos7 only. we should also update our Quick
Install Guide...


Thanks,

PL

Re: [DISCUSS] Config Drive: Using the OpenStack format?

[Pierre-Luc Dion]

Is this a format that would work for all hypervisor supported  by
cloudstack? Because openstack is highly focus on kvm.

Also,did anyone verified if that would work for coreos and
cloudbase-init(cloud-init for Windows)

Otherwise I like the idea of removing the VR dependency.

Le 22 mai 2017 7:40 AM, "Nux!"  a écrit :

> +1 using openstack format
>
> --
> Sent from the Delta quadrant using Borg technology!
>
> Nux!
> www.nux.ro
>
> - Original Message -
> > From: "Wido den Hollander" 
> > To: "dev" , "Kris Sterckx" <
> kris.ster...@nuagenetworks.net>
> > Sent: Friday, 19 May, 2017 14:15:28
> > Subject: [DISCUSS] Config Drive: Using the OpenStack format?
>
> > Hi,
> >
> > Yesterday at ApacheCon Kris from Nuage networks gave a great
> presentation about
> > alternatives for userdata from the VR: Config Drive
> >
> > In short, a CD-ROM/ISO attached to the Instance containing the
> meta/userdata
> > instead of having the VR serve it.
> >
> > The outstanding PR [0] uses it's own format on the ISO while cloud-init
> already
> > has support for config drive [1].
> >
> > This format uses 'openstack' in the name, but it seems to be in
> cloud-init
> > natively and well supported.
> >
> > I started the discussion yesterday during the talk and thought to take
> it to the
> > list.
> >
> > My opinion is that we should use the OpenStack format for the config
> drive:
> >
> > - It's already in cloud-init
> > - Easier to templates to be used on CloudStack
> > - Easier adoption
> >
> > We can always write a file like "GENERATED_BY_APACHE_CLOUDSTACK" or
> something on
> > the ISO.
> >
> > We can also symlink the 'openstack' directory to a directory called
> 'cloudstack'
> > on the ISO.
> >
> > Does anybody else have a opinion on this one?
> >
> > Wido
> >
> > [0]: https://github.com/apache/cloudstack/pull/2097
> > [1]:
> > http://cloudinit.readthedocs.io/en/latest/topics/
> datasources/configdrive.html#version-2
>

Re: Awesome CloudStack

[Pierre-Luc Dion]

Awesome! Great idea!

Le 12 mai 2017 4:17 AM, "Rohit Yadav"  a écrit :

> Nice, thanks for starting this and sharing René.
>
> Regards.
>
> 
> From: Rene Moser 
> Sent: 10 May 2017 22:53:55
> To: us...@cloudstack.apache.org; dev
> Subject: Awesome CloudStack
>
> Hi
>
> I started "A curated list of bookmarks, packages, tutorials, videos and
> other cool resources from the CloudStack ecosystem" on
> https://github.com/resmo/awesome-cloudstack
>
> Feel free to extend it by sending PRs ;)
>
> René
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
>

Re: Alternative Cloudstack UI for KVM and Basic Zones (with SG)

[Pierre-Luc Dion]

Wow, that looks great, good start!

Do you have intent to give it back as an Apache projet eventually?

Thanks for the initiative!

Le 25 avr. 2017 4:04 PM, "Nathan Johnson"  a écrit :

> Very nice!
>
> Ivan Kudryavtsev  wrote:
>
> > Hello, Cloudstack community.
> >
> > We are proud to present our last development effort to you. During the
> last
> > 5 months we spend some time to develop alternative Cloudstack UI for
> basic
> > zones with KVM hypervisor and security groups. This is basically the
> thing
> > we are using in our clouds. During the design of the software we tried to
> > fulfill the expectations of our average cloud users and simplify
> operations
> > as much as possible.
> >
> > The project is OSS and can be found at GitHub with bunch of screenshots
> and
> > deployment guide. It's under active development so, we will ge glad if
> you
> > join and provide us with additional feedback, UX considerations and other
> > interesting information.
> >
> > Project page at GitHub: https://bwsw.github.io/cloudstack-ui/
> > Source code: https://github.com/bwsw/cloudstack-ui
> >
> > Have a good day. Looking forward hearing your feedback.
> >
> > --
> > With best regards, Ivan Kudryavtsev
> > Bitworks Software, Ltd.
> > Cell: +7-923-414-1515
> > WWW: http://bw-sw.com/
>
>
>

Re: [PROPOSAL] Apache Cloudstack should join the gitbox experiment

[Pierre-Luc Dion]

Yes, I saw people put a lot of efforts for it :-)

On Apr 5, 2017 10:20, "Rohit Yadav"  wrote:

> +1 in addition to my support for joining the experiment, I would also like
> us to have the option to assign labels on the PRs. Such labels can be
> useful for CIs and RMs to determine and merge PRs that are fully tested and
> blessed by reviewers.
>
>
> Regards.
>
> 
>
> rohit.ya...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> From: Rohit Yadav
> Sent: 05 April 2017 19:49:01
> To: dev
> Subject: Re: [PROPOSAL] Apache Cloudstack should join the gitbox experiment
>
>
> +1 looking forward to this.
>
> 
> From: Daan Hoogland 
> Sent: 05 April 2017 12:10:19
> To: dev
> Subject: [PROPOSAL] Apache Cloudstack should join the gitbox experiment
>
> In the Apache foundation an experiment has been going on to host
> mirrors of Apache project on github with more write access then just
> to the mirror-bot. For those projects committers can merge on github
> and put labels on PRs. I want to propose that we join that project and
> intend to start a vote on it here, soon.
>
> questions, thoughts?
>
> --
> Daan
>

Re: :[VOTE] Apache Cloudstack 4.10.0.0

[Pierre-Luc Dion]

Look like we need a new systemvm named "systemvm-xenserver-4.10". t also
seams that older VR  ex: 4.7.x are still usable if the globalsetting
"minreq.sysvmtemplate.version" is changed after the first boot of
cloudstack-management that upgraded the database. I'll update the release
note acordingly...

So this 4.10 is looking promising! :-)




On Apr 1, 2017 16:02, "Pierre-Luc Dion" <pd...@cloudops.com> wrote:

I'm testing upgrade to 4.10 from latest master. I have the following error
when upgrading from 4.7.2 in management-server.log:

2017-04-01 15:58:12,558 DEBUG [c.c.u.d.Upgrade4920to41000]
(localhost-startStop-1:null) (logid:) Updating System Vm template IDs
2017-04-01 15:58:12,561 DEBUG [c.c.u.d.Upgrade4920to41000]
(localhost-startStop-1:null) (logid:) Updating KVM System Vms
2017-04-01 15:58:12,561 WARN  [c.c.u.d.Upgrade4920to41000]
(localhost-startStop-1:null) (logid:) 4.10.0.0KVM SystemVm template not
found. KVM hypervisor is not used, so not failing upgrade
2017-04-01 15:58:12,562 DEBUG [c.c.u.d.Upgrade4920to41000]
(localhost-startStop-1:null) (logid:) Updating VMware System Vms
2017-04-01 15:58:12,563 WARN  [c.c.u.d.Upgrade4920to41000]
(localhost-startStop-1:null) (logid:) 4.10.0.0VMware SystemVm template not
found. VMware hypervisor is not used, so not failing upgrade
2017-04-01 15:58:12,563 DEBUG [c.c.u.d.Upgrade4920to41000]
(localhost-startStop-1:null) (logid:) Updating XenServer System Vms
2017-04-01 15:58:12,565 ERROR [c.c.u.DatabaseUpgradeChecker]
(localhost-startStop-1:null) (logid:) Unable to upgrade the database
com.cloud.utils.exception.CloudRuntimeException: 4.10.0.0XenServer SystemVm
template not found. Cannot upgrade system Vms
at com.cloud.upgrade.dao.Upgrade4920to41000.updateSystemVmTempl
ates(Upgrade4920to41000.java:195)
at com.cloud.upgrade.dao.Upgrade4920to41000.performDataMigratio
n(Upgrade4920to41000.java:64)
at com.cloud.upgrade.DatabaseUpgradeChecker.upgrade(DatabaseUpg
radeChecker.java:426)
at com.cloud.upgrade.DatabaseUpgradeChecker.check(DatabaseUpgra
deChecker.java:507)
at org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLif
eCycle.checkIntegrity(CloudStackExtendedLifeCycle.java:65)
at org.apache.cloudstack.spring.lifecycle.CloudStackExtendedLif
eCycle.start(CloudStackExtendedLifeCycle.java:55)
at org.springframework.context.support.DefaultLifecycleProcesso
r.doStart(DefaultLifecycleProcessor.java:173)
at org.springframework.context.support.DefaultLifecycleProcesso
r.access$200(DefaultLifecycleProcessor.java:51)
at org.springframework.context.support.DefaultLifecycleProcesso
r$LifecycleGroup.start(DefaultLifecycleProcessor.java:346)
at org.springframework.context.support.DefaultLifecycleProcesso
r.startBeans(DefaultLifecycleProcessor.java:149)
at org.springframework.context.support.DefaultLifecycleProcesso
r.onRefresh(DefaultLifecycleProcessor.java:112)
at org.springframework.context.support.AbstractApplicationConte
xt.finishRefresh(AbstractApplicationContext.java:879)
at org.springframework.context.support.AbstractApplicationConte
xt.refresh(AbstractApplicationContext.java:545)
at org.apache.cloudstack.spring.module.model.impl.DefaultModule
DefinitionSet.loadContext(DefaultModuleDefinitionSet.java:145)
at org.apache.cloudstack.spring.module.model.impl.DefaultModule
DefinitionSet$2.with(DefaultModuleDefinitionSet.java:122)
at org.apache.cloudstack.spring.module.model.impl.DefaultModule
DefinitionSet.withModule(DefaultModuleDefinitionSet.java:245)
at org.apache.cloudstack.spring.module.model.impl.DefaultModule
DefinitionSet.withModule(DefaultModuleDefinitionSet.java:250)
at org.apache.cloudstack.spring.module.model.impl.DefaultModule
DefinitionSet.withModule(DefaultModuleDefinitionSet.java:233)
at org.apache.cloudstack.spring.module.model.impl.DefaultModule
DefinitionSet.loadContexts(DefaultModuleDefinitionSet.java:117)
at org.apache.cloudstack.spring.module.model.impl.DefaultModule
DefinitionSet.load(DefaultModuleDefinitionSet.java:79)
at org.apache.cloudstack.spring.module.factory.ModuleBasedConte
xtFactory.loadModules(ModuleBasedContextFactory.java:37)
at org.apache.cloudstack.spring.module.factory.CloudStackSpring
Context.init(CloudStackSpringContext.java:71)
at org.apache.cloudstack.spring.module.factory.CloudStackSpring
Context.(CloudStackSpringContext.java:58)
at org.apache.cloudstack.spring.module.factory.CloudStackSpring
Context.(CloudStackSpringContext.java:62)
at org.apache.cloudstack.spring.module.web.CloudStackContextLoaderListener.
contextInitialized(CloudStackContextLoaderListener.java:52)
at org.apache.catalina.core.StandardContext.listenerStart(Stand
ardContext.java:5068)
at org.apache.catalina.core.StandardContext.startInternal(Stand
ardContext.java:5584)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:147)
at org.apache.catalina.core.ContainerBase.addChildInternal(Cont
ainerBase.java:899)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:875)
at org.apache

Re: Need help in getting CentOS 7 templates to run on Cloudstack 4.9 and VMWare

[Pierre-Luc Dion]

Hi Syed, have you tried coud images from centos site, i think centos build
cloudimage as canonical for ubuntu.


On Mar 30, 2017 5:24 PM, "Rafael Weingärtner" 
wrote:

> Attachments are not forwarded with emails.
>
> On Thu, Mar 30, 2017 at 5:16 PM, Syed Ahmed  wrote:
>
> > FYI I'm attaching the screenshot of the cloud-init error
> >
> >
> >
> > On Thu, Mar 30, 2017 at 5:14 PM, Syed Ahmed  wrote:
> > > Hi All,
> > >
> > > I'm trying to run a CentOS 7 template on VMWare and ACS 4.9 but
> > > somehow cloud-init doesn't seem to pick up the IP form the VR. The
> > > default template which is bundled with ACS (CentOS 5) works.
> > >
> > > I got the CentOS7 template from
> > > http://dl.openvm.eu/cloudstack/centos/vanilla/7/
> x86_64/CentOS-7-x86_64-
> > vanilla-vmware.ova
> > >
> > > Is there any setting that I need to do for this template? Are there
> > > any other places I can get a working template for CentOS7 for
> > > Cloudstack?
> > >
> > > Thanks,
> > > -Syed
> >
>
>
>
> --
> Rafael Weingärtner
>

Re: :[VOTE] Apache Cloudstack 4.10.0.0

[Pierre-Luc Dion]

cts/cloudstack-
> > release-notes/en/4.10/upgrade/upgrade-4.9.html
> > >>
> > >> Using the above, I've tried to fix these issues here,
> please
> > >> review and merge for RC2:
> > >>
> > >> https://github.com/apache/cloudstack/pull/1983
> > >>
> > >> <https://github.com/apache/cloudstack/pull/1983>With above
> > fix,
> > >> the aim is that users only seed the 4.10 systemvmtemplate
> > before
> > >> upgrade and post-upgrade the upgrade paths fix the entries,
> > >> global setting etc.
> > >>
> > >> Regards.
> > >>
> > >> 
> > >> From: Tutkowski, Mike <mike.tutkow...@netapp.com>
> > >> Sent: 02 March 2017 22:39:08
> > >> To: dev@cloudstack.apache.org
> > >> Subject: Re: :[VOTE] Apache Cloudstack 4.10.0.0
> > >>
> > >> I rolled back to my master branch at
> > >> da66b06e7d562393da2e4b52206943f8bad49d10 and it works.
> > >>
> > >> It appears something that went into after that commit has
> > broken
> > >> this. It looks like this SHA is about two weeks old and
> that
> > 43
> > >> commits have gone into master since it.
> > >>
> > >> On 3/2/17, 7:06 AM, "Tutkowski, Mike"
> > >> <mike.tutkow...@netapp.com> wrote:
> > >>
> > >> According to where the code fails, though, it appears to be
> a
> > >> networking problem. If I set a breakpoint before the
> failure
> > and
> > >> change a variable to say that security groups are not being
> > used,
> > >> then the VM starts.
> > >>
> > >> I think this is a recently introduced problem because I
> have
> > >> another branch based off of a slightly older version of
> master
> > >> and it works fine here.
> > >>
> > >>> On Mar 2, 2017, at 6:51 AM, Pierre-Luc Dion
> > >> <pd...@cloudops.com> wrote:
> > >>>
> > >>> Hi Mike,
> > >>> Try vm with at least 512MB for memory.
> > >>>
> > >>>> On Mar 1, 2017 15:01, "Tutkowski, Mike"
> > >> <mike.tutkow...@netapp.com> wrote:
> > >>>>
> > >>>> I see the following exception when trying to deploy a
> user VM
> > >> in a Basic
> > >>>> Zone with two XenServer 6.5 hosts in one cluster. My
> system
> > >> VMs have all
> > >>>> deployed properly. The user template gets downloaded
> fine. I
> > >> can see the
> > >>>> user VM begin to start on a XenServer host, then it goes
> > >> away. We then
> > >>>> automatically try on the other host. I can see the VM
> begin
> > >> to start there
> > >>>> for a moment, then it goes away.
> > >>>>
> > >>>> I am just deploying the user VM’s template and root disk
> to
> > >> NFS (same
> > >>>> place where the template and root disks of my system VMs
> > >> are).
> > >>>>
> > >>>> I am using the built-in XenServer CentOS 5.6 (64 bit)
> > >> template with 1
> > >>>> vCPU, 500 MHz, and 256 MB memory.
> > >>>>
> > >>>> WARN [c.c.a.r.v.VirtualRoutingResource]
> > >> (DirectAgent-7:ctx-35aded78)
> > >>>> (logid:aab9c320) Expected 1 answers while executing
> > >> VmDataCommand but
> > >>>> received 2
> > >>>> WARN [c.c.v.VirtualMachinePowerStateSyncImpl]
> > >> (DirectAgentCronJob-14:ctx-27fb1ac3)
> > >>>> (logid:2c342f23) VM state was updated but update time is
> > >> null?! vm id: 6
> > >>>> INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
> > >>>> (logid:a56a9a8c) Begin cleanup expired async-jobs
> > >>>> INFO [o.a.c.f.j.i.AsyncJobManagerImpl]
> > >> (AsyncJobMgr-Heartbeat-1:ctx-2c7d2dce)
> >

Re: :[VOTE] Apache Cloudstack 4.10.0.0

[Pierre-Luc Dion]

ManagerImpl.java:502)
> at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> INFO  [o.a.c.f.j.i.AsyncJobMonitor] (Work-Job-Executor-2:ctx-bc104380
> job-25/job-27) (logid:aab9c320) Remove job-27 from job monitoring
> WARN  [o.a.c.alerts] (API-Job-Executor-1:ctx-f787201d job-25
> ctx-56356c1a) (logid:aab9c320)  alertType:: 8 // dataCenterId:: 1 //
> podId:: 1 // clusterId:: null // message:: Failed to deploy Vm with Id: 6,
> on Host with Id: null
> ERROR [c.c.a.ApiAsyncJobDispatcher] (API-Job-Executor-1:ctx-f787201d
> job-25) (logid:aab9c320) Unexpected exception while executing
> org.apache.cloudstack.api.command.admin.vm.DeployVMCmdByAdmin
> com.cloud.utils.exception.CloudRuntimeException: Unable to start a VM due
> to insufficient capacity
> at com.cloud.vm.VirtualMachineManagerImpl.start(
> VirtualMachineManagerImpl.java:623)
> at org.apache.cloudstack.engine.cloud.entity.api.VMEntityManagerImpl.
> deployVirtualMachine(VMEntityManagerImpl.java:242)
> at org.apache.cloudstack.engine.cloud.entity.api.
> VirtualMachineEntityImpl.deploy(VirtualMachineEntityImpl.java:212)
> at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(
> UserVmManagerImpl.java:4084)
> at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(
> UserVmManagerImpl.java:3682)
> at com.cloud.vm.UserVmManagerImpl.startVirtualMachine(
> UserVmManagerImpl.java:3670)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke(
> NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke(
> DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at org.springframework.aop.support.AopUtils.
> invokeJoinpointUsingReflection(AopUtils.java:333)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.
> invokeJoinpoint(ReflectiveMethodInvocation.java:190)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.
> proceed(ReflectiveMethodInvocation.java:157)
> at org.apache.cloudstack.network.contrail.management.
> EventUtils$EventInterceptor.invoke(EventUtils.java:107)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.
> proceed(ReflectiveMethodInvocation.java:168)
> at com.cloud.event.ActionEventInterceptor.invoke(
> ActionEventInterceptor.java:51)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.
> proceed(ReflectiveMethodInvocation.java:168)
> at org.springframework.aop.interceptor.ExposeInvocationInterceptor.
> invoke(ExposeInvocationInterceptor.java:92)
> at org.springframework.aop.framework.ReflectiveMethodInvocation.
> proceed(ReflectiveMethodInvocation.java:179)
> at org.springframework.aop.framework.JdkDynamicAopProxy.
> invoke(JdkDynamicAopProxy.java:213)
> at com.sun.proxy.$Proxy186.startVirtualMachine(Unknown Source)
> at org.apache.cloudstack.api.command.admin.vm.
> DeployVMCmdByAdmin.execute(DeployVMCmdByAdmin.java:50)
> at com.cloud.api.ApiDispatcher.dispatch(ApiDispatcher.java:150)
> at com.cloud.api.ApiAsyncJobDispatcher.runJob(
> ApiAsyncJobDispatcher.java:108)
> at org.apache.cloudstack.framework.jobs.impl.AsyncJobManagerImpl$5.
> runInContext(AsyncJobManagerImpl.java:554)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable$1.run(
> ManagedContextRunnable.java:49)
> at org.apache.cloudstack.managed.context.impl.
> DefaultManagedContext$1.call(DefaultManagedContext.java:56)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.
> callWithContext(DefaultManagedContext.java:103)
> at org.apache.cloudstack.managed.context.impl.DefaultManagedContext.
> runWithContext(DefaultManagedContext.java:53)
> at org.apache.cloudstack.managed.context.ManagedContextRunnable.run(
> ManagedContextRunnable.java:46)
> at org.apache.cloudstack.framework.jobs.impl.
> AsyncJobManagerImpl$5.run(AsyncJobManagerImpl.java:502)
> at java.util.concurrent.Executors$RunnableAdapter.
> call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(
> ThreadPoolExecutor.java:1142)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(
> ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: com.cloud.exception.InsufficientServerCapacityException:
> Unable to create a deployment for

Re: :[VOTE] Apache Cloudstack 4.10.0.0

[Pierre-Luc Dion]

Do we support  centos6  with this 4.10 on jdk8?

Because I did not had much success to install 4.10 jdk8 on centos6  and it
would make more sense to drop support of centos6 so our packages would use
centos7 with distro packages for tomcat7 and jdk8.  Should also be the same
with ubuntu 16.04 that use jdk8 and tomcat7 by default ?


thanks,


On Wed, Mar 1, 2017 at 9:01 AM, Rene Moser  wrote:

> Hi
>
> While not be directly related to the clodustack java source code, any
> RPM created using the specs from the repo e.g. from packages/centos7
> and proceeding an upgrade, will hit CLOUDSTACK-9765, PR
> https://github.com/apache/cloudstack/pull/1923 fixes the issue.
>
> Regards
> René
>
>
> On 03/01/2017 02:12 AM, Rajani Karuturi wrote:
> > Hi All,
> >
> > I've created a 4.10.0.0 release, with the following artifacts up for a
> vote:
> >
> > Git Branch and Commit
> > SH:https://git-wip-us.apache.org/repos/asf?p=cloudstack.
> git;a=shortlog;h=refs/heads/4.10.0.0-RC20170301T0634
> > Commit:7c1d003b5269b375d87f4f6cfff8a144f0608b67
> >  git;a=shortlog;h=refs/heads/4.10.0.0-RC20170301T0634Commit:
> 7c1d003b5269b375d87f4f6cfff8a144f0608b67>
> >
> > Source release (checksums and signatures are available at the same
> > location):https://dist.apache.org/repos/dist/dev/cloudstack/4.10.0.0/
> >
> > PGP release keys (signed using
> > CBB44821):https://dist.apache.org/repos/dist/release/cloudstack/KEYS
> >
> > Vote will be open for 72 hours.
> >
> > For sanity in tallying the vote, can PMC members please be sure to
> > indicate "(binding)" with their vote?
> >
> > [ ] +1  approve
> > [ ] +0  no opinion
> > [ ] -1  disapprove (and reason why)
> >
> >
> >
> > ~Rajani
> > http://cloudplatform.accelerite.com/
> >
>

Re: Modern template hosting

[Pierre-Luc Dion]

hi!

I think we should work with distro provider to have their cloud builds work
with cloudstack. Good example is CoreOS, it work out of the box from their
channel builds.
it shouldn't be too complicated to have centos, ubuntu and debian, unless
...

For our systemvm templates, can we just change the URL for somthing like
cloudstack.apache.org/systemvm/... ?
and for old systemvm that are depricated but that we might want to keep,
could we archive them into a github repo in
https://github.com/apachecloudstack ?




On Mon, Feb 27, 2017 at 6:28 PM, Chiradeep Vittal 
wrote:

> My stance is that the current workflow does a disservice to the user
> community by letting them install / use outdated and insecure templates.
> Now, let's assume download.cloud.com is gone forever. What do we tell ACS
> users pre-4.11 as far as *built-in templates* go?
> 1. Direct them to update templates.sql with some new URL, but with the same
> dirty old templates
> 2. Direct them to update templates.sql with some new URL, but with nice
> templates (e.g., open.vm.eu)
> 3. Same as (2), but document more choices.
>
> Now, why should things be different for 4.11 and later? Documenting the
> steps to install templates offline is trivial (and can be scripted to a
> large part, like cloud-install-sys-tmplt)
>
> For pre-4.11 users, for *systemvms*, anyway we tell them to use
> http://cloudstack.apt-get.eu which is not controlled by ACS.
>
>
> On Mon, Feb 27, 2017 at 2:50 PM, Rafael Weingärtner <
> rafaelweingart...@gmail.com> wrote:
>
> > Agree with you.
> > We need to support the current working flow. And then, define the first
> > version that will start using the new approach.
> >
> > On Mon, Feb 27, 2017 at 5:36 PM, Will Stevens 
> > wrote:
> >
> > > I think we almost need a two pronged approach.
> > >
> > > 1) Get a solution in place which will enable us to document and serve
> > > templates for legacy systems.  I will work on this.
> > > 2) Discuss and understand how we SHOULD be handling this problem in the
> > > future and in what release we can expect it.
> > >
> > > I think we need to do both.  I think we should start to try to really
> > > understand what we want to deliver in (2) going forward.
> > >
> > > *Will STEVENS*
> > > Lead Developer
> > >
> > > 
> > >
> > > On Mon, Feb 27, 2017 at 4:53 PM, Rafael Weingärtner <
> > > rafaelweingart...@gmail.com> wrote:
> > >
> > > > My worry is exactly with system VMs templates.
> > > >
> > > > Currently, we indicate administrators to download them from
> > > > http://cloudstack.apt-get.eu/systemvm/4.6/ [1]. However, the
> > > installation
> > > > docs do not mention the expected hashes for the file that is going to
> > be
> > > > downloaded.
> > > > Also, I do not know the code that downloads system VMs templates
> (when
> > > > upgrading), but if the hash being checked is taken from the mirror
> used
> > > to
> > > > download the file; the only thing it checks is that if the download
> > > > finished successfully (no transmission errors). If we want to check
> > > > integrity, check that the template we created is untampered; we need
> to
> > > > host and serve the hash in a secure manner.
> > > >
> > > > [1]
> > > > http://docs.cloudstack.apache.org/projects/cloudstack-
> > > installation/en/4.9/
> > > > management-server/index.html#prepare-the-system-vm-template
> > > >
> > > >
> > > > On Mon, Feb 27, 2017 at 4:36 PM, Chiradeep Vittal <
> > chirade...@gmail.com>
> > > > wrote:
> > > >
> > > > > Hashes are checked (md5 IIRC) today.
> > > > > But given the issues, I think the project should steer away from
> > > hosting
> > > > > templates except the systemvm template.
> > > > >
> > > > > On Mon, Feb 27, 2017 at 1:31 PM, Rafael Weingärtner <
> > > > > rafaelweingart...@gmail.com> wrote:
> > > > >
> > > > > > Will, I think we could support different path structures. This
> can
> > > > > > facilitate different deployment of mirrors based on the structure
> > the
> > > > > host
> > > > > > has.
> > > > > >
> > > > > > Could I add something else to the discussion? Have we discussed
> the
> > > > > > security impacts of setting up this mirrors approach?
> > > > > > I mean, if any of the mirrors gets corrupted (let`s say by a
> > hacker),
> > > > and
> > > > > > the templates are injected with malicious code, an attacker could
> > > > > > potentially get un-monitored and unlimited access to a cloud
> > > > environment.
> > > > > >
> > > > > > If we assume that the mirror may get malicious (it is not that I
> do
> > > not
> > > > > > trust you guys, but bad things happen), we cannot host hashes
> > there.
> > > > > Where
> > > > > > do you think we could store Sha512 or another hash type for these
> > > > > > templates? Could we host in the newly proposed Github repo or
> maybe
> > > > some
> > > > > > place in the ACS website?
> > > > > >
> > > > > > This would have an impact on clients (needing clear
> documentation)
> > 

Re: [QUESTION] Upgrade path to JDK8

[Pierre-Luc Dion]

That's quite interesting Chiradeep!

so I could do something like this I guest:

mvn clean install

and then this one to build the systemvm.iso:
mvn -Psystemvm -source 1.7 -target 1.7 install


I'll give it a try! but for now, I'm worried about existing VR, they must
continue to work while running on jdk7.  newer VPC would be ok to run with
jdk8.  but we need to make sure while upgrading the management-server we
are not in the obligation to upgrade VR's.

For sure it is required for strongswan + JDK8 to ugprade the VR, but a
 existing VR should remain usable for port forwarding, vm creation and
such...

I'll post my finding...

Thanks !



On Mon, Feb 20, 2017 at 3:59 PM, Chiradeep Vittal <chirade...@gmail.com>
wrote:

> You can build the system vm with  -source 1.7 -target 1.7
> Also unless you are using Java8 features (lambda) the classfiles produced
> by javac 8 should work in a 1.7 JVM
>
> Sent from my iPhone
>
> > On Feb 20, 2017, at 11:51 AM, Will Stevens <wstev...@cloudops.com>
> wrote:
> >
> > yes, that is what I was expecting.  which is why I was asking about Wei's
> > setup because he seems to have worked around that problem.  Or he has a
> > custom SystemVM template running with both JDK7 and JDK8.
> >
> > *Will STEVENS*
> > Lead Developer
> >
> > <https://goo.gl/NYZ8KK>
> >
> >> On Mon, Feb 20, 2017 at 2:20 PM, Syed Ahmed <sah...@cloudops.com>
> wrote:
> >>
> >> The problem is that systemvm.iso is built with java 8 whereas java on
> the
> >> VR is java 7
> >>> On Mon, Feb 20, 2017 at 13:20 Will Stevens <wstev...@cloudops.com>
> wrote:
> >>>
> >>> Did it work after resetting a VPC or when blowing away the SSVM or
> >> CPVM?  I
> >>> would not expect the SSVM or the CPVM to come up if the management
> server
> >>> was built with JDK8 and the system vm template is only using JDK7.  Can
> >> you
> >>> confirm?​
> >>>
> >>> *Will STEVENS*
> >>> Lead Developer
> >>>
> >>> <https://goo.gl/NYZ8KK>
> >>>
> >>>> On Mon, Feb 20, 2017 at 1:15 PM, Wei ZHOU <ustcweiz...@gmail.com>
> wrote:
> >>>>
> >>>> We've tested management server 4.7.1 with ubuntu 16.04/openjdk8 and
> >>>> systemvm 4.6 with debian7/openjdk7.
> >>>> The systemvms (ssvm, cpvm) work fine.
> >>>>
> >>>> I agree we need consider the openjdk upgrade in systemvm template.
> >>>>
> >>>> -Wei
> >>>>
> >>>> 2017-02-20 18:15 GMT+01:00 Will Stevens <wstev...@cloudops.com>:
> >>>>
> >>>>> Regarding my question. Is it because of the version of Java that the
> >>>>> systemvm.iso is built on?
> >>>>>
> >>>>> On Feb 20, 2017 11:58 AM, "Will Stevens" <wstev...@cloudops.com>
> >>> wrote:
> >>>>>
> >>>>>> A question that is hidden in here is:
> >>>>>> - Why does the JDK version on the management server have to match
> >> the
> >>>> JDK
> >>>>>> version of the System VM?
> >>>>>>
> >>>>>> *Will STEVENS*
> >>>>>> Lead Developer
> >>>>>>
> >>>>>> <https://goo.gl/NYZ8KK>
> >>>>>>
> >>>>>> On Mon, Feb 20, 2017 at 11:50 AM, Pierre-Luc Dion <
> >>> pd...@cloudops.com>
> >>>>>> wrote:
> >>>>>>
> >>>>>>> Hi,
> >>>>>>>
> >>>>>>> In the context of deployment of CloudStack with VPCs,
> >>>>>>> What will happen to a cloud when upgrading to 4.10 that now use
> >>> jdk8?
> >>>>>>>
> >>>>>>> Does upgrading the management-server to 4.10 jdk8 and then keep
> >> the
> >>>> old
> >>>>>>> VRs
> >>>>>>> run for a while that run on JDK7 will still work ?
> >>>>>>>
> >>>>>>> Because If we upgrade the management-server to jdk8, we need to
> >> keep
> >>>> VR
> >>>>> to
> >>>>>>> work until they get upgraded but we can't force an upgrade of VR
> >>> just
> >>>>>>> because the management-server is now using JDK8.
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>
> >>
>

[QUESTION] Upgrade path to JDK8

[Pierre-Luc Dion]

Hi,

In the context of deployment of CloudStack with VPCs,
What will happen to a cloud when upgrading to 4.10 that now use jdk8?

Does upgrading the management-server to 4.10 jdk8 and then keep the old VRs
run for a while that run on JDK7 will still work ?

Because If we upgrade the management-server to jdk8, we need to keep VR to
work until they get upgraded but we can't force an upgrade of VR just
because the management-server is now using JDK8.

Thanks,

Re: maven settings.xml

[Pierre-Luc Dion]

Hi Raja,

You don't need a custom setting.xml in Maven for cloudstack unless you want
to define a private maven repo such as nexus



On Mon, Dec 26, 2016 at 1:22 PM, Raja Guru Thirumavalavan <
geeky.rajag...@gmail.com> wrote:

> Hi,
>
> Can some one please send me maven settings.xml for cloudstack or point me
> to a link where I can download from.
>
> Regards,
> Raja Guru T
>

[DISCUSS] VR metric collection

[Pierre-Luc Dion]

I'm looking for a way to collect metrics from VR's in a way that I would
call "aggressive" to collect CPU usage, disk, memory, network and VPN logs
in a way that I can almost have realtime data of them (every 15seconds).

My use case is to help troubleshoot network performance like haproxy and
VPNs, healthcheck and network usage, detect potantial issues. I've made a
POC intenally and I'd like to see how to go forward with this with the
community.

So what I did  is deployed tcollector and forward rsyslog on the VR and
point them on the IP:169.254.0.1.
I created iptables PortForward on hypervisors so data was send to OpenTSDB
and log into Logstash
so to me it's kind of simple and safe.

here is a quick list of change required on the VR and infra:
- add an agent on the VR
- forward log of rsyslogd
- haproxy stats to a socket instead of public ip
- Configure port forwarding on hypervisor
- some new zone settings ideally

Would I need to create a new systemvm-template for that ? or our
systemvm.iso support installation of package ?

So, what about adding telegraf [1] into our VR? Does is MIT license is
compatible with our's?
any one else interested by such as feature?

[1] https://github.com/influxdata/telegraf


Pierre-Luc

Re: Current status of IPv6 in Basic Networking

[Pierre-Luc Dion]

Thanks Wido,

It's cool seeing update like this!

On Dec 20, 2016 15:48, "Wido den Hollander"  wrote:

> Hi,
>
> Just wanted to give an update on my IPv6 progress since my last [0] update.
>
> The Wiki [1] still contains the most data about this, but the first PR [2]
> is already open which should fix CLOUDSTACK-9359 [3].
>
> When PR #1700 merges (I hope in to 4.10!) I will submit a second PR which
> brings full Security Group support for IPv6 under KVM with Basic Networking.
>
> The code [4] is already running on a private cloud at PCextreme and
> working fine there. I really hope that code can also make it into 4.10
> since that will be a huge IPv6 step for CloudStack.
>
> Afterwards I will be working on getting better IPv6 support for the SSVMs.
> Main focus is still Basic Networking however. Anything additional is a good
> side-effect.
>
> I ask everybody to look at PR #1700 and give feedback. The sooner it
> merges, the better.
>
> Thanks!
>
> Wido
>
> [0]: http://mail-archives.apache.org/mod_mbox/cloudstack-dev/
> 201610.mbox/%3C803597888.5548.1475520180772%40ox.pcextreme.nl%3E
> [1]: https://cwiki.apache.org/confluence/display/CLOUDSTACK/
> IPv6+in+Basic+Networking
> [2]: https://github.com/apache/cloudstack/pull/1700
> [3]: https://issues.apache.org/jira/browse/CLOUDSTACK-9359
> [4]: https://github.com/wido/cloudstack/commits/ipv6-basic-
> networking-secgroup-ports
>

Re: [VOTE] Release Apache CloudStack CloudMonkey 5.3.3

[Pierre-Luc Dion]

+1 (binding)

Got it to work on osX-sierra.


On Thu, Nov 3, 2016 at 10:00 AM, Sigert GOEMINNE <
sigert.goemi...@nuagenetworks.net> wrote:

> +1
>
> On Thu, Nov 3, 2016 at 5:32 AM, Rohit Yadav 
> wrote:
>
> > Hi All,
> >
> > I've created a 5.3.3 release-candidate of CloudMonkey, with the following
> > artifacts up for a vote:
> >
> > Git Branch and Commit SH:
> > https://git-wip-us.apache.org/repos/asf?p=cloudstack-
> > cloudmonkey.git;a=shortlog;h=refs/heads/master
> > Commit: 319c2dc097fcf7607d352d9fb26f13e8051074ff
> >
> > List of changes:
> > https://git-wip-us.apache.org/repos/asf?p=cloudstack-
> > cloudmonkey.git;a=blob_plain;f=CHANGES;hb=master
> >
> > Source release (checksums and signatures are available at the same
> > location):
> > https://dist.apache.org/repos/dist/dev/cloudstack/cloudmonkey-5.3.3/
> >
> > PGP release keys (signed using 0EE3D884):
> > https://dist.apache.org/repos/dist/release/cloudstack/KEYS
> >
> > Vote will be open for 72 hours.
> >
> > For sanity in tallying the vote, can PMC members please be sure to
> > indicate "(binding)" with their vote?
> >
> > [ ] +1  approve
> > [ ] +0  no opinion
> > [ ] -1  disapprove (and reason why)
> >
> >
> > Regards.
> >
> > rohit.ya...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> > @shapeblue
> >
> >
> >
> >
>

Re: [DISCUSS] Replacing the VR

[Pierre-Luc Dion]

I'm not sure how it would help to containerized VR services, but it would
certainly help on maintaining them.

I really like the idea of using a configuration management tool on the VR
that would be responsible to maintain the VR at the expected state by
CloudStack.

At the moment on our side we are struggling with old libraries and
configuration issues on the current VR, I am still wondering where the
effort worth between a new default VR based on vyos or container based or
fix what we have and make it better :-S

But I find the thread very healthy, thank you :)

On Tue, Sep 20, 2016 at 5:54 AM, Murali Reddy 
wrote:

> Configuration management of network appliances particularly for Cloud and
> NFV scenarios is still evolving area. Programmability is the not the
> strength for even the most popular network operating systems like IOS,
> JunoS etc. So its not surprising why CloudStack integrates in a archaic way
> with stock linux for the VR.
>
> VR was never integral/hardcoded option in CloudStack. Its always been a
> plugin. CloudStack network orchestration is well abstracted and designed
> with vision to compose a network with different set of providers for
> different services. Yes that vision is not fully realised yet, and we don’t
> have true service function chains. That would be different discussion topic.
>
> I tend to agree with Simon, as alternate/interim option we can take hard
> look whats causing the problems with current VR integration. Personally, I
> think it would be easier we take a cue from configuration managers and
> network configuration solutions out there (for e.g promise theory based
> Cisco ACI) move to more declarative model of expressing desired state of
> network configuration. Infact current VR from 4.6, actually holds the
> desired state per service basis, seems to be in that direction.
>
> It does make sense to evaluate new appliances which can provide rich
> semantics (like programmable API, declarative configuration, versioning,
> commit/rollback etc), but will need significant engineering effort and time
> to stabilise. We may make same mistakes with integration of other appliance
> as well, if we fully don’t understand the nature of the current problems
> with CloudStack core and service provider interaction and current VR
> integration.
>
>
> On 16/09/16, 11:59 PM, "Simon Weller"  wrote:
>
> >I think our other option is to take a real look at what it would take to
> fix the VR. In my opinion, a lot of the problems are related to the
> monolithic python code base and the fact nothing is actually separated.
> >
> >Secondly, the python scripts (and bash scripts) don't use any established
> libraries to complete tasks and instead shell out and run commands that are
> both hard to track and hard to parse on return.
> >
> >
> >If we daemonized this, used a real api for Agent to VR communication,
> used common already existing libraries for the system service and network
> interactions and spent a bit of time separating out code into distinct
> modules, everything would behave a lot better.
> >
> >
> >The pain and suffering is due to years and years of patches and constant
> shelling out to complete tasks in my opinion. If we spend time to rethink
> how we interact with the VR in general and we abstract the systems and
> networking stuff and use well known and stable libraries to do the work,
> the VR would be much easier to maintain.
> >
> >
> >- Si
> >
> >
> >
> >
> >
> >From: Marty Godsey 
> >Sent: Friday, September 16, 2016 12:24 PM
> >To: dev@cloudstack.apache.org
> >Subject: RE: [DISCUSS] Replacing the VR
> >
> >So based upon this discussion would it be prudent to wait on VyOS 2.0?
> The current VR is giving us issues but would the time invested in another
> "solution" be wasted especially if by the time another option is chose,
> then coded, then tested, then implemented and right as that time happened
> to be when VyOS 2.0 is released.  Of course you said they are just in the
> scoping range so this could still be a year or more out.
> >
> >Thoughts?
> >
> >Regards,
> >Marty Godsey
> >nSource Solutions
> >
> >-Original Message-
> >From: williamstev...@gmail.com [mailto:williamstev...@gmail.com] On
> Behalf Of Will Stevens
> >Sent: Friday, September 16, 2016 10:31 AM
> >To: dev@cloudstack.apache.org
> >Cc: dan...@baturin.org
> >Subject: Re: [DISCUSS] Replacing the VR
> >
> >I just had a quick chat with a couple of the guys over on the VyOS chat.
> >I have CC'ed one of them in case we have more licensing questions.
> >
> >So here is the status with the license "the code inherited from Vyatta
> and our modifications from it is GPLv2 (strict, not v2+). The config
> reading library is GPLv2 too, so anything that links to is is GPLv2.
> >Some auxiliary components we made after the fork are more permissive,
> >LGPLv2+ or MIT."
> >
> >They are currently in the process of 

Re: is this a bug

[Pierre-Luc Dion]

Hi Jeroen,

You should need at least 1Gb but I don t think it provide a warning if you
don't have enough space.

On Sep 20, 2016 07:33, "Wei ZHOU"  wrote:

>  2 GB ?
>
> 2016-09-20 12:40 GMT+02:00 Jeroen Baten :
>
> > Could it be that  in /usr/share/cloudstack-common/
> > scripts/storage/secondary/cloud-install-sys-tmplt
> >
> > the variable DISKSPACE as one zero too many?
> >
> >
> >
> > Or, if you really insist on having a tmp space of 2T, you might want to
> > add this to the requirements in the doc section J
> >
> >
> >
> > Greets,
> >
> >
> >
> > Jeroen Baten
> >
> > Specialist ICT
> >
> >
> >
> >
> >
> > T +31(0)88335 7941
> >
> > M+31(0)6 4691 4649
> >
> > [image: Description: Description: cid:image002.png@01CE9846.E058C610]
> >
> > P.O. Box 177
> >
> > 2600 MH Delft
> >
> > The Netherlands
> >
> >
> >
> >
> >
> >
> > DISCLAIMER: This message is intended exclusively for the addressee(s) and
> > may contain confidential and privileged information. If you are not the
> > intended recipient please notify the sender immediately and destroy this
> > message. Unauthorized use, disclosure or copying of this message is
> > strictly prohibited. The foundation 'Stichting Deltares', which has its
> > seat at Delft, The Netherlands, Commercial Registration Number 41146461,
> is
> > not liable in any way whatsoever for consequences and/or damages
> resulting
> > from the improper, incomplete and untimely dispatch, receipt and/or
> content
> > of this e-mail.
> >
>

builds.cloudstack.org (new jenkins system)

[Pierre-Luc Dion]

Hi,

we now have a nice URL pointing to the replacement jenkins of   j.bac.o:
builds.cloudstack.org

I'll make sure the enable SSL for user authentication, whoever is commiter
that want admin access to Jenkins, let me know, if you have CI system that
we could leverage via a centralized portal, feel free to post on dev@

Cheers,

PL