Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Ivan Kudryavtsev]

Hello, guys.

What kind of hardware do you need actually to improve?
I host own DC and probably could give you our spare hardware connected to
dedicated managed switch. I can not guarantee the resources will be granted
and when the customer need it I'll take it, but we usually have 5-10 spare
servers with Xeon E5620, E3-1230.

Best wishes, Ivan.

2017-07-04 19:57 GMT+07:00 Syed Ahmed :

> As Paul said, in theory, running KVM as your base hypervisor in Trillain
> shoud be possible. We have done nested KVM in the past with XenServer and
> KVM as the nested hypervisors and with KVM being the base hypervisor. I am
> not completely sure about how VMWare handles being in a nested environment.
> Having said that, I believe if we get KVM and XenServer support with KVM as
> being the base hypervisor, we will have a lot more adaptability for
> Trillian. I will work with Paul and team on this.
>
> As a side note, I have been working on getting to run integration testing
> from a docker container. We need this because we require our tests to be
> done on real hardware for cloud.ca. I really like the container approach
> as
> it bundles all dependencies required by Marvin into a single container
> which can be launched from any machine which has docker. This would hugely
> benefit running it via Jenkins for example. I will open-source it as soon
> as I am happy with it.
>
>
>
> On Mon, Jul 3, 2017 at 2:34 PM, Will Stevens 
> wrote:
>
> > I have added Syed to this.  He has done some initial review to get a port
> > to KVM working, but I am not sure how far he got yet.
> >
> > *Will Stevens*
> > CTO
> >
> > <https://goo.gl/NYZ8KK>
> >
> > On Mon, Jul 3, 2017 at 2:26 PM, Paul Angus 
> > wrote:
> >
> >> I will take an action to look at trillian on KVM.  There's nothing
> >> explicit or implicit in trillian itself that it requires vmware as long
> as
> >> we can trunk the guest VLANs and virtualise the hypervisors.
> >>
> >> ____
> >> From: Will Stevens 
> >> Sent: 3 Jul 2017 2:06 am
> >> To: dev@cloudstack.apache.org
> >> Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof
> >>
> >> Sorry, I have been keeping up with these threads while on vacation at a
> >> campsite.  :)  Finally back to a computer.
> >>
> >> Yes, ideally we would have more people actually committing code and
> >> validating the PRs are ready for merge.  Right now, we have VERY limited
> >> CI
> >> setups, so we are bottlenecked on the ability to actually test the PRs
> in
> >> a
> >> timely fashion.  This leads to PRs sitting for a week at a time in some
> >> 'phase' of the process.  This means that the developers get pushed off
> to
> >> other items to pay the bills and it then causes a lag everywhere.
> Because
> >> of this, the RM has basically had to fill the role of making sure
> >> everything is moving and understanding and unblocking the different PRs
> as
> >> they move through the review/test/commit phases.
> >>
> >> Yes, we need more reviewers.  That is very true.
> >>
> >> Also true, is that we need more CI environments.  I would love to see
> >> Trillian be able to be run on KVM on a developers laptop to at least
> test
> >> the core components.  We could then start to standardize the dev/test
> >> cycle
> >> for developers so we can start focusing on a 'minimum support feature
> >> set'.  We could also hopefully leverage the developers setups to run CI
> >> overnight if they choose to participate (or at least their own PRs).  If
> >> we
> >> could standardize well enough to push the workload to the edge, I think
> we
> >> would end up with more active rigs in the game.
> >>
> >> I personally feel that if we can put the CI on rails and standardize our
> >> dev environment, a lot of our 'we need an full time RM' problems go
> away.
> >> However, I do think we will need a full time RM for at least a year to
> be
> >> able to shepherd that into existence though.
> >>
> >> *Will Stevens*
> >> CTO
> >>
> >> <https://goo.gl/NYZ8KK>
> >>
> >>
> >> paul.an...@shapeblue.com
> >> www.shapeblue.com
> >> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> >> @shapeblue
> >>
> >>
> >>
> >> On Sun, Jul 2, 2017 at 8:06 PM, Syed Ahmed  wrote:
> >>
> >> > Agree with Ro

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Syed Ahmed]

As Paul said, in theory, running KVM as your base hypervisor in Trillain
shoud be possible. We have done nested KVM in the past with XenServer and
KVM as the nested hypervisors and with KVM being the base hypervisor. I am
not completely sure about how VMWare handles being in a nested environment.
Having said that, I believe if we get KVM and XenServer support with KVM as
being the base hypervisor, we will have a lot more adaptability for
Trillian. I will work with Paul and team on this.

As a side note, I have been working on getting to run integration testing
from a docker container. We need this because we require our tests to be
done on real hardware for cloud.ca. I really like the container approach as
it bundles all dependencies required by Marvin into a single container
which can be launched from any machine which has docker. This would hugely
benefit running it via Jenkins for example. I will open-source it as soon
as I am happy with it.



On Mon, Jul 3, 2017 at 2:34 PM, Will Stevens  wrote:

> I have added Syed to this.  He has done some initial review to get a port
> to KVM working, but I am not sure how far he got yet.
>
> *Will Stevens*
> CTO
>
> <https://goo.gl/NYZ8KK>
>
> On Mon, Jul 3, 2017 at 2:26 PM, Paul Angus 
> wrote:
>
>> I will take an action to look at trillian on KVM.  There's nothing
>> explicit or implicit in trillian itself that it requires vmware as long as
>> we can trunk the guest VLANs and virtualise the hypervisors.
>>
>> 
>> From: Will Stevens 
>> Sent: 3 Jul 2017 2:06 am
>> To: dev@cloudstack.apache.org
>> Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof
>>
>> Sorry, I have been keeping up with these threads while on vacation at a
>> campsite.  :)  Finally back to a computer.
>>
>> Yes, ideally we would have more people actually committing code and
>> validating the PRs are ready for merge.  Right now, we have VERY limited
>> CI
>> setups, so we are bottlenecked on the ability to actually test the PRs in
>> a
>> timely fashion.  This leads to PRs sitting for a week at a time in some
>> 'phase' of the process.  This means that the developers get pushed off to
>> other items to pay the bills and it then causes a lag everywhere.  Because
>> of this, the RM has basically had to fill the role of making sure
>> everything is moving and understanding and unblocking the different PRs as
>> they move through the review/test/commit phases.
>>
>> Yes, we need more reviewers.  That is very true.
>>
>> Also true, is that we need more CI environments.  I would love to see
>> Trillian be able to be run on KVM on a developers laptop to at least test
>> the core components.  We could then start to standardize the dev/test
>> cycle
>> for developers so we can start focusing on a 'minimum support feature
>> set'.  We could also hopefully leverage the developers setups to run CI
>> overnight if they choose to participate (or at least their own PRs).  If
>> we
>> could standardize well enough to push the workload to the edge, I think we
>> would end up with more active rigs in the game.
>>
>> I personally feel that if we can put the CI on rails and standardize our
>> dev environment, a lot of our 'we need an full time RM' problems go away.
>> However, I do think we will need a full time RM for at least a year to be
>> able to shepherd that into existence though.
>>
>> *Will Stevens*
>> CTO
>>
>> <https://goo.gl/NYZ8KK>
>>
>>
>> paul.an...@shapeblue.com
>> www.shapeblue.com
>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> @shapeblue
>>
>>
>>
>> On Sun, Jul 2, 2017 at 8:06 PM, Syed Ahmed  wrote:
>>
>> > Agree with Ron about this being a role of the commiter but in what I
>> have
>> > seen, it is mostly the RM who has to run around and ask for updates/make
>> > sure fixes are done.
>> >
>> > Part of the problem also is that there is a lack of reviewers. We've had
>> > some issues recently [1] which were code which was committed was not
>> > properly reviewed and later lead to problems. Having a RM and some core
>> > reviewers is essential to maintain quality and sanity of the release.
>> >
>> > I also agree with testing on a known setup with known parameters. The
>> > community can pool resources for hardware and Trillian can be used as
>> the
>> > CI framework. There was supposedly hadware donated by Citrix to the ASF.
>> > Anyone know what happened to that?
>> >
&

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Will Stevens]

I have added Syed to this.  He has done some initial review to get a port
to KVM working, but I am not sure how far he got yet.

*Will Stevens*
CTO

<https://goo.gl/NYZ8KK>

On Mon, Jul 3, 2017 at 2:26 PM, Paul Angus  wrote:

> I will take an action to look at trillian on KVM.  There's nothing
> explicit or implicit in trillian itself that it requires vmware as long as
> we can trunk the guest VLANs and virtualise the hypervisors.
>
> 
> From: Will Stevens 
> Sent: 3 Jul 2017 2:06 am
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof
>
> Sorry, I have been keeping up with these threads while on vacation at a
> campsite.  :)  Finally back to a computer.
>
> Yes, ideally we would have more people actually committing code and
> validating the PRs are ready for merge.  Right now, we have VERY limited CI
> setups, so we are bottlenecked on the ability to actually test the PRs in a
> timely fashion.  This leads to PRs sitting for a week at a time in some
> 'phase' of the process.  This means that the developers get pushed off to
> other items to pay the bills and it then causes a lag everywhere.  Because
> of this, the RM has basically had to fill the role of making sure
> everything is moving and understanding and unblocking the different PRs as
> they move through the review/test/commit phases.
>
> Yes, we need more reviewers.  That is very true.
>
> Also true, is that we need more CI environments.  I would love to see
> Trillian be able to be run on KVM on a developers laptop to at least test
> the core components.  We could then start to standardize the dev/test cycle
> for developers so we can start focusing on a 'minimum support feature
> set'.  We could also hopefully leverage the developers setups to run CI
> overnight if they choose to participate (or at least their own PRs).  If we
> could standardize well enough to push the workload to the edge, I think we
> would end up with more active rigs in the game.
>
> I personally feel that if we can put the CI on rails and standardize our
> dev environment, a lot of our 'we need an full time RM' problems go away.
> However, I do think we will need a full time RM for at least a year to be
> able to shepherd that into existence though.
>
> *Will Stevens*
> CTO
>
> <https://goo.gl/NYZ8KK>
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>
>
>
> On Sun, Jul 2, 2017 at 8:06 PM, Syed Ahmed  wrote:
>
> > Agree with Ron about this being a role of the commiter but in what I have
> > seen, it is mostly the RM who has to run around and ask for updates/make
> > sure fixes are done.
> >
> > Part of the problem also is that there is a lack of reviewers. We've had
> > some issues recently [1] which were code which was committed was not
> > properly reviewed and later lead to problems. Having a RM and some core
> > reviewers is essential to maintain quality and sanity of the release.
> >
> > I also agree with testing on a known setup with known parameters. The
> > community can pool resources for hardware and Trillian can be used as the
> > CI framework. There was supposedly hadware donated by Citrix to the ASF.
> > Anyone know what happened to that?
> >
> > [1] https://github.com/apache/cloudstack/pull/1834
> >
> >
> > On Sun, Jul 2, 2017 at 9:55 AM, Ron Wheeler  > com>
> > wrote:
> >
> > > I think you are describing the roles of all of the committers
> > >
> > > Is it normal at Apache for the RM to be doing all of this stuff?
> > >
> > > I would expect that the RM has a QC role in these activities but others
> > > are doing the work.
> > >
> > > Ron
> > >
> > >
> > > On 01/07/2017 7:18 PM, Will Stevens wrote:
> > >
> > >> Oh, and if a system VM is touched, then you have to add in a new
> system
> > VM
> > >> build and install into the CI setup prior to testing...
> > >>
> > >> On Jul 1, 2017 6:41 PM, wrote:
> > >>
> > >> Which is part of the reason the RM job is hard and time consuming.
> > >> - checking the PRs have the appropriate tests.
> > >> - updating the CI to include the new tests.
> > >> - run and report CI for the PR (with very limited CI infra community
> > >> wide).
> > >> - chase PR authors to get their PRs to a point where you are happy
> they
> > >> are
> > >> not breaking master
&

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Paul Angus]

I will take an action to look at trillian on KVM.  There's nothing explicit or 
implicit in trillian itself that it requires vmware as long as we can trunk the 
guest VLANs and virtualise the hypervisors.


From: Will Stevens 
Sent: 3 Jul 2017 2:06 am
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof

Sorry, I have been keeping up with these threads while on vacation at a
campsite.  :)  Finally back to a computer.

Yes, ideally we would have more people actually committing code and
validating the PRs are ready for merge.  Right now, we have VERY limited CI
setups, so we are bottlenecked on the ability to actually test the PRs in a
timely fashion.  This leads to PRs sitting for a week at a time in some
'phase' of the process.  This means that the developers get pushed off to
other items to pay the bills and it then causes a lag everywhere.  Because
of this, the RM has basically had to fill the role of making sure
everything is moving and understanding and unblocking the different PRs as
they move through the review/test/commit phases.

Yes, we need more reviewers.  That is very true.

Also true, is that we need more CI environments.  I would love to see
Trillian be able to be run on KVM on a developers laptop to at least test
the core components.  We could then start to standardize the dev/test cycle
for developers so we can start focusing on a 'minimum support feature
set'.  We could also hopefully leverage the developers setups to run CI
overnight if they choose to participate (or at least their own PRs).  If we
could standardize well enough to push the workload to the edge, I think we
would end up with more active rigs in the game.

I personally feel that if we can put the CI on rails and standardize our
dev environment, a lot of our 'we need an full time RM' problems go away.
However, I do think we will need a full time RM for at least a year to be
able to shepherd that into existence though.

*Will Stevens*
CTO

<https://goo.gl/NYZ8KK>


paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 

On Sun, Jul 2, 2017 at 8:06 PM, Syed Ahmed  wrote:

> Agree with Ron about this being a role of the commiter but in what I have
> seen, it is mostly the RM who has to run around and ask for updates/make
> sure fixes are done.
>
> Part of the problem also is that there is a lack of reviewers. We've had
> some issues recently [1] which were code which was committed was not
> properly reviewed and later lead to problems. Having a RM and some core
> reviewers is essential to maintain quality and sanity of the release.
>
> I also agree with testing on a known setup with known parameters. The
> community can pool resources for hardware and Trillian can be used as the
> CI framework. There was supposedly hadware donated by Citrix to the ASF.
> Anyone know what happened to that?
>
> [1] https://github.com/apache/cloudstack/pull/1834
>
>
> On Sun, Jul 2, 2017 at 9:55 AM, Ron Wheeler  com>
> wrote:
>
> > I think you are describing the roles of all of the committers
> >
> > Is it normal at Apache for the RM to be doing all of this stuff?
> >
> > I would expect that the RM has a QC role in these activities but others
> > are doing the work.
> >
> > Ron
> >
> >
> > On 01/07/2017 7:18 PM, Will Stevens wrote:
> >
> >> Oh, and if a system VM is touched, then you have to add in a new system
> VM
> >> build and install into the CI setup prior to testing...
> >>
> >> On Jul 1, 2017 6:41 PM, wrote:
> >>
> >> Which is part of the reason the RM job is hard and time consuming.
> >> - checking the PRs have the appropriate tests.
> >> - updating the CI to include the new tests.
> >> - run and report CI for the PR (with very limited CI infra community
> >> wide).
> >> - chase PR authors to get their PRs to a point where you are happy they
> >> are
> >> not breaking master
> >> - rinse repeat for 200+ PRs...
> >>
> >> On Jul 1, 2017 6:34 PM, "Will Stevens" 
> wrote:
> >>
> >> Yes, we can totally reject PRs until we are happy with the associated
> >> tests.
> >>
> >> On Jul 1, 2017 5:48 PM, "Alex Hitchins"  wrote:
> >>
> >> Out of interest, are there any guidelines/rules in place to reject PR's
> >>> without unit tests?
> >>>
> >>>
> >>>
> >>>
> >>> Alexander Hitchins
> >>> 
> >>> E: a...@alexhitchins.com
> >>> W: alexhitchins.com
> >>> M: 

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Will Stevens]

Sorry, I have been keeping up with these threads while on vacation at a
campsite.  :)  Finally back to a computer.

Yes, ideally we would have more people actually committing code and
validating the PRs are ready for merge.  Right now, we have VERY limited CI
setups, so we are bottlenecked on the ability to actually test the PRs in a
timely fashion.  This leads to PRs sitting for a week at a time in some
'phase' of the process.  This means that the developers get pushed off to
other items to pay the bills and it then causes a lag everywhere.  Because
of this, the RM has basically had to fill the role of making sure
everything is moving and understanding and unblocking the different PRs as
they move through the review/test/commit phases.

Yes, we need more reviewers.  That is very true.

Also true, is that we need more CI environments.  I would love to see
Trillian be able to be run on KVM on a developers laptop to at least test
the core components.  We could then start to standardize the dev/test cycle
for developers so we can start focusing on a 'minimum support feature
set'.  We could also hopefully leverage the developers setups to run CI
overnight if they choose to participate (or at least their own PRs).  If we
could standardize well enough to push the workload to the edge, I think we
would end up with more active rigs in the game.

I personally feel that if we can put the CI on rails and standardize our
dev environment, a lot of our 'we need an full time RM' problems go away.
However, I do think we will need a full time RM for at least a year to be
able to shepherd that into existence though.

*Will Stevens*
CTO

<https://goo.gl/NYZ8KK>

On Sun, Jul 2, 2017 at 8:06 PM, Syed Ahmed  wrote:

> Agree with Ron about this being a role of the commiter but in what I have
> seen, it is mostly the RM who has to run around and ask for updates/make
> sure fixes are done.
>
> Part of the problem also is that there is a lack of reviewers. We've had
> some issues recently [1] which were code which was committed was not
> properly reviewed and later lead to problems. Having a RM and some core
> reviewers is essential to maintain quality and sanity of the release.
>
> I also agree with testing on a known setup with known parameters. The
> community can pool resources for hardware and Trillian can be used as the
> CI framework. There was supposedly hadware donated by Citrix to the ASF.
> Anyone know what happened to that?
>
> [1] https://github.com/apache/cloudstack/pull/1834
>
>
> On Sun, Jul 2, 2017 at 9:55 AM, Ron Wheeler  com>
> wrote:
>
> > I think you are describing the roles of all of the committers
> >
> > Is it normal at Apache for the RM to be doing all of this stuff?
> >
> > I would expect that the RM has a QC role in these activities but others
> > are doing the work.
> >
> > Ron
> >
> >
> > On 01/07/2017 7:18 PM, Will Stevens wrote:
> >
> >> Oh, and if a system VM is touched, then you have to add in a new system
> VM
> >> build and install into the CI setup prior to testing...
> >>
> >> On Jul 1, 2017 6:41 PM, wrote:
> >>
> >> Which is part of the reason the RM job is hard and time consuming.
> >> - checking the PRs have the appropriate tests.
> >> - updating the CI to include the new tests.
> >> - run and report CI for the PR (with very limited CI infra community
> >> wide).
> >> - chase PR authors to get their PRs to a point where you are happy they
> >> are
> >> not breaking master
> >> - rinse repeat for 200+ PRs...
> >>
> >> On Jul 1, 2017 6:34 PM, "Will Stevens" 
> wrote:
> >>
> >> Yes, we can totally reject PRs until we are happy with the associated
> >> tests.
> >>
> >> On Jul 1, 2017 5:48 PM, "Alex Hitchins"  wrote:
> >>
> >> Out of interest, are there any guidelines/rules in place to reject PR's
> >>> without unit tests?
> >>>
> >>>
> >>>
> >>>
> >>> Alexander Hitchins
> >>> 
> >>> E: a...@alexhitchins.com
> >>> W: alexhitchins.com
> >>> M: 07788 423 969
> >>> T: 01892 523 587
> >>>
> >>> -Original Message-
> >>> From: Paul Angus [mailto:paul.an...@shapeblue.com]
> >>> Sent: 30 June 2017 21:58
> >>> To: dev@cloudstack.apache.org
> >>> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
> >>>
> >>> Taken from a talk on CloudStack testing [1]...
> >>>
> >>> There are Many, many, 

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Syed Ahmed]

Agree with Ron about this being a role of the commiter but in what I have
seen, it is mostly the RM who has to run around and ask for updates/make
sure fixes are done.

Part of the problem also is that there is a lack of reviewers. We've had
some issues recently [1] which were code which was committed was not
properly reviewed and later lead to problems. Having a RM and some core
reviewers is essential to maintain quality and sanity of the release.

I also agree with testing on a known setup with known parameters. The
community can pool resources for hardware and Trillian can be used as the
CI framework. There was supposedly hadware donated by Citrix to the ASF.
Anyone know what happened to that?

[1] https://github.com/apache/cloudstack/pull/1834


On Sun, Jul 2, 2017 at 9:55 AM, Ron Wheeler 
wrote:

> I think you are describing the roles of all of the committers
>
> Is it normal at Apache for the RM to be doing all of this stuff?
>
> I would expect that the RM has a QC role in these activities but others
> are doing the work.
>
> Ron
>
>
> On 01/07/2017 7:18 PM, Will Stevens wrote:
>
>> Oh, and if a system VM is touched, then you have to add in a new system VM
>> build and install into the CI setup prior to testing...
>>
>> On Jul 1, 2017 6:41 PM, wrote:
>>
>> Which is part of the reason the RM job is hard and time consuming.
>> - checking the PRs have the appropriate tests.
>> - updating the CI to include the new tests.
>> - run and report CI for the PR (with very limited CI infra community
>> wide).
>> - chase PR authors to get their PRs to a point where you are happy they
>> are
>> not breaking master
>> - rinse repeat for 200+ PRs...
>>
>> On Jul 1, 2017 6:34 PM, "Will Stevens"  wrote:
>>
>> Yes, we can totally reject PRs until we are happy with the associated
>> tests.
>>
>> On Jul 1, 2017 5:48 PM, "Alex Hitchins"  wrote:
>>
>> Out of interest, are there any guidelines/rules in place to reject PR's
>>> without unit tests?
>>>
>>>
>>>
>>>
>>> Alexander Hitchins
>>> --------
>>> E: a...@alexhitchins.com
>>> W: alexhitchins.com
>>> M: 07788 423 969
>>> T: 01892 523 587
>>>
>>> -Original Message-
>>> From: Paul Angus [mailto:paul.an...@shapeblue.com]
>>> Sent: 30 June 2017 21:58
>>> To: dev@cloudstack.apache.org
>>> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>>>
>>> Taken from a talk on CloudStack testing [1]...
>>>
>>> There are Many, many, MANY permutations of a CloudStack deployment….
>>> • Basic / Advanced
>>> • Local / shared / mixed storage
>>> • More than 8 common hypervisor types/versions • 4 or 5 Management server
>>> OS possibilities • That’s 144 combinations only looking the basics.
>>>
>>> [1] https://www.slideshare.net/ShapeBlue/cloudstack-eu-user-grou
>>> p-trillian?qid=74ff2be0-664c-4bca-a3dc-f30d880ca088&v=&b=&from_search=1
>>>
>>> Trillian [2], can create any of those, and multiple at the same time, but
>>> the amount of hardware required to do that means that we have to pick our
>>> battles. Running the blueorangutan bot command '@blueorangutan matrix'
>>> in a
>>> PR will run the smoke test suite against a PR using 3 environments, one
>>> each of KVM, XenServer and vSphere and takes around 8 hours.
>>>
>>> But that is only looking for major regressions.  A full component test
>>> run
>>> takes around 5 days to run and is riddled with bugs in the tests.
>>>
>>> Ultimately these are still of limited scope, few people are as diligent
>>> as
>>> say Mike T in creating practical marvin tests for their code / features.
>>>
>>> [2] https://github.com/shapeblue/Trillian
>>>
>>> Therefore we need hardware to run tests on, but more importantly we need
>>> the tests to exist and work in the first place.  Then we can really do
>>> something.
>>>
>>>
>>>
>>> paul.an...@shapeblue.com
>>> www.shapeblue.com
>>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>>>
>>>
>>>
>>>
>>> -Original Message-
>>> From: Alex Hitchins [mailto:a...@alexhitchins.com]
>>> Sent: 30 June 2017 21:34
>>> To: dev@cloudstack.apache.org; dev@cloudstack.apache.org
>>> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>>

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Ron Wheeler]

I think you are describing the roles of all of the committers

Is it normal at Apache for the RM to be doing all of this stuff?

I would expect that the RM has a QC role in these activities but others 
are doing the work.


Ron

On 01/07/2017 7:18 PM, Will Stevens wrote:

Oh, and if a system VM is touched, then you have to add in a new system VM
build and install into the CI setup prior to testing...

On Jul 1, 2017 6:41 PM, wrote:

Which is part of the reason the RM job is hard and time consuming.
- checking the PRs have the appropriate tests.
- updating the CI to include the new tests.
- run and report CI for the PR (with very limited CI infra community wide).
- chase PR authors to get their PRs to a point where you are happy they are
not breaking master
- rinse repeat for 200+ PRs...

On Jul 1, 2017 6:34 PM, "Will Stevens"  wrote:

Yes, we can totally reject PRs until we are happy with the associated
tests.

On Jul 1, 2017 5:48 PM, "Alex Hitchins"  wrote:


Out of interest, are there any guidelines/rules in place to reject PR's
without unit tests?




Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Paul Angus [mailto:paul.an...@shapeblue.com]
Sent: 30 June 2017 21:58
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

Taken from a talk on CloudStack testing [1]...

There are Many, many, MANY permutations of a CloudStack deployment….
• Basic / Advanced
• Local / shared / mixed storage
• More than 8 common hypervisor types/versions • 4 or 5 Management server
OS possibilities • That’s 144 combinations only looking the basics.

[1] https://www.slideshare.net/ShapeBlue/cloudstack-eu-user-grou
p-trillian?qid=74ff2be0-664c-4bca-a3dc-f30d880ca088&v=&b=&from_search=1

Trillian [2], can create any of those, and multiple at the same time, but
the amount of hardware required to do that means that we have to pick our
battles. Running the blueorangutan bot command '@blueorangutan matrix' in a
PR will run the smoke test suite against a PR using 3 environments, one
each of KVM, XenServer and vSphere and takes around 8 hours.

But that is only looking for major regressions.  A full component test run
takes around 5 days to run and is riddled with bugs in the tests.

Ultimately these are still of limited scope, few people are as diligent as
say Mike T in creating practical marvin tests for their code / features.

[2] https://github.com/shapeblue/Trillian

Therefore we need hardware to run tests on, but more importantly we need
the tests to exist and work in the first place.  Then we can really do
something.



paul.an...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue




-Original Message-
From: Alex Hitchins [mailto:a...@alexhitchins.com]
Sent: 30 June 2017 21:34
To: dev@cloudstack.apache.org; dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

Consultation with users is something that should definite be done. Canvas
as many as possible.

I agree that most people will be running test environments before full
rollout of any technology, I guess see it a little from a CTO eyes - why
shortlist a technology that doesn't even endorse its own releases?

Hopefully we will get some more replies to this thread from other
CloudStack enthusiasts to help shape this conversation.

I'm setting up a new development environment now to get my hands mildly
soiled. Going the Windows route again. Fancy a challenge for the weekend.




Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Ron Wheeler [mailto:rwhee...@artifact-software.com]
Sent: 30 June 2017 21:08
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof


On 30/06/2017 3:28 PM, Alex Hitchins wrote:

We can't validate all scenarios no, hence suggesting several common

setups as a reasonable baseline. I like the idea of users posting their
hardware and versions as a community endeavour.

I strongly feel there should be an established, physical setup that the

release team have access to in order to validate a release.

This is perhaps something that should be requested from the user community.
I would expect that anyone running Cloudstack in production on a major
site would have a test setup and might be very happy to have the dev team
test on their setup.
This would save them a lot of resources at the expense of a bit of time on
their test environment.


If this was some random cat meme generator on GitHub, I'd accept the

risk in running an untested version. For something I'd be running my
business on however I'd expect there being some weight behind the release.

Perhaps I have un

RE: [DISCUSS] - Releases, Project Management & Funding Thereof

[Will Stevens]

Oh, and if a system VM is touched, then you have to add in a new system VM
build and install into the CI setup prior to testing...

On Jul 1, 2017 6:41 PM, wrote:

Which is part of the reason the RM job is hard and time consuming.
- checking the PRs have the appropriate tests.
- updating the CI to include the new tests.
- run and report CI for the PR (with very limited CI infra community wide).
- chase PR authors to get their PRs to a point where you are happy they are
not breaking master
- rinse repeat for 200+ PRs...

On Jul 1, 2017 6:34 PM, "Will Stevens"  wrote:

Yes, we can totally reject PRs until we are happy with the associated
tests.

On Jul 1, 2017 5:48 PM, "Alex Hitchins"  wrote:

> Out of interest, are there any guidelines/rules in place to reject PR's
> without unit tests?
>
>
>
>
> Alexander Hitchins
> 
> E: a...@alexhitchins.com
> W: alexhitchins.com
> M: 07788 423 969
> T: 01892 523 587
>
> -Original Message-
> From: Paul Angus [mailto:paul.an...@shapeblue.com]
> Sent: 30 June 2017 21:58
> To: dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>
> Taken from a talk on CloudStack testing [1]...
>
> There are Many, many, MANY permutations of a CloudStack deployment….
> • Basic / Advanced
> • Local / shared / mixed storage
> • More than 8 common hypervisor types/versions • 4 or 5 Management server
> OS possibilities • That’s 144 combinations only looking the basics.
>
> [1] https://www.slideshare.net/ShapeBlue/cloudstack-eu-user-grou
> p-trillian?qid=74ff2be0-664c-4bca-a3dc-f30d880ca088&v=&b=&from_search=1
>
> Trillian [2], can create any of those, and multiple at the same time, but
> the amount of hardware required to do that means that we have to pick our
> battles. Running the blueorangutan bot command '@blueorangutan matrix' in a
> PR will run the smoke test suite against a PR using 3 environments, one
> each of KVM, XenServer and vSphere and takes around 8 hours.
>
> But that is only looking for major regressions.  A full component test run
> takes around 5 days to run and is riddled with bugs in the tests.
>
> Ultimately these are still of limited scope, few people are as diligent as
> say Mike T in creating practical marvin tests for their code / features.
>
> [2] https://github.com/shapeblue/Trillian
>
> Therefore we need hardware to run tests on, but more importantly we need
> the tests to exist and work in the first place.  Then we can really do
> something.
>
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>
>
>
>
> -Original Message-
> From: Alex Hitchins [mailto:a...@alexhitchins.com]
> Sent: 30 June 2017 21:34
> To: dev@cloudstack.apache.org; dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>
> Consultation with users is something that should definite be done. Canvas
> as many as possible.
>
> I agree that most people will be running test environments before full
> rollout of any technology, I guess see it a little from a CTO eyes - why
> shortlist a technology that doesn't even endorse its own releases?
>
> Hopefully we will get some more replies to this thread from other
> CloudStack enthusiasts to help shape this conversation.
>
> I'm setting up a new development environment now to get my hands mildly
> soiled. Going the Windows route again. Fancy a challenge for the weekend.
>
>
>
>
> Alexander Hitchins
> --------
> E: a...@alexhitchins.com
> W: alexhitchins.com
> M: 07788 423 969
> T: 01892 523 587
>
> -Original Message-
> From: Ron Wheeler [mailto:rwhee...@artifact-software.com]
> Sent: 30 June 2017 21:08
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof
>
>
> On 30/06/2017 3:28 PM, Alex Hitchins wrote:
> > We can't validate all scenarios no, hence suggesting several common
> setups as a reasonable baseline. I like the idea of users posting their
> hardware and versions as a community endeavour.
> >
> > I strongly feel there should be an established, physical setup that the
> release team have access to in order to validate a release.
>
> This is perhaps something that should be requested from the user community.
> I would expect that anyone running Cloudstack in production on a major
> site would have a test setup and might be very happy to have the dev team
> test on their setup.
> This would save them a lot of resources at the expense of a bit of time on
> their test environment.
>
> &

RE: [DISCUSS] - Releases, Project Management & Funding Thereof

[Will Stevens]

Which is part of the reason the RM job is hard and time consuming.
- checking the PRs have the appropriate tests.
- updating the CI to include the new tests.
- run and report CI for the PR (with very limited CI infra community wide).
- chase PR authors to get their PRs to a point where you are happy they are
not breaking master
- rinse repeat for 200+ PRs...

On Jul 1, 2017 6:34 PM, "Will Stevens"  wrote:

Yes, we can totally reject PRs until we are happy with the associated
tests.

On Jul 1, 2017 5:48 PM, "Alex Hitchins"  wrote:

> Out of interest, are there any guidelines/rules in place to reject PR's
> without unit tests?
>
>
>
>
> Alexander Hitchins
> 
> E: a...@alexhitchins.com
> W: alexhitchins.com
> M: 07788 423 969
> T: 01892 523 587
>
> -Original Message-
> From: Paul Angus [mailto:paul.an...@shapeblue.com]
> Sent: 30 June 2017 21:58
> To: dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>
> Taken from a talk on CloudStack testing [1]...
>
> There are Many, many, MANY permutations of a CloudStack deployment….
> • Basic / Advanced
> • Local / shared / mixed storage
> • More than 8 common hypervisor types/versions • 4 or 5 Management server
> OS possibilities • That’s 144 combinations only looking the basics.
>
> [1] https://www.slideshare.net/ShapeBlue/cloudstack-eu-user-grou
> p-trillian?qid=74ff2be0-664c-4bca-a3dc-f30d880ca088&v=&b=&from_search=1
>
> Trillian [2], can create any of those, and multiple at the same time, but
> the amount of hardware required to do that means that we have to pick our
> battles. Running the blueorangutan bot command '@blueorangutan matrix' in a
> PR will run the smoke test suite against a PR using 3 environments, one
> each of KVM, XenServer and vSphere and takes around 8 hours.
>
> But that is only looking for major regressions.  A full component test run
> takes around 5 days to run and is riddled with bugs in the tests.
>
> Ultimately these are still of limited scope, few people are as diligent as
> say Mike T in creating practical marvin tests for their code / features.
>
> [2] https://github.com/shapeblue/Trillian
>
> Therefore we need hardware to run tests on, but more importantly we need
> the tests to exist and work in the first place.  Then we can really do
> something.
>
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>
>
>
>
> -Original Message-
> From: Alex Hitchins [mailto:a...@alexhitchins.com]
> Sent: 30 June 2017 21:34
> To: dev@cloudstack.apache.org; dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>
> Consultation with users is something that should definite be done. Canvas
> as many as possible.
>
> I agree that most people will be running test environments before full
> rollout of any technology, I guess see it a little from a CTO eyes - why
> shortlist a technology that doesn't even endorse its own releases?
>
> Hopefully we will get some more replies to this thread from other
> CloudStack enthusiasts to help shape this conversation.
>
> I'm setting up a new development environment now to get my hands mildly
> soiled. Going the Windows route again. Fancy a challenge for the weekend.
>
>
>
>
> Alexander Hitchins
> --------
> E: a...@alexhitchins.com
> W: alexhitchins.com
> M: 07788 423 969
> T: 01892 523 587
>
> -Original Message-
> From: Ron Wheeler [mailto:rwhee...@artifact-software.com]
> Sent: 30 June 2017 21:08
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof
>
>
> On 30/06/2017 3:28 PM, Alex Hitchins wrote:
> > We can't validate all scenarios no, hence suggesting several common
> setups as a reasonable baseline. I like the idea of users posting their
> hardware and versions as a community endeavour.
> >
> > I strongly feel there should be an established, physical setup that the
> release team have access to in order to validate a release.
>
> This is perhaps something that should be requested from the user community.
> I would expect that anyone running Cloudstack in production on a major
> site would have a test setup and might be very happy to have the dev team
> test on their setup.
> This would save them a lot of resources at the expense of a bit of time on
> their test environment.
>
> > If this was some random cat meme generator on GitHub, I'd accept the
> risk in running an untested version. For something I'd be running my
> 

RE: [DISCUSS] - Releases, Project Management & Funding Thereof

[Will Stevens]

Yes, we can totally reject PRs until we are happy with the associated
tests.

On Jul 1, 2017 5:48 PM, "Alex Hitchins"  wrote:

> Out of interest, are there any guidelines/rules in place to reject PR's
> without unit tests?
>
>
>
>
> Alexander Hitchins
> 
> E: a...@alexhitchins.com
> W: alexhitchins.com
> M: 07788 423 969
> T: 01892 523 587
>
> -Original Message-
> From: Paul Angus [mailto:paul.an...@shapeblue.com]
> Sent: 30 June 2017 21:58
> To: dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>
> Taken from a talk on CloudStack testing [1]...
>
> There are Many, many, MANY permutations of a CloudStack deployment….
> • Basic / Advanced
> • Local / shared / mixed storage
> • More than 8 common hypervisor types/versions • 4 or 5 Management server
> OS possibilities • That’s 144 combinations only looking the basics.
>
> [1] https://www.slideshare.net/ShapeBlue/cloudstack-eu-user-
> group-trillian?qid=74ff2be0-664c-4bca-a3dc-f30d880ca088&v=
> &b=&from_search=1
>
> Trillian [2], can create any of those, and multiple at the same time, but
> the amount of hardware required to do that means that we have to pick our
> battles. Running the blueorangutan bot command '@blueorangutan matrix' in a
> PR will run the smoke test suite against a PR using 3 environments, one
> each of KVM, XenServer and vSphere and takes around 8 hours.
>
> But that is only looking for major regressions.  A full component test run
> takes around 5 days to run and is riddled with bugs in the tests.
>
> Ultimately these are still of limited scope, few people are as diligent as
> say Mike T in creating practical marvin tests for their code / features.
>
> [2] https://github.com/shapeblue/Trillian
>
> Therefore we need hardware to run tests on, but more importantly we need
> the tests to exist and work in the first place.  Then we can really do
> something.
>
>
>
> paul.an...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>
>
>
>
> -----Original Message-
> From: Alex Hitchins [mailto:a...@alexhitchins.com]
> Sent: 30 June 2017 21:34
> To: dev@cloudstack.apache.org; dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>
> Consultation with users is something that should definite be done. Canvas
> as many as possible.
>
> I agree that most people will be running test environments before full
> rollout of any technology, I guess see it a little from a CTO eyes - why
> shortlist a technology that doesn't even endorse its own releases?
>
> Hopefully we will get some more replies to this thread from other
> CloudStack enthusiasts to help shape this conversation.
>
> I'm setting up a new development environment now to get my hands mildly
> soiled. Going the Windows route again. Fancy a challenge for the weekend.
>
>
>
>
> Alexander Hitchins
> --------
> E: a...@alexhitchins.com
> W: alexhitchins.com
> M: 07788 423 969
> T: 01892 523 587
>
> -Original Message-
> From: Ron Wheeler [mailto:rwhee...@artifact-software.com]
> Sent: 30 June 2017 21:08
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof
>
>
> On 30/06/2017 3:28 PM, Alex Hitchins wrote:
> > We can't validate all scenarios no, hence suggesting several common
> setups as a reasonable baseline. I like the idea of users posting their
> hardware and versions as a community endeavour.
> >
> > I strongly feel there should be an established, physical setup that the
> release team have access to in order to validate a release.
>
> This is perhaps something that should be requested from the user community.
> I would expect that anyone running Cloudstack in production on a major
> site would have a test setup and might be very happy to have the dev team
> test on their setup.
> This would save them a lot of resources at the expense of a bit of time on
> their test environment.
>
> > If this was some random cat meme generator on GitHub, I'd accept the
> risk in running an untested version. For something I'd be running my
> business on however I'd expect there being some weight behind the release.
> >
> > Perhaps I have unrealistic expectations!
>
> Not at all.
> Your expectations might be the key to making a pitch to the user community
> for some help from people and organizations that are not interested in
> writing code but have a major interest in testing.
> In addition to access to test equ

RE: [DISCUSS] - Releases, Project Management & Funding Thereof

[Alex Hitchins]

Out of interest, are there any guidelines/rules in place to reject PR's without 
unit tests?




Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Paul Angus [mailto:paul.an...@shapeblue.com] 
Sent: 30 June 2017 21:58
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

Taken from a talk on CloudStack testing [1]...

There are Many, many, MANY permutations of a CloudStack deployment…. 
• Basic / Advanced
• Local / shared / mixed storage
• More than 8 common hypervisor types/versions • 4 or 5 Management server OS 
possibilities • That’s 144 combinations only looking the basics.

[1] 
https://www.slideshare.net/ShapeBlue/cloudstack-eu-user-group-trillian?qid=74ff2be0-664c-4bca-a3dc-f30d880ca088&v=&b=&from_search=1

Trillian [2], can create any of those, and multiple at the same time, but the 
amount of hardware required to do that means that we have to pick our battles. 
Running the blueorangutan bot command '@blueorangutan matrix' in a PR will run 
the smoke test suite against a PR using 3 environments, one each of KVM, 
XenServer and vSphere and takes around 8 hours.

But that is only looking for major regressions.  A full component test run 
takes around 5 days to run and is riddled with bugs in the tests. 

Ultimately these are still of limited scope, few people are as diligent as say 
Mike T in creating practical marvin tests for their code / features.

[2] https://github.com/shapeblue/Trillian

Therefore we need hardware to run tests on, but more importantly we need the 
tests to exist and work in the first place.  Then we can really do something.



paul.an...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
  
 


-Original Message-
From: Alex Hitchins [mailto:a...@alexhitchins.com]
Sent: 30 June 2017 21:34
To: dev@cloudstack.apache.org; dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

Consultation with users is something that should definite be done. Canvas as 
many as possible.

I agree that most people will be running test environments before full rollout 
of any technology, I guess see it a little from a CTO eyes - why shortlist a 
technology that doesn't even endorse its own releases?

Hopefully we will get some more replies to this thread from other CloudStack 
enthusiasts to help shape this conversation.

I'm setting up a new development environment now to get my hands mildly soiled. 
Going the Windows route again. Fancy a challenge for the weekend.




Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Ron Wheeler [mailto:rwhee...@artifact-software.com]
Sent: 30 June 2017 21:08
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof


On 30/06/2017 3:28 PM, Alex Hitchins wrote:
> We can't validate all scenarios no, hence suggesting several common setups as 
> a reasonable baseline. I like the idea of users posting their hardware and 
> versions as a community endeavour.
>
> I strongly feel there should be an established, physical setup that the 
> release team have access to in order to validate a release.

This is perhaps something that should be requested from the user community.
I would expect that anyone running Cloudstack in production on a major site 
would have a test setup and might be very happy to have the dev team test on 
their setup.
This would save them a lot of resources at the expense of a bit of time on 
their test environment.

> If this was some random cat meme generator on GitHub, I'd accept the risk in 
> running an untested version. For something I'd be running my business on 
> however I'd expect there being some weight behind the release.
>
> Perhaps I have unrealistic expectations!

Not at all.
Your expectations might be the key to making a pitch to the user community for 
some help from people and organizations that are not interested in writing code 
but have a major interest in testing.
In addition to access to test equipment, this might actually get new people on 
the team with the right skills required to extend the test scripts and test 
procedure documentation.

Does anyone have a list of the configuration specifications that are required 
to test a new release?

Would it help to approach major users of Cloudstack with a direct request for 
use of their test equipment and QA staff in return for early access to new 
releases and testing on their hardware?

Ron

>
> Alexander Hitchins
> 
> E: a...@alexhitchins.com
> W: alexhitchins.com
> M: 07788 423 969
> T: 01892 523 587
>
> -Original Message-
>

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Ron Wheeler]

Do all of these combinations need to be fully tested for each release?

What are the combinations that have been tested for the current release?

How many of these combinations are known to be running in production?

How many of these production organizations have test environments that 
could be used? And operations staff that could run the tests.
They will test anyway so it is mostly a change to timing and the actual 
scripts.
They may be able to augment the existing scripts with the test scripts 
that they are using or work on the completion of the scripts already 
planned.


I am unsure what Paul means by "we need hardware to run tests on".
Clearly hardware is required for testing but it would not seem to matter 
where the hardware exists or who owns it as long as it is available.


Is there a list of tests that are missing?
Is the test suite documented so that end-users can actually use the 
tests on their own test systems?


This is a bit of a switch in thinking about testing and about the role 
of the users in the release management process but it has some benefits.
The testing function of the release team switches to a project 
management role that involves tracking and coaching the testing ecosystem.


Ron




On 30/06/2017 4:57 PM, Paul Angus wrote:

Taken from a talk on CloudStack testing [1]...

There are Many, many, MANY permutations of a CloudStack deployment….
• Basic / Advanced
• Local / shared / mixed storage
• More than 8 common hypervisor types/versions
• 4 or 5 Management server OS possibilities
• That’s 144 combinations only looking the basics.

[1] 
https://www.slideshare.net/ShapeBlue/cloudstack-eu-user-group-trillian?qid=74ff2be0-664c-4bca-a3dc-f30d880ca088&v=&b=&from_search=1

Trillian [2], can create any of those, and multiple at the same time, but the 
amount of hardware required to do that means that we have to pick our battles. 
Running the blueorangutan bot command '@blueorangutan matrix' in a PR will run 
the smoke test suite against a PR using 3 environments, one each of KVM, 
XenServer and vSphere and takes around 8 hours.

But that is only looking for major regressions.  A full component test run 
takes around 5 days to run and is riddled with bugs in the tests.

Ultimately these are still of limited scope, few people are as diligent as say 
Mike T in creating practical marvin tests for their code / features.

[2] https://github.com/shapeblue/Trillian

Therefore we need hardware to run tests on, but more importantly we need the 
tests to exist and work in the first place.  Then we can really do something.



paul.an...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
   
  



-Original Message-
From: Alex Hitchins [mailto:a...@alexhitchins.com]
Sent: 30 June 2017 21:34
To: dev@cloudstack.apache.org; dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

Consultation with users is something that should definite be done. Canvas as 
many as possible.

I agree that most people will be running test environments before full rollout 
of any technology, I guess see it a little from a CTO eyes - why shortlist a 
technology that doesn't even endorse its own releases?

Hopefully we will get some more replies to this thread from other CloudStack 
enthusiasts to help shape this conversation.

I'm setting up a new development environment now to get my hands mildly soiled. 
Going the Windows route again. Fancy a challenge for the weekend.




Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Ron Wheeler [mailto:rwhee...@artifact-software.com]
Sent: 30 June 2017 21:08
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof


On 30/06/2017 3:28 PM, Alex Hitchins wrote:

We can't validate all scenarios no, hence suggesting several common setups as a 
reasonable baseline. I like the idea of users posting their hardware and 
versions as a community endeavour.

I strongly feel there should be an established, physical setup that the release 
team have access to in order to validate a release.

This is perhaps something that should be requested from the user community.
I would expect that anyone running Cloudstack in production on a major site 
would have a test setup and might be very happy to have the dev team test on 
their setup.
This would save them a lot of resources at the expense of a bit of time on 
their test environment.


If this was some random cat meme generator on GitHub, I'd accept the risk in 
running an untested version. For something I'd be running my business on 
however I'd expect there being some weight behind the release.

Perhaps I have unrealistic expectations!

Not at all.
Your expectations might be the key to making a pitch to the user community f

RE: [DISCUSS] - Releases, Project Management & Funding Thereof

[Paul Angus]

Taken from a talk on CloudStack testing [1]...

There are Many, many, MANY permutations of a CloudStack deployment…. 
• Basic / Advanced 
• Local / shared / mixed storage 
• More than 8 common hypervisor types/versions 
• 4 or 5 Management server OS possibilities 
• That’s 144 combinations only looking the basics.

[1] 
https://www.slideshare.net/ShapeBlue/cloudstack-eu-user-group-trillian?qid=74ff2be0-664c-4bca-a3dc-f30d880ca088&v=&b=&from_search=1

Trillian [2], can create any of those, and multiple at the same time, but the 
amount of hardware required to do that means that we have to pick our battles. 
Running the blueorangutan bot command '@blueorangutan matrix' in a PR will run 
the smoke test suite against a PR using 3 environments, one each of KVM, 
XenServer and vSphere and takes around 8 hours.

But that is only looking for major regressions.  A full component test run 
takes around 5 days to run and is riddled with bugs in the tests. 

Ultimately these are still of limited scope, few people are as diligent as say 
Mike T in creating practical marvin tests for their code / features.

[2] https://github.com/shapeblue/Trillian

Therefore we need hardware to run tests on, but more importantly we need the 
tests to exist and work in the first place.  Then we can really do something.



paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: Alex Hitchins [mailto:a...@alexhitchins.com] 
Sent: 30 June 2017 21:34
To: dev@cloudstack.apache.org; dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

Consultation with users is something that should definite be done. Canvas as 
many as possible.

I agree that most people will be running test environments before full rollout 
of any technology, I guess see it a little from a CTO eyes - why shortlist a 
technology that doesn't even endorse its own releases?

Hopefully we will get some more replies to this thread from other CloudStack 
enthusiasts to help shape this conversation.

I'm setting up a new development environment now to get my hands mildly soiled. 
Going the Windows route again. Fancy a challenge for the weekend.




Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Ron Wheeler [mailto:rwhee...@artifact-software.com] 
Sent: 30 June 2017 21:08
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof


On 30/06/2017 3:28 PM, Alex Hitchins wrote:
> We can't validate all scenarios no, hence suggesting several common setups as 
> a reasonable baseline. I like the idea of users posting their hardware and 
> versions as a community endeavour.
>
> I strongly feel there should be an established, physical setup that the 
> release team have access to in order to validate a release.

This is perhaps something that should be requested from the user community.
I would expect that anyone running Cloudstack in production on a major site 
would have a test setup and might be very happy to have the dev team test on 
their setup.
This would save them a lot of resources at the expense of a bit of time on 
their test environment.

> If this was some random cat meme generator on GitHub, I'd accept the risk in 
> running an untested version. For something I'd be running my business on 
> however I'd expect there being some weight behind the release.
>
> Perhaps I have unrealistic expectations!

Not at all.
Your expectations might be the key to making a pitch to the user community for 
some help from people and organizations that are not interested in writing code 
but have a major interest in testing.
In addition to access to test equipment, this might actually get new people on 
the team with the right skills required to extend the test scripts and test 
procedure documentation.

Does anyone have a list of the configuration specifications that are required 
to test a new release?

Would it help to approach major users of Cloudstack with a direct request for 
use of their test equipment and QA staff in return for early access to new 
releases and testing on their hardware?

Ron

>
> Alexander Hitchins
> 
> E: a...@alexhitchins.com
> W: alexhitchins.com
> M: 07788 423 969
> T: 01892 523 587
>
> -Original Message-
> From: Ron Wheeler [mailto:rwhee...@artifact-software.com]
> Sent: 30 June 2017 20:13
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] - Releases, Project Management & Funding 
> Thereof
>
> On 30/06/2017 2:19 PM, Alex Hitchins wrote:
>> Releasing against a defined reference rig would be a very good idea, 
>> especially if we could replicate several.
>>
>> It concerns

RE: [DISCUSS] - Releases, Project Management & Funding Thereof

[Alex Hitchins]

Consultation with users is something that should definite be done. Canvas as 
many as possible.

I agree that most people will be running test environments before full rollout 
of any technology, I guess see it a little from a CTO eyes - why shortlist a 
technology that doesn't even endorse its own releases?

Hopefully we will get some more replies to this thread from other CloudStack 
enthusiasts to help shape this conversation.

I'm setting up a new development environment now to get my hands mildly soiled. 
Going the Windows route again. Fancy a challenge for the weekend.




Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Ron Wheeler [mailto:rwhee...@artifact-software.com] 
Sent: 30 June 2017 21:08
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof


On 30/06/2017 3:28 PM, Alex Hitchins wrote:
> We can't validate all scenarios no, hence suggesting several common setups as 
> a reasonable baseline. I like the idea of users posting their hardware and 
> versions as a community endeavour.
>
> I strongly feel there should be an established, physical setup that the 
> release team have access to in order to validate a release.

This is perhaps something that should be requested from the user community.
I would expect that anyone running Cloudstack in production on a major site 
would have a test setup and might be very happy to have the dev team test on 
their setup.
This would save them a lot of resources at the expense of a bit of time on 
their test environment.

> If this was some random cat meme generator on GitHub, I'd accept the risk in 
> running an untested version. For something I'd be running my business on 
> however I'd expect there being some weight behind the release.
>
> Perhaps I have unrealistic expectations!

Not at all.
Your expectations might be the key to making a pitch to the user community for 
some help from people and organizations that are not interested in writing code 
but have a major interest in testing.
In addition to access to test equipment, this might actually get new people on 
the team with the right skills required to extend the test scripts and test 
procedure documentation.

Does anyone have a list of the configuration specifications that are required 
to test a new release?

Would it help to approach major users of Cloudstack with a direct request for 
use of their test equipment and QA staff in return for early access to new 
releases and testing on their hardware?

Ron

>
> Alexander Hitchins
> 
> E: a...@alexhitchins.com
> W: alexhitchins.com
> M: 07788 423 969
> T: 01892 523 587
>
> -Original Message-
> From: Ron Wheeler [mailto:rwhee...@artifact-software.com]
> Sent: 30 June 2017 20:13
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] - Releases, Project Management & Funding 
> Thereof
>
> On 30/06/2017 2:19 PM, Alex Hitchins wrote:
>> Releasing against a defined reference rig would be a very good idea, 
>> especially if we could replicate several.
>>
>> It concerns me slightly that we are building a platform we want to promote 
>> people to deploy in enterprise environments with the caveat 'run at your own 
>> risk'.
> There is no choice as near as I can tell.
> It seems that there are too many combinations of hardware, network 
> configurations and OSs to guarantee that a release will work on all of them 
> and still get a release delivered.
> As Will pointed out, the Release Team does not have access to every 
> combination where previous releases are in production use, to test the new 
> release candidate.
>
> Currently it may be  not very explicit about what are the fully tested 
> configurations and from what Will said, I gather that there is no policy 
> saying what the minimum test set is to declare a release ready to go.
>
> There is no reason preventing a release being tested after release by an 
> end-user or a developer and adding to the release documentation something to 
> the effect that "Users have reported that this release has been put into 
> production on XYZ configuration with no modifications."
> This at least gets the release out the door for the 95% of the users that do 
> not have an XYZ rather than waiting for someone with an XYZ to find time to 
> test it.
>
> It may also encourage companies using or selling XYZs to put up some 
> resources (hardware and people) dedicated to testing so that they get into 
> the initial release.
>
> Ron
>
>> We need to up our game.
>>
>> 'We' he says, after two years MIA!
>>
>>> On 30 Jun 2017, at 18:41, Ron Wheeler 

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Ron Wheeler]

On 30/06/2017 3:28 PM, Alex Hitchins wrote:

We can't validate all scenarios no, hence suggesting several common setups as a 
reasonable baseline. I like the idea of users posting their hardware and 
versions as a community endeavour.

I strongly feel there should be an established, physical setup that the release 
team have access to in order to validate a release.


This is perhaps something that should be requested from the user community.
I would expect that anyone running Cloudstack in production on a major 
site would have a test setup and might be very happy to have the dev 
team test on their setup.
This would save them a lot of resources at the expense of a bit of time 
on their test environment.



If this was some random cat meme generator on GitHub, I'd accept the risk in 
running an untested version. For something I'd be running my business on 
however I'd expect there being some weight behind the release.

Perhaps I have unrealistic expectations!


Not at all.
Your expectations might be the key to making a pitch to the user 
community for some help from people and organizations that are not 
interested in writing code but have a major interest in testing.
In addition to access to test equipment, this might actually get new 
people on the team with the right skills required to extend the test 
scripts and test procedure documentation.


Does anyone have a list of the configuration specifications that are 
required to test a new release?


Would it help to approach major users of Cloudstack with a direct 
request for use of their test equipment and QA staff in return for early 
access to new releases and testing on their hardware?


Ron



Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Ron Wheeler [mailto:rwhee...@artifact-software.com]
Sent: 30 June 2017 20:13
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof

On 30/06/2017 2:19 PM, Alex Hitchins wrote:

Releasing against a defined reference rig would be a very good idea, especially 
if we could replicate several.

It concerns me slightly that we are building a platform we want to promote 
people to deploy in enterprise environments with the caveat 'run at your own 
risk'.

There is no choice as near as I can tell.
It seems that there are too many combinations of hardware, network 
configurations and OSs to guarantee that a release will work on all of them and 
still get a release delivered.
As Will pointed out, the Release Team does not have access to every combination 
where previous releases are in production use, to test the new release 
candidate.

Currently it may be  not very explicit about what are the fully tested 
configurations and from what Will said, I gather that there is no policy saying 
what the minimum test set is to declare a release ready to go.

There is no reason preventing a release being tested after release by an end-user or a 
developer and adding to the release documentation something to the effect that 
"Users have reported that this release has been put into production on XYZ 
configuration with no modifications."
This at least gets the release out the door for the 95% of the users that do 
not have an XYZ rather than waiting for someone with an XYZ to find time to 
test it.

It may also encourage companies using or selling XYZs to put up some resources 
(hardware and people) dedicated to testing so that they get into the initial 
release.

Ron


We need to up our game.

'We' he says, after two years MIA!


On 30 Jun 2017, at 18:41, Ron Wheeler  wrote:

How much time is there between a feature freeze and the RC being cut.?
Do people know far enough in advance that their feature is in or out and if in 
must be ready to go to a RC release by such and such a date?

Is the use case testing well defined - hardware, configurations, etc.
Can you put out a release that says: "This release has been tested on these 
configurations (A, B ,C) but the following configurations/use cases are not yet fully 
tested and other configuration may be used at your own risk after your own internal tests 
have been run successfully."
Is there any concept that "Cloudstack is verified to run on the following 
configurations and should also run on these configurations but has not been tested fully. 
It may run on these configurations but is not tested during the release cycle."

Ron


On 30/06/2017 1:14 PM, Will Stevens wrote:
Have not looked at Release Tsar, but worth checking out.

In general, the biggest problem we have with releasing on a schedule
is the lack of a CI setup which covers the entire software. Or at
least a 'supported' set of features. This means that the release is
always bound to a bunch of volunteers getting around to testing
their use case. Solidfire and Nuage are pretty good

RE: [DISCUSS] - Releases, Project Management & Funding Thereof

[Alex Hitchins]

We can't validate all scenarios no, hence suggesting several common setups as a 
reasonable baseline. I like the idea of users posting their hardware and 
versions as a community endeavour.

I strongly feel there should be an established, physical setup that the release 
team have access to in order to validate a release.

If this was some random cat meme generator on GitHub, I'd accept the risk in 
running an untested version. For something I'd be running my business on 
however I'd expect there being some weight behind the release.

Perhaps I have unrealistic expectations!


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Ron Wheeler [mailto:rwhee...@artifact-software.com] 
Sent: 30 June 2017 20:13
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof

On 30/06/2017 2:19 PM, Alex Hitchins wrote:
> Releasing against a defined reference rig would be a very good idea, 
> especially if we could replicate several.
>
> It concerns me slightly that we are building a platform we want to promote 
> people to deploy in enterprise environments with the caveat 'run at your own 
> risk'.
There is no choice as near as I can tell.
It seems that there are too many combinations of hardware, network 
configurations and OSs to guarantee that a release will work on all of them and 
still get a release delivered.
As Will pointed out, the Release Team does not have access to every combination 
where previous releases are in production use, to test the new release 
candidate.

Currently it may be  not very explicit about what are the fully tested 
configurations and from what Will said, I gather that there is no policy saying 
what the minimum test set is to declare a release ready to go.

There is no reason preventing a release being tested after release by an 
end-user or a developer and adding to the release documentation something to 
the effect that "Users have reported that this release has been put into 
production on XYZ configuration with no modifications." 
This at least gets the release out the door for the 95% of the users that do 
not have an XYZ rather than waiting for someone with an XYZ to find time to 
test it.

It may also encourage companies using or selling XYZs to put up some resources 
(hardware and people) dedicated to testing so that they get into the initial 
release.

Ron

>
> We need to up our game.
>
> 'We' he says, after two years MIA!
>
>> On 30 Jun 2017, at 18:41, Ron Wheeler  wrote:
>>
>> How much time is there between a feature freeze and the RC being cut.?
>> Do people know far enough in advance that their feature is in or out and if 
>> in must be ready to go to a RC release by such and such a date?
>>
>> Is the use case testing well defined - hardware, configurations, etc.
>> Can you put out a release that says: "This release has been tested on these 
>> configurations (A, B ,C) but the following configurations/use cases are not 
>> yet fully tested and other configuration may be used at your own risk after 
>> your own internal tests have been run successfully."
>> Is there any concept that "Cloudstack is verified to run on the following 
>> configurations and should also run on these configurations but has not been 
>> tested fully. It may run on these configurations but is not tested during 
>> the release cycle."
>>
>> Ron
>>
>>> On 30/06/2017 1:14 PM, Will Stevens wrote:
>>> Have not looked at Release Tsar, but worth checking out.
>>>
>>> In general, the biggest problem we have with releasing on a schedule 
>>> is the lack of a CI setup which covers the entire software. Or at 
>>> least a 'supported' set of features. This means that the release is 
>>> always bound to a bunch of volunteers getting around to testing 
>>> their use case. Solidfire and Nuage are pretty good about getting some CI 
>>> run on some pieces.
>>> Trillian is great for covering a portion of the tests, but it 
>>> currently does not cover the whole software use case. We also need 
>>> more trillian deployments in the wild to support the CI initiative.
>>>
>>> We do need to be stricter about nothing going in after an RC is cut 
>>> but blockers. The limited CI coverage and the dependence on a few 
>>> people for testing exasperates this problem.
>>>
>>> So there is multiple layers to this. I think someone dedicates to 
>>> the RM role would help this a lot because they would have a single 
>>> community focus mandate, so it is in their best interest to 
>&g

Re: ***UNCHECKED*** Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Ron Wheeler]

Rewritten to have sentences that parse into some understandable English.


If the plan is to do several releases each year, something has to change 
in the process.


Any idea about what tasks took up most of the time?

Were there any specific issues that ate up more time than you expected.

Would breaking Cloudstack into separately modules that are separately 
released by different teams make things more manageable?
At some point, one would expect that the APIs would get stabilized so 
that modules could be upgraded without affecting the whole system.


Obviously some changes would require mods to more than one module but 
depending on how one defines the releasable packages, many changes 
should not.


This may get into the case where the current releases only supports part 
of the functionality that the end user needs and they may have to wait a 
week or 2 to find a set of packages that fully supports their particular 
case.
However in the meantime, new functionality can be released to the rest 
of the user community that does not need this case.


This would allow bug fix releases to get out the door quicker if they 
only affected one module.
It would also reduce the testing of each release by a lot and might make 
tests to be more complete on key areas.


It might help users get new functionality into production if they are 
only upgrade part of the system at one time. I would expect a lot less 
testing to be required if only the admin user interface is changing.


It might also expand the dev community as people with interests limited 
to one area (UI, networking hardware) might feel sufficiently 
knowledgeable to contribute.



Ron


On 30/06/2017 1:48 PM, Will Stevens wrote:

I am not doing much right now because our company has many other things on
the go.

For about the first 6 months of 2016 CloudOps donated my time full time to
act as the release manager of 4.9. That is not something we or I can
sustain. Which is part of the problem.

On Jun 30, 2017 1:28 PM, "Ron Wheeler" 
wrote:


How many companies are funding staff now to work on Cloudstack? How much
time?
How many FTEs does that come to if one adds it all up?

It is harder to get people who are working on their own time to do
administrative tasks on a tight schedule.

If someone is working for a company that is expecting the person to be
doing "cloudstack stuff", it may be possible to convince the company to
dedicate part of that person's time to release management.

A RM doing it all may be harder to fund/organize than a Release Team. Not
all of the tasks have to be done in sequence or by one person.

Ron

On 30/06/2017 1:13 PM, Wido den Hollander wrote:


Op 30 juni 2017 om 18:09 schreef Paul Angus :


We could probably split this topic down also

I think I may have mentioned previously 😊 my view on how we have
somewhat shot ourselves in the foot with the release process this time
around.  I think that for the most part, people have been well intentioned,
and have been trying to 'make this release as good as possible' which is
counter-productive, as it's been introducing new blockers.

True. But still, somebody who dedicated 5 days a week on releases and

keeping track of the project is still very welcome I think.

I'm not sure we have a problem in our 'loosely-agreed' process, it's just

that repeatedly people have ignored it.

I wouldn't say ignore it, but maybe forgotten about the process with all

the best intentions.

WRT a full-time release manager, I suspect that they would find that "you

can lead a horse to water, but you can't make it drink".  They would not be
able to compel anyone to 'hurry up and fix that bug you created', although
I guess maybe they could pull a feature if the author(s) didn't sort it out.

Because ultimately a release manager, paid or otherwise should only be
doing what the 'community' decides the release manager's role is.  So we
need to be clear about how we want releases to work before worrying about
who manages that.

Somebody who reverts a PR or commit to get to a proper release is

probably a good thing. RM is a busy task and done in spare time. That's not
always easy.

Other projects like Ceph have a dedicated RM who is busy the whole week
with just the new release.

We could use such a person, but we would need the funding.

How much would that cost? Well, you need to keep the overhead down. A few
companies donating 10k per year should probably allow you to hire a person.

Wido

Kind regards,

Paul Angus

paul.an...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue


-Original Message-
From: Alex Hitchins [mailto:a...@alexhitchins.com]
Sent: 30 June 2017 15:05
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

I am in complete agreement with you. Also on your other reply 

RE: ***UNCHECKED*** Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Marty Godsey]

"" Would breaking Cloudstack into separately modules that are separately 
released by different teams make things more manageable?""


Please don’t do this. This is basically OpenStack at that point. I would write 
more on this but ZI am on a phone call.. :)  I just wanted say I don’t think 
that is a good idea since the fact that Cloudtsack IS one project is one of its 
strengths..

Regards,
Marty Godsey
Principal Engineer
nSource Solutions, LLC

-Original Message-
From: Ron Wheeler [mailto:rwhee...@artifact-software.com] 
Sent: Friday, June 30, 2017 2:54 PM
To: dev@cloudstack.apache.org
Subject: Re: ***UNCHECKED*** Re: [DISCUSS] - Releases, Project Management & 
Funding Thereof

If the plan is to do several releases each year, something has to change in the 
process.

Any idea about what tasks took up most of the time?

Were there any specific issues that ate up more time than you expected.

Would breaking Cloudstack into separately modules that are separately released 
by different teams make things more manageable?
At some point, one would expect that the APIs would get stabilized so that 
modules could be upgraded without affecting the whole system.
Obviously some changes would require mods to more than one module but depending 
on how one defines the releasable packages, many changes should not.
This may get into the case where part of the current releases on support part 
of the functionality that the end user needs and they may have to wait a week 
or 2 to find a set of packages that fully supports their particular case but in 
the meantime, new functionality can be released to the rest of the user 
community that does not need this case.
This would allow bug fix releases to get out the door quicker if they only 
affected one module.
It would also reduce the testing of each release by a lot and might tests to be 
more complete on key areas.

Ron


On 30/06/2017 1:48 PM, Will Stevens wrote:
> I am not doing much right now because our company has many other 
> things on the go.
>
> For about the first 6 months of 2016 CloudOps donated my time full 
> time to act as the release manager of 4.9. That is not something we or 
> I can sustain. Which is part of the problem.
>
> On Jun 30, 2017 1:28 PM, "Ron Wheeler" 
> 
> wrote:
>
>> How many companies are funding staff now to work on Cloudstack? How 
>> much time?
>> How many FTEs does that come to if one adds it all up?
>>
>> It is harder to get people who are working on their own time to do 
>> administrative tasks on a tight schedule.
>>
>> If someone is working for a company that is expecting the person to 
>> be doing "cloudstack stuff", it may be possible to convince the 
>> company to dedicate part of that person's time to release management.
>>
>> A RM doing it all may be harder to fund/organize than a Release Team. 
>> Not all of the tasks have to be done in sequence or by one person.
>>
>> Ron
>>
>> On 30/06/2017 1:13 PM, Wido den Hollander wrote:
>>
>>> Op 30 juni 2017 om 18:09 schreef Paul Angus :
>>>>
>>>> We could probably split this topic down also
>>>>
>>>> I think I may have mentioned previously 😊 my view on how we have 
>>>> somewhat shot ourselves in the foot with the release process this 
>>>> time around.  I think that for the most part, people have been well 
>>>> intentioned, and have been trying to 'make this release as good as 
>>>> possible' which is counter-productive, as it's been introducing new 
>>>> blockers.
>>>>
>>>> True. But still, somebody who dedicated 5 days a week on releases 
>>>> and
>>> keeping track of the project is still very welcome I think.
>>>
>>> I'm not sure we have a problem in our 'loosely-agreed' process, it's 
>>> just
>>>> that repeatedly people have ignored it.
>>>>
>>>> I wouldn't say ignore it, but maybe forgotten about the process 
>>>> with all
>>> the best intentions.
>>>
>>> WRT a full-time release manager, I suspect that they would find that 
>>> "you
>>>> can lead a horse to water, but you can't make it drink".  They 
>>>> would not be able to compel anyone to 'hurry up and fix that bug 
>>>> you created', although I guess maybe they could pull a feature if the 
>>>> author(s) didn't sort it out.
>>>>
>>>> Because ultimately a release manager, paid or otherwise should only 
>>>> be doing what the 'community' decides the release manager's role 
>>>>

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Ron Wheeler]

rote:

We could probably split this topic down also

I think I may have mentioned previously 😊 my view on how we have
somewhat shot ourselves in the foot with the release process this time
around.  I think that for the most part, people have been well intentioned,
and have been trying to 'make this release as good as possible' which is
counter-productive, as it's been introducing new blockers.

I'm not sure we have a problem in our 'loosely-agreed' process, it's just
that repeatedly people have ignored it.

WRT a full-time release manager, I suspect that they would find that "you
can lead a horse to water, but you can't make it drink".  They would not be
able to compel anyone to 'hurry up and fix that bug you created', although
I guess maybe they could pull a feature if the author(s) didn't sort it out.

Because ultimately a release manager, paid or otherwise should only be
doing what the 'community' decides the release manager's role is.  So we
need to be clear about how we want releases to work before worrying about
who manages that.

Kind regards,

Paul Angus

paul.an...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue


-Original Message-
From: Alex Hitchins [mailto:a...@alexhitchins.com]
Sent: 30 June 2017 15:05
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

I am in complete agreement with you. Also on your other reply regards to
a FT release manager.

If 'we' don't go down this line, more and more people will follow the
Cosmic/Schuberg Philis path or even use Cosmic instead.

I'm encouraged by your response. Sounds like a few others hold the same
concerns.


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-----Original Message-----
From: Will Stevens [mailto:williamstev...@gmail.com]
Sent: 30 June 2017 14:54
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

Yes, Schuberg Philis, a very active community member forked Cosmic off of
CloudStack and has been developing their fork for their needs.

I do think we need to have a more consistent front on this matter. I
think it would make a big difference on the quality, release cadence and
perception of the project.




On Jun 30, 2017 9:48 AM, "Alex Hitchins"  wrote:

Thanks Will,

I understand it's something that comes with a big bag of troublesome
worries.

If this topic comes up again in any discussions, I'd be interested to
hear their thoughts on what I see as the alternative; without a dedicated
RM/PM/Captain, people will fork off CS so they can achieve the same thing,
and CS ultimately looses out long term. I can't remember the name of the
fork, but I think I'm right that a previous large CS contributor/user
forked off as they wanted greater management in the areas we are discussing
here.


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Will Stevens [mailto:williamstev...@gmail.com]
Sent: 30 June 2017 14:31
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof

Apache has been historically against the idea of a cloudstack foundation
and there is a bit of a pandoras box there which we will want to be careful
about opening.

Apache added direct contribution, but it was unusable for us historically
because it required a minimum contribution of 50k, which none of us can
afford. However, there have been some changes to the board recently which
are in our favour if we want to put pressure to lower that to say 5-10k.

Even if we do solve for smaller direct contributions, we will have to
jump through hoops to be able to use those funds for a dedicated release
manager. I do think this is a possibility if we manage our needs and
communications very well. I had some preliminary discussions with some
apache foundation folks to express these specific concerns. I played off
the fact that i know they dont want to entertain a cloudstack foundation
and tried to see if i could get them to move on the direct contribution
mechanism to make it usable for us, specifically with the goal of hiring a
full time release manager. I definitely had their ear and they acknowledged
the problems we are facing (and currently discussing).  They expressed
concerns about being able to hire someone with the direct contributions,
but brainstormed a bit to potentially hire an agency who actually does the
hire and they pay the persons salary through the agency with the direct
contribution funds.

All to say, there are potential options here, but there be dragons, so we
have to handle this topic with care.

On Jun 30, 2017 9:12 AM, "Ron Wheeler" 
wrot

Re: ***UNCHECKED*** Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Ron Wheeler]

If the plan is to do several releases each year, something has to change 
in the process.


Any idea about what tasks took up most of the time?

Were there any specific issues that ate up more time than you expected.

Would breaking Cloudstack into separately modules that are separately 
released by different teams make things more manageable?
At some point, one would expect that the APIs would get stabilized so 
that modules could be upgraded without affecting the whole system.
Obviously some changes would require mods to more than one module but 
depending on how one defines the releasable packages, many changes 
should not.
This may get into the case where part of the current releases on support 
part of the functionality that the end user needs and they may have to 
wait a week or 2 to find a set of packages that fully supports their 
particular case but in the meantime, new functionality can be released 
to the rest of the user community that does not need this case.
This would allow bug fix releases to get out the door quicker if they 
only affected one module.
It would also reduce the testing of each release by a lot and might 
tests to be more complete on key areas.


Ron


On 30/06/2017 1:48 PM, Will Stevens wrote:

I am not doing much right now because our company has many other things on
the go.

For about the first 6 months of 2016 CloudOps donated my time full time to
act as the release manager of 4.9. That is not something we or I can
sustain. Which is part of the problem.

On Jun 30, 2017 1:28 PM, "Ron Wheeler" 
wrote:


How many companies are funding staff now to work on Cloudstack? How much
time?
How many FTEs does that come to if one adds it all up?

It is harder to get people who are working on their own time to do
administrative tasks on a tight schedule.

If someone is working for a company that is expecting the person to be
doing "cloudstack stuff", it may be possible to convince the company to
dedicate part of that person's time to release management.

A RM doing it all may be harder to fund/organize than a Release Team. Not
all of the tasks have to be done in sequence or by one person.

Ron

On 30/06/2017 1:13 PM, Wido den Hollander wrote:


Op 30 juni 2017 om 18:09 schreef Paul Angus :


We could probably split this topic down also

I think I may have mentioned previously 😊 my view on how we have
somewhat shot ourselves in the foot with the release process this time
around.  I think that for the most part, people have been well intentioned,
and have been trying to 'make this release as good as possible' which is
counter-productive, as it's been introducing new blockers.

True. But still, somebody who dedicated 5 days a week on releases and

keeping track of the project is still very welcome I think.

I'm not sure we have a problem in our 'loosely-agreed' process, it's just

that repeatedly people have ignored it.

I wouldn't say ignore it, but maybe forgotten about the process with all

the best intentions.

WRT a full-time release manager, I suspect that they would find that "you

can lead a horse to water, but you can't make it drink".  They would not be
able to compel anyone to 'hurry up and fix that bug you created', although
I guess maybe they could pull a feature if the author(s) didn't sort it out.

Because ultimately a release manager, paid or otherwise should only be
doing what the 'community' decides the release manager's role is.  So we
need to be clear about how we want releases to work before worrying about
who manages that.

Somebody who reverts a PR or commit to get to a proper release is

probably a good thing. RM is a busy task and done in spare time. That's not
always easy.

Other projects like Ceph have a dedicated RM who is busy the whole week
with just the new release.

We could use such a person, but we would need the funding.

How much would that cost? Well, you need to keep the overhead down. A few
companies donating 10k per year should probably allow you to hire a person.

Wido

Kind regards,

Paul Angus

paul.an...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue


-Original Message-
From: Alex Hitchins [mailto:a...@alexhitchins.com]
Sent: 30 June 2017 15:05
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

I am in complete agreement with you. Also on your other reply regards to
a FT release manager.

If 'we' don't go down this line, more and more people will follow the
Cosmic/Schuberg Philis path or even use Cosmic instead.

I'm encouraged by your response. Sounds like a few others hold the same
concerns.


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Will Stevens [mailto:williamstev...@gma

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Alex Hitchins]

 sure we have a problem in our 'loosely-agreed' process, it's just
>>>> that repeatedly people have ignored it.
>>>> 
>>>> WRT a full-time release manager, I suspect that they would find that "you
>>>> can lead a horse to water, but you can't make it drink".  They would not be
>>>> able to compel anyone to 'hurry up and fix that bug you created', although
>>>> I guess maybe they could pull a feature if the author(s) didn't sort it 
>>>> out.
>>>> 
>>>> Because ultimately a release manager, paid or otherwise should only be
>>>> doing what the 'community' decides the release manager's role is.  So we
>>>> need to be clear about how we want releases to work before worrying about
>>>> who manages that.
>>>> 
>>>> Kind regards,
>>>> 
>>>> Paul Angus
>>>> 
>>>> paul.an...@shapeblue.com
>>>> www.shapeblue.com
>>>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>>>> @shapeblue
>>>> 
>>>> 
>>>> -Original Message-
>>>> From: Alex Hitchins [mailto:a...@alexhitchins.com]
>>>> Sent: 30 June 2017 15:05
>>>> To: dev@cloudstack.apache.org
>>>> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>>>> 
>>>> I am in complete agreement with you. Also on your other reply regards to
>>>> a FT release manager.
>>>> 
>>>> If 'we' don't go down this line, more and more people will follow the
>>>> Cosmic/Schuberg Philis path or even use Cosmic instead.
>>>> 
>>>> I'm encouraged by your response. Sounds like a few others hold the same
>>>> concerns.
>>>> 
>>>> 
>>>> Alexander Hitchins
>>>> 
>>>> E: a...@alexhitchins.com
>>>> W: alexhitchins.com
>>>> M: 07788 423 969
>>>> T: 01892 523 587
>>>> 
>>>> -Original Message-
>>>> From: Will Stevens [mailto:williamstev...@gmail.com]
>>>> Sent: 30 June 2017 14:54
>>>> To: dev@cloudstack.apache.org
>>>> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>>>> 
>>>> Yes, Schuberg Philis, a very active community member forked Cosmic off of
>>>> CloudStack and has been developing their fork for their needs.
>>>> 
>>>> I do think we need to have a more consistent front on this matter. I
>>>> think it would make a big difference on the quality, release cadence and
>>>> perception of the project.
>>>> 
>>>> 
>>>> 
>>>> 
>>>> On Jun 30, 2017 9:48 AM, "Alex Hitchins"  wrote:
>>>> 
>>>> Thanks Will,
>>>> 
>>>> I understand it's something that comes with a big bag of troublesome
>>>> worries.
>>>> 
>>>> If this topic comes up again in any discussions, I'd be interested to
>>>> hear their thoughts on what I see as the alternative; without a dedicated
>>>> RM/PM/Captain, people will fork off CS so they can achieve the same thing,
>>>> and CS ultimately looses out long term. I can't remember the name of the
>>>> fork, but I think I'm right that a previous large CS contributor/user
>>>> forked off as they wanted greater management in the areas we are discussing
>>>> here.
>>>> 
>>>> 
>>>> Alexander Hitchins
>>>> 
>>>> E: a...@alexhitchins.com
>>>> W: alexhitchins.com
>>>> M: 07788 423 969
>>>> T: 01892 523 587
>>>> 
>>>> -Original Message-
>>>> From: Will Stevens [mailto:williamstev...@gmail.com]
>>>> Sent: 30 June 2017 14:31
>>>> To: dev@cloudstack.apache.org
>>>> Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof
>>>> 
>>>> Apache has been historically against the idea of a cloudstack foundation
>>>> and there is a bit of a pandoras box there which we will want to be careful
>>>> about opening.
>>>> 
>>>> Apache added direct contribution, but it was unusable for us historically
>>>> because it required a minimum contribution of 50k, which none of us can
>>>> afford. However, there have been some change

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Ron Wheeler]

How much time is there between a feature freeze and the RC being cut.?
Do people know far enough in advance that their feature is in or out and 
if in must be ready to go to a RC release by such and such a date?


Is the use case testing well defined - hardware, configurations, etc.
Can you put out a release that says: "This release has been tested on 
these configurations (A, B ,C) but the following configurations/use 
cases are not yet fully tested and other configuration may be used at 
your own risk after your own internal tests have been run successfully."
Is there any concept that "Cloudstack is verified to run on the 
following configurations and should also run on these configurations but 
has not been tested fully. It may run on these configurations but is not 
tested during the release cycle."


Ron

On 30/06/2017 1:14 PM, Will Stevens wrote:

Have not looked at Release Tsar, but worth checking out.

In general, the biggest problem we have with releasing on a schedule is the
lack of a CI setup which covers the entire software. Or at least a
'supported' set of features. This means that the release is always bound to
a bunch of volunteers getting around to testing their use case. Solidfire
and Nuage are pretty good about getting some CI run on some pieces.
Trillian is great for covering a portion of the tests, but it currently
does not cover the whole software use case. We also need more trillian
deployments in the wild to support the CI initiative.

We do need to be stricter about nothing going in after an RC is cut but
blockers. The limited CI coverage and the dependence on a few people for
testing exasperates this problem.

So there is multiple layers to this. I think someone dedicates to the RM
role would help this a lot because they would have a single community focus
mandate, so it is in their best interest to implement a flow which does not
inhibit their ability to deliver on their mandate.

On Jun 30, 2017 12:53 PM, "Ron Wheeler" 
wrote:


Perhaps a Release Tsar would be a better solution.
The RM needs to have absolute control over what is in or out.
Reasonable discussion allowed and then a decision once the RM feels that
the case has been fully explored and that a positive vote is expected.

The importance on meeting deadlines needs to have a higher priority. If a
feature/fix can not meet the quality/testing threshold on time then it gets
dropped from the RC and scheduled for the next release.

A few cycles of a bit of ruthlessness should get everyone`s intention and
shorten the release cycle.

Meeting release schedules would also reduce the pain of a feature being
deferred.
According to the schedule proposed last year,(https://cwiki.apache.org
/confluence/display/CLOUDSTACK/%5BPROPOSAL%5D+2016-2017+
Release+Cycle+and+Calendar)
  Cloudstack 4.9.10 (LTS) , 5.04.0 (LTS) as well as 5.1.3.0 (maintenance)
5.2.1.0 (Maintenance) were released June 2017.

https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+Procedure
seems to be pretty reasonable. The RM probably needs to moderate the vote
and explain what -1 votes mean to product credibility if they delay the
release. Negative votes because someone`s new feature did not make it
should be ignored.

Ron

On 30/06/2017 12:09 PM, Paul Angus wrote:


We could probably split this topic down also

I think I may have mentioned previously 😊 my view on how we have
somewhat shot ourselves in the foot with the release process this time
around.  I think that for the most part, people have been well intentioned,
and have been trying to 'make this release as good as possible' which is
counter-productive, as it's been introducing new blockers.

I'm not sure we have a problem in our 'loosely-agreed' process, it's just
that repeatedly people have ignored it.

WRT a full-time release manager, I suspect that they would find that "you
can lead a horse to water, but you can't make it drink".  They would not be
able to compel anyone to 'hurry up and fix that bug you created', although
I guess maybe they could pull a feature if the author(s) didn't sort it out.

Because ultimately a release manager, paid or otherwise should only be
doing what the 'community' decides the release manager's role is.  So we
need to be clear about how we want releases to work before worrying about
who manages that.

Kind regards,

Paul Angus

paul.an...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue


-Original Message-----
From: Alex Hitchins [mailto:a...@alexhitchins.com]
Sent: 30 June 2017 15:05
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

I am in complete agreement with you. Also on your other reply regards to
a FT release manager.

If 'we' don't go down this line, more and more people will follow the
Cosmic/Schuberg Phil

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Alex Hitchins]

This makes me think of the OpenSSL (I think) that have some 'foundation' where 
there is a paid engineer and donated servers, switches etc to maintain builds 
and tests.

In my opinion, we need the same.

Who out there in a position to comment thinks they would be able to commit some 
budget towards some sort of dedicated role of which everyone benefits. You can 
contact me personally if you want to remain anonymous. Not asking for figures 
at this stage, just general intent.

Alex

> On 30 Jun 2017, at 18:14, Will Stevens  wrote:
> 
> Have not looked at Release Tsar, but worth checking out.
> 
> In general, the biggest problem we have with releasing on a schedule is the
> lack of a CI setup which covers the entire software. Or at least a
> 'supported' set of features. This means that the release is always bound to
> a bunch of volunteers getting around to testing their use case. Solidfire
> and Nuage are pretty good about getting some CI run on some pieces.
> Trillian is great for covering a portion of the tests, but it currently
> does not cover the whole software use case. We also need more trillian
> deployments in the wild to support the CI initiative.
> 
> We do need to be stricter about nothing going in after an RC is cut but
> blockers. The limited CI coverage and the dependence on a few people for
> testing exasperates this problem.
> 
> So there is multiple layers to this. I think someone dedicates to the RM
> role would help this a lot because they would have a single community focus
> mandate, so it is in their best interest to implement a flow which does not
> inhibit their ability to deliver on their mandate.
> 
> On Jun 30, 2017 12:53 PM, "Ron Wheeler" 
> wrote:
> 
>> Perhaps a Release Tsar would be a better solution.
>> The RM needs to have absolute control over what is in or out.
>> Reasonable discussion allowed and then a decision once the RM feels that
>> the case has been fully explored and that a positive vote is expected.
>> 
>> The importance on meeting deadlines needs to have a higher priority. If a
>> feature/fix can not meet the quality/testing threshold on time then it gets
>> dropped from the RC and scheduled for the next release.
>> 
>> A few cycles of a bit of ruthlessness should get everyone`s intention and
>> shorten the release cycle.
>> 
>> Meeting release schedules would also reduce the pain of a feature being
>> deferred.
>> According to the schedule proposed last year,(https://cwiki.apache.org
>> /confluence/display/CLOUDSTACK/%5BPROPOSAL%5D+2016-2017+
>> Release+Cycle+and+Calendar)
>> Cloudstack 4.9.10 (LTS) , 5.04.0 (LTS) as well as 5.1.3.0 (maintenance)
>> 5.2.1.0 (Maintenance) were released June 2017.
>> 
>> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+Procedure
>> seems to be pretty reasonable. The RM probably needs to moderate the vote
>> and explain what -1 votes mean to product credibility if they delay the
>> release. Negative votes because someone`s new feature did not make it
>> should be ignored.
>> 
>> Ron
>> 
>>> On 30/06/2017 12:09 PM, Paul Angus wrote:
>>> 
>>> We could probably split this topic down also
>>> 
>>> I think I may have mentioned previously 😊 my view on how we have
>>> somewhat shot ourselves in the foot with the release process this time
>>> around.  I think that for the most part, people have been well intentioned,
>>> and have been trying to 'make this release as good as possible' which is
>>> counter-productive, as it's been introducing new blockers.
>>> 
>>> I'm not sure we have a problem in our 'loosely-agreed' process, it's just
>>> that repeatedly people have ignored it.
>>> 
>>> WRT a full-time release manager, I suspect that they would find that "you
>>> can lead a horse to water, but you can't make it drink".  They would not be
>>> able to compel anyone to 'hurry up and fix that bug you created', although
>>> I guess maybe they could pull a feature if the author(s) didn't sort it out.
>>> 
>>> Because ultimately a release manager, paid or otherwise should only be
>>> doing what the 'community' decides the release manager's role is.  So we
>>> need to be clear about how we want releases to work before worrying about
>>> who manages that.
>>> 
>>> Kind regards,
>>> 
>>> Paul Angus
>>> 
>>> paul.an...@shapeblue.com
>>> www.shapeblue.com
>>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
&

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Alex Hitchins]

I know of four who have at least one full time resource dedicated, some with 
considerably more.

These people are working on features and support as this is where the income 
is. 

I feel if the cost of a full time RM was split across n sponsors, it would be a 
massive ROI.

> On 30 Jun 2017, at 18:27, Ron Wheeler  wrote:
> 
> How many companies are funding staff now to work on Cloudstack? How much time?
> How many FTEs does that come to if one adds it all up?
> 
> It is harder to get people who are working on their own time to do 
> administrative tasks on a tight schedule.
> 
> If someone is working for a company that is expecting the person to be doing 
> "cloudstack stuff", it may be possible to convince the company to dedicate 
> part of that person's time to release management.
> 
> A RM doing it all may be harder to fund/organize than a Release Team. Not all 
> of the tasks have to be done in sequence or by one person.
> 
> Ron
> 
> On 30/06/2017 1:13 PM, Wido den Hollander wrote:
>>> Op 30 juni 2017 om 18:09 schreef Paul Angus :
>>> 
>>> 
>>> We could probably split this topic down also
>>> 
>>> I think I may have mentioned previously 😊 my view on how we have somewhat 
>>> shot ourselves in the foot with the release process this time around.  I 
>>> think that for the most part, people have been well intentioned, and have 
>>> been trying to 'make this release as good as possible' which is 
>>> counter-productive, as it's been introducing new blockers.
>>> 
>> True. But still, somebody who dedicated 5 days a week on releases and 
>> keeping track of the project is still very welcome I think.
>> 
>>> I'm not sure we have a problem in our 'loosely-agreed' process, it's just 
>>> that repeatedly people have ignored it.
>>> 
>> I wouldn't say ignore it, but maybe forgotten about the process with all the 
>> best intentions.
>> 
>>> WRT a full-time release manager, I suspect that they would find that "you 
>>> can lead a horse to water, but you can't make it drink".  They would not be 
>>> able to compel anyone to 'hurry up and fix that bug you created', although 
>>> I guess maybe they could pull a feature if the author(s) didn't sort it out.
>>> 
>>> Because ultimately a release manager, paid or otherwise should only be 
>>> doing what the 'community' decides the release manager's role is.  So we 
>>> need to be clear about how we want releases to work before worrying about 
>>> who manages that.
>>> 
>> Somebody who reverts a PR or commit to get to a proper release is probably a 
>> good thing. RM is a busy task and done in spare time. That's not always easy.
>> 
>> Other projects like Ceph have a dedicated RM who is busy the whole week with 
>> just the new release.
>> 
>> We could use such a person, but we would need the funding.
>> 
>> How much would that cost? Well, you need to keep the overhead down. A few 
>> companies donating 10k per year should probably allow you to hire a person.
>> 
>> Wido
>> 
>>> Kind regards,
>>> 
>>> Paul Angus
>>> 
>>> paul.an...@shapeblue.com
>>> www.shapeblue.com
>>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>>> @shapeblue
>>> 
>>> 
>>> -Original Message-
>>> From: Alex Hitchins [mailto:a...@alexhitchins.com]
>>> Sent: 30 June 2017 15:05
>>> To: dev@cloudstack.apache.org
>>> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>>> 
>>> I am in complete agreement with you. Also on your other reply regards to a 
>>> FT release manager.
>>> 
>>> If 'we' don't go down this line, more and more people will follow the 
>>> Cosmic/Schuberg Philis path or even use Cosmic instead.
>>> 
>>> I'm encouraged by your response. Sounds like a few others hold the same 
>>> concerns.
>>> 
>>> 
>>> Alexander Hitchins
>>> 
>>> E: a...@alexhitchins.com
>>> W: alexhitchins.com
>>> M: 07788 423 969
>>> T: 01892 523 587
>>> 
>>> -Original Message-
>>> From: Will Stevens [mailto:williamstev...@gmail.com]
>>> Sent: 30 June 2017 14:54
>>> To: dev@cloudstack.apache.org
>>> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>>> 
>>

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Will Stevens]

I am not doing much right now because our company has many other things on
the go.

For about the first 6 months of 2016 CloudOps donated my time full time to
act as the release manager of 4.9. That is not something we or I can
sustain. Which is part of the problem.

On Jun 30, 2017 1:28 PM, "Ron Wheeler" 
wrote:

> How many companies are funding staff now to work on Cloudstack? How much
> time?
> How many FTEs does that come to if one adds it all up?
>
> It is harder to get people who are working on their own time to do
> administrative tasks on a tight schedule.
>
> If someone is working for a company that is expecting the person to be
> doing "cloudstack stuff", it may be possible to convince the company to
> dedicate part of that person's time to release management.
>
> A RM doing it all may be harder to fund/organize than a Release Team. Not
> all of the tasks have to be done in sequence or by one person.
>
> Ron
>
> On 30/06/2017 1:13 PM, Wido den Hollander wrote:
>
>> Op 30 juni 2017 om 18:09 schreef Paul Angus :
>>>
>>>
>>> We could probably split this topic down also
>>>
>>> I think I may have mentioned previously 😊 my view on how we have
>>> somewhat shot ourselves in the foot with the release process this time
>>> around.  I think that for the most part, people have been well intentioned,
>>> and have been trying to 'make this release as good as possible' which is
>>> counter-productive, as it's been introducing new blockers.
>>>
>>> True. But still, somebody who dedicated 5 days a week on releases and
>> keeping track of the project is still very welcome I think.
>>
>> I'm not sure we have a problem in our 'loosely-agreed' process, it's just
>>> that repeatedly people have ignored it.
>>>
>>> I wouldn't say ignore it, but maybe forgotten about the process with all
>> the best intentions.
>>
>> WRT a full-time release manager, I suspect that they would find that "you
>>> can lead a horse to water, but you can't make it drink".  They would not be
>>> able to compel anyone to 'hurry up and fix that bug you created', although
>>> I guess maybe they could pull a feature if the author(s) didn't sort it out.
>>>
>>> Because ultimately a release manager, paid or otherwise should only be
>>> doing what the 'community' decides the release manager's role is.  So we
>>> need to be clear about how we want releases to work before worrying about
>>> who manages that.
>>>
>>> Somebody who reverts a PR or commit to get to a proper release is
>> probably a good thing. RM is a busy task and done in spare time. That's not
>> always easy.
>>
>> Other projects like Ceph have a dedicated RM who is busy the whole week
>> with just the new release.
>>
>> We could use such a person, but we would need the funding.
>>
>> How much would that cost? Well, you need to keep the overhead down. A few
>> companies donating 10k per year should probably allow you to hire a person.
>>
>> Wido
>>
>> Kind regards,
>>>
>>> Paul Angus
>>>
>>> paul.an...@shapeblue.com
>>> www.shapeblue.com
>>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>>> @shapeblue
>>>
>>>
>>> -Original Message-
>>> From: Alex Hitchins [mailto:a...@alexhitchins.com]
>>> Sent: 30 June 2017 15:05
>>> To: dev@cloudstack.apache.org
>>> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>>>
>>> I am in complete agreement with you. Also on your other reply regards to
>>> a FT release manager.
>>>
>>> If 'we' don't go down this line, more and more people will follow the
>>> Cosmic/Schuberg Philis path or even use Cosmic instead.
>>>
>>> I'm encouraged by your response. Sounds like a few others hold the same
>>> concerns.
>>>
>>>
>>> Alexander Hitchins
>>> 
>>> E: a...@alexhitchins.com
>>> W: alexhitchins.com
>>> M: 07788 423 969
>>> T: 01892 523 587
>>>
>>> -Original Message-
>>> From: Will Stevens [mailto:williamstev...@gmail.com]
>>> Sent: 30 June 2017 14:54
>>> To: dev@cloudstack.apache.org
>>> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>>>
>>> Yes, Schuberg Philis, a very active community member 

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Alex Hitchins]

Thanks Wido.

Sounds like a warm reception from yourself. I will do more work on costs etc. 
Be a fun exercise even if likely proves academic. 



> On 30 Jun 2017, at 18:13, Wido den Hollander  wrote:
> 
> 
>> Op 30 juni 2017 om 18:09 schreef Paul Angus :
>> 
>> 
>> We could probably split this topic down also
>> 
>> I think I may have mentioned previously 😊 my view on how we have somewhat 
>> shot ourselves in the foot with the release process this time around.  I 
>> think that for the most part, people have been well intentioned, and have 
>> been trying to 'make this release as good as possible' which is 
>> counter-productive, as it's been introducing new blockers.
>> 
> 
> True. But still, somebody who dedicated 5 days a week on releases and keeping 
> track of the project is still very welcome I think.
> 
>> I'm not sure we have a problem in our 'loosely-agreed' process, it's just 
>> that repeatedly people have ignored it.
>> 
> 
> I wouldn't say ignore it, but maybe forgotten about the process with all the 
> best intentions.
> 
>> WRT a full-time release manager, I suspect that they would find that "you 
>> can lead a horse to water, but you can't make it drink".  They would not be 
>> able to compel anyone to 'hurry up and fix that bug you created', although I 
>> guess maybe they could pull a feature if the author(s) didn't sort it out.
>> 
>> Because ultimately a release manager, paid or otherwise should only be doing 
>> what the 'community' decides the release manager's role is.  So we need to 
>> be clear about how we want releases to work before worrying about who 
>> manages that.
>> 
> 
> Somebody who reverts a PR or commit to get to a proper release is probably a 
> good thing. RM is a busy task and done in spare time. That's not always easy.
> 
> Other projects like Ceph have a dedicated RM who is busy the whole week with 
> just the new release.
> 
> We could use such a person, but we would need the funding.
> 
> How much would that cost? Well, you need to keep the overhead down. A few 
> companies donating 10k per year should probably allow you to hire a person.
> 
> Wido
> 
>> Kind regards,
>> 
>> Paul Angus
>> 
>> paul.an...@shapeblue.com 
>> www.shapeblue.com
>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> @shapeblue
>> 
>> 
>> 
>> 
>> -Original Message-
>> From: Alex Hitchins [mailto:a...@alexhitchins.com] 
>> Sent: 30 June 2017 15:05
>> To: dev@cloudstack.apache.org
>> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>> 
>> I am in complete agreement with you. Also on your other reply regards to a 
>> FT release manager.
>> 
>> If 'we' don't go down this line, more and more people will follow the 
>> Cosmic/Schuberg Philis path or even use Cosmic instead.
>> 
>> I'm encouraged by your response. Sounds like a few others hold the same 
>> concerns. 
>> 
>> 
>> Alexander Hitchins
>> 
>> E: a...@alexhitchins.com
>> W: alexhitchins.com
>> M: 07788 423 969
>> T: 01892 523 587
>> 
>> -Original Message-
>> From: Will Stevens [mailto:williamstev...@gmail.com]
>> Sent: 30 June 2017 14:54
>> To: dev@cloudstack.apache.org
>> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>> 
>> Yes, Schuberg Philis, a very active community member forked Cosmic off of 
>> CloudStack and has been developing their fork for their needs.
>> 
>> I do think we need to have a more consistent front on this matter. I think 
>> it would make a big difference on the quality, release cadence and 
>> perception of the project.
>> 
>> 
>> 
>> 
>> On Jun 30, 2017 9:48 AM, "Alex Hitchins"  wrote:
>> 
>> Thanks Will,
>> 
>> I understand it's something that comes with a big bag of troublesome worries.
>> 
>> If this topic comes up again in any discussions, I'd be interested to hear 
>> their thoughts on what I see as the alternative; without a dedicated 
>> RM/PM/Captain, people will fork off CS so they can achieve the same thing, 
>> and CS ultimately looses out long term. I can't remember the name of the 
>> fork, but I think I'm right that a previous large CS contributor/user forked 
>> off as they wanted greater management in the areas we are discussing here.
>> 
>> 
>>

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Ron Wheeler]

How many companies are funding staff now to work on Cloudstack? How much 
time?

How many FTEs does that come to if one adds it all up?

It is harder to get people who are working on their own time to do 
administrative tasks on a tight schedule.


If someone is working for a company that is expecting the person to be 
doing "cloudstack stuff", it may be possible to convince the company to 
dedicate part of that person's time to release management.


A RM doing it all may be harder to fund/organize than a Release Team. 
Not all of the tasks have to be done in sequence or by one person.


Ron

On 30/06/2017 1:13 PM, Wido den Hollander wrote:

Op 30 juni 2017 om 18:09 schreef Paul Angus :


We could probably split this topic down also

I think I may have mentioned previously 😊 my view on how we have somewhat shot 
ourselves in the foot with the release process this time around.  I think that 
for the most part, people have been well intentioned, and have been trying to 
'make this release as good as possible' which is counter-productive, as it's 
been introducing new blockers.


True. But still, somebody who dedicated 5 days a week on releases and keeping 
track of the project is still very welcome I think.


I'm not sure we have a problem in our 'loosely-agreed' process, it's just that 
repeatedly people have ignored it.


I wouldn't say ignore it, but maybe forgotten about the process with all the 
best intentions.


WRT a full-time release manager, I suspect that they would find that "you can lead a 
horse to water, but you can't make it drink".  They would not be able to compel 
anyone to 'hurry up and fix that bug you created', although I guess maybe they could pull 
a feature if the author(s) didn't sort it out.

Because ultimately a release manager, paid or otherwise should only be doing 
what the 'community' decides the release manager's role is.  So we need to be 
clear about how we want releases to work before worrying about who manages that.


Somebody who reverts a PR or commit to get to a proper release is probably a 
good thing. RM is a busy task and done in spare time. That's not always easy.

Other projects like Ceph have a dedicated RM who is busy the whole week with 
just the new release.

We could use such a person, but we would need the funding.

How much would that cost? Well, you need to keep the overhead down. A few 
companies donating 10k per year should probably allow you to hire a person.

Wido


Kind regards,

Paul Angus

paul.an...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
   
  



-Original Message-
From: Alex Hitchins [mailto:a...@alexhitchins.com]
Sent: 30 June 2017 15:05
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

I am in complete agreement with you. Also on your other reply regards to a FT 
release manager.

If 'we' don't go down this line, more and more people will follow the 
Cosmic/Schuberg Philis path or even use Cosmic instead.

I'm encouraged by your response. Sounds like a few others hold the same 
concerns.


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Will Stevens [mailto:williamstev...@gmail.com]
Sent: 30 June 2017 14:54
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

Yes, Schuberg Philis, a very active community member forked Cosmic off of 
CloudStack and has been developing their fork for their needs.

I do think we need to have a more consistent front on this matter. I think it 
would make a big difference on the quality, release cadence and perception of 
the project.




On Jun 30, 2017 9:48 AM, "Alex Hitchins"  wrote:

Thanks Will,

I understand it's something that comes with a big bag of troublesome worries.

If this topic comes up again in any discussions, I'd be interested to hear 
their thoughts on what I see as the alternative; without a dedicated 
RM/PM/Captain, people will fork off CS so they can achieve the same thing, and 
CS ultimately looses out long term. I can't remember the name of the fork, but 
I think I'm right that a previous large CS contributor/user forked off as they 
wanted greater management in the areas we are discussing here.


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Will Stevens [mailto:williamstev...@gmail.com]
Sent: 30 June 2017 14:31
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof

Apache has been historically against the idea of a cloudstack foundation and 
there is a bit of a pandoras box there which we will 

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Will Stevens]

Have not looked at Release Tsar, but worth checking out.

In general, the biggest problem we have with releasing on a schedule is the
lack of a CI setup which covers the entire software. Or at least a
'supported' set of features. This means that the release is always bound to
a bunch of volunteers getting around to testing their use case. Solidfire
and Nuage are pretty good about getting some CI run on some pieces.
Trillian is great for covering a portion of the tests, but it currently
does not cover the whole software use case. We also need more trillian
deployments in the wild to support the CI initiative.

We do need to be stricter about nothing going in after an RC is cut but
blockers. The limited CI coverage and the dependence on a few people for
testing exasperates this problem.

So there is multiple layers to this. I think someone dedicates to the RM
role would help this a lot because they would have a single community focus
mandate, so it is in their best interest to implement a flow which does not
inhibit their ability to deliver on their mandate.

On Jun 30, 2017 12:53 PM, "Ron Wheeler" 
wrote:

> Perhaps a Release Tsar would be a better solution.
> The RM needs to have absolute control over what is in or out.
> Reasonable discussion allowed and then a decision once the RM feels that
> the case has been fully explored and that a positive vote is expected.
>
> The importance on meeting deadlines needs to have a higher priority. If a
> feature/fix can not meet the quality/testing threshold on time then it gets
> dropped from the RC and scheduled for the next release.
>
> A few cycles of a bit of ruthlessness should get everyone`s intention and
> shorten the release cycle.
>
> Meeting release schedules would also reduce the pain of a feature being
> deferred.
> According to the schedule proposed last year,(https://cwiki.apache.org
> /confluence/display/CLOUDSTACK/%5BPROPOSAL%5D+2016-2017+
> Release+Cycle+and+Calendar)
>  Cloudstack 4.9.10 (LTS) , 5.04.0 (LTS) as well as 5.1.3.0 (maintenance)
> 5.2.1.0 (Maintenance) were released June 2017.
>
> https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+Procedure
> seems to be pretty reasonable. The RM probably needs to moderate the vote
> and explain what -1 votes mean to product credibility if they delay the
> release. Negative votes because someone`s new feature did not make it
> should be ignored.
>
> Ron
>
> On 30/06/2017 12:09 PM, Paul Angus wrote:
>
>> We could probably split this topic down also
>>
>> I think I may have mentioned previously 😊 my view on how we have
>> somewhat shot ourselves in the foot with the release process this time
>> around.  I think that for the most part, people have been well intentioned,
>> and have been trying to 'make this release as good as possible' which is
>> counter-productive, as it's been introducing new blockers.
>>
>> I'm not sure we have a problem in our 'loosely-agreed' process, it's just
>> that repeatedly people have ignored it.
>>
>> WRT a full-time release manager, I suspect that they would find that "you
>> can lead a horse to water, but you can't make it drink".  They would not be
>> able to compel anyone to 'hurry up and fix that bug you created', although
>> I guess maybe they could pull a feature if the author(s) didn't sort it out.
>>
>> Because ultimately a release manager, paid or otherwise should only be
>> doing what the 'community' decides the release manager's role is.  So we
>> need to be clear about how we want releases to work before worrying about
>> who manages that.
>>
>> Kind regards,
>>
>> Paul Angus
>>
>> paul.an...@shapeblue.com
>> www.shapeblue.com
>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
>> @shapeblue
>>
>>
>> -Original Message-
>> From: Alex Hitchins [mailto:a...@alexhitchins.com]
>> Sent: 30 June 2017 15:05
>> To: dev@cloudstack.apache.org
>> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
>>
>> I am in complete agreement with you. Also on your other reply regards to
>> a FT release manager.
>>
>> If 'we' don't go down this line, more and more people will follow the
>> Cosmic/Schuberg Philis path or even use Cosmic instead.
>>
>> I'm encouraged by your response. Sounds like a few others hold the same
>> concerns.
>>
>>
>> Alexander Hitchins
>> 
>> E: a...@alexhitchins.com
>> W: alexhitchins.com
>> M: 07788 423 969
>> T: 01892 523 587
>>
>> -Original Messa

RE: [DISCUSS] - Releases, Project Management & Funding Thereof

[Wido den Hollander]

> Op 30 juni 2017 om 18:09 schreef Paul Angus :
> 
> 
> We could probably split this topic down also
> 
> I think I may have mentioned previously 😊 my view on how we have somewhat 
> shot ourselves in the foot with the release process this time around.  I 
> think that for the most part, people have been well intentioned, and have 
> been trying to 'make this release as good as possible' which is 
> counter-productive, as it's been introducing new blockers.
> 

True. But still, somebody who dedicated 5 days a week on releases and keeping 
track of the project is still very welcome I think.

> I'm not sure we have a problem in our 'loosely-agreed' process, it's just 
> that repeatedly people have ignored it.
> 

I wouldn't say ignore it, but maybe forgotten about the process with all the 
best intentions.

> WRT a full-time release manager, I suspect that they would find that "you can 
> lead a horse to water, but you can't make it drink".  They would not be able 
> to compel anyone to 'hurry up and fix that bug you created', although I guess 
> maybe they could pull a feature if the author(s) didn't sort it out.
> 
> Because ultimately a release manager, paid or otherwise should only be doing 
> what the 'community' decides the release manager's role is.  So we need to be 
> clear about how we want releases to work before worrying about who manages 
> that.
> 

Somebody who reverts a PR or commit to get to a proper release is probably a 
good thing. RM is a busy task and done in spare time. That's not always easy.

Other projects like Ceph have a dedicated RM who is busy the whole week with 
just the new release.

We could use such a person, but we would need the funding.

How much would that cost? Well, you need to keep the overhead down. A few 
companies donating 10k per year should probably allow you to hire a person.

Wido

> Kind regards,
> 
> Paul Angus
> 
> paul.an...@shapeblue.com 
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK
> @shapeblue
>   
>  
> 
> 
> -Original Message-
> From: Alex Hitchins [mailto:a...@alexhitchins.com] 
> Sent: 30 June 2017 15:05
> To: dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
> 
> I am in complete agreement with you. Also on your other reply regards to a FT 
> release manager.
> 
> If 'we' don't go down this line, more and more people will follow the 
> Cosmic/Schuberg Philis path or even use Cosmic instead.
> 
> I'm encouraged by your response. Sounds like a few others hold the same 
> concerns. 
> 
> 
> Alexander Hitchins
> ----
> E: a...@alexhitchins.com
> W: alexhitchins.com
> M: 07788 423 969
> T: 01892 523 587
> 
> -Original Message-
> From: Will Stevens [mailto:williamstev...@gmail.com]
> Sent: 30 June 2017 14:54
> To: dev@cloudstack.apache.org
> Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof
> 
> Yes, Schuberg Philis, a very active community member forked Cosmic off of 
> CloudStack and has been developing their fork for their needs.
> 
> I do think we need to have a more consistent front on this matter. I think it 
> would make a big difference on the quality, release cadence and perception of 
> the project.
> 
> 
> 
> 
> On Jun 30, 2017 9:48 AM, "Alex Hitchins"  wrote:
> 
> Thanks Will,
> 
> I understand it's something that comes with a big bag of troublesome worries.
> 
> If this topic comes up again in any discussions, I'd be interested to hear 
> their thoughts on what I see as the alternative; without a dedicated 
> RM/PM/Captain, people will fork off CS so they can achieve the same thing, 
> and CS ultimately looses out long term. I can't remember the name of the 
> fork, but I think I'm right that a previous large CS contributor/user forked 
> off as they wanted greater management in the areas we are discussing here.
> 
> 
> Alexander Hitchins
> 
> E: a...@alexhitchins.com
> W: alexhitchins.com
> M: 07788 423 969
> T: 01892 523 587
> 
> -Original Message-
> From: Will Stevens [mailto:williamstev...@gmail.com]
> Sent: 30 June 2017 14:31
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof
> 
> Apache has been historically against the idea of a cloudstack foundation and 
> there is a bit of a pandoras box there which we will want to be careful about 
> opening.
> 
> Apache added direct contribution, but it was unusable for us historically 
> because it 

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Ron Wheeler]

Perhaps a Release Tsar would be a better solution.
The RM needs to have absolute control over what is in or out.
Reasonable discussion allowed and then a decision once the RM feels that 
the case has been fully explored and that a positive vote is expected.


The importance on meeting deadlines needs to have a higher priority. If 
a feature/fix can not meet the quality/testing threshold on time then it 
gets dropped from the RC and scheduled for the next release.


A few cycles of a bit of ruthlessness should get everyone`s intention 
and shorten the release cycle.


Meeting release schedules would also reduce the pain of a feature being 
deferred.
According to the schedule proposed last 
year,(https://cwiki.apache.org/confluence/display/CLOUDSTACK/%5BPROPOSAL%5D+2016-2017+Release+Cycle+and+Calendar)
 Cloudstack 4.9.10 (LTS) , 5.04.0 (LTS) as well as 5.1.3.0 
(maintenance) 5.2.1.0 (Maintenance) were released June 2017.


https://cwiki.apache.org/confluence/display/CLOUDSTACK/Release+Procedure 
seems to be pretty reasonable. The RM probably needs to moderate the 
vote and explain what -1 votes mean to product credibility if they delay 
the release. Negative votes because someone`s new feature did not make 
it should be ignored.


Ron

On 30/06/2017 12:09 PM, Paul Angus wrote:

We could probably split this topic down also

I think I may have mentioned previously 😊 my view on how we have somewhat shot 
ourselves in the foot with the release process this time around.  I think that 
for the most part, people have been well intentioned, and have been trying to 
'make this release as good as possible' which is counter-productive, as it's 
been introducing new blockers.

I'm not sure we have a problem in our 'loosely-agreed' process, it's just that 
repeatedly people have ignored it.

WRT a full-time release manager, I suspect that they would find that "you can lead a 
horse to water, but you can't make it drink".  They would not be able to compel 
anyone to 'hurry up and fix that bug you created', although I guess maybe they could pull 
a feature if the author(s) didn't sort it out.

Because ultimately a release manager, paid or otherwise should only be doing 
what the 'community' decides the release manager's role is.  So we need to be 
clear about how we want releases to work before worrying about who manages that.

Kind regards,

Paul Angus

paul.an...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
   
  



-Original Message-
From: Alex Hitchins [mailto:a...@alexhitchins.com]
Sent: 30 June 2017 15:05
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

I am in complete agreement with you. Also on your other reply regards to a FT 
release manager.

If 'we' don't go down this line, more and more people will follow the 
Cosmic/Schuberg Philis path or even use Cosmic instead.

I'm encouraged by your response. Sounds like a few others hold the same 
concerns.


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Will Stevens [mailto:williamstev...@gmail.com]
Sent: 30 June 2017 14:54
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

Yes, Schuberg Philis, a very active community member forked Cosmic off of 
CloudStack and has been developing their fork for their needs.

I do think we need to have a more consistent front on this matter. I think it 
would make a big difference on the quality, release cadence and perception of 
the project.




On Jun 30, 2017 9:48 AM, "Alex Hitchins"  wrote:

Thanks Will,

I understand it's something that comes with a big bag of troublesome worries.

If this topic comes up again in any discussions, I'd be interested to hear 
their thoughts on what I see as the alternative; without a dedicated 
RM/PM/Captain, people will fork off CS so they can achieve the same thing, and 
CS ultimately looses out long term. I can't remember the name of the fork, but 
I think I'm right that a previous large CS contributor/user forked off as they 
wanted greater management in the areas we are discussing here.


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Will Stevens [mailto:williamstev...@gmail.com]
Sent: 30 June 2017 14:31
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof

Apache has been historically against the idea of a cloudstack foundation and 
there is a bit of a pandoras box there which we will want to be careful about 
opening.

Apache added direct contribution, but it was unusable for us historically 
because it required a m

RE: [DISCUSS] - Releases, Project Management & Funding Thereof

[Paul Angus]

We could probably split this topic down also

I think I may have mentioned previously 😊 my view on how we have somewhat shot 
ourselves in the foot with the release process this time around.  I think that 
for the most part, people have been well intentioned, and have been trying to 
'make this release as good as possible' which is counter-productive, as it's 
been introducing new blockers.

I'm not sure we have a problem in our 'loosely-agreed' process, it's just that 
repeatedly people have ignored it.

WRT a full-time release manager, I suspect that they would find that "you can 
lead a horse to water, but you can't make it drink".  They would not be able to 
compel anyone to 'hurry up and fix that bug you created', although I guess 
maybe they could pull a feature if the author(s) didn't sort it out.

Because ultimately a release manager, paid or otherwise should only be doing 
what the 'community' decides the release manager's role is.  So we need to be 
clear about how we want releases to work before worrying about who manages that.

Kind regards,

Paul Angus

paul.an...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK
@shapeblue
  
 


-Original Message-
From: Alex Hitchins [mailto:a...@alexhitchins.com] 
Sent: 30 June 2017 15:05
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

I am in complete agreement with you. Also on your other reply regards to a FT 
release manager.

If 'we' don't go down this line, more and more people will follow the 
Cosmic/Schuberg Philis path or even use Cosmic instead.

I'm encouraged by your response. Sounds like a few others hold the same 
concerns. 


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Will Stevens [mailto:williamstev...@gmail.com]
Sent: 30 June 2017 14:54
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

Yes, Schuberg Philis, a very active community member forked Cosmic off of 
CloudStack and has been developing their fork for their needs.

I do think we need to have a more consistent front on this matter. I think it 
would make a big difference on the quality, release cadence and perception of 
the project.




On Jun 30, 2017 9:48 AM, "Alex Hitchins"  wrote:

Thanks Will,

I understand it's something that comes with a big bag of troublesome worries.

If this topic comes up again in any discussions, I'd be interested to hear 
their thoughts on what I see as the alternative; without a dedicated 
RM/PM/Captain, people will fork off CS so they can achieve the same thing, and 
CS ultimately looses out long term. I can't remember the name of the fork, but 
I think I'm right that a previous large CS contributor/user forked off as they 
wanted greater management in the areas we are discussing here.


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Will Stevens [mailto:williamstev...@gmail.com]
Sent: 30 June 2017 14:31
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof

Apache has been historically against the idea of a cloudstack foundation and 
there is a bit of a pandoras box there which we will want to be careful about 
opening.

Apache added direct contribution, but it was unusable for us historically 
because it required a minimum contribution of 50k, which none of us can afford. 
However, there have been some changes to the board recently which are in our 
favour if we want to put pressure to lower that to say 5-10k.

Even if we do solve for smaller direct contributions, we will have to jump 
through hoops to be able to use those funds for a dedicated release manager. I 
do think this is a possibility if we manage our needs and communications very 
well. I had some preliminary discussions with some apache foundation folks to 
express these specific concerns. I played off the fact that i know they dont 
want to entertain a cloudstack foundation and tried to see if i could get them 
to move on the direct contribution mechanism to make it usable for us, 
specifically with the goal of hiring a full time release manager. I definitely 
had their ear and they acknowledged the problems we are facing (and currently 
discussing).  They expressed concerns about being able to hire someone with the 
direct contributions, but brainstormed a bit to potentially hire an agency who 
actually does the hire and they pay the persons salary through the agency with 
the direct contribution funds.

All to say, there are potential options here, but there be dragons, so we have 
to handle this topic with care.

On Jun 30, 

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Ron Wheeler]

1) Are there any good models in the Apache community of projects that 
maintain a quality release process without full-time staff?


2) Are there things that cause most of the grief around releases? 
Anything on the Release Manager's list of aggravations that could be 
eliminated?

Is there a good history of moving from Release Candidates to Releases?
Are there tasks that the RM has to do that should be shifted to the 
community?



Ron

Just provide the rest of the joke.
Q: How many psychiatrists does it take to change a lightbulb ?
A: Only one, but the lightbulb must want to change.
A: None; the bulb will change itself when it is ready.
A: How long have you been having this fantasy ?
A: How many do *you* think it takes?

On 30/06/2017 9:53 AM, Will Stevens wrote:

Yes, Schuberg Philis, a very active community member forked Cosmic off of
CloudStack and has been developing their fork for their needs.

I do think we need to have a more consistent front on this matter. I think
it would make a big difference on the quality, release cadence and
perception of the project.




On Jun 30, 2017 9:48 AM, "Alex Hitchins"  wrote:

Thanks Will,

I understand it's something that comes with a big bag of troublesome
worries.

If this topic comes up again in any discussions, I'd be interested to hear
their thoughts on what I see as the alternative; without a dedicated
RM/PM/Captain, people will fork off CS so they can achieve the same thing,
and CS ultimately looses out long term. I can't remember the name of the
fork, but I think I'm right that a previous large CS contributor/user
forked off as they wanted greater management in the areas we are discussing
here.


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Will Stevens [mailto:williamstev...@gmail.com]
Sent: 30 June 2017 14:31
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof

Apache has been historically against the idea of a cloudstack foundation
and there is a bit of a pandoras box there which we will want to be careful
about opening.

Apache added direct contribution, but it was unusable for us historically
because it required a minimum contribution of 50k, which none of us can
afford. However, there have been some changes to the board recently which
are in our favour if we want to put pressure to lower that to say 5-10k.

Even if we do solve for smaller direct contributions, we will have to jump
through hoops to be able to use those funds for a dedicated release
manager. I do think this is a possibility if we manage our needs and
communications very well. I had some preliminary discussions with some
apache foundation folks to express these specific concerns. I played off
the fact that i know they dont want to entertain a cloudstack foundation
and tried to see if i could get them to move on the direct contribution
mechanism to make it usable for us, specifically with the goal of hiring a
full time release manager. I definitely had their ear and they acknowledged
the problems we are facing (and currently discussing).  They expressed
concerns about being able to hire someone with the direct contributions,
but brainstormed a bit to potentially hire an agency who actually does the
hire and they pay the persons salary through the agency with the direct
contribution funds.

All to say, there are potential options here, but there be dragons, so we
have to handle this topic with care.

On Jun 30, 2017 9:12 AM, "Ron Wheeler" 
wrote:


https://www.apache.org/foundation/contributing.html says:
"If you have a specific target or project that you wish to directly
support, pleasecontact us <https://www.apache.org/founda
tion/contributing.html#Fundraising>and we will do our best to satisfy
your wishes."

1) Is Apache willing to allow projects to set up their own
foundations? I doubt but someone would need to check this out.
Does the PMC have the project charter or the agreement that was signed
when Cloudstack moved.

2) Has anyone tried to contact Apache about directing support to
Cloudstack.

I am not convinced that lack of paid staff is the issue.
This discussion reminded me of this.
Q: How many psychiatrists does it take to change a lightbulb ?
A: Only one, but the lightbulb must want to change

http://www.lightbulbjokes.com/directory/p.html


Ron


On 30/06/2017 6:48 AM, Alex Hitchins wrote:


As per Giles's comment to the previous thread, I thought I would
start a discussion on the subject to canvas peoples thoughts, opinions

and fears.

My question for discussion, is there is any mileage in someone
creating a "CloudStack Foundation" as a non-profit entity, funded
largely by key CloudStack players with the sole function of employing
dedicated resource (part or full time) to handle all releases and
other essential 'back office&#x

RE: [DISCUSS] - Releases, Project Management & Funding Thereof

[Alex Hitchins]

I am in complete agreement with you. Also on your other reply regards to a FT 
release manager.

If 'we' don't go down this line, more and more people will follow the 
Cosmic/Schuberg Philis path or even use Cosmic instead.

I'm encouraged by your response. Sounds like a few others hold the same 
concerns. 


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Will Stevens [mailto:williamstev...@gmail.com] 
Sent: 30 June 2017 14:54
To: dev@cloudstack.apache.org
Subject: RE: [DISCUSS] - Releases, Project Management & Funding Thereof

Yes, Schuberg Philis, a very active community member forked Cosmic off of 
CloudStack and has been developing their fork for their needs.

I do think we need to have a more consistent front on this matter. I think it 
would make a big difference on the quality, release cadence and perception of 
the project.




On Jun 30, 2017 9:48 AM, "Alex Hitchins"  wrote:

Thanks Will,

I understand it's something that comes with a big bag of troublesome worries.

If this topic comes up again in any discussions, I'd be interested to hear 
their thoughts on what I see as the alternative; without a dedicated 
RM/PM/Captain, people will fork off CS so they can achieve the same thing, and 
CS ultimately looses out long term. I can't remember the name of the fork, but 
I think I'm right that a previous large CS contributor/user forked off as they 
wanted greater management in the areas we are discussing here.


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Will Stevens [mailto:williamstev...@gmail.com]
Sent: 30 June 2017 14:31
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof

Apache has been historically against the idea of a cloudstack foundation and 
there is a bit of a pandoras box there which we will want to be careful about 
opening.

Apache added direct contribution, but it was unusable for us historically 
because it required a minimum contribution of 50k, which none of us can afford. 
However, there have been some changes to the board recently which are in our 
favour if we want to put pressure to lower that to say 5-10k.

Even if we do solve for smaller direct contributions, we will have to jump 
through hoops to be able to use those funds for a dedicated release manager. I 
do think this is a possibility if we manage our needs and communications very 
well. I had some preliminary discussions with some apache foundation folks to 
express these specific concerns. I played off the fact that i know they dont 
want to entertain a cloudstack foundation and tried to see if i could get them 
to move on the direct contribution mechanism to make it usable for us, 
specifically with the goal of hiring a full time release manager. I definitely 
had their ear and they acknowledged the problems we are facing (and currently 
discussing).  They expressed concerns about being able to hire someone with the 
direct contributions, but brainstormed a bit to potentially hire an agency who 
actually does the hire and they pay the persons salary through the agency with 
the direct contribution funds.

All to say, there are potential options here, but there be dragons, so we have 
to handle this topic with care.

On Jun 30, 2017 9:12 AM, "Ron Wheeler" 
wrote:

>
> https://www.apache.org/foundation/contributing.html says:
> "If you have a specific target or project that you wish to directly 
> support, pleasecontact us <https://www.apache.org/founda 
> tion/contributing.html#Fundraising>and we will do our best to satisfy 
> your wishes."
>
> 1) Is Apache willing to allow projects to set up their own 
> foundations? I doubt but someone would need to check this out.
> Does the PMC have the project charter or the agreement that was signed 
> when Cloudstack moved.
>
> 2) Has anyone tried to contact Apache about directing support to 
> Cloudstack.
>
> I am not convinced that lack of paid staff is the issue.
> This discussion reminded me of this.
> Q: How many psychiatrists does it take to change a lightbulb ?
> A: Only one, but the lightbulb must want to change
>
> http://www.lightbulbjokes.com/directory/p.html
>
>
> Ron
>
>
> On 30/06/2017 6:48 AM, Alex Hitchins wrote:
>
>> As per Giles's comment to the previous thread, I thought I would 
>> start a discussion on the subject to canvas peoples thoughts, 
>> opinions
and fears.
>>
>> My question for discussion, is there is any mileage in someone 
>> creating a "CloudStack Foundation" as a non-profit entity, funded 
>> largely by key CloudStack players with the sole function

RE: [DISCUSS] - Releases, Project Management & Funding Thereof

[Will Stevens]

Yes, Schuberg Philis, a very active community member forked Cosmic off of
CloudStack and has been developing their fork for their needs.

I do think we need to have a more consistent front on this matter. I think
it would make a big difference on the quality, release cadence and
perception of the project.




On Jun 30, 2017 9:48 AM, "Alex Hitchins"  wrote:

Thanks Will,

I understand it's something that comes with a big bag of troublesome
worries.

If this topic comes up again in any discussions, I'd be interested to hear
their thoughts on what I see as the alternative; without a dedicated
RM/PM/Captain, people will fork off CS so they can achieve the same thing,
and CS ultimately looses out long term. I can't remember the name of the
fork, but I think I'm right that a previous large CS contributor/user
forked off as they wanted greater management in the areas we are discussing
here.


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Will Stevens [mailto:williamstev...@gmail.com]
Sent: 30 June 2017 14:31
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof

Apache has been historically against the idea of a cloudstack foundation
and there is a bit of a pandoras box there which we will want to be careful
about opening.

Apache added direct contribution, but it was unusable for us historically
because it required a minimum contribution of 50k, which none of us can
afford. However, there have been some changes to the board recently which
are in our favour if we want to put pressure to lower that to say 5-10k.

Even if we do solve for smaller direct contributions, we will have to jump
through hoops to be able to use those funds for a dedicated release
manager. I do think this is a possibility if we manage our needs and
communications very well. I had some preliminary discussions with some
apache foundation folks to express these specific concerns. I played off
the fact that i know they dont want to entertain a cloudstack foundation
and tried to see if i could get them to move on the direct contribution
mechanism to make it usable for us, specifically with the goal of hiring a
full time release manager. I definitely had their ear and they acknowledged
the problems we are facing (and currently discussing).  They expressed
concerns about being able to hire someone with the direct contributions,
but brainstormed a bit to potentially hire an agency who actually does the
hire and they pay the persons salary through the agency with the direct
contribution funds.

All to say, there are potential options here, but there be dragons, so we
have to handle this topic with care.

On Jun 30, 2017 9:12 AM, "Ron Wheeler" 
wrote:

>
> https://www.apache.org/foundation/contributing.html says:
> "If you have a specific target or project that you wish to directly
> support, pleasecontact us <https://www.apache.org/founda
> tion/contributing.html#Fundraising>and we will do our best to satisfy
> your wishes."
>
> 1) Is Apache willing to allow projects to set up their own
> foundations? I doubt but someone would need to check this out.
> Does the PMC have the project charter or the agreement that was signed
> when Cloudstack moved.
>
> 2) Has anyone tried to contact Apache about directing support to
> Cloudstack.
>
> I am not convinced that lack of paid staff is the issue.
> This discussion reminded me of this.
> Q: How many psychiatrists does it take to change a lightbulb ?
> A: Only one, but the lightbulb must want to change
>
> http://www.lightbulbjokes.com/directory/p.html
>
>
> Ron
>
>
> On 30/06/2017 6:48 AM, Alex Hitchins wrote:
>
>> As per Giles's comment to the previous thread, I thought I would
>> start a discussion on the subject to canvas peoples thoughts, opinions
and fears.
>>
>> My question for discussion, is there is any mileage in someone
>> creating a "CloudStack Foundation" as a non-profit entity, funded
>> largely by key CloudStack players with the sole function of employing
>> dedicated resource (part or full time) to handle all releases and
>> other essential 'back office' functions. The idea being it's in
>> everyone's interest to chip in a little each to fund core project and
release management.
>>
>> The idea might be utterly irrelevant, pointless and/or straight up daft.
>> I urge you all to let me know.
>>
>> Something for you all to think over this weekend.
>>
>>
>> Alexander Hitchins
>> 
>> E: a...@alexhitchins.com
>> W: alexhitchins.com
>> M: 07788 423 969
>> T: 01892 523 587
>>
>> -Original Message--

RE: [DISCUSS] - Releases, Project Management & Funding Thereof

[Will Stevens]

I personally think it is important to have a person working consistently on
the ACS releases. It is a ton of work and with the job rotating right now,
it makes it harder for everyone. That said, no organization can afford to
hire a dedicated RM for year(s) at a time.  I have considered it, but even
if I did, I would depend on other organizations to contribute towards that
person's salary to be able to actually make it work.

On Jun 30, 2017 9:33 AM, "Alex Hitchins"  wrote:

> I'll read those links, thank you for providing them.
>
> Do you think this is a move in the wrong direction or just an unnecessary
> move to begin with?
>
> Alexander Hitchins
> 
> E: a...@alexhitchins.com
> W: alexhitchins.com
> M: 07788 423 969
> T: 01892 523 587
>
> -Original Message-
> From: Ron Wheeler [mailto:rwhee...@artifact-software.com]
> Sent: 30 June 2017 14:06
> To: dev@cloudstack.apache.org
> Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof
>
>
> https://www.apache.org/foundation/contributing.html says:
> "If you have a specific target or project that you wish to directly
> support, pleasecontact us <https://www.apache.org/
> foundation/contributing.html#Fundraising>and we will do our best to
> satisfy your wishes."
>
> 1) Is Apache willing to allow projects to set up their own foundations?
> I doubt but someone would need to check this out.
> Does the PMC have the project charter or the agreement that was signed
> when Cloudstack moved.
>
> 2) Has anyone tried to contact Apache about directing support to
> Cloudstack.
>
> I am not convinced that lack of paid staff is the issue.
> This discussion reminded me of this.
> Q: How many psychiatrists does it take to change a lightbulb ?
> A: Only one, but the lightbulb must want to change
>
> http://www.lightbulbjokes.com/directory/p.html
>
>
> Ron
>
>
> On 30/06/2017 6:48 AM, Alex Hitchins wrote:
> > As per Giles's comment to the previous thread, I thought I would start a
> discussion on the subject to canvas peoples thoughts, opinions and fears.
> >
> > My question for discussion, is there is any mileage in someone creating
> a "CloudStack Foundation" as a non-profit entity, funded largely by key
> CloudStack players with the sole function of employing dedicated resource
> (part or full time) to handle all releases and other essential 'back
> office' functions. The idea being it's in everyone's interest to chip in a
> little each to fund core project and release management.
> >
> > The idea might be utterly irrelevant, pointless and/or straight up daft.
> I urge you all to let me know.
> >
> > Something for you all to think over this weekend.
> >
> >
> > Alexander Hitchins
> > 
> > E: a...@alexhitchins.com
> > W: alexhitchins.com
> > M: 07788 423 969
> > T: 01892 523 587
> >
> > -Original Message-
> > From: Giles Sirett [mailto:giles.sir...@shapeblue.com]
> > Sent: 30 June 2017 09:51
> > To: dev@cloudstack.apache.org
> > Subject: RE: JIRA - PLEASE READ
> >
> > All
> > This thread seems to have turned into 2 quite different discussions:
> >
> > 1. The use (or not) of Jira - which was the original discussion
> >
> > 2. Ways/means of encouraging (and paying for more structured
> > contributors)
> >
> > I know that it could be argued that these are related. Could I suggest
> > opening up a thread on "release and project management and funding it"
> > and keeping this thread to the original discussion
> >
> > (I will weigh in on both of these at some stage)
> >
> > Kind regards
> > Giles
> >
> > giles.sir...@shapeblue.com
> > www.shapeblue.com
> > 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
> >
> >
> >
> >
> > -Original Message-
> > From: Alex Hitchins [mailto:a...@alexhitchins.com]
> > Sent: 29 June 2017 18:49
> > To: dev@cloudstack.apache.org
> > Subject: Re: JIRA - PLEASE READ
> >
> > If it isn't being treated as a product it will be very impossible to
> market it as enterprise ready.
> >
> > I know we all know this.
> >
> > Similar sized projects under the Apache banner must have the same issue,
> what is the best way to gather experience of these projects? See how they
> handle these growing pains.
> >
> > A cloudstack foundation entity funded by companies earning from
> cloudstack seems a good way forward.
> >
> > Another tuppence, this is getting

RE: [DISCUSS] - Releases, Project Management & Funding Thereof

[Alex Hitchins]

Thanks Will,

I understand it's something that comes with a big bag of troublesome worries. 

If this topic comes up again in any discussions, I'd be interested to hear 
their thoughts on what I see as the alternative; without a dedicated 
RM/PM/Captain, people will fork off CS so they can achieve the same thing, and 
CS ultimately looses out long term. I can't remember the name of the fork, but 
I think I'm right that a previous large CS contributor/user forked off as they 
wanted greater management in the areas we are discussing here.


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Will Stevens [mailto:williamstev...@gmail.com] 
Sent: 30 June 2017 14:31
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof

Apache has been historically against the idea of a cloudstack foundation and 
there is a bit of a pandoras box there which we will want to be careful about 
opening.

Apache added direct contribution, but it was unusable for us historically 
because it required a minimum contribution of 50k, which none of us can afford. 
However, there have been some changes to the board recently which are in our 
favour if we want to put pressure to lower that to say 5-10k.

Even if we do solve for smaller direct contributions, we will have to jump 
through hoops to be able to use those funds for a dedicated release manager. I 
do think this is a possibility if we manage our needs and communications very 
well. I had some preliminary discussions with some apache foundation folks to 
express these specific concerns. I played off the fact that i know they dont 
want to entertain a cloudstack foundation and tried to see if i could get them 
to move on the direct contribution mechanism to make it usable for us, 
specifically with the goal of hiring a full time release manager. I definitely 
had their ear and they acknowledged the problems we are facing (and currently 
discussing).  They expressed concerns about being able to hire someone with the 
direct contributions, but brainstormed a bit to potentially hire an agency who 
actually does the hire and they pay the persons salary through the agency with 
the direct contribution funds.

All to say, there are potential options here, but there be dragons, so we have 
to handle this topic with care.

On Jun 30, 2017 9:12 AM, "Ron Wheeler" 
wrote:

>
> https://www.apache.org/foundation/contributing.html says:
> "If you have a specific target or project that you wish to directly 
> support, pleasecontact us <https://www.apache.org/founda 
> tion/contributing.html#Fundraising>and we will do our best to satisfy 
> your wishes."
>
> 1) Is Apache willing to allow projects to set up their own 
> foundations? I doubt but someone would need to check this out.
> Does the PMC have the project charter or the agreement that was signed 
> when Cloudstack moved.
>
> 2) Has anyone tried to contact Apache about directing support to 
> Cloudstack.
>
> I am not convinced that lack of paid staff is the issue.
> This discussion reminded me of this.
> Q: How many psychiatrists does it take to change a lightbulb ?
> A: Only one, but the lightbulb must want to change
>
> http://www.lightbulbjokes.com/directory/p.html
>
>
> Ron
>
>
> On 30/06/2017 6:48 AM, Alex Hitchins wrote:
>
>> As per Giles's comment to the previous thread, I thought I would 
>> start a discussion on the subject to canvas peoples thoughts, opinions and 
>> fears.
>>
>> My question for discussion, is there is any mileage in someone 
>> creating a "CloudStack Foundation" as a non-profit entity, funded 
>> largely by key CloudStack players with the sole function of employing 
>> dedicated resource (part or full time) to handle all releases and 
>> other essential 'back office' functions. The idea being it's in 
>> everyone's interest to chip in a little each to fund core project and 
>> release management.
>>
>> The idea might be utterly irrelevant, pointless and/or straight up daft.
>> I urge you all to let me know.
>>
>> Something for you all to think over this weekend.
>>
>>
>> Alexander Hitchins
>> 
>> E: a...@alexhitchins.com
>> W: alexhitchins.com
>> M: 07788 423 969
>> T: 01892 523 587
>>
>> -Original Message-
>> From: Giles Sirett [mailto:giles.sir...@shapeblue.com]
>> Sent: 30 June 2017 09:51
>> To: dev@cloudstack.apache.org
>> Subject: RE: JIRA - PLEASE READ
>>
>> All
>> This thread seems to have turned into 2 quite different discussions:
>>
>> 1. The use (or no

RE: [DISCUSS] - Releases, Project Management & Funding Thereof

[Alex Hitchins]

I'll read those links, thank you for providing them.

Do you think this is a move in the wrong direction or just an unnecessary move 
to begin with?

Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Ron Wheeler [mailto:rwhee...@artifact-software.com] 
Sent: 30 June 2017 14:06
To: dev@cloudstack.apache.org
Subject: Re: [DISCUSS] - Releases, Project Management & Funding Thereof


https://www.apache.org/foundation/contributing.html says:
"If you have a specific target or project that you wish to directly support, 
pleasecontact us 
<https://www.apache.org/foundation/contributing.html#Fundraising>and we will do 
our best to satisfy your wishes."

1) Is Apache willing to allow projects to set up their own foundations? 
I doubt but someone would need to check this out.
Does the PMC have the project charter or the agreement that was signed when 
Cloudstack moved.

2) Has anyone tried to contact Apache about directing support to Cloudstack.

I am not convinced that lack of paid staff is the issue.
This discussion reminded me of this.
Q: How many psychiatrists does it take to change a lightbulb ?
A: Only one, but the lightbulb must want to change

http://www.lightbulbjokes.com/directory/p.html


Ron


On 30/06/2017 6:48 AM, Alex Hitchins wrote:
> As per Giles's comment to the previous thread, I thought I would start a 
> discussion on the subject to canvas peoples thoughts, opinions and fears.
>
> My question for discussion, is there is any mileage in someone creating a 
> "CloudStack Foundation" as a non-profit entity, funded largely by key 
> CloudStack players with the sole function of employing dedicated resource 
> (part or full time) to handle all releases and other essential 'back office' 
> functions. The idea being it's in everyone's interest to chip in a little 
> each to fund core project and release management.
>
> The idea might be utterly irrelevant, pointless and/or straight up daft. I 
> urge you all to let me know.
>
> Something for you all to think over this weekend.
>
>
> Alexander Hitchins
> 
> E: a...@alexhitchins.com
> W: alexhitchins.com
> M: 07788 423 969
> T: 01892 523 587
>
> -Original Message-
> From: Giles Sirett [mailto:giles.sir...@shapeblue.com]
> Sent: 30 June 2017 09:51
> To: dev@cloudstack.apache.org
> Subject: RE: JIRA - PLEASE READ
>
> All
> This thread seems to have turned into 2 quite different discussions:
>
> 1. The use (or not) of Jira - which was the original discussion
>
> 2. Ways/means of encouraging (and paying for more structured 
> contributors)
>
> I know that it could be argued that these are related. Could I suggest 
> opening up a thread on "release and project management and funding it"  
> and keeping this thread to the original discussion
>
> (I will weigh in on both of these at some stage)
>
> Kind regards
> Giles
>
> giles.sir...@shapeblue.com
> www.shapeblue.com
> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>
>   
>
>
> -Original Message-
> From: Alex Hitchins [mailto:a...@alexhitchins.com]
> Sent: 29 June 2017 18:49
> To: dev@cloudstack.apache.org
> Subject: Re: JIRA - PLEASE READ
>
> If it isn't being treated as a product it will be very impossible to market 
> it as enterprise ready.
>
> I know we all know this.
>
> Similar sized projects under the Apache banner must have the same issue, what 
> is the best way to gather experience of these projects? See how they handle 
> these growing pains.
>
> A cloudstack foundation entity funded by companies earning from cloudstack 
> seems a good way forward.
>
> Another tuppence, this is getting expensive.
>
>
>
>> On 29 Jun 2017, at 18:18, Ron Wheeler  wrote:
>>
>> I understand that it is a volunteer organization.
>> I do not know how many (if any) of the committers and PMC members are funded 
>> by their organizations (allowed or ordered to work on Cloudstack during 
>> company time) which is often the way that Apache projects get staffed.
>>
>> Clearly it is hard to tell someone who is being funded by a company to fix a 
>> problem or who is working on their own time, to do or not do something.
>>
>> On the other hand, the PMC has to  build a community culture that is good 
>> for the project.
>> That means describing a vision, planning and enforcing a roadmap, and  
>> maintaining a focused project "marketing" effort.
>>
>> There is a lot of extremely talented individuals working on Cloudstack and 
>> it appears to have a very strong and 

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Will Stevens]

Apache has been historically against the idea of a cloudstack foundation
and there is a bit of a pandoras box there which we will want to be careful
about opening.

Apache added direct contribution, but it was unusable for us historically
because it required a minimum contribution of 50k, which none of us can
afford. However, there have been some changes to the board recently which
are in our favour if we want to put pressure to lower that to say 5-10k.

Even if we do solve for smaller direct contributions, we will have to jump
through hoops to be able to use those funds for a dedicated release
manager. I do think this is a possibility if we manage our needs and
communications very well. I had some preliminary discussions with some
apache foundation folks to express these specific concerns. I played off
the fact that i know they dont want to entertain a cloudstack foundation
and tried to see if i could get them to move on the direct contribution
mechanism to make it usable for us, specifically with the goal of hiring a
full time release manager. I definitely had their ear and they acknowledged
the problems we are facing (and currently discussing).  They expressed
concerns about being able to hire someone with the direct contributions,
but brainstormed a bit to potentially hire an agency who actually does the
hire and they pay the persons salary through the agency with the direct
contribution funds.

All to say, there are potential options here, but there be dragons, so we
have to handle this topic with care.

On Jun 30, 2017 9:12 AM, "Ron Wheeler" 
wrote:

>
> https://www.apache.org/foundation/contributing.html says:
> "If you have a specific target or project that you wish to directly
> support, pleasecontact us  tion/contributing.html#Fundraising>and we will do our best to satisfy
> your wishes."
>
> 1) Is Apache willing to allow projects to set up their own foundations? I
> doubt but someone would need to check this out.
> Does the PMC have the project charter or the agreement that was signed
> when Cloudstack moved.
>
> 2) Has anyone tried to contact Apache about directing support to
> Cloudstack.
>
> I am not convinced that lack of paid staff is the issue.
> This discussion reminded me of this.
> Q: How many psychiatrists does it take to change a lightbulb ?
> A: Only one, but the lightbulb must want to change
>
> http://www.lightbulbjokes.com/directory/p.html
>
>
> Ron
>
>
> On 30/06/2017 6:48 AM, Alex Hitchins wrote:
>
>> As per Giles's comment to the previous thread, I thought I would start a
>> discussion on the subject to canvas peoples thoughts, opinions and fears.
>>
>> My question for discussion, is there is any mileage in someone creating a
>> "CloudStack Foundation" as a non-profit entity, funded largely by key
>> CloudStack players with the sole function of employing dedicated resource
>> (part or full time) to handle all releases and other essential 'back
>> office' functions. The idea being it's in everyone's interest to chip in a
>> little each to fund core project and release management.
>>
>> The idea might be utterly irrelevant, pointless and/or straight up daft.
>> I urge you all to let me know.
>>
>> Something for you all to think over this weekend.
>>
>>
>> Alexander Hitchins
>> 
>> E: a...@alexhitchins.com
>> W: alexhitchins.com
>> M: 07788 423 969
>> T: 01892 523 587
>>
>> -Original Message-
>> From: Giles Sirett [mailto:giles.sir...@shapeblue.com]
>> Sent: 30 June 2017 09:51
>> To: dev@cloudstack.apache.org
>> Subject: RE: JIRA - PLEASE READ
>>
>> All
>> This thread seems to have turned into 2 quite different discussions:
>>
>> 1. The use (or not) of Jira - which was the original discussion
>>
>> 2. Ways/means of encouraging (and paying for more structured contributors)
>>
>> I know that it could be argued that these are related. Could I suggest
>> opening up a thread on "release and project management and funding it"  and
>> keeping this thread to the original discussion
>>
>> (I will weigh in on both of these at some stage)
>>
>> Kind regards
>> Giles
>>
>> giles.sir...@shapeblue.com
>> www.shapeblue.com
>> 53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
>>
>>
>> -Original Message-
>> From: Alex Hitchins [mailto:a...@alexhitchins.com]
>> Sent: 29 June 2017 18:49
>> To: dev@cloudstack.apache.org
>> Subject: Re: JIRA - PLEASE READ
>>
>> If it isn't being treated as a product it will be very impossible to
>> market it as enterprise ready.
>>
>> I know we all know this.
>>
>> Similar sized projects under the Apache banner must have the same issue,
>> what is the best way to gather experience of these projects? See how they
>> handle these growing pains.
>>
>> A cloudstack foundation entity funded by companies earning from
>> cloudstack seems a good way forward.
>>
>> Another tuppence, this is getting expensive.
>>
>>
>>
>> On 29 Jun 2017, at 18:18, Ron Wheeler 
>>> wrote:
>>>
>>> I understand th

Re: [DISCUSS] - Releases, Project Management & Funding Thereof

[Ron Wheeler]

https://www.apache.org/foundation/contributing.html says:
"If you have a specific target or project that you wish to directly 
support, pleasecontact us 
and we 
will do our best to satisfy your wishes."


1) Is Apache willing to allow projects to set up their own foundations? 
I doubt but someone would need to check this out.
Does the PMC have the project charter or the agreement that was signed 
when Cloudstack moved.


2) Has anyone tried to contact Apache about directing support to Cloudstack.

I am not convinced that lack of paid staff is the issue.
This discussion reminded me of this.
Q: How many psychiatrists does it take to change a lightbulb ?
A: Only one, but the lightbulb must want to change

http://www.lightbulbjokes.com/directory/p.html


Ron


On 30/06/2017 6:48 AM, Alex Hitchins wrote:

As per Giles's comment to the previous thread, I thought I would start a 
discussion on the subject to canvas peoples thoughts, opinions and fears.

My question for discussion, is there is any mileage in someone creating a 
"CloudStack Foundation" as a non-profit entity, funded largely by key 
CloudStack players with the sole function of employing dedicated resource (part or full 
time) to handle all releases and other essential 'back office' functions. The idea being 
it's in everyone's interest to chip in a little each to fund core project and release 
management.

The idea might be utterly irrelevant, pointless and/or straight up daft. I urge 
you all to let me know.

Something for you all to think over this weekend.


Alexander Hitchins

E: a...@alexhitchins.com
W: alexhitchins.com
M: 07788 423 969
T: 01892 523 587

-Original Message-
From: Giles Sirett [mailto:giles.sir...@shapeblue.com]
Sent: 30 June 2017 09:51
To: dev@cloudstack.apache.org
Subject: RE: JIRA - PLEASE READ

All
This thread seems to have turned into 2 quite different discussions:

1. The use (or not) of Jira - which was the original discussion

2. Ways/means of encouraging (and paying for more structured contributors)

I know that it could be argued that these are related. Could I suggest opening up a 
thread on "release and project management and funding it"  and keeping this 
thread to the original discussion

(I will weigh in on both of these at some stage)

Kind regards
Giles

giles.sir...@shapeblue.com
www.shapeblue.com
53 Chandos Place, Covent Garden, London  WC2N 4HSUK @shapeblue
   
  



-Original Message-
From: Alex Hitchins [mailto:a...@alexhitchins.com]
Sent: 29 June 2017 18:49
To: dev@cloudstack.apache.org
Subject: Re: JIRA - PLEASE READ

If it isn't being treated as a product it will be very impossible to market it 
as enterprise ready.

I know we all know this.

Similar sized projects under the Apache banner must have the same issue, what 
is the best way to gather experience of these projects? See how they handle 
these growing pains.

A cloudstack foundation entity funded by companies earning from cloudstack 
seems a good way forward.

Another tuppence, this is getting expensive.




On 29 Jun 2017, at 18:18, Ron Wheeler  wrote:

I understand that it is a volunteer organization.
I do not know how many (if any) of the committers and PMC members are funded by 
their organizations (allowed or ordered to work on Cloudstack during company 
time) which is often the way that Apache projects get staffed.

Clearly it is hard to tell someone who is being funded by a company to fix a 
problem or who is working on their own time, to do or not do something.

On the other hand, the PMC has to  build a community culture that is good for 
the project.
That means describing a vision, planning and enforcing a roadmap, and  maintaining a 
focused project "marketing" effort.

There is a lot of extremely talented individuals working on Cloudstack and it 
appears to have a very strong and valuable code-base.

To me the key question is about the PMC and the core committers' ability to make 
Cloudstack a "product" that can compete for market share and acceptance.

Is Cloudstack at a point in its development where it should be treated like a 
product?
- sufficient functionality to compete
- sufficient user base to be a competitor in the market
- production reliability and stability
- business model for supporting companies to justify their continued
support

This may not require more effort but requires different policies and different 
activities.

There has to be someone or a PMC  that can say "No".
- This change can not be included in this release because it will delay the 
release.
- This change adds an unacceptable level of complexity
- This bug fix will have to wait for the next release because it is too late to 
test it and fix the docs.
- This fix breaks the docs
- The release can not be made until this doc is updated.

Does the core group want to make it a competitive product or is it sufficient 
for the interested players to contin