Eliminating Support for Ubuntu 12.04

[John Burwell]

All,

PR 1647 [1] proposes dropping support for Ubuntu 12.04 from 4.9.2.0+.  The 
primary motivation for its removal is that the age of its libvirt and qemu 
versions greatly complicate maintenance of the KVM integration.  However, 
Ubuntu 12.04 will be supported until April 2017 [2]. What would be the impact 
to our user community of removing support for Ubuntu 12.04 before its EOL in 
April 2017?  If we don’t drop support for it in 4.x, would it be acceptable to 
drop support for it in 5.0.0 which is currently scheduled for release at the 
end of 2016 [3]?

If we do chose to drop support for Ubuntu 12.04 in 4.x, I propose that we 
remove it in 4.10.0.0 rather than 4.9.2.0.  First, it is reasonable for users 
to expect that upgrading between patch releases (i.e. 4.9.x.x -> 4.9.x+1.x) 
would not require system changes.  Dropping a distribution would violate such 
an expectation.  Second, 4.9 is an LTS branch.  Therefore, maintaining 12.04 
support in 4.9 would provide LTS users with support for Ubuntu 12.04 until May 
2018 — well after its EOL.  Does this approach seem reasonable if we elect drop 
Ubuntu 12.04 in 4.x?

Thanks,
-John

[1]: https://github.com/apache/cloudstack/pull/1647
[2]: https://wiki.ubuntu.com/Releases
[3]: 
https://cwiki.apache.org/confluence/display/CLOUDSTACK/%5BPROPOSAL%5D+2016-2017+Release+Cycle+and+Calendar



john.burw...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
@shapeblue
  
 

Re: [PROPOSAL] Early LTS Initial Release

[John Burwell]

Rajani,

As I mentioned in my previous email, the original motivation for a completely 
separate branch was based on objections by some community members that the 
longer, more conservative LTS release cycle would place a drag on the velocity 
of regular releases.  Additionally, users interested in LTS releases voiced 
their desire to have fewer releases a year.  Therefore, where the regular 
release cycle is scheduled to make major releases every 2 months and 
maintenance releases every month, the LTS cycle makes major releases every 6 
months and maintenance releases less frequently (e.g. every 3 months).  These 
longer cycles allow users with longer acceptance test/verification cycles to 
more easily keep up with upgrades.  Completely separate branches and release 
cycles were proposed to serve both use cases (rapid, leading upgrades and more 
traditional maintenance cycles).  

I am open to collapsing LTS into the regular maintenance releases (e.g. 4.9 
simply becomes supported for 20 months instead of 4 months).  Ultimately, I 
would like that decision to be based on user feedback since separate release 
branches/cycles have been previously discussed with no objections [1].  I have 
CC’ed users@ to solicit thoughts from our users on which approach would be 
preferred.

Thanks,
-John

[1]: http://markmail.org/thread/zh43rc6ahs4te46l  

> 
john.burw...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
@shapeblue
  
 

On Aug 2, 2016, at 3:57 AM, Rajani Karuturi <raj...@apache.org> wrote:
> 
> We already maintain the release branches and do regular
> releases(of the past two releases) on them. What are we achieving
> through this LTS model? How is 4.9.1 different from 4.9.0.0_1.0?
> 
> ~ Rajanihttp://cloudplatform.accelerite.com/
> On August 2, 2016 at 2:33 AM, John Burwell
> (john.burw...@shapeblue.com) wrote:Wido,
> 
> As proposed, LTS will be a branch of 4.9.0 with a six (6) week
> period of additional testing (i.e. soak/endurance, scalability,
> and more extensive plugin testing). Therefore, LTS releases will
> be named _ meaning that the first LTS
> release would be 4.9.0.0_1.0. The original motivation for this
> approach was that the regular release cycle performed testing for
> a week which was not enough time to execute long running tests
> (e.g. the entire test suite requires roughly 4 days to run, a
> good endurance/soak test should run for 5-7 days,
> etc). Additionally, there was concern that LTS would impede the
> velocity of the regular release cycle. By decoupling the
> regular and LTS release branches, there would be no opportunity
> for LTS constraints to impede velocity of the regular release
> cycle.
> 
> Since my original proposal, a number of aspects about the release
> cycle have changed. I am open to adjust LTS to simply be an
> extension of the support period on the 4.9.0.0
> release. Personally, I think the risk of the LTS cycle impeding
> regular releases is very low. I also think it would be more
> consistent with the way we have managed long running releases in
> the past. We would still perform the additional test we planned
> for LTS, but it would on the 4.9 release branch rather than a
> separate LTS branch.
> 
> Are there any objections to this change to the way LTS branches
> are managed and releases named? For now, I will leave the LTS
> releases in the schedule as defined in the original
> proposal. If/when we gain consensus on this change, I will
> adjust the schedule.
> 
> Thanks,
> -John
> From: Wido den Hollander
> <w...@widodh.nl ( w...@widodh.nl )>Sent: Friday, July 15, 2016
> 4:15:36 AMTo: d...@cloudstack.apache.org (
> d...@cloudstack.apache.org ); John BurwellSubject: Re: [PROPOSAL]
> Early LTS Initial Release
> Op 13 juli 2016 om 18:25 schreef John Burwell
> <john.burw...@shapeblue.com ( john.burw...@shapeblue.com )>:
> 
> All,
> Since LTS introduces a new release stream, I would like to
> propose that we cut the first LTS quickly to verify that various
> aspects of the release cycle and version number dependent
> components will work properly with the new release naming
> scheme. It will also allow us to flesh out distribution issues
> such as download locations (e.g. for system VMs) and publishing
> LTS documentation alongside regular releases. My thinking is
> that we would push an RC for this release within a week of the
> 4.9.0.0 release. If this additional release is acceptable, I
> will update the release calendar to reflect the following
> changes:
> * LTS 4.9.0.0_1.0* Development/Testing: 1 week after 4.9.0.0
> release* RC Voting: 2 weeks after 4.9.0.0 release* LTS
> 4.9.0.0_2.0* Development/Testing: From 3 to 9 weeks of the
> 4.9.0.0 release* RC Voting: 10t

Re: Organizing CCCNA16 Hackathon

[John Burwell]

Pierre-Luc and WIll,

While testing and CI are important topics, there may be additional topics of 
high importance to those attending the conference to discuss and work.  
Therefore, I think it is a good idea to collect everyone’s thoughts to ensure 
the priorities of all attendees are addressed.  Additionally, my hope is that 
some advance organization and coordination will allow people to participate 
virtually.

Thanks,
-John

> 
Regards,

John Burwell

john.burw...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
@shapeblue
On May 5, 2016, at 1:32 PM, Pierre-Luc Dion <pd...@cloudops.com> wrote:
> 
> Thanks John  for that initiative.
> 
> Good idea to start organizing teams and topics since we won't have much
> time on site. As Will said the main theme would be around CI, and we are
> working to get enought hardware and ressource so all teams could have their
> own setup...
> 
> Cheers,
> 
> PL
> 
> On Thu, May 5, 2016 at 12:02 PM, Will Stevens <williamstev...@gmail.com>
> wrote:
> 
>> Hey John,
>> Thanks for the initiative.  We will be focusing the hackathon mainly
>> around testing.  I will be setting up full environments for the teams to
>> work with and those details will be made available on the day of the
>> hackathon.
>> 
>> I think it is a good idea to have a general theme for the hackathon to
>> help focus us on getting actionable work done.
>> 
>> I like the idea of having topics submitted early and teams getting
>> organized prior to the actual hackathon.
>> 
>> Cheers,
>> 
>> Will
>> 
>> On Thu, May 5, 2016 at 10:23 AM, John Burwell <john.burw...@shapeblue.com>
>> wrote:
>> 
>>> All,
>>> 
>>> In advance of the CCNA16 Hackathon on Wednesday, 1 June 2016, it seems
>>> like it would be wise to organize the topics which people are interested in
>>> addressing.  My thought is anyone may suggest a topic and/or register their
>>> interest in a topic.  Also, since there will likely be a number of people
>>> unable to physically attend, we can create a Google Hangout for each
>>> group.  Hopefully, by organizing the topics in advance and broadcasting our
>>> work, we can further extend participation across the community.
>>> 
>>> To begin organizing topics, I have created a public Google Spreadsheet
>>> [1] with columns for group name, organizer/proposer,  interested
>>> participants, and a Hangout URL.  If you would like to propose a topic,
>>> create a new row with the name of the group, your name and email, and a
>>> Hangout URL (instructions[2]).  If you are interested in one or more
>>> topics, please add your name and email to the row.  I would also like to
>>> propose that we record the Hangout sessions (upload location TBD), and that
>>> the organizer of the topic report any results of their efforts with a URL
>>> of the recording back to dev@.
>>> 
>>> Finally, I have included users@ and marketing@ as they may also wish to
>>> participate.
>>> 
>>> Thanks,
>>> -John
>>> 
>>> [1]:
>>> https://docs.google.com/spreadsheets/d/14U0E1YpgZvsBc88SHVojo-XzOLAKs-WFEkMxXlrAl_o/edit#gid=0
>>> [2]: https://support.google.com/hangouts/answer/3111943?hl=en
>>> 
>>> 
>>> 
>>> 
>>> 
>>> john.burw...@shapeblue.com
>>> www.shapeblue.com
>>> @shapeblue
>>> 
>> 
>> 

Organizing CCCNA16 Hackathon

[John Burwell]

All,

In advance of the CCNA16 Hackathon on Wednesday, 1 June 2016, it seems like it 
would be wise to organize the topics which people are interested in addressing. 
 My thought is anyone may suggest a topic and/or register their interest in a 
topic.  Also, since there will likely be a number of people unable to 
physically attend, we can create a Google Hangout for each group.  Hopefully, 
by organizing the topics in advance and broadcasting our work, we can further 
extend participation across the community.

To begin organizing topics, I have created a public Google Spreadsheet [1] with 
columns for group name, organizer/proposer,  interested participants, and a 
Hangout URL.  If you would like to propose a topic, create a new row with the 
name of the group, your name and email, and a Hangout URL (instructions[2]).  
If you are interested in one or more topics, please add your name and email to 
the row.  I would also like to propose that we record the Hangout sessions 
(upload location TBD), and that the organizer of the topic report any results 
of their efforts with a URL of the recording back to dev@.

Finally, I have included users@ and marketing@ as they may also wish to 
participate.

Thanks,
-John

[1]: 
https://docs.google.com/spreadsheets/d/14U0E1YpgZvsBc88SHVojo-XzOLAKs-WFEkMxXlrAl_o/edit#gid=0
[2]: https://support.google.com/hangouts/answer/3111943?hl=en

Regards,

John Burwell

john.burw...@shapeblue.com 
www.shapeblue.com
53 Chandos Place, Covent Garden, London VA WC2N 4HSUK
@shapeblue

[DISCUSS][PROPOSAL] CloudStack Upgrade Policy

[John Burwell]

All,

A recently proposed change [1] to remove older upgrade code has prompted a 
wider discussion regarding the policy we should employ regarding upgrade 
support.  Currently, we have no policy regarding how far back the process will 
function.  As the system evolves, it is simply unsustainable to expect a 
deliver a reliable upgrade from releases that are many years old (e.g. 3.x to 
the latest 4.x). Currently, we do not verify that all upgrade paths work for 
each release.

My straw-man proposal is that effective with the 4.6 release, we only support 
upgrades from two minor releases back of release with revisions inclusive.  For 
example, upgrades to 4.6.x would be supported from 4.4.x and 4.5.x.  The 
upgrade process from releases prior to 4.4.0 would require a two step process — 
upgrade to the latest version 4.5 release and then upgrade from the latest 4.5 
release to the latest 4.6 release.  This approach provides backwards 
compatibility while establishing an upgrade process that is reasonably testable.

In my opinion, any upgrade policy should satisfy two criteria — support the 
operational processes of the majority and be verifiable through automated 
testing.

Thanks,
-John

[1]: https://github.com/apache/cloudstack/pull/608

---
John Burwell (@john_burwell)
VP of Software Engineering, ShapeBlue
(571) 403-2411 | +44 20 3603 0542
http://www.shapeblue.com
Please join us at CloudStack Collab EU — 
http://events.linuxfoundation.org/events/cloudstack-collaboration-conference-europe




Find out more about ShapeBlue and our range of CloudStack related services

IaaS Cloud Design  Buildhttp://shapeblue.com/iaas-cloud-design-and-build//
CSForge – rapid IaaS deployment frameworkhttp://shapeblue.com/csforge/
CloudStack Consultinghttp://shapeblue.com/cloudstack-consultancy/
CloudStack Software 
Engineeringhttp://shapeblue.com/cloudstack-software-engineering/
CloudStack Infrastructure 
Supporthttp://shapeblue.com/cloudstack-infrastructure-support/
CloudStack Bootcamp Training Courseshttp://shapeblue.com/cloudstack-training/

This email and any attachments to it may be confidential and are intended 
solely for the use of the individual to whom it is addressed. Any views or 
opinions expressed are solely those of the author and do not necessarily 
represent those of Shape Blue Ltd or related companies. If you are not the 
intended recipient of this email, you must neither take any action based upon 
its contents, nor copy or show it to anyone. Please contact the sender if you 
believe you have received this email in error. Shape Blue Ltd is a company 
incorporated in England  Wales. ShapeBlue Services India LLP is a company 
incorporated in India and is operated under license from Shape Blue Ltd. Shape 
Blue Brasil Consultoria Ltda is a company incorporated in Brasil and is 
operated under license from Shape Blue Ltd. ShapeBlue SA Pty Ltd is a company 
registered by The Republic of South Africa and is traded under license from 
Shape Blue Ltd. ShapeBlue is a registered trademark.

[DISCUSS/PROPOSAL] CCC13 Hackfest: Storage Architecture Summary

[John Burwell]

All,

During the CloudStack Collab 2013 Hackfest, a group of users and developers got 
together to discuss the current storage architecture and ideas for future 
evolution.  The group focused on the following topics:

* Storage architecture overview and 4.2 enhancements
* Storage use cases and deployment models
* Vendor driver needs
* Prioritization of desired storage enhancements

From the storage enhancement prioritization discussion, I would like to bring 
forward the following storage prioritization proposal for the next (and 
possibly subsequent) CloudStack releases:

1. Breaking Storage - Hypervisor Layer Dependencies:  Currently, the 
storage layer includes hypervisor-specific code for each supported hypervisor.  
Additionally, the hypervisor layer includes storage-type specific code for each 
storage type.  This circular dependency bloats the storage layer and greatly 
complicates storage device driver implementation.  Additionally, it makes 
future enhancement much more complex and risky.  This effort would represent a 
set of enhancements to break down the storage layer into a set of composable 
primitives that would be consumed by dependent layers (e.g. Hypervisor).  I 
will initiate a separate discussion thread to flesh out the nature of the 
dependencies and high-level approaches to addressing them.
2. Streamlined Storage Driver Model: As part of the hypervisor/storage 
decoupling, refactoring the storage device driver model to support a set of 
basic, composable operations that function in terms of logical URIs and Java 
I/O streams.  As we discussed storage devices, we realized they minimally 
perform seven I/O operations -- read, write, copyTo, clone, delink 
(non-destructive delete), destroy (destructive delete), and list (?) with the 
relevant paths expressed as URIs.  Additionally, drivers would describe their 
capabilities (e.g. manageable, snapshotable, etc).  I plan to include my 
options on this topic as part of the storage - hypervisor discussion 
decoupling thread.
3. Storage Device Maintenance Mode:  Provide a generalized 
orchestration mechanism to put a storage device into maintenance mode.  This 
capability would likely include an asynchronous internal API for storage layer 
clients (e.g. Hypervisor) to be notified when a device plans to go into 
maintenance mode and, if necessary, abort it.
4. Generic Properties/Details:  Enhance the DataStore support storing a 
property bag of additional configuration information specific to the associated 
storage device driver.  In order to support proper validation and UI display of 
this information, the storage device driver model would include a mechanism to 
describe the nature of the properties and callbacks to perform runtime 
validation of the property bag before persistence.  Finally, storage 
orchestration would ensure that this information is always passed into the 
driver for each operation.
5. Backup/Storage Snapshots:  Support transfer of storage snapshots 
from device to device (e.g. from a SAN to an object store).  Dependent on the 
flexibility of the streamlined storage driver enhancements, this capability may 
be able to implemented completely in the orchestration layer.  If the 
Storage/Hypervisor Decoupling work does not split the notions of storage and 
hypervisor snapshots, this enhancement would likely require it. 

For those in attendance, please correct and/or expand on my 
capture/recollection.  

Thanks,
-John

P.S. I have CC'ed the users@ list to gain notice of the users involved for 
their thoughts/feedback as well.

Apache CloudStack + Riak CS: Building a Complete Private Cloud Meetup

[John Burwell]

All,

The Silicon Valley Riak users group will be hosting a CloudStack/Riak CS meet 
up entitled Apache CloudStack + Riak CS: Building a Complete Private Cloud on 
26 June 2013 (the day after CCC13 ends) at The Innovation Center in Santa 
Clara, CA.  In the talk, we will cover the following topics:
Why Private Cloud?
Anatomy of a Private Cloud
Building a Apache CloudStack Compute Offering
Large Object Storage using Riak CS
Your Own Private Cloud: The Riak CS Apache CloudStack Integration Roadmap
For more information, contact me via email (jburw...@apache.org) or on Freenode 
(jburwell).  To RSVP, please visit 
http://www.meetup.com/Silicon-Valley-Riak/events/124134742/.

Thanks,
-John

Re: Apache CloudStack + Riak CS: Building a Complete Private Cloud Meetup

[John Burwell]

Frans,

Currently, I don't know of any plans for an Indonesian event, but one our 
Technical Evangelists, Pavan Venkatesh, is CC'ed on this email chain.  He may 
have more information regarding international events.  

Riak installation documentation is available @ 
http://docs.basho.com/riak/latest/tutorials/installation/ and Riak CS 
installation documentation is available @ 
http://docs.basho.com/riakcs/latest/cookbooks/installing/Installing-Riak-CS/.  
If you are looking to setup a local development environment, you can quickly 
spin up Riak/Riak CS instance up and going very quickly using Vagrant [1] and 
our vagrant-riak-cs-cluster [2] project:

If you don't have it installed, download and install VirtualBox [3] for your 
development platform
If you don't have it installed, download and install Vagrant [1] for your your 
development platform
Download and unzip the latest release of the vagrant-riak-cs-cluster [2] to a 
local directory (e.g. vagrant-riak-cs-cluster)
From the directory created in step 3, execute RIAK_CS_CREATE_ADMIN_USER=1 
vagrant up from the command line.  Make a note of the access and secret key 
outputted to the terminal -- these are your access credentials for Riak CS.  
Riak requires no access credentials.

Following completion of the vagrant up command, you will have a running virtual 
machine with Riak and Riak CS exposing the following ports on localhost:

8080: Riak CS
8087: Riak Protocol Buffers interface
8098: Riak HTTP interface

Once you have the virtual machine up and running, I suggest working through the 
Riak Fast Track [4] and Riak CS Fast Track [5] using the VM (Note: Using the 
steps described above, you can skip the installation portions of each Fast 
Track).  I also suggest checking out the Basho blog [6], as well as, our RICON 
conference series [7] for more in-depth discussions about Riak, Riak CS, and 
distributed systems.

Please feel free to ping me if you have any further questions or issues getting 
up and running via email (jburw...@basho.com) or on Freenode (jburwell).
-John

[1]: http://www.vagrantup.com
[2]: https://github.com/basho/vagrant-riak-cs-cluster/archive/v1.0.1.zip
[3]: http://www.virtualbox.org
[4]: http://docs.basho.com/riak/latest/tutorials/fast-track/
[5]: http://docs.basho.com/riakcs/latest/riakcs-tutorials/fast-track/
[6]: http://ricon.io/

On Jun 17, 2013, at 10:13 AM, Frans Thamura fr...@meruvian.org wrote:

 Hi john.
 
 Any idea to do smiliar event here.in indonesia?
 
 Any tips for us to know deeply riak?
 
 Do u have installation guide.
 
 F
 On Jun 17, 2013 6:46 PM, John Burwell jburw...@basho.com wrote:
 
 All,
 
 The Silicon Valley Riak users group will be hosting a CloudStack/Riak CS
 meet up entitled Apache CloudStack + Riak CS: Building a Complete Private
 Cloud on 26 June 2013 (the day after CCC13 ends) at The Innovation Center
 in Santa Clara, CA.  In the talk, we will cover the following topics:
 Why Private Cloud?
 Anatomy of a Private Cloud
 Building a Apache CloudStack Compute Offering
 Large Object Storage using Riak CS
 Your Own Private Cloud: The Riak CS Apache CloudStack Integration Roadmap
 For more information, contact me via email (jburw...@apache.org) or on
 Freenode (jburwell).  To RSVP, please visit
 http://www.meetup.com/Silicon-Valley-Riak/events/124134742/.
 
 Thanks,
 -John

DC CloudStack Meetup

[John Burwell]

All,

Basho Technologies [1] will be hosting Chip Childers for a meet up entitled 
Cloudstack – What Is It and What’s Next  on 29 May 2013 at our Herndon, VA 
office.  For more information and/or to RSVP, please see the meetup.com 
announcement [2].

Thanks,
-John

[1]: http://www.basho.com
[2]: http://www.meetup.com/Riak-DC/events/118224772/