Bluntness appreciated, this process is already in motion. http://opscode.com/openstack was launched 2 weeks ago and I promptly left for conferences and vacation. I am consolidating GitHub repos here:
https://github.com/opscode/openstack-chef-repo https://github.com/opscode-cookbooks/nova https://github.com/opscode-cookbooks/glance https://github.com/opscode-cookbooks/horizon https://github.com/opscode-cookbooks/keystone https://github.com/opscode-cookbooks/swift Work is being done in my own repos until it's ready for an initial release, which will include a Getting Started with Chef and OpenStack document. https://github.com/mattray/ I'm working with quite a few folks already, Rackspace, Dell, DreamHost and others and Intel is sponsoring this work. Jay and I chatted a bit in IRC, we're quite aligned in how we plan on working this and the goal will be to get github.com/openstack/chef-repo gated with Gerrit and CI and pulling from Opscode's repos soon. Please feel free to join the discussion on our new mailing list focused on this effort here: http://groups.google.com/group/opscode-chef-openstack And an IRC channel: #openstack-chef on irc.freenode.net Thanks, Matt Ray Senior Technical Evangelist | Opscode Inc. [email protected] | (512) 731-2218 Twitter, IRC, GitHub: mattray On Tue, Jul 10, 2012 at 11:25 AM, Jay Pipes <[email protected]> wrote: > Apologies in advance for my blunt and somewhat dour response, Matt. I'm > not singling you out at all, and I know you've tried your best to get > the various Chef stakeholders to work together. Also apologies for > top-posting, but there's not a whole lot of use inline posting this. > > tl;dr > ----- > > We need to stop the needless fracturing of the operational knowledge of > the Chef community and try working as a team towards some common goals > instead of creating fork after fork of repos of Chef cookbooks. > > There is a ton of wasted effort in this area. > > Proposal: > > * Get our act together and treat Chef repos (and puppet/juju/whatever) > as we do other OpenStack core and supporting projects -- use Gerrit, use > a CI gating system, do real code reviews on it, and in general treat > them as a supporting OpenStack project > * Mark ALL Chef repos that are not currently maintained with an OBSELETE > marker and/or DELETE the repo on Github > * Consolidate all *cookbooks* into a repository in > github.com/openstack/chef-repo > * Use git submodules to manage cookbooks that are upstreamed in > github.com/opscode/ that have little to no changes in them > * Actually fix the documentation of the dang cookbooks -- right now, > half of them include the documentation from the memcache cookbook, as > they were lazily copy-pasted around, or the standard example doc that is > created when using something like knife. > * Put as much variation in deployment philosophy into *roles* and > attribute overrides/defaults > > More/Rant/Details > ----------------- > > Maybe it's just the open source developer in me, but I don't understand > why there is such an aversion to coordination in the deployment/ops > community around the scripts and deployment > cookbooks/modules/charms/whatever. > > Is it that everyone has a different idea of what is best? Is it because > deployers/ops folks think that coordinating with other contributors is > too time-consuming? Is it because the chef repos are not controlled in > the same way as, say, devstack or the core projects, with an automated > patch queue manager and code review system that actually encourages > debate over patches? A combination of all of the above? > > Over the last 2 years, I've worked at 3 companies in the OpenStack > ecosystem. All three companies had their own repos of Chef cookbooks > (still do to this day). 50-60% of the content of these cookbooks is > identical. 10-20% of the content of these cookbooks is different -- but > only slightly or cosmetically. And a good portion of the remaining > 20-40% are differences that are incorrectly (IMHO) placed in the > cookbooks and recipes instead of where they should be: in roles and > environments, with cookbooks created that deal with variations in > deployments with attributes and the occasional if/else block. > > In trying to determine the appropriate Chef repo to use for the TryStack > project, we found the following repo: > > https://github.com/osops/chef-repo > > to have the most up-to-date. I've since been told this repo is no longer > maintained. This is very frustrating, not because of this particular > repo, but because this is just one in a long line of neglected and > forgotten forks of chef cookbook repositories. The fact that the default > Chef behaviour and Opscode documentation encourages the copy/pasting of > cookbooks all over the place and GitHub itself encourages the random and > promiscuous forking of repos doesn't help. > > Let's get real about the deployment/ops code and > cookbooks/modules/charms. Let's treat them the same way we do code in > the core projects and supporting projects. > > Thanks for your time, > -jay > > On 07/10/2012 11:42 AM, Matt Ray wrote: >> Just a heads up, I'm working on building unified community-driven >> cookbooks over in https://github.com/opscode/openstack-chef-repo (and >> repos for the individual cookbooks). These are forked from Rackspace's >> cookbooks and I'm working with them and others to make reusable, >> well-documented and supported Chef cookbooks for OpenStack. I'll make >> a larger announcement around them once I have a working quickstart >> document for them. >> >> tl;dr; https://github.com/opscode/openstack-chef-repo >> >> Thanks, >> Matt Ray >> Senior Technical Evangelist | Opscode Inc. >> [email protected] | (512) 731-2218 >> Twitter, IRC, GitHub: mattray >> >> >> On Tue, Jul 10, 2012 at 10:22 AM, Jay Pipes <[email protected]> wrote: >>> Gah... probably would be good if you guys either shut down the repo or >>> made a big notice on the README then :( >>> >>> -jay >>> >>> On 07/09/2012 05:25 PM, Joe Breu wrote: >>>> Hi Jay, >>>> >>>> The chef cookbooks at https://github.com/osops are no longer maintained. >>>> Our current cookbooks are at https://github.com/rcbops/chef-cookbooks >>>> >>>> >>>> >>>> --- >>>> Joseph Breu >>>> Deployment Engineer >>>> Rackspace Cloud Builders >>>> 210-312-3508 >>>> >>>> On Jul 9, 2012, at 8:01 AM, Jay Pipes wrote: >>>> >>>>> Vish and Ron, just getting back to this... see inline continued >>>>> questions for you both. >>>>> >>>>> On 07/02/2012 04:24 PM, Vishvananda Ishaya wrote: >>>>>> >>>>>> On Jul 2, 2012, at 7:28 AM, Jay Pipes wrote: >>>>>> >>>>>>> Hi Ron, cc'ing the openstack ML for extra eyes and opinions... >>>>>>> >>>>>>> So, Nati and I are looking to use either the osops chef-repo or >>>>>>> something similar as the basis of the new TryStack zone chef deployment. >>>>>>> I've been going through the recipes and roles and I have a question on >>>>>>> the nova-compute *role*: >>>>>>> >>>>>>> https://github.com/osops/chef-repo/blob/master/roles/nova-compute.rb >>>>>>> >>>>>>> I'm wondering why the nova-api recipe is in the runlist for >>>>>>> nova-compute? >>>>>> >>>>>> Because metadata needs to run on the compute hosts in the default >>>>>> layout. This should >>>>>> be switched to use nova-api-metadata instead of nova-api, but the >>>>>> split out hasn't been done yet. >>>>> >>>>> OK, I will work on splitting this out a bit more effectively. >>>>> >>>>> One additional question, though. In opening up the >>>>> /cookbooks/nova/recipes/nova/compute.rb file, you will notice this at >>>>> the top: >>>>> >>>>> include_recipe "nova::api" >>>>> >>>>> Therefore, unless I am mistaken, the nova-compute *role*'s runlist >>>>> actually doesn't need to contain both nova-api AND nova-compute since >>>>> apparently the nova-compute *recipe* already includes all of the >>>>> nova-api recipe. >>>>> >>>>> Would you agree with that? >>>>> >>>>>>> In addition, I was wondering if y'all had considered making more use of >>>>>>> roles instead of recipes to allow most of the attribute assignment and >>>>>>> variation to be in the combination of roles assigned to a host, instead >>>>>>> of recipes combined in a role? >>>>>>> >>>>>>> For example, right now, there is a "nova-controller" role that looks >>>>>>> like this: >>>>>>> >>>>>>> name "nova-controller" >>>>>>> description "Nova controller node (vncproxy + rabbit)" >>>>>>> run_list( >>>>>>> "role[base]", >>>>>>> "recipe[nova::controller]" >>>>>>> ) >>>>>>> >>>>>>> Because most of the special sauce is in the nova::controller recipe, I >>>>>>> have to go into that recipe to see what the role is composed of: >>>>>>> >>>>>>> include_recipe "mysql::server" >>>>>>> include_recipe "openssh::default" >>>>>>> >>>>>>> include_recipe "rabbitmq::default" >>>>>>> include_recipe "keystone::server" >>>>>>> include_recipe "glance::registry" >>>>>>> include_recipe "glance::api" >>>>>>> include_recipe "nova::nova-setup" >>>>>>> include_recipe "nova::scheduler" >>>>>>> include_recipe "nova::api" >>>>>>> >>>>>>> if platform?(%w{fedora}) >>>>>>> # Fedora skipping vncproxy for right now >>>>>>> else >>>>>>> include_recipe "nova::vncproxy" >>>>>>> end >>>>>>> >>>>>>> But what this recipe really is is an opinionated description of a >>>>>>> "controller role". If the role was, instead, structured like so: >>>>>>> >>>>>>> name "nova-controller" >>>>>>> description "Nova Controller - All major API services and control >>>>>>> servers like MySQL and Rabbit" >>>>>>> run_list( >>>>>>> "role[base]", >>>>>>> "recipe[mysql::server]", >>>>>>> "recipe[openssh::default]", >>>>>>> "recipe[rabbitmq::default]", >>>>>>> "recipe[keystone::server]", >>>>>>> "recipe[glance::api]", >>>>>>> "recipe[glance::registry]", >>>>>>> "recipe[nova::scheduler]", >>>>>>> "recipe[nova::api]", >>>>>>> ) >>>>>>> >>>>>>> Then the deployer can more easily switch up the way they deploy >>>>>>> OpenStack servers by merely changing the role -- say, removing the >>>>>>> Rabbit service and putting it somewhere else -- WITHOUT having to modify >>>>>>> a recipe in a Git submodule in the upstream cookbooks. >>>>>>> >>>>>>> Furthermore, if we broke out more roles -- such as "control-services" >>>>>>> which might be MySQL and Rabbit only -- than we could make the "super >>>>>>> roles" ,like the nova-controller role above, more of a "put this and >>>>>>> that role together, like so: >>>>>>> >>>>>>> name "nova-controller" >>>>>>> description "Nova Controller - All major API services and control >>>>>>> servers like MySQL and Rabbit" >>>>>>> run_list( >>>>>>> "role[base]", >>>>>>> "role[control_services]", >>>>>>> "recipe[keystone::server]", >>>>>>> "recipe[glance::api]", >>>>>>> "recipe[glance::registry]", >>>>>>> "recipe[nova::scheduler]", >>>>>>> "recipe[nova::api]", >>>>>>> ) >>>>>> >>>>>> This all makes sense to me. Ron? >>>>> >>>>> Ron, any disagreement here? >>>>> >>>>>>> Finally, I've noticed that there are aren't any HA options in the osops >>>>>>> recipes -- specifically around MySQL. Are there plans to do so? We use >>>>>>> heartbeat/Pacemaker/DRBD in the original TryStack cookbooks [1] and >>>>>>> environments to get simple HA solutions up and would love to see those >>>>>>> in the upstream. >>>>> >>>>> Either of you, any thoughts on this front? >>>>> >>>>> Thanks! >>>>> -jay >>>>> >>>>>>> Thanks and all the best guys, >>>>>>> -jay >>>>>>> >>>>>>> [1] >>>>>>> https://github.com/trystack/openstack-chef/tree/stable/diablo/cookbooks/heartbeat >>>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Mailing list: https://launchpad.net/~openstack >>>>> Post to : [email protected] >>>>> <mailto:[email protected]> >>>>> Unsubscribe : https://launchpad.net/~openstack >>>>> More help : https://help.launchpad.net/ListHelp >>>> >>> >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~openstack >>> Post to : [email protected] >>> Unsubscribe : https://launchpad.net/~openstack >>> More help : https://help.launchpad.net/ListHelp > _______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : [email protected] Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
