Judd, Ok. Here are some of the thoughts I've had (and have mostly working, but hitting some swift snags..) Maybe we can collaborate on this?
Since Chef promotes idempotent operations and cookbooks, I put some effort in making sure that changes are only made if they're required, particularly around destructive or expensive operations. The 2 main cases are: - partitioning disks, which is obviously destructive - building / rebuilding the ring files - rebalancing the ring is relatively expensive. for both cases I've built a LWRP which reads the current state of affairs, and decides if and what are the required changes. For disks, I'm using ' 'parted', which produces machine friendly output. For ring files I'm using the output from ring-builder. the LWRP are driven by recipes which inspect databag - very similar to your approach. However, they also utilize inventory information about available resources created by crowbar in chef during the initial bring up of the deployment. (As a side note - Dell has announced plans to opensource most of crowbar, but there's legalese involved) I'd be more than happy to elaborate and collaborate on this ! a On Thu, Apr 28, 2011 at 11:35 AM, Judd Maltin <[email protected]> wrote: > Hi Andi, > > Indeed, the swift recipes hadn't been updated since mid 2010, so I pushed > forward with my own. > > Thanks! > -judd > > > On Thu, Apr 28, 2011 at 10:03 AM, andi abes <[email protected]> wrote: > >> Judd, >> >> this is a great idea... actually so great, that some folks @Dell and >> OpsCode, me included, have been working on it. >> >> Have a peek on : >> https://github.com/opscode/openstack-cookbooks/tree/master/cookbooks >> >> This effort is also being included into Crowbar (take a peek here: >> http://robhirschfeld.com/tag/crowbar/) which adds the steps needed to >> start with bare metal (rather than installed OS), then using chef to get to >> a working Open stack deployment. >> (if you're at the design meeting, there are demos scheduled). >> >> That said - I'm updating the swift cookbook, and hope to update github >> soon. >> >> a. >> >> >> >> >> >> On Wed, Apr 27, 2011 at 9:55 PM, Jay Payne <[email protected]> wrote: >> >>> Judd, >>> >>> I'm not that familiar with Chef (I'll do some research) but I have a >>> couple of questions and some thoughts: >>> >>> 1. Is this for a multi-server environment? >>> 2. Are all your proxy nodes going to have "allow_account_management = >>> true" in the configs? It might be a good idea to have a second proxy >>> config for account management only >>> 3. Have you looked at using swauth instead of auth? >>> 4. Have you thought about an admin or client node that has utilities >>> on it like st and stats-report? >>> 5. How where will you do on-going ring management or changes? >>> 6. I would think about including some type of functional test at the >>> end of the deployment process to verify everything was created >>> properly and that all nodes can communicate. >>> >>> >>> >>> --J >>> >>> On Wed, Apr 27, 2011 at 6:18 PM, Judd Maltin <[email protected]> >>> wrote: >>> > Hi Folks, >>> > >>> > I've been hacking away at creating an automated deployment system for >>> Swift >>> > using Chef. I'd like to drop a design idea on you folks (most of which >>> I've >>> > already implemented) and get feedback from this esteemed group. >>> > >>> > My end goal is to have a "manifest" (apologies to Puppet) which will >>> define >>> > an entire swift cluster, deploy it automatically, and allow edits to >>> the >>> > ingredients to manage the cluster. In this case, a "manifest" is a >>> > combination of a chef databag describing the swift settings, and a >>> > spiceweasel infrastructure.yaml file describing the OS configuration. >>> > >>> > Ingredients: >>> > - swift cookbook with base, proxy and server recipes. proxy nodes also >>> > (provisionally) contain auth services. storage nodes handle object, >>> > container and account services. >>> > -- Base recipe handles common package install, OS user creation. Sets >>> up >>> > keys. >>> > -- Proxy recipe handles proxy nodes: network config, package install, >>> > memcache config, proxy and auth package config, user creation, ring >>> > management (including builder file backup), user management >>> > -- Storage recipe handles storage nodes: network config, storage device >>> > config, package install, ring management. >>> > >>> > - chef databag that describes a swift cluster (eg: >>> mycluster_databag.json) >>> > -- proxy config settings >>> > -- memcached settings >>> > -- settings for all rings and devices >>> > -- basic user settings >>> > -- account management >>> > >>> > - chef "spiceweasel" file that auto-vivifies the infrastructure: (eg: >>> > mycluster_infra.yaml) >>> > -- uploads cookbooks >>> > -- uploads roles >>> > -- uploads the cluster's databag >>> > -- kicks off node provisioning by requesting from infrastructure API >>> (ec2 or >>> > what have you) the following: >>> > --- chef roles applied (role[swift:proxy] or role[swift:storage]) >>> > --- server flavor >>> > --- storage device configs >>> > --- hostname >>> > --- proxy and storage network details >>> > >>> > By calling this spiceweasel file, the infrastructure can leap into >>> > existence. >>> > >>> > I'm more or less done with all this stuff - and I'd really appreciate >>> > conceptual feedback before I take out all the non-sense code I have in >>> the >>> > files and publish. >>> > >>> > Many thanks! Happy spring, northern hemispherians! >>> > -judd >>> > >>> > Judd Maltin >>> > T: 917-882-1270 >>> > F: 501-694-7809 >>> > A loving heart is never wrong. >>> > >>> > >>> > >>> > >>> > _______________________________________________ >>> > Mailing list: https://launchpad.net/~openstack >>> > Post to : [email protected] >>> > Unsubscribe : https://launchpad.net/~openstack >>> > More help : https://help.launchpad.net/ListHelp >>> > >>> > >>> >>> _______________________________________________ >>> Mailing list: https://launchpad.net/~openstack >>> Post to : [email protected] >>> Unsubscribe : https://launchpad.net/~openstack >>> More help : https://help.launchpad.net/ListHelp >>> >> >> > > > -- > Judd Maltin > T: 917-882-1270 > F: 501-694-7809 > A loving heart is never wrong. > > > >
_______________________________________________ Mailing list: https://launchpad.net/~openstack Post to : [email protected] Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
