Re: [Openstack] [Swift] Administration Interfaces/Tools for Swift
Theres actually 2 diamond plugin's for swift now. I added the one we use internally at Rackspace a week or two ago https://github.com/BrightcoveOS/Diamond/wiki/collectors-OpenstackSwiftReconCollector . In addition to these and Zenoss theres also been nagios and collectd plugins floating around. At least in our deployments Swifts built in statsd metrics become a bit overwhelming (both in volume of metrics and what metrics are actually being reported). This is probably just a side effect of the size of our clusters. We ended up sticking with our original statsd/metric setup (a combo of some proxy middleware, and translating a bunch of Swift's syslog lines to statsd events). -- Florian Hines | @pandemicsyn http://about.me/pandemicsyn On Tuesday, January 29, 2013 at 12:16 PM, Dieter Plaetinck wrote: On Tue, 29 Jan 2013 15:24:31 -0200 Gui Maluf guimal...@gmail.com (mailto:guimal...@gmail.com) wrote: Hi guys, I'm searching for a Swift Interface that integrates Administration and maybe monitoring. I've checked SwiftStack http://swiftstack.com/ and Gladinethttp://gladinet.blogspot.com.br/2012/06/basic-layer-on-top-of-openstack-swift.html, but both are not opensource and require money to be used. There is something available that could ease admins tasks on swift? Thanks in advance! for monitoring, you can use https://github.com/Dieterbe/graph-explorer (graphite dashboard) which displays data (amongst others) from https://github.com/BrightcoveOS/Diamond (which has a swift plugin) as well as from the built in statsd support into swift i'm the author of graph-explorer and the diamond openstack swift plugin (i wrote these things cause i also couldn't find anything existing) i actually plan to write a blog post about these things soonish. Dieter ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net (mailto:openstack@lists.launchpad.net) Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift] Interested in implementing swift ring builder server
Hi Mark, I forgot to grab the blueprint but I finally got around to start working on this. I'll hopefully have something to put up for review next week. This version just support's basic GET/POST ops to retrieve the rings or to view/alter/add devices to the ring in bulk. Having someone else adding features as well would be awesome! -- Florian Hines | @pandemicsyn http://about.me/pandemicsyn On Monday, June 18, 2012 at 2:23 PM, Mark Gius wrote: Hello Swifters, I've got some interns working with me this summer and I had a notion that they might take a stab at the swift ring builder server blueprint that's been sitting around for a while (https://blueprints.launchpad.net/swift/+spec/ring-builder-server). As a first step I figured that the ring-builder-server would be purely an alternative for the swift-ring-builder CLI, with a future iteration adding support for deploying the rings to all servers in the cluster. I'm currently planning on making the ring-builder server be written and deployed like the account/container/etc servers, although I imagine the implementation will be a lot simpler. Is anybody else already working on this and forgot to update the blueprint? If not can I get the blueprint assigned to me on launchpad? Username 'markgius'. Or if there's some other process I need to go through please let me know. Mark ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net (mailto:openstack@lists.launchpad.net) Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Swift] Interested in implementing swift ring builder server
I just need to clean it up at this point (mainly the rebalance). Since the rebalance (i.e. swift-ring-builder rebalance) can take minutes to complete when you have lots of devices I need to move it off to a background task. -- Florian Hines | @pandemicsyn http://about.me/pandemicsyn On Tuesday, June 19, 2012 at 2:06 PM, Mark Gius wrote: Alright. Is this something you're more or less done with and are just cleaning up or have you just started? I've got my guys reading up on swift and are about ready to dive in with the coding, so if you're more or less done I'll find another project for them to work on. If you're just started (or even better, not yet started), would you mind if I stole this from you? Mark On Tue, Jun 19, 2012 at 11:59 AM, Florian Hines florian.hi...@gmail.com (mailto:florian.hi...@gmail.com) wrote: Hi Mark, I forgot to grab the blueprint but I finally got around to start working on this. I'll hopefully have something to put up for review next week. This version just support's basic GET/POST ops to retrieve the rings or to view/alter/add devices to the ring in bulk. Having someone else adding features as well would be awesome! -- Florian Hines | @pandemicsyn http://about.me/pandemicsyn On Monday, June 18, 2012 at 2:23 PM, Mark Gius wrote: Hello Swifters, I've got some interns working with me this summer and I had a notion that they might take a stab at the swift ring builder server blueprint that's been sitting around for a while (https://blueprints.launchpad.net/swift/+spec/ring-builder-server). As a first step I figured that the ring-builder-server would be purely an alternative for the swift-ring-builder CLI, with a future iteration adding support for deploying the rings to all servers in the cluster. I'm currently planning on making the ring-builder server be written and deployed like the account/container/etc servers, although I imagine the implementation will be a lot simpler. Is anybody else already working on this and forgot to update the blueprint? If not can I get the blueprint assigned to me on launchpad? Username 'markgius'. Or if there's some other process I need to go through please let me know. Mark ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net (mailto:openstack@lists.launchpad.net) Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] availability/performance sensors/probes
If John Dickinson can steal me a 30 minute block at the conference I'll probably be giving a talk about it, but we (Rackspace) started switching to Graphite back in December. We're basically just following the etsy cookbook to graph all the things!. We're using https://github.com/pandemicsyn/swift-informant to fire events to statsd. It takes care of answering questions like: How many Object GET 200's are we currently getting per second. How many container ops are we doing per second. What was the average request time of container HEAD's between 4-5PM last tuesday (which always seems to lead to the question of why are they so much slower today…oh look that node is having a weird hw issue)? Swift's also really good about dumping info to the error log. We convert the majority of those log lines to events thats get fired to statsd using https://github.com/pandemicsyn/statsdlog. That lets us track everything from container-replicator timeouts, auth service retries, to OSError's on the object servers (think we're tracking about 25-30 log line patterns at the moment). The last piece is just a hacked version of the swift-recon cli. It's what reports async-pending's, replication times, etc to graphite. Right now it gets tied together by tiny hackish Flask app that generates some tv dashboard's and will probably start doing the monitoring/alerting for the traffic prediction/confidence bands (experimenting with just doing it with an irc bot). -- Florian Hines | @pandemicsyn http://about.me/pandemicsyn On Wednesday, February 22, 2012 at 2:50 AM, Jasper Capel wrote: I've uploaded the checks we use in production here at Spil Games to https://github.com/spilgames/swift. Besides check_swift (which is a functional test) everything's meant to gather statistics from the cluster and we're looking to replace that with a Graphite-based solution to avoid having to parse access logs and having more real-time metrics available. Nothing fancy, but it may be of use to someone. Jasper On Feb 21, 2012, at 11:54 PM, Tim Bell wrote: This does bring up a more generic problem of sharing the availability/performance code for all of the OpenStack components. At the design summit, this was proposed as one of the example use cases of the OpenStack community forge (I forget the exact name) but it was intended as a place for sharing code/procedures which were not intended to be part of the core but may be of interest to others. Was anything set up along these lines ? A set of production quality Nagios/Ganglia sensors would be very interesting if someone has these Tim -Original Message- From: openstack-bounces+tim.bell=cern...@lists.launchpad.net (mailto:cern...@lists.launchpad.net) [mailto:openstack-bounces+tim.bell=cern...@lists.launchpad.net (mailto:cern...@lists.launchpad.net)] On Behalf Of Jasper Capel Sent: 21 February 2012 18:29 To: John Dickinson Cc: openstack@lists.launchpad.net (mailto:openstack@lists.launchpad.net) Subject: Re: [Openstack] swprobe: swift middleware for sending metrics to graphite using statsd Hi John, Apparently my google-fu is not up to snuff, as I wasn't aware of that project. Had I been, I probably would've just extemded that one. :) Cheers, Jasper From: John Dickinson [m...@not.mn (mailto:m...@not.mn)] Sent: Tuesday, February 21, 2012 5:44 PM To: Jasper Capel Cc: openstack@lists.launchpad.net (mailto:openstack@lists.launchpad.net) Subject: Re: [Openstack] swprobe: swift middleware for sending metrics to graphite using statsd That's great. Have you by any chance seen https://github.com/pandemicsyn/swift-informant? It's something similar that we've been playing with at Rackspace. --John On Feb 21, 2012, at 10:36 AM, Jasper Capel wrote: Hi all, I'm announcing a piece of Swift middleware, swprobe [1], designed to gather run-time metrics and ship them off to Graphite [2] for near real-time monitoring. Currently it sends out bytes up- and downloaded per account, http methods and response codes and timings in miliseconds on each call. To be able to use this you need Graphite [2]. You also need statsd running, preferably on the local machine since there potentially many small UDP packets are being sent out. Please also note that we have not yet tested this with production workloads. [1] - https://github.com/spilgames/swprobe [2] - http://graphite.wikidot.com/ [3] - https://github.com/etsy/statsd Best regards, -- Jasper Capel Lead Infrastructure Engineer W http://www.spilgames.com | S jwcapel-spil ___ Mailing list: https://launchpad.net/~openstack
Re: [Openstack] HA for Swift proxy and object server processes
On Friday, January 20, 2012 at 12:20 AM, Michael Barton wrote: There isn't any monitoring built into Swift, but basic availability and such shouldn't be difficult to monitor with the third-party tools of your choosing. -Michael The Zenoss guys actually have pretty complete Swift zenpack: https://github.com/zenoss/ZenPacks.zenoss.OpenStackSwift#readme You'll need to enable the recon middleware on the object servers to use it though. -- Florian Hines | @pandemicsyn http://about.me/pandemicsyn ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Separation of IRC Channels
On May 3, 2011, at 5:22 PM, Ant Messerli wrote: Hey all, I wanted to see what everyones opinions were on separating out some of the IRC channels. When the project started we decided to keep #openstack as the primary channel since the project was small at the time. Over the last year with the project growing, the traffic in the #openstack channel have increased quite a bit. The #openstack channel is the place people go for dev talk, support, and Openstack conversation. My proposal is that we separate out development talk into separate channels and keep #openstack as the place for new people to the project to ask questions and get help. It might help separate a lot of the cross project talk. I think it helps out for those trying to focus on a specific project and may actually get more talk to occur in the rooms. We could use these to start: #openstack – Help and Getting Started #openstack-meeting – Community Meetings #openstack-nova – All Nova Development #openstack-swift – All Swift Development #openstack-glance – All Glance Development Obviously there are other projects as well that we could look at making channels for but I think these are the ones that are getting a majority of the traffic today. Thoughts, comments? -Ant I think you're totally right. There's 229 people in the channel right now. Between all the bot chatter and chatter about project's that don't matter/relate to me, I've kind of stopped paying attention to the channel unless a few specific keywords get said. Florian Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
