[Openstack] StackTach and Stacy github repos have moved ...

2013-03-25 Thread Sandy Walsh
Hi, how are you? Me? Right as rain, thanks for asking. Due to a recent reorg of the Rackspace github repo, StackTach and Stacky are now moved under the rackerlabs organization. The new coordinates are: https://github.com/rackerlabs/stacktach https://github.com/rackerlabs/stacky Please update

Re: [Openstack] StackTach and Stacy github repos have moved ...

2013-03-25 Thread Sandy Walsh
working towards that. We just had some internal tactical stuff we had to deal with. Getting everyone freed up now. Cheers! -S Best, -jay On 03/25/2013 09:42 AM, Sandy Walsh wrote: Hi, how are you? Me? Right as rain, thanks for asking. Due to a recent reorg of the Rackspace github repo

[Openstack] [StackTach][Metering] Nova summary report for the PHB in your life ...

2013-02-14 Thread Sandy Walsh
Hey! We just added a feature to StackTach for generating 24-hr summary reports. The output can be seen here: https://gist.github.com/SandyWalsh/4946226 It will identify requests failures, with timing information, by major actions (create, resize, rescue, delete, snapshot, etc) by image type.

Re: [Openstack] [openstack-dev] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ...

2012-11-08 Thread Sandy Walsh
From: Doug Hellmann [doug.hellm...@dreamhost.com] Sent: Thursday, November 08, 2012 1:54 PM To: Sandy Walsh Cc: Eoghan Glynn; OpenStack Development Mailing List; openstack@lists.launchpad.net Subject: Re: [Openstack] [openstack-dev] [metering][ceilometer] Unified

Re: [Openstack] [openstack-dev] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ...

2012-11-07 Thread Sandy Walsh
Hey! (sorry for the top-posting, crappy web client) There is a periodic task already in the compute manager that can handle this: https://github.com/openstack/nova/blob/master/nova/compute/manager.py#L3021 There seems to be some recent (to me) changes in the manager now wrt the

Re: [Openstack] A point in my mind which may be already implemented

2012-11-04 Thread Sandy Walsh
to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp I think the notification system is what you really want. http://wiki.openstack.org/SystemUsageData Sandy Walsh made some

Re: [Openstack] [openstack-dev] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ...

2012-11-02 Thread Sandy Walsh
Agreed! Code good. -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Jeffrey Budzinski [jeffr...@yahoo-inc.com] Sent: Friday, November 02, 2012 10:11

Re: [Openstack] Is there any documents explain nova source code in detail?

2012-11-01 Thread Sandy Walsh
I wrote a couple of articles on this: http://www.sandywalsh.com/2012/04/openstack-nova-internals-pt1-overview.html http://www.sandywalsh.com/2012/09/openstack-nova-internals-pt2-services.html and, to a lesser extent: http://www.sandywalsh.com/2012/10/debugging-openstack-with-stacktach-and.html

[Openstack] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ...

2012-11-01 Thread Sandy Walsh
Hey! Here's a first pass at a proposal for unifying StackTach/Ceilometer and other instrumentation/metering/monitoring efforts. It's v1, so bend, spindle, mutilate as needed ... but send feedback! http://wiki.openstack.org/UnifiedInstrumentationMetering Thanks, Sandy

Re: [Openstack] [metering][ceilometer] Unified Instrumentation, Metering, Monitoring ...

2012-11-01 Thread Sandy Walsh
with different sinks. Hmm, really not a fan of the decorator approach. Makes the code really ugly. They'd be everywhere. Not sure if I can get over it. :D -S On Nov 1, 2012, at 1:17 PM, Sandy Walsh wrote: Hey! Here's a first pass at a proposal for unifying StackTach/Ceilometer and other

[Openstack] Debugging OpenStack - Video on StackTach v2 and Stacky

2012-10-31 Thread Sandy Walsh
Hey! As promised at the summit the latest changes to StackTach are up on github. This is a major change from the original StackTach I introduced earlier this year (and left to wither). Also, I'm including Stacky, which is a new command line tool for StackTach. Here's a video that explains

Re: [Openstack] about nova-schedule queues

2012-10-30 Thread Sandy Walsh
As with most services there are two queues, but the scheduler has one extra: 1. The general round-robin queue. Any worker of that class can process the event. But, only one worker will handle the event. 2. The specific worker queue. Used when I want an event to go to a specific worker, for

Re: [Openstack] Instrumentation Monitoring Next Step - quick meet up

2012-10-29 Thread Sandy Walsh
Ugh, I just realized I have a conflict. Can we push it an hour later? (sorry!) -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Annie Cheng

Re: [Openstack] Instrumentation Monitoring Next Step - quick meet up

2012-10-29 Thread Sandy Walsh
np ... I realize it's tricky to change this stuff. I'll see if I can jump on remote. -S From: Annie Cheng [ann...@yahoo-inc.com] Sent: Monday, October 29, 2012 1:44 PM To: 'doug.hellm...@dreamhost.com'; Sandy Walsh Cc: 'openstack@lists.launchpad.net' Subject: Re

Re: [Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup

2012-10-25 Thread Sandy Walsh
, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup On 24/10/12 23:35 +, Sandy Walsh wrote: Hey y'all, Great to chat during the summit last week, but it's been a crazy few days of catch-up since then. The main takeaway for me was the urgent need to get some common

[Openstack] Ceilometer, StackTach, Tach / Scrutinize, CloudWatch integration ... Summit followup

2012-10-24 Thread Sandy Walsh
Hey y'all, Great to chat during the summit last week, but it's been a crazy few days of catch-up since then. The main takeaway for me was the urgent need to get some common libraries under these efforts. So, to that end ... 1. To those that asked, I'm going to get my slides / video

Re: [Openstack] (no subject)

2012-10-16 Thread Sandy Walsh
+1 From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Joe Savak [joe.sa...@rackspace.com] Sent: Tuesday, October 16, 2012 5:50 PM To: openstack@lists.launchpad.net Subject: [Openstack] (no

[Openstack] StackTach unconference talk

2012-10-15 Thread Sandy Walsh
... is Wednesday 11:50 in 'Maggie' room. If you're interested in debugging tools and performance monitoring ... see you there! -Sandy ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe :

Re: [Openstack] Versioning for notification messages

2012-10-10 Thread Sandy Walsh
4:47 AM To: Sandy Walsh Subject: RE: Versioning for notification messages Hi Sandy, Are you making changes to StackTach to keep up with the notification system, or changes to the notification system to make StackTach even better ? Cheers, Phil -Original Message- From: Sandy Walsh

Re: [Openstack] Bottleneck of message queue

2012-10-10 Thread Sandy Walsh
Hey Howard, Queues are generally in memory, but you may turn on persistent (disk) queues in your environment. So that's your limitation. Having rabbitmq on a different server is a good idea. Also, Queues are only used for control, not user data, so they shouldn't be that big of a burden.

Re: [Openstack] Versioning for notification messages

2012-10-09 Thread Sandy Walsh
While I think the idea has merit, it's going to be a tough thing to version. (I'm neck deep in the notifications right now making a major revision to StackTach) We'd have to version the context, the instance state dictionary, the cpu info and the related payloads for each type of message since

[Openstack] Blog post: OpenStack Nova Internals - Pt2 - Services

2012-09-27 Thread Sandy Walsh
Went a little crazy on this one ... please let me know about the many blatant mistakes so I can edit quickly :) http://www.sandywalsh.com/2012/09/openstack-nova-internals-pt2-services.html Thanks! -S ___ Mailing list: https://launchpad.net/~openstack

[Openstack] Blueprint Proposal: Inflight Monitoring Service ...

2012-08-28 Thread Sandy Walsh
Not sure if an email like this gets sent automagically by LP anymore, so here goes ... I'd love to get some feedback on it. Thanks! -S The Blueprint https://blueprints.launchpad.net/nova/+spec/monitoring-service The Spec http://wiki.openstack.org/PerformanceMonitoring The Branch:

Re: [Openstack] OpenStack Summit Tracks Topics

2012-08-14 Thread Sandy Walsh
Perhaps off topic, but ... One of the things I've noticed at the last couple of summits are the number of new attendees that could really use an OpenStack 101 session. Many of them are on fact-finding missions and their understanding of the architecture is 10,000'+. Usually when conf's get

[Openstack] Removing support for KVM Hypervisor ...

2012-08-10 Thread Sandy Walsh
Sorry George, couldn't resist. :) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp

Re: [Openstack] Removing support for KVM Hypervisor ...

2012-08-10 Thread Sandy Walsh
haha ... oh no. It's joke. Long story. Not going to happen :) From: chaohua wang [chwang...@gmail.com] Sent: Friday, August 10, 2012 5:31 PM To: Sandy Walsh Subject: Re: [Openstack] Removing support for KVM Hypervisor ... Hi I posted my question to openstack

Re: [Openstack] Cannot pass hint to Nova Scheduler

2012-08-07 Thread Sandy Walsh
to check the stats in HostState class as exists in memory? cc'ing Sandy Walsh, who is vastly more familiar with the scheduler than I am :) Sandy, see Heng's question above... seems like a great question to me -- and also a possible mini-project for someone to work on that would add a scheduler

Re: [Openstack] Keyring support in openstack

2012-07-31 Thread Sandy Walsh
I added similar functionality to novaclient. There is a --nocache option to ignore the cache and the 'import keyring' check is just to keep it from crashing on non-supported systems. I key off the following: auth url + username + service name + region to prevent conflicts with other users.

Re: [Openstack] [nova] Proposal to add Yun Mao to nova-core

2012-07-18 Thread Sandy Walsh
Ab-so-lutely! +1 From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Vishvananda Ishaya [vishvana...@gmail.com] Sent: Wednesday, July 18, 2012 8:10 PM To:

Re: [Openstack] Happy Birthday OpenStack!

2012-07-17 Thread Sandy Walsh
US Only ... booo! From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of John Purrier [j...@openstack.org] Sent: Tuesday, July 17, 2012 11:34 AM To:

Re: [Openstack] Performance metrics

2012-06-20 Thread Sandy Walsh
Hi That's been my focus for a while now. We're using Tach to instrument openstack, quantum, glance and a bunch of other components. https://github.com/ohthree/tach Also, there's StackTach which will consume the notifications and give you a real-time display of what's happening in the system

Re: [Openstack] Where to add the performance increasing method in openstack

2012-05-24 Thread Sandy Walsh
Hi! You would want to report this information to the Scheduler (probably via the db) so it can make more informed decisions. A new Weight Function in the scheduler would be the place to add it specifically. We currently track the number of VM I/O operations being performed on each Compute

Re: [Openstack] nova state machine simplification and clarification

2012-05-24 Thread Sandy Walsh
I don't think so. .../nova/etc seems to be where the conf files are living. Look forward to it! -S From: Yun Mao [yun...@gmail.com] Sent: Thursday, May 24, 2012 11:46 AM To: Sandy Walsh Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] nova

Re: [Openstack] Where to add the performance increasing method in openstack

2012-05-24 Thread Sandy Walsh
candidates. Cheers, Sandy From: Diego Parrilla [diego.parrilla.santama...@gmail.com] Sent: Thursday, May 24, 2012 2:01 PM To: Sandy Walsh Cc: sarath zacharia; Nagaraju Bingi; openstack@lists.launchpad.net Subject: Re: [Openstack] Where to add the performance increasing method in openstack Hi Sandy

Re: [Openstack] nova state machine simplification and clarification

2012-05-23 Thread Sandy Walsh
Hi Yun, I like the direction you're going with this. Unifying these three enums would be a great change. Honestly it's really just combing two enums (vm task) and using power state as a tool for reconciliation (actual != reality). Might I suggest using graphvis instead of a spreadsheet? That

Re: [Openstack] Nova idear, thoughts wanted.

2012-05-03 Thread Sandy Walsh
Agreed. That's largely the effort of the Orchestration group. From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Joshua Harlow [harlo...@yahoo-inc.com] Sent:

Re: [Openstack] Compute State Machine diagram ... (orchestration? docs?)

2012-05-03 Thread Sandy Walsh
PM To: Sandy Walsh Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Compute State Machine diagram ... (orchestration? docs?) Hi Sandy: On May 2, 2012, at 12:10 PM, Sandy Walsh wrote: Here's a little diagram I did up this morning for the required vm_state / task_state transitions

[Openstack] Compute State Machine diagram ... (orchestration? docs?)

2012-05-02 Thread Sandy Walsh
Here's a little diagram I did up this morning for the required vm_state / task_state transitions for compute api operations. http://dl.dropbox.com/u/166877/PowerStates.pdf Might be useful to the orchestration effort (or debugging in general) Cheers, Sandy

Re: [Openstack] Question on notifications

2012-04-26 Thread Sandy Walsh
. Simple and Change will turn to filters/weights soon. Depends on your installation. -Sandy From: Joshua Harlow [harlo...@yahoo-inc.com] Sent: Thursday, April 26, 2012 5:07 PM To: Sandy Walsh; openstack Subject: Re: [Openstack] Question on notifications Thx. With these messages, instead

Re: [Openstack] Question on notifications

2012-04-25 Thread Sandy Walsh
You want these events: scheduler.run_instance.start (generated when scheduling begins) scheduler.run_instance.scheduled (when a host is selected. one per instance) scheduler.run_instance.end (all instances placed) The .scheduled event will have the target hostname in it in the weighted_host key

Re: [Openstack] Monitoring / Billing Architecture proposed

2012-04-24 Thread Sandy Walsh
I think we have support for this currently in some fashion, Dragon? -S On 04/24/2012 12:55 AM, Loic Dachary wrote: Metering needs to account for the volume of data sent to external network destinations ( i.e. n4 in http://wiki.openstack.org/EfficientMetering ) or the disk I/O etc. This

Re: [Openstack] Using Nova APIs from Javascript: possible?

2012-04-24 Thread Sandy Walsh
Due to the redirect nature of the auth system we may need JSONP support for this to work. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help :

Re: [Openstack] Using Nova APIs from Javascript: possible?

2012-04-24 Thread Sandy Walsh
are mandatory. -S Nick On Tue, Apr 24, 2012 at 8:57 PM, Sandy Walsh sandy.wa...@rackspace.com mailto:sandy.wa...@rackspace.com wrote: Due to the redirect nature of the auth system we may need JSONP support for this to work

Re: [Openstack] Monitoring / Billing Architecture proposed

2012-04-23 Thread Sandy Walsh
Flavor information is copied to the Instance table on creation so the Flavors can change and still be tracked in the Instance. It may just need to be sent in the notification payload. The current events in the system are documented here: http://wiki.openstack.org/SystemUsageData -Sandy On

Re: [Openstack] Monitoring / Billing Architecture proposed

2012-04-23 Thread Sandy Walsh
StackTach is a Django-based web interface for capturing, displaying and navigating OpenStack notifications https://github.com/rackspace/stacktach -S On 04/23/2012 04:26 PM, Luis Gervaso wrote: Joshua, I have performed a create instance operation and here is an example data obtained from

Re: [Openstack] New Gerrit version (and server)

2012-04-13 Thread Sandy Walsh
Sounds awesome! Looking forward to Draft Changes, much needed. -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of James E. Blair [cor...@inaugust.com]

Re: [Openstack] [Nova] removing nova-direct-api

2012-04-09 Thread Sandy Walsh
/me wears black armband. Would love to see DirectApi v2 be the de facto OS API implementation. The spec could be the same, it's just a (better) implementation detail. -S On 04/09/2012 03:58 PM, Vishvananda Ishaya wrote: +1 to removal. I just tested to see if it still works, and due to our

Re: [Openstack] [Nova-orchestration] Preliminary analysis of SpiffWorkflow

2012-04-06 Thread Sandy Walsh
forward to seeing that larger project! -S From: Ziad Sawalha Sent: Friday, April 06, 2012 4:53 PM To: Sriram Subramanian; Dugger, Donald D; Sandy Walsh Cc: nova-orchestrat...@lists.launchpad.net; openstack@lists.launchpad.net Subject: Re: [Openstack] [Nova

Re: [Openstack] [Nova-orchestration] Preliminary analysis of SpiffWorkflow

2012-04-06 Thread Sandy Walsh
From what I've seen Spliff doesn't specify ... the containing application has to deal with persistence. -S From: Yun Mao [yun...@gmail.com] Sent: Friday, April 06, 2012 5:38 PM To: Ziad Sawalha Cc: Sriram Subramanian; Dugger, Donald D; Sandy Walsh

Re: [Openstack] [Nova-orchestration] Thoughts on Orchestration (was Re: Documentation on Caching)

2012-04-03 Thread Sandy Walsh
Can't wait to hear about it Ziad! Very cool! -S From: Ziad Sawalha Sent: Tuesday, April 03, 2012 6:56 PM To: Sriram Subramanian; Dugger, Donald D; Sandy Walsh Cc: nova-orchestrat...@lists.launchpad.net; openstack@lists.launchpad.net Subject: Re

Re: [Openstack] Running code on instance start/terminate

2012-03-28 Thread Sandy Walsh
named Carrot that uses amqplib. Would it be better to use it instead? Em 27 de março de 2012 12:42, Sandy Walsh sandy.wa...@rackspace.com mailto:sandy.wa...@rackspace.com escreveu: I believe '.exists' is sent via a periodic update in compute.manager

Re: [Openstack] Caching strategies in Nova ...

2012-03-23 Thread Sandy Walsh
caching stuff is easy-peasy, making sure it is invalidated on all servers in all conditions, not so easy... On 3/22/12 4:26 PM, Sandy Walsh sandy.wa...@rackspace.com wrote: We're doing tests to find out where the bottlenecks are, caching is the most obvious solution, but there may

Re: [Openstack] Caching strategies in Nova ...

2012-03-23 Thread Sandy Walsh
in all conditions, not so easy... On 3/22/12 4:26 PM, Sandy Walsh sandy.wa...@rackspace.com wrote: We're doing tests to find out where the bottlenecks are, caching is the most obvious solution, but there may be others. Tools like memcache do a really good job of sharing memory across servers so

Re: [Openstack] Caching strategies in Nova ...

2012-03-23 Thread Sandy Walsh
into a general (albeit obvious) solution such as caching. Sandy Walsh sandy.wa...@rackspace.com said: We're doing tests to find out where the bottlenecks are, caching is the most obvious solution, but there may be others. Tools like memcache do a really good job of sharing memory across servers so

Re: [Openstack] Caching strategies in Nova ...

2012-03-23 Thread Sandy Walsh
... On 3/22/12 4:26 PM, Sandy Walsh sandy.wa...@rackspace.com wrote: We're doing tests to find out where the bottlenecks are, caching is the most obvious solution, but there may be others. Tools like memcache do a really good job of sharing memory across servers so we don't have

Re: [Openstack] Caching strategies in Nova ...

2012-03-23 Thread Sandy Walsh
@lists.launchpad.net [mailto:openstack- bounces+gabe.westmaas=rackspace@lists.launchpad.net] On Behalf Of Sandy Walsh Sent: Friday, March 23, 2012 7:58 AM To: Joshua Harlow Cc: openstack Subject: Re: [Openstack] Caching strategies in Nova ... Was reading up some more on cache invalidation

Re: [Openstack] Caching strategies in Nova ...

2012-03-23 Thread Sandy Walsh
You can. The sanctioned approach is to use Yagi with a feed into something like PubSubHubBub that lives on the public interweeb. It's just an optional component. -S On 03/23/2012 12:20 PM, Kevin L. Mitchell wrote: On Fri, 2012-03-23 at 13:43 +, Gabe Westmaas wrote: However, I kind of

Re: [Openstack] Caching strategies in Nova ...

2012-03-23 Thread Sandy Walsh
Ugh (reply vs reply-all again) On 03/23/2012 02:58 PM, Joshua Harlow wrote: Right, Lets fix the problem, not add a patch that hides the problem. U can’t put lipstick on a pig, haha. Its still a pig... When stuff is expensive to compute, caching is the only option (yes?). Whether that

Re: [Openstack] Caching strategies in Nova ...

2012-03-23 Thread Sandy Walsh
Was the db on a separate server or loopback? On 03/23/2012 05:26 PM, Mark Washenberger wrote: Johannes Erdfelt johan...@erdfelt.com said: MySQL isn't exactly slow and Nova doesn't have particularly large tables. It looks like the slowness is coming from the network and how many queries

Re: [Openstack] Caching strategies in Nova ...

2012-03-23 Thread Sandy Walsh
. I just see a ton of specific performance problems that are easier to address one by one, rather than diving into a general (albeit obvious) solution such as caching. Sandy Walsh sandy.wa...@rackspace.com mailto:sandy.wa...@rackspace.com said

[Openstack] Caching strategies in Nova ...

2012-03-22 Thread Sandy Walsh
o/ Vek and myself are looking into caching strategies in and around Nova. There are essentially two approaches: in-process and external (proxy). The in-process schemes sit in with the python code while the external ones basically proxy the the HTTP requests. There are some obvious pro's and

Re: [Openstack] Caching strategies in Nova ...

2012-03-22 Thread Sandy Walsh
We're doing tests to find out where the bottlenecks are, caching is the most obvious solution, but there may be others. Tools like memcache do a really good job of sharing memory across servers so we don't have to reinvent the wheel or hit the db at all. In addition to looking into caching

Re: [Openstack] nova zone and availability_zone

2012-03-20 Thread Sandy Walsh
Availability Zone is an EC2 concept. Zones were a sharding scheme for Nova. Zones are being renamed to Cells to avoid further confusion. Availability Zones will remain the same. Hope it helps! -S From:

Re: [Openstack] [OpenStack] using xenapi hypervisor

2012-03-20 Thread Sandy Walsh
http://wiki.openstack.org/XenServer/Development#Legacy_way_to_Prepare_XenServer From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Eduardo Nunes

Re: [Openstack] Announcing StackTach ...

2012-02-21 Thread Sandy Walsh
Thanks ... I'll have a look! -S On 02/21/2012 04:57 PM, Cole wrote: very cool. If there is any interest in extending the tool and making it pluggable to work with other wire protocols i'd think the openmama http://www.openmama.org/ project would be an interesting possibility. nice work!

Re: [Openstack] Announcing StackTach ...

2012-02-21 Thread Sandy Walsh
this ability be lost with Horizon integration? -S On 02/20/2012 07:40 PM, Devin Carlen wrote: Sandy, this is great work! I think it would be worth integrating this into a view in Horizon for Folsom timeframe. Devin On Monday, February 20, 2012 at 12:15 PM, Sandy Walsh wrote: Hey! Last

[Openstack] Announcing StackTach ...

2012-02-20 Thread Sandy Walsh
Hey! Last week I started on a little debugging tool for OpenStack based on AMQP events that I've been calling StackTach. It's really handy for watching the flow of an operation through the various parts of OpenStack. It consists of two parts: 1. The Worker. Sits somewhere on your OpenStack

Re: [Openstack] Remove Zones code - FFE

2012-02-12 Thread Sandy Walsh
were working on providing the necessary functionality in Keystone but stopped when we heard of the alternative solution. We could resume the conversation about what is needed on the Keystone side and implement if needed. Z From: Sandy Walsh sandy.wa...@rackspace.com mailto:sandy.wa

Re: [Openstack] Propose to make Monsyne Dragon a nova core developer

2012-02-07 Thread Sandy Walsh
Yessir! +1 From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Matt Dietz [matt.di...@rackspace.com] Sent: Monday, February 06, 2012 6:48 PM To:

[Openstack] Remove Zones code - FFE

2012-02-01 Thread Sandy Walsh
As part of the new (and optional) Zones code coming down the pipe, part of this is to remove the old Zones implementation. More info in the merge prop: https://review.openstack.org/#change,3629 So, can I? can I? Huh? ___ Mailing list:

Re: [Openstack] Remove Zones code - FFE

2012-02-01 Thread Sandy Walsh
integration was the biggest pita. I can keep this branch fresh with trunk for when we're ready to pull the trigger. -S From: Joshua McKenty [jos...@pistoncloud.com] Sent: Wednesday, February 01, 2012 4:45 PM To: Vishvananda Ishaya Cc: Sandy Walsh; openstack

Re: [Openstack] [Nova] Essex dead wood cutting

2012-01-27 Thread Sandy Walsh
I'll be taking the existing Zones code out of API and Distributed Scheduler. The new Zones infrastructure is an optional component. -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net

[Openstack] [Scaling][Orchestration] Zone changes. WAS: [Question #185840]: Multi-Zone finally working on ESSEX but cant nova list (KeyError: 'uuid') + doubts

2012-01-26 Thread Sandy Walsh
blueprint soon. Hope it helps, Sandy From: boun...@canonical.com [boun...@canonical.com] on behalf of Alejandro Comisario [question185...@answers.launchpad.net] Sent: Thursday, January 26, 2012 8:50 AM To: Sandy Walsh Subject: Re: [Question #185840]: Multi

Re: [Openstack] [Scaling][Orchestration] Zone changes. WAS: [Question #185840]: Multi-Zone finally working on ESSEX but cant nova list (KeyError: 'uuid') + doubts

2012-01-26 Thread Sandy Walsh
this is what you're looking for? -S From: Blake Yeager [blake.yea...@gmail.com] Sent: Thursday, January 26, 2012 12:13 PM To: Sandy Walsh Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] [Scaling][Orchestration] Zone changes. WAS: [Question #185840

Re: [Openstack] Proposal to limit decorator usage

2012-01-17 Thread Sandy Walsh
Seems kind of arbitrary doesn't it? Perhaps something about not using decorators with arguments instead? (since they're the biggest wart on the ass of Python) -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net

Re: [Openstack] Using fake drivers for testing

2012-01-16 Thread Sandy Walsh
I used the fake virt driver when I was testing zones. Very handy. From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Brebner, Gavin [gavin.breb...@hp.com] Sent:

Re: [Openstack] Nova Subteam Changes

2011-12-08 Thread Sandy Walsh
+1 From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Soren Hansen [so...@linux2go.dk] Sent: Thursday, December 08, 2011 6:09 AM To: Vishvananda Ishaya Cc:

[Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Sandy Walsh
For orchestration (and now the scheduler improvements) we need to know when an operation fails ... and specifically, which resource was involved. In the majority of the cases it's an instance_uuid we're looking for, but it could be a security group id or a reservation id. With most of the

Re: [Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Sandy Walsh
Sure, the problem I'm immediately facing is reclaiming resources from the Capacity table when something fails. (we claim them immediately in the scheduler when the host is selected to lessen the latency). The other situation is Orchestration needs it for retries, rescheduling, rollbacks and

Re: [Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Sandy Walsh
Exactly! ... or it could be handled in the notifier itself. From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Mark Washenberger

Re: [Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Sandy Walsh
To: Sandy Walsh Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] [Orchestration] Handling error events ... explicit vs. implicit Hi Sandy, I'm wondering if it is possible to change the scheduler's rpc cast to rpc call. This way the exceptions should be magically propagated back

Re: [Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Sandy Walsh
*removing our Asynchronous nature. (heh, such a key point to typo on) From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Sandy Walsh [sandy.wa

[Openstack] HPC with Openstack?

2011-12-02 Thread Sandy Walsh
I've recently had inquiries about High Performance Computing (HPC) on Openstack. As opposed to the Service Provider (SP) model, HPC is interested in fast provisioning, potentially short lifetime instances with precision metrics and scheduling. Real-time vs. Eventually. Anyone planning on using

Re: [Openstack] HPC with Openstack?

2011-12-02 Thread Sandy Walsh
Good point ... thanks for the clarification. -S From: Lorin Hochstein [lo...@isi.edu] Sent: Friday, December 02, 2011 9:47 AM To: Sandy Walsh Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] HPC with Openstack? As a side note, HPC means very different

Re: [Openstack] Proposal for Lorin Hochstein to join nova-core

2011-11-30 Thread Sandy Walsh
+1 ... good call! From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Vishvananda Ishaya [vishvana...@gmail.com] Sent: Tuesday, November 29, 2011 2:03 PM To:

Re: [Openstack] [nova-testing] Efforts for Essex

2011-11-23 Thread Sandy Walsh
method call so I need to do that How much effort would it be to make it into a better/more generic mox library? -S From: Soren Hansen [so...@linux2go.dk] Sent: Tuesday, November 22, 2011 7:38 PM To: Sandy Walsh Cc: openstack@lists.launchpad.net Subject: Re

Re: [Openstack] [nova-testing] Efforts for Essex

2011-11-23 Thread Sandy Walsh
:) yeah, you're completely misunderstanding me. So, you've made a much better StubOutWithMock() and slightly better stubs.Set() by (essentially) ignoring the method parameter checks and just focusing on the return type. Using your example: def test_something(self): def

Re: [Openstack] [nova-testing] Efforts for Essex

2011-11-23 Thread Sandy Walsh
I understand what you're proposing, but I'm backtracking a little. (my kingdom for you and a whiteboard in the same room :) I think that you could have a hybrid of your db.do_something(desired_return_value) and self.stubs.Set(nova.db, 'instance_get_by_uuid', fake_instance_get) (which I don't

Re: [Openstack] [nova-testing] Efforts for Essex

2011-11-23 Thread Sandy Walsh
haha ... worse email thread ever. I'll catch you on IRC ... we've diverged too far to make sense. -S From: Soren Hansen [so...@linux2go.dk] Sent: Wednesday, November 23, 2011 6:30 PM To: Sandy Walsh Cc: openstack@lists.launchpad.net Subject: Re

Re: [Openstack] [nova-testing] Efforts for Essex

2011-11-22 Thread Sandy Walsh
Excellent! I wrote a few blog posts recently, mostly based on my experience with openstack automated tests: http://www.sandywalsh.com/2011/06/effective-units-tests-and-integration.html http://www.sandywalsh.com/2011/08/pain-of-unit-tests-and-dynamically.html Would love to see some of those

Re: [Openstack] [nova-testing] Efforts for Essex

2011-11-22 Thread Sandy Walsh
I'm not a big fan of faking a database, not only for the reasons outlined already, but because it makes the tests harder to understand. I much prefer to mock the db call on a per-unit-test basis so you can see everything you need in a single file. Yes, this could mean some duplication across

Re: [Openstack] [nova-testing] Efforts for Essex

2011-11-22 Thread Sandy Walsh
the fake for both of these scenarios will be any easier than just having a stub. It seems like an unnecessary abstraction. -S From: Soren Hansen [so...@linux2go.dk] Sent: Tuesday, November 22, 2011 4:37 PM To: Sandy Walsh Cc: Jay Pipes; openstack

[Openstack] python-novaclient moved into gerrit

2011-11-16 Thread Sandy Walsh
Thanks the James and the rest of the CI team, python-novaclient is now moved from github to gerrit. Once we've migrated the old bugs over we'll nuke that repo. You know the drill ... see ya there. -S ___ Mailing list:

Re: [Openstack] OSAPI and Zones

2011-11-14 Thread Sandy Walsh
Jorge is correct. The zones stuff was added before the API was finalized and before the extensions mechanism was in place. We simply haven't taken the time to convert it yet. -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net

Re: [Openstack] Four compute-node, everytime the 1st and 2nd are choosen

2011-11-10 Thread Sandy Walsh
Are you using Diablo or Trunk? If you're using trunk the default scheduler is MultiScheduler, which uses Chance scheduler. I think Diablo uses Chance by default? --scheduler_driver Unless you've explicitly selected the LeastCostScheduler (which only exists in Diablo now) I wouldn't worry

Re: [Openstack] Host Aggregates ...

2011-11-10 Thread Sandy Walsh
, HasGPU, GeneratorBackup, PriorityNetwork Are we saying the same thing? Are there use cases that this approach couldn't handle? -S From: Armando Migliaccio [armando.migliac...@eu.citrix.com] Sent: Thursday, November 10, 2011 8:50 AM To: Sandy Walsh Cc

Re: [Openstack] Proposal to add Johannes Erdfelt to nova-core

2011-11-09 Thread Sandy Walsh
9 * 3 - 26 From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Brian Waldon [brian.wal...@rackspace.com] Sent: Wednesday, November 09, 2011 10:02 AM To:

Re: [Openstack] Proposal to add Kevin Mitchell to nova-core

2011-11-09 Thread Sandy Walsh
3746 ^ 0 From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Brian Waldon [brian.wal...@rackspace.com] Sent: Wednesday, November 09, 2011 9:59 AM To:

[Openstack] Host Aggregates ...

2011-11-09 Thread Sandy Walsh
Hi Armando, I finally got around to reading https://blueprints.launchpad.net/nova/+spec/host-aggregates. Perhaps you could elaborate a little on how this differs from host capabilities (key-value pairs associated with a service) that the scheduler can use when making decisions? The

Re: [Openstack] Submitting code to Novaclient

2011-11-02 Thread Sandy Walsh
You're doing it right, we've all just been tied up with some production stuff. I'll try and take some time this afternoon to clear the queue. Sorry for the delay ... and thanks for the submissions! -S From:

  1   2   3   >