Re: [Openstack] The right way to deprecate things in nova?

2012-06-13 Thread Mark Washenberger
Sean Dague sda...@linux.vnet.ibm.com said: On 06/12/2012 05:53 PM, Dan Prince wrote: snip Here's my current suggested path forward, which I'd like comments on: * keep the existing nova.utils deprecation functions (don't remove them) My take is why keep a 200-300 line set of

Re: [Openstack] cfg usage - option registration, global objects

2012-06-06 Thread Mark Washenberger
is going into common and in what form? I think I can be less disruptive if I'm involved in these discussions much earlier. Mark McLoughlin mar...@redhat.com said: On Tue, 2012-06-05 at 17:25 -0400, Mark Washenberger wrote: Mark McLoughlin mar...@redhat.com said: On Tue, 2012-06-05 at 12:21

Re: [Openstack] cfg usage - option registration, global objects

2012-05-31 Thread Mark Washenberger
Jay Pipes jaypi...@gmail.com said: On 05/29/2012 04:04 AM, Mark McLoughlin wrote: Adopting this pattern across all projects will actually help openstack-common more generally. For example, Russell is moving the RPC code into openstack-common and it has a bunch of configuration options. If

Re: [Openstack] nova state machine simplification and clarification

2012-05-18 Thread Mark Washenberger
Hi Yun, This proposal looks very good to me. I am glad you included in it the requirement that hard deletes can take place in any vm/task/power state. I however feel that a similar requirement exists for revert resize. It should be possible to issue a RevertResize command for any task_state

Re: [Openstack] Caching strategies in Nova ...

2012-03-23 Thread Mark Washenberger
it comin'! -S On 03/22/2012 11:53 PM, Mark Washenberger wrote: Working on this independently, I created a branch with some simple performance logging around the nova-api, and individually around glance, nova.db, and nova.rpc calls. (Sorry, I only have a local copy and its on a different

Re: [Openstack] Caching strategies in Nova ...

2012-03-23 Thread Mark Washenberger
Johannes Erdfelt johan...@erdfelt.com said: MySQL isn't exactly slow and Nova doesn't have particularly large tables. It looks like the slowness is coming from the network and how many queries are being made. Avoiding joins would mean even more queries, which looks like it would slow

Re: [Openstack] Caching strategies in Nova ...

2012-03-23 Thread Mark Washenberger
to a vm on a different hypervisor. Not sure why that is the case! In any case it is trivial. . ~3 ms for first ping, ~0.3 ms for subsequent pings. Sandy Walsh sandy.wa...@rackspace.com said: Was the db on a separate server or loopback? On 03/23/2012 05:26 PM, Mark Washenberger wrote: Johannes

Re: [Openstack] Caching strategies in Nova ...

2012-03-23 Thread Mark Washenberger
, how many VMs do you have, are you creating/destroying/migrating VMs, volumes, networks? Thanks, Yun On Fri, Mar 23, 2012 at 4:26 PM, Mark Washenberger mark.washenber...@rackspace.com wrote: Johannes Erdfelt johan...@erdfelt.com said: MySQL isn't exactly slow and Nova doesn't have

Re: [Openstack] Caching strategies in Nova ...

2012-03-22 Thread Mark Washenberger
What problems are caching strategies supposed to solve? On the nova compute side, it seems like streamlining db access and api-view tables would solve any performance problems caching would address, while keeping the stale data management problem small. Sandy Walsh sandy.wa...@rackspace.com

Re: [Openstack] Caching strategies in Nova ...

2012-03-22 Thread Mark Washenberger
to reinvent the wheel or hit the db at all. In addition to looking into caching technologies/approaches we're gluing together some tools for finding those bottlenecks. Our first step will be finding them, then squashing them ... however. -S On 03/22/2012 06:25 PM, Mark Washenberger wrote: What

Re: [Openstack] Caching strategies in Nova ...

2012-03-22 Thread Mark Washenberger
. Our first step will be finding them, then squashing them ... however. -S On 03/22/2012 06:25 PM, Mark Washenberger wrote: What problems are caching strategies supposed to solve? On the nova compute side, it seems like streamlining db access and api-view tables would solve any performance

Re: [Openstack] Quota classes

2012-03-19 Thread Mark Washenberger
Out of curiosity, why prefer keystone for centrally managing quota groups rather than an admin api in nova? From my perspective, a nova admin api would save a data migration and preserve nova-manage backwards compatibility. Also, since quota clearly isn't an auth-n thing, is keystone way more

Re: [Openstack] cfg usage - option registration, global objects

2012-03-09 Thread Mark Washenberger
Really not trying to derail, but Mark McLoughlin mar...@redhat.com said: [..] Also, these global objects force us to do a bunch of hacks in unit tests. We need to do tricks to ensure the object is initialized as we want. We also need to save and restore its state between

Re: [Openstack] eventlet weirdness

2012-03-04 Thread Mark Washenberger
While we are on the topic of api performance and the database, I have a few thoughts I'd like to share. TL;DR: - we should consider refactoring our wsgi server to leverage multiple processors - we could leverage compute-cell database responsibility separataion to speedup our api database

Re: [Openstack] eventlet weirdness

2012-03-01 Thread Mark Washenberger
Someone might have already said this (sure wish the listserv sent me mail faster), but we tried out PyMysql and it was exceptionally slow, even under almost no load. I have a branch in my github that I was using to test out unblocking the database access. For my cases I found that it was

Re: [Openstack] Remove Zones code - FFE

2012-02-19 Thread Mark Washenberger
Remember that for many deployments, the entire system will be a single zone, so whatever term is used should make sense in a singular sense. That rules out names such as 'slice' or 'fragment'. I think this is a slightly outdated concept of zones. The key to scalability in nova is to

Re: [Openstack] Remove Zones code - FFE

2012-02-15 Thread Mark Washenberger
Gabe Westmaas gabe.westm...@rackspace.com said: I think both these approaches are valid, and speaks to the fact that there isn't really a relationship between the two concepts. Absolutely. An availability zone is about partitioning user instance infrastructure and exposing that partitioning

Re: [Openstack] nova and trusted computing

2012-01-12 Thread Mark Washenberger
, 2012 12:33 PM To: Mark Washenberger Cc: Openstack Mailing List Subject: Re: [Openstack] nova and trusted computing Hey Mark, I agree with the comments you have made on the merge prop so far, and I'm glad you've been working with the authors to find something more amenable. I'm all

Re: [Openstack] openstack-common

2012-01-04 Thread Mark Washenberger
Openstack-common could be great. There are lots of use cases that make a lot of sense to put in openstack common. Configuration loading, context, some aspects of logging, wsgi middleware, some parts of utils--those seem to me like great opportunities to save time and effort, both writing and

[Openstack] nova and trusted computing

2012-01-03 Thread Mark Washenberger
: Is there a consensus among nova-core that the approach given in the blueprint needs to be changed? Or the other way around, is there a consensus approving of this approach? Thanks Mark Washenberger Rackspace Hosting Software Developer mark.washenber...@rackspace.com

Re: [Openstack] using objects returned from DB layer

2011-12-16 Thread Mark Washenberger
Johannes Erdfelt johan...@erdfelt.com said: I'm not saying you need to do it, but this is something that doesn't have an obvious design and implementation. It would be easier to understand and discuss with some real meat behind it. From what Monsyne Dragon and Jonathan LaCour have said on

Re: [Openstack] using objects returned from DB layer

2011-12-15 Thread Mark Washenberger
Johannes Erdfelt johan...@erdfelt.com said: On Thu, Dec 15, 2011, Kevin L. Mitchell kevin.mitch...@rackspace.com wrote: 2. However, I violently disagree with the idea that the DB layer must return dicts. It does not, even if you start talking about allowing use of other

Re: [Openstack] trusted computing and nova

2011-12-13 Thread Mark Washenberger
Fred, I can see the plugin-like behavior of the approach you have taken. However, there are a few components of it that could be improved in order to avoid adding extra complexity to the scheduler and to nova. IMO, the parts that add complexity are the additional integrity caching service,

Re: [Openstack] trusted computing and nova

2011-12-09 Thread Mark Washenberger
@lists.launchpad.net [mailto:openstack-bounces+fred.yang=intel@lists.launchpad.net] On Behalf Of Vishvananda Ishaya Sent: Friday, December 09, 2011 11:33 AM To: Michael Pittaro Cc: OpenStack Mailing List; Mark Washenberger Subject: Re: [Openstack] trusted computing and nova I suggested a couple

[Openstack] trusted computing and nova

2011-12-08 Thread Mark Washenberger
expected to support). Context: https://blueprints.launchpad.net/nova/+spec/trusted-computing-pools http://wiki.openstack.org/TrustedComputingPools https://review.openstack.org/1899 Mark Washenberger Rackspace Hosting Software Developer mark.washenber...@rackspace.com

Re: [Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Mark Washenberger
Can you talk a little more about how you want to apply this failure notification? That is, what is the case where you are going to use the information that an operation failed? In my head I have an idea of getting code simplicity dividends from an everything succeeds approach to some of our

Re: [Openstack] [Orchestration] Handling error events ... explicit vs. implicit

2011-12-07 Thread Mark Washenberger
From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Mark Washenberger [mark.washenber...@rackspace.com] Sent: Wednesday, December 07, 2011 11:53 AM To: openstack@lists.launchpad.net

[Openstack] virt driver interface and exceptions

2011-12-04 Thread Mark Washenberger
, could you talk about it a bit? I just want to understand the reasoning behind this choice. Thanks! Mark Washenberger Rackspace Hosting Software Developer mark.washenber...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post

Re: [Openstack] [nova-testing] Efforts for Essex

2011-11-22 Thread Mark Washenberger
I'm tending to agree with Sandy's comments. I think we all agree that we have a mess with the database stubbing that is going on. And I'm confident that the db fake would make that mess more manageable. But the way I see the mess, it comes from having a giant flat db interface and really

Re: [Openstack] Injecting user data into instances

2011-06-08 Thread Mark Washenberger
I don't know much about Cloudpipe and VPN, so I hope I don't hijack the thread. However, regarding inject_file Another interesting situation is with inject_file compute APIs … on API level there is no even file/contents fields, only def inject_file(self, context, instance_id): but

Re: [Openstack] Standardizing resource IDs on UUID?

2011-05-27 Thread Mark Washenberger
+1 for UUIDs. If we agree on this approach, there is some difficulty incorporating it into nova as Ed has identified. However, any other projects, especially those hoping to be adopted as Openstack projects by the PPB, can probably switch to this approach more immediately. Just a thought. Ed

Re: [Openstack] OpenStack API, Reservation ID's and Num Instances ...

2011-05-23 Thread Mark Washenberger
I'm totally on board with this as a future revision of the OS api. However it sounds like we need some sort of solution for 1.1. 1. We can't treat the InstanceID as a ReservationID since they do two different things. InstanceID's are unique per instance and ReservationID's might span N

Re: [Openstack] Floating IP in OpenStack API

2011-04-18 Thread Mark Washenberger
, there is implementation of floating ips in Nova. In implementation of approach #1 we just care about auto assigning/deassigning. As I know floating ip implemented like NAT from network nodes. 2011/4/17 Mark Washenberger [mailto:mark.washenber...@rackspace.com] mark.washenber...@rackspace.com

Re: [Openstack] Floating IP in OpenStack API

2011-04-17 Thread Mark Washenberger
Eldar, I'm having some trouble finding the diff for your implementation of approach #1. Any chance you can share it on the list? Thanks Erik Carlin erik.car...@rackspace.com said: Cool. Got it. Floating IPs or what Amazon calls Elastic IPs. How are you solving the cross L2 problem?

Re: [Openstack] distributed and heterogeneous schedulers

2011-04-14 Thread Mark Washenberger
1) Continue to add fields to the instances table (or compute_nodes table) for these main attributes like cpu_arch, etc. 2) Use the custom key/value table (instance_metadata) to store these attribute names and their values 3) Do both 1) and 2) I've no particular preference here, but if we

Re: [Openstack] Instance IDs and Multiple Zones

2011-03-22 Thread Mark Washenberger
However, if we don't have documentation of the decision, then I vote that it never happened, and instance ids are strings. We've always been at war with Eastasia, and all ids have always been strings. This approach might help us in fixing some of the nastier bits of the openstack api images

Re: [Openstack] server affinity

2011-03-02 Thread Mark Washenberger
[W]e shouldn't be overloading that functionality by performing some action based on user-defined metadata. That is exactly what I've been trying to say, but you have stated it much more succinctly. Thanks! My specific concern is with quotas. If the current osapi metadata is overloaded with

Re: [Openstack] OS API server password generation

2011-03-02 Thread Mark Washenberger
setting passwords into their boxes (especially after reading this thread). Is bypassing password generation in scope, or should I open a new bug? On Wed, Mar 2, 2011 at 5:57 PM, Mark Washenberger mark.washenber...@rackspace.com wrote: Each time I call random.seed() on my box, it grabs

Re: [Openstack] How to deal with 'tangential' bugs?

2011-02-28 Thread Mark Washenberger
I think it is commendable to identify bugs even if you can't fix them at the time. I hope that we don't create incentives to ignore bugs you find during development just to get your own merge through. But I'm worried about staleness and usefulness with known bugs. If the known bugs test cases

Re: [Openstack] server affinity

2011-02-28 Thread Mark Washenberger
This is great stuff. It sounds like there is a real distinction to be made between the data central to the apis and the user-defined properties. Also, as time and compatibility allow, we should probably change what we were calling metadata to be called properties or somesuch. Jay Pipes

Re: [Openstack] Should the OpenStack API re-use the EC2 credentials?

2011-02-24 Thread Mark Washenberger
We need an unstable trunk: I could not possibly disagree more. Trunk is about releasability and stability. As developers we need a stable well-protected trunk so that we can actually work successfully in parallel on our own branches. My ideal for trunk is that when it comes time for tagging a

Re: [Openstack] Should the OpenStack API re-use the EC2 credentials?

2011-02-24 Thread Mark Washenberger
This is what we're working on, and what Justin is proposing, Mark. Basically, in Drizzle-land, people propose a merge into trunk, Hudson picks up that proposal, pulls the brnach into lp:drizzle/staging, builds Drizzle on all supported platforms (12 OS/distro combos), then runs all automated