Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-17 Thread Soren Hansen
2011/2/16 Eric Day : > On Wed, Feb 16, 2011 at 11:20:31PM +0100, Soren Hansen wrote: >> 2011/2/16 Sandy Walsh : >> >> Hmmm... I am not sure about exposing internal structure to customers in >> >> this >> >> way.  Would you really want the more 'internal' zones exposed? >> > To Jay's point, the "co

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Sandy Walsh
___ From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Eric Day [e...@oddments.org] Sent: Wednesday, February 16, 2011 6:21 PM To: Soren Hansen Cc: openstack@lists.launchpad.net Sub

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Eric Day
On Wed, Feb 16, 2011 at 11:20:31PM +0100, Soren Hansen wrote: > 2011/2/16 Sandy Walsh : > >> Hmmm... I am not sure about exposing internal structure to customers in > >> this > >> way.  Would you really want the more 'internal' zones exposed? > > To Jay's point, the "control panel" would hide all

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Ed Leafe
On Feb 16, 2011, at 5:12 PM, Soren Hansen wrote: > * Whenever I ask something for information and I get out-of-date, > cached data back I feel like I'm back in 2003. And 2003 sucked, I > might add. The other side of that coin was the implementation of pub/sub. New/deleted/changed instanc

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Soren Hansen
2011/2/16 Eric Day : > This doesn't help the 'list all instances' search. This would be > very expensive when dealing with a large number of accounts and > instances. We need a more active caching policy, which ends up being > more of a replication subset than a cache. Did we ever discuss a gossip

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Eric Day
Good points Soren, and this is why I was suggesting we not solve this problem with a cache, but instead an eventually consistent replication stream of the aggregate data. -Eric On Wed, Feb 16, 2011 at 11:12:50PM +0100, Soren Hansen wrote: > 2011/2/16 Ed Leafe : > > This was one of the issues we d

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Soren Hansen
2011/2/16 Sandy Walsh : >> Hmmm... I am not sure about exposing internal structure to customers in this >> way.  Would you really want the more 'internal' zones exposed? > To Jay's point, the "control panel" would hide all that switching. I agree with this. It's an API, not a UI. Doing redirects o

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Soren Hansen
2011/2/16 Ed Leafe : > This was one of the issues we discussed during the sprint planning. I believe > (check with cyn) that the consensus was to use a caching strategy akin to > DNS: e.g., if zone A got a request for instance ID=12345, it would check to > see if it had id 12345 in its cache. If

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Eric Day
On Wed, Feb 16, 2011 at 04:33:06PM -0500, Jay Pipes wrote: > > While I would agree with this most of the time, there are some cases > > where "optimizing later" just doesn't work. Or, "optimizing" means > > rewriting everything you've done and replacing it with something > > that will scale seamles

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Jay Pipes
On Wed, Feb 16, 2011 at 4:25 PM, Eric Day wrote: > On Wed, Feb 16, 2011 at 03:59:10PM -0500, Jay Pipes wrote: >> But, as I mentioned to Sandy on IRC, caching and performance should be >> a secondary concern. The primary concern, right now, is just making >> this all work. In other words, getting m

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Eric Day
On Wed, Feb 16, 2011 at 03:59:10PM -0500, Jay Pipes wrote: > But, as I mentioned to Sandy on IRC, caching and performance should be > a secondary concern. The primary concern, right now, is just making > this all work. In other words, getting multiple zones up and running, > each knowing about thei

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Jay Pipes
On Wed, Feb 16, 2011 at 2:17 PM, Eric Day wrote: > On Wed, Feb 16, 2011 at 01:02:22PM -0500, Jay Pipes wrote: >> >> [Sorry, yes the instance name is passed in on the request, but the >> >> instance ID is what's needed (assuming of course instance ID is unique >> >> across zones.)] >> > >> >    

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Eric Day
On Wed, Feb 16, 2011 at 01:02:22PM -0500, Jay Pipes wrote: > >> [Sorry, yes the instance name is passed in on the request, but the > >> instance ID is what's needed (assuming of course instance ID is unique > >> across zones.)] > > > >        The ID is determined early in the process; well before

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Eric Day
Hi Sandy, On Wed, Feb 16, 2011 at 06:19:52PM +, Sandy Walsh wrote: > Hmm. You wouldn't really need to re-marshall the request. Just > copy the needed headers & url, and pass along the body as you received > it. Basically you are just > acting as a sort of http proxy.

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Sandy Walsh
Thanks Dragon! Had another conversation with Jay on this as well. I think the caching with the smart use of instance/customer naming will go a long way to helping the search. More comments below ... -S From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sa

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Monsyne Dragon
On 2/16/11 9:26 AM, Sandy Walsh wrote: Hi y'all Like they say "The devil is in the details". I'm at the stage where the parent zones will talk to the child zones and there are some interesting implementation issues: *Problem 1.* I'd like to pass the incoming HTTP Request object along to the

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Jay Pipes
On Wed, Feb 16, 2011 at 12:16 PM, Ed Leafe wrote: > On Feb 16, 2011, at 11:54 AM, Sandy Walsh wrote: >>>        Isn't the instance name usually supplied by the user/originator? >> >> [Sorry, yes the instance name is passed in on the request, but the instance >> ID is what's needed (assuming of co

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Ed Leafe
On Feb 16, 2011, at 11:54 AM, Sandy Walsh wrote: > Thanks for feedback Ed. Comments in [] below ... Yeesh - that makes for ugly quoting. :) Let me try to reformat the quoting. >>Isn't the instance name usually supplied by the user/originator? > > [Sorry, yes the instance name

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Sandy Walsh
Thanks for feedback Ed. Comments in [] below ... From: Ed Leafe [e...@leafe.com] On Feb 16, 2011, at 10:26 AM, Sandy Walsh wrote: Isn't the instance name usually supplied by the user/originator? [Sorry, yes the instance name is passed in on the r

Re: [Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Ed Leafe
On Feb 16, 2011, at 10:26 AM, Sandy Walsh wrote: > But this introduces a problem. Consider this use-case: > > a. I issue a "create-instance" via the top-level API in zone-A > b. the request is relayed down to zone-C > c. the instance is created some time later >Q1. How does the user learn wh

[Openstack] Multi-Cluster/Zone - Devil in the Details ...

2011-02-16 Thread Sandy Walsh
Hi y'all Like they say "The devil is in the details". I'm at the stage where the parent zones will talk to the child zones and there are some interesting implementation issues: Problem 1. I'd like to pass the incoming HTTP Request object along to the Scheduler so I don't have to remarshall the