Re: [Openstack] KVM Block Migration

2011-05-11 Thread Masanori ITOH
Hi Mikhail, Vish, Kei,

I filed the blueprint below.

  https://blueprints.launchpad.net/nova/+spec/kvm-block-migration

and, please check the full specification page too.

  http://etherpad.openstack.org/kvm-block-migration


BTW, who is the appropriate person as the approver?
Also, I'm wondering which milestone target we should shoot for..
diablo-1 or diablo-2?

Regards,
Masanori

From: Vishvananda Ishaya vishvana...@gmail.com
Subject: Re: [Openstack] KVM Block Migration
Date: Wed, 11 May 2011 03:57:47 +0900

 I don't know if anyone has started work on this feature.  Someone should file 
 a blueprint.
 
 Vish
 
 On May 10, 2011, at 3:35 AM, Mikhail Shcherbakov wrote:
 
 Hi,
 Is there any progress on KVM block migration?
 
 I'd like to test it and possibly make some changes to nova to support block 
 devices,
 but don't want to reinventing the wheel.
 
 Thanks,
 
 2011/4/12 Masanori ITOH itou...@nttdata.co.jpmailto:itou...@nttdata.co.jp
 
 Hi,
 
 Vish also mentioned that we should support the KVM block migration feature
 instead of stability which I mentioned because it's very much useful.
 I agree with Vish of course. :)
 Actually, we discussed inclusion of block migration feature at San Antonio. :)
 
 
 I've also seen the page Hisaki mentioned. My point is that
 the page explains interacting qemu monitor but we need to invoke
 the feature through libvirt layer not the qemu monitor directly.
 Here, A colleague of mine mentioned that --copy-storage-all option of
 virsh looked like not-supported yet in Ubuntu 10.10 at least.
 
 Hisaki,
 do you have any information if it's enabled in Ubuntu 11.04 libvirt?
 I mean using virsh, did you succeed invoking block live migration
 something like the following?
 
  $ virsh migrate --live --copy-storage-all DOMID DESTURL
 
 Unfortunately, we haven't been able to make enough time trying Natty alpha
 because of our schedule delay caused by the earthquake in Northeastern Japan. 
 :(
 At least, libvirt(virsh) included in RHEL6 does not support the feature.
 
 But, anyway I will talk about the issue with Kei and Muneyuki before going to
 the Design Summit. :)
 
 Regards,
 Masanori
 
 
 From: igoigo246 igoigo...@gmail.commailto:igoigo...@gmail.com
 Subject: Re: [Openstack] KVM Block Migration
 Date: Tue, 12 Apr 2011 09:28:38 +0900
 
  Hi,
 
  I look Japanease Site.
  http://www.cuspy.org/blog/archives/917
 
  This site wrote.
 
  image file
  vda.qcow
 
  sending host
 
  qemu --enable-kvm -m 512 \
  -drive file=vda.qcow,if=virtio,boot=on \
  -net nic,macaddr=00:16:3E:00:FF:32,model=virtio
 
  receive host
 
  touch vda.qcow
  qemu -enable-kvm -m 512 \
  -drive file=vda.qcow,if=virtio \
  -net nic,macaddr=00:16:3E:00:FF:32,model=virtio \
  -incoming tcp:0:
 
  sending host
 
  migrate -d -b tcp:wasabi:
 
 
  (qemu) info migrate
  Migration status: active
  transferred ram: 48 kbytes
  remaining ram: 147792 kbytes
  total ram: 147840 kbytes
  transferred disk: 206848 kbytes
  remaining disk: 10278912 kbytes
  total disk: 10485760 kbytes
 
  Thanks,
 
  --
  Hisashi Ikari
 
 
 
 
 
  2011-04-11 (月) の 16:50 +0900 に Masanori ITOH さんは書きました:
   Hi,
  
   We are considering if it's possible to support KVM block migration
   as the next step of live migration.
  
   Actually, our main issue at this moment is if that kvm feature is enough
   stable or not because we got several errors during our try of it
   using Ubuntu 10.10 code base. Especially, I'm not sure if the feature
   is enabled or not in the qemu-kvm bundled in Ubuntu/RHEL.
  
   Do you have any information about stability?
  
   Thanks,
   Masanori
  
   ---
   Masanori ITOH  RD Headquarters, NTT DATA CORPORATION
  e-mail: itou...@nttdata.co.jpmailto:itou...@nttdata.co.jp
  
   From: igoigo246 igoigo...@gmail.commailto:igoigo...@gmail.com
   Subject: [Openstack] (no subject)
   Date: Mon, 11 Apr 2011 14:41:20 +0900
  
hi all
   
KVM Block Migration is wonderful function.
   
http://www.linux-kvm.com/content/qemu-kvm-012-adds-block-migration-feature
   
this allow   that live migration do  without shared storage.
   
   
When KVM Block migration Support ?
   
   
Thanks for reading.
--
Hisashi Ikari
   
   
___
Mailing list: https://launchpad.net/~openstack
Post to : 
openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp
  
   ___
   Mailing list: https://launchpad.net/~openstack
   Post to : 
   openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net
   Unsubscribe : https://launchpad.net/~openstack
   More help   : https://help.launchpad.net/ListHelp
 
 
 
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : 
 

Re: [Openstack] PREROUTING 169.254.169.254 rule shoud not on Compute node.......

2011-05-11 Thread Narayan Desai
We're using quagga for route exchange (via ospf) between the network
servers, and the api server. The api server peers with each network
server. This is about as simple a quagga setup as you can have.
Basically, when nova-network starts up a new project network, quagga
begins to advertise that network into ospf. The api server sees that
advertisement and can use that to reach the node (for metadata
requests).

Incidentally, you might want to lock down your POSTROUTING rule to
ensure that vms can't hit your management network. IIRC, we were able
to use a similar rule, except with a specific host and port as opposed
to a blanket rule for the whole network.
 -nld

2011/5/10 郭耀謙 tonyt...@gmail.com:
 #iptables -t nat -A  nova-network-POSTROUTING -s 10.0.0.0/12 -d
 192.168.1.0/24 -j ACCEPT
 That's what I did on nova-network host .
 btw , I always isolate nova-network.
 I'm interesting about quagga in your environment.

 2011/5/11 Narayan Desai narayan.de...@gmail.com

 For what it's worth, we're running in a configuration similar to the
 one in the attached diagram using VlanManager. When we moved the
 nova-network service off of the machine with nova-api, we needed to
 add an additional prerouting rule on the network server that prevented
 the traffic from being sent out via NAT (which caused the source
 address to be lost, resulting in a metadata resolution error). Once
 the packets arrive at the api service with the correct source address,
 they need a route back, via the nova-network server in order to get
 the response packets onto the correct vlan. With a single nova-network
 server, a static route will do. With multiple nova-network instances
 on different systems, things get a little more complicated. We ended
 up setting up route distribution via quagga between the nova-api
 server, and the nova-network servers. This ensures that nova-api knows
 which nova-network instance to use to reach any particular project
 network.
  -nld

 On Tue, May 10, 2011 at 9:08 PM, 郭耀謙 tonyt...@gmail.com wrote:
  Hello , guys
  There's a problem while separate instance's network and nova-management
  network.
  EX.
  Nova management network : 192.168.1.0/24  eth0
  Instance network   :  10.0.0.0/12  eth1 bridge to br100
  During cloud-setup :
  Instance try to retrieve metadata from 169.254.169.254.
  Instances(10.0.0.0/12) request 169.254.169.254 PREROUTING from
  gateway(nova-network).
  But If PREROUTING rule is already been set on nova-Compute node,
  instance
  request will be redirected on VM host instead of nova-network host.
  So If your topology is like A diadram from StackOps , Plz Check iptables
  rule on Compute nodes.
  -A PREROUTING -d 169.254.169.254/32 -p tcp -m tcp --dport 80 -j DNAT
  --to-destination 192.168.1.2:8773
  And del this rule , your instance will get metadata correctly
 
 
 
 
  ___
  Mailing list: https://launchpad.net/~openstack
  Post to : openstack@lists.launchpad.net
  Unsubscribe : https://launchpad.net/~openstack
  More help   : https://help.launchpad.net/ListHelp
 
 


 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Zones / distributed scheduler question

2011-05-11 Thread Sandy Walsh
Hi Lorin,

Zones can have multiple parents. Child Zones don't know their parents ... 
parents only know about children.

Sharing services across Zones isn't permitted (since they would need to share a 
DB and AMQP).

You could solve the problem by using the Capabilities to determine where 
instances are created (GPU only or not) or create smaller sub-zones for 
GPU/not-GPU and let the parent zones select between them.

-S



From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net 
[openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of 
Lorin Hochstein [lo...@isi.edu]
Sent: Wednesday, May 11, 2011 12:06 PM
To: Openstack
Subject: [Openstack] Zones / distributed scheduler question

All:

For the proposed zones / distributed scheduler functionality 
https://blueprints.launchpad.net/nova/+spec/zones101, can we defines  zones 
that overlap without being nested?

Consider the scenario where:

* We have two datacenters: one in Arlington, Virginia (ARL), and one in Marina 
del Rey, California (MDR).
* At each datacenter, we have both conventional x86 nodes and  nodes with GPUs

Can we define our zones such that:

* We have a zone for each datacenter (ARL, MDR)
* We have a zone for GPU nodes  (GPU)

In this case, the GPU zone would both overlap the ARL and MDR zones, but ARL 
and MDR would not be fully nested within GPU.


Lorin
--
Lorin Hochstein, Computer Scientist
USC Information Sciences Institute
703.812.3710
http://www.east.isi.edu/~lorin



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at ab...@rackspace.com, and delete the original message.
Your cooperation is appreciated.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] KVM Block Migration

2011-05-11 Thread Vishvananda Ishaya
I assigned it to milestone 2 since there are already a lot of changes we are 
trying to get in in milestone 1

Vish

On May 10, 2011, at 11:37 PM, Masanori ITOH wrote:

 Hi Mikhail, Vish, Kei,
 
 I filed the blueprint below.
 
  https://blueprints.launchpad.net/nova/+spec/kvm-block-migration
 
 and, please check the full specification page too.
 
  http://etherpad.openstack.org/kvm-block-migration
 
 
 BTW, who is the appropriate person as the approver?
 Also, I'm wondering which milestone target we should shoot for..
 diablo-1 or diablo-2?
 
 Regards,
 Masanori
 
 From: Vishvananda Ishaya vishvana...@gmail.com
 Subject: Re: [Openstack] KVM Block Migration
 Date: Wed, 11 May 2011 03:57:47 +0900
 
 I don't know if anyone has started work on this feature.  Someone should 
 file a blueprint.
 
 Vish
 
 On May 10, 2011, at 3:35 AM, Mikhail Shcherbakov wrote:
 
 Hi,
 Is there any progress on KVM block migration?
 
 I'd like to test it and possibly make some changes to nova to support block 
 devices,
 but don't want to reinventing the wheel.
 
 Thanks,
 
 2011/4/12 Masanori ITOH itou...@nttdata.co.jpmailto:itou...@nttdata.co.jp
 
 Hi,
 
 Vish also mentioned that we should support the KVM block migration feature
 instead of stability which I mentioned because it's very much useful.
 I agree with Vish of course. :)
 Actually, we discussed inclusion of block migration feature at San Antonio. 
 :)
 
 
 I've also seen the page Hisaki mentioned. My point is that
 the page explains interacting qemu monitor but we need to invoke
 the feature through libvirt layer not the qemu monitor directly.
 Here, A colleague of mine mentioned that --copy-storage-all option of
 virsh looked like not-supported yet in Ubuntu 10.10 at least.
 
 Hisaki,
 do you have any information if it's enabled in Ubuntu 11.04 libvirt?
 I mean using virsh, did you succeed invoking block live migration
 something like the following?
 
 $ virsh migrate --live --copy-storage-all DOMID DESTURL
 
 Unfortunately, we haven't been able to make enough time trying Natty alpha
 because of our schedule delay caused by the earthquake in Northeastern 
 Japan. :(
 At least, libvirt(virsh) included in RHEL6 does not support the feature.
 
 But, anyway I will talk about the issue with Kei and Muneyuki before going to
 the Design Summit. :)
 
 Regards,
 Masanori
 
 
 From: igoigo246 igoigo...@gmail.commailto:igoigo...@gmail.com
 Subject: Re: [Openstack] KVM Block Migration
 Date: Tue, 12 Apr 2011 09:28:38 +0900
 
 Hi,
 
 I look Japanease Site.
 http://www.cuspy.org/blog/archives/917
 
 This site wrote.
 
 image file
 vda.qcow
 
 sending host
 
 qemu --enable-kvm -m 512 \
 -drive file=vda.qcow,if=virtio,boot=on \
 -net nic,macaddr=00:16:3E:00:FF:32,model=virtio
 
 receive host
 
 touch vda.qcow
 qemu -enable-kvm -m 512 \
 -drive file=vda.qcow,if=virtio \
 -net nic,macaddr=00:16:3E:00:FF:32,model=virtio \
 -incoming tcp:0:
 
 sending host
 
 migrate -d -b tcp:wasabi:
 
 
 (qemu) info migrate
 Migration status: active
 transferred ram: 48 kbytes
 remaining ram: 147792 kbytes
 total ram: 147840 kbytes
 transferred disk: 206848 kbytes
 remaining disk: 10278912 kbytes
 total disk: 10485760 kbytes
 
 Thanks,
 
 --
 Hisashi Ikari
 
 
 
 
 
 2011-04-11 (月) の 16:50 +0900 に Masanori ITOH さんは書きました:
 Hi,
 
 We are considering if it's possible to support KVM block migration
 as the next step of live migration.
 
 Actually, our main issue at this moment is if that kvm feature is enough
 stable or not because we got several errors during our try of it
 using Ubuntu 10.10 code base. Especially, I'm not sure if the feature
 is enabled or not in the qemu-kvm bundled in Ubuntu/RHEL.
 
 Do you have any information about stability?
 
 Thanks,
 Masanori
 
 ---
 Masanori ITOH  RD Headquarters, NTT DATA CORPORATION
   e-mail: itou...@nttdata.co.jpmailto:itou...@nttdata.co.jp
 
 From: igoigo246 igoigo...@gmail.commailto:igoigo...@gmail.com
 Subject: [Openstack] (no subject)
 Date: Mon, 11 Apr 2011 14:41:20 +0900
 
 hi all
 
 KVM Block Migration is wonderful function.
 
 http://www.linux-kvm.com/content/qemu-kvm-012-adds-block-migration-feature
 
 this allow   that live migration do  without shared storage.
 
 
 When KVM Block migration Support ?
 
 
 Thanks for reading.
 --
 Hisashi Ikari
 
 
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : 
 openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp
 
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : 
 openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp
 
 
 
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : 
 

Re: [Openstack] Notifications proposal

2011-05-11 Thread Monsyne Dragon

On May 11, 2011, at 10:47 AM, Matt Dietz wrote:

Hey Seshu,

1) Yes, that will be contained within the publisher_id field of the message body
2) We should be able to get customer related data from the message where it 
makes sense. It would be contained within the payload dictionary. Given that 
not all messages are going to be related to customers, we shouldn't explicitly 
force this in the standard message attributes
3) The mandatory fields on the nova side of things are: message_id, 
publisher_id, timestamp, priority, event_type, and payload. Payload is a 
dictionary containing the data that Nova is sending a notification for in the 
first place. On the Rackspace side of things, outside of System Usages, no 
message headers have been defined to my knowledge. We're simply designing the 
transport mechanism. Dragon can weigh in more on what usages will look like.

Ya, the usages are pretty simple notifications. They all have the account 
ID/user on them.
By IP address, do you mean source IP for the api request?  That we do not have 
(the usages are often generated by the worker processes long after the api call 
is over)
If you mean IP of the instance, that can be looked up from the instance id.

For instance usages (create/delete/resize, etc) they have the instance ID, and 
type (flavor).
For things like adding/removing IP's they have  the IP and the instance id.

For bandwidth usages, they have the bandwidth used, the period, and the 
instance.


Seshu Vavilikolanu 
seshu.vavilikol...@rackspace.commailto:seshu.vavilikol...@rackspace.com 
wrote:

I am part of cloud integration team in RackSpace, Austin.  I am working on 
Analytics project.
I went through the blue-prints of notifications and related topics.I have 
few questions, I am still trying to learn about openstack, so please let me 
know if any of these are already documented and available somewhere.

  *   Can we know what service generated this particular notification?  e.g.:  
Auth, etc…
 *   Can we get the service name from publisher id or some other field?
  *   Can we get the IP address/customer/account ID from the message?I am 
interested to know how this is handled when we are using load balancer.
  *   Are there any mandatory fields in message?   I guess these notifications 
are generated by Nova using Atom events, right?

--
--Monsyne M. Dragon



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at ab...@rackspace.com, and delete the original message.
Your cooperation is appreciated.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Zones / distributed scheduler question

2011-05-11 Thread Lorin Hochstein
Hi Sandy:

Do you know if there are there any nova branches floating out there that 
implement Capabilities? I've seen the following related wiki pages and 
blueprints, but I haven't found any implementations out there yet.

https://blueprints.launchpad.net/nova/+spec/extra-data
https://blueprints.launchpad.net/nova/+spec/use-metadata-tags-for-capabilities
http://wiki.openstack.org/RequestCapabilities

The zone scheduler slides mention capabilities as well, but those are at the 
zone level, and we're potentially interested in compute nodes with different 
capabilities within the same zone. 

(Note that for our use case, we'll probably be using flavors to specify 
different capabilities, so we don't (yet) need the users to be able to send 
arbitrary strings using the API). 

Lorin
--
Lorin Hochstein, Computer Scientist
USC Information Sciences Institute
703.812.3710
http://www.east.isi.edu/~lorin

On May 11, 2011, at 11:54 AM, Sandy Walsh wrote:

 Hi Lorin,
 
 Zones can have multiple parents. Child Zones don't know their parents ... 
 parents only know about children.
 
 Sharing services across Zones isn't permitted (since they would need to share 
 a DB and AMQP). 
 
 You could solve the problem by using the Capabilities to determine where 
 instances are created (GPU only or not) or create smaller sub-zones for 
 GPU/not-GPU and let the parent zones select between them. 
 
 -S
 
 
 From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net 
 [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf 
 of Lorin Hochstein [lo...@isi.edu]
 Sent: Wednesday, May 11, 2011 12:06 PM
 To: Openstack
 Subject: [Openstack] Zones / distributed scheduler question
 
 All:
 
 For the proposed zones / distributed scheduler functionality 
 https://blueprints.launchpad.net/nova/+spec/zones101, can we defines  zones 
 that overlap without being nested? 
 
 Consider the scenario where: 
 
 * We have two datacenters: one in Arlington, Virginia (ARL), and one in 
 Marina del Rey, California (MDR).
 * At each datacenter, we have both conventional x86 nodes and  nodes with 
 GPUs 
 
 Can we define our zones such that:
 
 * We have a zone for each datacenter (ARL, MDR)
 * We have a zone for GPU nodes  (GPU)
 
 In this case, the GPU zone would both overlap the ARL and MDR zones, but ARL 
 and MDR would not be fully nested within GPU. 
 
 
 Lorin
 --
 Lorin Hochstein, Computer Scientist
 USC Information Sciences Institute
 703.812.3710
 http://www.east.isi.edu/~lorin
 
 Confidentiality Notice: This e-mail message (including any attached or
 embedded documents) is intended for the exclusive and confidential use of the
 individual or entity to which this message is addressed, and unless otherwise
 expressly indicated, is confidential and privileged information of Rackspace.
 Any dissemination, distribution or copying of the enclosed material is 
 prohibited.
 If you receive this transmission in error, please notify us immediately by 
 e-mail
 at ab...@rackspace.com, and delete the original message.
 Your cooperation is appreciated.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [nova-core] Proposal to add Mark Washenberger to nova-core

2011-05-11 Thread Jay Pipes
Mark's been a very good reviewer and an invaluable resource on the API
side, particularly regarding serialization. I propose he join
nova-core.

-jay

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] [nova-core] Removing myself from nova-core

2011-05-11 Thread Jay Pipes
Hey all,

I'm going to remove myself from the nova-core group. I'll still do
reviews and stuff, but I've been focusing a lot on Glance, and there
are much more knowledgeable folks already on and proposed for
nova-core.

Not sure if this is the way to go about resigning from nova-core,
but since nobody's done it so far... here goes nothing :)

-jay

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [nova-core] Proposal to add Brian Waldon to nova-core

2011-05-11 Thread Vishvananda Ishaya
+1
On May 11, 2011, at 10:06 AM, Jay Pipes wrote:

 Subject says it all. I think Brian's demonstrated that his code
 reviews are thoughtful and thorough, and he knows the OpenStack API
 controller stuff as well as anyone else I believe.
 
 Cheers,
 jay
 
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [nova-core] Proposal to add Mark Washenberger to nova-core

2011-05-11 Thread Vishvananda Ishaya
+1
On May 11, 2011, at 10:07 AM, Jay Pipes wrote:

 Mark's been a very good reviewer and an invaluable resource on the API
 side, particularly regarding serialization. I propose he join
 nova-core.
 
 -jay
 
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


[Openstack] Code Reviews

2011-05-11 Thread Vishvananda Ishaya
Hello Everyone,

We have quite a large backlog of merge proposals here:

https://code.launchpad.net/~rlane/nova/lp773690/+merge/59565

I've been attempting to go through them to find some high priority ones to 
review.  It seems like people are being pretty active in reviewing branches, 
but there are a lot old branches that haven't been touched in a while.  So 
first I have a general request that anyone who has old branches in for review:  
please update your branches or mark them Work In Progress to remove them from 
the review queue.

I'd also like to propose a change to our process that will make the ready to 
review branches easier to find. I'd like for nova-core to set branches to WIP 
if there are two significant needs fixings or or needs information.  That way 
everyone doesn't have to sort through branches that have already been reviewed 
but are waiting on updates.  We may need to use our judgement here, so if a 
large branch has a needs fixing for a minor typo or some such, you could leave 
it under needs review so it gets viewed by more people.

Here is an example where i think this policy will be useful:

You see a branch that already has a 'Needs Fixing: this needs a failing test.  
If you look at the branch and reach the same conclusion, you can mark it Needs 
Fixing: I agree, needs a test like xxx and then set the branch to Work In 
Progress.  When the author has added the test or needs to make more comments, 
he can set it back to Needs Review.

I think this will generally keep the review board a little cleaner and also 
each branch will end up with a couple of people that are queued to review once 
the changes have come in. Does this seem acceptable to everyone?  If I don't 
here any major dissents, I will add this info to the wiki and we can put it 
into practice.

Vish___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [nova-core] Proposal to add Brian Waldon to nova-core

2011-05-11 Thread Ed Leafe
On May 11, 2011, at 1:06 PM, Jay Pipes wrote:

 Subject says it all. I think Brian's demonstrated that his code
 reviews are thoughtful and thorough, and he knows the OpenStack API
 controller stuff as well as anyone else I believe.


Definitely! +1


-- Ed Leafe



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace. 
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at ab...@rackspace.com, and delete the original message. 
Your cooperation is appreciated.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [nova-core] Removing myself from nova-core

2011-05-11 Thread Eric Day
I've been thinking of doing the same to focus on Burrow since I've
not been following many of the nova discussions lately and there are
so many new folks involved in -core. I think I'll follow your lead,
but I'll still be around for reviews where I'm useful.

-Eric

On Wed, May 11, 2011 at 01:10:04PM -0400, Jay Pipes wrote:
 Hey all,
 
 I'm going to remove myself from the nova-core group. I'll still do
 reviews and stuff, but I've been focusing a lot on Glance, and there
 are much more knowledgeable folks already on and proposed for
 nova-core.
 
 Not sure if this is the way to go about resigning from nova-core,
 but since nobody's done it so far... here goes nothing :)
 
 -jay
 
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Code Reviews

2011-05-11 Thread Jay Pipes
++ on your suggestions.

On Wed, May 11, 2011 at 3:38 PM, Vishvananda Ishaya
vishvana...@gmail.com wrote:
 Hello Everyone,
 We have quite a large backlog of merge proposals here:
 https://code.launchpad.net/~rlane/nova/lp773690/+merge/59565
 I've been attempting to go through them to find some high priority ones to
 review.  It seems like people are being pretty active in reviewing branches,
 but there are a lot old branches that haven't been touched in a while.  So
 first I have a general request that anyone who has old branches in for
 review:  please update your branches or mark them Work In Progress to remove
 them from the review queue.
 I'd also like to propose a change to our process that will make the ready to
 review branches easier to find. I'd like for nova-core to set branches to
 WIP if there are two significant needs fixings or or needs information.
  That way everyone doesn't have to sort through branches that have already
 been reviewed but are waiting on updates.  We may need to use our judgement
 here, so if a large branch has a needs fixing for a minor typo or some such,
 you could leave it under needs review so it gets viewed by more people.
 Here is an example where i think this policy will be useful:
 You see a branch that already has a 'Needs Fixing: this needs a failing
 test.  If you look at the branch and reach the same conclusion, you can
 mark it Needs Fixing: I agree, needs a test like xxx and then set the
 branch to Work In Progress.  When the author has added the test or needs to
 make more comments, he can set it back to Needs Review.
 I think this will generally keep the review board a little cleaner and also
 each branch will end up with a couple of people that are queued to review
 once the changes have come in. Does this seem acceptable to everyone?  If I
 don't here any major dissents, I will add this info to the wiki and we can
 put it into practice.
 Vish
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to     : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Code Reviews

2011-05-11 Thread Chris Behrens
I'm not nova-core, but this makes a heck of a lot of sense to me, too.

On May 11, 2011, at 1:16 PM, Jay Pipes wrote:

 ++ on your suggestions.
 
 On Wed, May 11, 2011 at 3:38 PM, Vishvananda Ishaya
 vishvana...@gmail.com wrote:
 Hello Everyone,
 We have quite a large backlog of merge proposals here:
 https://code.launchpad.net/~rlane/nova/lp773690/+merge/59565
 I've been attempting to go through them to find some high priority ones to
 review.  It seems like people are being pretty active in reviewing branches,
 but there are a lot old branches that haven't been touched in a while.  So
 first I have a general request that anyone who has old branches in for
 review:  please update your branches or mark them Work In Progress to remove
 them from the review queue.
 I'd also like to propose a change to our process that will make the ready to
 review branches easier to find. I'd like for nova-core to set branches to
 WIP if there are two significant needs fixings or or needs information.
  That way everyone doesn't have to sort through branches that have already
 been reviewed but are waiting on updates.  We may need to use our judgement
 here, so if a large branch has a needs fixing for a minor typo or some such,
 you could leave it under needs review so it gets viewed by more people.
 Here is an example where i think this policy will be useful:
 You see a branch that already has a 'Needs Fixing: this needs a failing
 test.  If you look at the branch and reach the same conclusion, you can
 mark it Needs Fixing: I agree, needs a test like xxx and then set the
 branch to Work In Progress.  When the author has added the test or needs to
 make more comments, he can set it back to Needs Review.
 I think this will generally keep the review board a little cleaner and also
 each branch will end up with a couple of people that are queued to review
 once the changes have come in. Does this seem acceptable to everyone?  If I
 don't here any major dissents, I will add this info to the wiki and we can
 put it into practice.
 Vish
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp
 
 
 
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [nova-core] Proposal to add Brian Waldon to nova-core

2011-05-11 Thread Paul Voccio
+1

On 5/11/11 3:01 PM, Ed Leafe ed.le...@rackspace.com wrote:

On May 11, 2011, at 1:06 PM, Jay Pipes wrote:

 Subject says it all. I think Brian's demonstrated that his code
 reviews are thoughtful and thorough, and he knows the OpenStack API
 controller stuff as well as anyone else I believe.


Definitely! +1


-- Ed Leafe



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of
the
individual or entity to which this message is addressed, and unless
otherwise
expressly indicated, is confidential and privileged information of
Rackspace. 
Any dissemination, distribution or copying of the enclosed material is
prohibited.
If you receive this transmission in error, please notify us immediately
by e-mail
at ab...@rackspace.com, and delete the original message.
Your cooperation is appreciated.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at ab...@rackspace.com, and delete the original message.
Your cooperation is appreciated.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] [nova-core] Proposal to add Mark Washenberger to nova-core

2011-05-11 Thread Paul Voccio
+1

On 5/11/11 3:02 PM, Ed Leafe e...@leafe.com wrote:

On May 11, 2011, at 1:07 PM, Jay Pipes wrote:

 Mark's been a very good reviewer and an invaluable resource on the API
 side, particularly regarding serialization. I propose he join
 nova-core.

+1


-- Ed Leafe




___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace. 
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at ab...@rackspace.com, and delete the original message. 
Your cooperation is appreciated.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Code Reviews

2011-05-11 Thread Soren Hansen
2011/5/11 Vishvananda Ishaya vishvana...@gmail.com:
 I'd also like to propose a change to our process that will make the ready to
[...]
 once the changes have come in. Does this seem acceptable to everyone?  If I
 don't here any major dissents, I will add this info to the wiki and we can
 put it into practice.

Sounds good to me.

-- 
Soren Hansen        | http://linux2go.dk/
Ubuntu Developer    | http://www.ubuntu.com/
OpenStack Developer | http://www.openstack.org/

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Code Reviews

2011-05-11 Thread Matt Dietz
+1

On 5/11/11 3:16 PM, Jay Pipes jaypi...@gmail.com wrote:

++ on your suggestions.

On Wed, May 11, 2011 at 3:38 PM, Vishvananda Ishaya
vishvana...@gmail.com wrote:
 Hello Everyone,
 We have quite a large backlog of merge proposals here:
 https://code.launchpad.net/~rlane/nova/lp773690/+merge/59565
 I've been attempting to go through them to find some high priority ones
to
 review.  It seems like people are being pretty active in reviewing
branches,
 but there are a lot old branches that haven't been touched in a while.
So
 first I have a general request that anyone who has old branches in for
 review:  please update your branches or mark them Work In Progress to
remove
 them from the review queue.
 I'd also like to propose a change to our process that will make the
ready to
 review branches easier to find. I'd like for nova-core to set branches
to
 WIP if there are two significant needs fixings or or needs information.
  That way everyone doesn't have to sort through branches that have
already
 been reviewed but are waiting on updates.  We may need to use our
judgement
 here, so if a large branch has a needs fixing for a minor typo or some
such,
 you could leave it under needs review so it gets viewed by more people.
 Here is an example where i think this policy will be useful:
 You see a branch that already has a 'Needs Fixing: this needs a failing
 test.  If you look at the branch and reach the same conclusion, you can
 mark it Needs Fixing: I agree, needs a test like xxx and then set the
 branch to Work In Progress.  When the author has added the test or
needs to
 make more comments, he can set it back to Needs Review.
 I think this will generally keep the review board a little cleaner and
also
 each branch will end up with a couple of people that are queued to
review
 once the changes have come in. Does this seem acceptable to everyone?
If I
 don't here any major dissents, I will add this info to the wiki and we
can
 put it into practice.
 Vish
 ___
 Mailing list: https://launchpad.net/~openstack
 Post to : openstack@lists.launchpad.net
 Unsubscribe : https://launchpad.net/~openstack
 More help   : https://help.launchpad.net/ListHelp



___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp



Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at ab...@rackspace.com, and delete the original message.
Your cooperation is appreciated.


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Zones / distributed scheduler question

2011-05-11 Thread Sandy Walsh
Hey again!

dabo, sirp and myself have been working on the Distributed Scheduler branch, 
which does all this.

It's a large branch, so while Ed (dabo) continues on getting things functional, 
myself and Rick (sirp) have started pulling chunks of code from the branch and 
getting smaller merge-props in.

Expect to see the first to land, hopefully, tomorrow. This will give you a good 
idea of the capabilities functionality and the extension points. You can have a 
sneak peak at lp:~sandy-walsh/nova/dist-sched-1

Cheers,
-S


From: Lorin Hochstein [lo...@isi.edu]
Sent: Wednesday, May 11, 2011 1:47 PM
To: Sandy Walsh
Cc: Openstack
Subject: Re: [Openstack] Zones / distributed scheduler question

Hi Sandy:

Do you know if there are there any nova branches floating out there that 
implement Capabilities? I've seen the following related wiki pages and 
blueprints, but I haven't found any implementations out there yet.

https://blueprints.launchpad.net/nova/+spec/extra-data
https://blueprints.launchpad.net/nova/+spec/use-metadata-tags-for-capabilities
http://wiki.openstack.org/RequestCapabilities

The zone scheduler slides mention capabilities as well, but those are at the 
zone level, and we're potentially interested in compute nodes with different 
capabilities within the same zone.

(Note that for our use case, we'll probably be using flavors to specify 
different capabilities, so we don't (yet) need the users to be able to send 
arbitrary strings using the API).

Lorin
--
Lorin Hochstein, Computer Scientist
USC Information Sciences Institute
703.812.3710
http://www.east.isi.edu/~lorin

On May 11, 2011, at 11:54 AM, Sandy Walsh wrote:

Hi Lorin,

Zones can have multiple parents. Child Zones don't know their parents ... 
parents only know about children.

Sharing services across Zones isn't permitted (since they would need to share a 
DB and AMQP).

You could solve the problem by using the Capabilities to determine where 
instances are created (GPU only or not) or create smaller sub-zones for 
GPU/not-GPU and let the parent zones select between them.

-S



From: 
openstack-bounces+sandy.walsh=rackspace@lists.launchpad.netmailto:openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net
 [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of 
Lorin Hochstein [lo...@isi.edu]
Sent: Wednesday, May 11, 2011 12:06 PM
To: Openstack
Subject: [Openstack] Zones / distributed scheduler question

All:

For the proposed zones / distributed scheduler functionality 
https://blueprints.launchpad.net/nova/+spec/zones101, can we defines  zones 
that overlap without being nested?

Consider the scenario where:

* We have two datacenters: one in Arlington, Virginia (ARL), and one in Marina 
del Rey, California (MDR).
* At each datacenter, we have both conventional x86 nodes and  nodes with GPUs

Can we define our zones such that:

* We have a zone for each datacenter (ARL, MDR)
* We have a zone for GPU nodes  (GPU)

In this case, the GPU zone would both overlap the ARL and MDR zones, but ARL 
and MDR would not be fully nested within GPU.


Lorin
--
Lorin Hochstein, Computer Scientist
USC Information Sciences Institute
703.812.3710
http://www.east.isi.edu/~lorin


Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace.
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at ab...@rackspace.commailto:ab...@rackspace.com, and delete the original 
message.
Your cooperation is appreciated.




Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for the exclusive and confidential use of the
individual or entity to which this message is addressed, and unless otherwise
expressly indicated, is confidential and privileged information of Rackspace. 
Any dissemination, distribution or copying of the enclosed material is 
prohibited.
If you receive this transmission in error, please notify us immediately by 
e-mail
at ab...@rackspace.com, and delete the original message. 
Your cooperation is appreciated.

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Code Reviews

2011-05-11 Thread Dan Prince
+1

I just SmokeStack'ed a couple more suspiciously old branches and marked them 
needs fixing because they didn't merge w/ trunk. Most of which already had 
needs fixing for other reasons anyway.

---

Also, I've been sitting on lp:~dan-prince/nova/chown_vdb_device for a month 
now. I'm happy to mark that as WIP to keep things clean. I am however okay with 
the merge prop as is (for the reasons in I listed in the merge prop).


Dan

-Original Message-
From: Vishvananda Ishaya vishvana...@gmail.com
Sent: Wednesday, May 11, 2011 3:38pm
To: openstack@lists.launchpad.net
Subject: [Openstack] Code Reviews

___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp
Hello Everyone,

We have quite a large backlog of merge proposals here:

https://code.launchpad.net/~rlane/nova/lp773690/+merge/59565

I've been attempting to go through them to find some high priority ones to 
review.  It seems like people are being pretty active in reviewing branches, 
but there are a lot old branches that haven't been touched in a while.  So 
first I have a general request that anyone who has old branches in for review:  
please update your branches or mark them Work In Progress to remove them from 
the review queue.

I'd also like to propose a change to our process that will make the ready to 
review branches easier to find. I'd like for nova-core to set branches to WIP 
if there are two significant needs fixings or or needs information.  That way 
everyone doesn't have to sort through branches that have already been reviewed 
but are waiting on updates.  We may need to use our judgement here, so if a 
large branch has a needs fixing for a minor typo or some such, you could leave 
it under needs review so it gets viewed by more people.

Here is an example where i think this policy will be useful:

You see a branch that already has a 'Needs Fixing: this needs a failing test.  
If you look at the branch and reach the same conclusion, you can mark it Needs 
Fixing: I agree, needs a test like xxx and then set the branch to Work In 
Progress.  When the author has added the test or needs to make more comments, 
he can set it back to Needs Review.

I think this will generally keep the review board a little cleaner and also 
each branch will end up with a couple of people that are queued to review once 
the changes have come in. Does this seem acceptable to everyone?  If I don't 
here any major dissents, I will add this info to the wiki and we can put it 
into practice.

Vish


___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help   : https://help.launchpad.net/ListHelp


Re: [Openstack] Zones / distributed scheduler question

2011-05-11 Thread Lorin Hochstein
Sandy:

I took a quick look at see how capabilities are represented and how the 
information flows from the ComputeManager to the ZoneManager, it looks great. I 
see you're using the message queue (RPC call) instead of storing the capability 
info in the database. Anxiously awaiting for this code to make it to the trunk 

I also see that there's a complete-looking implementation of the 
XenAPIConnection.get_host_stats method, but LibvirtConnection.get_host_stats is 
a no-op. Do you plan on implementing similar functionality for KVM (e.g., 
reporting memory and disk free), or will that be up to other people to get that 
code in there? 

Take care,

Lorin
--
Lorin Hochstein, Computer Scientist
USC Information Sciences Institute
703.812.3710
http://www.east.isi.edu/~lorin

On May 11, 2011, at 8:12 PM, Sandy Walsh wrote:

 Hey again!
 
 dabo, sirp and myself have been working on the Distributed Scheduler branch, 
 which does all this. 
 
 It's a large branch, so while Ed (dabo) continues on getting things 
 functional, myself and Rick (sirp) have started pulling chunks of code from 
 the branch and getting smaller merge-props in. 
 
 Expect to see the first to land, hopefully, tomorrow. This will give you a 
 good idea of the capabilities functionality and the extension points. You can 
 have a sneak peak at lp:~sandy-walsh/nova/dist-sched-1
 
 Cheers,
 -S
 
 From: Lorin Hochstein [lo...@isi.edu]
 Sent: Wednesday, May 11, 2011 1:47 PM
 To: Sandy Walsh
 Cc: Openstack
 Subject: Re: [Openstack] Zones / distributed scheduler question
 
 Hi Sandy:
 
 Do you know if there are there any nova branches floating out there that 
 implement Capabilities? I've seen the following related wiki pages and 
 blueprints, but I haven't found any implementations out there yet.
 
 https://blueprints.launchpad.net/nova/+spec/extra-data
 https://blueprints.launchpad.net/nova/+spec/use-metadata-tags-for-capabilities
 http://wiki.openstack.org/RequestCapabilities
 
 The zone scheduler slides mention capabilities as well, but those are at the 
 zone level, and we're potentially interested in compute nodes with 
 different capabilities within the same zone. 
 
 (Note that for our use case, we'll probably be using flavors to specify 
 different capabilities, so we don't (yet) need the users to be able to send 
 arbitrary strings using the API). 
 
 Lorin
 --
 Lorin Hochstein, Computer Scientist
 USC Information Sciences Institute
 703.812.3710
 http://www.east.isi.edu/~lorin
 
 On May 11, 2011, at 11:54 AM, Sandy Walsh wrote:
 
 Hi Lorin,
 
 Zones can have multiple parents. Child Zones don't know their parents ... 
 parents only know about children.
 
 Sharing services across Zones isn't permitted (since they would need to 
 share a DB and AMQP). 
 
 You could solve the problem by using the Capabilities to determine where 
 instances are created (GPU only or not) or create smaller sub-zones for 
 GPU/not-GPU and let the parent zones select between them. 
 
 -S
 
 
 From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net 
 [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf 
 of Lorin Hochstein [lo...@isi.edu]
 Sent: Wednesday, May 11, 2011 12:06 PM
 To: Openstack
 Subject: [Openstack] Zones / distributed scheduler question
 
 All:
 
 For the proposed zones / distributed scheduler functionality 
 https://blueprints.launchpad.net/nova/+spec/zones101, can we defines  
 zones that overlap without being nested? 
 
 Consider the scenario where: 
 
 * We have two datacenters: one in Arlington, Virginia (ARL), and one in 
 Marina del Rey, California (MDR).
 * At each datacenter, we have both conventional x86 nodes and  nodes with 
 GPUs 
 
 Can we define our zones such that:
 
 * We have a zone for each datacenter (ARL, MDR)
 * We have a zone for GPU nodes  (GPU)
 
 In this case, the GPU zone would both overlap the ARL and MDR zones, but ARL 
 and MDR would not be fully nested within GPU. 
 
 
 Lorin
 --
 Lorin Hochstein, Computer Scientist
 USC Information Sciences Institute
 703.812.3710
 http://www.east.isi.edu/~lorin
 
 Confidentiality Notice: This e-mail message (including any attached or
 embedded documents) is intended for the exclusive and confidential use of the
 individual or entity to which this message is addressed, and unless otherwise
 expressly indicated, is confidential and privileged information of Rackspace.
 Any dissemination, distribution or copying of the enclosed material is 
 prohibited.
 If you receive this transmission in error, please notify us immediately by 
 e-mail
 at ab...@rackspace.com, and delete the original message.
 Your cooperation is appreciated.
 
 Confidentiality Notice: This e-mail message (including any attached or
 embedded documents) is intended for the exclusive and confidential use of the
 individual or entity to which this message is addressed, and unless otherwise
 expressly indicated, is confidential and privileged information of Rackspace. 
 Any 

Re: [Openstack] Zones / distributed scheduler question

2011-05-11 Thread Sandy Walsh
Cool! Glad to hear we're on the right path. Yes, we're using rabbit to send the 
metrics to the schedulers since they're largely ephemeral.

We'll be using the existing scheduler.host_filter:HostFilter code to fill out 
the filter_hosts() method. Other groups will be making more drivers for that 
component as well. The overall plan is in our Zones101 summit presentation 
available here: http://dl.dropbox.com/u/166877/Zones-101.pdf

We're not working on KVM ourselves ... hoping for the community to jump in 
there.

Hope it helps,
-S


From: Lorin Hochstein [lo...@isi.edu]
Sent: Thursday, May 12, 2011 12:26 AM
To: Sandy Walsh
Cc: Openstack; do...@pandora.east.isi.edu
Subject: Re: [Openstack] Zones / distributed scheduler question

Sandy:

I took a quick look at see how capabilities are represented and how the 
information flows from the ComputeManager to the ZoneManager, it looks great. I 
see you're using the message queue (RPC call) instead of storing the capability 
info in the database. Anxiously awaiting for this code to make it to the trunk

I also see that there's a complete-looking implementation of the 
XenAPIConnection.get_host_stats method, but LibvirtConnection.get_host_stats is 
a no-op. Do you plan on implementing similar functionality for KVM (e.g., 
reporting memory and disk free), or will that be up to other people to get that 
code in there?

Take care,

Lorin
--
Lorin Hochstein, Computer Scientist
USC Information Sciences Institute
703.812.3710
http://www.east.isi.edu/~lorin

On May 11, 2011, at 8:12 PM, Sandy Walsh wrote:

Hey again!

dabo, sirp and myself have been working on the Distributed Scheduler branch, 
which does all this.

It's a large branch, so while Ed (dabo) continues on getting things functional, 
myself and Rick (sirp) have started pulling chunks of code from the branch and 
getting smaller merge-props in.

Expect to see the first to land, hopefully, tomorrow. This will give you a good 
idea of the capabilities functionality and the extension points. You can have a 
sneak peak at lp:~sandy-walsh/nova/dist-sched-1

Cheers,
-S


From: Lorin Hochstein [lo...@isi.edu]
Sent: Wednesday, May 11, 2011 1:47 PM
To: Sandy Walsh
Cc: Openstack
Subject: Re: [Openstack] Zones / distributed scheduler question

Hi Sandy:

Do you know if there are there any nova branches floating out there that 
implement Capabilities? I've seen the following related wiki pages and 
blueprints, but I haven't found any implementations out there yet.

https://blueprints.launchpad.net/nova/+spec/extra-data
https://blueprints.launchpad.net/nova/+spec/use-metadata-tags-for-capabilities
http://wiki.openstack.org/RequestCapabilities

The zone scheduler slides mention capabilities as well, but those are at the 
zone level, and we're potentially interested in compute nodes with different 
capabilities within the same zone.

(Note that for our use case, we'll probably be using flavors to specify 
different capabilities, so we don't (yet) need the users to be able to send 
arbitrary strings using the API).

Lorin
--
Lorin Hochstein, Computer Scientist
USC Information Sciences Institute
703.812.3710
http://www.east.isi.edu/~lorin

On May 11, 2011, at 11:54 AM, Sandy Walsh wrote:

Hi Lorin,

Zones can have multiple parents. Child Zones don't know their parents ... 
parents only know about children.

Sharing services across Zones isn't permitted (since they would need to share a 
DB and AMQP).

You could solve the problem by using the Capabilities to determine where 
instances are created (GPU only or not) or create smaller sub-zones for 
GPU/not-GPU and let the parent zones select between them.

-S



From: 
openstack-bounces+sandy.walsh=rackspace@lists.launchpad.netmailto:openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net
 [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of 
Lorin Hochstein [lo...@isi.edu]
Sent: Wednesday, May 11, 2011 12:06 PM
To: Openstack
Subject: [Openstack] Zones / distributed scheduler question

All:

For the proposed zones / distributed scheduler functionality 
https://blueprints.launchpad.net/nova/+spec/zones101, can we defines  zones 
that overlap without being nested?

Consider the scenario where:

* We have two datacenters: one in Arlington, Virginia (ARL), and one in Marina 
del Rey, California (MDR).
* At each datacenter, we have both conventional x86 nodes and  nodes with GPUs

Can we define our zones such that:

* We have a zone for each datacenter (ARL, MDR)
* We have a zone for GPU nodes  (GPU)

In this case, the GPU zone would both overlap the ARL and MDR zones, but ARL 
and MDR would not be fully nested within GPU.


Lorin
--
Lorin Hochstein, Computer Scientist
USC Information Sciences Institute
703.812.3710
http://www.east.isi.edu/~lorin


Confidentiality Notice: This e-mail message (including any attached or
embedded documents) is intended for