[Openstack] Reinstating Trey Morris for Nova Core
All, I think Trey Morris has been doing really well on reviews again, so I'd like to propose him to be reinstated for Nova core. Thoughts? -Dietz ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Quark, an NVP plugin alternative
Hey all, We just started work on a new plugin for Quantum that we're calling Quark. The blueprint lives here: https://blueprints.launchpad.net/quantum/+spec/quark In short, we want to provide an alternative to the existing NVP plugin that's a bit more cognizant of how much it should be interacting with the backend implementation. We've been running a hybridized mishmash of one-off Nova network manager, melange and one-off, v1 API Quantum for some time now, and we decided it was time to start pushing some work upstream that provides everyone with the performance benefits we've been trying to squeeze out of our implementation. Essentially, Quark will try to denormalize a lot of the data the existing plugin normally goes to NVP for, and provides an IPAM solution more like what Melange gives us. All work will be in the open. Comments, patches and snarky insight welcome. https://github.com/jkoelker/quark -Matt ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [nova] Proposal for Sean Dague to join nova-core
+1 From: Tong Li liton...@us.ibm.commailto:liton...@us.ibm.com Date: Friday, July 20, 2012 1:34 PM To: Vishvananda Ishaya vishvana...@gmail.commailto:vishvana...@gmail.com Cc: openstack-bounces+litong01=us.ibm@lists.launchpad.netmailto:openstack-bounces+litong01=us.ibm@lists.launchpad.net openstack-bounces+litong01=us.ibm@lists.launchpad.netmailto:openstack-bounces+litong01=us.ibm@lists.launchpad.net, Openstack (openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net) (openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net) openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] [nova] Proposal for Sean Dague to join nova-core +1, Sean will be a great addition to the nova-core. He has always been willing to lend a helping hand in the community. Tong Li Emerging Technologies Standards Building 501/B205 liton...@us.ibm.commailto:liton...@us.ibm.com [Inactive hide details for Vishvananda Ishaya ---07/20/2012 01:53:48 PM---Hello Everyone, When I was going through the list of r]Vishvananda Ishaya ---07/20/2012 01:53:48 PM---Hello Everyone, When I was going through the list of reviewers to see who would be good for nova-cor From: Vishvananda Ishaya vishvana...@gmail.commailto:vishvana...@gmail.com To: Openstack (openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net) (openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net) openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Date: 07/20/2012 01:53 PM Subject: [Openstack] [nova] Proposal for Sean Dague to join nova-core Sent by: openstack-bounces+litong01=us.ibm@lists.launchpad.netmailto:openstack-bounces+litong01=us.ibm@lists.launchpad.net Hello Everyone, When I was going through the list of reviewers to see who would be good for nova-core a few days ago, I left one out. Sean has been doing a lot of reviews lately[1] and did the refactor and cleanup of the driver loading code. I think he would also be a great addition to nova-core. Vish [1] https://review.openstack.org/#/dashboard/2750 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp attachment: graycol.gif___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [nova] Proposal to add Padraig Brady to nova-core
+1 On 7/19/12 9:46 AM, Johannes Erdfelt johan...@erdfelt.com wrote: On Wed, Jul 18, 2012, Vishvananda Ishaya vishvana...@gmail.com wrote: Padraig has been contributing a lot of code to all parts of nova, and has been contributing a lot to reviews[1]. I think he would make a great addition to nova-core. +1! JE ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nova Core Cleanup
The problem here is there are two opposing points: the idea that there are too many core reviewers, and the idea that patches aren't being reviewed fast enough. *Drags a yak into the room* Beyond that, what makes 20 better than 25, or 15, especially in light of the fact that we're not happy with how quickly reviews are moving? It seems like the answer here isn't just quantity, but a matter of focus and relative skill sets. I'm a core developer but, beyond code quality, I couldn't review in the best interests of functionality in and around KVM. Conversely, I'm sure most core developers couldn't do any better for XenServer. I liked the idea of lieutenants when it first came up almost 2 years ago. Have we talked about reinstating that? On 5/9/12 5:34 AM, Mark McLoughlin mar...@redhat.com wrote: On Tue, 2012-05-08 at 15:44 -0700, Vishvananda Ishaya wrote: The -2 issue is a good point. I personally treat a -1 (or +1) from the author of a given piece of code quite strongly when I do reviews, but you're right that the -1 could be more trivially overridden. Coincidentally, dprince and I discussed this a bit yesterday. I rarely -2, because I see it as a strong veto which blocks the patch or later revisions of the patch until I remove the -2. Maybe it's just the fact that I know I'm likely to be slow to come back and review later revisions of a patch that I've put a -2 on. Maybe I should just fix that :-) On the flip-side, I respect (and expect others to respect) a -1 from a core reviewer when reviewing later revisions of a patch - i.e. if I see a -1 from a core reviewer, then I check that the later revisions of the patch have actually addressed the previous concerns. If core reviewers' concerns are consistently ignored, that'd definitely cause me to pull out the big old -2. The removal is primarily to keep core a manageable size. We currently have 25 core members and still have many patches that are not being quickly reviewed. Giving too many people the ability to approve patches leads to inconsistency in code and the review process. It seems like overkill to have 20 people. I expect this number to decrease further if out plans to create substystem branches materialize. I agree, FWIW. 20 seems like the right number, more than that and I think folks assume someone else will do it, core membership isn't for life, the requirement that core members are actually active is a great incentive to do reviews, folks can easily be re-added again later, etc. etc. Cheers, Mark. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Monitoring / Billing Architecture proposed
If you wanted to use Yagi, it would be trivial to add a JSON only notifier to Yagi. If you're interested and need a hand, feel free to hit Dragon or I for assistance. From: Monsyne Dragon mdra...@rackspace.commailto:mdra...@rackspace.com Date: Mon, 23 Apr 2012 16:39:08 + To: Luis Gervaso l...@woorea.esmailto:l...@woorea.es Cc: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] Monitoring / Billing Architecture proposed This already exists in trunk. The Notification system was designed specifically to feed billing and monitoring systems. Basically, we don't want Nova/Glance/etc to be in the business of trying to determine billing logic, since it is different for pretty much everyone, so we just emit notifications to a queue and the interested pull what they want, and aggregate according to their own rules. On Apr 22, 2012, at 1:50 PM, Luis Gervaso wrote: Hi, I want to share the architecture i am developing in order to perform the monitorig / billing OpenStack support: 1. AMQP Client which listen to RabbitMQ / QPid (this should be interchangeable) (Own Stuff or ServiceMix / Camel) 2. Events should be stored on a NoSQL document oriented database (I think mongodb is perfect, since we can query in a super easy fashion) We have an existing system called Yagi (https://github.com/Cerberus98/yagi/) that listens to the notification queues and persists events to a Redis database. It then provides feeds as ATOM formatted documents that a billing system can pull to aggregate data, It also can support PubSub notification of clients thru the pubsubhubub protocol, and push events to a long-term archiving store thru the AtomPub protocol. That said, the notification system outputs its events as JSON, so it should be easy to pipe into a json document-oriented db if that's what you need. (we only use ATOM because we have a atom-based archiving/search/aggregation engine (it's open source: http://atomhopper.org/ ) our in-house systems already plug into. ) 3a. The monitoring system can pull/push MongoDB 3b. The billing system can pull to create invoices 4. A mediation EIP should be necessary to integrate a billing/monitoring product. (ServiceMix / Camel) This is to receive your feedback. So please, critics are welcome! Cheers! -- --- Luis Alberto Gervaso Martin Woorea Solutions, S.L CEO CTO mobile: (+34) 627983344 luis@mailto:luis.gerv...@gmail.comwoorea.eshttp://woorea.es/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Monsyne M. Dragon OpenStack/Nova cell 210-441-0965 work x 5014190 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Moving nova.rpc into openstack.common
I started looking into moving the notification drivers as well, but punted because of the reliance on multiple things, RPC included. For us, it's a nice to have but not a need. As for your plan, it seems sensible to me. On 4/2/12 3:26 PM, Russell Bryant rbry...@redhat.com wrote: Greetings, There was a thread on this list a little while ago about moving the notification drivers that are in nova and glance into openstack.common since they provide very similar functionality, but have implementations that have diverged over time. Is anyone actively working on this? If so, please let me know. For the message queue notifiers, nova uses nova.rpc to do the heavy lifting. Glance has notifiers written against the messaging libs directly. I think it makes sense to take the nova approach. This would mean moving nova.rpc into openstack.common before the notifiers can get moved. I have started looking at moving nova.rpc to openstack.common.rpc. My plan is: 1) Write a series of patches that reduces coupling in nova.rpc on the rest of nova. 2) Submit changes needed to support this decoupling to openstack.common. 3) Once nova.rpc is sufficiently decoupled, move it to openstack.common. While doing the above, I want to aim for the least amount of disruption to the rpc interface as is practical. While we're at it, is it time to drop nova.rpc.impl_carrot? It is marked as deprecated in Essex. Is there any reason anyone would still use impl_carrot over impl_kombu? Thoughts? Thanks, -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] yagi caching messages for time
max_messages actually corresponds to how many messages we try to send to the configured recipient before it moves onto the next queue in the list. Basically, it's super rudimentary QoS. As for expiration, that's controlled by redis itself. Under the persistence configuration section, you can set a key called entry_ttl which takes a value in seconds. If you look in yagi/persistence/redis_driver.py you can see the default value specified at the top of the file. From: Craig Vyvial cp16...@gmail.commailto:cp16...@gmail.com Date: Tue, 13 Mar 2012 13:50:19 -0500 To: openstack openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: [Openstack] yagi caching messages for time I noticed that in the configuration file for yagi that there is no option for storing messages for a time period but rather a limit of max_messages. Does anyone know if there is a way to configure yagi to keep messages for a set amount of time? ###yagi/etc/yagi.conf### [consumer:notifications.warn] apps = yagi.handler.pubsubhubbub_handler.PubSubHubBubHandler, yagi.handler.redis_handler.RedisHandler exchange = nova exchange_type = topic routing_key = notifications.warn durable = False max_messages = 1000 -Craig Vyvial ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Propose to make Monsyne Dragon a nova core developer
Hey guys, Dragon has really stepped up lately on reviewing patches into Nova, and has a ton of knowledge around Nova proper, so I propose he be added to Nova core. I think he'd be a great addition to the team. Matt ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Proposal for Lorin Hochstein to join nova-core
+1! On 11/30/11 8:05 AM, Sandy Walsh sandy.wa...@rackspace.com wrote: +1 ... good call! From: openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net [openstack-bounces+sandy.walsh=rackspace@lists.launchpad.net] on behalf of Vishvananda Ishaya [vishvana...@gmail.com] Sent: Tuesday, November 29, 2011 2:03 PM To: openstack (openstack@lists.launchpad.net) Subject: [Openstack] Proposal for Lorin Hochstein to join nova-core Lorin has been a great contributor to Nova for a long time and has been participating heavily in reviews over the past couple of months. I think he would be a great addition to nova-core. Vish ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Proposal for Mark McLoughlin to join nova-core
+1 from me, too On 11/30/11 4:47 AM, Soren Hansen so...@linux2go.dk wrote: 2011/11/29 Vishvananda Ishaya vishvana...@gmail.com: Mark is maintaining openstack for Fedora and has made some excellent contributions to nova. He has also been very prolific with reviews lately. Lets add him to core and make his reviews count towards potential merges! I'd be delighted to have Mark on the core team. +1 -- Soren Hansen| http://linux2go.dk/ Ubuntu Developer| http://www.ubuntu.com/ OpenStack Developer | http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] +1, All services should have WADLs
Couldn't agree more with this On a side-note, I'm now going to sign all emails as Weird, -Matt On 10/28/11 12:54 PM, Jay Pipes jaypi...@gmail.com wrote: On Fri, Oct 28, 2011 at 1:39 AM, John Dickinson m...@not.mn wrote: I am concerned about some of the implications that are being discussed. 1) A WADL is part of documentation of an API. Nobody is going to object to more documentation. 2) Being an open-source project, if somebody wants to commit to creating and maintaining a WADL for a particular part of Openstack, they are free to. Alternately, persuade somebody else to do it. However, having a WADL to describe a particular component of openstack is not something that can be forced onto that component. Phrases like All services should have WADLs are either meaningless (unenforcible or not really all services) or oppressive (mandating requirements on a project). 3) A WADL is not a replacement for any sort of dev documentation, and in fact, still requires there to be human-readable dev docs. Specifically for swift, not one of the current developers are going to either write or maintain a WADL for the swift API. However, we'll be happy to assist anyone who wants to write and maintain docs for swift, including WADLs. The important thing is that code talks. If you want WADLs (or your flavor of WADLs), make them! Stop trying to architect systems for architects. These things are meant to be used. Let's focus on what is necessary for getting a reliable system into the hands of those who will be using it. (Just about all of the above goes for things like API versioning, too. And packaging vs tarballs vs python libraries. And polling vs pushing. And the true meaning of what a ReST interface is.) Not sure what this means from a personal existential viewpoint, but I completely agree with John. Weird. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenSTack Minimum Spec
For the record, I'm currently running XenServer 6 inside of a VMWare Fusion install on my Mac, with 2GB of RAM dedicated to the entire VM. I created a new instance_type that creates 64MB instances. Works great so far! On 10/28/11 3:31 PM, Jay Pipes jaypi...@gmail.com wrote: 2011/10/28 Frans Thamura fr...@meruvian.org: 2011/10/29 Jay Pipes jaypi...@gmail.com: FWIW, my test machine at home is from System76. It's got 24G RAM and an i7 12-core processor and runs OpenStack via devstack.sh pretty well :) All for around $2400USD. You can get a 4-core, 16G system for aroun $700USD a box. hi jay, r u sure openstack must run minimum spec 24GB RAM or 16GB? how about 8GB? I didn't say that :) It will run on much less, yes. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] describing APIs for OpenStack consumers
To answer this, yes, it's possible. Rails does it already, in some fashion. They have some convention for generating all routes, which I think is one of the most important take aways here, and they provide a DSL for implementing other routes very easily. From there, one could readily generate the majority of the WADL. Jay is right though, it doesn't help with non-existant APIs :-) On 10/26/11 10:48 AM, Kevin L. Mitchell kevin.mitch...@rackspace.com wrote: On Tue, 2011-10-25 at 15:30 -0700, Joseph Heck wrote: It sounds like even though most of us hate WADL, it's what we're expending effort after to make a consolidated API set. So unless Nati and Ravi want to switch to using Swagger (or something else), WADL is the direction we're heading. I totally agree with Daryl that reading it is a PITA, and am finding (from my part) that the only definitive way to know about writing the docs and documenting the authoritative API is to read the underlying code. (which is what I suspect Nati likely did with the pull request that adds in WADL for the Nova/OpenCompute extension API) I wonder if it would be possible to generate much of the WADL from introspecting the code itself...surely the URL structure itself can be extracted from the paste setup, and the XML templates code I recently contributed could easily be traversed to provide at least a basic description of the output. That could at least provide a starting point for generating WADLs... (Of course, I propose this, having little idea of what actually goes in a WADL, but still... ;) -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Error while resizing. Nova compute on XenServer 5.6
Giuseppe,
I'm not sure that's the error that's causing the problem, but it seems
indicative of a larger issue: namely that the other compute node (the one
that received the VHD in the migration) perhaps cannot talk to the dom0?
Matt
On 9/20/11 11:29 AM, Giuseppe Civitella giuseppe.civite...@gmail.com
wrote:
Hi all,
I'm trying to resize an instance between two XenServer, each managed
by a domU running Natty and current Diablo milestone.
When I schedule the resize the instance gets a snapshot and its vhd
file get copyed by rsync on the other host. At this point the task
fails.
At this time there is a record on migrations table of cloud controller's
mysql.
On source host's nova-compute log i can see:
2011-09-20 18:11:34,446 DEBUG nova.rpc [-] received
{u'_context_roles': [], u'_context_request_id':
u'da2ac014-1600-4fa4-9be7-4c53add5caa4', u'_context_read_deleted':
False, u'args': {u'instance_id':
u'4548a3ed-2989-4fe7-bca9-bdf55bc9635a', u'migration_id': 12},
u'_context_auth_token': None, u'_context_is_admin': True,
u'_context_project_id': u'gcivitella_proj', u'_context_timestamp':
u'2011-09-20T16:11:33.724787', u'_context_user_id':
u'gcivitella_name', u'method': u'resize_instance',
u'_context_remote_address': u'127.0.0.1'} from (pid=20478)
process_data /usr/lib/pymodules/python2.7/nova/rpc/amqp.py:200
2011-09-20 18:11:34,447 DEBUG nova.rpc [-] unpacked context:
{'user_id': u'gcivitella_name', 'roles': [], 'timestamp':
u'2011-09-20T16:11:33.724787', 'auth_token': None, 'msg_id': None,
'remote_address': u'127.0.0.1', 'is_admin': True, 'request_id':
u'da2ac014-1600-4fa4-9be7-4c53add5caa4', 'project_id':
u'gcivitella_proj', 'read_deleted': False} from (pid=20478)
_unpack_context /usr/lib/pymodules/python2.7/nova/rpc/amqp.py:432
2011-09-20 18:11:34,447 INFO nova.compute.manager
[da2ac014-1600-4fa4-9be7-4c53add5caa4 gcivitella_name gcivitella_proj]
check_instance_lock: decorating: |function resize_instance at
0x362d758|
2011-09-20 18:11:34,448 INFO nova.compute.manager
[da2ac014-1600-4fa4-9be7-4c53add5caa4 gcivitella_name gcivitella_proj]
check_instance_lock: arguments: |nova.compute.manager.ComputeManager
object at 0x2f56810| |nova.rpc.amqp.RpcContext object at 0x4b46e50|
|4548a3ed-2989-4fe7-bca9-bdf55bc9635a|
2011-09-20 18:11:34,448 DEBUG nova.compute.manager
[da2ac014-1600-4fa4-9be7-4c53add5caa4 gcivitella_name gcivitella_proj]
instance 4548a3ed-2989-4fe7-bca9-bdf55bc9635a: getting locked state
from (pid=20478) get_lock
/usr/lib/pymodules/python2.7/nova/compute/manager.py:1121
2011-09-20 18:11:34,522 INFO nova.compute.manager
[da2ac014-1600-4fa4-9be7-4c53add5caa4 gcivitella_name gcivitella_proj]
check_instance_lock: locked: |False|
2011-09-20 18:11:34,522 INFO nova.compute.manager
[da2ac014-1600-4fa4-9be7-4c53add5caa4 gcivitella_name gcivitella_proj]
check_instance_lock: admin: |True|
2011-09-20 18:11:34,523 INFO nova.compute.manager
[da2ac014-1600-4fa4-9be7-4c53add5caa4 gcivitella_name gcivitella_proj]
check_instance_lock: executing: |function resize_instance at
0x362d758|
2011-09-20 18:11:34,716 DEBUG nova [-] Starting snapshot for VM
nova.db.sqlalchemy.models.Instance object at 0x496eb10 from
(pid=20478) _get_snapshot
/usr/lib/pymodules/python2.7/nova/virt/xenapi/vmops.py:515
2011-09-20 18:11:34,723 DEBUG nova.virt.xenapi.vm_utils [-]
Snapshotting VM OpaqueRef:defbf1c9-f3e6-3822-f110-7b61a35b7c18 with
label 'instance-0048-snapshot'... from (pid=20478) create_snapshot
/usr/lib/pymodules/python2.7/nova/virt/xenapi/vm_utils.py:338
2011-09-20 18:11:36,337 INFO nova.virt.xenapi [-] Task
[Async.VM.snapshot] OpaqueRef:7ca66e5a-261e-5e7b-441e-dca99ab2a1a8
status: success
valueOpaqueRef:05b68453-bee2-9c9b-db33-2bebe97cbdf7/value
2011-09-20 18:11:36,361 DEBUG nova.virt.xenapi.vm_utils [-] Created
snapshot OpaqueRef:05b68453-bee2-9c9b-db33-2bebe97cbdf7 from VM
OpaqueRef:defbf1c9-f3e6-3822-f110-7b61a35b7c18. from (pid=20478)
create_snapshot
/usr/lib/pymodules/python2.7/nova/virt/xenapi/vm_utils.py:352
2011-09-20 18:11:36,361 DEBUG nova.virt.xenapi.vm_utils [-]
Re-scanning SR OpaqueRef:d02abfe8-c781-f49f-81b2-6b981ed98097 from
(pid=20478) scan_sr
/usr/lib/pymodules/python2.7/nova/virt/xenapi/vm_utils.py:804
2011-09-20 18:11:36,916 INFO nova.virt.xenapi [-] Task [Async.SR.scan]
OpaqueRef:9514755f-8b94-dea1-5c0b-689bbe282804 status: success
2011-09-20 18:11:36,940 DEBUG nova.virt.xenapi.vm_utils [-] VHD
9d2ec65d-aeb4-4e6c-811f-78111d5835fc has parent
OpaqueRef:1f83df6e-d9aa-90cb-e2d9-b3c672c7ec4d from (pid=20478)
get_vhd_parent
/usr/lib/pymodules/python2.7/nova/virt/xenapi/vm_utils.py:840
and then the error that stops the task:
2011-09-20 18:12:28,311 DEBUG nova.manager [-] Notifying Schedulers of
capabilities ... from (pid=20478) periodic_tasks
/usr/lib/pymodules/python2.7/nova/manager.py:111
2011-09-20 18:12:28,312 DEBUG nova.rpc [-] Making asynchronous fanout
cast... from (pid=20478) fanout_cast
/usr/lib/pymodules/python2.7/nova/rpc/amqp.py:545
2011-09-20 18:12:28,312 INFO nova.rpc [-] Creating
Re: [Openstack] Integration tests
Regarding novaclient, I was debating this myself last night. Our backfire suite currently uses it. However, I can see scenarios where nova has advanced but novaclient has yet to, and because all three projects are independently developed, we're stuck temporarily. I can see utility in tests specifically for Novaclient, and ones that use a built in client (as the suite that Soren provided currently does.) Also, Backfire uses a module Kevin Mitchell wrote named Dtest that actually provides a lot of that same functionality you mention, Tim. It also gives you the ability to spawn the tests individually in greenthreads so they execute in parallel. We got push back when we initially tried to merge our tests into Nova, partially because it uses a 3rd party library for executing the tests. We can try merging these things together, if that's what everyone wants. On 9/12/11 11:39 AM, Tim Simpson tim.simp...@rackspace.com wrote: I'm with the Reddwarf (Database as a Service) team at Rackspace. We employ a large set of integration / functional tests which run in a Vagrant controlled VM. Our tests use python-novaclient as well as a similar client for Reddwarf we've written ourselves. The motivation is to eat our own dog food- if the novaclient is the official rich client for Python, why shouldn't the test make use of it? We also use a library called Proboscis so we can order our tests without prefixing numbers to them, automatically skip related tests when something in a dependency fails, and avoid global variables even as we pass state between related tests. I think the openstack-integration-tests would definitely benefit from it. I'd love to have a conversation about getting the traits outlined above adopted into this standardized testing solution. :) Output of our tests running: http://pastie.org/2521835 Proboscis documentation: http://packages.python.org/proboscis/ From: openstack-bounces+tim.simpson=rackspace@lists.launchpad.net [openstack-bounces+tim.simpson=rackspace@lists.launchpad.net] on behalf of Jay Pipes [jaypi...@gmail.com] Sent: Monday, September 12, 2011 8:20 AM To: Matt Dietz Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Integration tests Wow, another integration test framework emerges out of the woodwork :) Looking forward to getting all of these into a single place... and clearing up the lack of communication between teams on this subject! There's Kong, Stacktester, and Backfire. Any others out there we should know about? Cheers, jay On Sun, Sep 11, 2011 at 9:09 PM, Matt Dietz matt.di...@rackspace.com wrote: Suite looks great, Soren! Wanted to mention that we actually developed our own suite of tests, which you can find at https://github.com/ohthree/backfire I'm planning to reconcile the difference so we can stop the independent efforts, but it's going to take time. Something else I'd like to see in the suite, that we currently have in backfire via a custom test module, is the ability to parallelize the tests for execution. From: Soren Hansen so...@linux2go.dk Date: Sat, 10 Sep 2011 00:21:10 +0200 To: openstack@lists.launchpad.net Subject: Re: [Openstack] Integration tests That link shouldn't have included the +bug bit. Copy/paste fail. :( Sent from my phone. Pardon my brevity. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Integration tests
Ditto On 9/12/11 2:10 PM, Sandy Walsh sandy.wa...@rackspace.com wrote: From: Jay Pipes [jaypi...@gmail.com] Can we discuss pulling novaclient into Nova's source tree at the design summit? +1 This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Integration tests
Actually to extend this, Novaclient is used for zone communication inside of nova. It not being included is silly, at this point On 9/12/11 1:37 PM, Jay Pipes jaypi...@gmail.com wrote: On Mon, Sep 12, 2011 at 1:22 PM, Matt Dietz matt.di...@rackspace.com wrote: Regarding novaclient, I was debating this myself last night. Our backfire suite currently uses it. However, I can see scenarios where nova has advanced but novaclient has yet to, and because all three projects are independently developed, we're stuck temporarily. This, I believe, is a problem. novaclient is currently the only client that isn't in the core project itself. Can we discuss pulling novaclient into Nova's source tree at the design summit? I can see utility in tests specifically for Novaclient, and ones that use a built in client (as the suite that Soren provided currently does.) Sure. This is actually something that Glance's functional test suite does. It uses httplib2 as a direct HTTP client and uses bin/glance to test the supplied Glance client. Also, Backfire uses a module Kevin Mitchell wrote named Dtest that actually provides a lot of that same functionality you mention, Tim. It also gives you the ability to spawn the tests individually in greenthreads so they execute in parallel. We got push back when we initially tried to merge our tests into Nova, partially because it uses a 3rd party library for executing the tests. We can try merging these things together, if that's what everyone wants. I'd be totally cool with bringing DTest goodness into https://github.com/sorenh/openstack-integration-tests. :) Less duplication of effort, the better IMHO. openstack-integration-tests should be in the business of writing integration tests, not multi-processing testing libraries. -jay This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Integration tests
Suite looks great, Soren! Wanted to mention that we actually developed our own suite of tests, which you can find at https://github.com/ohthree/backfire I'm planning to reconcile the difference so we can stop the independent efforts, but it's going to take time. Something else I'd like to see in the suite, that we currently have in backfire via a custom test module, is the ability to parallelize the tests for execution. From: Soren Hansen so...@linux2go.dkmailto:so...@linux2go.dk Date: Sat, 10 Sep 2011 00:21:10 +0200 To: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: Re: [Openstack] Integration tests That link shouldn't have included the +bug bit. Copy/paste fail. :( Sent from my phone. Pardon my brevity. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] New nova service proposal
I would think we have enough tracking information to support the goal of identifying failures. In any scenario, some of the failures will simply be unrecoverable. Regarding the process crashing, who's to say the retry process also wouldn't crash? We could endlessly argue the arbiter/watchdog processes will crash at each tier. As such, I think it's better to say we need a simpler mechanism for identifying failures and perhaps a best-effort retry. Retrying can be scary, to say the least. You can't possibly handle all of the possible failure scenarios, and some of the ones you think you can might be different in subtle ways such that retrying them only causes more issues. I agree with Lamar that we could make things significantly more reliable, and I think that's where we should start. We may find that, after some stabilization work, the failure rate is acceptably low and any retry mechanism is no longer required. On 8/29/11 11:24 AM, Kevin L. Mitchell kevin.mitch...@rackspace.com wrote: On Fri, 2011-08-26 at 23:10 +, Monsyne Dragon wrote: First off, I think it would be better if whatever had the failure responded by sending a request somewhere (a cast) to say Hey, this bombed. Retry it. What if the failure was due to the process crashing, so that it can't possibly send a request/cast off for retry? -- Kevin L. Mitchell kevin.mitch...@rackspace.com This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Proposal for Brian Lamar to join Nova-Core
+1 from me On 5/31/11 3:10 PM, Jay Pipes jaypi...@gmail.com wrote: +1 On Tue, May 31, 2011 at 4:00 PM, Todd Willey t...@ansolabs.com wrote: +1 On Tue, May 31, 2011 at 3:16 PM, Vishvananda Ishaya vishvana...@gmail.com wrote: While I was checking branch merges, I noticed that Brian Lamar (blamar), is not listed as a nova-core developer. This is most definitely a travesty, as he has been one of the most prolific coders/reviewers over the past few months. So I'm proposing that he is added as a nova-core member. Vish ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [nova-core] Proposal to add Brian Waldon to nova-core
+1 On 5/11/11 4:24 PM, Paul Voccio paul.voc...@rackspace.com wrote: +1 On 5/11/11 3:01 PM, Ed Leafe ed.le...@rackspace.com wrote: On May 11, 2011, at 1:06 PM, Jay Pipes wrote: Subject says it all. I think Brian's demonstrated that his code reviews are thoughtful and thorough, and he knows the OpenStack API controller stuff as well as anyone else I believe. Definitely! +1 -- Ed Leafe Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [nova-core] Proposal to add Mark Washenberger to nova-core
+1 On 5/11/11 12:07 PM, Jay Pipes jaypi...@gmail.com wrote: Mark's been a very good reviewer and an invaluable resource on the API side, particularly regarding serialization. I propose he join nova-core. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Code Reviews
+1 On 5/11/11 3:16 PM, Jay Pipes jaypi...@gmail.com wrote: ++ on your suggestions. On Wed, May 11, 2011 at 3:38 PM, Vishvananda Ishaya vishvana...@gmail.com wrote: Hello Everyone, We have quite a large backlog of merge proposals here: https://code.launchpad.net/~rlane/nova/lp773690/+merge/59565 I've been attempting to go through them to find some high priority ones to review. It seems like people are being pretty active in reviewing branches, but there are a lot old branches that haven't been touched in a while. So first I have a general request that anyone who has old branches in for review: please update your branches or mark them Work In Progress to remove them from the review queue. I'd also like to propose a change to our process that will make the ready to review branches easier to find. I'd like for nova-core to set branches to WIP if there are two significant needs fixings or or needs information. That way everyone doesn't have to sort through branches that have already been reviewed but are waiting on updates. We may need to use our judgement here, so if a large branch has a needs fixing for a minor typo or some such, you could leave it under needs review so it gets viewed by more people. Here is an example where i think this policy will be useful: You see a branch that already has a 'Needs Fixing: this needs a failing test. If you look at the branch and reach the same conclusion, you can mark it Needs Fixing: I agree, needs a test like xxx and then set the branch to Work In Progress. When the author has added the test or needs to make more comments, he can set it back to Needs Review. I think this will generally keep the review board a little cleaner and also each branch will end up with a couple of people that are queued to review once the changes have come in. Does this seem acceptable to everyone? If I don't here any major dissents, I will add this info to the wiki and we can put it into practice. Vish ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Notifications proposal
George,
Unless I'm completely mistaken, I think our proposal satisfies this suggestion.
What you have here looks like a slight variation on PSHB. Our stuff is coded
such that the responsibility of any heavy lifting falls outside of Nova. In our
case, we'll be implementing the PubSub publisher externally; I.e. I don't think
any of the infrastructure for making PSHB work belongs in Nova. We can then
follow all of the other rules of PSHB(feed discovery and subscriptions,
callbacks etc.)
Does this make sense?
Matt
From: George Reese
george.re...@enstratus.commailto:george.re...@enstratus.com
Date: Mon, 9 May 2011 23:17:29 -0500
To: Jorge Williams
jorge.willi...@rackspace.commailto:jorge.willi...@rackspace.com
Cc: Matt Dietz matt.di...@rackspace.commailto:matt.di...@rackspace.com,
openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net
openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net
Subject: Re: [Openstack] Notifications proposal
I think notifications need to be kept really simple. I put out a proposal a few
months ago at:
http://broadcast.oreilly.com/2011/04/proposal-for-cloud-state-notifications.html
Let the subscribers do any heavy lifting. Just provide them enough information
that they can make the right requests.
-George
On May 9, 2011, at 10:58 PM, Jorge Williams wrote:
On May 9, 2011, at 6:39 PM, Matt Dietz wrote:
Jorge,
Thanks for the feedback!
Regarding the message format, we actually don't need the unique id in the
generic event format because that's implementation specific. The external
publisher I've implemented actually does all of the pubsubhubbub specific heavy
lifting for you. The idea behind keeping this system simple at the nova layer
is allowing people to implement anything they'd like, such as emails or paging.
I guess, I'm not seeing the whole picture. So these are internal messages?
Will they cross service boundaries / zones? I'm sorry I missed the
conversation at the summit :-) Is there a blueprint I should be reading?
For categories, were you considering this to be a list? Could you give an
example of an event that would span multiple categories?
From an Atom perspective, I suppose anything a client might want to key in on
or subscribe to may be a category. So create may be a category -- a billing
layer may key in on all create messages and ignore others. compute may also
be a category -- you can aggregate messages from other services so It'd be
nice for messages from compute to have their own category. To my knowledge,
atom doesn't have the concept of priority so WARN may also be a category. I
suppose if these are internal messages an external publisher can split the
event_type and priority into individual categories.
Finally, I can make the changes to the timestamp. This as just a hypothetical
example, anyway.
Okay cool, thanks Matt.
On May 9, 2011, at 6:13 PM, Jorge Williams
jorge.willi...@rackspace.commailto:jorge.willi...@rackspace.com wrote:
On May 9, 2011, at 5:20 PM, Matt Dietz wrote:
Message example:
{ 'publisher_id': 'compute.host1',
'timestamp': '2011-05-09 22:00:14.621831',
'priority': 'WARN',
'event_type': 'compute.create_instance',
'payload': {'instance_id': 12, ... }}
There was a lot of concern voiced over messages backing up in any of the
queueing implementations, as well as the intended priority of one message over
another. There are couple of immediately obvious solutions to this. We think
the simplest solution is to implement N queues, where N is equal the number of
priorities. Afterwards, consuming those queues is implementation specific and
dependent on the solution that works best for the user.
The current plan for the Rackspace specific implementation is to use
PubSubHubBub, with a dedicated worker consuming the notification queues and
providing the glue necessary to work with a standard Hub implementation. I have
a very immature worker implementation at https://github.com/Cerberus98/yagi
https://github.com/Cerberus98/yagi if you're interested in checking that out.
Some thoughts:
In order to support PubSubHubBub you'll also need each message to also contain
a globally unique ID. It would also be nice if you had the concept of
categories. I realize you kinda get that with the event type
compute.create_instance but there are always going to be messages that may
belong to multiple categories. Also, ISO timestamps with a T :
2011-05-09T22:00:14.621831 are way more interoperable -- I would also include
a timezone designator Z for standard time 2011-05-09T22:00:14.621831Z --
otherwise some implementation assume the local timezone.
-jOrGe W.
___
Mailing list: https://launchpad.net/~openstack
Post to :
openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
--
George
Re: [Openstack] Notifications proposal
These all sound perfect to me. I'm hoping our PSHB implementation solves that
problem. More specifically, the publisher worker that I linked to earlier I
think solves most of what you're referring to, and works well with the Google
reference hub. There's a lot more work to be done, but I think it's on target
with what you're suggesting
Thanks!
From: George Reese
george.re...@enstratus.commailto:george.re...@enstratus.com
Date: Tue, 10 May 2011 12:07:22 -0500
To: Matt Dietz matt.di...@rackspace.commailto:matt.di...@rackspace.com
Cc: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net
openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net
Subject: Re: [Openstack] Notifications proposal
I came into the conversation late and it struck me this proposal was a bit
heavier than what I was proposing.
I agree with letting something outside of Nova do the heavy lifting. Much more
scaleable. The base things I would like to see are:
a) the minimal amount of information to let a subscriber know that there was a
change (not the details of the change)
b) only information that is safe to deliver over a public network to an
untrusted target
c) that the subscriber be able to be a programmatic endpoint (not simply
email/SMS)
d) the subscriber should not assume anything about the message, including its
authenticity (it should use its credentials to verify the truth of the message
and details of change with provider)
-George
On May 10, 2011, at 12:01 PM, Matt Dietz wrote:
George,
Unless I'm completely mistaken, I think our proposal satisfies this suggestion.
What you have here looks like a slight variation on PSHB. Our stuff is coded
such that the responsibility of any heavy lifting falls outside of Nova. In our
case, we'll be implementing the PubSub publisher externally; I.e. I don't think
any of the infrastructure for making PSHB work belongs in Nova. We can then
follow all of the other rules of PSHB(feed discovery and subscriptions,
callbacks etc.)
Does this make sense?
Matt
From: George Reese
george.re...@enstratus.commailto:george.re...@enstratus.com
Date: Mon, 9 May 2011 23:17:29 -0500
To: Jorge Williams
jorge.willi...@rackspace.commailto:jorge.willi...@rackspace.com
Cc: Matt Dietz matt.di...@rackspace.commailto:matt.di...@rackspace.com,
openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net
openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net
Subject: Re: [Openstack] Notifications proposal
I think notifications need to be kept really simple. I put out a proposal a few
months ago at:
http://broadcast.oreilly.com/2011/04/proposal-for-cloud-state-notifications.html
Let the subscribers do any heavy lifting. Just provide them enough information
that they can make the right requests.
-George
On May 9, 2011, at 10:58 PM, Jorge Williams wrote:
On May 9, 2011, at 6:39 PM, Matt Dietz wrote:
Jorge,
Thanks for the feedback!
Regarding the message format, we actually don't need the unique id in the
generic event format because that's implementation specific. The external
publisher I've implemented actually does all of the pubsubhubbub specific heavy
lifting for you. The idea behind keeping this system simple at the nova layer
is allowing people to implement anything they'd like, such as emails or paging.
I guess, I'm not seeing the whole picture. So these are internal messages?
Will they cross service boundaries / zones? I'm sorry I missed the
conversation at the summit :-) Is there a blueprint I should be reading?
For categories, were you considering this to be a list? Could you give an
example of an event that would span multiple categories?
From an Atom perspective, I suppose anything a client might want to key in on
or subscribe to may be a category. So create may be a category -- a billing
layer may key in on all create messages and ignore others. compute may also
be a category -- you can aggregate messages from other services so It'd be
nice for messages from compute to have their own category. To my knowledge,
atom doesn't have the concept of priority so WARN may also be a category. I
suppose if these are internal messages an external publisher can split the
event_type and priority into individual categories.
Finally, I can make the changes to the timestamp. This as just a hypothetical
example, anyway.
Okay cool, thanks Matt.
On May 9, 2011, at 6:13 PM, Jorge Williams
jorge.willi...@rackspace.commailto:jorge.willi...@rackspace.com wrote:
On May 9, 2011, at 5:20 PM, Matt Dietz wrote:
Message example:
{ 'publisher_id': 'compute.host1',
'timestamp': '2011-05-09 22:00:14.621831',
'priority': 'WARN',
'event_type': 'compute.create_instance',
'payload': {'instance_id': 12, ... }}
There was a lot of concern voiced over messages backing up in any of the
queueing implementations, as well as the intended priority of one message over
another. There are couple
Re: [Openstack] Proposal for Nova Core
+1 On 5/10/11 3:20 PM, Jay Pipes jaypi...@gmail.com wrote: +1 On Tue, May 10, 2011 at 4:13 PM, Paul Voccio paul.voc...@rackspace.com wrote: All, I would like to nominate Dan Prince (https://launchpad.net/~dan-prince) for nova-core. He has been a solid contributor in terms of code, reviews and discussions during the summit. Thanks, pvo Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Notifications proposal
Hey guys,
Monsyne Dragon and myself are proposing an implementation for notifications
going forward. Currently my branch exists under
https://code.launchpad.net/~cerberus/nova/nova_notifications. you'll see that's
it been proposed for merge, but we're currently refactoring it around changes
proposed at the summit during the notifications discussion, which you can see
at http://etherpad.openstack.org/notifications
At the heart of the above branch is the idea that, because nova is about
compute, we get notifications away from Nova as quickly as possible. As such,
we've implemented a simple modular driver system which merely pushes messages
out. The two sample drivers are for pushing messages into Rabbit, or doing
nothing at all. There's been talk about adding Burrow as a third possible
driver, which I don't think would be an issue.
One of the proposals is to have priority levels for each notification. What
we're proposing is emulating the standard Python logging module and providing
levels like WARN' and CRITICAL in the notification. Additionally, the
message format we're proposing will be a JSON dictionary containing the
following attributes:
publisher_id - the source worker_type.host of the message.
timestamp - the GMT timestamp the notification was sent at
event_type - the literal type of event (ex. Instance Creation)
priority - patterned after the enumeration of Python logging levels in
the set (DEBUG, WARN, INFO, ERROR, CRITICAL)
payload - A python dictionary of attributes
Message example:
{ 'publisher_id': 'compute.host1',
'timestamp': '2011-05-09 22:00:14.621831',
'priority': 'WARN',
'event_type': 'compute.create_instance',
'payload': {'instance_id': 12, ... }}
There was a lot of concern voiced over messages backing up in any of the
queueing implementations, as well as the intended priority of one message over
another. There are couple of immediately obvious solutions to this. We think
the simplest solution is to implement N queues, where N is equal the number of
priorities. Afterwards, consuming those queues is implementation specific and
dependent on the solution that works best for the user.
The current plan for the Rackspace specific implementation is to use
PubSubHubBub, with a dedicated worker consuming the notification queues and
providing the glue necessary to work with a standard Hub implementation. I have
a very immature worker implementation at https://github.com/Cerberus98/yagi if
you're interested in checking that out.
We'll be going forward with this plan immediately, but we'd love feedback if
you have it. Questions, comments, concerns are very much welcomed!
Matt Dietz
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Lieutenants (Again)
I'll add myself to the pool for Xen From: Vishvananda Ishaya vishvana...@gmail.commailto:vishvana...@gmail.com Date: Tue, 3 May 2011 17:08:35 -0700 To: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: [Openstack] Lieutenants (Again) Hey Everyone, A couple people have stepped up to lead development on different sections of the code. Thanks to Brian Waldon and Brian Lamar for listing themselves at http://wiki.openstack.org/NovaLieutenants I'm still looking for points of contact for the remaining sections. Even though alternate projects are being created for volumes and networking, I still need someone to manage the existing nova code in these areas. Even more important are POCs for the various hypervisors. We have a breaking change coming in the multinic code, and I want to make sure that changes like this make it into all hypervisors as quickly as possible. I'm assuming someone from citrix is right for both Xen and ESX hypervisors. Volunteers? Vish ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Creating a forum
Fair points. I can see it being used for user support. Another way to have these sorts of discussions would be an openstack-users list, but I think lists present much more friction to tire-kickers or intrigued admins. Forums have a much lower barrier to entry, and consequently (IMHO) they are better tools for building communities. Controlling forum spam is an amazing pain, but that's another issue. :) Can you explain this a little? I don't necessarily object, but I frankly don't see the difference, either. On 5/2/11 5:01 PM, Ron Pedde ron.pe...@rackspace.com wrote: On 5/2/11 4:03 PM, Matt Dietz matt.di...@rackspace.com wrote: I think a forum as a means of communication is great. However, I'm not sure I feel it's the right fit here. My main concern in this regard is that there would be a separation of important discussions. I think the class of questions on a forum would be wildly different than the questions on a dev mailing list. Forums would be a great place to ask questions like How do I set up my bridge interface to persist on reboot? Questions like these aren't the right questions for the openstack mailing list, and end-users don't want to bother devs with this sort of thing, so they walk away from the project before getting it set up. Properly moderated, the forums could push dev questions to the mailing list, while removing distraction from devs and building a community of users. I would also be concerned about a feeling of false consensus on hot-button topics that see activity on one channel but not the other. Finally, we'd be introducing yet another fire hose for project communications, and frankly I personally wouldn't feel compelled to check both, and I'm sure I'm not the only one. I don't see forums as a channel for project communication or consensus building. I see it more as a way for users-to-user discussion on topics like how I implemented X on top of openstack, or How can I integrate system X with my openstack cluster. Things that don't get discussed on the dev list. Another way to have these sorts of discussions would be an openstack-users list, but I think lists present much more friction to tire-kickers or intrigued admins. Forums have a much lower barrier to entry, and consequently (IMHO) they are better tools for building communities. Controlling forum spam is an amazing pain, but that's another issue. :) Just my opinion, but I think end-user/sysadmin focused forums are a great idea. -- Ron Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] core dev
Your reviews have been very thorough for FF. +1 On 3/24/11 4:58 PM, Jay Pipes jaypi...@gmail.com wrote: +1 from me. On Thu, Mar 24, 2011 at 5:40 PM, Trey Morris trey.mor...@rackspace.com wrote: All, consider me as a nova core dev. Seems we could use a few more and I need an excuse to spend more time reviewing code :) Thanks, -tr3buchet Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Proposal to be a member of Nova Core ...
+1 to this! On 2/17/11 3:28 PM, Soren Hansen so...@ubuntu.com wrote: +1! (Yup, that was +factorial(1), for those keeping score at home) 2011/2/17 Sandy Walsh sandy.wa...@rackspace.com: I'd like to help out on the review process as per http://wiki.openstack.org/Governance/Approved/CoreDevProcess I like quiet walks in the park and black and white movies. -Sandy Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Soren Hansen Ubuntu Developerhttp://www.ubuntu.com/ OpenStack Developer http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Proposal to be a member of Nova Core
+1 :-D From: Josh Kearney josh.kear...@rackspace.commailto:josh.kear...@rackspace.com Date: Thu, 17 Feb 2011 10:32:04 -0600 To: openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Subject: [Openstack] Proposal to be a member of Nova Core I'd like to volunteer my time to help out with reviews in Nova by becoming a member of Nova-Core. -Josh Kearney ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.netmailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
