Re: [Openstack] [Horizon] [UX] phabriactor/pholio as a possible UX option
Hi! Sorry - we've been a bit busy and this got put on the back burner. I believe that ttx has done some work over the last couple of weeks ... Theirry, any updates from your end? Github as an option is problematic for several reasons. The ones that come to mind are that it's not opensource, it doesn't integrate into any of the rest of our tooling or SSO, and we're actually working to clarify to people that we do not do our development on github, and adding the usage of a github issue tracker somewhere would kindof undercut that. So we're still trying to find an option that we can run that does work for everyone. I will redouble those efforts. We also have an effort underway to spin up an owncloud instance ... so for filesharing needs that might be a choice. On 07/12/2013 08:47 PM, Toshiyuki Hayashi wrote: Hi, So far, my understanding is that requirements are: - discussion - messages containing images - possibly specific image annotation/commenting Also if there is file sharing space, that would be great. e.g.) - photoshop template for designing and prototyping - html template for designing and prototyping - wireframe data (it seems Jaromir has created good one already :-) ) - some document regarding UI Discourse seems good, but still Github is better for me. Why Github was evaluated as not suitable? On Tue, Jul 9, 2013 at 4:52 AM, Jaromir Coufal jcou...@redhat.com wrote: On 2013/27/06 09:37, Thierry Carrez wrote: As a data point, Discourse could also be a solution: http://www.discourse.org/ It's clearly a discussion tool (including pretty advanced threading, post likes, etc.), and messages can contain images. See a design discussion for example at: http://test.ubuntu-discourse.org/t/a-ubuntu-ish-theme-for-the-site/177 Discourse actually looks pretty good. I was playing around that a little bit and like it. We can consider labels as categories - not optimal, but can work if we have only design discussions. Only problem is that we might want to extend the tool for other discussions as well and then it will be less optimal. Do you guys see any other possibilities apart from this one? -- Jarda ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Horizon] [UX] phabriactor/pholio as a possible UX option
Hi! Thanks for looking in to Phabricator for us! The feedback is helpful. I think we've also got some concerns around elements of it as well. However, I still don't feel like I fully understand what the requirements list are here. github isn't a requirement, it's a suggested solution, and one with already distressing massive negative implications. so I'd like us to work on what requirements the tool needs to have so that we can figure out solutions that will solve them. So far, my understanding is that requirements are: - discussion - messages containing images - possibly specific image annotation/commenting Are there any others I've missed? To summarize things we've learned so far about possible solutions: - Launchpad Bugs don't work due to lack of images - Phabricator is too image centric, and also confusing - github issues is not open source, and also increases confusion about OpenStack's use of github, and is not integrated with the rest of the project - mailing list is too text oriented and has a bad threading model We've got folks working on the area - so let's figure out what we need and then we can move forward. thanks! Monty On 06/25/2013 12:16 AM, Jaromir Coufal wrote: Hey All, I investigated and played with few tools for team collaboration, mainly focused on designs and discussions. They are mostly similar as Phabricator [1], what Monty suggested. You can see inVision [2] or GoVisually [3] for example. And of course there are more, however they are all somehow similar. I have few notes which covers most of them (from my point of view): * I can't help myself, but I have disorganized feeling. - Might be only my problem, but the whole system navigation is just... Strange. Maybe too much graphical :) * Main focus on pictures. - You can't start a thread without picture. - It's just a little bit weird, if everything is focused on the picture, which from my point of view shouldn't be the main point. Pictures and other documents should be supportive material - discussion matters here. - Due on pictures focus, discussions are just somehow neglected. * I love the inline comments for pictures, but... - Having possibility to attach comment to any place of the picture is cool, but... still this tool will fail for example in sequence of screens, if you are presenting workflow. * Mainly - I don't see developers coming to this tool and actively ask discuss. Therefore, also thanks to comments from Toshi and Kyle, I tried to focus a little bit more on GitHub. I asked couple of colleagues and friends what would they prefer. From developers, the answer was obvious - GitHub. Designers said that they wouldn't mind GH, they are ok with it. Anyway, it's a normal discussion tool, nothing to be afraid of. The reasons why GitHub matters are already covered in my first e-mail and I still see it as the best possibility. Another reason for GitHub occurred in last conversation on G+ community site. There started thread about design question, which got solved, but then followed implementation discussion how to implement such thing. And here you can see, that any tool focusing on designers in the first place, would fail. I really don't want to discourage creative people from proposals and discussions - completely the opposite. I want them to connect to developers and vice versa. That's why I believe that GitHub worths trying. -- Jarda [1] http://www.invisionapp.com/ [2] http://www.invisionapp.com/ [3] http://www.govisually.com/ On 2013/19/06 03:49, Monty Taylor wrote: Hey all! I spoke with Gabriel about this briefly on IRC, but there is an app for Phabricator called Pholio which seems to do the things the UI folks are looking for. To play with it further, I've spun up a phabricator here: http://phab.mnorp.com/ You should have gotten account signup emails (if not, look in your spam folder - it's a throwaway machine) Check out: http://phab.mnorp.com/pholio/ I've put up one design review here: http://phab.mnorp.com/M1 that Jim and I have discussed a little bit. We're not thrilled with Phabricator for things like bug tracking or code review yet - but it's configurable enough that we could turn off everything except design review and move forward with that. Then, if we get to a point where more of its features are useful to us, then great - or if this winds up something we only ever use for design reviews - well, that's great too. David/Olaph - we'll need an OpenID SSO provider landed upstream before we can use this. (we are NOT going to carry local patches) There is an upstream auth refactor going on: https://secure.phabricator.com/T1536 Also, you'll see on http://phab.mnorp.com/M1 a lorem ipsum over to the side. We should finish that work. Then we'll need to do a proper puppet install and skinning. Don't anybody do work yet - mainly I want to see if this is suitable for the UI folks, and if we
Re: [Openstack] New code name for networks
On 05/13/2013 11:03 AM, Doug Hellmann wrote: On Sat, May 11, 2013 at 5:48 PM, Anne Gentle a...@openstack.org mailto:a...@openstack.org wrote: On Sat, May 11, 2013 at 3:12 PM, Monty Taylor mord...@inaugust.com mailto:mord...@inaugust.com wrote: On 05/11/2013 04:07 PM, Asher Newcomer wrote: Or even better, just continue to call it openstack networking. The code names only serve to confuse the uninitiated. They needlessly steepen the learning curve and slow uptake. The problem with OpenStack Networking (or getting rid of codenames) is seen with pre-incubation-incubation-integrated cycle. A project cannot call itself OpenStack Foo until it actually _is_ openstack foo. So any new project by necessity has to start off as something else. But - if we then require them to drop that name and become openstack foo when they become incubated or integrated, then we've got what's become a stable project with a decent amount of contributors renaming itself. Every. Time. The code names aren't just cute. I kind of wish they were, because it would make several things much easier if we could just ditch them and do a pure openstack. code namespace. But the reality is that it gets really really tricky to deal with a bunch of things if they go away. I told Monty and the TC this at the Summit (sorry I couldn't attend the session about code names). I find this trickiness a weak argument in the face of the invented names that are getting to be as bad as U.S. pharmaceuticals. Plus it forces us to put a lookup table in the docs forever. [1] Let's find a process for naming that meets a few reqs: - describes the service - goes through a legal review - enables new eyes to understanding it by reading the English word that the service represents (that can be translated into a meaningful word in other languages) If we have to preface with Stackforge to indicate pre-incubation, that's fine. Use Incubating or some such to indicate incubation. Meaningful words have meaning. +1 Use a namespace package openstack then each project has a unique package under that for their meaningfully named package (compute, networking, etc.). We only have to rename projects if they start out with bad names to begin with. Projects that want to be integrated should be developing on stackforge and using the openstack namespace package. It's just logically infeasible. If we were to make that choice, I'm pretty sure we'd have to stop everything that everyone is doing and just deal the fallout from doing it. In any case - we did have a session on this at the summit, and we did lay out a strategy for moving forward. The etherpad is here https://etherpad.openstack.org/ProjectsReNaming tl;dr - we are going to rename quantum to a new codename, and we are going to do our best as a community to lessen the external usage of our codenames. I acknowledge we still have to indicate what commands, daemon names, and so on are related to a particular service. That issue is also fixable with some resources and effort and pays off in easier adoption and understanding. The unified command line project will resolve the command issue because all commands will be openstack $noun $verb (end users shouldn't have to know which OpenStack component owns a resource in order to operate on it via the command line). Daemon startup scripts could use a similar framework, or just a naming convention (openstack-compute-api, openstack-network-api, etc.). I agree - having the unified command line client will be very helpful to lessening our externally-facing usages of code names. Doug Anne 1. http://docs.openstack.org/grizzly/openstack-compute/install/yum/content/terminology-codenames.html On May 11, 2013 3:59 PM, Davanum Srinivas dava...@gmail.com mailto:dava...@gmail.com mailto:dava...@gmail.com mailto:dava...@gmail.com wrote: Lattice -- dims On Sat, May 11, 2013 at 3:52 PM, Mark Turner m...@amerine.net mailto:m...@amerine.net mailto:m...@amerine.net mailto:m...@amerine.net wrote: Tubes ;-) On Sat, May 11, 2013 at 12:51 PM, Jason Smith jason.sm...@rackspace.com mailto:jason.sm...@rackspace.com mailto:jason.sm...@rackspace.com mailto:jason.sm...@rackspace.com wrote: Hello, I understand why we had to give up Quantum code name but rather
Re: [Openstack] New code name for networks
On 05/11/2013 08:58 PM, Anne Gentle wrote: On Sat, May 11, 2013 at 6:43 PM, Monty Taylor mord...@inaugust..com mailto:mord...@inaugust.com wrote: On 05/11/2013 05:48 PM, Anne Gentle wrote: On Sat, May 11, 2013 at 3:12 PM, Monty Taylor mord...@inaugust..com mailto:mord...@inaugust.com mailto:mord...@inaugust.com wrote: On 05/11/2013 04:07 PM, Asher Newcomer wrote: Or even better, just continue to call it openstack networking. The code names only serve to confuse the uninitiated. They needlessly steepen the learning curve and slow uptake. The problem with OpenStack Networking (or getting rid of codenames) is seen with pre-incubation-incubation-integrated cycle. A project cannot call itself OpenStack Foo until it actually _is_ openstack foo. So any new project by necessity has to start off as something else. But - if we then require them to drop that name and become openstack foo when they become incubated or integrated, then we've got what's become a stable project with a decent amount of contributors renaming itself. Every. Time. The code names aren't just cute. I kind of wish they were, because it would make several things much easier if we could just ditch them and do a pure openstack. code namespace. But the reality is that it gets really really tricky to deal with a bunch of things if they go away. I told Monty and the TC this at the Summit (sorry I couldn't attend the session about code names). I promise, it wasn't the world's most fun session. :) I'm sure. :) I think I don't have much regret but do feel sorry that I don't know more. I find this trickiness a weak argument in the face of the invented names that are getting to be as bad as U.S. pharmaceuticals. Plus it forces us to put a lookup table in the docs forever. [1] Let's find a process for naming that meets a few reqs: - describes the service - goes through a legal review - enables new eyes to understanding it by reading the English word that the service represents (that can be translated into a meaningful word in other languages) I don't think it's a weak argument at all. There are real technical issues. The technical issues, to me, and I may be missing something, are when the name is used as: - service/daemon name - command/CLI name And the directories in the code where those things live, and the name of the python package that gets installed, and the name of the client library used to connect to it. You can use any pet name you want for your team and project while addressing technical issues some other way? Here's another way I'm looking at the naming autonomy/process. Why incubate? - you get to pick your cool name OR - you get access to infrastructure, tools, events, community, and branding that is OpenStack The naming can't be THAT crucial. I get that we want projects to be fun to work on. But it can't be just the naming that brings the fun. I don't think having a cool name is interesting or important at all. Not one little bit. If any part of this was about esprit de corps or team bonding or identity I'd be 100% on board with the no-codenames approach. Also, to be clear, I don't think there are any problems with using non-codename for identification. Already, as part of the upgrade of our build stuff to PBR I've been setting the project short description to the non-codename. So, nova's is OpenStack Compute I think that's a great idea, and it's important. Equally as important, although harder, is that we should all try our best to use the non-codenames when we're talking about official projects. It's not going to work or be 100% covered - but we should all make a best-effort. (these are all things that did come out of that session - perhaps one of us should do a writeup on that?) The thing that _I'm_ sticking on is the above list of technical issues. What is the daemon named? What is the command line tool named? What is directory in which the code lives? Those may seem trivial, but those are the primary interface that a developer and deployer has with the project. And it's an issue because of the lifecycle of these code projects before they are integrated. It's not ok for moniker to call it's API daemon openstack-dns-api until it's actually openstack DNS. It has to call itself something though. That's where names come in. It's a practical thing that there must be a name. And let's be honest - it's my least favorite part of making a project. So much so that our CI project which makes jenkins jobs from yaml files is called jenkins-job-builder
Re: [Openstack] New code name for networks
Don't get me wrong... I don't disagree with you. I think lawyers are super important. At the risk of some of my best friends are... I took the lsat a few years ago (did quite well ;) ) with a view towards law school and my girlfriend is in law school. TOTALLY respect for lawyers. I don't want to keep lawyers away from the project any more than I want to keep business people away. What I do want to do is keep random lawyers away from implementation details of our source code repositories. Here's the rub... The technical leadership of the project has no body from which it can receive legal counsel. The foundation has counsel, but that is different than the TC having actual counsel to whom we can give intent direction and receive advice. We have many interested and involved parties who are top lawyers, and I regularly talk with them. But they're all quite clear that they cannot directly advise us as the collected technical leadership. So we are in a sticky situation of needing to make decisions that have actual technical ramifications, and that may have legal consequences, but no counsel to whom we can give priorities and direct to help us find solutions. From the foundation perspective, the risk/reward is clear, and from that perspective discussions with counsel can proceed. But in addition to a board who can have an executive session with counsel to which privilege attaches, the foundation has executive director who can also have privileged discussions. Those are from a particular POV, and it's a totally valid one. The technical leadership, OTOH, does not have this ability, nor are we interested in doing things in private. We are tasked with technical leadership, which is what we intend to do. So, with all due respect to the many awesome lawyers who are involved with the project, what I'm getting at is that we need to do the right thing as best we can based on unofficial advice from well meaning people who may or may not have legal backgrounds as well as input and help from Jonathan Bryce and the foundation board where appropriate. I wish there was a more straightforward way to approach this, but it's not a straightforward situation. However, maybe if we play our cards right and are conscientious in our approach to the problems we can help forge ground for more people who want to run a thing like we are without getting into too much trouble along the way. Brad Knowles bknow...@momentumsi.com wrote: On May 12, 2013, at 9:52 AM, Monty Taylor mord...@inaugust.com wrote: I would really like to keep the marketing/business folks out of our source code. Most importantl, I would really like to keep the lawyers out of our source code. My wife is a lawyer, so maybe I'm particularly sensitive to this issue. However, I don't believe that the lawyers themselves are the problem. I think the problem is that maybe you haven't gotten the *RIGHT* lawyers involved.. The RIGHT lawyers can help you spot potential problems while they're still just mole hills in the distance, and can help you avoid having them get turned into mountains. But they can't just give you a set of rules and have you follow them blindly -- the reason they are the RIGHT lawyers is because of all their experiences and the lessons they've learned, and the places where they've seen clients trip up in the past, and therefore they know what to look for. They can't necessarily tell you what to look for, they'll just know it when they see it. Sure, they have rules that will cover the easy 80%, but it's the hard 20% that you have to really worry about. The RIGHT lawyers will know when they look at a contract what kinds of things don't need to be written down, because they're covered by laws on the books, or by existing case law. And when they look at contracts in a foreign country, they'll have some idea of what kinds of questions to ask -- and the different kind of legal systems around the world, how they differ, etc. This is the kind of thing that got BazaarVoice in trouble -- they went out of their way to structure the buyout of their major competitor in such a way that it didn't need to be approved in advance by the regulators. But then they got hit with a lawsuit by the federal government, and they ended up having to divest themselves of most of the assets they had bought. Had they structured the deal differently, they could have at least known in advance whether or not the regulators would have approved it, and if approved would never had to divest themselves of their acquisition. The RIGHT lawyers can help you see and avoid these problems before you ever get there, but even they can't necessarily help you if you take the attitude that all lawyers are bad and therefore you have to keep them out of your business until there is simply no other choice. Ask yourself this -- do you want the first time you turn to a lawyer to be when you're in the dock for a crime you committed
Re: [Openstack] New code name for networks
I have been arguing for: mutnuaq Granted, it takes a minute to learn how to type, but it's just a little snarky, and it takes up the exact same number of letter. However, it does screw with sorting. SO - what about: qumutna It's a little bit easier to wrap your head around, it's still clearly an homage, and it should be super easy to bulk cut/replace. On 05/11/2013 03:58 PM, Davanum Srinivas wrote: Lattice -- dims On Sat, May 11, 2013 at 3:52 PM, Mark Turner m...@amerine.net wrote: Tubes ;-) On Sat, May 11, 2013 at 12:51 PM, Jason Smith jason.sm...@rackspace.com wrote: Hello, I understand why we had to give up Quantum code name but rather than just refer to it as networking let's come up with a new code name! Thoughts? Thanks, -js ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] New code name for networks
On 05/11/2013 04:07 PM, Asher Newcomer wrote: Or even better, just continue to call it openstack networking. The code names only serve to confuse the uninitiated. They needlessly steepen the learning curve and slow uptake. The problem with OpenStack Networking (or getting rid of codenames) is seen with pre-incubation-incubation-integrated cycle. A project cannot call itself OpenStack Foo until it actually _is_ openstack foo. So any new project by necessity has to start off as something else. But - if we then require them to drop that name and become openstack foo when they become incubated or integrated, then we've got what's become a stable project with a decent amount of contributors renaming itself. Every. Time. The code names aren't just cute. I kind of wish they were, because it would make several things much easier if we could just ditch them and do a pure openstack. code namespace. But the reality is that it gets really really tricky to deal with a bunch of things if they go away. On May 11, 2013 3:59 PM, Davanum Srinivas dava...@gmail.com mailto:dava...@gmail.com wrote: Lattice -- dims On Sat, May 11, 2013 at 3:52 PM, Mark Turner m...@amerine.net mailto:m...@amerine.net wrote: Tubes ;-) On Sat, May 11, 2013 at 12:51 PM, Jason Smith jason.sm...@rackspace.com mailto:jason.sm...@rackspace.com wrote: Hello, I understand why we had to give up Quantum code name but rather than just refer to it as networking let's come up with a new code name! Thoughts? Thanks, -js ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Davanum Srinivas :: http://davanum.wordpress.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] New code name for networks
Jeremy Stanly on IRC just suggested kumquat... but to that I respond: qumkuat Same benefits as qumutna - except it's more pronouncable. On 05/11/2013 04:07 PM, Monty Taylor wrote: I have been arguing for: mutnuaq Granted, it takes a minute to learn how to type, but it's just a little snarky, and it takes up the exact same number of letter. However, it does screw with sorting. SO - what about: qumutna It's a little bit easier to wrap your head around, it's still clearly an homage, and it should be super easy to bulk cut/replace. On 05/11/2013 03:58 PM, Davanum Srinivas wrote: Lattice -- dims On Sat, May 11, 2013 at 3:52 PM, Mark Turner m...@amerine.net wrote: Tubes ;-) On Sat, May 11, 2013 at 12:51 PM, Jason Smith jason.sm...@rackspace.com wrote: Hello, I understand why we had to give up Quantum code name but rather than just refer to it as networking let's come up with a new code name! Thoughts? Thanks, -js ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Guest PXE Boot
Neat!
Have you seen any of the work around nova baremetal (which is
transitioning to be called ironic?) Related to that is a set of virtual
power drivers which allow for treating virtual machines like real
machines - so that you can use nova to pxe boot a kvm or a virtualbox or
a vmware instance.
I know it's not exactly the same thing, but I don't know what you're
trying to accomplish. Perhaps what you want is similar enough to work
together?
Monty
On 05/10/2013 12:55 PM, David Hill wrote:
Hi guys,
I was trying to PXE boot a guest for quite some time now and I think
I’ve found a solution that is kind of hackish but pretty simple. I’m
not quite sure it’s good to go in trunk but felt like I’d share it since
I’ve been messing a while on this.
If anybody have a better solution, I would really like to hear/see/try it …
Here is how I did it:
First, patch the libvirt/driver.py file:
--- /usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py.orig
2013-05-10 16:25:17.787862177 +
+++ /usr/lib/python2.6/site-packages/nova/virt/libvirt/driver.py
2013-05-10 16:26:39.442022870 +
@@ -87,6 +87,9 @@
LOG = logging.getLogger(__name__)
libvirt_opts = [
+cfg.StrOpt('default_guest_boot_dev',
+ default='hd',
+ help='Sets the default guest boot device'),
cfg.StrOpt('rescue_image_id',
default=None,
help='Rescue ami image'),
@@ -1792,7 +1795,7 @@
instance['name'],
ramdisk)
else:
-guest.os_boot_dev = hd
+guest.os_boot_dev = FLAGS.default_guest_boot_dev
if FLAGS.libvirt_type != lxc and FLAGS.libvirt_type != uml:
guest.acpi = True
And add to nova.conf:
default_guest_boot_dev=network
And finally add to /etc/dnsmasq.conf
dhcp-boot=boot\x86\pxelinux.com,host_name,host_ip
dhcp-no-override
And restart dnsmasq.conf
In my actual setup, the guest will PXE boot, show the menu 60 seconds
and then boot from hard disk after the 60 seconds timeout.
Thank you very much,
Dave
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] New code name for networks
On 05/11/2013 05:48 PM, Anne Gentle wrote: On Sat, May 11, 2013 at 3:12 PM, Monty Taylor mord...@inaugust..com mailto:mord...@inaugust.com wrote: On 05/11/2013 04:07 PM, Asher Newcomer wrote: Or even better, just continue to call it openstack networking. The code names only serve to confuse the uninitiated. They needlessly steepen the learning curve and slow uptake. The problem with OpenStack Networking (or getting rid of codenames) is seen with pre-incubation-incubation-integrated cycle. A project cannot call itself OpenStack Foo until it actually _is_ openstack foo. So any new project by necessity has to start off as something else. But - if we then require them to drop that name and become openstack foo when they become incubated or integrated, then we've got what's become a stable project with a decent amount of contributors renaming itself. Every. Time. The code names aren't just cute. I kind of wish they were, because it would make several things much easier if we could just ditch them and do a pure openstack. code namespace. But the reality is that it gets really really tricky to deal with a bunch of things if they go away. I told Monty and the TC this at the Summit (sorry I couldn't attend the session about code names). I promise, it wasn't the world's most fun session. :) I find this trickiness a weak argument in the face of the invented names that are getting to be as bad as U.S. pharmaceuticals. Plus it forces us to put a lookup table in the docs forever. [1] Let's find a process for naming that meets a few reqs: - describes the service - goes through a legal review - enables new eyes to understanding it by reading the English word that the service represents (that can be translated into a meaningful word in other languages) I don't think it's a weak argument at all. There are real technical issues. That assumes that OpenStack is involved with the project pre-incubation. That's was the case with Quantum and Ceilometer and Ironic. On the other hand, the heat folks had heat-api.org and a heat github org and other things before they started down the stackforge road. Looking right now at non-incubated projects just off the top of my head: Libra is an LBaaS solution that was started on stackforge and which it's increasingly unlikely will go to incubation rather than just atttempt to merge stuff in with Quantum LBaaS. Moniker is a DNSaaS that hasn't applied for incubation, might do so, but as of right now has been around for almost a year yet has no formal relationship with OpenStack. healthnmon may or may not be an incubation candidate. Before that, reddwarf was not-incubated for pretty much the entire time OpenStack has existed until just now. All of these things have had to have lifecycles during a period of time before they have any interaction with us formally. On the other hand, if we did checking pre-incubation, we'd be in a very odd position of granting permission/blessing/tentative interest in projects before they come close to sorting things out. Moniker is great, but 6 months ago there were as many as 4 different DNSaaS possibilities. In any case, I don't think there is any way that we can, become directly involved in projects before they are incubated. Stackforge helps already, in that moniker is stackforge/moniker rather than openstack/moniker. But neither has any effect on the fact that there is a directory named moniker in moniker repository. If we go with 'descriptive' names, then there are two choices: a) moniker starts life with a directory structure underneath openstack/dns inside of its repository, even though it is not an openstack project. b) moniker starts life with a moniker directory, and then when incubated, renames that directory from moniker to openstack/dns. We can't stop anybody from doing (a) of course, but let me tell you - you wanna talk about confusing and bad messaging - we had apt repos at the beginning of the project with giant letters on them FOR TESTING ONLY but because they had our name on them people assumed we supported them. (b) sound easy, until you reckon with things like line-wrapping changes, sort order changes, and client library/API consumption changes. If something is using python-monikerclient and thus the monikerclient namespace of the API, that person would have to port their code to now do something like openstack.dns.client or similar. Even ignoring people who may have already been using the code (many of these things start life as a service by one of our member companies and then enters incubation when it's become baked a little bit - reddwarf was in production at RAX and HP before it got incubated, moniker is in production at HP, etc) and just focusing on our own code base, the massive undertaking that it is to change the name of the project inside of the project is daunting enough that we're
[Openstack] TC Election Results
The Sprint 2013 TC Election has concluded. The at-large members elected are: vishy ttx For a term of one year. mikal For a term of six months. Congratulations. http://www.cs.cornell.edu/w8/~andru/cgi-perl/civs/results.pl?id=E_5af0b5341a01b892 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] REMINDER: TC election closes tomorrow
Hey all! 186 of 618 voters have cast ballots in the TC election. Tomorrow is the last day ... go vote now! Monty PS. Look for an email titled Poll: Spring 2013 OpenStack TC Election and click the link ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] TC candidacy
I confirm that Thierry is eligible to run for a seat on the TC. On 03/15/2013 11:32 AM, Thierry Carrez wrote: Hi everyone, I'd like to run for reelection to one of the Technical Committee directly-elected seats. For those who don't know me, I've been handling release management duties for OpenStack since November 2010, a work currently sponsored by the OpenStack Foundation. My involvement is mostly around project coordination, keeping a global view and trying to anticipate issues as our development community grows even larger. I'm also heading the Vulnerability Management team, which handles incoming security issues reports. On the development side, I authored the rootwrap framework which is being used by a few of our projects, and whenever I find some free time, I'm working on improving it. I've been regularly elected by our community to the Project Policy Board and Technical Committee directly-elected seats for the last two years. I was heavily involved in the transition to our new governance, authored the Technical Committee charter, and have been chosen to chair it for the past 6 months. I think it's important that the Technical Committee contains representation from the horizontal functions within the project (Docs, QA, Infrastructure, Vulnerability management, Release management...), since each project is already represented by the seats granted for all PTLs. Over the last 30 months we grew from 2 projects to 10 projects, and I'm proud to be part of this community which successfully managed to handle growth and adoption while preserving our ideals of open design and open development. The challenges ahead of us include accommodating further growth, resist fragmentation, and maintaining efficiency and coherence as we grow well past Dunbar's number. I hope that you place me in a position where I can help us through those challenges. Thanks, ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] TC candidacy
I confirm that Carl is eligible to run for a seat on the TC. On 03/15/2013 07:08 PM, Carl Perry wrote: Greetings - I would like to run for a TC seat as well. My platform is a focus on deployment and operations for OpenStack. I'm not going to mince words: deploying OpenStack is hard. Maintaining it is even harder. I don't think it needs to be this way. I spent two years designing and deploying a cloud based on OpenStack and it really seemed much harder to do than it should have been, especially when you factor in configuration management tools. I would like to promote a more operational approach to the decisions made inside the OpenStack community. Some of it is easy, some of it is hard. I don't expect things to change overnight, but I do feel that a course correction is needed, that it's going to need to come from a whole project approach as well as code contribution, and that now is the time to make it. As I said, I spent the last two years as an implementor of OpenStack with the DreamCompute project at DreamHost and have now moved on to a new position as a solutions architect at Midokura. These positions have given (and continue to give) me a customer perspective of what it takes to deploy, maintain, and upgrade OpenStack in production. While my company supports me and my desire to run for the TC, I'm running independent of my company in order to keep a vendor neutral approach. Thanks for your time! -Carl ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] TC candidacy
I confirm that Chris is eligible to run for a seat on the TC. On 03/18/2013 06:11 PM, Chris Behrens wrote: Hi all, I'd like to announce my candidacy for a seat on the OpenStack Technical Committee. - General background - I have over 15 years of experience designing and building distributed systems. I am currently a Senior Software Developer at Rackspace, where I have been for a 2 and a half years now. Most of my time at Rackspace has been spent working on OpenStack as both a developer and a technical leader. My first week at Rackspace was spent at the very first OpenStack Design Summit in Austin where the project was announced. Prior to working at Rackspace, I held various roles over 14 years at Concentric Network Corporation/XO Communications including Senior Software Architect and eventually Director of Engineering. My main focus there was on an award winning web/email hosting platform which we'd built to be extremely scalable and fault tolerant. While my name is not on this patent, I was heavily involved with the development and design that led to US6611861. - Why am I interested? - I have strong feelings for OpenStack and I want to help take it to the next level. I have a lot of technical knowledge and experience building scalable distributed systems. During most of my past experience, I haven't had the luxury of having access to a lot extremely fast hardware, so it's been important to make software as performant as possible. I've also had to put lots of effort into having 0 downtime, meaning code can be updated seamlessly without dropping clients. I've also been one to lead host and software security efforts so I have a lot of strong feelings in this area. I am extremely interested in using this experience to make OpenStack perform well, be secure, be more easily pluggable, and easy to use! - OpenStack contributions - As I mentioned above, I was at the very first design summit, so I've been involved with the project from the beginning. I started the initial work for nova-scheduler shortly after the project was opened. I also implemented the RPC support for kombu, making sure to properly support reconnecting and so forth which didn't work quite so well with the carrot code. I've contributed a number of improvements designed to make nova-api more performant. I've worked on the filter scheduler as well as designing and implementing the first version of the Zones replacement that we named 'Cells'. I'm currently looking forward to restructuring our use of DB API to better support upgrades w/ schema changes as well as committing an alternative DB backend implementation for mysql that significantly reduces how long we block on DB API calls compared to sqlalchemy. - Summary - I feel my years of experience contributing to and leading large scale technical projects along with my knowledge of the OpenStack projects will provide a good foundation for technical leadership. Thanks, - Chris ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] TC candidacy
I confirm that Eric is eligible to run for a seat on the TC. On 03/18/2013 03:23 PM, Eric Windisch wrote: Hello, I'd like to run for a seat on the Technical Committee. I am a Principal Engineer at Cloudscaling, but I am running as an individual. For over two years, beginning with Bexar, I have been working to bring working OpenStack solutions to customers. My first code contributions landed in Cactus, but my contributions have not been limited to code alone. I propose and drive summit and mailing list discussions, contribute to those discussions led by others, have begun contributing to packaging efforts, and soon intend to land documentation changes. The code I do write and review tends to be in Oslo. Working with Oslo, and with OpenStack deployments, I have the perspective to work across projects. Before my involvement with OpenStack, in my former role as the owner and technical director of a VPS hosting company, I was already building open source cloud infrastructure as early as 2007. For years preceding the existence of OpenStack, I've worked on many of the technical problems and challenges facing a cloud, not only those related to virtualization, networking, and storage, but also billing, metering, and user interfaces. The number of companies and contributors involved in OpenStack is exploding. Each conference has been shockingly larger than the last. We have new projects being added in each 6-month release. These are problems that the TC must be prepared to deal with. We need technical leadership that understands the problems of these projects, can help them succeed, and especially for those individually elected seats -- can bring them to together. I am deeply committed to the success of OpenStack and to open source cloud. I thank you for your consideration and ask that you please vote for me. Regards, Eric Windisch ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] TC election open
If you are an ATC, you should by this point have received a link that will let you vote in the OpenStack TC election. You should vote. If you read it, you will note that it says February 28 is the end date. That is an error - March 28 is the end date. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] TC candidacy
I confirm that Michael is eligible to run for a seat on the TC. On 03/15/2013 11:27 AM, Michael Still wrote: Hi. I'd like to run for the TC Spring 2013 election. I am a senior software engineer at Rackspace in their OpenStack group, and have worked in a variety of cloud devops roles for the last seven years. I think my operations experience gives me an interesting perspective into where OpenStack should be going in the next few years. My basic platform is the same as for the Nova PTL election [1] -- I think we need to get better at closing bugs, and selecting defaults which work out of the box for OpenStack deployers. We are blessed with a very engaged user community, and we need to focus us much as possible on giving new users a good experience. I am an active Nova and Oslo core reviewer and frequently appear in the top ten contributors for Nova in a given month. I am a very active code reviewer, especially for Nova. I am also serve on the OpenStack Vulnerability Management Team. I strongly believe in the future of OpenStack and want to be part of that success in any way I can. For the last two years my non-OpenStack open source contributions have mainly been as Director for linux.conf.au 2013, which was the largest OpenStack event to be run in Australia so far. Thanks, Michael 1: http://lists.openstack.org/pipermail/openstack-dev/2013-March/006417.html ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] TC Candidacy
I confirm that Chuck is eligible to run for a seat on the TC. On 03/16/2013 12:59 PM, Chuck Thier wrote: Hello all, I would like to run for a seat on the TC. I am one of the original developers of Rackspace Cloud Files which became Openstack Swift, and was deeply involved with the creation of Openstack. I also lead the team at Rackspace to create Cloud Block Storage which built off the foundation of Openstack Cinder, and during that time contributed directly and indirectly to Nova and Cinder. I have the unique experience of not only developing across several Openstack projects, but also being responsible for deploying the projects at a very large scale. I have a track record for fighting for reasonable APIs, upgradeability, and maintainability across projects. I have also fought to ensure that we have equal and fair representation from all projects in the Openstack community. The purpose of the TC is not to legislate from the bench, but when questions and issues are brought to the TC I will continue to support these ideals. I deeply care for Openstack and its future success, so please consider me for this position. Thanks, -- Chuck Thier @creiht ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] TC candidacy
I confirm that Gary is eligible to run for a seat on the TC. On 03/15/2013 02:13 PM, Gary Kotton wrote: Hi, I'd like to run for the Technical Committee in the up and coming elections. I am a Principle Software Engineer at Red Hat. I have been actively developing OpenStack since the Essex release. I am currently a Quantum core developer. In addition to this I am also core on the Stable Maintenance team. I also have contribute to Nova, OSLO, documentation and devstack. In Nova the work was mainly focused on the Quantum integrations. My latest contribution was the VM ensembles, part of which was added in Grizzly release and hopefully will be completed in the up and coming Havana release. I spend most of my days reviewing, testing, debugging, documenting and developing with the goal of making OpenStack better. I am thankful to my employer Red Hat to have the opportunity and time to work on such and amazing project. I have close to 18 years of experience in the industry. Over that course of time I have strived to produce quality, usable, robust and optimal solutions. I would like to bring all that experience to the table to ensure that we have a better product. We are working in a very healthy, dynamic and vibrant community. A few things that I would like to improve are the following: - cross project interaction - growth of the community - sharing of ideas and information In my spare time I run. Thanks Gary ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] TC candidacy
I confirm that Vishvananda is eligible to run for a seat on the TC. On 03/15/2013 05:41 PM, Vishvananda Ishaya wrote: Hello all, I would like to run for a seat on The Technical Comittee. I have been working on Nova since it was a project as Nasa and I have been heavily involved in openstack since it was founded. I was elected to the precursor to TC (the Project Oversight Committee, later named the Project Policy Board) when it was first created. I was also elected as the first PTL for Nova and have been filling that role for the last two years. I am the top contributor to Nova over the lifetime of the project, and the third most frequent contributor over the past 12 months. I helped to create Devstack, Keystone, and Cinder. In addition, I have contributed to Oslo and I am a member of the stable-maintenance team. Despite passing on the mantle of Nova PTL, I am still deeply involved with OpenStack and I want to make sure that it continues to be a huge success. As OpenStack grows, one of the most important challenges we face is integration. It is vital that we have technical leaders that are focused cross-project and dedicated to making OpenStack as a whole successful. I currently work as the Director of Open Source at Nebula, Inc. Previously I was a principal engineer on the private cloud team at Rackspace, and before that I was a senior developer on the Nebula project at NASA where Nova was created. Thanks, Vish ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] PTL Election Results
The PTL elections for the Havana cycle have completed. The new PTL's are: Nova: Russell Bryant Ceilometer: Julien Danjou Keystone: Dolph Matthews Congratulations! As a side note, we had over 50% participation in each of the three elections, which I have been told is actually a really good turnout. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Technical Committee Nominations are Open
Now that the TC elections have ended, we now have three at-large seats open. https://wiki.openstack.org/wiki/TC_Elections_Spring_2013 Mar 15 - 21: Open candidacy to directly-elected TC positions Mar 22 - 28: TC elections The persons ranking 1st and 2nd will get one-year seats on the TC, and the person ranking 3rd will be elected for a 6-month seat as a replacement for Russell Bryant who is now granted a PTL seat. The electorate for the TC direct seats election are the Foundation individual members that are also committers for an official OpenStack project, over the Folsom-Grizzly timeframe, up to 23:59 PST on February 28, 2013. Any member of the election electorate can propose his candidacy for this election. No nomination is required. They do so by sending an email to the openstack@lists.launchpad.net mailing-list, which the subject: TC candidacy. The email can include a description of the candidate platform. The candidacy is then confirmed by one of the election officials, after verification of the electorate status of the candidate. Note: PTLs that just got elected in the previous election are automatically granted a 6-month term seat on the TC, and therefore cannot run for this election. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] PTL Elections are Open!
If you are an ATC for a keystone, nova or ceilometer, you should have now received in the mail your link to vote in the PTL election for that project be sure to vote! The elections end March 14. For the other projects, there was only one person standing for election, so congratulations guys, you win! Havana PTLs so far: Glance: Mark Washenberger Heat: Steven Dake Oslo: Mark McLoughlin Quantum: Marck McClain Swift: John Dickenson Horizon: Gabriel Hurley Cinder: John Griffith Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] orchestration engine wrt to openstack api
On 03/04/2013 10:02 AM, Nirlay Kundu wrote: Which tool would you use for configuration management geared towards Openstack api : Chef, Puppet, Saltstack ? If anybody has experience with Saltstack, please let me know the advantages , shortcomings. I would use Heat to orchestrate thing WRT the openstack APIs. On a per-machine basis, chef, puppet and salt should all be fine for doing configuration management and can read the metadata that heat provides onto the machine. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] PTL and Technical Committee Election Time
Hi everyone! Spring is upon us, which means it's time to overthrow our leadership and replace it with new (or the same) leadership! First we elect PTL's, then we elect TC at-large members. So first things first: ** PTL Candidacy Nomination Period is open from now until March 7 ** If you would like to announce that you would like to run for PTL for a project, please send an email to openstack-...@lists.openstack.org and tell us. You should probably tell us WHY you're a good choice, but I'll leave that decision up to you. The nomination period will end at 23:59 PST March 7. Each PTL is elected by the Active Project Contributors for the project - which means you have to have landed a change to the project in question in year preceding the election. We apparently still haven't amended our process to define a person who has only reviewed code as an Active Contributor - so if all you've done over the past year is code review - thank you! - but you don't get a vote. Also, to be an APC, the Foundation bylaws also mandate that you be an individual member of the Foundation. That means that *in order to vote* (or propose your candidacy), you additionally have to *join the Foundation* if you haven't done so yet. The email you specify there will be the one used in the election process. The cut-off date for joining the Foundation to be able to participate to this election is set to the end of day (23:59 PST), Thursday March 7. You can do so at: http://openstack.org/join For those of you following at home, this means we are about to have 10 (ten) different simultaneous elections. If you aren't following at home, the following projects are electing PTLs right now: Ceilometer Cinder Glance Heat Horizon Keystone Nova Oslo Quantum Swift After that, the following will happen: March 8-14: Vote for PTL March 15-21: Nominations for direct seats March 22-28: Vote for direct seats Thanks everybody! PS. Yes. I know, we're not supposed to cross-post - but this is important and I'm pretty sure that there are people out there who do not read both lists. It's also a subject I'm not expecting tons of chatter in response to. PPS. If any of you are wondering why this email sounds less official than usual, ttx is eligible for a seat this term, wheras I'm not since my term is still active - so you're stuck with me running this thing. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Name it Hood!
On 01/25/2013 12:54 PM, Thomas Goirand wrote: On Fri Jan 25 2013 06:29:32 AM CST, Monty Taylor mord...@inaugust.com wrote: f) Hood is only 4 letters. Think about that when you think about typing hatfield a lot. Also, if we name it hatfield, we're going to have to have the M summit somewhere that has a town called McCoy. Oh! I didn't realized that was the reason why the next summit will be in Gortland ;) Oh dear god. PLEASE let the next summit be in a place called Gortland. g) I'll buy you a beer at the summit if you vote for Hood. And will you also sign my PGP key? These go 2gether! Only if you can prove to my satisfaction that you are, actually, Thomas Goirand. :) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Name it Hood!
Hey all! Here's my pitch for Hood: a) It's the tallest mountain in Oregon, and honestly, it's a pretty kick-ass mountain in general b) Being in the pacific northwest, the mountain itself is quite regularly in the clouds. That's gotta count for something. c) It's actually a volcano. d) Mount Hood is CLEARLY an Oregon thing. Havana is clearly a town in Cuba. (We should have a design summit in cuba!!!) e) Harbor is super-problematic because of the US/UK clash in spelling. Half of us will spell it wrong no matter what. f) Hood is only 4 letters. Think about that when you think about typing hatfield a lot. Also, if we name it hatfield, we're going to have to have the M summit somewhere that has a town called McCoy. g) I'll buy you a beer at the summit if you vote for Hood. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Name it Hood!
On 01/25/2013 07:08 AM, Matt Joyce wrote: I think we all know M is for Manhattan. I will vote for whatever thing gets me a summit in NYC - although I'd love to not have to wait until fall 2015 On Thu, Jan 24, 2013 at 12:07 PM, Sean Dague sda...@linux.vnet.ibm.com mailto:sda...@linux.vnet.ibm.com wrote: On 01/24/2013 02:50 PM, Monty Taylor wrote: Hey all! Here's my pitch for Hood: a) It's the tallest mountain in Oregon, and honestly, it's a pretty kick-ass mountain in general b) Being in the pacific northwest, the mountain itself is quite regularly in the clouds. That's gotta count for something. c) It's actually a volcano. d) Mount Hood is CLEARLY an Oregon thing. Havana is clearly a town in Cuba. (We should have a design summit in cuba!!!) e) Harbor is super-problematic because of the US/UK clash in spelling. Half of us will spell it wrong no matter what. f) Hood is only 4 letters. Think about that when you think about typing hatfield a lot. Also, if we name it hatfield, we're going to have to have the M summit somewhere that has a town called McCoy. Yes, but I'm totally cool with that. +1 for Hatfield. Just means that we have to go to Florida for the M summit - http://en.wikipedia.org/wiki/__Fort_McCoy,_Florida http://en.wikipedia.org/wiki/Fort_McCoy,_Florida g) I'll buy you a beer at the summit if you vote for Hood. Monty _ Mailing list: https://launchpad.net/~__openstack https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~__openstack https://launchpad.net/~openstack More help : https://help.launchpad.net/__ListHelp https://help.launchpad.net/ListHelp -- Sean Dague IBM Linux Technology Center email: sda...@linux.vnet.ibm.com mailto:sda...@linux.vnet.ibm.com alt-email: slda...@us.ibm.com mailto:slda...@us.ibm.com _ Mailing list: https://launchpad.net/~__openstack https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~__openstack https://launchpad.net/~openstack More help : https://help.launchpad.net/__ListHelp https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Name it Hood!
I vote for hoodies. On 01/25/2013 09:28 AM, Doug Hellmann wrote: Will we have hoodies instead of t-shirts for ODS attendees? Doug On Thu, Jan 24, 2013 at 2:50 PM, Monty Taylor mord...@inaugust.com mailto:mord...@inaugust.com wrote: Hey all! Here's my pitch for Hood: a) It's the tallest mountain in Oregon, and honestly, it's a pretty kick-ass mountain in general b) Being in the pacific northwest, the mountain itself is quite regularly in the clouds. That's gotta count for something. c) It's actually a volcano. d) Mount Hood is CLEARLY an Oregon thing. Havana is clearly a town in Cuba. (We should have a design summit in cuba!!!) e) Harbor is super-problematic because of the US/UK clash in spelling. Half of us will spell it wrong no matter what. f) Hood is only 4 letters. Think about that when you think about typing hatfield a lot. Also, if we name it hatfield, we're going to have to have the M summit somewhere that has a town called McCoy. g) I'll buy you a beer at the summit if you vote for Hood. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] TC Candidacy
Hi everybody! I'd like to run for a seat on the Technical Committee. - OpenStack Experience - I run the CI and Developer Automation team for OpenStack. There hasn't always been a team, but as long as there has been, I've been doing it. Before there was a proper team, there was Soren and I, and before that there was just me. I set up the original Tarmac-based trunk gate that we used before Gerrit, and I'm the original owner on many of the Launchpad resources (because it turns out some human must be the ultimate owner of things there) So it's pretty easy to say without boasting that there is very little about the tooling that runs the project that I don't know about, both in its current form and the history that got us to its current form. My commits and reviews can be seen here: https://review.openstack.org/#/q/reviewer:mordred%2540inaugust.com+OR+owner:mordred%2540inaugust.com,n,z - Other Experience - I work at HP where I lead a team of people who staff the rest of the CI team. I do what I can as part of my job there to ensure that the OpenStack Dev systems always have the resources they need to operate effectively. Before HP I worked at Rackspace doing the same thing. Before OpenStack at Rackspace I was a core developer on Drizzle, having moved with the rest of the Drizzle team to Rackspace when Sun got bought by Oracle. I was lucky enough to get involved in Drizzle hacking from the beginning of that project while I was a Senior Consultant for MySQL Inc (and then for Sun when we got bought) Although it doesn't come up in this context very often, I'm pretty stinking good at MySQL Scaling and High Availability. I've been a Python hacker since I wrote the SNPP protocol library for Python in 2000, and have experience as both a developer and a *nix sysadmin stretching back to 1994. Recently I became a member of the Python Software Foundation, and I sit on the OpenStack Foundation Board. - Why I should be on the Technical Committee - The TC provides direction to the project as a whole and acts as a collaboration point and focus for consensus between the projects. Often times things that are decided by the TC have a direct effect on the automation systems that we use to run things ... so having someone from the CI and Automation teams represented seems like a really good idea. Additionally, since I tend to be cross-project focused rather than involved in any one specific project, I'd like to think that I have a decent unbiased perspective on issues that are related to project agreement and consistency. Thanks for your consideration! Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Bug 1030646] [NEW] Devstack vs. Horizon client versions
On 07/29/2012 07:47 PM, Monty Taylor wrote: Hey! This might be fairly tricky to fix properly given the design of the devstack gate. I could be wrong, it might not be terrible now that we can release new client lib versions to PyPI more quickly. The devstack gate explicitly tests proposed change to trunk of one project against tip of trunk across all the other projects. That's great for the dev purpose, and can solidify through sequencing during the milestone-proposed period, but if we have people with standalone horizon installations that are tracking trunk more closely, we might need to do some more thinking. Ok. jeblair and I chatted about this some last night in prep, and it might not be as tricky as I was thinking. In fact, we're already doing some testing that you could take advantage of to prevent the kind of breakage you're talking about, and a few things that are new enough that we haven't really taken advantage of them yet. Let me 'splain: The devstack gate works as I mentioned above, and will test you against trunk of other branches. That's important, because that's how we ensure that everything works together as we're developing on it. However, we also do unittest runs, which will test the project in question using the specific dependency versions the project declares. Those tests for horizon will use the released versions of all of the client libs, and would be the automated trap against patches to horizon that depend on changes that have not yet been released to pypi from client libs. Of course, horizon is tricky, being a django app, and unittests themselves don't always tell the full story. We have just in the last week added selenium test running (which you know, but for completeness I'm including here) Those tests also will run with released versions of libs from pypi. Also important to note here is that we have just recently gotten everything hooked up properly so that the client libs can release to pypi whenever they need to - so while we haven't started taking advantage of this fully yet, I think as people add functionality to the client libs, we may not have to wait as long for it to land. Anyhow - you can also totally just not accept patches to horizon that use things that aren't in released versions via code review - but if it's helpful, we are running a set of tests against horizon and the declared/released depends, so if you can figure out some sensible tests to add to unit or selenium tests that would trip this, they will definitely get run. We'll still be there at the CI meeting of course, but I wanted to set the stage a little bit so that we didn't have to cover all that in IRC. Could I request that you drop by the CI meeting on Tuesday so we can chat about it interactively? I think this is important and that we should do everything we can to get this right. Monty On 07/29/2012 06:34 PM, Gabriel Hurley wrote: Public bug reported: We patched this bug: https://bugs.launchpad.net/horizon/+bug/1027210 However, that fix worked for devstack and broke stand-alone installations of horizon because devstack pulled the clients from Github while Horizon's requirements file pulls them from PyPI. This is no good. Now anytime you try to list out images from glance (except when using devstack) you get this error: list() got an unexpected keyword argument 'page_size' We need to reconcile devstack with Horizon, and in the future Horizon will *only* accept changes that work with *released* client versions. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Bug 1030646] [NEW] Devstack vs. Horizon client versions
Hey! This might be fairly tricky to fix properly given the design of the devstack gate. I could be wrong, it might not be terrible now that we can release new client lib versions to PyPI more quickly. The devstack gate explicitly tests proposed change to trunk of one project against tip of trunk across all the other projects. That's great for the dev purpose, and can solidify through sequencing during the milestone-proposed period, but if we have people with standalone horizon installations that are tracking trunk more closely, we might need to do some more thinking. Could I request that you drop by the CI meeting on Tuesday so we can chat about it interactively? I think this is important and that we should do everything we can to get this right. Monty On 07/29/2012 06:34 PM, Gabriel Hurley wrote: Public bug reported: We patched this bug: https://bugs.launchpad.net/horizon/+bug/1027210 However, that fix worked for devstack and broke stand-alone installations of horizon because devstack pulled the clients from Github while Horizon's requirements file pulls them from PyPI. This is no good. Now anytime you try to list out images from glance (except when using devstack) you get this error: list() got an unexpected keyword argument 'page_size' We need to reconcile devstack with Horizon, and in the future Horizon will *only* accept changes that work with *released* client versions. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Openstack JAVA SDK
Hi! You probably want to check out jclouds as well. It's extremely mature, is in production all over the world and has support for openstack compute and storage. http://www.jclouds.org/ Monty On 07/18/2012 08:50 AM, Jyothsna Padavala wrote: Thanks Vincent for your reply. I lean towards using java cloud-files too since my primary work would be involving swift. How did you build the java cloud-files code? My existing Java application is done on Netbeans IDE and i'm bound to use that. Do we need any extra build tool like Ant / Maven etc? Thanks again, Jyothsna. - Original Message - From: Sheng Bo Hou sb...@cn.ibm.com To: Jyothsna Padavala jyothsna.padav...@strongauth.com Cc: openstack@lists.launchpad.net Sent: Wednesday, July 18, 2012 2:28:30 AM (GMT-0800) America/Los_Angeles Subject: Re: [Openstack] Openstack JAVA SDK Hi Jyothsna, I think they are different java codes for the client sides. I was using java cloud-files for accessing the swift. Open-java-sdk can be used as the client side of nova. And I used the source code directly. Best wishes. Vincent Hou (侯胜博) Software Engineer, Standards Growth Team, Emerging Technology Institute, IBM China Software Development Lab Tel: 86-10- 82450778 Fax: 86-10- 82453660 Notes ID: Sheng Bo Hou/China/IBM@IBMCN E-mail: sb...@cn.ibm.com Address:3F Ring, Building 28 Zhongguancun Software Park, 8 Dongbeiwang West Road, Haidian District, Beijing, P.R.C. 100193 Jyothsna Padavala jyothsna.padav...@strongauth.com Sent by: openstack-bounces+sbhou= cn.ibm@lists.launchpad.net 2012-07-18 06:09 Toopenstack openstack@lists.launchpad.net cc Subject [Openstack] Openstack JAVA SDK Hello all, I've the openstack (only keystone and swift)setup ready. Now, i want to talk to this setup from my java application. When i tried to google for java sdk for openstack, i got two options. 1) java cloud-files from rackspace ( https://github.com/rackspace/java-cloudfiles ) 2) Java SDK from ( https://github.com/woorea/openstack-java-sdk ) My question is which one is reliable and easy to use. How do i go about installing them? Thanks much, Jyothsna ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Release Upgrades (was [nova] [cinder] Nova-volume vs. Cinder in Folsom)
On 07/12/2012 04:36 PM, Vishvananda Ishaya wrote: On Jul 12, 2012, at 2:22 PM, Narayan Desai wrote: I think that the long term maintenance or removal of nova-volume in its existing form is orthogonal to the actual issue of continuity from one release to the next. Agreed. Discussion the volume/cinder strategy is a separate topic. I've taken the liberty of updating the subject to keep the discussions on point At this point, we've now run cactus, diablo and are in testing with essex. Each of these has effectively been a flag day for us; we build the new system, migrate users, images, etc, and let users do a bunch of manual migration of volume data, etc, while running both systems in parallel. This hasn't been as painful as it sounds because our understanding of best practices for running openstack is moving pretty quickly and each system has been quite different from the previous. Upgrading has been painful and we are striving to improve this process as much as possible. The lack of an effective process to move from one major release to the next is the major issue here in my mind. It would be fantastic if (some day, ha ha ha) you could apt-get upgrade from folsom to grizzly, but i think that is likely to be more trouble than it is worth. A reasonable compromise would be a well documented process as well as tools to aid in the process. Each real deployment will have a substantial set of local customizations, particularly if they are running at any sort of scale. While it won't be feasible to support any upgrade with these customizations, tools for the process (which may only be used a straw man in complex cases) would go a long way. I would like to take this a bit further. Documentation is a great first step, but I would actually like to have an actual Jenkins test that does the upgrade from essex to Folsom with live resources created. I think this the only way that we can be sure that the upgrade is working properly. ++ The first version of this doesn't even have to be on a full cluster. I'm thinking something as simple as: * configure devstack to checkout stable/essex from all of the projects * run the system, launch some instances and volumes * terminate the workers * upgrade all of the code to folsom * do any manual upgrade steps (nova-manage db sync, cinder migrate, etc.) * run all the workers * make sure the existing data still works and new api commands run The manual upgrade steps should be contained in a single script so that the distress can use it to help make their package upgrades and deployers can use it for reference when upgrading their clusters. Yes - especially if it's a self contained thing like devstack is currently. For the upgrade all the code to folsom step - let's chat about making sure that we get the right hooks/env vars in there so that we can make that upgrade to tip of trunk in most projects + proposed patch in one of them - same as we do for devstack runs today. This is something we can start working on today and we can run after each commit. Then we will immediately know if we do something that breaks upgradability, and we will have a testable documented process for upgrading. The creation of the self-contained script devstack was a HUGE step forward for us for integration testing. I think a similar thing for upgradability would similarly be huge. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [CHEF] Clarification on osops/chef-repo/roles/nova-compute.rb
On 07/10/2012 04:23 PM, Matt Ray wrote: Bluntness appreciated, this process is already in motion. http://opscode.com/openstack was launched 2 weeks ago and I promptly left for conferences and vacation. I am consolidating GitHub repos here: Awesome. https://github.com/opscode/openstack-chef-repo https://github.com/opscode-cookbooks/nova https://github.com/opscode-cookbooks/glance https://github.com/opscode-cookbooks/horizon https://github.com/opscode-cookbooks/keystone https://github.com/opscode-cookbooks/swift Work is being done in my own repos until it's ready for an initial release, which will include a Getting Started with Chef and OpenStack document. https://github.com/mattray/ I'm working with quite a few folks already, Rackspace, Dell, DreamHost and others and Intel is sponsoring this work. Jay and I chatted a bit in IRC, we're quite aligned in how we plan on working this and the goal will be to get github.com/openstack/chef-repo gated with Gerrit and CI and pulling from Opscode's repos soon. Only really replying because I saw the word gated. :) I'd love to be part of any conversations that are being had on this subject, sooner rather than later. Please feel free to join the discussion on our new mailing list focused on this effort here: http://groups.google.com/group/opscode-chef-openstack And an IRC channel: #openstack-chef on irc.freenode.net Thanks, Matt Ray Senior Technical Evangelist | Opscode Inc. m...@opscode.com | (512) 731-2218 Twitter, IRC, GitHub: mattray On Tue, Jul 10, 2012 at 11:25 AM, Jay Pipes jaypi...@gmail.com wrote: Apologies in advance for my blunt and somewhat dour response, Matt. I'm not singling you out at all, and I know you've tried your best to get the various Chef stakeholders to work together. Also apologies for top-posting, but there's not a whole lot of use inline posting this. tl;dr - We need to stop the needless fracturing of the operational knowledge of the Chef community and try working as a team towards some common goals instead of creating fork after fork of repos of Chef cookbooks. There is a ton of wasted effort in this area. Proposal: * Get our act together and treat Chef repos (and puppet/juju/whatever) as we do other OpenStack core and supporting projects -- use Gerrit, use a CI gating system, do real code reviews on it, and in general treat them as a supporting OpenStack project * Mark ALL Chef repos that are not currently maintained with an OBSELETE marker and/or DELETE the repo on Github * Consolidate all *cookbooks* into a repository in github.com/openstack/chef-repo * Use git submodules to manage cookbooks that are upstreamed in github.com/opscode/ that have little to no changes in them * Actually fix the documentation of the dang cookbooks -- right now, half of them include the documentation from the memcache cookbook, as they were lazily copy-pasted around, or the standard example doc that is created when using something like knife. * Put as much variation in deployment philosophy into *roles* and attribute overrides/defaults More/Rant/Details - Maybe it's just the open source developer in me, but I don't understand why there is such an aversion to coordination in the deployment/ops community around the scripts and deployment cookbooks/modules/charms/whatever. Is it that everyone has a different idea of what is best? Is it because deployers/ops folks think that coordinating with other contributors is too time-consuming? Is it because the chef repos are not controlled in the same way as, say, devstack or the core projects, with an automated patch queue manager and code review system that actually encourages debate over patches? A combination of all of the above? Over the last 2 years, I've worked at 3 companies in the OpenStack ecosystem. All three companies had their own repos of Chef cookbooks (still do to this day). 50-60% of the content of these cookbooks is identical. 10-20% of the content of these cookbooks is different -- but only slightly or cosmetically. And a good portion of the remaining 20-40% are differences that are incorrectly (IMHO) placed in the cookbooks and recipes instead of where they should be: in roles and environments, with cookbooks created that deal with variations in deployments with attributes and the occasional if/else block. In trying to determine the appropriate Chef repo to use for the TryStack project, we found the following repo: https://github.com/osops/chef-repo to have the most up-to-date. I've since been told this repo is no longer maintained. This is very frustrating, not because of this particular repo, but because this is just one in a long line of neglected and forgotten forks of chef cookbook repositories. The fact that the default Chef behaviour and Opscode documentation encourages the copy/pasting of cookbooks all over the place and GitHub itself encourages the random and
Re: [Openstack] Single global dependency list
On 07/03/2012 08:43 AM, Doug Hellmann wrote: On Jul 2, 2012, at 6:40 PM, Monty Taylor mord...@inaugust.com wrote: Hey all! One of the tasks from the last ODS was to implement a single global dependency list. Turns out the more you think about it, the more important it is... because of the way we use devstack as part of the gate, we actually _currently_ have a de facto global dependency list, it's just not declared anywhere. (oops) Anyway - the original thought was to put the depends in openstack-common. We'd use update.py to copy the depends in to the project, so that projects could align on their own timeframe. Additionally, we'd make the copy only copy in the versions from openstack-common for package that were already listed in the target project, so that we wouldn't add django to python-swiftclient, for instance. The mechanics of that all work and are ready - but then bcwaldon pointed out that it didn't make a ton of sense for them to go in openstack-common, since that has its own lifecycle and is a place for common code to go - not just a catch all place. To that end, I took the code we had written for the update logic and put it, along with the depends lists, into its own repo. I think we're ready to start actually moving forward with it - but we've run up against the hardest problem we every have: naming openstack-depends already got vetoed on IRC because it makes people think of adult diapers. I'm proposing openstack-requires, since the files we're talking about are actually python requirements files. Any objections? +0 on the name As an alternative, how about combining the requirements file with the other packaging related stuff from openstack-common and calling the result openstack-packaging? Interesting. Which other packaging stuff? Do you mean the stuff in openstack/common/setup.py? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Single global dependency list
On 07/03/2012 10:09 AM, Eric Windisch wrote: I have to agree with others that copying files around is not ideal, and I can see the maintenance of this getting more involved as Nova becomes more coupled with common. Additionally, we'd make the copy only copy in the versions from openstack-common for package that were already listed in the target project, so that we wouldn't add django to python-swiftclient, for instance. This seems to be a reasonable argument against using git submodules, but I'm afraid we might be losing more than we're gaining here. Just because python-swiftclient depends on openstack-common, and django-using code exists there, doesn't mean that django needs to be installed for python-swiftclient. We might do better to use git submodules and solve the dependency problem, than continuing down this copy-everything path. We're explicitly NOT doing a copy-everything path. That's the whole point. We're only copying the needed depends from the master list. git submodules actually make the problem worse, not better. Alternatively, speed up the movement from incubation to library. Yeah - that's kind of the reason that bcwaldon was saying this shouldn't be in openstack-common. openstack-common wants to be a library, and then we're back at not having an appropriate place for the master list. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Single global dependency list
It's a good and valid question and I don't really know. In this case, I'm merely the pack-horse who was told global synchronized dependencies lists! (not that I'm not the evil person cooking up schemes) That said - most patches from new contributors don't actually come with new library dependencies. If they are adding a new depend, I think it's reasonable to expect it to be slightly harder to get that landed. I do think that we need an answer to who approves changes to this list. Getting stuff merged to openstack-common is often hard because it's a smaller list of people who work on it. I'd hate to see this be only PTLs. However, things like let's upgrade webob seem to _actually_ need more eyes than it seems like at the time. meh. On 07/03/2012 03:12 PM, Gabriel Hurley wrote: So, I understand the rationales, and I think of those three options the one chosen is the most reasonable. I'm gonna just come out and say I hate this whole idea, but let's set this aside for a minute 'cuz I have a genuine question: What will the process be for merging changes to this requirements list? Having yet another roadblock to getting your contribution merged is a huge developer disincentive. We're really making it exceptionally hard for new contributors, and frustrating even for the old hands. So, with the goal of making the coordinated management of the projects possible, what can we do to respect developers? - Gabriel -Original Message- From: openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net [mailto:openstack- bounces+gabriel.hurley=nebula@lists.launchpad.net] On Behalf Of Monty Taylor Sent: Tuesday, July 03, 2012 7:54 AM To: Eric Windisch Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Single global dependency list On 07/03/2012 10:09 AM, Eric Windisch wrote: I have to agree with others that copying files around is not ideal, and I can see the maintenance of this getting more involved as Nova becomes more coupled with common. Additionally, we'd make the copy only copy in the versions from openstack-common for package that were already listed in the target project, so that we wouldn't add django to python-swiftclient, for instance. This seems to be a reasonable argument against using git submodules, but I'm afraid we might be losing more than we're gaining here. Just because python-swiftclient depends on openstack-common, and django-using code exists there, doesn't mean that django needs to be installed for python-swiftclient. We might do better to use git submodules and solve the dependency problem, than continuing down this copy-everything path. We're explicitly NOT doing a copy-everything path. That's the whole point.. We're only copying the needed depends from the master list. git submodules actually make the problem worse, not better. Alternatively, speed up the movement from incubation to library. Yeah - that's kind of the reason that bcwaldon was saying this shouldn't be in openstack-common. openstack-common wants to be a library, and then we're back at not having an appropriate place for the master list. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack G naming poll
tl;dr - Screw the rules, I agree Let's at least add it to the poll. Also - I think we should further amend the rules such that we select the NEXT release by the summit for the current release. That means two things: At the g summit, we'd tell everyone where the next summit is: At the g summit, we'd vote and announce the name of h We wouldn't have to spend half the cycle saying h, or whatever when we mean we're going to defer that crazy idea until next time I wouldn't have had to use the letter g by itself twice just above here. On 07/03/2012 06:50 PM, Brian Waldon wrote: TL;DR - Screw the rules, let's call the next release 'Grizzly' As California is rather lacking in the 'municipality names starting with a G that we should use for an OpenStack release' department, I have had to look *slightly* outside the ruleset to find a suitable 'G' release name - that name being 'Grizzly'. The rules clearly state that a release name must represent a city or county near the corresponding design summit and be comprised of a single word of ten characters or less - the problem here being that 'Grizzly' is actually 'Grizzly Flats.' Having already polled a small subset of the community, I feel like there would be enough support for 'Grizzly' to win if it were on the ballot. As I'm more interested in selecting a suitable name than accurately representing some arbitrary territory, I'd love to either permanently amend the rules to make this acceptable or grant an exception in this one case. As Thierry said, if this reaches critical mass, we will figure out what to do. Otherwise, I'll shut up and deal with '/Gazelle/'. Brian On Jul 3, 2012, at 3:20 PM, Thierry Carrez wrote: Yes, it's that time of the year again... time for us to choose the name of the next OpenStack release ! This time, cities and counties in California (San Diego, CA being the location of the G design summit) I set up a poll with the available options (based on our current rules of naming) at: https://launchpad.net/~openstack/+poll/g-release-naming Poll is accessible to all members of ~openstack group in Launchpad, and ends next Tuesday, 21:30 UTC. Please cast your vote! I'm aware that a subversive movement wants to try to amend the rules so that another name option becomes available. Since we can't stop (or modify) the poll now that it's been launched, if that movement reaches critical mass, we may organize a second round of polling :) -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] coding standards (was: review for implement dhcp agent for quantum)
On 07/03/2012 05:07 PM, Duncan McGreggor wrote: On Tue, Jul 3, 2012 at 5:39 PM, Dan Wendlandt d...@nicira.com wrote: Lately, Quantum reviewers have been doing their best to enforce python style guidelines above and beyond the programmatically enforced pep8 checks. This has happened for many recent reviews, so Mark isn't being singled out here, My objection isn't to Mark being singled-out -- my objection is to *anyone* engaging in this level of nit-pickery. This is death to projects. This is coming from a guy who's incredibly anal about his code and coding standards, too. I've been coding in Python for over a decade, adhering to PEP8 for a considerable period of that time, am a member of the notoriously picky Twisted project, and even I was surprised by the flood of review comments -- a high number of which contributed nothing to the improved readability, maintaiability, or functionality of this code under review. There were definitely some good points/comments. But there was a lot in there that you had to wade through the rest, before you saw them. I actually am going to need to side with Duncan here, although also I'm going to slightly disagree- but hopefully we're all used to that by now. Duncan is right - nitpickery can be quite deadly, but I think what's worse is when it's vague, not codified, and not checkable. With pep8, there is a clear document, and there is a tool that a dev can use to simply check his code. It's not like pylint, where it's literally impossible to write code which satisfies all of the warnings - it is completely possible to write code which is pep8 clean (as we all know, since we are all required to do so) But the best part about having a tool (other than my single-minded devotion to automated gating) isn't that we can use it to gate - it's that a dev can use it locally to verify things before sending them in for review... and that's great. The death cycle is really about the lag time. If you write some stuff, then run pep8 - or even nova's hacking.py - and it tells you things like Hey Duncan, I don't like it when you write methods that have the word is in the name - you may think it's ridiculous, but the feedback cycle is quick and deterministic and it's not nearly as frustrating. I think this is why the extra pedanticness in nova has worked out ok without killing people. The rules are in HACKING and are clear, but they're also in tools/hacking.py - and we use them as part of the pep8 gate. Because the code is clean to begin with, they're not very onerous to deal with... they're also simple and deterministic enough, because someone had to code a flipping check for them. Once there is a predictable and quick feedback cycle that can be locally tested, a developer can train himself to write the code that way in the first place - and they also don't feel like they're being picked on. SO - I'd recommend a middle ground here - if you want to add additional strictness in style checking, do what nova did with hacking.py ... we'll happily add it to the gate if you like. However... just remember that we're not here to write python style guidelines, or to write python programs enforcing those guidelines (not even those of us on the CI team) ... so if you find yourself spending weeks on a new version of hacking.py, something has probably gone wrong. though admittedly there's a lot of code previously accepted to the codebase that wasn't held to such a high bar. This attention to style guidelines is generally a good thing, I *strongly* disagree. /Attention/ to style guidelines is a huge boon to open source projects. But /this/ attention seems beyond the pale, like a good idea was taken too far and the intent of the guidelines has been lost. though I understand that it can be frustrating, especially for new developers unfamiliar with the rules (I personally like garyk's comment about how he felt dealing with PEP-257, see: http://www.youtube.com/watch?v=lYU-SeVofHs) But that's just it: I'm *not* a new developer! I'm a seasoned Python hacker, PSF member, obsessive-compulsive neat-freak with code. etc., etc. I haven't ever seen this level of zealous syntax pursuit in any well-functioning open source project. As long as reviewer comments are inline with items covered in https://github.com/openstack/quantum/blob/master/HACKING.rst, I may have missed something, but a lot of the comments I saw did not reference something particular in the HACKING file, nor were many of these marked as CONSIDER ... then I consider them fair game for reviews. If they go beyond that, they should be generally be expressed as a CONSIDER. If we're unhappy with what is or is not enforced, I'm definitely unhappy with what is being enforced and how. But even more: if reviews devolve to this level of non-code minutiae, how long do you think you will have the hearts and minds of enthusiastic contributing coders? What about sponsoring
Re: [Openstack] OpenStack G naming poll
On 07/03/2012 07:29 PM, Brian Waldon wrote: On Jul 3, 2012, at 5:21 PM, Monty Taylor wrote: tl;dr - Screw the rules, I agree Let's at least add it to the poll. Also - I think we should further amend the rules such that we select the NEXT release by the summit for the current release. That means two things: At the g summit, we'd tell everyone where the next summit is: At the g summit, we'd vote and announce the name of h We wouldn't have to spend half the cycle saying h, or whatever when we mean we're going to defer that crazy idea until next time I wouldn't have had to use the letter g by itself twice just above here. Fantastic idea. I haven't been involved in choosing the next location, so I'm not sure how hard it would be to choose it that far in advance. Maybe somebody can comment on how doable this is? I actually think it's HARDER to not choose it that far in advance, because the longer you wait, the more places are booked already. Although it's not exactly the same - linux conf australia always announces the location of the next conference at the current conference - and those are yearly. Also, they have a rotating set of host teams... so basically a team from a town puts in a proposal to linux.au saying we'd like to host the next one, here's what we're going to do, here's where we're going to have it, blah blah blah - and then one of them gets selected, and then they're the poor sods that have to actually run the darned thing. So to stick my nose WAY in where it doesn't belong, once we have the foundation - what if we move to a model of having folks propose that they'd like to host the design summit? We can make a CFP-style deadline, and people can all pitch their location, and one can get chosen and announced by Thierry at a closing session or whatnot. That way if I wanted to get together with some other folks and say hey guys, I've got 10 rooms that NYU has donated, and I'll provide these facilities - then great, or if mercado libre wanted to say zomg - we totally going to bring you all to the Shearton WTC in Sao Paulo - or NTT was all dude, we've got a thousand rooms in the middle of Tokyo or Rackspace went I don't know if anybody noticed, but we bought a shopping mall a few years ago and it's got conference rooms ... the folks can sit in a room, look at the proposals, say things like wow, I really don't want to sit in the castle all week when I sit there all week anyway - but drinking with Monty in the Lower East Side at a bar that has freezer they'll lock you in with a bottle of vodka sounds like a great way to plan the Houston release (that's how-ston for all you Texans) -- it would almost be like distributed development with code review. Of course, if that flys I guess I'm going to have to figure out how to take everyone to Mehanata... Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Single global dependency list
On 07/03/2012 04:23 PM, Gabriel Hurley wrote: Agreed on all points, and I know you're not evil, Monty. ;-) (mostly) Dammit. I'm going to HAVE to try harder... see my other post about Bulgarian bars with freezers... You're totally right that this particular case won't stymie new contributors, but as we've seen for changes to common--and sometimes even to the client libraries or devstack--reviewers are in short supply and getting the change you need in one of the gate projects merged can often add days of impedance to otherwise fruitful work. It's bitten me plenty of times. Yes. It's super hard and frustrating- especially as someone who rarely makes patches to the projects themselves UNLESS it's something happening through openstack-common ... or something that just broke between devstack and our infrastructure. So the need for balance is critical. Being able to vet the impact of a change on every project consuming it is difficult for either automated systems or human reviewers, so we do our best. Perhaps the simplest answer for now is devising a reasonable set of automated gate tests for this os-requires module that humans can trust, and working to expand the circle of reviewers on these centralized projects that have the power to block everyone yet are so easy to ignore... TOTALLY agree - especially on expanding the circle of reviewers. All the best, - Gabriel -Original Message- From: Monty Taylor [mailto:mord...@inaugust.com] Sent: Tuesday, July 03, 2012 12:49 PM To: Gabriel Hurley Cc: Eric Windisch; openstack@lists.launchpad.net Subject: Re: [Openstack] Single global dependency list It's a good and valid question and I don't really know. In this case, I'm merely the pack-horse who was told global synchronized dependencies lists! (not that I'm not the evil person cooking up schemes) That said - most patches from new contributors don't actually come with new library dependencies. If they are adding a new depend, I think it's reasonable to expect it to be slightly harder to get that landed. I do think that we need an answer to who approves changes to this list. Getting stuff merged to openstack-common is often hard because it's a smaller list of people who work on it. I'd hate to see this be only PTLs. However, things like let's upgrade webob seem to _actually_ need more eyes than it seems like at the time. meh. On 07/03/2012 03:12 PM, Gabriel Hurley wrote: So, I understand the rationales, and I think of those three options the one chosen is the most reasonable. I'm gonna just come out and say I hate this whole idea, but let's set this aside for a minute 'cuz I have a genuine question: What will the process be for merging changes to this requirements list? Having yet another roadblock to getting your contribution merged is a huge developer disincentive. We're really making it exceptionally hard for new contributors, and frustrating even for the old hands. So, with the goal of making the coordinated management of the projects possible, what can we do to respect developers? - Gabriel -Original Message- From: openstack- bounces+gabriel.hurley=nebula@lists.launchpad.net [mailto:openstack- bounces+gabriel.hurley=nebula@lists.launchpad.net] On Behalf Of Monty Taylor Sent: Tuesday, July 03, 2012 7:54 AM To: Eric Windisch Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] Single global dependency list On 07/03/2012 10:09 AM, Eric Windisch wrote: I have to agree with others that copying files around is not ideal, and I can see the maintenance of this getting more involved as Nova becomes more coupled with common. Additionally, we'd make the copy only copy in the versions from openstack-common for package that were already listed in the target project, so that we wouldn't add django to python-swiftclient, for instance. This seems to be a reasonable argument against using git submodules, but I'm afraid we might be losing more than we're gaining here. Just because python-swiftclient depends on openstack-common, and django-using code exists there, doesn't mean that django needs to be installed for python-swiftclient. We might do better to use git submodules and solve the dependency problem, than continuing down this copy-everything path. We're explicitly NOT doing a copy-everything path. That's the whole point.. We're only copying the needed depends from the master list. git submodules actually make the problem worse, not better. Alternatively, speed up the movement from incubation to library. Yeah - that's kind of the reason that bcwaldon was saying this shouldn't be in openstack-common. openstack-common wants to be a library, and then we're back at not having an appropriate place for the master list. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack
Re: [Openstack] best practices for merging common into specific projects
Interestingly enough - gerrit supports submodules pretty well... and it does exactly what Eric said below ... if both the project and superproject are in gerrit, and a change is made to the project, gerrit can automatically update the superproject reference. Here's the thing though (and one of the reasons we haven't actually earnestly suggested using this for anything yet) ... testing If you propose a change to openstack-common and we were going to use the auto-update feature, we'd need to test that that change doesn't break ALL of the projects that use it. Now, what if nova is going to need a patch to use the new version of the code. Oops. Complexity. Overload. Everyone dies. However, with a versioned library model, the projects can consume things pinned to specific versions, and then they can submit a change that updates the version depend and the code which needs to be updated to support the version change, and that change can be atomic. So honestly, I'd say the real key is getting us closer to the point where openstack-common is a proper library, because all of the rest of the complexity is stuff we're inventing to make life harder on ourselves, when the standard library with api-contract and a version model of the world works pretty fine without needing coordinated changes across multiple repositories. On 07/03/2012 06:54 PM, Gabriel Hurley wrote: I’m pretty -1 on triggering changes in other projects from common. That’s gonna result in a broken code (whether subtle or obvious) no matter how good your gates are. At least as an external library you can freeze a version requirement until such time as you see fit to properly updated that code and **ensure** compatibility in your project. Or if your project likes ridin’ trunk, then don’t pin a version and you’ve got the same effect as an automatic trigger. - Gabriel *From:*openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net [mailto:openstack-bounces+gabriel.hurley=nebula@lists.launchpad.net] *On Behalf Of *Andrew Bogott *Sent:* Tuesday, July 03, 2012 3:54 PM *To:* Eric Windisch; openstack@lists.launchpad.net *Subject:* Re: [Openstack] best practices for merging common into specific projects On 7/3/12 5:47 PM, Eric Windisch wrote: git submodules don't have to be linked to the head of a branch. Instead of double-commiting (for every commit), we can do a single commit in each project to change the git reference of the submodule. This would not be too far from the existing behavior, except that it would minimize the double commits. Oh, I guess I left out an important part of my vision, which is that there would be a commit hook in common which moves the submodule reference in the parent projects anytime a patch is merged in common. So, in short: once a patch passed review for inclusion in common, that patch would automatically go live in all other project heads simultaneously. -- Eric Windisch On Tuesday, July 3, 2012 at 15:47 PM, Andrew Bogott wrote: On 7/3/12 1:59 PM, Gabriel Hurley wrote: The notion that copying code is any protection against APIs that may change is a red herring. It's the exact same effect as pegging a version of a dependency (whether it's a commit hash or a real version number), except now you have code duplication. An unstable upgrade path is an unstable upgrade path, and copying the code into the project doesn't alleviate the pain for the project if the upstream library decides to change its APIs. Also, we're really calling something used by more or less all the core projects incubated? ;-) Seems like it's past the proof-of-concept phase now, at least for many parts of common. Questions of API stability are an issue unto themselves. Anyhow, I'm +1 on turning it into a real library of its own, as a couple people suggested already. - Gabriel I feel like I should speak up since I started this fight in the first place :) Like most people in this thread, I too long for an end to the weird double-commit process that we're using now. So I'm happy to set aside my original Best Practices proposal until there's some consensus regarding how much longer we're going to use that process. Presumably opinions about how to handle merge-from-common commits will vary in the meantime, but that's something we can live with. In terms of promoting common into a real project, though, I want to raise another option that's guaranteed to be unpopular: We make openstack-common a git-submodule that is automatically checked out within the directory tree of each other project. Then each commit to common would need to be gated by the full set of tests on every
Re: [Openstack] OpenStack G naming poll
On 07/03/2012 07:33 PM, James E. Blair wrote: Brian Waldon brian.wal...@rackspace.com writes: TL;DR - Screw the rules, let's call the next release 'Grizzly' As California is rather lacking in the 'municipality names starting with a G that we should use for an OpenStack release' department, I have had to look *slightly* outside the ruleset to find a suitable 'G' release name - that name being 'Grizzly'. The rules clearly state that a release name must represent a city or county near the corresponding design summit and be comprised of a single word of ten characters or less - the problem here being that 'Grizzly' is actually 'Grizzly Flats.' Having already polled a small subset of the community, I feel like there would be enough support for 'Grizzly' to win if it were on the ballot. As I'm more interested in selecting a suitable name than accurately representing some arbitrary territory, I'd love to either permanently amend the rules to make this acceptable or grant an exception in this one case. As Thierry said, if this reaches critical mass, we will figure out what to do. Otherwise, I'll shut up and deal with 'Gazelle'. I will join your Bear Flag Revolt. Best. Picture. Ever. We have a G winner... and honestly, if that's not the t-shirt then I'm going to have my own bear flag revolt. Grizzly FTW We could amend the rules to add official symbols of the territory in question. Despite being one of the most recognized symbols of California, named the state animal, and appearing on the state flag, the Grizzly bear (Ursus californicus) has been extinct here since 1922. [1] http://en.wikipedia.org/wiki/California_Republic#Bear_Flag_Revolt -Jim Though as a Firesign Theatre fan, I like Goshen too. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] PEP8 checks
On 07/02/2012 06:46 AM, John Garbutt wrote: Hi, I noticed I can now run the pep8 tests like this (taken from Jenkins job): tox -v -epep8 ... pep8: commands succeeded congratulations :) But the old way to run tests seems to fail: ./run-tests.sh -p ... File /home/johngar/openstack/nova/.venv/local/lib/python2.7/site-packages/pep8.py, line 1220, in check_logical for result in self.run_check(check, argument_names): TypeError: 'NoneType' object is not iterable Is this expected? Did I just miss an email about this change? It's not really expected, and I honestly don't understand why run_tests.sh -p would have problems running pep8. Although we do not use run_tests.sh for anything in jenkins, we have not done anything to disable or change what it's doing. I'm looking at it right now, lemme see what's up. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] RFC: Thoughts on improving OpenStack GIT commit practice/history
On 07/02/2012 05:14 AM, Daniel P. Berrange wrote: I must say that this has been driving me mad this last week. IIUC, only members of the core review team have permission to retrigger Jenkins, but I feel it is putting too much burden on them to have to track every patch with a bogus Jenkins failure. If we can't get Jenkins to be more reliable, then can we see about letting patch submitters retrigger Jenkins builds for their own patches. There's a whole other thread about this at the moment, which outlines the problems and what we're doing about it. One of the main issues should be already solved. The primary goal is to get jenkins to be more reliable so that hiccups don't happen in the first place. In any case, if you want all the details, please see James Blair's recent email to the Jenkins and Transient Failures thread. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] Issues with run_tests.sh, no tests are run when import libvirt is present
On 07/02/2012 06:02 AM, Leander Bessa Beernaert wrote:
Thanks, that let me see the real problem now:
./tools/with_venv.sh nosetests -svx nova
nose.config: INFO: Set working dir to /home/gsd/nova/nova/tests
nose.config: INFO: Working directory /home/gsd/nova/nova/tests is a
package; adding to sys.path
nose.config: INFO: Ignoring files matching ['^\\.', '^_',
'^setup\\.py$']
nose.plugins.cover: INFO: Coverage report will include only
packages: ['nova']
nose.selector: INFO: /home/gsd/nova/nova/auth/opendj.sh is
executable; skipped
nose.selector: INFO: /home/gsd/nova/nova/auth/slap.sh is executable;
skipped
nose.selector: INFO:
/home/gsd/nova/nova/cloudpipe/bootscript.template is executable; skipped
Failure: ImportError (No module named libvirt) ... ERROR
==
ERROR: Failure: ImportError (No module named libvirt)
--
Traceback (most recent call last):
File
/home/gsd/nova/.venv/local/lib/python2.7/site-packages/nose/loader.py,
line 390, in loadTestsFromName
addr.filename, addr.module)
File
/home/gsd/nova/.venv/local/lib/python2.7/site-packages/nose/importer.py,
line 39, in importFromPath
return self.importFromDir(dir_path, fqname)
File
/home/gsd/nova/.venv/local/lib/python2.7/site-packages/nose/importer.py,
line 86, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File /home/gsd/nova/nova/test.py, line 41, in module
from nova.tests import fake_flags
File /home/gsd/nova/nova/tests/fake_flags.py, line 24, in module
flags.DECLARE('compute_scheduler_driver', 'nova.scheduler.multi')
File /home/gsd/nova/nova/flags.py, line 52, in DECLARE
__import__(module_string, globals(), locals())
File /home/gsd/nova/nova/scheduler/multi.py, line 27, in module
from nova.scheduler import driver
File /home/gsd/nova/nova/scheduler/driver.py, line 53, in module
flags..DECLARE('libvirt_type', 'nova.virt.libvirt.connection')
File /home/gsd/nova/nova/flags.py, line 52, in DECLARE
__import__(module_string, globals(), locals())
File /home/gsd/nova/nova/virt/libvirt/connection.py, line 73, in
module
from nova.virt.libvirt import diagnostics as libvirt_diagnostics
File /home/gsd/nova/nova/virt/libvirt/diagnostics.py, line 75,
in module
import libvirt as virt
ImportError: No module named libvirt
--
Ran 1 test in 0.002s
FAILED (errors=1)
Libvirt is present on my system, i can import it through the python
interpreter from any location. Yet, it still says it is not present.
Could this have to do with the virtual_env or fakelibvirt?
Yes. Our virtualenvs are currently configured to not allow system
packages to be used (so that they only use python libs installed into
the virtualenv. However, libvirt is not installable via pip, which is a
problem.
We've recently added a new tox environment to help with this, so try
running:
tox -v -efull
Which is set up to allow system site packages.
However, if you're adding unittests, you should look at the other ones
which test for libvirt existence and skip the tests if they can't find it.
On Fri, Jun 29, 2012 at 6:54 PM, Jay Pipes jaypi...@gmail.com
mailto:jaypi...@gmail.com wrote:
Hi Leander,
I've noticed some weirdness with the openstack.nose_plugin (which is
used by default for the Nova test runner) sometimes either throwing
errors (particularly errors raised by nosetests) away and/or coming
up with a different set of skip tests than when running just with
nosetests.
So, I'd recommend running just this:
./tools/with_venv.sh nosetests -svx nova
That will stop at the first error and probably will give you better
insight into the actual errors that are likely being suppressed by
the openstack nose plugin.
best,
-jay
On 06/29/2012 09:02 AM, Leander Bessa Beernaert wrote:
Hello,
I'm sorry to restart the topic
(https://lists.launchpad.net/__openstack/msg13621.html
https://lists.launchpad.net/openstack/msg13621.html), but
i accidentally deleted the message in my inbox :S.
I'm still having the same problem, each time i add import
libvirt to
the file diangostics.py
(https://review.openstack.org/__#/c/8839/
https://review.openstack.org/#/c/8839/) the
entire test suit won't run.
*With the import* present i get the following output:
--__--__--
Ran 0 tests
Re: [Openstack] PEP8 checks
On 07/02/2012 06:46 AM, John Garbutt wrote: Hi, I noticed I can now run the pep8 tests like this (taken from Jenkins job): tox -v -epep8 ... pep8: commands succeeded congratulations :) But the old way to run tests seems to fail: ./run-tests.sh -p ... File /home/johngar/openstack/nova/.venv/local/lib/python2.7/site-packages/pep8.py, line 1220, in check_logical for result in self.run_check(check, argument_names): TypeError: 'NoneType' object is not iterable Is this expected? Did I just miss an email about this change? I cannot reproduce this on my system. Can you run bash -x run_tests.sh -p and pastebin the output? Also, tools/with_venv.sh pep8 --version just to be sure. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] Issues with run_tests.sh, no tests are run when import libvirt is present
On 07/02/2012 08:43 AM, Leander Bessa Beernaert wrote:
So, if no system packages can be imported, how do you test the
connection class for the libvirt driver?
We're working on that - but as I said, please try running tox -efull
which _should_ run tests with libvirt support enabled.
How does that particular test case wrap around the fact that it requires
the libvirt module? The only thing i could find are these lines of code
in the driver's __init__ method. Do these somehow detect if this is a
unit test environment and import the fakelibvirt driver instead? I'm no
expert in python so i'm not sure what's happening there :s
global libvirt
if libvirt is None:
libvirt = __import__('libvirt')
Regards,
Leander
On Mon, Jul 2, 2012 at 1:27 PM, Monty Taylor mord...@inaugust.com
mailto:mord...@inaugust.com wrote:
On 07/02/2012 06:02 AM, Leander Bessa Beernaert wrote:
Thanks, that let me see the real problem now:
./tools/with_venv.sh nosetests -svx nova
nose.config: INFO: Set working dir to /home/gsd/nova/nova/tests
nose.config: INFO: Working directory /home/gsd/nova/nova/tests
is a
package; adding to sys.path
nose.config: INFO: Ignoring files matching ['^\\.', '^_',
'^setup\\.py$']
nose.plugins.cover: INFO: Coverage report will include only
packages: ['nova']
nose.selector: INFO: /home/gsd/nova/nova/auth/opendj.sh is
executable; skipped
nose.selector: INFO: /home/gsd/nova/nova/auth/slap.sh is
executable;
skipped
nose.selector: INFO:
/home/gsd/nova/nova/cloudpipe/bootscript.template is
executable; skipped
Failure: ImportError (No module named libvirt) ... ERROR
==
ERROR: Failure: ImportError (No module named libvirt)
--
Traceback (most recent call last):
File
/home/gsd/nova/.venv/local/lib/python2.7/site-packages/nose/loader.py,
line 390, in loadTestsFromName
addr.filename, addr.module)
File
/home/gsd/nova/.venv/local/lib/python2.7/site-packages/nose/importer.py,
line 39, in importFromPath
return self.importFromDir(dir_path, fqname)
File
/home/gsd/nova/.venv/local/lib/python2.7/site-packages/nose/importer.py,
line 86, in importFromDir
mod = load_module(part_fqname, fh, filename, desc)
File /home/gsd/nova/nova/test.py, line 41, in module
from nova.tests import fake_flags
File /home/gsd/nova/nova/tests/fake_flags.py, line 24, in
module
flags.DECLARE('compute_scheduler_driver',
'nova.scheduler.multi')
File /home/gsd/nova/nova/flags.py, line 52, in DECLARE
__import__(module_string, globals(), locals())
File /home/gsd/nova/nova/scheduler/multi.py, line 27, in
module
from nova.scheduler import driver
File /home/gsd/nova/nova/scheduler/driver.py, line 53, in
module
flags..DECLARE('libvirt_type', 'nova.virt.libvirt.connection')
File /home/gsd/nova/nova/flags.py, line 52, in DECLARE
__import__(module_string, globals(), locals())
File /home/gsd/nova/nova/virt/libvirt/connection.py, line
73, in
module
from nova.virt.libvirt import diagnostics as
libvirt_diagnostics
File /home/gsd/nova/nova/virt/libvirt/diagnostics.py, line 75,
in module
import libvirt as virt
ImportError: No module named libvirt
--
Ran 1 test in 0.002s
FAILED (errors=1)
Libvirt is present on my system, i can import it through the python
interpreter from any location. Yet, it still says it is not present.
Could this have to do with the virtual_env or fakelibvirt?
Yes. Our virtualenvs are currently configured to not allow system
packages to be used (so that they only use python libs installed into
the virtualenv. However, libvirt is not installable via pip, which is a
problem.
We've recently added a new tox environment to help with this, so try
running:
tox -v -efull
Which is set up to allow system site packages.
However, if you're adding unittests, you should look at the other ones
which test for libvirt existence and skip the tests if they can't
find it.
On Fri, Jun 29, 2012 at 6:54 PM, Jay Pipes jaypi...@gmail.com
mailto:jaypi...@gmail.com
mailto:jaypi...@gmail.com mailto:jaypi...@gmail.com wrote:
Hi Leander
Re: [Openstack] PEP8 checks
Awesome. On 07/02/2012 08:46 AM, John Garbutt wrote: I had pep8 v1.2 in my virtual env from this: https://github.com/openstack/nova/commit/2fa2cd2254a4044aaa584c4fcf5d6c3e1ec60d1f#tools/test-requires Rebased with trunk and re-creating my virtualenv (i.e. move back to pep8 v1..1) seemed to fix things. Thanks for your help, John -Original Message- From: Monty Taylor [mailto:mord...@inaugust.com] Sent: 02 July 2012 1:28 To: John Garbutt Cc: openstack@lists.launchpad.net Subject: Re: [Openstack] PEP8 checks On 07/02/2012 06:46 AM, John Garbutt wrote: Hi, I noticed I can now run the pep8 tests like this (taken from Jenkins job): tox -v -epep8 ... pep8: commands succeeded congratulations :) But the old way to run tests seems to fail: ./run-tests.sh -p ... File /home/johngar/openstack/nova/.venv/local/lib/python2.7/site- packages/pep8.py, line 1220, in check_logical for result in self.run_check(check, argument_names): TypeError: 'NoneType' object is not iterable Is this expected? Did I just miss an email about this change? I cannot reproduce this on my system. Can you run bash -x run_tests.sh -p and pastebin the output? Also, tools/with_venv.sh pep8 --version just to be sure. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] Issues with run_tests.sh, no tests are run when import libvirt is present
Run:
sudo pip install tox
And you will get the tox command.
Does ./run_tests.sh -N nova.tests.test_libvirt work fine when you
_don't_ have libvirt? It needs to skip the test if you don't have
libvirt installed, and it needs to run it and pass if you do.
Jenkins is going to run tox -v -epy27 and then eventually also tox -v
-efull - so make sure both of those work and you'll be set.
Monty
On 07/02/2012 10:30 AM, Leander Bessa Beernaert wrote:
Running with ./run_tests.sh -N nova.tests.test_libvirt works just
fine, however i don't know if this is enough to get it past jenkins :/
On Mon, Jul 2, 2012 at 3:26 PM, Daniel P. Berrange berra...@redhat.com
mailto:berra...@redhat.com wrote:
On Mon, Jul 02, 2012 at 01:43:31PM +0100, Leander Bessa Beernaert wrote:
So, if no system packages can be imported, how do you test the
connection
class for the libvirt driver?
How does that particular test case wrap around the fact that it
requires
the libvirt module? The only thing i could find are these lines of
code in
the driver's __init__ method. Do these somehow detect if this is a
unit
test environment and import the fakelibvirt driver instead? I'm no
expert
in python so i'm not sure what's happening there :s
global libvirt
if libvirt is None:
libvirt = __import__('libvirt')
If you have installed all the neccessary python packages on your
local host, then it is entirely possible to run the Nova test
suites without using virtualenv. You just need to pass the '-N'
arg to the run_tests.sh script, eg on my Fedora 17 host, I can
run
./run_tests.sh -N nova.tests.test_libvirt
Regards,
Daniel
--
|: http://berrange.com -o-
http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org -o-
http://virt-manager.org :|
|: http://autobuild.org -o-
http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org -o-
http://live.gnome.org/gtk-vnc :|
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
[Openstack] Single global dependency list
Hey all! One of the tasks from the last ODS was to implement a single global dependency list. Turns out the more you think about it, the more important it is... because of the way we use devstack as part of the gate, we actually _currently_ have a de facto global dependency list, it's just not declared anywhere. (oops) Anyway - the original thought was to put the depends in openstack-common. We'd use update.py to copy the depends in to the project, so that projects could align on their own timeframe. Additionally, we'd make the copy only copy in the versions from openstack-common for package that were already listed in the target project, so that we wouldn't add django to python-swiftclient, for instance. The mechanics of that all work and are ready - but then bcwaldon pointed out that it didn't make a ton of sense for them to go in openstack-common, since that has its own lifecycle and is a place for common code to go - not just a catch all place. To that end, I took the code we had written for the update logic and put it, along with the depends lists, into its own repo. I think we're ready to start actually moving forward with it - but we've run up against the hardest problem we every have: naming openstack-depends already got vetoed on IRC because it makes people think of adult diapers. I'm proposing openstack-requires, since the files we're talking about are actually python requirements files. Any objections? We've also been discussing an idea that we could write some gating tests that are only run against milestone-proposed branches that would verify that the requires files in a given project match the versions in openstack-requires - that way there is some lee-way throughout the cycle, but the expectation is that by release time the requires files will be brought in line with the rest of the project. (it's an option if people find that useful - certainly not required) Finally, as an added bonus to this approach, once we have the list and we're even mostly aligned on it, since a library version would need to be in openstack-requires before it hits a project, we can use it as the main basis for our pypi mirror and turn off the upstream pypi source from our jenkins slaves. This would remove the last of the ephemeral build errors that are due to upstream network timeouts. I'm sure everyone will be pleased about that. Anyway - I think general buy in on at _least_ the name openstack-requires will let us move forward with the next step. After that point, it'll all be normal code reviews. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] new locations for some things
Hey all! (It must be a busy day - I'm writing you all so many emails...) A little while ago, after chatting with Anne Gentle, we started publishing the sphinx documentation to docs.openstack.org/developer/$project ... instead of to $project.openstack.org. That went really well and we're happy about it ... but we didn't put in any redirects from the old locations because we still had tarballs to content with (which we uploaded to $project.openstack.org/tarballs) That's fixed now - we now have tarballs going to tarballs.openstack.org/$project, docs going to docs.openstack.org/developer/$project, and redirects from the old locations to the new locations. While doing this, we added a couple of features. First of all - client libs get their own dir, so there should be no confusion there. Secondly, in addition to the normal per-commit tarballs, we're now publishing tarballs of the form $project-$branch.tar.gz which will get overwritten with each commit - that way, if you need to track trunk from a pip-requires file, (such as ceilometer, which needs to track nova trunk) you can simply plop in something like http://tarballs.openstack.org/nova/nova-master.tar.gz - and it'll work for both pip installs AND easy_install/distutils based installs. Yay! Finally, we've got code landing that will properly publish documentation for tagged revisions to a subdir ... so when nova gets tagged 2012.2 - there will be a docs.openstack.org/developer/nova/2012.2 dir with those docs in it and they will not change over time. Just wanted to let everyone know. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] need help about jenkins
Hi! We were doing some maintenance this afternoon on jenkins and gerrit - you were unlucky enough to push right in the middle of that. Try amending your commit (git commit --amend) and change the commit message a little (put a space after the comma here: lp:1019348,update and then run git review again. This will cause a new changeset to be uploaded and will re-trigger tests. Monty On 07/01/2012 01:30 PM, heut2008 wrote: Hi,all recently,I made a commit push to gerrit ,see https://review.openstack.org/#/c/9204/. jenkins always show failed ,how can I figure out what's wrong with my commit,or some thing wrong with jenkins? thanks in advance:) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] RFC: Thoughts on improving OpenStack GIT commit practice/history
Hey! I agree with most of this in general. A few comments about a section I just happened to read: On 06/27/2012 06:52 AM, Daniel P. Berrange wrote: snip Including external references - The commit message is primarily targetted towards human interpretation, but there is always some metadata provided for machine use. In the case of OpenStack this includes at least the 'Change-id', but also optional bug ID references and blueprint name references. Although GIT records the author committer of a patch, it is common practice across many open source projects to include a Signed-off-by tag. Though OpenStack does not mandate its use, the latter is still useful to include if a patch is a combination of work by many different developers, since GIT only records a single author. All machine targetted metadata, however, is of secondary consequence to humans and thus it should all be grouped together at the end of the commit message. For example: Switch libvirt get_cpu_info method over to use config APIs The get_cpu_info method in the libvirt driver currently uses XPath queries to extract information from the capabilities XML document. Switch this over to use the new config class LibvirtConfigCaps. Also provide a test case to validate the data being returned Fixes: bug #1003373 Implements: blueprint libvirt-xml-cpu-model These two are fine in general - although we actually parse and do bug and blueprint linking a bit more permissively. I believe our current regexes should still apply to those though. Change-Id: I4946a16d27f712ae2adf8441ce78e6c0bb0bb657 Signed-off-by: Daniel P. Berrange berra...@redhat.com As well as the 'Signed-off-by' tag, there are various other ad-hoc tags that can be used to credit other people involved in a patch who aren't the author. - 'Reviewed-by: ...some name.. ...email...' Although Gerrit tracks formal review by project members, some patches have been reviewed by people outside the community prior to submission I'm not in support of this one. Our review system has no restrictions on who can review things - and I value the actual content of their reviews, which itself is stored in git notes along with the change. So, in general, you can tell me that someone else reviewed it somewhere else, but I don't really care. They should review it in the same place everyone else does. - 'Suggested-by: ...some name.. ...email...' If a person other than the patch author suggested the code enhancement / influnced the design Great idea. - 'Reported-by: ...some name.. ...email...' If a person reported the bug / problem being fixed but did not otherwise file a launchpad bug report. I think this should be strongly discouraged. If there is a bug or problem, it should be filed as a bug report. I don't really know of a valid use case where someone can't do this, but where referencing them here is valid. We have systems, and just as you're suggesting overall that we be a bit stricter in how we do things, I think that being more lax in how we record some of this metadata is counter productive. However - strongly in favor of better commit message practice! Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] setuptools-git
On 06/29/2012 04:06 AM, Alan Pevec wrote: On Thu, Jun 28, 2012 at 9:56 PM, Monty Taylor mord...@inaugust.com wrote: https://review.openstack.org/9109 Why is setuptools_git added in pip-requires, I thought that's for run-time, not build-time dependencies? Hey Alan! We use pip-requires as part of building the virtualenvs. Once we start using it, setuptools-git is pretty much required for running setup.py, so many common actions in our workflow will require it. It's a good question though - and I consider the current existence of it in pip-requires purely a convenience hack. We don't have a strong place in our infrastructure to indicate setup_requires entries. I'll work on that next. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] setuptools-git
On 06/29/2012 10:30 AM, Joshua Harlow wrote: Should there be a separation of build-time setup.py and run-time setup.py?? I’m not sure if something like that is possible (maybe with a setuptools variant, distribute or something similar??) Well, we use distribute already - but the problem isn't really build-time vs. run-time as much as it has to do with our use of virtualenv. HOWEVER - I think I just had an idea of how to make this slighly cleaner. Let me poke at it. On 6/29/12 4:06 AM, Alan Pevec ape...@gmail.com wrote: On Thu, Jun 28, 2012 at 9:56 PM, Monty Taylor mord...@inaugust.com wrote: https://review.openstack..org/9109 https://review.openstack.org/9109 Why is setuptools_git added in pip-requires, I thought that's for run-time, not build-time dependencies? Cheers, Alan ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] setuptools-git
On 06/29/2012 10:48 AM, Alan Pevec wrote: On Fri, Jun 29, 2012 at 7:19 PM, Monty Taylor mord...@inaugust.com wrote: We use pip-requires as part of building the virtualenvs. Once we start using it, setuptools-git is pretty much required for running setup.py, so many common actions in our workflow will require it. It's a good question though - and I consider the current existence of it in pip-requires purely a convenience hack. We don't have a strong place in our infrastructure to indicate setup_requires entries. I'll work on that next. test-requires is also used when building venv and until there are separate build-requires, seems to be a better place for deps such as this Trouble is that openstack.common.setup.parse_requirements() parses pip-requires and puts everything there into egg-info, which should be only run-time deps. Yeah - good call. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] setuptools-git
On 06/29/2012 10:48 AM, Alan Pevec wrote: On Fri, Jun 29, 2012 at 7:19 PM, Monty Taylor mord...@inaugust.com wrote: We use pip-requires as part of building the virtualenvs. Once we start using it, setuptools-git is pretty much required for running setup.py, so many common actions in our workflow will require it. It's a good question though - and I consider the current existence of it in pip-requires purely a convenience hack. We don't have a strong place in our infrastructure to indicate setup_requires entries. I'll work on that next. test-requires is also used when building venv and until there are separate build-requires, seems to be a better place for deps such as this Trouble is that openstack.common.setup.parse_requirements() parses pip-requires and puts everything there into egg-info, which should be only run-time deps. I just put up another possibility for review before I read this - check it out and see what you think - we can also go test-requires if you like that better. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Jenkins vs SmokeStack tests Gerrit merge blockers
On 06/28/2012 07:32 AM, Daniel P. Berrange wrote: Today we face a situation where Nova GIT master fails to pass all the libvirt test cases. This regression was accidentally introduced by the following changeset https://review.openstack.org/#/c/8778/ If you look at the history of that, the first SmokeStack test run fails with some (presumably) transient errors, and added negative karma to the change against patchset 2. If it were not for this transient failure, it should have shown the regression in the libvirt test case. The libvirt test case in question was one that is skipped, unless libvirt is actually present on the host running the tests. SmokeStack had made sure the tests would run on such a host. There were then further patchsets uploaded, and patchset 4 was approved for merge. Jenkins ran its gate jobs and these all passed successfully. I am told that Jenkins will actually run the unittests that are included in Nova, so I would have expected it to see the flawed libvirt test case, but it didn't. I presume therefore, that Jenkins is not running on a libvirt enabled host. Kind of - it's sadly more complex than that ... The end result was that the broken changeset was merged to master, which in turns means any other developers submitting changes touching the libvirt area will get broken tests reported that were not actually their own fault. This leaves me with the following questions... 1. Why was the recorded failure from SmokeStack not considered to be a blocker for the merge of the commit by Gerrit or Jenkins or any of the reviewers ? 2. Why did SmokeStack not get re-triggered for the later patch set revisions, before it was merged ? The answer to 1 and 2 is largely the same - SmokeStack is a community contributed resources and is not managed by the CI team. Dan Prince does a great job with it, but it's not a resource that we have the ability to fix should it start messing up, so we have not granted it the permissions to file blocking votes. The tests that smokestack runs could all be written such that they are run by jenkins. The repos that run the jenkins tests are all in git and managed by openstack's gerrit. If there are testing profiles that it runs that we as a community value and want to see part of the gate, anyone is welcome to port them. 3. Why did Jenkins not ensure that the tests were run on a libvirt enabled host ? This is a different, and slightly more complex. We run tests in virtualenvs so that the process used to test the code can be consistently duplicated by all of the developers in the project. This is the reason that we no longer do ubuntu package creation as part of the gate - turns out that's really hard for a developer running on OSX to do locally on their laptop - and if Jenkins reports an blocking error in a patch, we want a developer to be able to reproduce the problem locally so that they can have a chance at fixing it. Problem arise in paradise though. libvirt being one of them. It's not possible to install libvirt into a virtualenv, because it's a swig-based module built as part of the libvirt source itself. One of the solutions to this is to allow the testing virtual environments to use packages installed at the system level. We suggested this a little while ago, but this was rejected by the nova team who valued the benefit of having a restricted test run so that we know we've got all of the depends properly specified. To that end, after chatting with Brian Waldon, I put this up as a possible next try: https://review.openstack.org/#/c/8949/ Which adds an additional testing environment that has system software enabled and also installs additional optional things. With that environment, we should be able to run a jenkins gate that tests things with full libvirt, and also tests the mysql upgrade paths, without screwing our fine friends who run OSX. Fundamentally though - we're at a point of trying to have our cake and eat it too. Either we want comprehensive testing of all of the unit tests, or we want to be careful about not making the test environment to hard for a developer to exactly mimic. I'm obviously on the side of having us have gating tests that some devs might not be able to do on their laptops - such as running the libvirt tests properly. We're working on cloud software - worst case scenario if there's an intractable problem, as dev can always spin up an ubuntu image somewhere. Obviously this was all made worse by the transient problems we've had with the tests suite infrastructure these past 2 days, but regardless it seems like we have a gap in our merge approval procedures here. IMHO, either SmokeStack needs to be made compulsory, or Jenkins needs to ensure tests are run on suitable hosts like SmokeStack does, or both. The second is much more possible and as I've pointed out is in work - but I do think we should develop a clear sense that it's important to us that we
Re: [Openstack] Adding docs gating jobs?
Yeah - just to make sure that docs are actually produced (basically, anything that would cause a doc job failure) I don't think any of us could deal with a spell checker. :) On 06/28/2012 09:19 AM, Chmouel Boudjnah wrote: Just to make sure this gating test will just run python setup.py build_sphinx, right? (if it was a gating spellcheck I'll be in big trouble :)) On Tue, Jun 26, 2012 at 4:02 PM, Monty Taylor mord...@inaugust.com wrote: Hey guys! We have all of the projects properly and consistently building and uploading sphinx docs from in tree. This is pretty exciting, because it means one more resource we can expect to work. So related to that, we were talking about putting in a gating job for each project to prevent changes from breaking the docs. I don't really expect these jobs to fail builds very often, as the jobs themselves are pretty stable - but obviously it's the kind of thing people might have an opinion on. Thoughts? Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Jenkins vs SmokeStack tests Gerrit merge blockers
On 06/28/2012 12:05 PM, Vishvananda Ishaya wrote: On Jun 28, 2012, at 8:13 AM, Monty Taylor wrote: Which adds an additional testing environment that has system software enabled and also installs additional optional things. With that environment, we should be able to run a jenkins gate that tests things with full libvirt, and also tests the mysql upgrade paths, without screwing our fine friends who run OSX. Just fyi, libvirt is installable on OSX with brew and the tests currently run and pass. Great! As a follow up - the full env got merged today. I'll add jenkins jobs to start running it. If you want to give it a spin locally, try: tox -v -efull And it should both install MySQL-python and set up the virtualenv with system-site-packages turned to true, so libvirt tests should run and not skip. In theory. In practice, it's virtualenv, so all bets are off. ;) Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] setuptools-git
Hey all! Using quantum as a little bit of a guinea pig, we've been poking at setuptools-git, which is a setuptools plugin which adds git vcs support to setuptools. Why would we care? Well, setuptools itself has baked in support for cvs and svn (yay! such future thought!) One of the nice bits is that if you use those, you don't have to list all of your extra files in MANIFEST.in - it puts everything that's in your VCS into your tarball. That's pretty much always what we want. So- I added it in a change to quantum after we had an issue with files missing from the tarball, in case anyone wants a look-see: https://review.openstack.org/9109 I'd like to transition all of the projects to doing this, because, well, let's be honest, it's less work long term and will be more correct. Thing is- it requires a setup_requires= stanza in setup.py, which is a facility we have not used in the project before, and is not covered in any sensible way by either tox or install_venv. It'll still WORK, but if you don't run off and pip install setuptools-git you're going to wind up with an egg file in your source tree. Anyway - we use git for everything and it doesn't hurt - so I recommend adding setuptools-git to the list of crap you pip install outside of the virtualenv. I'm going to try to get patches done for tox and install_venv to slightly change how stuff gets injected so that it's not wonky... That is, unless there are massive objections or anything. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Jenkins vs SmokeStack tests Gerrit merge blockers
On 06/28/2012 01:49 PM, Dan Prince wrote: - Original Message - From: Monty Taylor mord...@inaugust.com To: openstack@lists.launchpad.net Sent: Thursday, June 28, 2012 11:13:28 AM Subject: Re: [Openstack] Jenkins vs SmokeStack tests Gerrit merge blockers On 06/28/2012 07:32 AM, Daniel P. Berrange wrote: Today we face a situation where Nova GIT master fails to pass all the libvirt test cases. This regression was accidentally introduced by the following changeset https://review.openstack.org/#/c/8778/ If you look at the history of that, the first SmokeStack test run fails with some (presumably) transient errors, and added negative karma to the change against patchset 2. If it were not for this transient failure, it should have shown the regression in the libvirt test case. The libvirt test case in question was one that is skipped, unless libvirt is actually present on the host running the tests. SmokeStack had made sure the tests would run on such a host. There were then further patchsets uploaded, and patchset 4 was approved for merge. Jenkins ran its gate jobs and these all passed successfully. I am told that Jenkins will actually run the unittests that are included in Nova, so I would have expected it to see the flawed libvirt test case, but it didn't. I presume therefore, that Jenkins is not running on a libvirt enabled host. Kind of - it's sadly more complex than that ... The end result was that the broken changeset was merged to master, which in turns means any other developers submitting changes touching the libvirt area will get broken tests reported that were not actually their own fault. This leaves me with the following questions... 1. Why was the recorded failure from SmokeStack not considered to be a blocker for the merge of the commit by Gerrit or Jenkins or any of the reviewers ? 2. Why did SmokeStack not get re-triggered for the later patch set revisions, before it was merged ? The answer to 1 and 2 is largely the same - SmokeStack is a community contributed resources and is not managed by the CI team. Dan Prince does a great job with it, but it's not a resource that we have the ability to fix should it start messing up, so we have not granted it the permissions to file blocking votes. I would add that if anyone else is interested in collaborating on making SmokeStack better I'm more than happy to give access. Its all open source and has been since Cactus. As is now SmokeStack can can cast a -1 vote and hopefully this is proving to be useful. I'm open to suggestions. I think it's stellar! The tests that smokestack runs could all be written such that they are run by jenkins. I actually put in quite a bit of work to maintain an openstack_vpc job on Jenkins post-Cactus. When we talked about gating on this job at the Diablo conference the idea didn't seem to get very far... I kind of saw that as the end of the line for maintaining an openstack_vpc job and eventually it went away. Not sure who deleted it, but anyway. The way I see it there is value in both testing systems. Rather than complaining about why one system exists and/or doesn't port its tests to the other why don't we build on each others strengths. Seeing a green verified +1 from both Jenkins and SmokeStack on a review should be very encouraging... and if one of the two systems fails it might require further investigation. I completely agree with that. I'm still hoping we'll see more systems from more people so that the set of combinations get larger. I think also there's clearly value in running tests, like how SmokeStack is doing right now, that aren't necessarily part of the gate, but which pro-actively provide useful information to the reviewers. The repos that run the jenkins tests are all in git and managed by openstack's gerrit. If there are testing profiles that it runs that we as a community value and want to see part of the gate, anyone is welcome to port them. 3. Why did Jenkins not ensure that the tests were run on a libvirt enabled host ? This is a different, and slightly more complex. We run tests in virtualenvs so that the process used to test the code can be consistently duplicated by all of the developers in the project. This is the reason that we no longer do ubuntu package creation as part of the gate - turns out that's really hard for a developer running on OSX to do locally on their laptop - and if Jenkins reports an blocking error in a patch, we want a developer to be able to reproduce the problem locally so that they can have a chance at fixing it. The ability for developers to test things locally is very important. For that matter SmokeStack all started with a project called openstack_vpc, a project to spin up groups of cloud servers installed with the latest OpenStack code. A developer can use a project like openstack_vpc to spin up a set of servers in the cloud
[Openstack] Adding docs gating jobs?
Hey guys! We have all of the projects properly and consistently building and uploading sphinx docs from in tree. This is pretty exciting, because it means one more resource we can expect to work. So related to that, we were talking about putting in a gating job for each project to prevent changes from breaking the docs. I don't really expect these jobs to fail builds very often, as the jobs themselves are pretty stable - but obviously it's the kind of thing people might have an opinion on. Thoughts? Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Common openstack client library
Hi!
On 06/19/2012 09:43 AM, Alexey Ababilov wrote:
Hi!
Unfortunately, nova, keystone, and glance clients are very inconsistent.
A lot of code is copied between all these clients instead of moving it
to a common library. The code was edited without synchronization between
clients, so, they have different behaviour:
* all client constructors use different parameters (api_key in nova or
password in keystone and so on);
* keystoneclient authenticates immediately in __init__, while
novaclient does in lazily during first method call;
* {keystone,nova}client can manage service catalogs and accept
keystone's auth URI while glanceclient allows endpoints only;
* keystoneclient can support authorization with an unscoped token but
novaclient doesn't;
* novaclient uses class composition while keystoneclient uses inheritance.
I have developed a library to unify current clients. The library can be
used as-is, but it would be better if openstack clients dropped their
common code (base.py, exceptions.py and so on) and just began to import
common code.
There are two projects already in work focused on various aspects of
this. openstack-common is the place that we put code that should be
shared between the clients. python-openstackclient is a project that
aims at a single consistent interface.
I'm thrilled that you have done some work in this area, but it would be
great if you could do this in the context of the two fairly official
projects that already exist.
Thanks!
Monty
Here is an example of using unified clients.
from openstackclient_base import patch_clients
from openstackclient_base.client import HttpClient
http_client = HttpClient(username=..., password=..., tenant_name=...,
auth_uri=...)
from openstackclient_base.nova.client import ComputeClient
print ComputeClient(http_client).servers.list()
from openstackclient_base.keystone.client import IdentityPublicClient
print IdentityPublicClient(http_client).tenants.list()
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
[Openstack] Thoughts on client library releasing
We're trying to figure out how we release client libraries. We're really close - but there are some sticking points. First of all, things that don't really have dissent (with reasoning) - We should release client libs to PyPI Client libs are for use in other python things, so they should be able to be listed as dependencies. Additionally, proper releases to PyPI will make our cross project depends work more sensibly - They should not necessarily be tied to server releases There could be a whole version of the server which sees no needed changes in the client. Alternately, there could be new upcoming server features which need to go into a released version of the library even before the server is released. - They should not be versioned with the server See above. - Releases of client libs should support all published versions of server APIs An end user wants to talk to his openstack cloud - not necessarily to his Essex cloud or his Folsom cloud. That user may also have accounts on multiple providers, and would like to be able to write one program to interact with all of them - if the user needed the folsom version of the client lib to talk to the folsom cloud and the essex version to talk to the essex cloud, his life is very hard. However, if he can grab the latest client lib and it will talk to both folsom and essex, then he will be happy. There are three major points where there is a lack of clear agreement. Here they are, along with suggestions for what we do about them. - need for official stable branches I would like to defer on this until such a time as we actually need it, rather than doing the engineering for in case we need it. But first, I'd like to define we, and that is that we are OpenStack as an upstream. As a project, we are at the moment probably the single friendliest project for the distros in the history of software. But that's not really our job. Most people out there writing libraries do not have multiple parallel releases of those libraries - they have the stable library, and then they release a new one, and people either upgrade their apps to use the new lib or they don't. One of the reasons this has been brought up as a need is to allow for drastic re-writes of a library. I'll talk about that in a second, but I think that is a thing that needs to have allowances for happening. So the model that keystone-lite used - create an experimental branch for the new work, eventually propose that it becomes the new master - seems like a better fit for the drastic rewrite scenario than copying the stable/* model from the server projects, because I think the most common thing will be that library changes are evolutionary, and having two mildly different branches that both represent something that's actually pretty much stable will just be more confusing than helpful. That being said - at such a time that there is actually a pain-point or a specific need for a stable branch, creating branches is fairly easy ... but I think once we have an actual burning need for such a thing, it will make it easier for us to look at models of how we'll use it. - API or major-rewrite-driven versioning scheme I was wondering why bcwaldon and I were missing each other so strongly in the channel the other day when we were discussing this, and then I realized that it's because we have one word API that's getting overloaded for a couple of different meanings - and also that I was being vague in my usage of the word. So to clarify, a client library has: * programming level code APIs * supported server REST APIs So I back off everything I said about tying client libs version to server REST API support. Brian was right, I was wrong. The thing that's more important here is that the version should indicate programmer contract, and if it that is changed in a breaking manner, the major number should bump. If we combine that with the point from above that our libraries should always support the existing server REST APIs, then I think we can just purely have statements like support for compute v3 can be found in 2.7.8 and later and people will likely be fine, because it will map easily to the idea just grab the latest lib and you should be able to talk to the latest server Yea? So in that case, the client libs versions are purely whatever they are right now, and we'll increase them moving forward using normal library version thoughts. - room for deprecating old APIs The above then leads us to wanting to know what we do about supported server REST APIs over time, especially since I keep making sweeping statements about should support all available server versions ... How about this as a straw man: Since we're planning on beginning to run tests of the client libs against previous versions (so we'll test trunk novaclient against essex nova in addition to trunk nova) ... we need support for past versions of servers as long as our automation can sensibly spin up a past version. (Since the support for that
Re: [Openstack] Thoughts on client library releasing
On 06/18/2012 02:25 PM, Doug Hellmann wrote: How do these plans fit with the idea of creating a unified client library (either as one package or several, based on a common core)? They are kind of orthogonal. At the point where python-openstackclient is ready for release, we'd likely want to manage it the same way. On Mon, Jun 18, 2012 at 5:11 PM, Monty Taylor mord...@inaugust.com mailto:mord...@inaugust.com wrote: We're trying to figure out how we release client libraries. We're really close - but there are some sticking points. First of all, things that don't really have dissent (with reasoning) - We should release client libs to PyPI Client libs are for use in other python things, so they should be able to be listed as dependencies. Additionally, proper releases to PyPI will make our cross project depends work more sensibly - They should not necessarily be tied to server releases There could be a whole version of the server which sees no needed changes in the client. Alternately, there could be new upcoming server features which need to go into a released version of the library even before the server is released. - They should not be versioned with the server See above. - Releases of client libs should support all published versions of server APIs An end user wants to talk to his openstack cloud - not necessarily to his Essex cloud or his Folsom cloud. That user may also have accounts on multiple providers, and would like to be able to write one program to interact with all of them - if the user needed the folsom version of the client lib to talk to the folsom cloud and the essex version to talk to the essex cloud, his life is very hard. However, if he can grab the latest client lib and it will talk to both folsom and essex, then he will be happy. There are three major points where there is a lack of clear agreement. Here they are, along with suggestions for what we do about them. - need for official stable branches I would like to defer on this until such a time as we actually need it, rather than doing the engineering for in case we need it. But first, I'd like to define we, and that is that we are OpenStack as an upstream. As a project, we are at the moment probably the single friendliest project for the distros in the history of software. But that's not really our job. Most people out there writing libraries do not have multiple parallel releases of those libraries - they have the stable library, and then they release a new one, and people either upgrade their apps to use the new lib or they don't. One of the reasons this has been brought up as a need is to allow for drastic re-writes of a library. I'll talk about that in a second, but I think that is a thing that needs to have allowances for happening. So the model that keystone-lite used - create an experimental branch for the new work, eventually propose that it becomes the new master - seems like a better fit for the drastic rewrite scenario than copying the stable/* model from the server projects, because I think the most common thing will be that library changes are evolutionary, and having two mildly different branches that both represent something that's actually pretty much stable will just be more confusing than helpful. That being said - at such a time that there is actually a pain-point or a specific need for a stable branch, creating branches is fairly easy ... but I think once we have an actual burning need for such a thing, it will make it easier for us to look at models of how we'll use it. - API or major-rewrite-driven versioning scheme I was wondering why bcwaldon and I were missing each other so strongly in the channel the other day when we were discussing this, and then I realized that it's because we have one word API that's getting overloaded for a couple of different meanings - and also that I was being vague in my usage of the word. So to clarify, a client library has: * programming level code APIs * supported server REST APIs So I back off everything I said about tying client libs version to server REST API support. Brian was right, I was wrong. The thing that's more important here is that the version should indicate programmer contract, and if it that is changed in a breaking manner, the major number should bump. If we combine that with the point from above that our libraries should always support the existing server REST APIs, then I think we can just purely have statements like support for compute v3 can be found in 2.7.8 and later and people will likely be fine, because it will map easily to the idea just grab the latest lib and you
Re: [Openstack] [Openstack-poc] Thoughts on client library releasing
On 06/18/2012 02:26 PM, Joe Heck wrote: Monty - Thierry stated it as an assumption last PPB meeting, but I'd like it to be explicit that we have at least a tag on each client library release that we make so that it's possible to distribute a version of the clients. +1000 I didn't want to get too far into implementation details, but the way I'd really like to see this work for the client libs is that releases are actually trigger via jenkins by tags on the repo - so there would literally be no way to release something _without_ a tag. I've got a POC patch to do the tag-based-versioning here: https://review.openstack.org/#/c/8427/ We need to get (aiui) one thing landed to zuul so that we can appropriately trigger on tag events... but that's the plan in my brain hole. On Jun 18, 2012, at 2:11 PM, Monty Taylor wrote: We're trying to figure out how we release client libraries. We're really close - but there are some sticking points. First of all, things that don't really have dissent (with reasoning) - We should release client libs to PyPI Client libs are for use in other python things, so they should be able to be listed as dependencies. Additionally, proper releases to PyPI will make our cross project depends work more sensibly - They should not necessarily be tied to server releases There could be a whole version of the server which sees no needed changes in the client. Alternately, there could be new upcoming server features which need to go into a released version of the library even before the server is released. - They should not be versioned with the server See above. - Releases of client libs should support all published versions of server APIs An end user wants to talk to his openstack cloud - not necessarily to his Essex cloud or his Folsom cloud. That user may also have accounts on multiple providers, and would like to be able to write one program to interact with all of them - if the user needed the folsom version of the client lib to talk to the folsom cloud and the essex version to talk to the essex cloud, his life is very hard. However, if he can grab the latest client lib and it will talk to both folsom and essex, then he will be happy. There are three major points where there is a lack of clear agreement. Here they are, along with suggestions for what we do about them. - need for official stable branches I would like to defer on this until such a time as we actually need it, rather than doing the engineering for in case we need it. But first, I'd like to define we, and that is that we are OpenStack as an upstream. As a project, we are at the moment probably the single friendliest project for the distros in the history of software. But that's not really our job. Most people out there writing libraries do not have multiple parallel releases of those libraries - they have the stable library, and then they release a new one, and people either upgrade their apps to use the new lib or they don't. One of the reasons this has been brought up as a need is to allow for drastic re-writes of a library. I'll talk about that in a second, but I think that is a thing that needs to have allowances for happening. So the model that keystone-lite used - create an experimental branch for the new work, eventually propose that it becomes the new master - seems like a better fit for the drastic rewrite scenario than copying the stable/* model from the server projects, because I think the most common thing will be that library changes are evolutionary, and having two mildly different branches that both represent something that's actually pretty much stable will just be more confusing than helpful. That being said - at such a time that there is actually a pain-point or a specific need for a stable branch, creating branches is fairly easy ... but I think once we have an actual burning need for such a thing, it will make it easier for us to look at models of how we'll use it. - API or major-rewrite-driven versioning scheme I was wondering why bcwaldon and I were missing each other so strongly in the channel the other day when we were discussing this, and then I realized that it's because we have one word API that's getting overloaded for a couple of different meanings - and also that I was being vague in my usage of the word. So to clarify, a client library has: * programming level code APIs * supported server REST APIs So I back off everything I said about tying client libs version to server REST API support. Brian was right, I was wrong. The thing that's more important here is that the version should indicate programmer contract, and if it that is changed in a breaking manner, the major number should bump. If we combine that with the point from above that our libraries should always support the existing server REST APIs, then I think we can just purely have statements like support
[Openstack-poc] Thoughts on client library releasing
We're trying to figure out how we release client libraries. We're really close - but there are some sticking points. First of all, things that don't really have dissent (with reasoning) - We should release client libs to PyPI Client libs are for use in other python things, so they should be able to be listed as dependencies. Additionally, proper releases to PyPI will make our cross project depends work more sensibly - They should not necessarily be tied to server releases There could be a whole version of the server which sees no needed changes in the client. Alternately, there could be new upcoming server features which need to go into a released version of the library even before the server is released. - They should not be versioned with the server See above. - Releases of client libs should support all published versions of server APIs An end user wants to talk to his openstack cloud - not necessarily to his Essex cloud or his Folsom cloud. That user may also have accounts on multiple providers, and would like to be able to write one program to interact with all of them - if the user needed the folsom version of the client lib to talk to the folsom cloud and the essex version to talk to the essex cloud, his life is very hard. However, if he can grab the latest client lib and it will talk to both folsom and essex, then he will be happy. There are three major points where there is a lack of clear agreement. Here they are, along with suggestions for what we do about them. - need for official stable branches I would like to defer on this until such a time as we actually need it, rather than doing the engineering for in case we need it. But first, I'd like to define we, and that is that we are OpenStack as an upstream. As a project, we are at the moment probably the single friendliest project for the distros in the history of software. But that's not really our job. Most people out there writing libraries do not have multiple parallel releases of those libraries - they have the stable library, and then they release a new one, and people either upgrade their apps to use the new lib or they don't. One of the reasons this has been brought up as a need is to allow for drastic re-writes of a library. I'll talk about that in a second, but I think that is a thing that needs to have allowances for happening. So the model that keystone-lite used - create an experimental branch for the new work, eventually propose that it becomes the new master - seems like a better fit for the drastic rewrite scenario than copying the stable/* model from the server projects, because I think the most common thing will be that library changes are evolutionary, and having two mildly different branches that both represent something that's actually pretty much stable will just be more confusing than helpful. That being said - at such a time that there is actually a pain-point or a specific need for a stable branch, creating branches is fairly easy ... but I think once we have an actual burning need for such a thing, it will make it easier for us to look at models of how we'll use it. - API or major-rewrite-driven versioning scheme I was wondering why bcwaldon and I were missing each other so strongly in the channel the other day when we were discussing this, and then I realized that it's because we have one word API that's getting overloaded for a couple of different meanings - and also that I was being vague in my usage of the word. So to clarify, a client library has: * programming level code APIs * supported server REST APIs So I back off everything I said about tying client libs version to server REST API support. Brian was right, I was wrong. The thing that's more important here is that the version should indicate programmer contract, and if it that is changed in a breaking manner, the major number should bump. If we combine that with the point from above that our libraries should always support the existing server REST APIs, then I think we can just purely have statements like support for compute v3 can be found in 2.7.8 and later and people will likely be fine, because it will map easily to the idea just grab the latest lib and you should be able to talk to the latest server Yea? So in that case, the client libs versions are purely whatever they are right now, and we'll increase them moving forward using normal library version thoughts. - room for deprecating old APIs The above then leads us to wanting to know what we do about supported server REST APIs over time, especially since I keep making sweeping statements about should support all available server versions ... How about this as a straw man: Since we're planning on beginning to run tests of the client libs against previous versions (so we'll test trunk novaclient against essex nova in addition to trunk nova) ... we need support for past versions of servers as long as our automation can sensibly spin up a past version. (Since the support for that
Re: [Openstack] [Swift] swift.common.client and swift CLI has moved to its own project python-swiftclient
On 06/13/2012 10:58 AM, Juan J. Martinez wrote: On 13/06/12 15:42, Dan Prince wrote: Okay. It looks like Swift also still depends on swiftclient. Long term it would be nice if we could build and unit test swift without relying on the swiftclient package. Could we: I can't see the reason for that, swiftlclient is a dependency of swift (as webob, greenlet, pastedeploy, etc). I agree. At the moment it's slightly more ugly because we haven't finalized the pypi release process for our client libs... but we should have that in the next week or so, and then it should be perfectly possible to treat it just like anything else. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Errors running individual tests that call into the database
On 06/11/2012 12:04 PM, John Garbutt wrote: Hi, I am trying to run tests like test_xenapi and test_libvirt by themselves do things like: nosetests test_xenapi But it does work, I get DB errors relating to missing tables. However, I can successfully run all the tests. The way I understand it: - nova.tests.__init__.py setup() does the database setup - nova.test.py TestCase.setUp() does the resetting of the db It is almost like doing nosetests test_asdf skips the database setup in nova.tests.__init__.py, is that correct? I believe this is because nosetests test_asdf is taking advantage of the where=nova/tests in setup.cfg and actually just starts the test from there. So whereas run_tests.py aliased run_tests.sh test_asdf to run_tests nova.tests.test_asdf, I believe nosetests is now actually changing directory into nose/tests and then running, so is not actually loading nova/tests/__init__.py because it's not resolving the library path that way. Any ideas on how to run tests individually, but still get the database correclty initialized? Am I just calling the tests incorrectly? I like the patch that was submitted for this, actually. Part of the intent behind the recent patches it to get our test runner behaving a little bit more normal and to put extra things we need inside of test fixtures and the like so that as we continue to grow we can take advantage of the work of other people on things like automatic test run parallelization and what not. So actually, doing database setup in the test fixture is the more correct way to do things - whereas just relying on __init__ import semantics to do it is a bit of a hack. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Glance/Swift integration broken (kills devstack)
On 06/11/2012 07:06 PM, Gabriel Hurley wrote: I just thought I'd call out that Glance's swift storage module is currently broken, and apparently this escaped the devstack gate even though devstack actually fails to complete if swift is enabled. Are we not testing with swift in the ENABLED_SERVICES list? Bug report here: https://bugs.launchpad.net/devstack/+bug/1011885 We do not run swift through the gate: https://github.com/openstack-ci/devstack-gate/blob/master/devstack-vm-gate.sh#L28 When we first started doing the devstack gate, I do not believe that swift support in devstack worked fully, and since then, no one has turned it on in the gate. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] new version of gerrit - with new features!
Hey guys! We just upgraded to a new version of gerrit. This is based on the new upstream version 2.4, but in addition we've landed two additional features on top of that - so there's tons of new toys to play with. First of all, in 2.4 upstream has added a new button Rebase Change ... which you can use to rebase your change on top of the current tip of the target branch from within gerrit itself. Also, upstream has been working on adding a proper REST interface, instead of json-rpc which is what they have been using. I'm not sure how far that's gotten, but I believe it can do a decent amount of stuff for those of you who like, you know, REST-based scripting. In addition to that, we've got two main features we've landed as well. David Shrewsbury wrote a long-requested feature: a Work In Progress state. Changes will now have a Work In Progress button on them that can be used to mark the change as something that the dev is working on again, so that other folks know they don't need to review it. The button is available to the author of the patch, as well any group that we assign to have access. In our case, we'll be granting $project-core Work In Progress permission. Putting something back into the ready for review state can be done one of two ways - either by just uploading a new patch to the change, or by clicking the Ready for Review button. Hand in hand with that change, Clark Boylan has written an Important Changes view - which shows all on one page the changes that a developer should be looking at. On the page are changes that were uploaded by the developer (same as the My Changes page), then a section for changes that the developer should be reviewing, which are changes that the dev has either watched, starred, or that reviews have been requested, and that are no in work in progress state and additionally that the dev has not already reviewed. Finally, there is a section for changes that the developer has already reviewed, in case they need to be further tracked. We're hoping that some of these things help to reduce a little bit of the burden in terms of tracking which things should be watched. We'll be working on getting our patches upstreamed in the near future... but since they are new workflow possibilities, we're happy to get feedback on ways in which they could be more useful to folks. Also - we have an open question - which is, if the pre-approval check jobs fail in Jenkins, should the patch be automatically marked Work In Progress. We're not going to do that right out of the gate, but would love feedback on what people think. Thanks everybody! Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] busy day for the CI team
Hey guys! In a fit of things coming together... today has been an unnaturally good and productive day for us, so I thought I'd mention a bunch of stuff (most things long-in-work and just sort of came together nicely today) - builders are now updated to run precise - except for the python2.6 builders, which are still on oneiric since there's no 2.6 on oneiric - local pypi mirror is in use and effective for unittests and devstack - gerrit 2.4 rolled out (see separate email) - fix the horizon build problem, which was actually pip related - rolled out xunit output for unittests along with processing in jenkins for all projects - landed changes in glance and nova that use nosetests directly and remove run_tests.py. Additionally, logging output is done in such a way that nosetests processes it, so the above jenkins output should show tracelog info automatically along with failed tests. We'll carry this over to everyone else soon. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] quantumclient=2012.1
On 06/05/2012 07:54 AM, Gary Kotton wrote: Hi Monty and Dan, Background: A short while ago I started to port bug fixes for Quantum from Folsom-1 to Stable Essex. Jenkins did not accept the patches due to the fact that the automatic tests did not pass. The failures are due to 2 reasons: 1. pep8 checks (addressed in the tox.ini) 2. missing files for automatic tests A fix for this issue was proposed - https://review.openstack.org/#/c/8023. Mark McLoughlin rightly pointed out that the fix was risky by adding a considerable amount of code and suggested adding quantumclient==2012.1 to the pip-requires. The problem with this is that the quantumclient does not exist in pypi (http://pypi.python.org/pypi/python-quantumclient). How do you suggest that we proceed? For now, add: https://github.com/openstack/python-quantumclient/zipball/2012.1#egg=python-quantumclient to the quantum stable/essex pip-requires We're really close to figuring out the client lib upload issue overall (there are about 100 requirement from about 100 different people - but I think we've almost got it sorted) but until we can get it done properly, just do the github zipball bit. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] depend discrepancies
Hey guys! One of the things that came out of ODS is the idea of having a single global dependency list. There are two bits to that - the mechanics of managing the list (which I think we may have sorted) and then, you know, making the list. In compiling the list of what the current global list is, most of the things have clear and obvious answers. However, there are three problematic deps: kombu, PasteDeploy and xattr. Here's what we have for them: ./melange/tools/pip-requires:kombu==1.5.1 ./glance/tools/pip-requires:kombu ./nova/tools/pip-requires:kombu==1.0.4 ./cinder/tools/pip-requires:kombu==1.0.4 ./keystone/tools/pip-requires:PasteDeploy ./melange/tools/pip-requires:PasteDeploy ./quantum/tools/pip-requires:PasteDeploy==1.5.0 ./glance/tools/pip-requires:PasteDeploy ./nova/tools/pip-requires:PasteDeploy==1.5.0 ./swift/tools/pip-requires:pastedeploy==1.3.3 ./cinder/tools/pip-requires:PasteDeploy==1.5.0 ./glance/tools/pip-requires:xattr=0.6.0 ./swift/tools/pip-requires:xattr==0.4 I have no personal emotions towards any of these versions ... but if folks who matter could make some call on what they _should_ be, we can move forward with the first step. I should point out that at the moment we're looking at using update.py from openstack-common to copy in the relevant depends from the master list - so a project will only get the ones it needs. It also means that a decision on a version here does not mean that everyone needs to move to that version immediately ... just that we should be moving towards supporting those versions before folsom, really. Anywhoo... thoughts? Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] git-based jenkins jobs and pre-approval check jobs
On 05/28/2012 04:32 PM, Paul Belanger wrote: On 12-05-25 03:58 PM, Monty Taylor wrote: Hey guys! We just finished rolling out the first in a sequence of upcoming changes to gerrit and jenkins based on needs described at the design summit. We now have the basic jobs for all of the projects (except for horizon, because it's slightly different and I want to spend a little more time with it) applied to jenkins based on some scripts and some yaml files that are in a git repo. This means that anybody could conceivably do some hacking and submit a change to gerrit, instead of the current status quo which is that you have to be a jenkins admin to touch anything. If you want to look at it, it's all in this dir: https://github.com/openstack/openstack-ci-puppet/tree/master/modules/jenkins_jobs/files What was the reason for moving these files directly into puppet and not in a separate git repo? I'm surprise they are not actually packaged to be honest. WELL... a couple of things. First of all, this is stab one, and as we add more openstack projects and jobs we keep finding more features, etc. that we need. It seemed like just doing it directly in one place at the start while we're rapidly iterating on it was less painful. However, as it solidifies, I'm hoping we can have a thing that is the jenkins job filler that can be installed, and then the _config_ of that that's specific to a project is something that puppet can manage. So let's call it coming-soon. :) We're also still a little specifically openstack oriented at the moment ... but we do want to go back through and re-generalize some things as we get a handle on where we need config bits and whatnot. Additionally, it's grouped/templated, so we've actually got identical jenkins jobs running for all of the projects... which means we can hopefully stop running in to the whole oops, I forgot to update the glance jobs situation. But wait, that's not all ... By popular demand, as part of rolling out those jobs, we have added pre-approval check jobs. We are now running merge checks, pep8 checks and unittest jobs on the patch-uploaded event and reporting the results into the code review so that people can skip code reviewing patches that don't work yet. We're also still running tests post-approval so that we ensure the patch as it is to be merged is still good. We have several more great bits that will be rolling out in the next week or two, in the attempt to streamline the set of things that need review, and also speed up the testing of reviewed things. Happy hacking Nice work! I always enjoy looking to your work and see what you guys are doing in the world of automation. Thanks! ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] git-based jenkins jobs and pre-approval check jobs
Hey guys! We just finished rolling out the first in a sequence of upcoming changes to gerrit and jenkins based on needs described at the design summit. We now have the basic jobs for all of the projects (except for horizon, because it's slightly different and I want to spend a little more time with it) applied to jenkins based on some scripts and some yaml files that are in a git repo. This means that anybody could conceivably do some hacking and submit a change to gerrit, instead of the current status quo which is that you have to be a jenkins admin to touch anything. If you want to look at it, it's all in this dir: https://github.com/openstack/openstack-ci-puppet/tree/master/modules/jenkins_jobs/files Additionally, it's grouped/templated, so we've actually got identical jenkins jobs running for all of the projects... which means we can hopefully stop running in to the whole oops, I forgot to update the glance jobs situation. But wait, that's not all ... By popular demand, as part of rolling out those jobs, we have added pre-approval check jobs. We are now running merge checks, pep8 checks and unittest jobs on the patch-uploaded event and reporting the results into the code review so that people can skip code reviewing patches that don't work yet. We're also still running tests post-approval so that we ensure the patch as it is to be merged is still good. We have several more great bits that will be rolling out in the next week or two, in the attempt to streamline the set of things that need review, and also speed up the testing of reviewed things. Happy hacking Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Translation and Internationalization in OpenStack
On 05/08/2012 09:56 PM, Thierry Carrez wrote: Gabriel Hurley wrote: Having worked with all three tools, I would strongly suggest Transifex, particularly given that we as a community have to do almost no work to maintain it, it's the only tool that supports OpenStack as a project hub with shared teams and management, and it offers us a strong crowdsourced translation community. Getting a strong crowdsourced translation community is the most important aspect to me. We can relatively easily bridge the tooling/integration gap, but we can't invent a translation community :) I know for a fact that Launchpad has a magic translation community (just pushing stuff there makes it translated). If Transifex's community is as efficient and gives us better integration, then we should go for it. As per Thierry's call for volunteers, I will throw my hat in the ring to spearhead translation efforts in OpenStack for the time being. Great! I'm happy to defer the tool decision to the people that will own and push that work forward ;) Same here. Transifex seems like it meets our needs - and if it's got a champion who wants to do the work, then stellar! ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack-poc] [Bug 943336] Re: New requirements never get written out
Honestly, I think we might want to delete write_requirements() altogether. I don't think it's beneficial... asign this to me and I'll get it fixed up. On 05/04/2012 11:30 AM, Mark McLoughlin wrote: Could do with more details here, I'm not sure I follow what the issue is ** Changed in: openstack-common Assignee: (unassigned) = Jason Kölker (jason-koelker) ** Changed in: openstack-common Importance: Undecided = Low ** Changed in: openstack-common Milestone: None = folsom-1 -- You received this bug notification because you are a member of OpenStack Common Drivers, which is the registrant for openstack-common. https://bugs.launchpad.net/bugs/943336 Title: New requirements never get written out Status in openstack-common: Confirmed Bug description: write_requirements is failing to write out requirements.txt on a new virtualenv/checkout. Need to figure out what's going on. To manage notifications about this bug go to: https://bugs.launchpad.net/openstack-common/+bug/943336/+subscriptions ___ Mailing list: https://launchpad.net/~openstack-poc Post to : openstack-poc@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-poc More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nova subsystem branches and feature branches
On 05/03/2012 05:24 AM, Thierry Carrez wrote: (snip things I pretty much agree with) (I'm not sure gerrit is right for this. Why not just do it in folk's github forks? I think all people are looking for is for people to be more aware of feature branches. How about if you put details of your feature branch in the blueprint for the feature?) (If not using gerrit, can developers configure Jenkins to CI their branch? Or is Smokestack the right tool?) I think preserving the ability to run your branch through integration testing is a necessary prerequisite of the new model. Agree, otherwise it gets weird. HOWEVER - I also agree that the current UI as we're presenting it might not be optimal. Let me follow up with some sketched out thoughts on how we can address both concerns. Thanks again for starting the discussion on this. I'll meet with Jim Blair (and hopefully Monty) next week to brainstorm solutions, so discussing the needed properties of the system is a very nice step forward. Regards, ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nova subsystem branches and feature branches
On 05/03/2012 08:50 AM, Mark McLoughlin wrote: On Thu, 2012-05-03 at 16:46 +0100, Mark McLoughlin wrote: Hey, On Thu, 2012-05-03 at 14:24 +0200, Thierry Carrez wrote: Mark McLoughlin wrote: And how about feature branches? - Feature branches are relatively short-lived (i.e. weeks or months rather than years) branches for a specific feature. They are a mechanism for developers to work on a patch series in the open until the feature is complete enough to be merged into a subsystem branch or master. (I'm not sure gerrit is right for this. Why not just do it in folk's github forks? I think all people are looking for is for people to be more aware of feature branches. How about if you put details of your feature branch in the blueprint for the feature?) (If not using gerrit, can developers configure Jenkins to CI their branch? Or is Smokestack the right tool?) I think preserving the ability to run your branch through integration testing is a necessary prerequisite of the new model. Yes, it's only really needed at the point where the feature branch is merged into the subsystem tree. The developer should be able to break stuff on their WIP feature branch if they wish, since they'll be expect to rebase/fix before proposing it to be merged into the subsystem tree. So Jenkins/Smokestack would be nice tools to provide to folks working on feature branches, but they don't need to be gating. Put another way - the gating tests should be run on every single commit that is ultimately merged into master. However, this should not mean that we apply gating to every commit on feature branches, since feature branches are allowed to rebase until they are merged into a subsystem tree or directly into master. Yes. This all makes sense to me and I can be in support of this plan (and continue work on making that process of moving from feature branch to subsystem branch is not painful) Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Mailing-list split
On 04/27/2012 06:11 AM, Jim Meyering wrote: Daniel P. Berrange wrote: On Fri, Apr 27, 2012 at 12:04:34PM +0200, Thierry Carrez wrote: To avoid Launchpad list slowness, we would run the new openstack-dev list off lists.openstack.org. Given the potential hassle of dealing with spam and delivery issues on mission-critical MLs, we are looking into the possibility of outsourcing the maintenance of lists.openstack.org to a group with established expertise running mailman instances. Please let us know ASAP if you could offer such services. We are not married to mailman either -- if an alternative service offers good performance and better integration (like OpenID-based subscription to integrate with our SSO), we would definitely consider it. FYI for libvirt mailing lists we are using the GNU project's spam filter: https://savannah.gnu.org/maintenance/ListHelperAntiSpam I think the procedure documented there works fine for gnu projects, but that something special has to be done for projects hosted elsewhere. I'll check and get back to you. Awesome - and thanks. It's entirely possible that our Jim Blair already knows a decent amount of that, as he set up most of the FSF's mail and mailing list infrastructure, iirc. BUT - that was several years ago, so it's also possible something has changed. :) Thanks to this we get essentially zero spam on our mailing lists, with practically no burden on / work required by our list admins. Jim Meyering (CC'd) set it up for us originally may have some recommendations or tips if OpenStack want to make use of it too. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Mailing-list split
Hey everyone! On 04/27/2012 05:04 AM, Thierry Carrez wrote: To avoid Launchpad list slowness, we would run the new openstack-dev list off lists.openstack.org. Given the potential hassle of dealing with spam and delivery issues on mission-critical MLs, we are looking into the possibility of outsourcing the maintenance of lists.openstack.org to a group with established expertise running mailman instances. Please let us know ASAP if you could offer such services. We are not married to mailman either -- if an alternative service offers good performance and better integration (like OpenID-based subscription to integrate with our SSO), we would definitely consider it. Just to be clear - I definitely think that mailing lists are an important part of dev infrastructure and would love for this to be a fully integrated part of all of the rest of our tools. However, the current set of active infrastructure team members have huge todo lists at the moment. So the biggest home run from my perspective would be if someone out there had time or resources and wanted to join us on the infra team to manage this on our existing resources (turns out we have plenty of servers for running this, and even a decent amount of expertise, just missing manpower). The existing team would be more than happy to be involved, and it would help avoid get-hit-by-a-truck issues. We're a pretty friendly bunch, I promise. Any takers? Anybody want to pony up somebody with some bandwidth to admin a mailman? Respond back here or just find us in #openstack-infra and we'll get you plugged in and stuff. Thanks! Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Mailing-list split
On 04/27/2012 09:44 AM, Duncan McGreggor wrote: On Fri, Apr 27, 2012 at 8:46 AM, Monty Taylor mord...@inaugust.com wrote: Hey everyone! On 04/27/2012 05:04 AM, Thierry Carrez wrote: To avoid Launchpad list slowness, we would run the new openstack-dev list off lists.openstack.org. Given the potential hassle of dealing with spam and delivery issues on mission-critical MLs, we are looking into the possibility of outsourcing the maintenance of lists.openstack.org to a group with established expertise running mailman instances. Please let us know ASAP if you could offer such services. We are not married to mailman either -- if an alternative service offers good performance and better integration (like OpenID-based subscription to integrate with our SSO), we would definitely consider it. Just to be clear - I definitely think that mailing lists are an important part of dev infrastructure and would love for this to be a fully integrated part of all of the rest of our tools. However, the current set of active infrastructure team members have huge todo lists at the moment. So the biggest home run from my perspective would be if someone out there had time or resources and wanted to join us on the infra team to manage this on our existing resources (turns out we have plenty of servers for running this, and even a decent amount of expertise, just missing manpower). The existing team would be more than happy to be involved, and it would help avoid get-hit-by-a-truck issues. We're a pretty friendly bunch, I promise. Any takers? Anybody want to pony up somebody with some bandwidth to admin a mailman? Respond back here or just find us in #openstack-infra and we'll get you plugged in and stuff. Thanks! Monty Count me in, Monty. I've been managing mailman lists for about 12 years now (and, incidentally, Barry and I are bruthas from anutha mutha), so I'd be quite comfortable handling those responsibilities. I can couple it with the python.org SIG mail list that I manage, so there'd be zero context switching. You make me very happy! Let's work out the details and stuff... Check it out - it's like we're, you know, a collaborative community or something! ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Metering] Metering repository in stackforge
Hey! On 04/27/2012 06:20 PM, Loic Dachary wrote: Hi, I would like to create a repository ceilometer in https://github.com/stackforge to host the code for the newborn Metering project ( https://launchpad.net/ceilometer , first meeting held this thursday http://wiki.openstack.org/Meetings/MeteringAgenda ). We would be more than thrilled to get you set up. I'm including Andrew here, as he's been doing most of the StackForge work. I'll also file a bug on openstack-ci to make sure we don't lose this. I'm not sure how to proceed, could you please guide me ? My github account name is dachary Thanks in advance ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] openstack.common setup code
Hey guys, Quick follow up from the summit on things that should happen in projects from the setup module of openstack common as I understand it. (to make sure we're all on the same page) There are currently 5 essential things in openstack.common.setup: parse_requirements parse_dependency_links write_requirements write_git_changelog generate_authors that are being used to varying levels in the various projects. What should happen at this point is this: parse_requirements parse_dependency_links Should be in all of the client libraries and should be removed from all the not-client libraries. These are essential for pip installation of client libs (which is important) as they allow pip to follow the depends. The make things hard for non-client libs, as setuptools doesn't understand git urls, which we use in non-client lib pip-requires files. write_requirements Should die everywhere. It was an attempt to record in our tarballs the state of what was actually tested ... but did not actually provide benefit to anyone - and the distros hate it. write_git_changelog generate_authors Should be added/used everywhere. When generate_authors is added, unit tests testing authors content should be removed. Is this how everyone else understood the outcome of conversations at the summit too? Thanks! Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Using Foreign Keys
On 04/26/2012 10:14 AM, Sean Dague wrote: On 04/25/2012 05:17 PM, Vishvananda Ishaya wrote: The main issue is when the relevant tables are moved into a separate service a la quantum or cinder. We can't keep referential integrity across multiple databases, so the foreign keys in this case need to be removed. It leads to an odd situation when there is still an internal implementation in addition to the external implementation because the internal implementation no longer has foreign keys. As an example, we used to have foreign key relationships between instances and networks. We can no longer have these because we support networks declared externally. The internal network management now has no referential integrity, but this is the price we pay for separation of concerns. We are going through a similar set of relationship-breaking with the volume code. There are definitely the practical aspects of where this can't be done because the services have split out, and I think that's fine. But enforcing the ref constraints where possible just provides another level of safety in the data. A policy where we break FK relationships if the preferred core model is 2 services (i.e. Nova / Quantum), but we add FK constraints within a service might be a good idea. SO ... in a production MySQL service in this situation, under no circumstances should foreign keys actually be applied to the database. Specifying them as part of the SqlAlchemy model is fine, and I believe conveys the informational relationships that are important. But it turns out that in practice, especially with an ORM running things, the performance hit of adding them is pretty bad (generates tons of unneeded index scans, for one thing) If all of your db access is via the ORM layer, there is absolutely zero actual benefit. I think the real key is to have a config option to tell sqlalchemy to not, even if we're running innodb, add the foreign keys to the DDL sent to the database. If sqlalchemy doesn't have that ability, we should write it and contribute it, because anyone using MySQL at scale via sqlalchemy actually wants the feature, whether they recognize it yet or not. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] Minimum required code coverage per file
On 04/24/2012 10:08 PM, Lorin Hochstein wrote: On Apr 24, 2012, at 4:11 PM, Joe Gordon wrote: Hi All, I would like to propose a minimum required code coverage level per file in Nova. Say 80%. This would mean that any new feature/file should only be accepted if it has over 80% code coverage. Exceptions to this rule would be allowed for code that is covered by skipped tests (as long as 80% is reached when the tests are not skipped). I like the idea of looking at code coverage numbers. For any particular merge proposal, I'd also like to know whether it increases or decreases the overall code coverage of the project. I don't think we should gate on this, but it would be helpful for a reviewer to see that, especially for larger proposals. Yup... Nati requested this a couple of summits ago - main issue is that while we run code coverage and use the jenkins code coverage plugin to track the coverage numbers, the plugin doesn't fully support this particular kind of report. HOWEVER - if any of our fine java friends out there want to chat with me about adding support to the jenkins code coverage plugin to track and report this, I will be thrilled to put it in as a piece of reported information. With 193 python files in nova/tests, Nova unit tests produce 85% overall code coverage (calculated with ./run_test.sh -c [1]). But 23% of files (125 files) have lower then 80% code coverage (30 tests skipped on my machine). Getting all files to hit the 80% code coverage mark should be one of the goals for Folsom. I would really like to see a visualization of the code coverage distribution, in order to help spot the outliers. Along these lines, there's been a lot of work in the software engineering research community about predicting which parts of the code are most likely to contain bugs (fault prone is a good keyword to find this stuff, e.g.: http://scholar.google.com/scholar?q=fault+prone, big names include Nachi Nagappan at MS Research and Elaine Weyuker, formerly of ATT Research). I would *love* to see some academic researchers try to apply those techniques to OpenStack to help guide QA activities by identifying which parts of the code should get more rigorous testing and review. ++ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack][Nova] Minimum required code coverage per file
Hey - funny story - in responding to Justin I re-read the original email and realized it was asking for a static low number, which we _can_ do - at least project-wide. We can't do per-file yet, nor can we fail on a downward inflection... and I've emailed Justin about that. If we have consensus on gating on project-wide threshold, I can certainly add adding that to the gate to the todo list. (If we decide to do that, I'd really like to make that be openstack-wide rather than just nova... although I imagine it might take a few weeks to come to consensus on what the project-wide low number should be. Current numbers on project-wide lines numbers: nova: 79% glance: 75% keystone: 81% swift: 80% horizon: 91% Perhaps we get nova and glance up to 80 and then set the threshold for 80? Also, turns out we're not running this on the client libs... Monty On 04/25/2012 03:53 PM, Justin Santa Barbara wrote: If you let me know in a bit more detail what you're looking for, I can probably whip something up. Email me direct? Justin On Wed, Apr 25, 2012 at 6:59 AM, Monty Taylor mord...@inaugust.com mailto:mord...@inaugust.com wrote: On 04/24/2012 10:08 PM, Lorin Hochstein wrote: On Apr 24, 2012, at 4:11 PM, Joe Gordon wrote: Hi All, I would like to propose a minimum required code coverage level per file in Nova. Say 80%. This would mean that any new feature/file should only be accepted if it has over 80% code coverage. Exceptions to this rule would be allowed for code that is covered by skipped tests (as long as 80% is reached when the tests are not skipped). I like the idea of looking at code coverage numbers. For any particular merge proposal, I'd also like to know whether it increases or decreases the overall code coverage of the project. I don't think we should gate on this, but it would be helpful for a reviewer to see that, especially for larger proposals. Yup... Nati requested this a couple of summits ago - main issue is that while we run code coverage and use the jenkins code coverage plugin to track the coverage numbers, the plugin doesn't fully support this particular kind of report. HOWEVER - if any of our fine java friends out there want to chat with me about adding support to the jenkins code coverage plugin to track and report this, I will be thrilled to put it in as a piece of reported information. With 193 python files in nova/tests, Nova unit tests produce 85% overall code coverage (calculated with ./run_test.sh -c [1]). But 23% of files (125 files) have lower then 80% code coverage (30 tests skipped on my machine). Getting all files to hit the 80% code coverage mark should be one of the goals for Folsom. I would really like to see a visualization of the code coverage distribution, in order to help spot the outliers. Along these lines, there's been a lot of work in the software engineering research community about predicting which parts of the code are most likely to contain bugs (fault prone is a good keyword to find this stuff, e.g.: http://scholar.google.com/scholar?q=fault+prone, big names include Nachi Nagappan at MS Research and Elaine Weyuker, formerly of ATT Research). I would *love* to see some academic researchers try to apply those techniques to OpenStack to help guide QA activities by identifying which parts of the code should get more rigorous testing and review. ++ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack-poc] [Bug 983734] Re: Keystone fails badly if you miss one option
I agree with Mark. In my world, all options should have sane defaults. -- You received this bug notification because you are a member of OpenStack Common Drivers, which is the registrant for openstack-common. https://bugs.launchpad.net/bugs/983734 Title: Keystone fails badly if you miss one option Status in OpenStack Identity (Keystone): Confirmed Status in openstack-common: Invalid Bug description: If you misspell or forget one option in keystone.conf (like template_file for TemplatedCatalog backend), Keystone will fail with misguiding critical failure (in my case, TypeError: coercing to Unicode: need string or buffer, NoneType found). To manage notifications about this bug go to: https://bugs.launchpad.net/keystone/+bug/983734/+subscriptions ___ Mailing list: https://launchpad.net/~openstack-poc Post to : openstack-poc@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-poc More help : https://help.launchpad.net/ListHelp
Re: [Openstack] New Gerrit version (and server)
On 04/13/2012 04:36 AM, Kiall Mac Innes wrote: The new gerrit version also supports per user namespaces, if enabled. These allow users to create private branches with full push privileges etc.. Have these been enabled? They have not - we need to verify what the behavior looks like over the event interface so that we're not all of a sudden attempting to devstack a bunch of in-progress stuff without meaning to. We'll put it on the list of new things to investigate our possible use of! Monty On Apr 13, 2012 12:33 p.m., Thierry Carrez thie...@openstack.org mailto:thie...@openstack.org wrote: Mark McLoughlin wrote: One new addition in 2.3 is draft changes. The idea behind a draft change in Gerrit is that it is a change that is not ready for merging, or even general code review, but you would like to share it with some people to get early comments. If you upload a change as a draft, by default, no one else can see it. You must explicitly add each person you would like to share it with as a reviewer. Reviewers you add can leave comments, but can not vote at this stage. You can continue to upload new patchsets to the change as it evolves, and once it is ready for general review, you can click the Publish button. It will then become a normal change in Gerrit that everyone can see, including the earlier reviews from the draft stage. This is a one way transition; once a draft is published, it can't be made a draft again. This sounds cool. Will the vulnerability management team use this for embargoed security fixes? I'll definitely look into that possibility... Depends a bit on the security model around drafts (not being listed is not the same as being private). -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] DevStack stable/essex branch
It should Just Work™ On 04/11/2012 02:29 PM, Vishvananda Ishaya wrote: Yay! Awesome work Dean! Monty/Jim: Has the ci infrastructure been updated to use the stable/essex branch for integration tests on the stable/essex merges? Vish On Apr 11, 2012, at 1:57 PM, Dean Troyer wrote: The stable/essex branch of DevStack has been created in GitHub (https://github.com/openstack-dev/devstack/tree/stable/essex) with stackrc pre-configured to pull from stable/essex branches of the project repos as appropriate. http://devstack.org has been updated to reflect the current state. dt -- Dean Troyer dtro...@gmail.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Image API v2 Draft 4
On 04/09/2012 04:11 PM, Jay Pipes wrote: On 04/09/2012 07:07 PM, Jorge Williams wrote: On Apr 9, 2012, at 6:03 PM, Justin Santa Barbara wrote: How about we discuss this further at the summit :-) I think that's a sensible proposal. We're not likely to reach a good conclusion here. I think my viewpoint is that even json-dressed-as-xml is fine; no end-user gives two hoots what our JSON/XML/HPSTR looks like. I'd wager most users of the EC2 API have never even seen the data representation! I take it you didn't attend the glorious JSON debate of a couple of summits ago :-) Glorious it was indeed. I'm up for round two, Only with beer. :) I still owe you a couple I think! I refuse to not be there for that. Please make sure I'm in the room. With a video camera. And a bottle of whiskey. Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Jcloud and openstack relation
Hi! jclouds is an open source java library for connecting to clouds. As such, it supports connecting to OpenStack based clouds. The OpenStack CI team have been working with jclouds on both support for OpenStack as well as a plugin for jenkins so that we can have more direct control of build slaves on the clouds we use for that purpose. Does that help? Monty On 04/07/2012 07:02 PM, Frans Thamura wrote: hi all can share to me, what is the relateion between openstack and jclouds. F ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Jcloud and openstack relation
On 04/08/2012 07:21 PM, Justin Santa Barbara wrote: There is also a (friendly rival) OpenStack Java binding being developed: https://github.com/platformlayer/openstack-java https://github.com/woorea/openstack-java-sdk/ Friendly rivalries are the best! That library supports a direct-to-OpenStack Jenkins plugin which I confidently predict will rapidly surpass any lowest-common-denominator attempts (*): https://github.com/platformlayer/openstack-jenkins (*: This opinion may be biased) If you do evaluate JClouds against the OpenStack Java bindings (or the Jenkins plugin), please let me know of any areas where the OpenStack bindings aren't better, and I will make sure to improve that area. I'd say the same applies in the reverse, too. If we cross-file bugs against each project, both should eventually rule the world. Monty: We should port the OpenStack CI team's work to the OpenStack Jenkins plugin at the summit! Justin - I definitely want to sit down with you and Adrian Cole at the summit... the more we chat about usecases and whatnot the better everyone winds up being, I think. One of the things we're trying to do is provide a happy migration path for folks - but you know - face to face about this stuff often winds up being super-big win. Yay for Open Source! Monty On Sun, Apr 8, 2012 at 6:37 PM, Monty Taylor mord...@inaugust.com mailto:mord...@inaugust.com wrote: Hi! jclouds is an open source java library for connecting to clouds. As such, it supports connecting to OpenStack based clouds. The OpenStack CI team have been working with jclouds on both support for OpenStack as well as a plugin for jenkins so that we can have more direct control of build slaves on the clouds we use for that purpose. Does that help? Monty On 04/07/2012 07:02 PM, Frans Thamura wrote: hi all can share to me, what is the relateion between openstack and jclouds. F ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Jcloud and openstack relation
Yes - jclouds and OpenStack Java are both bindings to be used to launch and orchestrate cloud instances. Neither are aimed at running Java in any particular way inside of a cloud instance. Honestly, I don't see any reason that normal jvms shouldn't work just fine in a cloud server. On 04/08/2012 07:24 PM, Frans Thamura wrote: hi i see jclouds it is like binding, right... but should we launch a Java apps, which Java under JVM and Cloud/Virtualization also in Cloud, why dont auto launch i see JRockit VE run good in VMWare (i saw it 2007). F On Mon, Apr 9, 2012 at 9:21 AM, Justin Santa Barbara jus...@fathomdb.com mailto:jus...@fathomdb.com wrote: There is also a (friendly rival) OpenStack Java binding being developed: https://github.com/platformlayer/openstack-java https://github.com/woorea/openstack-java-sdk/ That library supports a direct-to-OpenStack Jenkins plugin which I confidently predict will rapidly surpass any lowest-common-denominator attempts (*): https://github.com/platformlayer/openstack-jenkins (*: This opinion may be biased) If you do evaluate JClouds against the OpenStack Java bindings (or the Jenkins plugin), please let me know of any areas where the OpenStack bindings aren't better, and I will make sure to improve that area. Monty: We should port the OpenStack CI team's work to the OpenStack Jenkins plugin at the summit! Justin On Sun, Apr 8, 2012 at 6:37 PM, Monty Taylor mord...@inaugust.com mailto:mord...@inaugust.com wrote: Hi! jclouds is an open source java library for connecting to clouds. As such, it supports connecting to OpenStack based clouds. The OpenStack CI team have been working with jclouds on both support for OpenStack as well as a plugin for jenkins so that we can have more direct control of build slaves on the clouds we use for that purpose. Does that help? Monty On 04/07/2012 07:02 PM, Frans Thamura wrote: hi all can share to me, what is the relateion between openstack and jclouds. F ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net mailto:openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Plugin for Jenkins
On 04/05/2012 01:22 AM, Justin Santa Barbara wrote: I've got Compute functionality working with the OpenStack Jenkins plugin, so it can launch nova instances as on-demand slaves now, run builds on them, and archive the results into swift. I'd like to open GitHub issues to track your requirements, but I have a few questions. I shall do my best to elaborate... We need disposable machines that are only used for one test, which means spinning up and terminating hundreds of machines per day. Sounds like we want a function to terminate the machine after the job has run. https://github.com/platformlayer/openstack-jenkins/issues/1 Yes. That seems sensible. We need to use machines from multiple providers simultaneously so that we're resilient against errors with one provider. Label expressions should work here; you would apply a full set of axis labels to each machine (rax oneiric python26) but then you would filter based only on the required axes (oneric python26). Are labels sufficient for this? Labels are sufficient for tying the jobs to the specific resource description. I think the idea here is that we definitely want to be able to configure multiple cloud providers, and for each provider (in some manner) be able to configure what a machine labeled oneiric would look like. (likely as a combination of image, flavor and setup script) After that - honestly - as long as we can actually get an oneiric labeled machine from _someone_ when we ask for it, we're good. We need to pull nodes from a pool of machines that have been spun up ahead of time for speed. This sounds like a custom NodeProvisioner implementation. The current implementation is optimized around minimizing CPU hours, by doing load prediction. You have a different criteria, based on minimizing launch latency. It looks like it should be relatively easy to implement a new algorithm, although perhaps a bit tricky to figure out how to plug it in. https://github.com/platformlayer/openstack-jenkins/issues/2 Yeah - average time to spin up a node and get it configured _when_it_works_ is between 5 and 10 minutes. devstack takes around that amount of time, so if we have to actually wait for the node to spin up each time, we'd be doubling the time it takes to test a change. Then there's the fact that clouds fail at giving us working node ALL THE TIME. So waiting for re-trys and such (although if it was happening at jenkins node provisioning time would be technically correct) could potentially lead to a terribly build queue! We need to be able to select from different kinds of images for certain tests. Are labels sufficient for this? Yes. Configuring the characteristics of an image and assigning a label to those characteristics will definitely let us associate tests with the right running environment. Machines need to be checked for basic functionality before being added to the pool (we frequently get nodes without a functioning network). I believe Jenkins does this anyway; a node which doesn't have networking won't be able to get the agent. And you can run your own scripts after the slave boots up (apt-get install openjdk, for example). Those scripts can basically do any checks you want. Is that enough? Yes- just pointing out that it's a case that we have to deal with at the moment so it needs to be handled. They need to be started from snapshots with cached data on them to avoid false negatives from network problems. Can you explain this a bit more? This is to protect against the apt repositories / python sources / github repos being down? Would an http proxy be enough? Yes. apt repositories, pypi and github are CONSTANTLY down, so we do a lot of work to pre-cache network fetched resources onto a machine so that running of the tests almost never have to involve a network fetch. (we've learned over the last year or so that any time a test system wants to fetch network resources, that the number of false-negatives due to github or pypi going away is unworkably high) It's possible that an http proxy _might_ help that - but the approach we've been taking so far is to have one process that spins up a node, does all the network fetching into local resources, and then snapshots that into an image which is the basis for subsequent node creation. The base image is updated nightly so that the amount of network update that has to happen at node instantiation time is minimized. jclouds itself (rather than the plugin) has a caching feature which does the auto-image creation based on node creation criteria. So if you combine the characteristics of a node (image, flavor, init-script, ram, volumes, etc) with a TTL, then the first time a node meeting those criteria is requested, it will create you one from scratch, but at the end of the user data script run, it will create an image snapshot which it can use for subsequent creation of nodes which match the same description. When we combine that
