Re: [Openstack] Horizon PTL Candidacy
+1 :) On Aug 31, 2012, at 5:14 PM, Gabriel Hurley wrote: I hereby officially throw my hat in the ring to be Horizon's PTL. Qualifications: I'm a highly active developer on Horizon, and a member of the Horizon Drivers group. I wrote the initial version of python-keystoneclient, and I'm a member of the Keystone Core group as well. I'm also a core committer for the Django web framework on which Horizon is based. I work hard to understand the workings of the entire stack and the needs of the ecosystem at large so we can work together to make the entirety of OpenStack consistent, stable and amazing. I believe VERY strongly in stability, backwards-compatibility, and consistent, properly-versioned APIs. I also have a strong belief in the importance of translation, internationalization and localization to support our international userbases. Contribution over the last six months: I've been the implementer on the majority of the large Folsom blueprints, most of which had to do with improving the flow and ease-of-use of various complicated workflows. I also re-implemented the keystone authentication in Horizon to be dramatically more robust and secure. I've provided a lot of feedback on API revisions in other projects (e.g. Keystone) since those APIs deeply affect Horizon's ability to deliver on a high quality experience. I guided the community's initiative to nail down a complete set of guidelines for developers around internationalization. I've been largely leading the Horizon project in terms of architecture and code review during the Folsom timeframe despite not being the official PTL. Most critical aspects for Horizon in the next 6 months: Full RBAC support; continued focus on reducing the user frustration in complex workflows and interactions; adding more introspective features to reduce boilerplate; cleaner separation of end-user and admin code flows; making the admin dashboard more useful to admins as opposed to just being the same user dashboard except you can see everything in the system; improved file upload mechanisms; Philosophical ideas regarding being a PTL: The foremost role of the PTL is to maintain and convey the long-term vision of the project through day-to-day work (in code, in architecture, in reviews, in answering questions, etc.). Right behind that it is crucial that the PTLs work to guide the community (in gathering consensus on difficult topics, in supporting community members of all types, in maintaining the principles of our community). A close third is that the PTLs must work with each other to ensure the consistency, compatibility, and commonality of all core projects. I'm more than happy to answer any questions or address any concerns people may have. :-) All the best, - Gabriel ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nova and asynchronous instance launching
On Jun 28, 2012, at 9:01 AM, Jay Pipes wrote: On 06/27/2012 06:51 PM, Doug Davis wrote: Consider the creation of a Job type of entity that will be returned from the original call - probably a 202. Then the client can check the Job to see how things are going. BTW - this pattern can be used for any async op, not just the launching of multiple instances since technically any op might be long-running (or queued) based on the current state of the system. Note that much of the job of launching an instance is already asynchronous -- the initial call to create an instance really just creates an instance UUID and returns to the caller -- most of the actual work to create the instance is then done via messaging calls and the caller can continue to call for a status of her instance to check on it. In this particular case, I believe Devin is referring to when you indicate you want to spawn a whole bunch of instances and in that case, things happen synchronously instead of asynchronously? Devin, is that correct? If so, it seems like returning a packet immediately that contains a list of the instance UUIDs that can be used for checking status is the best option? Yep, exactly. The client still waits synchronously for the underlying RPC to complete. An immediate 202 would be a great way to deal with this. Or am I missing something here? -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [keystone] proposing adding Adam Young (ayoung) to keystone-core
+1! On Jun 27, 2012, at 12:44 AM, Syed Armani wrote: Adam is a great guy, He is always there if you need any help or guidance. His work is incredible. Best! Syed On Wed, Jun 27, 2012 at 12:57 PM, Razique Mahroua razique.mahr...@gmail.com wrote: Not part of that group, but Adam's work is indeed fantastic Nuage Co - Razique Mahroua razique.mahr...@gmail.com NUAGECO-LOGO-Fblan_petit.jpg Le 26 juin 2012 à 23:06, Joseph Heck a écrit : Given his work in Keystone since the redux, I would like propose Adam Young (ayoung) be added to the group keystone-core. For a process in doing this, I thought we'd generally follow Nova's core-promotion process = lazy consensus over a week, and assuming no -1's and at least two +1's from current core members, it's good to go. -joe ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Nova and asynchronous instance launching
We filed a blueprint for this yesterday: https://blueprints.launchpad.net/nova/+spec/launch-instances-async Currently if a user attempts to create a lot of instances with a single API call (using min_count) the request will hang for a long time while all RPC calls are completed. For a large number of instances this can take a very long time. The API should return immediately and asynchronously make RPC calls. We are looking for creative ways to work around this problem, but in the meantime I'd like to hear from folks on what they think the preferred solution would be. Devin___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Tihomir Trifonov added to horizon-core
Welcome Tihomir to horizon-core! Thanks for all the great contributions in the past months! Devin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Need stable/essex review for Horizon
Hey all, We've had a number of stable/essex reviews up that were abandoned due to lack of reviews. We have re-enabled them and are hoping to get some eyes on these so we can release 2012.1.1: https://review.openstack.org/#/c/7718/ https://review.openstack.org/#/c/7723/ https://review.openstack.org/#/c/7729/ https://review.openstack.org/#/c/7731/ Thanks all! Devin___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Need stable/essex review for Horizon
Skipping that one as it breaks python 2.6 compatibility which we still target for the Essex version and we won't have time to prepare a proper patch for 2012.1.1. It's also quite low priority. Thanks, Devin On Jun 21, 2012, at 1:48 PM, Mark McLoughlin wrote: Hey, On Thu, 2012-06-21 at 13:21 -0700, Devin Carlen wrote: Hey all, We've had a number of stable/essex reviews up that were abandoned due to lack of reviews. We have re-enabled them and are hoping to get some eyes on these so we can release 2012.1.1: https://review.openstack.org/#/c/7718/ https://review.openstack.org/#/c/7723/ https://review.openstack.org/#/c/7729/ https://review.openstack.org/#/c/7731/ Ok, looking now. What about this one? https://review.openstack.org/#/c/7730/ Cheers, Mark. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nodejs in horizon
Long story short - we will work to make node.js an optional build time component and leave it as an distro packaging issue. node.js was being evaluated as a potential solution to https://blueprints.launchpad.net/horizon/+spec/realtime-communication, but that blueprint isn't targeted for Folsom, so it's very future. We'll have a lot of time to evaluate python based alternatives. Devin On May 29, 2012, at 10:26 AM, Adam Young wrote: On 05/29/2012 12:29 PM, Gabriel Hurley wrote: Without rehashing backstory which is available in public archives of this thread, while node is currently on the table for LESS it also may play a role in future needs as well. As for your link, yes there are LESS compilers in other languages (there's even a nascent one in Python that's very much not ready for prime time) but requiring PHP or any other language besides Python opens a similar can of worms, and PHP gets us nowhere on any other fronts. /me refrains from any other comments about PHP. Moreover, the canonical LESS compiler which is the best supported and most stable is the main one written for node.js. Unless there were a 100% working Python version I see no reason not to favor the real LESS compiler. All the best, For LESS, it think it is fine to use even server side Javascript. The CSS should be compiled at RPM/DEB build time, and not at run time for live deployments, so that is a bit of a non-issue. I'd also be fine with using the client side LESS implementation, especially if we want to use the same implementation at development and live deployment time, but I can understand if there are issues with doing this. Node.js is a whole different server side technology, and that should not be implemented at this time. - Gabriel On May 29, 2012, at 1:00 AM, Matthias Rungemru...@matthias-runge.de wrote: On 28/05/12 16:21, Thierry Carrez wrote: John Postlethwait wrote: Sorry if I've missed anything below, this thread has become rather fragmented and messy (at least in my email clients) but I will try to address the main points I have seen so far: Sorry, if I jump in late in this thread, I may have skipped some basics. If I get it right, nodejs is just required to compile LESS to css, right? There is at least one alternative without requiring nodejs: https://github.com/leafo/lessphp -- Matthias Rungemru...@matthias-runge.de ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nodejs in horizon
Hi Joshua, Node.js is in the standard repos for most modern distros. It's not an issue for Ubuntu/Fedora. We are using Node.js for a package called Less with does asset compression for us. Less is Apache 2 licensed so we have included it directly within Horizon: https://blueprints.launchpad.net/horizon/+spec/transition-to-lesscss Relying on Node.js actually opens up a lot of possibilities in the future for us to do realtime websocket communications via node.js and still rely on django to do the heavy lifting: https://blueprints.launchpad.net/horizon/+spec/realtime-communication All this aside, I'm a bit confused why you are asking us to justify a new dependency. (We don't have to.) What is your actual concern here? We are talking about a single package, and a single line patch to update devstack. Thanks, Devin On May 24, 2012, at 10:33 AM, Joshua Harlow wrote: Hi all, I was seeing that node.js is now being used in horizon. Is there any details on why that was needed, the reasoning, the technical docs on where it is used. Are there packages available in fedora/ubuntu for this? Such a change seems like it should have a little more reasoning/explanation that what I found @ https://github.com/openstack-dev/devstack/commit/0c2891558122aa9d030811109536caf5c81cfb75 or https://blueprints.launchpad.net/horizon/+spec/transition-to-lesscss Do we really need to have that ?? :-/ -Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nodejs in horizon
-1 to introducing formal processes around this. This will happen from time to time. Development may be briefly impacted on other platforms but hindering innovation and telling developers that they are responsible for package availability across every distro is not healthy. You are concerned about Fedora which is a perfectly reasonable stance. But what about CentOS, RHEL, Debian, etc. I for once don't want to have my technical options limited by whether or not a package is currently maintained by a distro. We have a lot of reach into these communities and we are still in the earliest stages of Folsom. There is plenty of time to deal with this. That said, this is why we made this change in the *first* milestone, and not a week before release. ;) Devin On May 24, 2012, at 4:09 PM, Joshua Harlow wrote: That’s what I was worried about, and why I (just my opinion) think that we need to be a lot stricter about vetting new dependencies. There needs to be time given to say, ensuring that its really needed, if it really is, documenting why it has to be there in depth, getting various PTL’s to agree on that, then comes the process of informing and/or discussing with the distributions about how to get that supported (if it isn’t). That second stage might influence the first and so on. Since problems like this, although yes, it slows down development overall, seem necessary in a larger project with multiple distributions being enabled (for development usage and for actual prod. usage). On 5/24/12 3:38 PM, Russell Bryant rbry...@redhat.com wrote: On 05/24/2012 03:40 PM, Devin Carlen wrote: Hi Joshua, Node.js is in the standard repos for most modern distros. It's not an issue for Ubuntu/Fedora. It actually is a problem for Fedora. node.js is not in Fedora. Once Horizon requires node.js, it will be broken for Fedora (and EPEL for use with RHEL and its derivatives). https://bugzilla.redhat.com/show_bug.cgi?id=815018 -- Russell Bryant ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack-poc] 3rd Party APIs
+1, primarily by process of elimination. The other options seem either too permissive or too strict. I think our job is to provide a way for the ecosystem to develop and give people a place and category for these projects to live, but not to micromanage every piece of the ecosystem. Devin On May 2, 2012, at 9:50 PM, Joshua McKenty wrote: I'm a fan of c), where the officialness is tied to a committed organization or team that is keeping the code up-to-date and tested. I'd also be a fan of making that a per-release designation, with an easy renewal if the commitment is still in place. Generally, a smaller core with a supported status for satellite projects is my favorite model, for much of OpenStack development. -- Joshua McKenty, CEO Piston Cloud Computing, Inc. w: (650) 24-CLOUD m: (650) 283-6846 http://www.pistoncloud.com Oh, Westley, we'll never survive! Nonsense. You're only saying that because no one ever has. On Wednesday, May 2, 2012 at 3:02 PM, Vishvananda Ishaya wrote: We discussed the policy on third party apis this week at the PPB meeting. We decided to take it to the mailing list for discussion so we can get to some reasonable things to vote on in next weeks meeting. tl; dr How do third party apis fit in OpenStack? Background This was inspired by the current proposals for OCCI and CDMI into nova and swift and the upcoming work and proposals for CIMI for nova. The basic question is: does this code belong in the core repositories and if not, where does it go. I see a number of groups with interest in this. I'm going to outline the major players and give my (biased) opinion on what they want a) Core Developers: would prefer to have these apis outside of core. It is already a burden to maintain the existing apis, so separating these into separate projects would be beneficial. b) Standards Bodies/Developers: would prefer to have some recognition/discoverability for the new apis, currently the only path forward is to be in core, so they are pushing to be included, but they might be ok with some other type of recognition. c) Deployers/Distributors: want an easy way to know that these external plugins work well. This can be accomplished by testing/etc. Probably don't really care too much about the new apis unless they get specific customer requests d) Users: some users (scientific community) would love to have access to these other apis. From a user perspective, the more apis the better, as long as they are stable and all work. Current Proposals a) ppb doesn't care and the projects decide individually b) third party apis are not part of openstack core, and we focus on building a strong ecosystem where these apis could exist as proxies or external plugins. It is up to deployers to decide which ecosystem projects to include in their distributions c) just like b, but there is additionally a process by which these third party tools could become 'official' in some sense or be 'recommended' for inclusion by the distros. d) third party standards are vetted for inclusion by the ppb and are added to core projects assuming they can pass certain testing requirements e) we have our own api, so we shouldn't be encouraging 3rd party apis at all. Tney are on their own. f) ??? Please discuss, Vish ___ Mailing list: https://launchpad.net/~openstack-poc Post to : openstack-poc@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-poc More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack-poc Post to : openstack-poc@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-poc More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack-poc Post to : openstack-poc@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack-poc More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [Nova] removing nova-direct-api
+1 It doesn't have to go home but it can't stay here. On Apr 9, 2012, at 12:31 PM, Kevin L. Mitchell wrote: On Mon, 2012-04-09 at 11:58 -0700, Vishvananda Ishaya wrote: +1 to removal. I just tested to see if it still works, and due to our policy checking and loading objects before sending them into compute.api, it no longer functions. Probably wouldn't be too hard to fix it, but clearly no one is using it so lets axe it. Also +1 for removal. I discovered this thing when I was first trying to figure out how the API worked, and it confused me no end… -- Kevin L. Mitchell kevin.mitch...@rackspace.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [OpenStack Foundation] Foundation Structure: An Alternative
Mark, one thing that I am curious about - there doesn't seem to be language in the foundation structure document that would prevent individual members from organizations that are also strategic members from also being on the board. Is there any language that prevents board stacking? Devin On Friday, March 9, 2012 at 2:27 PM, Mark Collier wrote: Josh, Thanks for taking the time to consider the proposed structure and provide your thoughts on it. I don't think your concerns are very far off from what we are wanting to address with the Associate Member category, which allows more companies to be represented at a lower price point. The current funding proposal is on the wiki for reference: http://wiki.openstack.org/Governance/Foundation/Funding Let me address a few of your points regarding the proposed plan: Accessibility: Associate Member fees start at $50k/year which is substantially lower than a $200k/year flat fee model. The members will elect resprenstatives as a class to the Board. Those Board members will have the same roles and responsibilities as any other Board member. Additionally, companies who wish to show their support for OpenStack may also become a sponsor without paying the full Associate Member fee. Sponsor levels have not been worked out, but I suspect they will start in the 20k/year range. Board Size: My sense is that with a flat fee of 200k/year, we would still get many more than 15-20 interested parties, which would quickly overwhelm the board, making it unmanageable. The Associate Member concept mitigates this issue by allowing the class to grow (for accessibility), without the seats balooning, by having elected representatives. It also reduces the risk of shaking up the board makeup and size through acquisition. Strategic Member fees: One small clarification: the proposal calls for Strategic Members to make a commitment of $500k/year for 5 years, paid annually (not up front). This figure was driven primarily by the need to arrive at a reasonable board size, while also raising substantial funds for foundation operations. Strategic Member dominance: Board members who are appointed by Strategic Members will make up only 1/3 of the board, and will have the same roles and responsibilties as the other board members. Requirement for Strategic Members to have full time staff: The current structure proposal states that Strategic Members must Provide dedicated resources (e.g. developers, legal resources) and the funding proposal states that ...the general expectation is a resource commitment that is consistent with staffing two full time equivalents so it looks like we are on the same page re: 2 FTE requirement. I hope many other folks take the opporunity to weigh in, both on this mailing list as well as in the webinars we'll be holding next week and in any other forum that suits your fancy. I'll also try to keep up with all of the blogging and tweeting, as if that's possible! Mark From: Joshua McKenty jos...@pistoncloud.com (mailto:jos...@pistoncloud.com) Date: Fri, 9 Mar 2012 11:20:17 -0800 To: OpenStack openstack@lists.launchpad.net (mailto:openstack@lists.launchpad.net), foundat...@lists.openstack.org (mailto:foundat...@lists.openstack.org) Subject: [OpenStack Foundation] Foundation Structure: An Alternative I'll be the first to admit that I'm not a diplomatic person. When we were launching OpenStack, this was a bit of an advantage (if we had waited for permission before releasing the Nova source code, we'd still be waiting) - but since the first summit, the community has grown so quickly, and become so diverse, that I have tried to leave discussions of governance, foundation structure, dispute resolution, and most particularly monetary corporate contributions, to others with more... tact. But now I feel I have no choice but to speak up; I'm deeply concerned. The biggest, splashiest openstack stories of the past two years have all had the names of huge, multi-national corporations in them - names like IBM, ATT, Dell, HP, and CISCO. And while their participation has been tremendously positive for the project (with Quantum and Crowbar standing as examples of this), I see things trending in a direction that makes me nervous for the smaller players - for the startups who will live or die on the strength of the OpenStack project. Like Piston Cloud. The current official proposal for the foundation creates a new class of super-members - with a sticker price of $2.5M (due up front) that puts it out of reach of all but a small handful of organizations. This is not a new idea - it was the first structural proposal for the foundation that I heard from the organizing team, and I have argued against it (at times seemingly successfully) continuously since last fall. I understand why it is appealing; it creates a small and
Re: [Openstack] eventlet weirdness
If the libvirt API (or other Native API) has an async mode, what you can do is provide a synchronos, python based wrapper that does the following. register_request callback() async_call() sleep() This can be set up like a more traditional multi-threaded model as well. You can eventlet.sleep while waiting for the callback handler to notify the greenthread. This of course assumes your i/o and callback are running in a different pthread (eventlet.tpool is fine). So it looks more like: condition = threading.Condition() # or something like it register_request_callback(condition) async_call() condition.wait() I found this post to be enormously helpful in understanding some of the nuances of dealing with green thread and process thread synchronization and communication: http://blog.devork.be/2011/03/synchronising-eventlets-and-threads.html Devin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] eventlet weirdness
As long as we allocate a thread in the eventlet thread pool for the number of mysql connections we want to actually maintain in our connection pool, we shouldn't have problems getting the results we want even with the blocking mysql c drivers. Devin On Thursday, March 1, 2012 at 5:23 PM, Kapil Thangavelu wrote: The standard python postgresql driver (psycopg2) does have an async mode. There are non db api compliant async mysql drivers for gevent. Excerpts from Vishvananda Ishaya's message of 2012-03-01 15:36:43 -0500: Yes it does. We actually tried to use a pool at diablo release and it was very broken. There was discussion about moving over to a pure-python mysql library, but it hasn't been tried yet. Vish On Mar 1, 2012, at 11:45 AM, Yun Mao wrote: There are plenty eventlet discussion recently but I'll stick my question to this thread, although it's pretty much a separate question. :) How is MySQL access handled in eventlet? Presumably it's external C library so it's not going to be monkey patched. Does that make every db access call a blocking call? Thanks, Yun On Wed, Feb 29, 2012 at 9:18 PM, Johannes Erdfelt johan...@erdfelt.com (mailto:johan...@erdfelt.com) wrote: On Wed, Feb 29, 2012, Yun Mao yun...@gmail.com (mailto:yun...@gmail.com) wrote: Thanks for the explanation. Let me see if I understand this. 1. Eventlet will never have this problem if there is only 1 OS thread -- let's call it main thread. In fact, that's exactly what Python calls it :) 2. In Nova, there is only 1 OS thread unless you use xenapi and/or the virt/firewall driver. 3. The python logging module uses locks. Because of the monkey patch, those locks are actually eventlet or green locks and may trigger a green thread context switch. Based on 1-3, does it make sense to say that in the other OS threads (i.e. not main thread), if logging (plus other pure python library code involving locking) is never used, and we do not run a eventlet hub at all, we should never see this problem? That should be correct. I'd have to double check all of the monkey patching that eventlet does to make sure there aren't other cases where you may inadvertently use eventlet primitives across real threads. JE ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net (mailto:openstack@lists.launchpad.net) Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net (mailto:openstack@lists.launchpad.net) Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net (mailto:openstack@lists.launchpad.net) Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Announcing StackTach ...
Sandy, this is great work! I think it would be worth integrating this into a view in Horizon for Folsom timeframe. Devin On Monday, February 20, 2012 at 12:15 PM, Sandy Walsh wrote: Hey! Last week I started on a little debugging tool for OpenStack based on AMQP events that I've been calling StackTach. It's really handy for watching the flow of an operation through the various parts of OpenStack. It consists of two parts: 1. The Worker. Sits somewhere on your OpenStack network. It listens to AMQP monitor.* notifications and sends them to the StackTach server. (I need this branch to land for it to work ... hint hint) https://review.openstack.org/#change,4194 2. The Web Interface Collects events via REST calls (poorman multi-tenant) and presents these events in a funky little web interface. You can play around with the UI here: http://darksecretsoftware.com/stacktach/1/ (this is with data coming from my personal OpenStack Dev env) What do I do? Click on anything and you'll see the particulars in the Details window. Click on [+] to see the JSON for the event. Hosts shows the last 20 events that have a Host defined. Instances shows the last 20 events that have the Instance field populated. Hosts and Instances windows are resize-able. You may see duplication between both windows. Click on Time to see any events around that time (+/- 1 minute I think) Where is the code? The code is hosted below. There's LOTS of work to do to make it ready for prime-time ... but please, contribute. https://github.com/rackspace/stacktach How do I install it? I need to make this process cleaner. Right know you need to know how to create a Django Project and stick StackTach in there. Look forward to the feedback. Cheers, Sandy ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net (mailto:openstack@lists.launchpad.net) Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Keystone: Redux (Dubstep Remix)
I agree fully with Jesse. I think given the timelines the first cut of Keystone was great. Moving forward we'll also have more folks that are burdened (honored?) with operating it in production environments which means that more rubber meets the road kinds of issues will be identified and dealt with quickly. Keystone was originally pushed into core a bit too soon, but this was simply a byproduct of the fact that the need for it was very real. All the groundwork done to unify the projects was a huge benefit to the community. Before that point, most of the time when someone was working on OpenStack, they were really just working on one individual component in an isolated environment. Keystone forced the issue for the community and led to the creation of the fabulous DevStack project. This integration made it far simpler to do integration testing, and projects like Tempest greatly benefited from it as well. Devin On Tuesday, February 14, 2012 at 6:20 PM, Jesse Andrews wrote: The major lessons of keystone: While keystone served as an effective proof of concept for unified authentication (before keystone each component had its own users/passwords), it didn't get enough attention from other developers and integration with other core projects. The pain caused by not having shared authentication caused it to grow up too fast. Keystone was in incubation during Diablo and is scheduled for official core at the Essex release. Going forward when something is added to core we need to make sure it has the project wide support necessary to present a consistent openstack during the transition and afterwards. As an example, before quantum becomes a core project we are documenting what becomes of Nova network and existing APIs. Glance integration into nova was a good example where the image list API call proxies to glance. Additional if the code is vastly different, it is harder to get existing contributors to participate. The original keystone team had a hard task and didn't get enough time and help due to circumstances (some within their control and some not) Jesse ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Instances with status error and what to do with them n the UI
The obvious answer is to have Horizon show whatever nova-client would show here. However, given that these things now stay around forever, I would argue that the functionality within nova-client itself is suspect and that perhaps there should be a flag to show instances/volumes with an error state. Otherwise something that fails just stays around forever, which is certainly not desirable. Devin On Saturday, January 21, 2012 at 10:29 AM, Tres Henry wrote: While trying to debug boot-from-volume I've ended up with instances (and volumes for that matter) in various states of failure. Currently Horizon simply displays all instances regardless of state (see attached ss). Horizon can filter out instances in an error state very simply, however, that is not necessarily the desired user experience. For example if a user attempts to launch a new instance and the instance goes into an error state before Horizon can render the instances page. In that case if we filtered out instances in an error state the user would never see the instance. It would appear as though nothing happened. This issue exists for any asynchronous process which could result in an object in the system with an unexpected state. This issue also impacts every client (not just Horizon). My question to the greater community is this: how should OpenStack present objects in unexpected states consistently so that every client (Horizon, CLI, third-party tool, etc) provides a consistent user experience? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net (mailto:openstack@lists.launchpad.net) Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp Attachments: - error_instances.png ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Horizon and translation timing
Hello all, We've been discussing how best to handle accepting translation changes since we are doing a fairly substantial reworking of the OpenStack Dashboard. We have decided to do a string freeze when the essex-4 milestone closes and then to rebuild translation files during the integrated-freeze period in between the essex-4 milestone and release. We have had an unusually large number of changes during this cycle, but we want to do the best we can to make sure language support is as complete as possible. Therefore if you are interested in translations, please plan on contributing updates between 2012-03-01 and 2012-03-15. I know this is a small window but this will prevent us from having to repeat the work as various pieces are still implemented. Best, Devin___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] using objects returned from DB layer
Yes, we should absolutely push to make this more consistent across the board. We are severely limiting possible future implementations. So, yea. +1! Devin On Dec 14, 2011, at 11:10 PM, Chris Behrens wrote: I've seen a number of patches lately that have code like this: instance = db.instance_get(...) instance_uuid = instance.uuid instead of: instance_uuid = instance['uuid'] There's a mix of usage throughout the code, and I know some people are just matching the surrounding code. But, in a number of cases, I've asked for these to be corrected to the latter, on assumption that the DB layer will be returning dictionaries at some point vs the models. It also pushes the code towards consistent usage. But I might be the only Nova Core member looking at this and/or maybe my assumption is wrong. So, I ask here: Should Nova Core make an effort to reject patches with the former format? Or did I miss any DB layer plans where the former format is now preferred? - Chris ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] OpenStack Dashboard Gerrit/Github Transition
Hello all, We have officially completed the Gerrit/Github transition for Horizon. This is the last time we'll have to move things for a while. I promise. :) The official Horizon repo is now at: https://github.com/openstack/horizon Gerrit workflow instructions are here: http://wiki.openstack.org/GerritWorkflow Thanks to James Blair, Thierry Carrez, and Monty Taylor for all their help lately. Best, Devin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Dashboard Gerrit/Github Transition
This will be fixed shortly. Until then we do have tags: https://github.com/openstack/horizon/tags Thanks, Devin On Oct 28, 2011, at 1:53 PM, Kiall Mac Innes wrote: Hi Devin, Should we expect a diablo/stable branch? I'm only asking because the old branches are now gone. Thanks, Kiall On Oct 28, 2011 9:48 p.m., Devin Carlen devin.car...@gmail.com wrote: Hello all, We have officially completed the Gerrit/Github transition for Horizon. This is the last time we'll have to move things for a while. I promise. :) The official Horizon repo is now at: https://github.com/openstack/horizon Gerrit workflow instructions are here: http://wiki.openstack.org/GerritWorkflow Thanks to James Blair, Thierry Carrez, and Monty Taylor for all their help lately. Best, Devin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] OpenStack Dashboard Cactus tag
Hello all, When we transitioned from Launchpad to Github, we didn't carry over the tag for Cactus. I've had a few people express interest lately in continuing to use it, so we have retagged for Cactus. You can grab it here if you need: https://github.com/4P/horizon/zipball/2011.2 Devin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] OpenStack Dashboard Update
Hello all, Milestones and feature roadmaps are now available at: https://blueprints.launchpad.net/openstack-dashboard Also we have chosen the official project codename, Horizon. We will be moving the official repo to the OpenStack GitHub account. In the interim, we have renamed the currently official repo. This is temporary while we make the transition. https://github.com/4P/horizon In the next few days we will be migrating to the Gerrit workflow as well. More updates soon as we make this migration. Thanks, Devin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Creating openstack-dev mailing list
+1 Devin On Oct 7, 2011, at 7:16 AM, John Dickinson m...@not.mn wrote: I would prefer to keep the list together. I do not think the volume of either dev or user conversations is too oppressive to the other, and I would like to avoid wither group being neglected by the other. Sent from my iPhone On Oct 7, 2011, at 2:16 AM, Wayne A. Walls wa...@openstack.org wrote: I remember having this discussion at the Cactus design summit, and the reoccurring theme was dev vs sysads/deployment. I was in favor of a split list then, but over the past year my stance has changed a bit. I don't know if I feel that OpenStack has reached the level where deployment knowledge is absent of dev knowhow. Deployers of OS are typically in irc/ml/forums pasting stack traces, working through problems with dev help. Once we start seeing more public deployments, reference architectures, etc, then maybe? Do devs think there is too much noise on the main ml? What has the impact been on splitting the irc channels out? To me, I just idle in two places now, but did that really have a big impact? I tend to agree with John's points, I don't want the presumably easiest ml to consume to be neglected by some of the brightest minds in the project. Thanks, Wayne Sent from my iPhone On Oct 6, 2011, at 10:11 PM, John Purrier j...@openstack.org wrote: My 2 cents... The traffic on the list is less than 100 messages per day, of that about 35% is bug notifications. I wonder if we redirect the developer oriented email on the list whether we will have 10 messages a day on the original openstack mailing list. Stefano, can you elaborate on why the developers feel a split is necessary at this point? Most (if not all) of the traffic is developer oriented, what is the problem we want to solve? Thanks, John -Original Message- From: openstack-bounces+john=openstack@lists.launchpad.net [mailto:openstack-bounces+john=openstack@lists.launchpad.net] On Behalf Of Stefano Maffulli Sent: Thursday, October 06, 2011 1:35 PM To: openstack Subject: [Openstack] Creating openstack-dev mailing list Jesse Andrews (anotherjesse)(Rackspace) Jonathan Bryce (jbryce)(Rackspace) Devin Carlen (devcamcar)(Nebula) Thierry Carrez (ttx)(Rackspace) John Dickinson (notmyname)(Rackspace)) Vish Ishaya (vishy)(Rackspace) Josh Kearney (jk0)(Rackspace) Joshua McKenty (jmckenty)(Piston) Ewan Mellor (ewanmellor)(Citrix) Jay Pipes (jaypipes)(Rackspace) John Purrier (johnpur)(HP) Monty Taylor (mordred)(Rackspace) Paul Voccio (pvo)(Rackspace) Ziad Sawalha (zns)(Rackspace) Hello folks, I've been talking to quite a few developers participating to the Design Summit and a recurring request I got is to create a new mailing list for the OpenStack developers to meet and discuss. There is also a concern that putting developers in another list will decrease their attention to the bigger part of the community. I believe this is a serious concern and that it's going to be our role as leaders of this community to prevent this from happening. I'd suggest to dedicate the existing mailing list for discussions about usage of OpenStack (deployment and development of applications on top of OpenStack API) and create a new one only for developers of OpenStack. Developers in this context should be developers of openstack projects (nova, swift, quantum, etc). If there is no opposition to this proposal in the next days, I'll proceed and create openstack-dev and invite developers to subscribe to it. cheers, stef ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] OpenStack Dashboard Diablo branch
Hello all, There is now an official Diablo branch on GitHub: https://github.com/4p/openstack-dashboard/tree/diablo I'll be sending out a more formal note about this later. Thanks, Devin___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] dashaboard+keystone+nova+glance work well？
There are a few bugs specific to admin functions in Keystone. It's about 95% there now and when those last issues are resolved we will be back in business. Best, Devin On Sep 28, 2011, at 6:37 PM, andi abes wrote: is this just to avoid the need for the long lived token, or are there other issues in dash/keystone integration? 2011/9/28 Devin Carlen devin.car...@gmail.com We should have a drop of Keystone by the end of the week that fully support Diablo. This will fix the Dashboard issues as well. Thanks, Devin On Sep 28, 2011, at 12:31 AM, Jesse Andrews wrote: at various points in time they have worked together. We (cloudbuilders) keep a list of repositories that work well together. # compute service NOVA_REPO=https://github.com/openstack/nova.git NOVA_BRANCH=2011.3 # image catalog service GLANCE_REPO=https://github.com/cloudbuilders/glance.git GLANCE_BRANCH=diablo # unified auth system (manages accounts/tokens) KEYSTONE_REPO=https://github.com/cloudbuilders/keystone.git KEYSTONE_BRANCH=diablo # a websockets/html5 or flash powered VNC console for vm instances NOVNC_REPO=https://github.com/cloudbuilders/noVNC.git NOVNC_BRANCH=master # django powered web control panel for openstack DASH_REPO=https://github.com/cloudbuilders/openstack-dashboard.git DASH_BRANCH=master # python client library to nova that dashboard (and others) use NOVACLIENT_REPO=https://github.com/cloudbuilders/python-novaclient.git NOVACLIENT_BRANCH=master # openstackx is a collection of extensions to openstack.compute nova # that is *deprecated*. The code is being moved into python-novaclient nova. OPENSTACKX_REPO=https://github.com/cloudbuilders/openstackx.git OPENSTACKX_BRANCH=diablo On Tue, Sep 27, 2011 at 7:11 PM, shake chen shake.c...@gmail.com wrote: No， Now the Dashbaord can not working. https://bugs.launchpad.net/openstack-dashboard/+bug/855142 I think need the bug beed fixed. On Wed, Sep 28, 2011 at 9:31 AM, l jv ljv...@gmail.com wrote: hi is there anybody config dashaboard+keystone+nova+glance sucess and work well? when i do as the http://docs.openstack.org/,but it does not work well,always has some wrong when use dashboard(glance and nova work well). So somebody can write a detail config process doc ？ thans a lot ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- 陈沙克 手机：13661187180 msn：shake.c...@hotmail.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] dashaboard+keystone+nova+glance work well？
We should have a drop of Keystone by the end of the week that fully support Diablo. This will fix the Dashboard issues as well. Thanks, Devin On Sep 28, 2011, at 12:31 AM, Jesse Andrews wrote: at various points in time they have worked together. We (cloudbuilders) keep a list of repositories that work well together. # compute service NOVA_REPO=https://github.com/openstack/nova.git NOVA_BRANCH=2011.3 # image catalog service GLANCE_REPO=https://github.com/cloudbuilders/glance.git GLANCE_BRANCH=diablo # unified auth system (manages accounts/tokens) KEYSTONE_REPO=https://github.com/cloudbuilders/keystone.git KEYSTONE_BRANCH=diablo # a websockets/html5 or flash powered VNC console for vm instances NOVNC_REPO=https://github.com/cloudbuilders/noVNC.git NOVNC_BRANCH=master # django powered web control panel for openstack DASH_REPO=https://github.com/cloudbuilders/openstack-dashboard.git DASH_BRANCH=master # python client library to nova that dashboard (and others) use NOVACLIENT_REPO=https://github.com/cloudbuilders/python-novaclient.git NOVACLIENT_BRANCH=master # openstackx is a collection of extensions to openstack.compute nova # that is *deprecated*. The code is being moved into python-novaclient nova. OPENSTACKX_REPO=https://github.com/cloudbuilders/openstackx.git OPENSTACKX_BRANCH=diablo On Tue, Sep 27, 2011 at 7:11 PM, shake chen shake.c...@gmail.com wrote: No， Now the Dashbaord can not working. https://bugs.launchpad.net/openstack-dashboard/+bug/855142 I think need the bug beed fixed. On Wed, Sep 28, 2011 at 9:31 AM, l jv ljv...@gmail.com wrote: hi is there anybody config dashaboard+keystone+nova+glance sucess and work well? when i do as the http://docs.openstack.org/,but it does not work well,always has some wrong when use dashboard(glance and nova work well). So somebody can write a detail config process doc ？ thans a lot ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- 陈沙克 手机：13661187180 msn：shake.c...@hotmail.com ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Glance client release?
The old ec2 based admin client for nova was in a separate repo. It's a bit tricky with pypi to do multiple distributions if you have a top level setup.py. There is a cleaner way to do it but it requires restructuring the top level structure a bit. Something like: server/ glance/ glance/client/ ... setup.py client/ setup.py So client only contains the setup.py to push the client, and server (or whatever) contains all the actual bits. I'm not a total wizard with pypi and setup.py, so maybe there's a less painful way to do this? Devin On Sep 28, 2011, at 11:56 AM, Jesse Andrews wrote: I know Sandy Walsh did novaclient and Devin Carlen did (the deprecated) adminclient for nova. They might be able to pitch in and help (irc?) On Wed, Sep 28, 2011 at 8:26 AM, Jay Pipes jaypi...@gmail.com wrote: We should be able to do that, yes. I have to figure out how to do it, but I will create a bug for it in Launchpad and track progress. Cheers, jay On Wed, Sep 28, 2011 at 11:05 AM, Huang Zhiteng winsto...@gmail.com wrote: I am also looking forward to python-glance being available on PyPI. Will it be released with Diablo? On Sat, Sep 24, 2011 at 3:02 AM, Jay Pipes jaypi...@gmail.com wrote: On Fri, Sep 23, 2011 at 2:51 PM, Devin Carlen devin.car...@gmail.com wrote: Awesome, thanks! Any plans to have the client available on PyPI? Makes testing a lot easier. Yup! -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp -- Regards Huang Zhiteng ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Nova DB Connection Pooling
We really need to hear from Sandy Walsh on this thread so he can elaborate on how the distributed scheduling works (with multiple mysql databases). Devin On Sep 26, 2011, at 6:41 AM, Soren Hansen wrote: 2011/9/26 Pitucha, Stanislaw Izaak stanislaw.pitu...@hp.com: The pain starts when your max memory usage crosses what you have available. Check http://dev.mysql.com/doc/refman/5.1/en/memory-use.html - especially comments which calculate the needed memory for N connections for both innodb and isam. (mysqltuner.pl will also calculate that for you) Hundreds of connections should be ok. Thousands... you should rethink it ;) Hm.. It doesn't take many racks full of blade servers to get into 4 digit numbers of compute nodes. Certainly fewer than I was expecting to see in a garden variety Nova zone. -- Soren Hansen| http://linux2go.dk/ Ubuntu Developer| http://www.ubuntu.com/ OpenStack Developer | http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] [openstack]diablo release dashboard error[Could not import dashboard.views. Error was: No module named kombu.connection]
Hello, can you please post this as a question on launchpad? https://answers.launchpad.net/openstack-dashboard Thanks, Devin On Sep 23, 2011, at 3:33 AM, darkfower wrote: Hi, my open dasboard url ,error : ViewDoesNotExist at / Could not import dashboard.views. Error was: No module named kombu.connection Request Method: GET Request URL: http://60.12.206.111:8000/ Django Version: 1.3 Exception Type: ViewDoesNotExist Exception Value: Could not import dashboard.views. Error was: No module named kombu.connection Exception Location: /opt/openstack-dashboard/openstack-dashboard/.dashboard-venv/local/lib/python2.7/site-packages/django/core/urlresolvers.py in _get_callback, line 167 Python Executable: /opt/openstack-dashboard/openstack-dashboard/.dashboard-venv/bin/python Python Version: 2.7.1 Python Path: ['/opt/openstack-dashboard/openstack-dashboard/dashboard', '/opt/openstack-dashboard/openstack-dashboard/.dashboard-venv/local/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg', '/opt/openstack-dashboard/openstack-dashboard/.dashboard-venv/local/lib/python2.7/site-packages/pip-1.0.2-py2.7.egg', '/opt/openstack-dashboard/openstack-dashboard/.dashboard-venv/src/openstack', '/opt/openstack-dashboard/openstack-dashboard/.dashboard-venv/src/openstackx', '/opt/openstack-dashboard/django-openstack', '/opt/openstack-dashboard/openstack-dashboard/.dashboard-venv/local/lib/python2.7/site-packages/mox-0.5.3-py2.7.egg', '/opt/openstack-dashboard/openstack-dashboard/.dashboard-venv/local/lib/python2.7/site-packages/xattr-0.6.2-py2.7-linux-x86_64.egg', '/opt/openstack-dashboard/openstack-dashboard/.dashboard-venv/lib/python2.7', '/opt/openstack-dashboard/openstack-dashboard/.dashboard-venv/lib/python2.7/plat-linux2', '/opt/openstack-dashboard/openstack-dashboard/.dashboard-venv/lib/python2.7/lib-tk', '/opt/openstack-dashboard/openstack-dashboard/.dashboard-venv/lib/python2.7/lib-old', '/opt/openstack-dashboard/openstack-dashboard/.dashboard-venv/lib/python2.7/lib-dynload', '/usr/lib/python2.7', '/usr/lib/python2.7/plat-linux2', '/usr/lib/python2.7/lib-tk', '/usr/lib64/python2.7/lib-tk', '/opt/openstack-dashboard/openstack-dashboard/.dashboard-venv/local/lib/python2.7/site-packages', '/opt/openstack-dashboard/openstack-dashboard', '/opt/openstack-dashboard/openstack-dashboard/dashboard', '/opt/openstack-dashboard/openstack-dashboard/dashboard'] Server time: 星期五, 23 九月 2011 17:53:33 +0800 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Glance client release?
Awesome, thanks! Any plans to have the client available on PyPI? Makes testing a lot easier. Best, Devin On Sep 23, 2011, at 11:44 AM, Jay Pipes wrote: Hey Devin, There is actually already a python-glance package that is intended to be only the client (glance.client), but the dependencies for the main glance packages servers + client seem to have crept into the python-glance package... I'll have a look into it. Cheers, -jay On Fri, Sep 23, 2011 at 2:13 PM, Devin Carlen devin.car...@gmail.com wrote: Hey Jay and glancers, Do you have plans to package the glance client separately? We keep running into issues on the dashboard side with a few of the glance dependencies. Logically there is no reason to have to install all of glance just to get to the client. This would really help to simplify things on our end. Best, Devin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Proposal to add Chris Behrens to nova-core
+1! On Sep 15, 2011, at 2:49 PM, Soren Hansen wrote: 2011/9/15 Brian Waldon brian.wal...@rackspace.com: Chris (comstud) has been a great help with reviews lately and has quite a bit of knowledge of the system overall. I think he would add a great amount to the team, so I'm proposing we add him to nova-core. I'm shocked he's not nova-core already. +1 -- Soren Hansen| http://linux2go.dk/ Ubuntu Developer| http://www.ubuntu.com/ OpenStack Developer | http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Diablo 4 dashboard error Unable to get usage info: (OperationalError) no such column: vm_state
It may help you to know that the issue you are having is a Nova issue, not a Dashboard issue. You should open a question on launchpad.net/nova and paste your full stack trace. On Sep 13, 2011, at 9:21 AM, atkisc wrote: hi when i run the diablo-4 dashboard,Has the following error : Unable to get usage info: (OperationalError) no such column: vm_state uselect id,image_ref,project_id,user_id,vcpus,hostname,display_name,host,vm_state,instance_type_id,launched_at,terminated_at from instances where (terminated_at is NULL or terminated_at '2011-09-01 00:00:00') and (launched_at '2011-09-13 15:55:32.288462') and project_id='cloud' () Don't know who met didn't ? 2011-09-14 atkisc ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Dashboard Diablo update
Hi Armando, Actually I misspoke and the volumes branch hasn't landed yet. But it will soon! Sorry for the confusion, Devin On Sep 2, 2011, at 8:02 AM, Armando Migliaccio wrote: Hi Devin, thanks for this update. Below you mentioned support for managing Nova volumes, floating IPs. Can you tell us a bit more about this? I cannot see a volume section in the dashboard UI, are there specific extensions to pull into the code? Thanks, Armando -Original Message- From: openstack-bounces+armando.migliaccio=eu.citrix@lists.launchpad.net [mailto:openstack- bounces+armando.migliaccio=eu.citrix@lists.launchpad.net] On Behalf Of Devin Carlen Sent: 01 September 2011 22:18 To: openstack@lists.launchpad.net Subject: [Openstack] Dashboard Diablo update Hey everyone, I wanted to update everyone on where we've made it so far in the Diablo release. First off, along with the Keystone project, the Dashboard project has been promoted to OpenStack core, meaning it is now an officially recognized part of the OpenStack ecosystem. There are a few outstanding things we need to do to fully integrate with existing processes, though we are pretty far along: * Move code reviews from Github to Gerrit * Synch up on packaging efforts * Redouble efforts on documentation * Pick an official codename for the project I'll be sending out a list of codename candidates soon for us to decide on as a community. :) We put a lot of work into Dashboard in the Diablo timeframe. Some of the highlights: * uses OpenStack API instead of EC2 API for underlying calls * full support for Keystone for authentication * support for managing Swift containers and objects * support for managing Quantum networks and ports * support for managing Nova volumes, floating IPs, etc (mostly got feature parity with EC2 API) * a new look and feel * new features for sysadmins * ... and more! There are three phases to a project's life: 1) make it work (austin/bexar) 2) more is better (cactus/diablo) 3) better is more I feel that as a project we've fulfilled our more is better phase, as in having more features is more important than perfecting each piece. During the Essex release, we're going into our better is more phase. We'll be focusing on user experience and interaction design and making this project a world class web application. We'll be talking quite a bit about this at the design summit as well, so we'll look for you all there! Devin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Dashboard newbie question
Hi Joshua, Dashboard is currently in pretty good shape but the latest nova code in trunk has a few issues. I recommend using the Diablo-3 milestone release of Nova. You should have better luck with that. On Aug 4, 2011, at 10:23 AM, Joshua Harlow wrote: Hi all, I was just wondering if it is recommended or if anyone has been successful in getting the openstack dashboard up and running with the lastest nova code. I seem to be encountering some 400 http responses when say deploying from the dashboard. Is that just because the dashboard isn’t fully integrated yet? Thanks, Josh ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Common
If by mini-projects you mean small and separate projects, then I don't think that makes sense. All we need for this is a single project that contains submodules that don't contain unnecessary dependencies on other submodules within the common project. So lots of bite size pieces that can be used or not used. Devin On Jul 25, 2011, at 10:20 AM, Glen Campbell wrote: Would it better to break it down even further? I.e., instead of trying to put ALL the common code into one project, create mini projects for common-logging, common-configuration, etc. That would permit other projects to adopt what they need, when they need it, rather than trying to integrate the entire common project at once. For example, we're working on converting Glance to use the logging/notification mechanism defined by Nova. The first step in the project was, however, cut and paste notification code from one to the other. That's disturbing to me, because it doubles the amount of effort required to implement changes to the notification system in the future. It would be much better IMHO to have a refactored common-logging module that could be included by multiple projects, with a standard interface each project could rely upon. There's no requirement, then, to implement common-rpc when you integrate common-logging, which lowers the barrier to entry for each project. On 7/25/11 12:11 PM, Brian Lamar brian.la...@rackspace.com wrote: All, I love the idea of having an openstack-common project. However, the prospect of creating such a project is daunting and quite difficult. It's my belief that standardizing/collecting common logic into a single module will be beneficial to our current code-base and allow for future projects to be created more quickly/easily. Currently the behemoth in the room is OpenStack Compute (aka Nova). The Compute project contains much more code than all other OpenStack projects (combined), and rightly so...virtualization is a pretty darn complex thing to do in a flexible way. This might be why other projects have been spawned to take away some of the logic from a single massive project and place that logic into smaller, more focused projects. However, I would argue that the barrier of entry is too high for this strategy to work. Projects such as Quantum, Burrow, Lunr, and Keystone suffer from the lack of a single cohesive strategy in the following areas: -Logging -Configuration -WSGI -RPC -Database -Testing -Deployment/Distribution of code These are the building blocks which most projects will require, yet every project has to create their own implementations? Sure, it's not going to be easy, and maybe some categories I've labeled won't make the final cut, but I would like to start a conversation with the community as to the feasibility of such an endeavor. This is not the first time such a project has been brought up. Much earlier in the year, a number of people suggested moving lazy loading code into a common project. I would like to think that project died due to complexity rather than the community rejecting the idea. To create a common library of this nature requires a bit of pushing aside one's partisan blinders and the abandonment of ideological entrenchments*, so hopefully this won't deteriorate into a FLAGS vs. argparse flame-war. TLDR: No * - Shamelessly stolen from The West Wing (tm) ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp This email may include confidential information. If you received it in error, please delete it. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cross-zone instance identifiers in EC2 API - Is it worth the effort?
Here's a few crazy questions for you guys to consider: 1) Why are we even trying to have the same ID for an instance or image across two different APIs? 2) How many people really switch back and forth between OpenStack and EC2 API once they pick one? 3) How many people really expect euca2ools and python-novatools to return the same IDs for the same instances? 4) Why not just store a EC2 ID and a OS ID alongside an actual row PK? My point with these questions is that very little is gained from forcing two different APIs to try to use the same ID, especially given that the format is different. I would argue that it has created a lot more problems than it has solved. Devin On Jul 8, 2011, at 3:12 PM, Soren Hansen wrote: 2011/7/8 Vishvananda Ishaya vishvana...@gmail.com: Yes they seem to apply to all ids r-XXX ami-XXX i-XXX vol-XXX snap-XXX That said we are using base-36 for the hex in reservation ids and no one has complained, so i don't think they are used by the tools that much. Not true. ElasticFox does not work with this. I don't think we're yet at a point where we can say that lack of complaints about stuff means something's fine. That sort of assumption only starts to be useful when we have lots of real users. The users we have so far are people who care about OpenStack for OpenStack's sake, not random users who are being given access to an OpenStack deployment and being told talk to this like you would talk to EC2. It'll work fine. Different expectations. -- Soren Hansen| http://linux2go.dk/ Ubuntu Developer| http://www.ubuntu.com/ OpenStack Developer | http://www.openstack.org/ ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cross-zone instance identifiers in EC2 API - Is it worth the effort?
On Jul 8, 2011, at 2:19 PM, Vishvananda Ishaya wrote: My hope was that turning ec2 into a compatibility api would force people adding the ec2 features to make them work in the openstack api as well. I really feel like we're failing in producing a cloud standard api if we don't have all of the features. It is quite sad that we are still missing security group support in the os api for example. Yes - and this is a perfect example of why it's important for people to think of the EC2 and OpenStack API's not as mid-layer APIs in some stack. These APIs are actually presentation layers. Currently there is far too much core logic happening in the EC2 API for sure. At this tier, there shouldn't be anything other than data transforming and API specific muck going on. Nova is currently missing a classic middle tier, and this is why it gets difficult to circle back and add support for security groups to the OpenStack API. There's no common layer that both APIs rely on. This way you end up with a middle tier that is a super set of both OpenStack and EC2 APIs, and then exposing functionality in one API or the other is truly just a presentation issue, because by forcing the core to be implemented in a common layer, you force the person(s) implementing the feature to deal with the bigger picture, and not favor one API over the other. That said, I'm not wedded to this approach; I don't mind ec2 talking directly to the service apis. I just want to make sure that we understand only way we are going to get to the goal of a standard ubiquitous os api is if we make sure to add the features there as well. Said another way, I have two goals: Goal 1. OS api becomes a complete standard that supports the entire nova feature set. Goal 2. EC2 api implements as close to amazon's apis as possible and supports as many features as possible without breaking compatibility. I think that there is little disagreement in the goals, it is just how to get there that we are quibbling over. Option 1 EC2 | OS | Internal This gets a huge -1 from me for the exact reasons I described above. Advantage: forces us to add extensions to the OS api to support ec2 features, which makes the OS api more complete. Disadvantage: harder to add features to the EC2 api Option 2 EC2 OS \ / Internal Advantage: easier to add the ec2 features, so we end up with a more feature-rich ec2 api Disadvantage: OS api can get skipped when new features are added Huge +1 here. Like I said above, even if the OS API gets skipped in the initial implementation, that isn't as big of a deal now. All of the core logic exists. Now all someone has to do is expose the data transforms and API specific bits into the OS API. The problem now is for someone to implement security groups in the OS API requires a lot of work because there's a fair amount of actual logic in the EC2 API code. If we gave that logic a better place to live, it's far less painful to expose in multiple APIs. I feel that goal 1 is slightly more important than goal 2 so I'm leaning towards implementation option 1. This is not a strong preference, though. I think with the right focus we can achieve both goals in either scenario. I feel very strongly that option 2 is the right move. Devin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Cross-zone instance identifiers in EC2 API - Is it worth the effort?
+1 Devin On Jul 8, 2011, at 10:29 AM, Lorin Hochstein lo...@isi.edu wrote: On Jul 8, 2011, at 10:36 AM, Sandy Walsh wrote: +1 to Soren's argument that ec2 is the 1000lb gorilla and should be central to nova. We definitely need to support it with as close to 100% compatibility as we can. Sounds like the only option is to embrace and extend it. Do everything it can do, and layer on what we need provided it doesn't break the core EC2 commands. If the customer wants pure EC2, they'll have to live with the limitations. That said, is this the proposal I'm hearing? ... Since our separation is done at the service.api layer, the service.api's get pulled in two directions with each change in ec2/os. The idea is to have ec2 be a translation layer to os api? Preventing ec2 api from calling [service].api directly? So, instead of EC2 Client OS Client | | EC2 APIOS API \ / [Service] API We'd be shifting to: EC2 Client EC2 API | OS Client -- OS API | [Service] API I need to think more about this, but at first blush, it doesn't seem like such a [bad] thing? At some point the abstraction layer needs to be locked down doesn't it? I think it actually looks more like this right now: EC2 Client OS Client | | EC2 APIOS API \ / [nova-*] service APIs There isn't really a single back-end API for the front-end APIs to call into. Instead, each of them makes calls to the multiple service APIs (e.g., scheduler, network, compute). I would advocate for something more like this: EC2 Client OS Client | | EC2 APIOS API \ / internal nova API | [nova-*] service APIs This is a single, unified API that is meant only for internal use. This would reduce the coupling between front-end and back-end. It would make it easier for someone with less expertise in the code (hello!) to find the location in the code that answers questions like: What does nova do when a user requests that an instance is launched? They would just look at the internal API and find the appropriate method. It would also make it easier to add additional front-ends, if there's ever any interest in that. Lorin -- Lorin Hochstein, Computer Scientist USC Information Sciences Institute 703.812.3710 http://www.east.isi.edu/~lorin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Overview of CI/Testing
+1 for chef over debs! On Jun 7, 2011, at 12:38 PM, Andy Smith wrote: Thanks for the update Monty :) That's just testing API in a VM though, and doesn't get us to testing actual bare-metal deployment or integration testing. At Rackspace, we have some machines set aside at the moment, and have had others offer chunks of machines to test various combinations of things. At its heart, the abstract version of this looks fairly identical to the smoketests job - pxe boot machines, shove version to be tested on them, run tests. However, there are several moving bits on the best way to actually do the how. At the moment, the fine folks at rPath have a Jenkins installing and testing rPath OpenStack images, so Mihai and I are going to look at getting that setup ported to our Jenkins. However, although that will be an excellent test of code, as our main target platform is Ubuntu, we're also looking at doing a straight-up cobbler install using generated .debs. Jesse and I had already gotten quite far along using chef to do the provisioning of baremetal boxes once we'd pxe booted them into ubuntu, it seems like chef or puppet (our current preference is chef) should be used there as well instead of generated .debs. At the moment the two closest things to being official installations for us (me? are the chef recipes and the nova.sh script (the nova.sh script obviously being only targeted at testing and dev though), those are what we use to verify that the system is functional and I think we'd like to use chef or puppet for baremetal deployments as well. TL;DR: Can we focus on the chef recipes instead of on .debs? In any case, this is the bit which is still in the planning and discussion phase, but so far all of the conversations I've had with folks have been great - and I'd love to get more folks involved in that (thus this email) However- latent goal here is that whatever mechanism we're having Jenkins use to deploy OpenStack onto real hardware should be consumable and one that actual people might actually use - otherwise what the heck are we testing? Additionally, as you may have surmised, it is also a goal to run as much of this as possible from the OpenStack Jenkins, because that way we can as a project choose to incorporate as much of the feedback/results of various forms of testing directly in to branch testing/approval as we want. For some things (spinning up 20 node OpenStack clusters) doing it on every merge proposal or giving all devs the ability to click a button and have it run on their branch will likely be overkill - but if it turns out not to be, it would be great to have the ability to do it. End goal is to have: - publicly accessible and usable system for testing and build automation - resources that it uses to spin up clouds in order to test them are themselves usable by people to spin up clouds - tooling around this is done in a manner that makes us of and contributes back to existing projects (jenkins plugins, patches back to cobbler/orchestra/whatever) If you didn't read my _other_ long email from a few moments ago, actual discussion of getting this done - and figuring out other people's needs/tools and how to integrate them - is hopefully happening next week right before the regular openstack-meeting. In the mean time, please either flame on right here in list, or ping me back personally. Thanks everyone! Monty ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Lunr question and GlusterFS integration
On Jun 6, 2011, at 3:17 AM, FUJITA Tomonori wrote: On Mon, 6 Jun 2011 09:59:19 + Shehjar Tikoo shehj...@gluster.com wrote: It is not very clear to me whether Lunr will be a replacement for Glance I don't think so. Glance is the image store which uses s3/swift api's to store machine images. Lunr is an effort to take the block volume code that is currently in nova and separate it into it's own more generic service. or will only be used for supporting application volumes. I'm not sure the meaning of application volumes but it's a replacement of the current nova-volume, provinding volume service for VMs. Yep, exactly. I believe Shehjart was referring to EBS style block storage, which nova-volume currently provides (and Lunr will replace in the future most likely). If I understand this correct, it is trying to overcome the iscsi-only access method currently provided for creating and accessing application volumes. Correct? nova-volume already supports non iSCSI protocols, AOE, Sheepdog, Ceph RBD. I think that you start working on GlusterFS integration with nova-volume right now. You can work with Lunr later (when it's released). Agreed - it seems most reasonable to begin this work in nova-volume and then add support to Lunr when it is closer to being ready for prime time. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Integrating GlusterFS with OpenStack
Hi Shehjar, Nova's block device volume service currently has support for several backends and is made in an abstract way so that adding support for Gluster should be a reasonably simple implementation. There is a new project called Lunr being developed that aims to separate the volume code from Nova and have it be implemented as its own stand alone service. There was also some conversations around using Gluster as a ring backend for OpenStack's object storage system named Swift. In short, I think there are quite a few places that Gluster can provide value to the OpenStack ecosystem. Devin On Jun 3, 2011, at 12:34 AM, Shehjar Tikoo wrote: Hi All We're aiming to integrate Gluster with Openstack in a way that allows GlusterFS to be used as the storage for application volumes as well as VMs. I am interested in hearing your ideas on how such an integration can be performed. Being the openstack experts, could you please share some issues I should be considering, the problems I may run into or anything else you think I should know. Thanks -Shehjar ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Conclusion on Pagination (hopefully!) :)
++ That's the right approach! On May 27, 2011, at 9:00 AM, Jay Pipes wrote: Thanks all for some awesome input on the pagination thread. I wanted to summarize what I think were the conclusions to come out of it. Please do let me know if I got it right. Proposal: 1) Push the LIMIT variable into the database API layer 2) Ensure that all queries that return a set of results have an ORDER BY expression to them 3) Push the marker into the database API layer. Continue to have the marker variable be a value of a unique key (primary key for now at least). Use a WHERE field $marker LIMIT $pagesize construct. I *think* this is what's agreed upon? It's basically the Swift model with a variation that the order of results is not static (it can be specified by the user). Please ++ if that looks good and I'll draw up a blueprint Thanks! jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
[Openstack] Proposal for Dashboard as an official OpenStack project
Hello all, For quite a while now, the OpenStack Dashboard has existed as an incubation project. There is quite a bit of development effort going into it now from a number of different companies. Current projects underway include, but are not limited to: * swift support * internationalization * a new theme * OpenStack API support * integration with Keystone project This project is quickly gaining contributors, and is in use by a large number of members of the community. I believe the time is right to make the dashboard an official OpenStack project. To my knowledge, no other projects have gone through the incubation process to become an official project, so in some ways we are in uncharted territory. Is there an officially documented process to make this happen? Devin ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Keystone Release #1 - seeking community input
Hi Ziad, This is great progress! The first question that pops up for me is where are you drawing the distinction between multi-tenancy and the concept of projects as exists today in nova? Are tenants the answer to this, and is the difference purely semantic? Thanks, Devin On May 26, 2011, at 12:04 AM, Ziad Sawalha wrote: Hi Everyone! It's been a while since the summit in Santa Clara. It was great meeting with everyone who was there – looking forward to the next one! Since the summit, we've been working on Keystone and figuring out how to integrate it into OpenStack (Nova, Swift, Glance, and the dashboard). There has been much activity on the project. The code, design, and API has been changing daily. Anyone interested, please join us. RELEASE 1 Milestone 1 for Diablo is right around the corner already! The goal remains to create a common auth system supporting existing use cases. There are a couple of proposals we'd like community input on before we get too far into the implementation: API spec Scope of first release API Spec We've published an API spec doc which we've been altering as requests come in for changes. The spec includes proposals for a core API that covers: tokens: for authentication tenants: for isolating and grouping resources to support multi-tenancy users: because we have to! roles: to support the Nova roles (see http://nova.openstack.org/runnova/managing.users.html for roles and users) credentials: to address the EC2, Rackspace auth, multiple-credentials question The draft spec is on github and includes both the core APIs and additional extensions needed to make Keystone function as a stand-alone system. We'd like to lock it down as soon as is feasible. R1 is too close (June 2nd) so we probably won't be done by then, but aiming for Friday June 10th gives us a good couple of weeks to get there and then a couple of weeks to firm up implementation and tests, so we should be able to hit R2 with a locked down API. Scope of R1 For the first Diablo milestone, we're aiming to support the user stories listed in http://wiki.openstack.org/KeystoneR1 ANNOUNCEMENTS Repo We're moving the source to the Rackspace repo (mainly because we can add multiple admins). Please start using the new repo. I will keep both in sync for a while. https://github.com/rackspace/keystone/ I was able to change my config with those commands: git remote rm origin git remote add origin -m master -t master https://your-lo...@github.com/rackspace/keystone.git As you open new issues, please use the Rackspace repo. Participate If you're interested in joining the team and working on Keystone, we'd love the input and help. Just let me know. And, of course, anyone is welcome to submit code, blueprints, issues, etc… Looking forward to hearing from ya'll. Ziad Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Proposal for Nova Core
+1 from me too :) On May 10, 2011, at 6:21 PM, Ed Leafe wrote: On May 10, 2011, at 4:13 PM, Paul Voccio wrote: I would like to nominate Dan Prince (https://launchpad.net/~dan-prince) for nova-core. He has been a solid contributor in terms of code, reviews and discussions during the summit. +1 from me. -- Ed Leafe Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Discussion on October Design Summit Locations
There was a lot of talk earlier about Seattle and there seemed to be a fair amount of interest. Devin On May 3, 2011, at 2:49 PM, Stephen Spector wrote: Team: As we all are interested in where the next event is located I thought I would send out a recap of the meeting I had last week at the Design Summit: • October 2011 Event – We will again run a Conference and Design Summit the first week of October which is 2 weeks after the Diablo Release • The developers are all in agreement that having the business development folks is a plus • There was a discussion on holding the event during code freeze of Diablo but there are many complications to this idea • The event will be held in the US; looking at New York, Orlando, Boston • GOAL: Look at various options and have a facility chosen in late May ; Planning will start in June • Event Projection – This could easily be a 700+ person event with about 200 to 300 developers • 2012 Events • OpenStack User Event Design Summit (April 2012) • Run a large User Event with the Design Summit running 1 or 2 days after start of User Event • Hold an exhibit hall for sponsors to promote their solutions to users • Event Projection - this could be a 1,000 to 2,000 person event based on how we promote, location we choose, etc • More discussion needed and location/date selection needs to occur this summer • OpenStack Conference and Design Summit (October 2012) • Target Europe or Asia for this event Finally, the marketing of the Conference and Design Summit needs more effort to clarify the Conference part being for the Ecosystem partners. I will work on this for our October event. Location – I was thinking about New York, Montreal, Boston, Washington, D.C, Orlando, etc and am focusing on Boston as it is a large technical hub similar to Silicon Valley and believe we should continue to take advantage of the local community of technologists to continue growing the event and promote OpenStack. As for Europe, the consensus in the room was that we don't have too many developers in Europe at this time and almost everyone would need to travel which would increase costs across the community as well as limit the number of developers who could attend as many companies are not allowing international travel. I agree that holding the event in Japan would be optimal and that was the original plan; however, we have some difficulties with high costs in Japan and local event sponsorship at this time. The community in Japan is growing but is not yet ready to put on an event. South Korea is another option; however everyone will need to travel again and costs will be higher than the US. Finally, Hawaii sounds great until you see the cost of a Diet Coke in a hotel – it is just too expensive. Please feel free to add your thoughts on this discussion. Thanks - - - Stephen Spector, Rackspace OpenStack Community Manager stephen.spec...@openstack.org OpenStack Blog | @opnstk_com_mgr Office +1 (512) 539-1162 | Mobile +1 (210) 415-0930 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Enhancements to Glance in Diablo? Input welcomed
Hey guys, I think there are two things I'd like to discuss to put some of this in context. First, the openstack-dashboard project itself is essentially nothing more than a sample site that shows how to integrate with django-nova. So I'd say if you're looking to build something, you don't want to build it on top of openstack-dashboard, you just want to expose it there, which django makes pretty easy. (I'm mainly just making the semantic distinction.) xtoddx just proposed a merge for a nova based system panel extension called something like django-nova-syspanel as a separate module next to django-nova. So we know we need to add syspanel stuff, swift stuff, glance stuff, burrow stuff, etc. So on to my second point, which is that we should rename django-nova to django-openstack and create submodules within. Then we can reuse exception handling, decorators, base views (now that django 1.3 supports class based views), etc. This is actually pretty easy to do and I'd call it a Medium Tshirt. So anyone building a Django based UI that used one or more pieces of django-openstack can then pick and choose which they want to expose using the actual django site container's urls.py. Django is really great at this. Then openstack-dashboard can continue to be a single reference implementation of all these things. Devin On Apr 19, 2011, at 10:58 AM, Eric Day wrote: Is the existing OpenStack dashboard project modular enough to support other applications? So far it seems to mostly be a Nova dashboard, but it would be nice if each project didn't need its own application/dashboard. Instead each project could have a tab or section within the OpenStack dashboard. Glance could then be accessed through the nova section (via Nova's OpenStack API) or the glance section (direct glance API). You could add/remove application modules as needed in the configuration. I'm already thinking of a dashboard for Burrow for monitoring queue sizes and other metrics, and it would be awesome if this could be built on top of openstack-dashboard. With this type of integration, we could provide group views (zone, cluster, etc) that had a summary and health report of each of the applications running in a group. -Eric On Tue, Apr 19, 2011 at 11:49:12AM -0400, Jay Pipes wrote: Hi! I'm open to any and all suggestions around a Glance dashboard. I think both a separate Glance dashboard application and integration with the OpenStack dashboard would be appropriate. Since Glance is a stand-alone component that can be run outside of Nova, I think having a Glance dashboard that allows people to query and update images is a good thing. The OpenStack dashboard should probably follow the OpenStack API, while the Glance dashboard can communicate directly via Glance's own REST-like API. -jay On Sun, Apr 17, 2011 at 10:40 PM, Devin Carlen devin.car...@gmail.com wrote: Hey Jay, What are your ideas for a glance dashboard app above and beyond what the OpenStack dashboard currently supports? This would be a great thing for us to spend a bit of time talking about at the summit. - Devin On Apr 12, 2011, at 11:11 AM, Jay Pipes wrote: Hey all, We're in the planning stages for Diablo now, working on putting together blueprints, which turn into sessions at the design summit. I know the Glance team is small and our project narrow in scope, but it would be great to get some feedback from the list about stuff you'd like to see included in Glance in the Diablo release. Some possible thoughts: [snip] * A Glance dashboard app? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Enhancements to Glance in Diablo? Input welcomed
Perfect - I am scheduled to give a talk about the dashboard at the summit next week. I was going to have it be a intro to dashboard sort of thing to try to get more people contributing to it. Instead I'm going to rework my talk to include all of this stuff, the openstack API migration, auth, etc. for the future of dashboard. Thanks guys for planning my talk for me! :) On Apr 19, 2011, at 11:57 AM, Jay Pipes wrote: Yes, I'll defer to Devin's judgment on this one... :) Devin, feel free to update the blueprint Ken mentioned below. -jay On Tue, Apr 19, 2011 at 2:52 PM, Ken Pepple ken.pep...@rabbityard.com wrote: On Apr 19, 2011, at 11:08 AM, Devin Carlen wrote: So anyone building a Django based UI that used one or more pieces of django-openstack can then pick and choose which they want to expose using the actual django site container's urls.py. Django is really great at this. Then openstack-dashboard can continue to be a single reference implementation of all these things. i am fine with the one dashboard to rule them all approach as long as it remains easy/possible to implement the sample site (openstack-dashboard) with limited/single project functionality (with the understanding that 80% of our users will probably use the stock sample site with little/no customization). i think my only concern is about authentication … i'd hate to see people entering several passwords or reauthenticating between switching tabs. i have only been casually watching the authn/authz discussions so this may not be the issue that i think it is. can we just broaden the current blueprint (http://wiki.openstack.org/GlanceDashboard) to talk about all this next week ? Or do we want to propose a more encompassing session ? i'd defer to Devin's judgement here. /k --- Ken Pepple http://ken.pepple.info ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Enhancements to Glance in Diablo? Input welcomed
Hey Jay, What are your ideas for a glance dashboard app above and beyond what the OpenStack dashboard currently supports? This would be a great thing for us to spend a bit of time talking about at the summit. - Devin On Apr 12, 2011, at 11:11 AM, Jay Pipes wrote: Hey all, We're in the planning stages for Diablo now, working on putting together blueprints, which turn into sessions at the design summit. I know the Glance team is small and our project narrow in scope, but it would be great to get some feedback from the list about stuff you'd like to see included in Glance in the Diablo release. Some possible thoughts: [snip] * A Glance dashboard app? ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Proposal for Justin Santa Barbara to join Nova-Core
+2 :) Sent from my iPhone On Apr 7, 2011, at 12:18 PM, Jay Pipes jaypi...@gmail.com wrote: Hey all, Subject basically says it all. Justin has proven to be a regular and thorough reviewer and unless he objects, I propose he join the nova-core team as a core reviewer. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Last day before GammaFreeze and Gamma milestone release !
These are all looking to be in good shape. I approved for merge all the ones that have sufficient reviews. On Apr 7, 2011, at 2:23 AM, Thierry Carrez wrote: Hello everyone, Today is the last day of unrestricted bugfix merging. At the end of the day (23:59 PST) we'll switch to release-critical bugfixing only, in order to avoid introducing a last-minute critical issue. That means we need to get as many of those 42+1+3 in progress fixes in by the end of the day, so please wear your reviewer suit today. The non-critical bugfixes that don't get merged will have to wait one week for Diablo to open. In particular, I'd like to see the following fixes make it: Nova: Some instances can't connect to metadata due to ARP failure https://code.launchpad.net/~vishvananda/nova/automatic-metadata/+merge/56672 Nova: Incoherent use of is_public creates havoc when using euca-describe-images https://code.launchpad.net/~vishvananda/nova/fix-describe-images/+merge/55617 Nova: live_migration failing due to network filter not found https://code.launchpad.net/~nttdata/nova/lp746821/+merge/56322 Nova: Sporadic failure when creating XS vms from machine-images https://code.launchpad.net/~johannes.erdfelt/nova/bug732801/+merge/56625 Glance: glance-manage not respecting config https://code.launchpad.net/~jaypipes/glance/bug742190/+merge/56220 Swift: .pot files not being generated in setup.py build https://code.launchpad.net/~jaypipes/swift/bug752610/+merge/56593 Thanks, -- Thierry Carrez (ttx) Release Manager, OpenStack ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Design Summit - Fall 2011 Location Request
+1 for Seattle :) On Mar 29, 2011, at 8:52 AM, Stephen Spector wrote: Developers: I have started early planning on the next OpenStack Design Summit in early October 2011 this year and would like to get some feedback on location options. The current thinking is to just host a Design Summit event for 3 days without the associated Conference portion that we are doing next month in Santa Clara. It is my intention to take this idea to the broader community in the coming weeks but I need to gather some data from the developers before I can put together a final proposal to the community, Location – My goal is to host the 3-day event in a gateway city so attendees from Asia, Europe, and the US can easily fly directly to this destination. This leads me to look at cities like New York, Chicago, London, Paris, Seoul, etc except these locations tend to be expensive for hotels and facilities. Other global cities such as Houston, Amsterdam, Frankfort, Seattle, Atlanta, Dallas are cheaper locations for the event. Thus, I am looking to see what locations are of interest to the developers; also remember weather in October as a factor. Host – Having an event at a hotel is more expensive then finding a corporate facility, university setting, or even co-locate with a conference. I am open to any ideas you have about interesting facilities or events to co-locate with as I want to ensure that we not only accomplish our goal of setting the project's direction for the future releases but also provide an excellent environment with top notch facilities. Feel free to provide your thoughts directly to myself via email or respond to this email should you wish a broader conversation. Thanks. - - - Stephen Spector, Rackspace OpenStack Community Manager stephen.spec...@openstack.org OpenStack Blog | @opnstk_com_mgr Office +1 (512) 539-1162 | Mobile +1 (210) 415-0930 ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] api wrappers?
Hi Jon, It is our goal to migrate the Dashboard to the OpenStack API. There are a few things in the EC2 API that the dashboard uses that aren't yet (or are about to be) in the OpenStack API. As soon as there is parity, we will begin the effort of moving everything over. Most OpenStack tools available today are based around the OpenStack API, and the goal is to use it for all new efforts. Best, Devin Carlen On Mar 18, 2011, at 11:29 AM, Jon Slenk wrote: hi, IIUC, it looks like the Nova Dashboard uses EC2 via boto. Questions along those lines: (a) will the community be wanting to move people off of EC2 and towards OpenStack API? (b) if yes, when will the OpenStack tools do that? (c) if yes, when will there be libraries for other languages (we're into C# for whatever reasons right now). thanks for any thoughts. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] updated_at and bug 722979
Hey Kevin,
704 session.execute('update instances set deleted=1,'
705 'deleted_at=:at where id=:id',
706 {'id': instance_id,
707 'at': datetime.datetime.utcnow()})
In this case, it isn't desirable to modify updated_at. We don't ever destroy
actual rows from mysql for a number of reasons (auditing, performance, etc).
We simply flagged them as deleted=1 and filter those rows out later.
So in this case it only makes sense to update deleted_at and not updated_at,
since, abstractly speaking, this is not an update operation.
Best,
Devin
On Mar 9, 2011, at 11:22 AM, Kevin L. Mitchell wrote:
Howdy; I'm looking into bug 722979 (convert calls to
session.execute(update blah) to SQLAlchemy code), and I noticed that
using session.execute() in places like
trunk/nova/db/sqlalchemy/api.py:704 has the effect of NOT updating the
'updated_at' column inherited from models.NovaBase; this would contrast
with the naive approach to changing this to use SQLAlchemy calls, which
would cause 'updated_at' to be updated.
My question is, is there any reason 'updated_at' should NOT be updated?
I can't imagine there's a problem with it, but I'm still just getting
started, and so I thought I'd check.
Thanks!
--
Kevin L. Mitchell kevin.mitch...@rackspace.com
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
___
Mailing list: https://launchpad.net/~openstack
Post to : openstack@lists.launchpad.net
Unsubscribe : https://launchpad.net/~openstack
More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Compute API for Cactus (critical!)
added a few inline On Mar 2, 2011, at 12:24 PM, Eric Day wrote: I havn't seen much activity on this, so though I'd do a quick brain dump of what I'm aware of: Auth/global: * Ability to lockout a user for some time period after failed auth. * Describe zones and regions (replaced with Sandy's zone work). Admin * Describe instance types (list flavors). * CRUD users and projects (should not be in nova, should be in common auth middleware/server project). * Associate roles between users and projects (probably belongs in common auth project too). * Start/stop/list running vpns (cloudpipe) Compute * CRUD key pairs. * Reference stored key pairs for injection. * Console output (this is just a dump, not an ajax console). * Ajax console works differently between ec2 and OS (I think). * Metadata handler for booting instances. Not needed if we switch to only using guest agents or injection. * Attach and detach volumes. * Attach and detach floating IP addresses (this is not the same as shared IP groups). * Associate and disassociate with network security groups. * Connect with VPNs. Network * CRUD floating IP addresses. * CRUD security groups. * CRUD VPNs Volume * CRUD Volumes. Security Groups (dashboard supports but is disabled at the moment) * CRUD Security Groups * CRUD Authorization rules I'm not sure if image management via new glance and what ec2 provides is ready yet, or if the semantics of all server actions are the same. I'll let the reseident experts weigh in there. -Eric On Mon, Feb 28, 2011 at 02:16:20PM -0600, John Purrier wrote: Has anyone done a gap analysis against the proposed OpenStack Compute API and a) the implemented code, and b) the EC2 API? It looks like we have had a breakdown in process, as the community review process of the proposed spec has not generated discussion of the missing aspects of the proposed spec. Here is what we said on Feb 3 as the goal for Cactus: OpenStack Compute API completed. We need to complete a working set of API's that are consistent and inclusive of all the exposed functionality. We need to *very* quickly identify the missing elements that are required in the OpenStack Compute API, and then discuss how we mobilize to get this work done for Cactus. As this is the #1 priority for this release there are implications on milestones dates depending on the results of this exercise. The 1.1 spec should be complete and expose all current Nova functionality (superset of EC2/RS). Dendrobates, please take the lead on this, anyone who can help please coordinate with Rick. Can we get a fairly complete view by EOD tomorrow? Please set up a wiki page to identify the gaps, I suggest 3 columns (Actual code / EC2 / OpenStack Compute). Thanks, John ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] OpenStack Compute API for Cactus (critical!)
Erik, Thanks for the clarification. I'd just like to reiterate that official support for the EC2 API is something that needs to be handled in parallel, since we've committed to supporting it in the past. Best, Devin On Feb 28, 2011, at 7:53 PM, Erik Carlin wrote: Devin - In a decomposed service model, OS APIs are per service, so the routing is straightforward. For services that need to consume other services (e.g. The compute service needs an IP from the network service), the queueing and worker model remains the same, it's just that the network worker calls out to the RESTful network service API (likely the admin API). For EC2 (and any other 3rd party API), the community is welcome to support them, although I see them as secondary to the canonical OS APIs themselves. Since the EC2 API combines a number of services, it is essentially a composition API. It probably makes sense to keep in nova (i.e. compute) but you are right, it would need to call out to glance, block, and network in the diablo timeframe. What was attached was intended simply to show the general approach, not be a detailed diagram of the API flows. Once we complete the gap analysis John has requested, these connections should become more clear. Erik From: Devin Carlen devin.car...@gmail.com Date: Mon, 28 Feb 2011 17:44:03 -0800 To: Erik Carlin erik.car...@rackspace.com Cc: John Purrier j...@openstack.org, openstack@lists.launchpad.net openstack@lists.launchpad.net Subject: Re: [Openstack] OpenStack Compute API for Cactus (critical!) Your diagram is deceptively simple because it makes no distinction about how block API would be handled in the EC2 API, where compute and block operations are very closely coupled. In order for the diagram to convey the requirements properly, it needs to show how compute/network/volume API requests are routed by both the EC2 and OpenStack API. Devin On Feb 28, 2011, at 3:52 PM, Erik Carlin wrote: I was talking with Will Reese about this more. If we are eventually going to decompose into independent services with separate endpoints, he thought we should do that now. I like that idea better. For cactus, we still have a single nova service black box but we put multiple OpenStack API endpoints on the front side, one for each future service. In other words, use separate endpoints instead of extensions in a single endpoint to expose the current capabilities. That way, it sets us on the right path and consumers don't have to refactor to support between cactus and diable. In diablo, we decompose into separate services and the endpoints move with them. It's a bit hard to visualize so I put together the attached pdf. I'm assuming glance is a separate service and endpoint for cactus (still need to figure out per my message below) and swift already is. Erik From: Erik Carlin erik.car...@rackspace.com Date: Mon, 28 Feb 2011 17:07:22 -0600 To: John Purrier j...@openstack.org, openstack@lists.launchpad.net Subject: Re: [Openstack] OpenStack Compute API for Cactus (critical!) That all sounds good. My only question is around images. Is glance ready to be an independent service (and thus have a separate API) in Cactus? Erik From: John Purrier j...@openstack.org Date: Mon, 28 Feb 2011 16:53:53 -0600 To: Erik Carlin erik.car...@rackspace.com, openstack@lists.launchpad.net Subject: RE: [Openstack] OpenStack Compute API for Cactus (critical!) Hi Erik, today we have compute, block/volume, and network all encompassed in nova. Along with image and object storage these make the whole of OpenStack today. The goal is to see where we are at wrt the OpenStack API (compute/network/volume/image) and coverage of the underlying implementation as well as what is available through the EC2 API today. I would propose that volume and network API’s be exposed not through the core compute API, but as extensions. Once we create separate services and factor network and volume services out of nova these API’s will form the core API’s for these services. We may also need to up-version these service API’s between Cactus and Diablo as they are currently under heavy discussion and design. John From: Erik Carlin [mailto:erik.car...@rackspace.com] Sent: Monday, February 28, 2011 3:16 PM To: John Purrier; openstack@lists.launchpad.net Subject: Re: [Openstack] OpenStack Compute API for Cactus (critical!) John - Are we just talking about compute aspects? IMO, we should NOT be exposing block functionality in the OS compute API. In Diablo, we will break out block into a separate service with it's own OS block API. That means for now, there may be functionality in nova that isn't exposed (an artifact of originally mimicing EC2) until we can fully decompose nova into independent services. Erik From: John Purrier j...@openstack.org Date: Mon, 28 Feb 2011 14:16
[Openstack] Announcing the OpenStack Dashboard
Hello all, I'd like to take this opportunity to formally announce the OpenStack Dashboard. It's been available on Launchpad for a little while now so I could gather a bit of initial feedback about it. Enough people have played with it and found it useful that it makes sense to unveil it to a larger audience. The OpenStack Dashboard is based primarily on code that was developed for the NASA Nebula Dashboard. We received the green light to release the code under the Apache licenese and have done so. The project is based on Django and Python and consists of two primary pieces: django-nova This is a Django module that contains all of the interesting bits. It's designed to be reusable and modular so it can be used in a variety of projects. The NASA Nebula Dashboard uses this module, as does the OpenStack Dashboard. The repo is available at: https://launchpad.net/django-nova As of Ubuntu Natty, it will be available in the apt repo: http://www.ubuntuupdates.org/packages/show/302171 openstack-dashboard This is a Django site that provides a bare minimum reference implementation around django-nova. This is essentially just CSS, django-registration for creating accounts, and the settings required to make django-nova function properly. The goal of this site is to provide something demoable with some OpenStack branding on it. Other organizations wanting to deploy a dashboard will want to roll their own, but using this as a reference. The repo is available at: https://launchpad.net/openstack-dashboard -- Future -- Migrate to OpenStack API Currently django-nova is based on the EC2 API for several reasons: * OpenStack API lacks support for Volumes (this is being remedied very soon) * OpenStack API has some conflicts with how project and user groups are handled. * OpenStack API lacks admin API functions such as creating users and projects, starting VPNs. We have cobbled together the admin functions in the EC2 based admin endpoint for now. The goal is to transition to the OpenStack API as soon as it is possible. There's no harm in starting this effort in a separate branch now so we can begin figuring out exactly where the pain points will be. Combine django-nova and openstack-dashboard repos Since these are complementary projects, I originally created them as separate Launchpad repos, but this created a lot of extra overhead when applying fixes that related to both projects. The code won't be combined anymore than it already is, but I will be restructuring the openstack-dashboard repo to include django-nova in a separate folder. This will make it much easier to deal with. Improve user experience We followed an agile development process and created the minimum of what was needed as we proceeded, but for the most part the functionality has stabilized. There is room for improvement in the general usability of the sight, such as adding more client side scripting, improving layouts, etc. There is much more to be done, but this is a good starting point for discussion! Best, Devin Carlen ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Queue service prototype
On Feb 22, 2011, at 8:58 AM, Eric Day wrote: On Tue, Feb 22, 2011 at 12:55:32AM -0800, Vishvananda Ishaya wrote: Awesome stuff! I had to do the following patch to get it to run (on python 2.6.1 on my mac) Yeah, the version in my junk repo was broken for a couple minutes last night, you must have pulled at just the right moment. :) I find it a little humorous that you went ahead and did a python implementation after the long discussion about the value of doing a python prototype. It is looking pretty good. I'm sure the client side and tests will be very useful as you develop the real version. Hehe. :) I was putting together a sample client with tests and before I knew it the whole server was written too. I agree now that this helps in the process, so Andy and the other folks advocating for a prototype were correct. Shh! Telling Termie he is right is like feeding the animals at the zoo. :) -Eric Vish On Feb 21, 2011, at 11:04 PM, Eric Day wrote: I decided to implement the current queue service specification in Python this weekend as it is described on the wiki page. It is functionally complete minus persistent queues and account verification (plugin points for the real implementation). You can find the links and description here (as well as more specification changes, examples, and diagrams): http://wiki.openstack.org/QueueService For a direct link, grab: bzr branch lp:~eday/+junk/osq or http://bazaar.launchpad.net/~eday/+junk/osq/view/head:/osq.py The top of the file has some example usage. To run the test suite, just run nosetests or: ./osq.py test Any feedback would be very much appreciated! -Eric ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Monsyne Dragon requesting Core-Dev status for nova.
I think the best way to get involved is to just start doing reviews. A branch still requires 2 core reviews to be merged, but that doesn't mean other people can't jump in and give their 2 cents in the meantime. So just dive on in and you can start learning about the review process and helping out! Devin On Feb 17, 2011, at 8:25 AM, Monsyne Dragon wrote: Reposting this in a separate thread as requested by Thierry. I'm volunteering to do code reviews/ other core-dev duties, so we can get some of our review logjam through. -- -- -Monsyne Dragon work: 210-312-4190 mobile210-441-0965 google voice: 210-338-0336 Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
Great thoughts Jay - I think a push to improve test coverage is a great goal for Cactus. It seems like we are getting new contributors faster than ever these days. I would like to suggest that we create a blueprint called something like improve test coverage and create a number of bug reports that reference the blueprint. Then label them as low hanging fruit, which will encourage everyone (especially new contributors) to look at them. As we all know, the best way to learn a codebase is to write tests for it. On Feb 17, 2011, at 12:57 PM, Jay Pipes wrote: On Wed, Feb 16, 2011 at 11:44 PM, Paul Voccio paul.voc...@rackspace.com wrote: Jay, Thanks for throwing this out. How would we build this with Hudson? What would a standard deploy of Nova even look like for integration tests? I replied with some specifics to Trey, who had a similar question, and created a bug report (that I subsequently assigned to Trey): https://bugs.launchpad.net/nova/+bug/720941 Let me know if that answered your questions and if you'd like some more explanation. We've also bounced the idea within our team of not allowing code commits if the code to test ratio decreases but I'm not sure if that would work for such a large project like this one. This is a good idea, but even if we were to add code to test ratios, the ratio would be mostly (only?) looking at unit tests. And I think we've seen that unit tests pass and functional tests blow up because of all the mocks/stubs in the unit tests that don't adequately represent a real-world deployment. Nothing wrong, of course, with exploring code to test ratio (basically, automated code coverage tests), but I think good functional and integration tests are more productive at this time. Just my two cents, though, nothing more, -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Steps that can help stabilize Nova's trunk
+1, this is a great suggestion. On Feb 17, 2011, at 1:06 PM, Jay Pipes wrote: Although Soren adequately explained why he thinks that running run_tests.sh on Hudson should *not* be against a pip/virtualenv setup, I would like to state that I think it would be a good idea to have Hudson do *both*. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Do these commands belong in the API or nova-manage?
Yep, you're right. I misread. There is still a longer conversation to be had about admin API in general. All the functions in the ec2 admin api will need to be available in the openstack admin api at some point. On Feb 8, 2011, at 5:01 PM, Jay Pipes wrote: I think you are confusing the EC2 admin API with what Sandy was talking about, which is the OpenStack admin API commands, which has been in Nova for months now. See /nova/api/openstack/__init__.py, lines 74 through 81. -jay On Tue, Feb 8, 2011 at 5:58 PM, Devin Carlen devin.car...@gmail.com wrote: Only place for it to go now is the ec2 admin extension api (nova/api/ec2/admin.py). Until we get the OpenStack API out in the open, we can't add anything to it. The goal is to move the admin functions to the OpenStack API when that becomes an option, but I'm guessing that won't be until diablo. On Feb 8, 2011, at 4:54 PM, Jay Pipes wrote: On Tue, Feb 8, 2011 at 5:09 PM, Sandy Walsh sandy.wa...@rackspace.com wrote: Quick question ... For multi-cluster/zones I have a bunch of commands that need to be exposed to administrators: 1. CRUD child zones 2. CRUD hosts to a zone 3. CRUD zone host capabilities to a zone Do you think these belong in the admin-only OpenStack API or only available via nova-manage? Admin-only OpenStack API, since they won't have anything to do with EC2 API, and at this point, nova-manage is very EC2-centric, and adding it to nova-manage could potentially be extremely confusing. Since this functionality is mostly going to be used by service providers, most of which have their own set of custom management tools (see: python-novatools, etc..), I think it's best to stick it only in the admin API for now. My 1.5 cents, -jay Perhaps just the (R)ead operations are available via the API and CUD via nova-manage? Opinions? -Sandy Confidentiality Notice: This e-mail message (including any attached or embedded documents) is intended for the exclusive and confidential use of the individual or entity to which this message is addressed, and unless otherwise expressly indicated, is confidential and privileged information of Rackspace. Any dissemination, distribution or copying of the enclosed material is prohibited. If you receive this transmission in error, please notify us immediately by e-mail at ab...@rackspace.com, and delete the original message. Your cooperation is appreciated. ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
Re: [Openstack] Network Service for L2/L3 Network Infrastructure blueprint
This has my support. For our time frame and the goal of robustness and stability for the upcoming release, this is the most reasonable course of action. Devin On Jan 31, 2011, at 10:40 AM, John Purrier wrote: In order to bring this discussion to a close and get everyone on the same page for Cactus development, here is where we have landed: 1. We will *not* be separating the network and volume controllers and API servers from the Nova project. 2. On-going work to extend the Nova capabilities in these areas will be done within the existing project and be based on extending the existing implementation. The folks working on these projects will determine the best approach for code re-use, extending functionality, and potential integration of additional community contributions in each area. 3. Like all efforts for Cactus, correct trade-offs must be made to maintain deployability, stability, and reliability (key themes of the release). 4. Core design concepts allowing each service to horizontally scale independently, present public/management/event interfaces through a documented OpenStack API, and allow services to be deployed independently of each other must be maintained. If issues arise that do not allow the current code structure to support these concepts the teams should raise the issues and open discussions on how to best address. We will target the Diablo design summit to discuss and review the progress made on these services and determine if the best approach to the project has been made. Thoughts? John From: Andy Smith [mailto:andys...@gmail.com] Sent: Friday, January 28, 2011 4:06 PM To: John Purrier Cc: Rick Clark; Jay Pipes; Ewan Mellor; Søren Hansen; openstack@lists.launchpad.net Subject: Re: [Openstack] Network Service for L2/L3 Network Infrastructure blueprint On Fri, Jan 28, 2011 at 1:19 PM, John Purrier j...@openstack.org wrote: Thanks for the response, Andy. I think we actually agree on this J. You said: This statement is invalid, nova is already broken into services, each of which can be dealt with individually and scaled as such, whether the code is part of the same repository has little bearing on that. The goals of scaling are orthogonal to the location of the code and are much more related to separation of concerns in the code, making sure that volume code does not rely on compute code for example (which at this point it doesn't particularly). The fact that the volume code and the compute code are not coupled make the separation easy. One factor that I did not mention is that each service will present public, management, and optional extension APIs, allowing each service to be deployed independently. So far that is all possible under the existing auspices of Nova. DirectAPI will happily sit in front of any of the services independently, any of the services when run can be configured with different instances of RabbitMQ to point at, DirectAPI supports a large amount of extensibility and pluggable managers/drivers support a bunch more. Decoupling of the code has always been a goal, as have been providing public, management, and extension APIs and we aren't doing so bad. I don't think we disagree about wanting to run things independently, but for the moment I have seen no convincing arguments for separating the codebase. You said: That suggestion is contradictory, first you say not to separate then you suggest creating separate projects. I am against creating separate projects, the development is part of Nova until at least Cactus. This is exactly my suggestion below. Keep Nova monolithic until Cactus, then integrate the new services once Cactus is shipped. There is work to be done to create the service frameworks, API engines, extension mechanisms, and porting the existing functionality. All of this can be done in parallel to the stability work being done in the Nova code base. As far as I know there are not major updates coming in either the volume or network management code for this milestone. Where is this parallel work being done if not in a separate project? --andy John From: Andy Smith [mailto:andys...@gmail.com] Sent: Friday, January 28, 2011 12:45 PM To: John Purrier Cc: Rick Clark; Jay Pipes; Ewan Mellor; Søren Hansen; openstack@lists.launchpad.net Subject: Re: [Openstack] Network Service for L2/L3 Network Infrastructure blueprint On Fri, Jan 28, 2011 at 10:18 AM, John Purrier j...@openstack.org wrote: Some clarification and a suggestion regarding Nova and the two new proposed services (Network/Volume). To be clear, Nova today contains both volume and network services. We can specify, attach, and manage block devices and also specify network related items, such as IP assignment and VLAN creation. I have heard there is some
Re: [Openstack] Deprecating nova-objectstore
On Jan 17, 2011, at 11:59 AM, Jay Pipes wrote: On Mon, Jan 17, 2011 at 2:19 PM, Thierry Carrez thie...@openstack.org wrote: Jay Pipes wrote: I think the big difference is that clients would be talking to Glance and its RESTlike/JSON API, not to an S3 API front-end like nova-objectstore. The big question is whether OpenStack wants to support Amazon S3 as an objectstore at all, instead of just Swift, which now can communicate via an S3 API (http://swift.openstack.org/misc.html#module-swift.common.middleware.swift3). OK, my understanding was that euca-upload-bundle is talking directly to an S3-like backend, so if we want to support the EC2 way of registering images we need some S3-compatible server... Precisely. This is why nova-objectstore cannot be removed even if Glance supports S3 as a backend :) For serious deployments I guess we would use Swift S3 frontend. For demo cases, do we need a simpler solution ? And if yes, which one ? Depends on whether supporting euca-upload-bundle is going to be a priority once alternate openstack-API-speaking client tools are completed... :) My guess is that it won't, and nova-objectstore will eventually move into /ext or /plugin for those who will want to continue using euca-upload-bundle directly. I tend to disagree with this. Anyone who is using eucatools is going to want to use euca-upload-bundle. This is something we should continue to support. Not supporting euca-upload-bundle means we don't have a fully compatible EC2 implementation. -jay ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp ___ Mailing list: https://launchpad.net/~openstack Post to : openstack@lists.launchpad.net Unsubscribe : https://launchpad.net/~openstack More help : https://help.launchpad.net/ListHelp
