Re: [openstack-dev] [nova] Create an instance with a custom uuid
On Wed, Sep 24, 2014 at 2:58 PM, Roman Podoliaka rpodoly...@mirantis.com wrote: Are there any known gotchas with support of this feature in REST APIs (in general)? I'd be worried about relying on a user-defined attribute in that use case, that's ripe for a DOS. Since these are cloud-unique I wouldn't even need to be in your project to block you from creating that clone instance if I knew your UUID. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Nova] Some ideas for micro-version implementation
On Tue, Sep 23, 2014 at 1:19 PM, Sean Dague s...@dague.net wrote: So today we have a proxy API in Nova which does some of this. I guess the question is is it better to do this in Nova in the future or divorce it from there. A stand-alone, production-quality configurable proxy would be useful to take over this work. It would also be useful for some other things I have in mind... For a while now I've wanted to find or write an API proxy/simulator to a) do API mock-like configurations to actually use and test it over the network; b) proxy requests on to real servers, with the option of transforming both request and response for further 'live' API change mockups. I'm sure this has already been done, suggestions and pointers to existing projects welcome. If not, suggestions for a good starting point/platform. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
tl;dr: we're not broken, but under stress...changing (outside) expectations requires changing the expression of the model...while it's called a 'stack' maybe it's multiple tiered stacks. MultiStack! On Mon, Sep 22, 2014 at 4:27 PM, Doug Hellmann d...@doughellmann.com wrote: The point of integration is to add the projects to the integrated *release*, not just the gate, because the release is the thing we have said is OpenStack. Integration was about our overall project identity and governance. The testing was a requirement to be accepted, not a goal. If there is no incubation process, and only a fixed list of projects will be in that new layer 1 group, then do contributors to the other projects have ATC status and vote for the TC? What is the basis for the TC accepting any responsibility for the project, and for the project agreeing to the TC’s leadership? This is one reason for multiple layers. The original 4 layer stack was meant as a technical dependency stack but has morphed into a social/project organizational stack. I don't think it is total coincidence that the technical hierarchy was interpreted as a social/governance hierarchy by some as there is a lot of similarity. The mapping between the two in my mind is fairly easy, but those details is not what is important. We love to look at the Apache Foundation for inspiration. In the current set of Apache projects most of them are not focused on adding value to a web server. They grew beyond that in multiple directions. I'm selfish and want to keep the layer nomenclature for the technical organization that helped re-structure DevStack and propose we think of these as *thing*[0] where we have Multiple *Things*: * IaaS thing: the stuff that builds excellent clouds * PaaS thing: the stuff that does amazing things that may or may not be built on top of excellent clouds * XaaS thing(s): more things I can not visualize through the fog of antihistamines * Non-aas developer things: what enables us s developers to make the above things (infra, qa, etc) * Non-aas consumer[1] things: what enables the rest of the world to enjoy the above things (docs, SDKs, clients, etc) This separates the technical hierarchical from the organizational relationships. All of the above things are still called OpenStack. But maybe it's a 'MultiStack'. - New projects wanting to join can apply and receive a 'provisional' attribute that tells the world we (OpenStack people) thing this looks promising and want to see if it matures to our standards. Similar to incubation but maybe the bar has some differences between the things. It _should_ require a higher bar to add to the foundation than a new deck on the side. - The integrated release still applies to a subset of the projects and/or things. The IaaS thing establishes the base release cycle other things apply common sense and follow along or release more often. - The test matrix is built using the technical layers and not the above organizational structure. I'm getting a bit hand-wavy here, but the point is to clearly state to the world that it isn't the organizational things that determine testing beyond having at least the 'provisional' attribute on a project. If the above feels very familiar it should. Some of it is terminology changes to help change expectations and some divisions based on use cases. What we have isn't totally broken, it is under growth related stress. Taking a different perspective on it will help expose where things can be improved. And what we are doing right. dt [0] TODO: cut-n-paste in actual allegorical noun [1] Yuck, better word here too! -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Fri, Sep 19, 2014 at 12:02 PM, Adam Lawson alaw...@aqorn.com wrote: Can someone do a small little Visio or other visual to explain what's being Sean's blog post included a nice diagram that is Monty's starting point: https://dague.net/2014/08/26/openstack-as-layers/ AIUI Monty's Layer 1 is basically the original layers 1+2. I had separated them originally as layer 2 is optional from a purely technical perspective, but not really from a cloud user perspective. layers tl;dr: For my purposes layers 1, 2 and a hand-wavey 'everything else' is useful. Others find combing layers 1 and 2 more useful, especially for organizational and other purposes. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] usability anti-pattern, part 2
[OK, I'll bite...just once more...because dammit I want this crap fixed too.] I know you know this Monty, but for the benefit of the folks who don't, the client library situation is a result of them belonging to the projects they serve, each one[0] forked from a different one forked from jkm's original, without having any sort of mechanism to stay in sync, like a cross-project (BINGO!) effort to keep things consistent. We may not want a BDFL, but we NEED someone to say NO when necessary for the sake of the entire project. Jeez, now I'm sounding all enterprisey. On Fri, Sep 19, 2014 at 9:01 PM, Monty Taylor mord...@inaugust.com wrote: except exc.Unauthorized: raise exc.CommandError(Invalid OpenStack credentials.) except exc.AuthorizationFailure: raise exc.CommandError(Unable to authorize user) This is pervasive enough that both of those exceptions come from openstack.common. If thats from apiclient, I have a guess. apiclient was an attempt (by someone who got frustrated and left us) to build a common core for the clients. However, in many ways wound up being a UNION of them. And scene. I'm guessing that what it actually is is that randomly some things return one, some things return the other, and there is absolutely no rhyme nor reason. Or, more likely, that termie liked the spelling of one of them better. I like that explanation but this isn't from OCL. Actually we'd have been much farther down the road if we had used Termie's bits a year ago. Whether that is a bug or a feature is left to the reader to decide. Code speaks, sometimes, so I'm going back to writing some more client bits. Someone come help. dt [0] except swift and glance, both of which were originally in the server repo. -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Thoughts on OpenStack Layers and a Big Tent model
On Thu, Sep 18, 2014 at 1:53 PM, Monty Taylor mord...@inaugust.com wrote: I've recently been thinking a lot about Sean's Layers stuff. So I wrote a blog post which Jim Blair and Devananda were kind enough to help me edit. Thanks for writing that Monty. Sean took a concept meant for organizing the relationships in DevStack and Grenade and made it relatable to OpenStack as a whole. This brings it out where we can actually use it as a model. I do think there is value in distinctions between the original layers 1,2 and 3 (you combined 1 and 2). But for non-technical purposes 1 and 2 are indeed the same. Suggestion 6: Isn't it funny how everything old is new again? The DevStack exercises started out as exactly this, then grew more functional over time until Tempest came along. How hipster is that? Suggestion 9: Wouldn't it be wonderful if a small set of cloud definition files could be used for the myriad of user tools out there? Standards, we don't have enough of them! In the long term I would like to see more of our base (toolsets and services) be pluggable so that the less-blessed projects can participate without requiring additions to the base repos. This should also be true for user-facing tools. OpenStackClient already picks up installed plugins with the proper entry points configured so everything doesn't need to be in the primary repo to play along. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)
Interestingly enough, the distros are doing exactly what they don't want us to do, ie, rebuilding things to use 'their' tested version of dependencies rather than the included one... On Wed, Sep 17, 2014 at 2:42 PM, Ian Cordasco ian.corda...@rackspace.com wrote: That aside, I’ve been mulling over how effectively the clients use requests. I haven’t investigated all of them, but many seem to reach into implementation details on their own. If I remember nova client has something it has commented as “connection pooling” while requests and urllib3 do that automatically. I haven’t started to investigate exactly why they do this. Likewise, glance client has custom certificate verification in glanceclient.common.https. Why? I’m not exactly certain yet. It seems for the most part from what little I’ve seen that requests is too high-level a library for OpenStack’s needs at best, and actively obscures details OpenStack developers need (or don’t realize requests provides in most cases). Part of that is my doing, the initial conversion from httplib2 to requests was intended to be as simple as possible in order to get the benefits of proper certificate verification. glanceclient never got this (maybe until recently?) because it uses OpenSSL. The come-back-and-clean-things-up work was intended to be Alessio's apiclient stuff that I think is still in oslo-incubator. That was never finished for a variety of reasons. Since that time you're seeing the results of other fixes (connection-pooling being one) that look at the existing code and not at the proper re-factor to push that stuff into requests. The real fix for the clients is to start over and re-build them on top of (in this case) requests to utilize all that it brings. This is already happening... FWIW I totally understand the desire for vendoring...I want to do the same thing with OSC because of the number of times we've been broken by requests, prettytable and others changing out from under us. It is easy enough for me to fix my box but a cloud user that just want to get his VMs running isn't going to be happy, especially on Windows. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Please do *NOT* use vendorized versions of anything (here: glanceclient using requests.packages.urllib3)
On Wed, Sep 17, 2014 at 3:53 PM, Robert Collins robe...@robertcollins.net wrote: On 18 September 2014 08:01, Dean Troyer dtro...@gmail.com wrote: Interestingly enough, the distros are doing exactly what they don't want us to do, ie, rebuilding things to use 'their' tested version of dependencies rather than the included one... Indeed - but the distros are solving for two specific issues: No argument, just observing the recursive nature of this... Also, if we pin to a version, is the downstream consequence different? IIRC Thomas has had to do this with Django (1.7?) and Horizon, probably with others too. As a provider of an app package directly to users, dealing with the front-line consequences of changing dependencies falls on me. And its one reason with this hat on I want static linking, or a Python equivalent of it (bundling/vendoring) at the app level. As an upstream to a distro, I'm happy to let them deal with all of that. Isn't it fun being in the middle? OOI were thouse changes API breaks or were we depending on nonpublic aspects? prettytable was packaging once and I don't recall the other. requests, aside from the recent 2.4.0 release, was the 1.0.0 release when we weren't expecting it and nothing was pinned 1.0.0. I think that was an API change that bit us. The 1.0.0 version was clear, but not having the control over the timing of the change is what makes me understand Kenneth's position on urllib3 and why those who bundle requests do that too... Is my Go-ness showing yet? dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] PostgreSQL jobs slow in the gate
Clark Boylan cboy...@sapwetik.org Wrotr : It appears that keystone eventlet is the source of the slowness in this job. With keystone eventlet all of the devstack jobs are slower and with keystone wsgi all of the jobs are quicker. Probably need to collect a bit more data but this doesn't look good for keystone eventlet. On Wed, Sep 17, 2014 at 11:02 PM, Morgan Fainberg morgan.fainb...@gmail.com wrote: I've kicked off a test[1] as well to check into some tunable options (eventlet workers) for improving keystone eventlet performance. I'll circle back with the infra team once we have a little more data on both fronts. The Keystone team will use this data to figure out the best way to approach this issue. Brant submitted https://review.openstack.org/#/c/121384/ to up the Keystone workers when API_WORKERS is set. I submitted https://review.openstack.org/#/c/122013/ to set a scaling default for API_WORKERS based on the available CPUs ((nproc+1)/2). There is a summary in that commit message of the current reviews addressing the workers in various projects. I think it has become clear that DevStack needs to set a default for most services that are currently either too big (nproc) or too small (Keystone at 1). Of course, moving things to mod_wsgi moots all of that, but it'll be a while before everything moves. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [Openstack] [Devstack][Icehouse] TENANT_ID Failure retrieving TENANT_ID for demo
On Wed, Sep 17, 2014 at 10:40 PM, foss geek thefossg...@gmail.com wrote: 2014-09-18 03:21:56.903 | ++ neutron net-create --tenant-id db7fd8d208db41ea8622be52b520d44e private 2014-09-18 03:21:56.904 | ++ grep ' id ' 2014-09-18 03:21:56.905 | ++ get_field 2 2014-09-18 03:21:56.905 | ++ read data 2014-09-18 03:21:58.466 | Not Found (HTTP 404) (Request-ID: req-d3c47a05-d5ed-4bed-aebf-69d95edb85a5) 2014-09-18 03:21:58.514 | + NET_ID= 2014-09-18 03:21:58.514 | + die_if_not_set 396 NET_ID 'Failure creating NET_ID for test-physnet1 db7fd8d208db41ea8622be52b520d44e' This is your error report, neutron net-create failed. You'll need to dig through neutron logs to figure out why. I'd start with q-svc and q-agt logs. It is possible that this is a side-effect of an earlier error. dt -- Dean Troyer dtro...@gmail.com ___ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Re: [openstack-dev] Error creating instances in DevStack
On Mon, Sep 8, 2014 at 1:40 PM, Sharan Kumar M sharan.monikan...@gmail.com wrote: I am trying to setup my own OpenStack cloud for development. I installed DevStack on a VM and the installation is fine. I am able to log in via ... I checked if the libvirt_type=qemu in nova.conf. Also the package nova-compute-qemu is installed. I am using Ubuntu 14.04. First thing is you should not combine DevStack and any packaged OpenStack installation. DevStack installs from the source repos and has no knowledge of any decisions made while packaging OpenStack by the distros. After sorting that out you'll need to dig further into the Nova log files to see why no working compute node is available. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[Openstack] OpenStackClient 0.4.1 released
OpenStackClient 0.4.1 has been released to PyPI. This release consists of mostly bug fixes and a few new commands. python-openstackclient can be installed from the following locations: PyPI: https://pypi.python.org/pypi/python-openstackclient OpenStack tarball: http://tarballs.openstack.org/python-openstackclient/python-openstackclient-0.4.1.tar.gz Release Highlights * Bug 1319381: remove insecure keyring support * Bug 1337245: add user password set command * Bug 1337684: add extension list --compute * Bug 1337684: add extension list --compute * add container create and container delete commands * add initial support for global --timing option (similar to nova CLI) * complete Python 3 compatibility * add authentication via --os-trust-id for Identity v3 ...and more... dt -- Dean Troyer dtro...@gmail.com ___ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Re: [openstack-dev] [nova][neutron] default allow security group
On Fri, Sep 5, 2014 at 10:27 AM, Monty Taylor mord...@inaugust.com wrote: I've decided that as I have problems with OpenStack while using it in the service of Infra, I'm going to just start spamming the list. User CLI/API feedback! neutron security-group-create default --allow-every-damn-thing You mean like this? https://review.openstack.org/#/c/119407/ dt *Disclaimer: For demonstration purposes on nova-network only; the views expressed here may not be those of the OpenStack Foundation, it's member companies or lackeys; in case of duplicates, ties will be awarded; your mileage may vary; allow 4 to 6 weeks for delivery; any resemblance to functional code, living or dead, is unintentional and purely coincidental; representations of this code may be freely reused without the express written consent of the Commissioner of the National Football League. -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Zaqar] Comments on the concerns arose during the TC meeting
On Fri, Sep 5, 2014 at 4:27 AM, Thierry Carrez thie...@openstack.org wrote: Tim Bell wrote: The one concern I have with a small core is that there is not an easy way to assess the maturity of a project on stackforge. The stackforge projects may be missing packaging, Red Hat testing, puppet modules, install/admin documentation etc. Thus, I need to have some indication that a project is deployable before looking at it with my user community to see if it meets a need that is sustainable. Do you see the optional layer services being blessed / validated in some way and therefore being easy to identify ? Yes, I think whatever exact shape this takes, it should convey some assertion of stability to be able to distinguish itself from random projects. Some way of saying this is good and mature, even if it's not in the inner circle. Being in The integrated release has been seen as a sign of stability forever, while it was only ensuring integration with other projects and OpenStack processes. We are getting better at requiring maturity there, but if we set up layers, we'll have to get even better at that. The layers are (or originally were) a purely technical organization, intentionally avoiding association with defcore and other groupings, and on reflection, maturity too. The problem that repeatedly bubbles up is that people (mostly outside the community) want a simple tag for maturity or blessedness and have been using the integrated/incubated status for that. Maturity has nothing to do with technical relationships between projects (required/optional layers). The good and mature blessing should be an independent attribute that is set on projects as a result of nomination by TC members or PTLs or existing core members or whatever trusted group we choose. I'd say for starters that anything from Stackforge that we use in integrated/incubated projects is on the short list for that status, as it already has that implicitly by our use. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [Openstack] is anyone using zeromq for RPC?
On Fri, Sep 5, 2014 at 2:50 PM, Doug Hellmann d...@doughellmann.com wrote: The zmq driver in oslo.messaging, used for internal communication between OpenStack services, has been without a maintainer for a significant period of time. It isn’t actively tested, and it isn’t clear whether or not it works. The Oslo team would like to drop support for it in Kilo, but before we do that we would like to find out if (a) anyone uses it and (b) if any of those people would like to contribute to maintaining it. I haven't seen any work on zmq in DevStack for some time either. We'll follow Oslo in dropping it if that decision is made. dt -- Dean Troyer dtro...@gmail.com ___ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Re: [openstack-dev] [Nova] Feature Freeze Exception process for Juno
On Wed, Sep 3, 2014 at 7:07 PM, Joe Gordon joe.gord...@gmail.com wrote: On Wed, Sep 3, 2014 at 2:50 AM, Nikola Đipanov ndipa...@redhat.com wrote: The reason many features including my own may not make the FF is not because there was not enough buy in from the core team (let's be completely honest - I have 3+ other core members working for the same company that are by nature of things easier to convince), but because of any of the following: I find the statement about having multiple cores at the same company very concerning. To quote Mark McLoughlin, It is assumed that all core team members are wearing their upstream hat and aren't there merely to represent their employers interests [0]. Your statement appears to be in direct conflict with Mark's idea of what core reviewer is, and idea that IMHO is one of the basic tenants of OpenStack development. FWIW I read Nikola's 'by nature of things' statement to be more of a representation of the higher-bandwith communication and relationships with co-workers rather than for the company. I hope my reading is not wrong. I know a while back some of the things I was trying to land in multiple projects really benefited from having both the relationships and high-bandwidth communication to 4 PTLs, three of whom were in the same room at the time. There is the perception problem, exactly what Mark also wrote about, when that happens off-line, and I think it is our responsibility (those advocating the reviews, and those responding to them) to note the outcome of those discussions on the record somewhere, IMO preferably in Gerrit. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [bashate] .bashateignore
On Tue, Sep 2, 2014 at 8:32 PM, Robert Collins robe...@robertcollins.net wrote: Well, git knows all the files in-tree, right? Or am I missing something here? if-has-bash-hashbang-and-is-versioned-then-bashate-it? It's not quote that simple, none of the include files have a shebang line; I've always felt that having that in an include file is an indication that the file is (also) a stand-alone script. Shocco (the docs processor) wants one too. I think I've given up attempting to mock os.walk so I'm going to post the latest version of my bashateignore review and we can use .bashateignore to both exclude as well as include the files to be processed using the gitignore syntax. Starting with the list of files in the repo is also an option, and excluding from there...but I'm not going to have that tonight. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [bashate] .bashateignore
On Fri, Aug 29, 2014 at 7:42 AM, Sean Dague s...@dague.net wrote: Integrating bashate into something as complicated as devstack, the file ignore problem has come up. We seem to have 3 approaches out under review right now: https://review.openstack.org/#/c/117425 : --exclude-dirs https://review.openstack.org/#/c/115794 : --exclude-dirs (different implementation) https://review.openstack.org/#/c/113892 : removing hidden directories I'm actually kind of convinced now that none of these approaches are what we need, and that we should instead have a .bashateignore file in the root dir for the project instead, which would be regex that would match files or directories to throw out of the walk. I think that would handle the concerns that everyone is having, and hopefully provides a more clear set of semantics in integrating. Anyone up for taking a stab at this patch? I started the other night and ran into the usual semantic problems wrt meaning...rather than re-invent this wheel I found the pathspec module another new dependency!) that purports to do .gitignore-style handling, only it doesn't. It's closer to rsync include file syntax. I managed to get it really close only to fail on handling bare directories properly. Example: Ignoring a doc directory in .gitignore: doc Ignoring a doc directory in my trial: doc/ It occurs to me that fixing this too means maybe I started down the wrong path. This matters to be because I want to also leverage the existing .gitignore files we have. Just to join the party I pushed up the working state in https://review.openstack.org/117772. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [bashate] .bashateignore
On Fri, Aug 29, 2014 at 9:02 AM, Sean Dague s...@dague.net wrote: If pathspec did the right thing, pulling in the extra dep would be fine, but it doesn't seem like it does. After looking at it with fresh eyes, the issue could be resolved by combining two methods from pathspec and still leveraging the regex compilation stuff, which is complicated...that part it gets right enough. I think I got it worked out properly, and it might be even flexible enough to replace discover_files() with the right patterns in .bashateignore. If not we can inject patterns too, I did that for .gitignore and .bashateifnore. ;) dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] gate debugging
On Thu, Aug 28, 2014 at 11:48 AM, Doug Hellmann d...@doughellmann.com wrote: In my case, a neutron call failed. Most of the other services seem to have a *-api.log file, but neutron doesn’t. It took a little while to find the API-related messages in screen-q-svc.txt (I’m glad I’ve been around long enough to know it used to be called “quantum”). I get that screen-n-*.txt would collide with nova. Is it necessary to abbreviate those filenames at all? Cleaning up the service names has been a background conversation for some time and came up again last night in IRC. I abbreviated them in the first place to try to get them all in my screen status bar, so that was a while ago... I don't think the current ENABLED_SERVICES is scaling well and using full names (nova-api, glance-registry, etc) will make it even harder to read. May that is misplaced concern? I do think though that making the logfile names and locations more obvious in the gate results will be helpful. I've started scratching out a plan to migrate to full names and will get it into an Etherpad soon. Also simplifying the log file configuration vars and locations. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] gate debugging
On Thu, Aug 28, 2014 at 12:44 PM, Doug Hellmann d...@doughellmann.com wrote: I usually use the functions for editing ENABLED_SERVICES. Is it still common to edit the variable directly? Not generally. It was looking at it in log files to see what was/was not enabled where I started to think about that. The default is already pretty long, however having full words might make the scan easier than x- does. I've started scratching out a plan to migrate to full names and will get it into an Etherpad soon. Also simplifying the log file configuration vars and locations. https://etherpad.openstack.org/p/devstack-logging Cool. Let us know if we can make any changes in oslo.log to simplify that work. I don't think olso.log is involved, this is all of the log files that DevStack generates or captures: screen windows and the stack.sh run itself. There might be room to optimize if we're capturing something that is also being logged elsewhere, but when using screen people seem to want it all in a window (see horizon and recent keystone windows) anyway. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [infra] [neutron] [tc] Neutron Incubator workflow
On Wed, Aug 27, 2014 at 6:59 PM, Mandeep Dhami dh...@noironetworks.com wrote: Also note that one of the features of the incubator is that it can be packaged/released independently (like python-neutronclient) and a feature branch is not designed for that workflow. Which is a good reason to not use the word 'incubator'. It also has the connotations from Oslo of being 'copy-pasta' code. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [ceilometer] repackage ceilometer and ceilometerclient
On Thu, Aug 21, 2014 at 11:14 PM, gordon chung g...@live.ca wrote: this idea sounds so familiar. i feel like i may have tried to move this in the past but gave up. i actually prefer having the middleware reside in ceilometerclient... it really doesn't make sense for the entire ceilometer package to be pulled in for just a middleware although i feel like that might require the oslo.messaging feature as well As one data point, the keystone middleware (auth_token) was just recently moved out of keystoneclient and into its own repo, partially because it had dependencies that otherwise were not required for pure client installations. I don't know what your middleware dependencies are, but I think it would be good to consider the effect that move would have on client-only installations. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [QA] Picking a Name for the Tempest Library
On Fri, Aug 15, 2014 at 4:31 PM, Jay Pipes jaypi...@gmail.com wrote: current Tempest repository, into their own repo called openstack-integration-tests or os-integration-tests. I see what you did there... +1 anyway dt Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all] The future of the integrated release
On Wed, Aug 13, 2014 at 2:24 PM, Doug Hellmann d...@doughellmann.com wrote: What “cross-project efforts” are we talking about? The liaison program in Oslo has been a qualified success so far. Would it make sense to extend that to other programs and say that each project needs at least one designated QA, Infra, Doc, etc. contact? ++1 I've done this informally with DevStack and it has worked well enough that Ian is introducing a MAINTAINERS file where we'll record these 'delegations'. For those areas that don't have obvious representation or volunteers I'll pick someone based on the commits for that feature. That specific bit might not scale to other projects, but I think documenting some informal connectivity like this would be helpful. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] Core team proposals
These changes have been completed. Welcome Ian! dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Bug#1231298 - size parameter for volume creation
On Mon, Aug 11, 2014 at 5:34 PM, Duncan Thomas duncan.tho...@gmail.com wrote: Making an previously mandatory parameter optional, at least on the command line, does break backward compatibility though, does it? Everything that worked before will still work. By itself, maybe that is ok. You're right, nothing _should_ break. But then the following is legal: cinder create What does that do? dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [cinder] Bug#1231298 - size parameter for volume creation
On Fri, Aug 8, 2014 at 12:36 AM, Ganapathy, Sandhya sandhya.ganapa...@hp.com wrote: This is to discuss Bug #1231298 – https://bugs.launchpad.net/cinder/+bug/1231298 ... Conclusion reached with this bug is that, we need to modify cinder client in order to accept optional size parameter (as the cinder’s API allows) and calculate the size automatically during volume creation from image. There is also an opinion that size should not be an optional parameter during volume creation – does this mean, Cinder’s API should be changed in order to make size a mandatory parameter. In cinderclient I think you're stuck with size as a mandatory argument to the 'cinder create' command, as you must be backward-compatible for at least a deprecation period.[0] Your option here[1] is to use a sentinel value for size that indicates the actual volume size should be calculated and let the client do the right thing under the hood to feed the server API. Other project CLIs have used both 'auto' and '0' in situations like this. I'd suggest '0' as it is still an integer and doesn't require potentially user-error-prone string matching to work. FWIW, this is why OSC changed 'volume create' to make --size an option and make the volume name be the positional argument. [0] The deprecation period for clients is ambiguous as the release cycle isn't timed but we think of deprecations that way. Using integrated release cycles is handy but less than perfect to correlate to the client's semver releases. [1] Bad pun alert...or is there such a thing as a bad pun??? dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [devstack] Core team proposals
I want to nominate Ian Wienand (IRC: ianw) to the DevStack core team. Ian has been a consistent contributor and reviewer for some time now. He also manages the Red Hat CI that runs tests on Fedora, RHEL and CentOS so those platforms have been a particular point of interest for him. Ian has also been active in the config and devstack-gate projects among others. Reviews: https://review.openstack.org/#/q/reviewer:%22Ian+Wienand+%22,n,z Stackalytics: http://stackalytics.com/?user_id=iwienandmetric=marksmodule=devstackrelease=all I also want to (finally?) remove long-standing core team members Vish Ishaya and Jesse Andrews, who between them were responsible for instigating the whole 'build a stack script' back in the day. Please respond in the usual manner, +1 or concerns. Thanks dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] DevStack program change
Thanks for the feedback everyone, I've proposed the change in https://review.openstack.org/112090. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] DevStack program change
I propose we de-program DevStack and consolidate it into the QA program. Some of my concerns about doing this in the beginning have proven to be a non-issue in practice. Also, I believe a program's focus can and should be wider than we have implemented up to now and this a step toward consolidating narrowly defined programs. I read the QA mission statement to already include DevStack's purpose so no change should be required there. I'll propose the governance changes following a few days of discussion. This is purely a program-level change, I do not anticipate changes to the DevStack project itself. dt (soon-to-be-former?) DevStack PTL -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [Openstack] OpenStack K naming poll
On Thu, Jul 10, 2014 at 10:19 AM, Gary Kotton gkot...@vmware.com wrote: What about Kilimanjaro? That'll get you sentenced to writing it on the chalkboard once for each commit during the cycle... dt Dean Troyer dtro...@gmail.com ___ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Re: [openstack-dev] Using tmux instead of screen in devstack
On Thu, Jul 3, 2014 at 2:59 AM, Anant Patil anant.tec...@gmail.com wrote: If we did move from screen to tmux. We aren't moving away... I have emphasized this enough. One should be able just choose tmux explicitly otherwise things will run in screen by default, as usual. And I am saying due to the nature of how we use screen, we are not going to support two. Adding a third set of semantics to consider when debugging process problems is not an option. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Using tmux instead of screen in devstack
I'm not going to get into a screen vs tmux debate, we removed tmux support two years ago and changing that now is going to be a high bar to get over...but it seems some expectations should be set here. On Wed, Jul 2, 2014 at 11:05 PM, Anant Patil anant.tec...@gmail.com wrote: [...] I understand that the changes will introduce complexity, but I will try to abstract out this complexity so that we don't deal with it in everywhere in devstack but one. As Sean suggested I will go though all the screen calls in devstack and see if we have equivalent tmux calls and how we can abstract these things out and put in one place. I don't think there's going to be any functional differences, but I will investigate and point them out in the blueprint. DevStack is an opinionated installer, and as such will not be all things to all people. Most of the screen usage is in the set of functions that handle process start and stop in functions-common. There is also some logging and support for rejoin-stack.sh set up in stack.sh (I think, too lazy to look this late at night) As I said above, but am going to repeat here, making this change is going to be a high bar to get over. screen has certainly been a challenge for us but at the moment we have about 90% of our issues with it solved. Changing the devil we know for a bag of unknown is not appealing to me at this point. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Creating new python-new_project_nameclient
On Wed, Jun 25, 2014 at 10:18 PM, Aaron Rosen aaronoro...@gmail.com wrote: I'm looking at creating a new python-new_project_nameclient and I was wondering if there was any on going effort to share code between the clients or not? I've looked at the code in python-novaclient and python-neutronclient and both of them seem to have their own homegrown HTTPClient and keystone integration. Figured I'd ping the mailing list here before I go on and make my own homegrown HTTPClient as well. For things in the library level of a client please consider using keystoneclient's fairly new session layer as the basis of your HTTP layer. That will also give you access to the new style auth plugins, assuming you want to do Keystone auth with this client. I'm not sure if Jamie has any examples of using this without leaning on the backward-compatibility bits that the existing clients need. The Python SDK is being built on a similar Session layer (without the backeard compat bits). dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[Openstack] OpenStackclient 0.4.0 release
OpenStackClient 0.4.0 has been released to PyPI. This release consists of a number of new commands and bug fixes. As of this release we feel it is ready for general consumption for the Compute, Identity, Image and Volume APIs. The commands for these APIs should be considered to be in their final form, although until the 1.0 release there is still the possibility of fixes/adjustments, particularly to command options. We will begin to observe a deprecation cycle to allow some time for adjustment. python-openstackclient can be installed from the following locations: PyPI: https://pypi.python.org/pypi/python-openstackclient OpenStack tarball: http://tarballs.openstack.org/python-openstackclient/python-openstackclient-0.4.0.tar.gz Release Highlights * Identity v2: rename 'token create' to 'token issue' and add 'token revoke' * Identity v3: rework the group/user/role list commands so each only lists its own data type; to get a list of the groups that user Xyzzy belongs to you would use 'group list –user Xyzzy'. * Identity v3: add new 'role assignment list' command add new 'extension list' command to list the available API extensions, currently supports the Identity API * Compute v2: rename 'agent' object to 'compute agent': 'compute agent list' * Identity v3: add 'identity provider create/delete/list/set/show' commands * Image v1: add –volume and –force to 'image create' * Volume v1: change –volume-type options for 'volume create' to –type fix quiet/verbose/debug output levels to be more consistent with other programs dt -- Dean Troyer dtro...@gmail.com ___ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Re: [openstack-dev] [Ironic] handling drivers that will not be third-party tested
On Thu, May 22, 2014 at 9:05 AM, Dan Prince dpri...@redhat.com wrote: - Original Message - From: Devananda van der Veen devananda@gmail.com [...] interface. However, I also don't expect the author to provide a full third-party CI environment, and as such, we should not claim the same level of test coverage and consistency as we would like to have with drivers in the gate. Not claiming the same level of support seems reasonable if we don't have 3rd party CI running on it. As was repeated many times last week: if it isn't tested it is broken. We get to argue over the definition and degree of 'tested' now... So, why not just put the driver in a separate library on github or stackforge? Because the driver can be easily broken due to internal Ironic driver API changes. :( I have similar issues with DevStack and including in-repo support for drivers/projects that are not integrated or incubated or even an OpenStack-affiliated project at all. And have come to the conclusion that at some point some things just make sense to include anyway. But the different status needs to be communicated somehow. As a user of an open source project it is frustrating for me to want/need to use something in a 'contrib' directory and not be able to know if that thing still works or if it is even maintained. Having a certain amount of testing to at least demonstrate non-brokenness should be a requirement for inclusion. It would be great if we would adopt common nomenclature here so user expectations are consistent: * 'experimental' - stuff not tested at all? * 'contrib' - stuff with some amount of testing short of gating * 'thirdparty' - stuff that requires hardware or licensed software to fully test Also, contact information should be required for anything 'special' to at least know who to notify if the thing is so broken that removal is contemplated. Projects are going to do what they want regarding inclusion/exclusion policies, I hope we can use common practices to implement those choices. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [Openstack] Using Nova client with SSH SOCKS proxy
On Fri, May 16, 2014 at 10:41 AM, Adrian Smith adr...@17od.com wrote: To access my controller I need to go through a intermediary box. I've created a local SOCKS proxy by ssh'ing to this intermediary with the parameters -D 13392. I then set the environment variable, export HTTP_PROXY=http://localhost:13392 But using nova list just gives an error, $ nova list ERROR: HTTPConnectionPool(host='localhost', port=13392): Max retries exceeded with url: http://x.x.x.x:5000/v2.0/tokens (Caused by class 'httplib.BadStatusLine': '') Should this work? Am I doing something wrong? The nova client lib uses requests/urllib3 for HTTP, they do not support SOCKS proxies out of the box. There have been some forks/patches to add that to requests or urllib3, we have not tested those. dt -- Dean Troyer dtro...@gmail.com ___ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Re: [openstack-dev] [Programs] Client Tools program discussion
On Wed, May 7, 2014 at 5:05 PM, Joe Gordon joe.gord...@gmail.com wrote: Would client tools be limited to only a pythonSDK or in the future could it potentially have other languages? There are a handful of other language projects active on Stackforge that have expressed interest in joining the program when they reach some state of maturity. That is a big part of the reason for structuring things the way we have. If any were ready today they would be included in the initial proposal. https://git.openstack.org/cgit/stackforge/python-openstacksdkhttps://github.com/stackforge/python-openstacksdk https://git.openstack.org/cgit/stackforge/openstack-sdk-php/http://git.openstack.org/cgit/stackforge/openstack-sdk-php/ https://git.openstack.org/cgit/stackforge/openstack-sdk-dotnet/http://git.openstack.org/cgit/stackforge/openstack-sdk-dotnet/ https://git.openstack.org/cgit/stackforge/golang-client/http://git.openstack.org/cgit/stackforge/golang-client/ https://git.openstack.org/cgit/stackforge/openstack-cli-powershell/http://git.openstack.org/cgit/stackforge/openstack-cli-powershell/ dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Programs] Client Tools program discussion
On Tue, May 6, 2014 at 7:02 AM, Thierry Carrez thie...@openstack.orgwrote: Would you take over the Python client libraries as well ? On one hand they need /some/ domain expertise, but on the other I see no reason to special-case Python against other SDKs, and that may give the libraries a bit more attention and convergence (they currently are the ugly stepchild in some programs, and vary a lot). The future of the existing client libs has not been settled, my working assumption is that they would remain with their home programs as they are now. From the start OpenStackClient was meant to be a clean-slate for the CLI and the Python SDK is taking the same basic approach. In the case you'd absorb the Python client libraries, it might make sense to ship the keystone middleware in a separate package that would still live in the Identity program. This needs to happen anyway, it's time for my semi-annual request to dolphm... I think we need people caring for the end user and their experience of interacting with an OpenStack-backed cloud. I understand that CLI/SDK specialists and GUI-oriented specialists are different crowds, but they share the same objective and would benefit IMHO from being in the same program. There could be two subteams to care for specialists in both areas (or even 3 if you separate the CLI and SDK folks). Overall from the TC perspective it would make a much stronger proposal if you somehow could present a united (and without overlap) proposal. To be honest, until the most recent ML thread I hadn't thought about the UX team or even if they were active. We have three basic categories of projects delivering code: web UI (Horizon), CLI (OpenStackClient) and SDK (at least three active language-based teams). They all should consume the output from a UX RD effort, I guess I am open on the program structure to make that work. Horizon is already a part of a program, OSC needs to be and the SDKs will also need to be in the near future. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Programs] Client Tools program discussion
On Tue, May 6, 2014 at 7:38 AM, Anne Gentle a...@openstack.org wrote: I believe that UX and DevEx are different areas of expertise and OpenStack could benefit more from a separate focused DevEx program Agreed. I'm not sure Dean's proposal is quite DevEx though, and I don't know if a rename alone would take care of what I think it is. - SDK maintenance and testing - App-dev focus on docs - Focus on improving automation tasks with unified CLI - Processes and governance that could be separate from contrib devs - Releases separate from the every six month timed release in order to serve app devs better with quicker turnaround (similar to the CLIs now) - Integrated client with the goal of eliminating the client-per-project mess Dean, what do you think of these ideas? Is there enough in common with the CLI user and app dev to consider this DevEx umbrella the program? Do you plan to get rid of project-clients, and does it make sense to do so? Also how do you plan to transition away from DevStack PTL role, have you found a replacement? Anne, this is why you are a writer and I am not. ;) I may steal that list... What you have listed fits with what I see happening as the Client Tools adds the SDK projects (#1 and 2). #3, 5 and 6 are already goals for OSC[1]; and #4 is what we are here talking about now. [1] I expect OSC to switch to use the Python OpenStack SDK as it becomes ready rather than attempt to salvage the existing client libs. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Programs] Client Tools program discussion
On Tue, May 6, 2014 at 4:45 PM, Joe Gordon joe.gord...@gmail.com wrote: 1. At first ClientTools consist of just the OpenStackClient 2. When the pythonSDK is ready to move off of stackforge, it will live in ClientTools Yes to both of these 3. Specific python-*clients will be rewritten (from scratch?) to use the PythonSDK. But this time they won't have a built in CLI. These libraries will live along side the respective servers (so nova's python-novaclient will live in Compute)? All while moving OpenStackClient to the new libraries I have no plans for the existing client libs. If it makes sense to consolidate them in Client Tools that is fine with me. Historically there has been pushback on suggestions of that sort so I have not included them here. OSC will use the SDK when it becomes viable to do so. No matter where the existing clients fit organizationally, I see their future use diminishing over time. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] SSL in Common client
On Fri, May 2, 2014 at 2:14 PM, Adam Young ayo...@redhat.com wrote: Did swift leave this behind when they switched to Requests? Swift and Glance clients were not changed to requests when I did the initial work in the fall of 2012 due to their use of chunked transfers. I've not really looked into this since then but do recall talk about being able to either implement the required changes in upstream requests or somehow hack it in from below. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] SSL in Common client
On Fri, May 2, 2014 at 2:06 PM, Rob Crittenden rcrit...@redhat.com wrote: I'm trying to get devstack to the point where it can configure all the services with SSL so it can be be part of the acceptance process. This is for client communication, there is no expectation that anyone would deploy native SSL endpoints. For the most part things just work. Most of the issues I've run into are server to server communication relating to passing in the CA certificate path. FWIW, DevStack has had the ability to do TLS termination using stud for all public API services, long before any of the individual service SSL/TLS configurations were usable. Using an external TLS termination solves the internal communication problem as long as internal services are configured properly. It also more closely matches what I have seen in real-world deployments. It has been a while since I've tested this and it is likely to need some love. The basic structure, including building a root and intermediate CA to issue certs that look like real-world certs, has been present for almost a year and a half. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [Programs] Client Tools program discussion
I want to open the discussion of an OpenStack Client Tools program proposal to a wider audience. It would initially consist of OpenStackClient and eventually add the existing SDK projects as they are ready to join. The initial wiki page is at https://wiki.openstack.org/wiki/ClientTools. I do want to have the proposal made before the summit, but not necessarily the TC consideration. There has recently been some discussion (specifically around summit sessions) regarding the overlap of client code and the user experience team. This is one of the things I want to get some feedback on before making a formal proposal. The mission statement and description are written with the anticipation of one or more SDK projects joining the program during the Juno cycle. dt Mission Statement The OpenStack Client Tools mission is to provide clean and consistent interfaces to OpenStack services via the published REST APIs. The intended audiences are command-line users (OpenStackClient) and application developers (SDKs as they join the program). Description The OpenStack Client Tools program encompasses a number of related projects that have both common contributors and common consumer interests regarding OpenStack services. The existing OpenStackClient project is targeted at command-line users: end-users as well as cloud operators, devops, system administrators or anyone needing a shell interface to OpenStack. The SDK projects (expected to join the program as they mature) implement bindings to the OpenStack REST APIs in multiple languages. Deliverables Releases for the Client Tools projects are on an independent schedule from the OpenStack integrated release just as the existing client CLI/libraries do today. - OpenStackClient delivers an integrated CLI as a PyPI package and the usual OpenStack tarballs. -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [UX] User Experience cross-project sessions at Summit
On Thu, Apr 24, 2014 at 8:33 AM, Liz Blanchard lsure...@redhat.com wrote: On Apr 24, 2014, at 3:58 AM, Thierry Carrez thie...@openstack.org wrote: After the UX session(s) there is a OpenStack clients and SDKs workshop that will go into the implementation details of some of that, but I still think that UX should set high-level goals for the whole user experience, not just the browser-driven one. I definitely will plan to attend this session, thanks for the heads up on this This is going to be an interesting session in that some of the original proposals were intended to be implementation-level and some conceptual and/or organizational level. Note that the people working on OpenStack clients / SDKs also look at becoming a program -- it may actually make sense to combine both efforts in a single end user experience program. As I've understood some of the criteria used for evaluating how projects belong under programs it included some of the overlap in project developer community. At the moment, there is no overlap that I know of between OSC+the set of SDK projects and Horizon, or the existing UX folk. I believe that ideally there would be some, but it hasn't happened yet. The Client Tools program proposal is done enough that I'll start an ML discussion later today. That'll be a good place to evaluate potential future options. I think working closely with the folks who are tackling efforts around OpenStack clients / SDKs makes a lot of sense. I will be sure to encourage conversations between the two groups on how we can work together towards a common UX goal! The things I long for in terms of what should overlap is a place to discuss things like agreements on terminology or how certain resources might be represented outside of the cloud; 'tenant' vs 'project' is a prime example. The ambiguousness of the term 'user' is not helping. It means different things to an SDK and a CLI and a webUI: an operator, a consumer, an app developer and devops/sysadmin, and more. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] How to re-compile Devstack Code
On Thu, Apr 24, 2014 at 12:53 AM, Matthew Oliver m...@oliver.net.au wrote: If you put OFFLINE=True in your localrc (or local.conf) file, then you can: ./unstack.sh; ./stack.sh without the stack.sh attempting to recheck out the latest code. If this works at all it is a side-effect of skipping a git clone and does not guarantee that the repos will be left untouched. In addition to what Steve mentions elsewhere in the thread about just restarting individual services, make sure you do not have RECLONE=True set (it defaults to False), as 'True' ensures that all repos are reset to the branches in stackrc/local.conf. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Devstack] add support for ceph
On Fri, Apr 18, 2014 at 10:51 AM, Scott Devoid dev...@anl.gov wrote: The issue is that it is very easy to suggest new features and refactoring when you are very familiar with the codebase. To a newcomer, though, you are basically asking me to do something that is impossible, so the logical interpretation is you're telling me to go away. This patch was dropped on us twice without any conversation or warning or discussion about how might be the best approach. Suggestions _were_ made and subsequently ignored. If there was a lack of understanding, asking questions is a good way to get past that. None were asked. I do not view that as saying 'go away'. The plugin mechanism is designed to facillitate these sort of additions. They can be completely drop-in, at minimum one file in extras.d and config in local.conf. The question is about the additional modifications required to the base DevStack scripts. We can address if anyone can articulate what is not possible with the existing setup. At this point I do not know what those needs are, nobody has contacted me directly to talk about any of this outside drive-by Gerrit review comments. That is not the place for a design discussion. I see your motivations here. There are systems to help us with this though: redirect them to ask.openstack.org or bugs.launchpad.net and have them ping you with the link. Delegate replies to others. I try to answer any questions that pop up on #openstack but I need to look at the ask.openstack.org queue more often. Perhaps we need to put more focus on organizing community support and offloading that task from PTLs and core devs. DevStack is an _opinionated_ OpenStack installer. It can not and will not be all things to all people. The first priority is to address services required by OpenStack projects (database, queue, web server, etc) and even then we only use what is provided in the underlying distributions. (BTW, does zmq even still work? I don't think it is tested.) Layered products that require 3rd party repos have a higher bar to get over to be included in the DevStack repo. If an OpenStack project changes to require such a product, and that change gets through the TC (see MongoDB discussions for an example), then we'll have to re-evaluate that position. All this said, I really do want to see Ceph support for Cinder, Glance, Swift, etc in DevStack as I think it is cool and useful. But it is not required to be in the DevStack repo to be useful. Chmouel has proposed a session on this subject and it is likely to be accepted as there are no other submissions for the last slot. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Help in re-running openstack
On Fri, Apr 4, 2014 at 3:43 AM, Deepak Shetty dpkshe...@gmail.com wrote: Shiva, Can u tell what exactly u r trying to change in /opt/stack/ ? My guess is that u might be running into stack.sh re-pulling the sources hence overriding ur changes ? Try with OFFLINE=True in localrc (create a localrc file in /opt/stack/ and put OFFLINE=True) and redo stack.sh FWIW, RECLONE controls the 'pull sources every time' behaviour without cutting off the rest of your net access. OFFLINE short-circuits functions that attempt network access to avoid waiting on the timeouts when you know they will fail. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [keystone] [oslo] Using oslo.cache in keystoneclient.middleware.auth_token
On Fri, Apr 4, 2014 at 10:51 AM, Kurt Griffiths kurt.griffi...@rackspace.com wrote: It appears the current version of oslo.cache is going to bring in quite a few oslo libraries that we would not want keystone client to depend on [1]. Moving the middleware to a separate library would solve that. +++ I think it makes a lot of sense to separate out the middleware. Would this be a new project under Identity or would it go to Oslo since it would be a shared library among the other programs? I think it really just needs to be a separate repo, similar to how keystoneclient is a separate repo but still part of the Keystone project. The primary problem being addressed is dependencies and packaging, not governance. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] DevStack PTL Candidacy
I am running for another term as DevStack PTL. I have been the acting/elected PTL since DevStack became a program and have been working on DevStack since its first public demo in a lightning talk at the Essex Design Summit. In addition I have also contributed to Grenade and am the primary instigator behind OpenStackClient. During the Icehouse development cycle we have continued to make DevStack more modular and plugin-friendly to allow projects to add support to DevStack without requiring changes to the core files. We have also spent a lot of time tuning the execution and logging environment to aid in debugging failures in the CI gate jobs. We also added Chmouel Boudjnah to the core team. The plans for the Juno cycle include further modularization of the project support and re-focusing the exercises back to one of their original purposes of demonstrating command-line use of OpenStack. Also, we plan to finally move the docs and devstack.org to foundation-controlled infrastructure. A bit of my background: I have worn many hats during my career ranging from UNIX system administrator, network engineer, application developer and other things that include the words 'Java', 'Perl' and 'black lacquer' in their description. I was on the Nebula team at NASA when Nova was born and currently work for Nebula Inc. remotely from my home outside Kansas City, MO. Commit history: https://review.openstack.org/#/q/owner:dtroyer%2540gmail.com,n,z Review history: https://review.openstack.org/#/q/reviewer:dtroyer%2540gmail.com,n,z dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [Openstack] devstack + ldap
On Thu, Mar 6, 2014 at 9:51 AM, Craig Jellick cjell...@godaddy.com wrote: I cannot get devstack + ldap working. I've tried on Ubuntu and CentOS vms and in both cases I get a similar error: In Ubuntu: + ldapdelete -x -w test -D cn=Manager,dc=openstack,dc=org -H ldap://localhost -r dc=openstack,dc=org ldap_search: No such object (32) This is another casualty of enabling errexit in stack.sh last week. This has always silently failed when that object was not present, such as on the first run. https://review.openstack.org/78675 fixes it for me locally, it's a one line change to lib/ldap. dt -- Dean Troyer dtro...@gmail.com ___ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Re: [openstack-dev] [devstack] [neutron] How to tell a compute host the control host is running Neutron
On Mon, Mar 3, 2014 at 8:36 PM, Kyle Mestery mest...@noironetworks.comwrote: In all cases today with Open Source plugins, Neutron agents have run on the hosts. For OpenDaylight, this is not the case. OpenDaylight integrates with Neutron as a ML2 MechanismDriver. But it has no Neutron code on the compute hosts. OpenDaylight itself communicates directly to those compute hosts to program Open vSwitch. devstack doesn't provide a way for me to express this today. On the compute hosts in the above scenario, there is no q-* services enabled, so the is_neutron_enabled function returns 1, meaning no neutron. True and working as designed. And then devstack sets Nova up to use nova-networking, which fails. This only happens if you have enabled nova-network. Since it is on by default you must disable it. The patch I have submitted [1] modifies is_neutron_enabled to check for the meta neutron service being enabled, which will then configure nova to use Neutron instead of nova-networking on the hosts. If this sounds wonky and incorrect, I'm open to suggestions on how to make this happen. From the review: is_neutron_enabled() is doing exactly what it is expected to do, return success if it finds any q-* service listed in ENABLED_SERVICES. If no neutron services are configured on a compute host, then this must not say they are. Putting 'neutron' in ENABLED_SERVICES does nothing and should do nothing. Since you are not implementing the ODS as a Neutron plugin (as far as DevStack is concerned) you should then treat it as a system service and configure it that way, adding 'opendaylight' to ENABLED_SERVICES whenever you want something to know it is being used. Note: I have another patch [2] which enables an OpenDaylight service, including configuration of OVS on hosts. But I cannot check if the opendaylight service is enabled, because this will only run on a single node, and again, not on each compute host. I don't understand this conclusion. in multi-node each node gets its own specific ENABLED_SERVICES list, you can check that on each node to determine how to configure that node. That is what I'm trying to explain in that last paragraph above, maybe not too clearly. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] OpenStackClient release 0.3.1
OpenStackClient 0.3.1 has been released to PyPI. This release consists of mostly bug fixes and one new command. It also works with the current development versions of its dependent client libraries. python-openstackclient can be installed from the following locations: * Via pip: https://pypi.python.org/pypi/python-openstackclient * By hand: http://tarballs.openstack.org/python-openstackclient/python-openstackclient-0.3.1.tar.gz Release Highlights * Add ``token create`` command, similar to ``keystone token-get`` * Bug 1100116: Prompt interactive user for passwords in ``user create`` and ``user set`` * Bug 1198171: Add domain support options for Identity v3 * Bug 1241177: Fix region handling in volume commands * Bug 1256935: Clean up ``security group rule list`` output format * Bug 1284957: Correctly pass ``-cacert`` and ``-insecure`` to Identity client in token flow auth dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] Bug in is_*_enabled functions?
On Fri, Feb 28, 2014 at 10:21 AM, Brian Haley brian.ha...@hp.com wrote: Of course now boot_from_volume.sh fails because it doesn't include lib/neutron, I've just pushed a patch for that, https://review.openstack.org/#/c/77212/ Thanks. It's absence from the gate left it the poor stepchild many times. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Tuskar CLI UX
On Thu, Feb 27, 2014 at 6:36 AM, Jiří Stránský ji...@redhat.com wrote: Thanks for bringing this up. It looks really interesting. Is it possible to not only add commands to the OpenStackClient, but also purposefully blacklist some from appearing? As Jay mentioned in his reply, we don't make use of many commands in the undercloud and having the others appear in --help is just confusing. E.g. there's a lot of commands in Nova CLI that we'll not use and we have no use for Cinder CLI at all. I didn't consider removing other API's commands as a use case, it might work but whether that is a Good Thing is TBD from our/user's point of view. If you are trying to hide the rest of the APIs then this is not the CLI that you seek. OSC doesn't assume that a user has access to only one or one type of cloud, so disabling/removing commands simply based on package installation is not a good user experience. Everything after that requires authentication to get server API versions and a service catalog to know what a specific cloud offers. Even if we decided to build TripleO CLI separate from OpenStackClient, i think being able to consume this plugin API would help us. We could plug in the particular commands we want (and rename them if we want node profiles instead of flavors) and hopefully not reimplement everything. The command names themselves are easy to change as they are just the key in the entry point key=value pair. Everything else about them, including the help text would be code changes. It sounds like you may be wanting something parallel to OSC, ie, a Tuskar-specific rebranding of it with a overlapping-but-different command set. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] Bug in is_*_enabled functions?
On Thu, Feb 27, 2014 at 10:34 AM, Brian Haley brian.ha...@hp.com wrote: Ok, part of this is my kernel background, where true=1 like it should be :) So there's a -EUSERERROR there. Right. This is Bourne/POSIX shell, forget everything logical. ;) That call to 'ip netns exec...' should be: sudo /usr/local/bin/neutron-rootwrap /etc/neutron/rootwrap.conf ip netns exec... Perhaps there's something up with this test - exercises/floating_ips.sh - I'll try running it by hand. There is a problem in two of DevStack's exercises, floating_ips.sh and volume.sh, where lib/neutron is not set up properly to handle the ping_check() function calls. That is what leads to what you see. https://review.openstack.org/#/c/76867/ fixes the problem in the exercises. As you know, the actual issue is that Q_RR_COMMAND is not set. Which is because 'is_service_enabled neutron' check appears to be failing. Note that is not the one you found in the logs as the one in question isn't actually logged. I've just re-submitted the above review with that logging turned on. The changes to is_service_enabled() are (so far) backward-compatible in that if is_neutron_enabled() is not declared (as is the case now) the old check in is_service_enabled for 'neutron' should still catch it. That happens everywhere else and in the DevStack test scripts, so I'm baffled and awaiting the current gate check on 76867. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] bug 1203680 - fix requires doc
On Tue, Feb 25, 2014 at 12:01 PM, Ben Nemec openst...@nemebean.com wrote: So is there some git magic that also keeps the repos in sync, or do you have to commit/pull/restart service every time you make changes? I ask because experience tells me I would inevitably forget one of those steps at some point and be stymied by old code still running in my devstack. Heck, I occasionally forget just the restart service step. ;-) This is why I set RECLONE=True and set the *_REPO and *_BRANCH as required to point to my working repo so re-running stack.sh just does the Right Thing. And it still allows short dev cycles in /opt/stack with manual restartes of the service(s) involved in screen. But the real work happens elsewhere, in my case on a different machine. Remember, DevStack's opinion of your work area is that it is a disposable VM. Doing long-term work in a disposable VM is just asking for trouble in a number of ways. Sean's local mounts via VBox and my remote git repos are only two ways to operate in that environment, I'm sure there are many others. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [glance] [heat] bug 1203680 - fix requires doc
On Wed, Feb 26, 2014 at 12:53 AM, Mike Spreitzer mspre...@us.ibm.comwrote: (I added some tags in the subject line, probably should have been there from the start.) Thanks guys, for an informative discussion. I have updated https://wiki.openstack.org/wiki/Gerrit_Workflow and https://wiki.openstack.org/wiki/Testing to incorporate what I have learned. Thanks for the updates, but I've massaged the project bits and restored/expanded the reasons to consider one or the other option. But I still do not detect consensus on what to do about bug 1203680. It looks like Sean thinks the fix is just wrong, but I'm not hearing much about what a good fix would look like. As best I can tell, it would involve wrapping tox in a script that recapitulates the system package install logic from DevStack (I know of no other scripting for installing needed system packages). That bug is closed and against Grizzly, is that the one you meant to reference? I added a note about INSTALL_TESTONLY_PACKAGES to the wiki page above and it will be in the next rev of devstack.org. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [python-*client] moving to WebOb exceptions
On Wed, Feb 26, 2014 at 11:20 AM, Andrey Kurilin akuri...@mirantis.comwrote: While working on unification of clients code we try to unify various exceptions in different client projects. We have module apiclient.exceptions in oslo-incubator[1]. Since our apiclient is an oslo-inclubator library and not a standalone lib this doesn't help in case we need to process exceptions from several clients. [...] The solution would be to use exceptions from external library - Module WebOb.exc[2] for example (since WebOb is already used in other openstack projects). This exceptions cover all our custom http exceptions. I would oppose adding WebOb as a requirement for the client libraries. I see keystoneclient has it today but that is only because the middleware is still in that repo (moving those is another topic). The pain of installing the exiting client libs and their prereqs is bad enough, adding to it is not tenable and is part of what is motivating the SDK efforts. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] Bug in is_*_enabled functions?
On Wed, Feb 26, 2014 at 11:51 AM, Brian Haley brian.ha...@hp.com wrote: While trying to track down why Jenkins was handing out -1's in a Neutron patch, I was seeing errors in the devstack tests it runs. When I dug deeper it looked like it wasn't properly determining that Neutron was enabled - ENABLED_SERVICES had multiple q-* entries, but 'is_service_enabled neutron' was returning 0. This is the correct return, 0 == success. Can you point to a specific log example? dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [nova] [glance] [heat] bug 1203680 - fix requires doc
On Wed, Feb 26, 2014 at 1:09 PM, Mike Spreitzer mspre...@us.ibm.com wrote: Thanks for the further updates. I have just one question about those. One way to do both unit testing and system (integration) testing is to: git clone your favorite project to make a working local repo somewhere on your testing machine, edit and commit there, then use DevStack to create a running system in /opt/stack using your modified project in place of the copy at git.openstack.org. Yes, that is similar to what Sean mentioned, the difference being that his repos are actually in VirtualBox Shared Folders so the Linux VM has access to them. He works on them natively on his laptop so the VM doesn't need his desktop toolset installed. I think the most direct approach would be to generalize the title of https://wiki.openstack.org/wiki/Gerrit_Workflow#Unit_Tests_Only and generalize the introductory remark to include a reference to https://wiki.openstack.org/wiki/Testing#Indirect_Approach . Does this make sense to you? Sure. My edits were prompted by the loss of some information useful to making a choice between the two and tweaking the phrasing and sections. There are a lot of ways to do this, enumerating them is beyond the scope of that doc IMHO. I think having the basic components/requirements available should be enough, but then I also have my workflow figured out so i probably am still missing something. As far as my experience goes, the fix to 1203680 would also cover 1203723: the only thing I find lacking for nova unit testing in Ubuntu is libmysqlclient-dev, which is among the testing requirements of glance (see DevStack's files/apts/glance). As far as I can tell, Sean Dague is saying the fix to 1203680 is wrong because it binds unit testing to DevStack and he thinks unit testing should be independent of DevStack. Let me summarize: Unit testing is not a default requirement for DevStack. If you want it, we have #testonly tags in the system packaging prereq files that will be installed when INSTALL_TESTONLY_PACKAGES=True. Following that, missing system package requirements that affect only unit testing can be added with the #testonly tag. It is good form to attempt to verify those added on both Ubuntu and Fedora, even better form to get RHEL and SuSE. Interestingly, installing DevStack with INSTALL_TESTONLY_PACKAGES set to True will have a global side-effect and so will provide the unit testing requirements even if the indirect procedure ( https://wiki.openstack.org/wiki/Testing#Indirect_Approach) that Sean advocates is used. In this way the only tie between unit testing and DevStack is that they be done on the same machine. Maybe this is the way to go? That is exactly what that variable is for. There are times we don't want those packages installed and it is MUCH easier to add them than to remove them. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Tuskar CLI UX
On Wed, Feb 26, 2014 at 1:43 PM, Jay Dobies jason.dob...@redhat.com wrote: I like the notion of OpenStackClient. I'll talk ideals for a second. If we had a standard framework and each project provided a command abstraction that plugged in, we could pick and choose what we included under the Tuskar umbrella. Advanced users with particular needs could go directly to the project clients if needed. This is a thing. https://github.com/dtroyer/python-oscplugin is an example of a stand-alone OSC plugin that only needs to be installed to be recognized. FWIW, four of the built-in API command sets in OSC also are loaded in this manner even though they are in the OSC repo so they represent additional examples of writing plugins. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] Documenting test environments (was Re: supported dependency versioning and testing)
[new thread for this...] On Fri, Feb 21, 2014 at 7:33 AM, Mark McLoughlin Perhaps rather than focusing on making this absolutely black and white, we should focus on better communicating what we actually focus our testing on? (i.e. rather than making the grey areas black, improve the white areas) Concretely, for every commit merged, we could publish: - the set of commits tested - details of the jobs passed: - the distro - installed packages and versions - output of pip freeze - configuration used - tests passed Long ago I created tools/info.sh to document the DevStack environment as a troubleshooting tool. Among other things it grabs information on the distro and system packages installed, pip-installed packages and repos and DevStack configuration (local.conf/localrc). The output is designed to be easily parsable, and with a bit of work I think it could provide the info above and be logged with the rest of the logfiles. Thoughts? dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] supported dependency versioning and testing
On Fri, Feb 21, 2014 at 1:42 PM, Mike Spreitzer mspre...@us.ibm.com wrote: Ah, so the only distro regularly tested is Ubuntu 12.04? Within the OpenStack Infrastructure Team -managed environment, generally yes (with the addition of CentOS 6 for Py26 as noted in your quote below). However, that is not the only platform that testing is performed on and certainly not the only platform that DevStack is supported on. That's consistent with the other clues I am getting. But it is inconsistent with the following remark found in http://devstack.org/overview.html : *The OpenStack Technical Committee (TC) has defined the current CI strategy to include the latest Ubuntu release and the latest RHEL release (for Python 2.6 testing).* The bullet list immediately following gets very specific regarding our distribution support policy. devstack.org documents (sometimes confusingly as you've noted) only DevStack itself. CI uses DevStack as the primary install/configuration to prepare for the Tempest tests, but is only one piece of that picture. When CI says they only test on Ubuntu LTS (I missed the letters LTS in that quote too) that doesn't mean that DevStack is not tested only on LTS. There are a number of vendors performing their own CI testing for configurations that are not covered by OpenStack CI tests, including other distributions. It may not be obvious to core developers, but for us newbies there is a lot of inconsistent and incomplete documentation of what to expect and how to do things --- and it is scattered in a variety of places, easy to miss some; it really is a drag on a beginner's time. I am trying to point out and fix doc problems as I discover them, but am not myself so sure what all the right answers are. Thaks for helping. OpenStack is huge; I've got the benefit of having learned it as it grew. We need this kind of input from fresh eyes to help flush out the inconsistencies that I know I am blind to. dt Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Version Discovery Standardization
FWIW, an early proposal to address this, as well as capability discovery, still lives at https://etherpad.openstack.org/p/api-version-discovery-proposal. I've lost track of where this went, and even which design summit this is from, but I've been using it as a sanity check for the discovery bits in OSC. On Thu, Feb 13, 2014 at 6:50 AM, Jamie Lennox jamielen...@redhat.comwrote: 6. GET '/' is unrestricted. GET '/vX' is often token restricted. Keystone allows access to /v2.0 and /v3 but most services give a HTTP Unauthorized. This is a real problem for discovery because we need to be able to evaluate the endpoints in the service catalog. I think we need to make these unauthorized. I agree, however from a client discovery process point-of-view, you do not necessarily have an endpoint until after you auth and get a service catalog anyway. For example, in the specific case of OpenStackClient Help command output, the commands listed may depend on the desired API version. To get the endpoints to query for version support still requires a service catalog so nothing really changes there. And this doesn't even touch on the SC endpoints that include things like tenant/project id... Please have a look over the wiki page and how it addresses the above and fits into the existing schemes and reply with any comments or problems that you see. Is this going to mess with any pre-existing clients? * id: Let's either make this a real semantic version so we can parse and use the major.minor.patch components (and dump the 'v') or make it an identifier that matches the URL path component. Right now * updated: I think it would be a friendly gesture to update this for unstable changes as the id is likely to not be updated mid-stream. During debugging I would want to be able to verify exactly which implementation I was talking to anyway. There are two transitional things to also consider: * We have to produce a discovery mechanism that takes in to account the historical authentication URLs published by deployments that include a version. ayoung's Ml thread last week discussed this a bit, we should document the approaches that we're testing and why they do or do not work. * There are real-world deployments that do not configure admin_endpoint and/or public_endpoint in keystone.conf. Discovery is really useless if the URL you are given is https://localhost:5000/v2.0. Do we need to talk about another horrible horrible hack to deal with these or are these deployments going to be left out in the cold? dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [OpenStack-Dev] [Cinder] Cinder driver verification
On Thu, Feb 13, 2014 at 4:51 AM, Thierry Carrez thie...@openstack.orgwrote: John Griffith wrote: To add some controversy and keep the original intent of having only known tested and working drivers in the Cinder release, I am going to propose that any driver that has not submitted successful functional testing by RC1 that that driver be removed. I'd at least like to see driver maintainers try... if the test fails a test or two that's something that can be discussed, but it seems that until now most drivers just flat out are not even being tested. +1 I think there are multiple stages here. Stage 0: noone knows if drivers work Stage 1: we know the (potentially sad) state of the drivers that are in the release Stage 2: only drivers that pass tests are added, drivers that don't pass tests have a gap analysis and a plan to fix it Stage 3: drivers that fail tests are removed before release Stage 4: 3rd-party testing rigs must run tests on every change in order to stay in tree At the very minimum you should be at stage 1 for the Icehouse release, so I agree with your last paragraph. I'd recommend that you start the Juno cycle at stage 2 (for new drivers), and try to reach stage 3 for the end of the Juno release. Are any of these drivers new for Icehouse? I think adding broken drivers in Icehouse is a mistake. The timing WRT Icehouse release schedule is unfortunate but so is shipping immature drivers that have to be supported and possibly deprecated. Should new drivers that are lacking have some not-quite-supported status to allow them to be removed in Juno if not brought up to par? Or moved into cinder/contrib? I don't mean to be picking on Cinder here, this seems to be recurring theme in OpenStack. I think we benefit from strengthening the precedent that makes it harder to get things in that are not ready even if the timing is inconvenient. We're seeing this in project incubation and I think we all benefit in the end. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] healthnmon and devstack
On Wed, Feb 12, 2014 at 6:13 PM, Paulo Rômulo p.romu...@gmail.com wrote: My name is Paulo, I'm a software engineer from Brazil and just getting started with OpenStack development using *devstack*. I'm working on a project which one the goals is to monitor resource usage (at first *cpu_util*), considering both VM instances and their hosts. Welcome! I'm using Ubuntu server 12.04. I've tried to build *healthnmon* from git, but the generated Debian package depends on python-novaclient and python-glance, whose *devstack* already installs. Dpkg doesn't detect the *devstack* installation of nova and glance, so I'm not sure of how can I enable *healthnmon* services from *devstack* (maybe editing *stack.sh* by hand?). DevStack builds everything supplied by OpenStack and Stackforge projects from source, so anything extra that you want to use with dependencies on OpenStack projects will also have to be installed in a similar manner. One approach would be to build a file in devstack/lib to do the install, configure and startup steps for healthmon. You can use lib/template as a starting point and refer to the other service files for reference. lib/glance is one of the simpler ones. You would also need a file in devstack/extras.d to hook into stack.sh, again use one of the existing files as a starting point, they are all very similar. You will need at a minimum to - check out the source repo - install_XXX() - do any configuration - configure_XXX() - start the service - maybe init_XXX(), definitely start_XXX() - stop the service - stop_XXX() The dispatch file in extras.d should start with 80 so it runs last, that way all of the dependencies that come from the OpenStack repos will be in place already. You should be able to just drop these files into an existing DevStack checkout and go. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Ugly Hack to deal with multiple versions
On Tue, Feb 4, 2014 at 9:00 AM, Sean Dague s...@dague.net wrote: Can you be more specific about what goes wrong here? I'm not entirely sure I understand why an old client of arbitrary age needs to be supported with new OpenStack. The contract is the API, not the client, and an old client that doesn't do version discovery is just a buggy client from what I'm concerned. Time to release a new version. Problem 1: API version discovery is not universally considered to be part of the API and therefore is not defined by most services beyond them responding to a '/' request with a 300 response and a list of versions. No two of these responses look alike except where the source was copied from an existing service. Problem 2: Identity is unique in that it is handed a deployment-defined URL to authenticate and get endpoints for all other services. Most of these auth URLs have a version hard-coded in them because the client didn't do version discovery or negotiation until recently. This is what we're talking about here, how to remove the version from this URL and not break old clients. We can't. Not without doing nasty things like detecting an old client and compensating for it server-side. So we have to work out a way for new clients to do discovery even when handed a URL that has a version in it. I've tested a couple of more generalized approaches, and the best solution I have found so far is to simply special-case the known legacy behaviour then drop in to the general discovery process. I also wonder if this is an issue with version discovery implementation. It seems like if we think this is going to be affecting multiple services before doing an odd hack for keystone, we should actually figure out a pattern that works for all services, and figure out why this has only just become an issue. Most of the other services have done The services that traditionally embed a version inside the URL followed by a tenant ID or something get even deeper into parsing the URL to hack the version. dual APIs at some point over the last 2 years, and this didn't seem to trip them up too badly. What happened differently in keystone that made this an issue? And what can be learned about how we structure APIs going forward. I think the difference is this is the first API we have actually tried to deprecate and we don't have the option to hide it in an updated SC endpoint. The service catalog has hidden a lot of this pain for other services because the clients generally can use whatever endpoint the SC gives it. a) Version discovery needs to be rationalized across the services. We've talked about this at summits before, and proposals have been written. And here we are. We'll do it again in Atlanta, hopefully for the last time. b) Define a common structured endpoint and let the client assemble the components into the final URL. If the service catalog had a base URL for compute, and a list of versions, and the additional bits to be appended the client could make an intelligent choice and assemble the endpoint. It isn't like the client doesn't already have to know how the REST URLs are constructed. b-alt) Stop putting things like tenant IDs in the SC. This has the same issue as the auth URL in how to do this without instantly breaking the existing clients. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Ugly Hack to deal with multiple versions
On Mon, Feb 3, 2014 at 1:50 PM, Adam Young ayo...@redhat.com wrote: HACK: In a new client, look at the URL. If it ends with /v2.0, chop it off and us the substring up to that point. Now, at this point you are probably going: That is ugly, is it really necessary? Can't we do something more correct? At this point I think we are stuck with hard-coding some legacy compatibility like this for the near future. Fortunately Identity is an easy one to handle, Compute is going to be a #$^%! as the commonly documented case has a version not at the end. I've been playing with variations on this strategy and I think it is our least bad option... Can we accept that this is necessary, and vow to never let this happen again by removing the versions from the URLs after the current set of clients are deprecated? +1 There is another hack to think about: if public_endpoint and/or admin_endpoint are not set in keystone.conf, all of the discovered urls use localhost: http://localhost:8770/v2.0/. Discovery falls over aga I don't know how common this is but I have encountered it at least once or twice. Is this the only place those config values are used? It seems like a better default could be worked out here too; is 'localhost' ever the right thing to advertise in a real-world deployment? dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] a common client library
On Thu, Jan 16, 2014 at 9:37 AM, Jesse Noller jesse.nol...@rackspace.comwrote: On Jan 16, 2014, at 9:26 AM, Justin Hammond justin.hamm...@rackspace.com wrote: I'm not sure if it was said, but which httplib using being used (urllib3 maybe?). Also I noticed many people were talking about supporting auth properly, but are there any intentions to properly support 'noauth' (python-neutronclient, for instance, doesn't support it properly as of this writing)? Can you detail out noauth for me; and I would say the defacto httplib in python today is python-requests - urllib3 is also good but I would say from a *consumer* standpoint requests offers more in terms of usability / extensibility requests is built on top of urllib3 so there's that... The biggest reaon I favor using Jamie Lennox's new session layer stuff in keystoneclient is that it better reflects the requests API instead of it being stuffed in after the fact. And as the one responsible for that stuffing, it was pretty blunt and really needs to be cleaned up more than Alessio did. only a few libs (maybe just glance and swift?) don't use requests at this point and I think the resistance there is the chunked transfers they both do. I'm really curious what 'noauth' means against APIs that have few, if any, calls that operate without a valid token. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] a common client library
On Thu, Jan 16, 2014 at 8:45 AM, Jesse Noller jesse.nol...@rackspace.comwrote: After speaking with people working on OSC and looking at the code base in depth; I don’t think this addresses what Chris is implying: OSC wraps the individual CLIs built by each project today, instead of the inverse: a common backend that the individual CLIs can wrap - the latter is an important distinction as currently, building a single binary install of OSC for say, Windows is difficult given the dependency tree incurred by each of the wrapped CLIs, difference in dependencies, structure, etc. OSC is the top of the cake and was the most beneficial to user experience so it went first. I would love to consume fewer libraries and dependencies, so much that I still have a project to do this in a language that can easily ship a single binary client. What I think we're talking about here is the bottom of the cake and eliminating duplication and accumulated cruft from repeated forking and later direction changes. The creamy gooey middle is the API-specific bits that right stay exactly where they are (for now??) and can continue to ship a stand-alone cli if they wish. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] a common client library
On Thu, Jan 16, 2014 at 9:54 AM, Alexei Kornienko alexei.kornie...@gmail.com wrote: Knowing usual openstack workflow I'm afraid that #1,#2 with a waterfall approach may take years to be complete. And after they'll be approved it will become clear that this architecture is already outdated. We try to use iterative approach for clients refactoring. We started our work from removing code duplication because it already gives a direct positive effect on client projects. If you can show us better way of moving forward please help us by uploading patches on this topic. Talk is cheap. Show me the code. (c) Linus python-keystoneclient has the Session bits commiited, and the auth bits in flight. So we'll all be using it one way or another. OSC has a very-slimmed-down model for replacing the REST and API layers already present underneath the object-store commands as an experiment to see how slim I could make then and still be usable. It isn't necessarily what I want to ship everywhere, but the REST layer is eerily similar to Jamie's in keystoneclient. There's why my bias... dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] a common client library
On Thu, Jan 16, 2014 at 10:23 AM, Jay Pipes jaypi...@gmail.com wrote: Right, but requests supports chunked-transfer encoding properly, so really there's no reason those clients could not move to a requests-based codebase. Absolutely...it was totally me chickening out at the time why they didn't get changed. I feel a bit braver now... :) dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] a common client library
On Thu, Jan 16, 2014 at 2:22 PM, Renat Akhmerov rakhme...@mirantis.comwrote: - Keeping all the clients physically separate/combining them in to a single library. Two things here: - In case of combining them, what exact project are we considering? If this list is limited to core projects like nova and keystone what policy could we have for other projects to join this list? (Incubation, graduation, something else?) FWIW, OpenStackClient has Identity built-in as very little else is useful without it. Compute, Image and Volume are in the repo but structured using the new plugin system to make sure that works well. Object-store is in the repo but I consider it experimental as it also includes the project API layer and my first cut at a common REST client. I've already written a POC for solum and some other things to demonstrate how to add additional projects simply by installing the python-*client package. https://github.com/dtroyer/python-oscplugin is a trivial example. - In terms of granularity and easiness of development I’m for keeping them separate but have them use the same boilerplate code, basically we need a OpenStack Rest Client Framework which is flexible enough to address all the needs in an abstract domain agnostic manner. I would assume that combining them would be an additional organizational burden that every stakeholder would have to deal with. If it is not a library that is actually shared you will get back to the current situation over time. - Has anyone ever considered an idea of generating a fully functional REST client automatically based on an API specification (WADL could be used for that)? Not sure how convenient it would be, it really depends on a particular implementation, but as an idea it could be at least thought of. Sounds a little bit crazy though, I recognize it :). When you have stable and accurate documents of this sort, let's talk. You may have noticed that few (any?) of the recent major API revs are documented in this manner... dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack] libvirt default log level
On Wed, Jan 15, 2014 at 11:35 AM, Daniel P. Berrange berra...@redhat.comwrote: I see that something close to this has already been added to devstack: https://review.openstack.org/#/c/65834/2/lib/nova which is the right way to tailor logging levels. So we should definitely revert https://review.openstack.org/#/c/63992/ [Who approved that anyway?? Oh...] FWIW I lean in this direction unless there is a strong need to make this configurable. We have enough config vars to track already so if there is a strong response to that I'll make it so this can be set using the local.conf mechanism. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] a common client library
On Wed, Jan 15, 2014 at 1:37 PM, Doug Hellmann doug.hellm...@dreamhost.comwrote: Several people have mentioned to me that they are interested in, or actively working on, code related to a common client library -- something meant to be reused directly as a basis for creating a common library for all of the openstack clients to use. There's a blueprint [1] in oslo, and I believe the keystone devs and unified CLI teams are probably interested in ensuring that the resulting API ends up meeting all of our various requirements. I would love to see a bit more detail on the structure of the lib(s), the blueprint really doesn't discuss the design/organization/intended API of the libs. For example, I would hope the distinction between the various layers of a client stack don't get lost, i.e. not mixing the low-level REST API bits with the higher-level CLI parsers and decorators. Does the long-term goals include a common caching layer? dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [DevStack] Nominate Chmouel Boudjnah for core team
Welcome to the DevStack core team Chmouel! dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Devstack on Fedora 20
On Thu, Jan 9, 2014 at 10:27 PM, Adam Young ayo...@redhat.com wrote: Tried wiping out the (installed) python-greenlet rpm and re running, and that was not installed afterwards, either. I am guessing that the package install step is getting skipped somehow, after the first run. That sounds like you need to remove the .prereqs file to force the install_prereqs.sh script to run again...normally this lets you skip running through all of the package checking on subsequent stack.sh runs...but if you change the package config within the time window (default is 2 hours) it gets missed until the end of the window. Setting FORCE_PREREQ=1 in local.conf will turn off the whole mechanism. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Less option
On Fri, Jan 10, 2014 at 5:16 AM, Flavio Percoco fla...@redhat.com wrote: On 10/01/14 11:28 +0100, Thierry Carrez wrote: Personally I think we should (and we can) say NO more often. As we get stronger as a dev community it becomes easier, and I think we see more opinionated choices in younger projects. That said, it's just harder for old projects which already have a lot of options to suddenly start denying someone's feature instead of just adding another option... +1 I've seen NOs flying around, which is a good sign. I think one of the issues right now is that we already have many configuration options in some projects and we should try to shrink them, if possible. +1 or maybe +1000 after looking at how to test all of those options... The trend of attempting to accommodate all possible options in configuration has led us down the slippery slope of complexity that is extremely hard to recover from. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Devstack on Fedora 20
On Thu, Jan 9, 2014 at 2:16 PM, Adam Young ayo...@redhat.com wrote: That didn't seem to make a difference, still no cache. The RPMS are not getting installed, even if I deliberately add a line for python-dogpile-cache Shouldn't it get installed via pip without the rpm line? Yes pip should install it based on requirements.txt. I just tried this and see the install in /opt/stack/logs/stack.sh.log and then see the import fail later. God I love pip. And there it is...xslt-config isn't present so a whole batch of installs fails. Add this to files/rpms/keystone: libxslt-devel # dist:f20 There are some additional tweaks that I'll ask Flavio to add to https://review.openstack.org/63647 as it needs at least one more patch set anyway. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [Openstack] this can crash devstack
On Wed, Jan 1, 2014 at 8:58 PM, Peter Cheung mcheun...@hotmail.com wrote: 1) import an qcow image to glance 2) start the instance for that image 3) init 6 4) delete that qcow image 5) rejoin the devstack devstack will still running, but unable to start any instance, even instances are not from that image. Restarting after a reboot is really outside what DevStack is intended to be able to do. rejoin_stack.sh is only meant to restart the screen sessions and nothing else. Even then it isn't perfect, which is why you should run stack.sh again after rebooting. dt -- Dean Troyer dtro...@gmail.com ___ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Re: [openstack-dev] Process for proposing patches attached to launchpad bugs?
On Mon, Dec 23, 2013 at 3:50 AM, Robert Collins robe...@robertcollins.netwrote: On 23 December 2013 17:35, Chet Burgess c...@metacloud.com wrote: It's unclear to me what exactly constitutes writing a new patch. I can check out oslo.messaging, and without trying to merge the patch just go and make the same change (its literarily a 2 line change). I can write the tests, and I can submit it (which I'm happy to do, I really want this bug fixed). Honestly though this change is so trivial I don't see how my patch would look all that different from the one already posted. I know there is prior art. The mixin class that kombu provides does the exact same thing. Is that Research the term 'de minimis' WRT copyright and decide (with the help of actual legal advice if necessary) when to just go ahead and submit a patch. Prior art is a patent concept, not related to copyright. Copyright is +1...this stuff gets confused too much these days... dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack]
On Wed, Dec 18, 2013 at 3:20 AM, Ganpat Agarwal gans.develo...@gmail.comwrote: I want to test the icehouse-1 milestone release (cinder and nova) on the setup. I looked for the instructions in the docs but could not find. Since you don't say I will assume you have your current setup from a DevStack stable/havana checkout. To get current master you need to check out the master branch of DevStack and re-run it. It will pick up the master branches of all of the repos in the process if you have not overridden them in local.conf. In general, use a matching branch of DevStack for the OpenStack bits that you want. Mixing them (ie, some Havana and some Icehouse) is not directly supported, you'll have to do that by hand. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [devstack]
On Wed, Dec 18, 2013 at 8:23 AM, Anne Gentle annegen...@justwriteclick.comwrote: Also, Dean correct me if I'm wrong, but I don't believe devstack does a precise milestone release. To guestimate it, you could figure out what This is correct, only stable/* releases. devstack looked like on 12/5 and get all the matching PROJECT_BRANCH= like this example localrc does: https://gist.github.com/everett-toews/4004430 I use this method to get stable/havana for example. Hat tip to Everett's blog at http://blog.phymata.com/2012/11/08/openstack-devstack-on-the-rackspace-open-cloud/ . Actually, to do a stable/* release, check out that branch of DevStack, it has the correct stable branch settings in stackrc already. But to do milestones, you would want DevStack somewhat close (for example, current DevStack master is still OK for the I-1 milestone) then set the XX_BRANCH variables manually. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Tuskar CLI after architecture changes
On Wed, Dec 11, 2013 at 9:35 AM, James Slagle james.sla...@gmail.comwrote: On Wed, Dec 11, 2013 at 7:33 AM, Jiří Stránský ji...@redhat.com wrote: Previously, we had planned Tuskar arcitecture like this: tuskar-ui - tuskarclient - tuskar-api - heat-api|ironic-api|etc. To be clear, tuskarclient is just a library right? So both the UI and CLI use tuskarclient, at least was that the original plan? I would expect tuskarclient above to be the Python API bindings without the business logic. I don't think we want the business logic in the UI. +1 Now this raises a question - how do we get CLI reasonably on par with abilities of the UI? (Or am i wrong that Anna the infrastructure administrator would want that?) IMO, we want an equivalent CLI and UI. A big reason is so that it can be sanely scripted/automated. At a minimum you need to be sure that all of the atomic operations in your business logic are exposed via _some_ API. ie, to script something the script may be where the business logic exists. Building on that is moving that logic into a library that calls multiple Python client APIs. This may or may not be part of tuskarclient. The next step up is to put your business logic into what we used to call middleware, the layer between client and backend. This is server-side and IMHO where it belongs. This is really the ONLY way you can ensure that various clients get the same experience. python-openstackclient consumes other clients :). Ok, that's probably not a great example :). :) No, not really. But it is also developing some 'value-added' functions that are cross-project APIs and has a similar problem. So far that is just smoke and mirrors hiding the duck tape behind the scenes but it is not unlike some of the things that Horizon does for user convenience. This approach makes the most sense to me. python-tuskarclient would make the decisions about if it can call the heat api directly, or the tuskar api, or some other api. The UI and CLI would then both use python-tuskarclient. If you do this keep the low-level API bindings separate from the higher-level logical API. 2) Make a thicker tuskar-api and put the business logic there. (This is the original approach with consuming other services from tuskar-api. The feedback on this approach was mostly negative though.) So, typically, I would say this is the right approach. However given what you pointed out above that sometimes we can use other API's directly, we then have a seperation where sometimes you have to use tuskar-api and sometimes you'd use heat/etc api. By using python-tuskarclient, you're really just pushing that abstraction into a library instead of an API, and I think that makes some sense. Consider that pushig out to the client requires that the client be in sync with what is deployed. You'll have to make sure your client logic can handle the multiple versions of server APIs that it will encounter. Putting that server-side lets you stay in sync with the other OpenStack APIs you need to use. 3) Keep tuskar-api and python-tuskarclient thin, make another library sitting between Tuskar UI and all python-***clients. This new project would contain the logic of using undercloud services to provide the tuskar experience it would expose python bindings for Tuskar UI and contain a CLI. (Think of it like traditional python-*client but instead of consuming a REST API, it would consume other python-*clients. I wonder if this is overengineering. We might end up with too many projects doing too few things? :) ) I don't follow how this new library would be different from python-tuskarclient. Unless I'm just misinterpreting what python-tuskarclient is meant to be, which may very well be true :). This is essentially what I suggested above. It need not be a separate repo or installable package, but the internal API should have its own namespace/modules/whatever. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [TripleO] Tuskar CLI after architecture changes
On Wed, Dec 11, 2013 at 10:41 AM, Jay Dobies jason.dob...@redhat.comwrote: I'm still fuzzy on what OpenStack means when it says *client. Is that just a bindings library that invokes a remote API or does it also contain the CLI bits? For the older python-*client projects they are both Python API bindings and a this CLI on top of them. Some of the newer clients may not include a CLI. By default I think most people mean the library API when referring to clients without 'CLI'. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [policy] Let's make topics mandatory for openstack-dev
On Thu, Dec 5, 2013 at 6:23 AM, Sean Dague s...@dague.net wrote: We already have that functionality with the Topics support in mailman. You can sign up for just specific topics there and only get those topics. Emails are tagged with an X-Topics: header for filtering (much more reliable than subject lines). This topic comes up regularly these days and this and another message from Stefano[1] are helpful to using topics so I've added it to the MailingList wiki page[2]. Plus pointers to split-the-list threads so we can maybe refresh on why things are the way they are. [1] http://lists.openstack.org/pipermail/openstack-dev/2013-November/019233.html [2] https://wiki.openstack.org/wiki/Mailing_Lists#Future_Development dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [Openstack] Regarding Openstack upgrade for havana onwards
On Thu, Dec 5, 2013 at 8:01 PM, Joe Hakim Rahme joe.hakim.ra...@enovance.com wrote: The only automated upgrade I know of is Grenade[1]. You should check it out. I should clarify this a bit to keep expectations realistic: Grenade is not a tool for performing upgrades, especially on production deployments. It is designed for testing the common upgrade path to detect when unexpected incompatabilities arise. It uses DevStack to set up the running system so right there it is not suitable for production use. It is useful to look at to get an idea of the minimum that needs to be done during an upgrade, though. dt -- Dean Troyer dtro...@gmail.com ___ Mailing list: http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack Post to : openstack@lists.openstack.org Unsubscribe : http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack
Re: [openstack-dev] [Solum] CLI minimal implementation
On Mon, Dec 2, 2013 at 10:09 PM, Adrian Otto adrian.o...@rackspace.comwrote: Sorry, I changed the link. We originally started with hyphenated noun-verbs but switched to the current proposal upon receipt of advice that it would be more compatible with the next version of the cliff based CLI for OpenStack. If I remember correctly this advice came from Doug Hellman. That may have been me, I had the same conversation three or four times during and right after the summit. Either way, I'm the one to blame for that command format... Etherpad for discussion notes: https://etherpad.openstack.org/p/MinimalCLI I left some comments about how I would format the commands to be consistent with OSC. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Keystone][Oslo] Future of Key Distribution Server, Trusted Messaging
On Sun, Nov 24, 2013 at 1:52 PM, Morgan Fainberg m...@metacloud.com wrote: The other concern is the library interfacing with KDS (I would assume this goes into keystoneclient? At least for the time being). I would rather see the client get its own repo, too. We still need to do that with the middleware. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] tenant or project
On Sat, Nov 23, 2013 at 11:35 AM, Dolph Mathews dolph.math...@gmail.comwrote:
+1 for using the term project across all services. Projects provide
multi-tenant isolation for resources across the cloud. Part of the reason
we prefer projects in keystone is that domains conceptually provide
multi-tenant isolation within keystone itself, so the overloaded tenant
terminology gets really confusing.
- keystoneclient already supports projects from a library perspective
(including auth_token)
Thanks you! I will eventually be able to remove my disparaging comments
and work-arounds in OSC for tenantId vs tenant_id!!!
- keystoneclient's CLI is deprecated in favor of openstackclient's CLI,
which supports the project terminology if you pass the
--identity-api-version=3 flag
FWIW I followed Horizon's lead in OSC and removed the tern 'tenant' from
all user-visible parts, except for the compatability OS_TENAMT_{ID,NAME}
variables and --os-tenant-{id,name} options. Neither of those is documented
anywhere though. This includes commands for all OS APIs it supports.
dt
--
Dean Troyer
dtro...@gmail.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] How to best make User Experience a priority in every project
On Wed, Nov 20, 2013 at 9:09 AM, Thierry Carrez thie...@openstack.orgwrote: However, as was apparent in the Technical Committee meeting discussion about it yesterday, most of us are not convinced that establishing and blessing a separate team is the most efficient way to give UX the attention it deserves. Ideally, UX-minded folks would get active *within* existing project teams rather than form some sort of counter-power as a separate team. In the same way we want scalability and security mindset to be present in every project, we want UX to be present in every project. It's more of an advocacy group than a program imho. Having been working on a cross-project project for a while now I can confirm that there is a startling lack of coordination between projects on the same/similar tasks. Oslo was a HUGE step toward reducing that and providing a good reference for code and libraries. There is nothing today for the intangible parts that are both user and developer facing such as common message (log) formats, common terms (tenant vs project) and so on. I think the model of the OSSG is a good one. After reading the log of yesterday's meeting I think I would have thrown in that the need from my perspective is for a coordination role as much as anything. The deliverables in the UX Program proposal seem a bit fuzzy to me as far as what might go into the repo. If it is interface specs, those should be in either the project code repos docs/ tree or in the docs project directly. Same for a Human Interface Guide (HIG) that both Horizon and OSC have (well, I did steal a lot of OSC's guide from Horizon's). The interaction with users seems very valuable to me. Frankly, user polls are not my favorite task, even if I always learn a lot about my project watching users bend it in ways I never imagined. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Openstack + OpenContrail
On Mon, Nov 18, 2013 at 12:50 AM, Pedro Roque Marques
pedro.r.marq...@gmail.com wrote:
Does it address the catch-22 problem that a Neutron reviewer asks for the
plugin to be accepted upstream into devstack as pre-condition for
acceptance whether a devstack reviewer as for the plugin to be upstreamed
into Neutron first ?
Yes, that is the intention. However, you have to do the work of
integrating it, it isn't going to go into the existing gate right off, if
ever. Mark's message this morning[1] lays it out from their point of view.
He mentions using the CI third party testing integration [2] in the
'Testing Requirements' section. Ideally, you get that running with a trunk
DevStack plus your plugin against the Nova and Neutron branches with your
contrail plugin/mods.
Also, DevStack does not install support services that are not packaged in
the underlying distro. Look at Docker's split between the support
service(s) that start before stack.sh runs and the parts that specifically
configure Nova.
Can you please elaborate as to what are support services in this context
?
Things required of the running environment that are not shipped with the
OS need to be in place before stack.sh runs. i.e., you are compiling a
kernel module. DevStack's workflow is not an appropriate place to do that.
As another example, we have an install script for Docker that takes care
of their service installation and startup that is up to the end-user ot
have run before running stack.sh. This keeps us (DevStack) out of the
business of trying to maintain something we {can not/do not} test.
The patches that are being applied by the script have been submitted for
review. Using a patch approach is helpful in attempting to demonstrate that
the resulting code works against the master branch of Nova/Neutron.
The OpenStack methodology for this is to fork the repo and create a branch
of the source tree with your patch and test against that.
For the gate testing, DevStack actually does NONE of the source branch
setup work. A CI component names Zuul does all of that, pulling branches
and repos as it requires to set up the correct environment. Patching the
source trees after that destroys the integrity of what
Gerrit/Jenkins/devstack-gate thinks it is testing.
Can you please suggest a sequence of steps... ?
I understand that it makes sense to follow the plugin documentation
specified above. That would be the first step. But it is not clear to me
how to break it down further while having something that is still
functional.
If you get it running as a plugin and there are no changes required to the
existing DevStack files then you are done. Any changes to DevStack
indicate a shortcoming in the plugin interface and may need to be
generalized anyway.
dt
[1]
http://www.mail-archive.com/openstack-dev@lists.openstack.org/msg08700.html
[2] http://ci.openstack.org/third_party.html
--
Dean Troyer
dtro...@gmail.com
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Openstack + OpenContrail
On Sat, Nov 16, 2013 at 11:15 AM, Harshad Nakil hna...@contrailsystems.comwrote: Sean, We have diff in three repositories. Nova, Neutron and devstack. Each review is requiring other to happen first. Do you have recommendation how do we deal with these dependencies? First off, if you rebase on to a current DevStack you will find a new plugin mechanism specifically designed to address this sort of problem. You've already worked out the plugin bits for Neutron, the parts for stack.sh are similar, located in extras.d. http://devstack.org/plugins.htmldescribes how it works. Also, DevStack does not install support services that are not packaged in the underlying distro. Look at Docker's split between the support service(s) that start before stack.sh runs and the parts that specifically configure Nova. The patches to Neutron and Nova should be handled by setting the *_BRANCH and *_REPO variables to point to your repo and branch. DevStack will check them out for you when it installs the project source. You should be able to re-arrange things to support this architecture. Also, expect to break the remaining DevStack changes into digestible bits. As it stands, your branch is unmergable, even if it was based on a semi-current commit. FWIW, the opencontrail.org website appears to be off the air making it harder to understand what it is you are trying to integrate here. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Shall backward compatibility env. vars be removed from python-clients?
On Mon, Nov 11, 2013 at 12:46 PM, Kevin L. Mitchell kevin.mitch...@rackspace.com wrote: The thing here is, these environment variables have already been deprecated for quite some time. The question is, have they been deprecated long enough that we're clear to drop them entirely? I say yes, over a year in the auth variable cases seems to be plenty long to me. Related but different are the now-long-undocumented option names with underscores in them. I've noticed new undocumented ones get added from time to time alongside the same option with dashes. ??? dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [Solum] Command Line Interface for Solum
On Mon, Nov 11, 2013 at 12:01 AM, Clayton Coleman ccole...@redhat.comwrote: The implication I heard from the session is the command infrastructure and client code have or will have a well defined interface from the framework, although I haven't looked through current state enough to say that's totally true today. If that interface is in place we certainly would be better isolated from upstream. OSC has a very simple plugin mechanism that is little more than pre-defined entry point groups that are scanned for additional commands. That works but doesn't properly leverage the desirable common bits, such as auth, that are available to the built-in clients. Enhancing this has not been a high priority before now but needs to be done. I guess it depends where we want to spend our focus as well - do we write something that in the long run we'd end up porting, or benefit from the work the common client is doing (doc, common patterns, keystone auth work with x509 and potentially Kerberos, bash completion, etc) as well as help them out. I'd suggest you focus on putting together a clean and well-defined library API that implements your REST API. Consider that work is going on to refactor the implementation of these layers in existing clients and that some of the library APIs will be changing. Specifically, the first of these efforts is a much-needed cleanup of the Identity Auth interfaces and the migration of existing clients to use that rather than their own embedded auth. Please don't just copy one of the existing clients, down that path lies madness. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Migrating to newer full projects from what used to be part of nova
On Fri, Nov 1, 2013 at 1:38 PM, Devananda van der Veen devananda@gmail.com wrote: Actually, anyone deploying nova with the baremetal driver will face a similar split when Ironic is included in the release. I'm targeting Icehouse, but of course, it's up to the TC when Ironic graduates. This should have a smaller impact than either the neutron or cinder splits, both of which were in widespread use, but I expect we'll see more usage of nova-baremetal crop up now that Havana is released. I didn't recall in which release baremetal was first a supported option, is it only now in Havana? Is it clear in the docs that this sort of situation is coming in the next release or two? (And no, I haven't gone to look for myself, maybe on the plane tomorrow...) dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Migrating to newer full projects from what used to be part of nova
On Thu, Oct 31, 2013 at 2:18 PM, Jesse Pretorius jesse.pretor...@gmail.comwrote: On 31 October 2013 18:46, John Griffith john.griff...@solidfire.comwrote: Upgrading Essex-Folsom introduced both the challenge of upgrading nova-volume to cinder and the challenge of upgrading nova-network to quantum. Upgrading Folsom-Grizzly presents the challenge of migrating nova-network to quantum and assumes that nova-volume has already been migrated to cinder. Upgrading Grizzly-Havana finally closes the door on nova-network as far as I can see, although I may be wrong. To clarify a bit, while deprecated, nova-network has not lost any functionality in Havana yet. It also hasn't gained much (if any), but it still works. The majority of the testing is performed using nova-network and not neutron (yet). It comes down to the fact that while we cater for migrating between versions of a particular project we don't cater particularly well for migrating between projects when a project splits out from a parent project as was the case for both of the above. I am not aware of any effort to do a migration from nova-network to neutron, we certainly don't have it in grenade's near future. The differences are large and the development cost for that sort of migration is significant for something that a given deployment is only going to use once. It isn't a technical problem but a resource problem. Also, FWIW, I don't see another one of these situations coming anytime soon. All of the new project activity is around new services/features. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] Remove vim modelines?
On Fri, Oct 25, 2013 at 2:43 PM, Robert Collins robe...@robertcollins.netwrote: On 26 October 2013 08:40, Dolph Mathews dolph.math...@gmail.com wrote: I'm all for low barriers of entry, so if there's any evidence that this is true, I'd want to make them more prolific. I'm not sure how to gather evidence for this, either for or against ;(. We do have a captive audience in a week or so to do unscientific room-temp surveys. Asking a question or two at the end of some sessions would at least give us a data point broader than ML participants. Who would be resistant to enforcing removal modelines? Who would be resistant to enforcing addition of modelines? Who doesn't know or care what a modeline is? For the record, I don't mind these things as long as they are at the end of the file. dt -- Dean Troyer dtro...@gmail.com ___ OpenStack-dev mailing list OpenStack-dev@lists.openstack.org http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
