Re: [openstack-dev] Re-initializing or dynamically configuring cinder driver

[Joshua Harlow]

I don't currently know of a one size fits all solution here. There was talk at 
the summit of having the cinder app respond to a SIGHUP signal and attempting 
to reload config on this signal. Dynamic reloading is tricky business 
(basically u need to unravel anything holding references to the old config 
values/affected by the old config values).

I would start with a simple trial of this if u want to so it, part if the issue 
will likely be oslo.config (can that library understand dynamic reloading?) and 
then cinder drivers themselves (perhaps u need to create a registry of drivers 
that can dynamically reload on config reloads?). Start out with something 
simple, isolate the reloading as much as u can to a single area (something like 
the mentioned registry of objects that can be reloaded when a SIGHUP arrives) 
and see how it goes.

It does seem like a nice feature if u can get it right :-)

Sent from my really tiny device...

On Dec 13, 2013, at 8:57 PM, "iKhan" 
mailto:ik.ibadk...@gmail.com>> wrote:

Hi All,

At present cinder driver can be only configured with adding entries in conf 
file. Once these driver related entries are modified or added in conf file, we 
need to restart cinder-volume service to validate the conf entries and create a 
child process that runs in background.

I am thinking of a way to re-initialize or dynamically configure cinder driver. 
So that I can accept the configuration from user on fly and perform operations. 
I think solution lies somewhere around "oslo.config.cfg", but I am still 
unclear about how re-initializing can be achieved.

Let know if anyone here is aware of any approach to re-initialize or 
dynamically configure a driver.

--
Thanks,
IK
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

[openstack-dev] [Cinder] Tracking mox to mock unit tests?

[Subramanian]

Is there a bug/bp tracking the overall cinder unit tests move from mox to
mock? Is this targeted to complete in IceHouse?

Thanks,
Subbu
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Generic question: Any tips for 'keeping up' with the mailing lists?

[James Bottomley]

On Thu, 2013-12-12 at 16:23 +, Justin Hammond wrote:
> I am a developer who is currently having troubles keeping up with the
> mailing list due to volume, and my inability to organize it in my client.
> I am nearly forced to use Outlook 2011 for Mac and I have read and
> attempted to implement
> https://wiki.openstack.org/wiki/MailingListEtiquette but it is still a lot
> to deal with. I read once a topic or wiki page on using X-Topics but I
> have no idea how to set that in outlook (google has told me that the
> feature was removed).
> 
> I'm not sure if this is a valid place for this question, but I *am* having
> difficulty as a developer.
> 
> Thank you for anyone who takes the time to read this.

I currently use the same technique I used to use with LKML (Linux Kernel
Mailing List) before we (finally) got subject specific lists and I could
just subscribe to those instead.

The essential requirement is to have a threaded mail reader with a "mark
thread read" function.  I use evolution where this is ctrl-H ctrl-K.  I
skip through the threads marking those I'm not interested in as read.
Sometimes I do this by subject other times by skimming the thread head
email.  When I'm finished, everything I didn't skip as read is usually
the interesting stuff.

Threads I'm really interested in, I tag with an imap flag which I have
an evolution search folder set on for flag+children (and which pops up
into my mail notification window), so I see immediately any activity on
interesting threads.

I also use procmail on the mail server with a set of heuristics mostly
on subject for things I may be interested in, these get flagged as well,
so they appear in my email notifications in evolution.  Likewise, things
I'm definitely not interested in get tagged as read by the heuristics.

The procmail thing is server side and means you need a sophisticated
mail server, but anything that runs Linux should have sieve which should
be able to do this (not sure about gmail, because I've never used it).

James



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Neutron] Cores - Prioritize merging migration fixes after tox change merges

[Jeremy Stanley]

On 2013-12-13 23:43:09 +0800 (+0800), Maru Newby wrote:
> As per Anita's email, we're not to approve anything until the
> following tox fix merges:  https://review.openstack.org/#/c/60825
[...]

Merged at 07:58 UTC today. Also, the gate is basically empty, so
those other changes will merge rather quickly if you decide to
revisit approving them before the Monday rush.
-- 
Jeremy Stanley

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] suggestion to a new driver

[John Griffith]

Hi Ronan,
Best advice I would give is start with the base driver class (
cinder.volume.driver.py) and the reference LVM driver (cinder
.volume.drivers.lvm.py).  Those will give you a template of the interfaces,
args needed and return values.

Also jump in IRC at #openstack-cinder and we can chat more there as well.

Thanks,
John
On Dec 10, 2013 2:04 AM, "Ronen Angluster"  wrote:

> Hello all!
>
> we're developing a new storage appliance and per one of our customers
> would like
> to build a cinder driver.
> i kept digging into the documentation for the past 2 weeks and could not
> find anything that described the code level of API. i.e. nothing describes
> what each function should
> receive and what it should return.
> is there a document that describe it and i missed it? if not, who can
> provide that missing information?
>
> Ronen
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Cinder] suggestion to a new driver

[Matt Riedemann]

On Tuesday, December 10, 2013 2:59:19 AM, Ronen Angluster wrote:

Hello all!

we're developing a new storage appliance and per one of our customers
would like
to build a cinder driver.
i kept digging into the documentation for the past 2 weeks and could not
find anything that described the code level of API. i.e. nothing
describes what each function should
receive and what it should return.
is there a document that describe it and i missed it? if not, who can
provide that missing information?

Ronen


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


I don't work on cinder, but I'm guessing this will get you started:

http://docs.openstack.org/api/openstack-block-storage/2.0/content/
https://github.com/openstack/cinder/blob/master/doc/source/devref/drivers.rst

I'm also not sure where the cinder team is with 3rd party CI 
requiements, but you'll want to at least read this also:


http://lists.openstack.org/pipermail/openstack-dev/2013-July/012557.html

--

Thanks,

Matt Riedemann


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Tuskar CLI after architecture changes

[Jay Pipes]

On Sat, 2013-12-14 at 10:52 -0500, Tzu-Mainn Chen wrote:
> > On Wed, 2013-12-11 at 17:48 +0100, Jiří Stránský wrote:
> > 
> > > >> When you say python- clients, is there a distinction between the CLI 
> > > >> and
> > > >> a bindings library that invokes the server-side APIs? In other words,
> > > >> the CLI is packaged as CLI+bindings and the UI as GUI+bindings?
> > > 
> > > python-tuskarclient = Python bindings to tuskar-api + CLI, in one project
> > > 
> > > tuskar-ui doesn't have it's own bindings, it depends on
> > > python-tuskarclient for bindings to tuskar-api (and other clients for
> > > bindings to other APIs). UI makes use just of the Python bindings part
> > > of clients and doesn't interact with the CLI part. This is the general
> > > OpenStack way of doing things.
> > 
> > Please everyone excuse my relative lateness in joining this discussion,
> > but I'm wondering if someone could point me to discussions (or summit
> > session etherpads?) where the decision was made to give Tuskar a
> > separate UI from Horizon? I'm curious what the motivations were around
> > this?
> > 
> > Thanks, and again, sorry for being late to the party! :)
> > 
> > -jay
> > 
> 
> Heya - just to clear up what I think might be a possible misconception here - 
> the
> Tuskar-UI is built on top of Horizon.  It's developed as a separate Horizon 
> dashboard
> - Infrastructure - that can be added into the OpenStack dashboard alongside 
> the existing
> dashboards - Project, Admin.  The Tuskar-UI developers are active within 
> Horizon, and
> there's currently an effort underway to get the UI placed under the Horizon 
> program.
> 
> Does that answer your question, or did I miss the thrust of it?

That precisely answers my questions! :) Thanks, Mainn!

-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [OpenStack-dev] How to modify a bug across multiple repo?

[wu jiang]

Hi Christopher,

Thanks again for your answering. :)

The issue I mentioned before, has already committed to Gerrit.
So if you have time, please see here:
https://review.openstack.org/#/c/62163/

P.S. Is it a good idea if I ask the contributor to participate in reviewing?

Any help will be appreciated.


wingwj



On Fri, Dec 13, 2013 at 8:17 PM, Christopher Yeoh  wrote:

> Hi wu jiang,
>
>
>
> On Wed, Dec 11, 2013 at 12:26 PM, wu jiang  wrote:
>
>> Hi Chris & Rob,
>>
>>
>> Thanks for your reply ,and sorry for my late response..
>>
>> - I tested again. The modification won't effect tempest test because it's
>> an optional argument, so I can commit it later in tempest. Lucky.
>>
>> - But the bug in Cinder exists at the API layer, so the modification will
>> effect the CinderClient behavior..  :(
>>
>> So, that's quite a complex problem.. Any ideas?
>>
>>
> To help I think I'd need more details on the exact bug. Does the change
> that you need to make comply with
> the guidelines here https://wiki.openstack.org/wiki/APIChangeGuidelines ?
>
> Chris
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] Unified Guest Agent proposal

[Steven Dake]

On 12/13/2013 01:13 PM, Clint Byrum wrote:

Excerpts from Dmitry Mescheryakov's message of 2013-12-13 12:01:01 -0800:

Still, what about one more server process users will have to run? I see
unified agent as library which can be easily adopted by both exiting and
new OpenStack projects. The need to configure and maintain Salt server
process is big burden for end users. That idea will definitely scare off
adoption of the agent. And at the same time what are the gains of having
that server process? I don't really see to many of them.



I tend to agree, I don't see a big advantage to using something like 
salt, when the current extremely simplistic cfn-init + friends do the job.


What specific problem does salt solve?  I guess I missed that context in 
this long thread.



The Salt devs already mentioned that we can more or less just import
salt's master code and run that inside the existing server processes. So
Savanna would have a salt master capability, and so would Heat Engine.
I really don't think we want to introduce a salt executive into the heat 
engine process address space, even if it is as simple as an import 
operation.  Sounds like a debugging nightmare!



If it isn't eventlet friendly we can just fork it off and run it as its
own child. Still better than inventing our own.
fork/exec is not the answer to scalability and availability I was 
looking for :)  So, given that we need scale+availability, we are back 
to managing a daemon outside the address space, which essentially 
introduces another daemon to be scaled and ha-ified (and documented, 
etc, see long zookeeper thread for my arguments against new server 
processes...).  import is not the answer, or atleast it won't be for heat...


Salt just seems like more trouble then it is worth, but I don't totally 
understand the rationale for introducing it as a dependency in this 
case, and I generally think dependencies are evil :)


What are we inventing our own again of?  cfn-init & friends already 
exist, are dead simple, and have no need of anything beyond a metadata 
server.  I would like to see that level of simplicity in any unified agent.


Regards
-steve


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][TripleO] Nested resources

[Clint Byrum]

Excerpts from Sylvain Bauza's message of 2013-12-14 06:23:48 -0800:
> 2013/12/9 Clint Byrum 
> 
> > Excerpts from Fox, Kevin M's message of 2013-12-09 09:34:06 -0800:
> > > I'm thinking more generic:
> > >
> > > The cloud provider will provide one or more "suballocating" images. The
> > one Triple O uses to take a bare metal node and make vm's available would
> > be the obvious one to make available initially. I think that one should not
> > have a security concern since it is already being used in that way safely.
> >
> > I like where you're going with this, in that the cloud should eventually
> > become "self aware" enough to be able to privision the baremetal resources
> > it has and spin nova up on them. I do think that is quite far out. Right
> > now, we have two nova's.. an undercloud nova which owns all the baremetal,
> > and an overcloud nova which owns all the vms. This is definitely nested,
> > but there is a hard line between the two.
> >
> > For many people, that hard line is a feature. For others, it is a bug.  :)
> >
> >
> Could we imagine that an end-user would like to provision one undercloud
> host plus a certain number of overcloud nodes so that the Scheduler for
> undercloud Nova would deny other hosts but the ones provisioned ?

Yes I could imagine that. I also imagine that does not require any special
knowledge of the undercloud that the overcloud's nova API doesn't already
have access to. The "host" is a thing in the overcloud after all.

> As a contrary,  Scheduler for other undercloud Nova's need to deny the
> provisioning of the nodes hosted by another tenant than the requester...
> I played with TripleO a few months ago (August/September, before the merge
> with Tuskar) so that's a bit unclear for me, but I'm just saying we could
> potentially achieve this by using Climate which does deploy its own
> Scheduler Filter for making sure the proper hosts are booted.
> 

Currently undercloud nova has one tenant: the overcloud operator. It is
single tenant, which means it has less complexity, but that also means you
can't hand hardware out directly to multiple tenants. That is why I say
that some consider it a feature, and some a bug. IMO that is how it should
remain, and we should just enhance systems like Climate to be more aware
of the topology of the hosts which are already an entity in the overcloud.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Tempest][Neutron] Network client and API tests refactoring.

[Jay Pipes]

On Sat, 2013-12-14 at 19:09 +0400, Eugene Nikanorov wrote:
> Hi Jay,
> 
> Sure, that is understood. In fact such refactoring could be a big
> change so I'd split it to two or more patches.
> Hope this will not overlap with ongoing neutron API tests development.

Hehe, given the sheer number of new tests that get added to Tempest
every week, I'd say that any sort of base refactoring like this will
need to be heavily coordinated with the Tempest core reviewers and other
contributors pushing code!

Best,
-jay



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Tuskar CLI after architecture changes

[Tzu-Mainn Chen]

> On Wed, 2013-12-11 at 17:48 +0100, Jiří Stránský wrote:
> 
> > >> When you say python- clients, is there a distinction between the CLI and
> > >> a bindings library that invokes the server-side APIs? In other words,
> > >> the CLI is packaged as CLI+bindings and the UI as GUI+bindings?
> > 
> > python-tuskarclient = Python bindings to tuskar-api + CLI, in one project
> > 
> > tuskar-ui doesn't have it's own bindings, it depends on
> > python-tuskarclient for bindings to tuskar-api (and other clients for
> > bindings to other APIs). UI makes use just of the Python bindings part
> > of clients and doesn't interact with the CLI part. This is the general
> > OpenStack way of doing things.
> 
> Please everyone excuse my relative lateness in joining this discussion,
> but I'm wondering if someone could point me to discussions (or summit
> session etherpads?) where the decision was made to give Tuskar a
> separate UI from Horizon? I'm curious what the motivations were around
> this?
> 
> Thanks, and again, sorry for being late to the party! :)
> 
> -jay
> 

Heya - just to clear up what I think might be a possible misconception here - 
the
Tuskar-UI is built on top of Horizon.  It's developed as a separate Horizon 
dashboard
- Infrastructure - that can be added into the OpenStack dashboard alongside the 
existing
dashboards - Project, Admin.  The Tuskar-UI developers are active within 
Horizon, and
there's currently an effort underway to get the UI placed under the Horizon 
program.

Does that answer your question, or did I miss the thrust of it?

Mainn


> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
> 

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Tuskar CLI after architecture changes

[Jay Pipes]

On Thu, 2013-12-12 at 15:22 +0100, Hugh O. Brock wrote:
> On Thu, Dec 12, 2013 at 03:11:14PM +0100, Ladislav Smola wrote:
> > Agree with this.
> > 
> > Though I am an optimist,  I believe that this time, we can avoid
> > calling multiple services in one request that depend on each other.
> > About the multiple users at once, this should be solved inside the
> > API calls of the services.
> > 
> > So I think we should forbid building these complex API calls
> > composites in the Tuskar-API. If we will want something like this,
> > we should implement
> > it properly inside the services itself. If we will not be able to
> > convince the community about it, maybe it's just not that good
> > feature. :-D
> > 
> 
> It's worth adding that in the particular case Radomir sites (the
> "Deploy" button), even with all the locks in the world, the resources
> that we have supposedly requisitioned in the undercloud for the user may
> have already been allocated to someone else by Nova -- because Nova
> currently doesn't allow reservation of resources. (There is work under
> way to allow this but it is quite a way off.) So we could find ourselves
> claiming for the user that we're going to deploy an overcloud at a
> certain scale and then find ourselves unable to do so.
> 
> Frankly I think the whole multi-user case for Tuskar is far enough off
> that I would consider wrapping a single-login restriction around the
> entire thing and calling it a day... except that that would be
> crazy. I'm just trying to make the point that making these operations
> really safe for multiple users is way harder than just putting a lock on
> the tuskar API.

That's actually not that crazy, Hugh :) We've deployed more than a half
dozen availability zones, and I've never seen anyone trample over each
other trying to deploy OpenStack to the same set of bare-metal machines
at the same time... it simply doesn't happen in the real world -- or at
least, it would be so exceedingly rare that trying to deal with this
kind of thing is more of an academic exercise than anything else.

Instead of focusing on locking issues -- which I agree are very
important in the virtualized side of things where resources are
"thinner" -- I believe that in the bare-metal world, a more useful focus
would be to ensure that the Tuskar API service treats related group
operations (like "deploy an undercloud on these nodes") in a way that
can handle failures in a graceful and/or atomic way.

For example, if the construction or installation of one compute worker
failed, adding some retry or retry-after-wait-for-event logic would be
more useful than trying to put locks in a bunch of places to prevent
multiple sysadmins from trying to deploy on the same bare-metal nodes
(since it's just not gonna happen in the real world, and IMO, if it did
happen, the sysadmins/deployers should be punished and have to clean up
their own mess ;)

Best,
-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Support for Pecan in Nova

[Doug Hellmann]

On Sat, Dec 14, 2013 at 7:55 AM, Christopher Yeoh  wrote:

>
> On Sat, Dec 14, 2013 at 8:48 AM, Doug Hellmann <
> doug.hellm...@dreamhost.com> wrote:
>
>> That covers routes. What about the properties of the inputs and outputs?
>>
>>
> I think the best way for me to describe it is that as the V3 API core and
> all the extensions
> are written, both the routes and input and output parameters are from a
> client's perspective fixed at application
> startup time. Its not an inherent restriction of the framework (an
> extension could for
> example dynamically load another extension at runtime if it really wanted
> to), but we just don't do that.
>

OK, good.



>
> Note that values of parameters returned can be changed by an extension
> though. For example os-hide-server-addresses
> can based on a runtime policy check and the vm_state of the server, filter
> whether the values in the
> addresses field are filtered out or not when returning information about a
> server. This isn't a new thing in the
> V3 API though, it already existed in the V2 API.
>

OK, it seems like as long as the fields are still present that makes the
API at least consistent for a given deployment's configuration.

Doug



>
> Chris
>
>
>>
>> On Fri, Dec 13, 2013 at 4:43 PM, Ryan Petrello <
>> ryan.petre...@dreamhost.com> wrote:
>>
>>> Unless there’s some other trickiness going on that I’m unaware of, the
>>> routes for the WSGI app are defined at application startup time (by methods
>>> called in the WSGI app’s __init__).
>>>
>>> ---
>>> Ryan Petrello
>>> Senior Developer, DreamHost
>>> ryan.petre...@dreamhost.com
>>>
>>> On Dec 13, 2013, at 12:56 PM, Doug Hellmann 
>>> wrote:
>>>
>>> >
>>> >
>>> >
>>> > On Thu, Dec 12, 2013 at 9:22 PM, Christopher Yeoh 
>>> wrote:
>>> > On Fri, Dec 13, 2013 at 4:12 AM, Jay Pipes  wrote:
>>> > On 12/11/2013 11:47 PM, Mike Perez wrote:
>>> > On 10:06 Thu 12 Dec , Christopher Yeoh wrote:
>>> > On Thu, Dec 12, 2013 at 8:59 AM, Doug Hellmann
>>> > >> > >wrote:
>>> >
>>> >
>>> >
>>> >
>>> > On Wed, Dec 11, 2013 at 3:41 PM, Ryan Petrello <
>>> > ryan.petre...@dreamhost.com
>>> > >
>>> > wrote:
>>> >
>>> > Hello,
>>> >
>>> > I’ve spent the past week experimenting with using Pecan for
>>> > Nova’s
>>> > API
>>> > and have opened an experimental review:
>>> >
>>> >
>>> > https://review.openstack.org/#/c/61303/6
>>> >
>>> > …which implements the `versions` v3 endpoint using pecan (and
>>> > paves the
>>> > way for other extensions to use pecan).  This is a *potential*
>>> >
>>> > approach
>>> > I've considered for gradually moving the V3 API, but I’m open
>>> > to other suggestions (and feedback on this approach).  I’ve
>>> > also got a few open questions/general observations:
>>> >
>>> > 1.  It looks like the Nova v3 API is composed *entirely* of
>>> > extensions (including “core” API calls), and that extensions
>>> > and their routes are discoverable and extensible via installed
>>> > software that registers
>>> > itself
>>> > via stevedore.  This seems to lead to an API that’s composed of
>>> >
>>> > installed
>>> > software, which in my opinion, makes it fairly hard to map out
>>> > the
>>> > API (as
>>> > opposed to how routes are manually defined in other WSGI
>>> > frameworks).  I
>>> > assume at this time, this design decision has already been
>>> > solidified for
>>> > v3?
>>> >
>>> >
>>> > Yeah, I brought this up at the summit. I am still having some
>>> > trouble understanding how we are going to express a stable core
>>> > API for compatibility testing if the behavior of the API can be
>>> > varied so significantly by deployment decisions. Will we just
>>> > list each
>>> > "required"
>>> > extension, and forbid any extras for a compliant cloud?
>>> >
>>> >
>>> > Maybe the issue is caused by me misunderstanding the term
>>> > "extension," which (to me) implies an optional component but is
>>> > perhaps reflecting a technical implementation detail instead?
>>> >
>>> >
>>> > Yes and no :-) As Ryan mentions, all API code is a plugin in the V3
>>> > API. However, some must be loaded or the V3 API refuses to start
>>> > up. In nova/api/openstack/__init__.py we have
>>> > API_V3_CORE_EXTENSIONS which hard codes which extensions must be
>>> > loaded and there is no config option to override this (blacklisting
>>> > a core plugin will result in the V3 API not starting up).
>>> >
>>> > So for compatibility testing I think what will probably happen is
>>> > that we'll be defining a minimum set (API_V3_CORE_EXTENSIONS) that
>>> > must be implemented and clients can rely on that always being
>>> > present
>>> > on a compliant cloud. But clients can also then query through
>>> > /extensions what other functionality (which is backwards compatible
>>> > with respect to core) may also be present on that specific cloud.
>>> >
>>> > This really seems similar to the idea of having a router class, some
>>> > controllers and you map them. From my o

Re: [openstack-dev] [TripleO] Tuskar CLI after architecture changes

[Jay Pipes]

On Wed, 2013-12-11 at 17:48 +0100, Jiří Stránský wrote:

> >> When you say python- clients, is there a distinction between the CLI and
> >> a bindings library that invokes the server-side APIs? In other words,
> >> the CLI is packaged as CLI+bindings and the UI as GUI+bindings?
> 
> python-tuskarclient = Python bindings to tuskar-api + CLI, in one project
> 
> tuskar-ui doesn't have it's own bindings, it depends on 
> python-tuskarclient for bindings to tuskar-api (and other clients for 
> bindings to other APIs). UI makes use just of the Python bindings part 
> of clients and doesn't interact with the CLI part. This is the general 
> OpenStack way of doing things.

Please everyone excuse my relative lateness in joining this discussion,
but I'm wondering if someone could point me to discussions (or summit
session etherpads?) where the decision was made to give Tuskar a
separate UI from Horizon? I'm curious what the motivations were around
this?

Thanks, and again, sorry for being late to the party! :)

-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [TripleO] Tuskar CLI after architecture changes

[Jay Pipes]

On Wed, 2013-12-11 at 09:32 -0500, Jay Dobies wrote:
> On 12/11/2013 07:33 AM, Jiří Stránský wrote:
> > 3) Keep tuskar-api and python-tuskarclient thin, make another library
> > sitting between Tuskar UI and all python-***clients. This new project
> > would contain the logic of using undercloud services to provide the
> > "tuskar experience" it would expose python bindings for Tuskar UI and
> > contain a CLI. (Think of it like traditional python-*client but instead
> > of consuming a REST API, it would consume other python-*clients. I
> > wonder if this is overengineering. We might end up with too many
> > projects doing too few things? :) )
> 
> This is the sort of thing I was describing with the facade image above. 
> Rather than beefing up python-tuskarclient, I'd rather we have a 
> specific logic layer that isn't the CLI nor is it the bindings, but is 
> specifically for the purposes of coordination across multiple APIs.
> 
> That said, I'm -1 to my own facade diagram. I think that should live 
> service-side in the API.

And this is what Dean said should be implemented as WSGI middleware,
which I agree is likely the most flexible and architecturally-sound way
of doing this.

Best,
-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Tempest][Neutron] Network client and API tests refactoring.

[Eugene Nikanorov]

Hi Jay,

Sure, that is understood. In fact such refactoring could be a big change so
I'd split it to two or more patches.
Hope this will not overlap with ongoing neutron API tests development.

Thanks,
Eugene.



On Sat, Dec 14, 2013 at 7:01 PM, Jay Pipes  wrote:

> On Sat, 2013-12-14 at 15:26 +0400, Eugene Nikanorov wrote:
> > Hi tempest folks,
> >
> > While adding new API tests for Neutron I found out that network client
> > used in tempest could be improved in at least two ways:
> > 1) code duplication between xml and json versions could be removed
> > 2) client API could be made more flexible and similar to real neutron
> > client API.
> > E.g. instead of accepting fixed subset of properties of the resource,
> > methods could accept full dict with resource description.
> >
> > Implementing this two points will also require some changes in the
> > tests: for example, instead of calling
> >  network_client.create_network(name)
> > test should call:
> >  network_data = {'name': name, 'property2': property, ...}
> >  network_client.create_network(network_data)
> >
> > Those both items require quite a bit of work and I'm working on such
> > improvement right now.
> > My personal motivation for this is that I hate to add new client
> > methods by copy-pasting existing and
> > hardcoding stuff in them (and btw, I need to do it for both xml and
> > json versions)
> >
> > So before committing more than two hours on this, I wanted to ask if
> > such kind of change is desirable?
> > If so, I'm going to present a sequence of patches that will implement
> > this plan.
>
> Hi Eugene!
>
> It's best to keep refactoring patches separate from any patches that add
> functionality or new tests to Tempest. I think the above improvements
> would be a welcome change, but it's easier for reviewers if that
> refactoring patch is all in one patch and not mixed in with new tests or
> added functionality.
>
> All the best,
> -jay
>
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Tempest][Neutron] Network client and API tests refactoring.

[Jay Pipes]

On Sat, 2013-12-14 at 15:26 +0400, Eugene Nikanorov wrote:
> Hi tempest folks,
> 
> While adding new API tests for Neutron I found out that network client
> used in tempest could be improved in at least two ways:
> 1) code duplication between xml and json versions could be removed
> 2) client API could be made more flexible and similar to real neutron
> client API. 
> E.g. instead of accepting fixed subset of properties of the resource,
> methods could accept full dict with resource description.
> 
> Implementing this two points will also require some changes in the
> tests: for example, instead of calling
>  network_client.create_network(name)
> test should call:
>  network_data = {'name': name, 'property2': property, ...}
>  network_client.create_network(network_data)
> 
> Those both items require quite a bit of work and I'm working on such
> improvement right now.
> My personal motivation for this is that I hate to add new client
> methods by copy-pasting existing and
> hardcoding stuff in them (and btw, I need to do it for both xml and
> json versions)
> 
> So before committing more than two hours on this, I wanted to ask if
> such kind of change is desirable?
> If so, I'm going to present a sequence of patches that will implement
> this plan.

Hi Eugene!

It's best to keep refactoring patches separate from any patches that add
functionality or new tests to Tempest. I think the above improvements
would be a welcome change, but it's easier for reviewers if that
refactoring patch is all in one patch and not mixed in with new tests or
added functionality.

All the best,
-jay


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][TripleO] Nested resources

[Sylvain Bauza]

2013/12/9 Clint Byrum 

> Excerpts from Fox, Kevin M's message of 2013-12-09 09:34:06 -0800:
> > I'm thinking more generic:
> >
> > The cloud provider will provide one or more "suballocating" images. The
> one Triple O uses to take a bare metal node and make vm's available would
> be the obvious one to make available initially. I think that one should not
> have a security concern since it is already being used in that way safely.
>
> I like where you're going with this, in that the cloud should eventually
> become "self aware" enough to be able to privision the baremetal resources
> it has and spin nova up on them. I do think that is quite far out. Right
> now, we have two nova's.. an undercloud nova which owns all the baremetal,
> and an overcloud nova which owns all the vms. This is definitely nested,
> but there is a hard line between the two.
>
> For many people, that hard line is a feature. For others, it is a bug.  :)
>
>
Could we imagine that an end-user would like to provision one undercloud
host plus a certain number of overcloud nodes so that the Scheduler for
undercloud Nova would deny other hosts but the ones provisioned ?
As a contrary,  Scheduler for other undercloud Nova's need to deny the
provisioning of the nodes hosted by another tenant than the requester...
I played with TripleO a few months ago (August/September, before the merge
with Tuskar) so that's a bit unclear for me, but I'm just saying we could
potentially achieve this by using Climate which does deploy its own
Scheduler Filter for making sure the proper hosts are booted.

-Sylvain
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova][TripleO] Nested resources

[Sylvain Bauza]

Hi Kevin,


2013/12/2 Fox, Kevin M 

> Hi all,
>
> I just want to run a crazy idea up the flag pole. TripleO has the concept
> of an under and over cloud. In starting to experiment with Docker, I see a
> pattern start to emerge.
>
>  * As a User, I may want to allocate a BareMetal node so that it is
> entirely mine. I may want to run multiple VM's on it to reduce my own cost.
> Now I have to manage the BareMetal nodes myself or nest OpenStack into them.
>  * As a User, I may want to allocate a VM. I then want to run multiple
> Docker containers on it to use it more efficiently. Now I have to manage
> the VM's myself or nest OpenStack into them.
>  * As a User, I may want to allocate a BareMetal node so that it is
> entirely mine. I then want to run multiple Docker containers on it to use
> it more efficiently. Now I have to manage the BareMetal nodes myself or
> nest OpenStack into them.
>
> I think this can then be generalized to:
> As a User, I would like to ask for resources of one type (One AZ?), and be
> able to delegate resources back to Nova so that I can use Nova to subdivide
> and give me access to my resources as a different type. (As a different AZ?)
>
>

Have you ever heard about Climate [1], our Stackforge project for
reservations as a service in Openstack ? I'm just saying that because your
usecases seem pretty identical as the ones we defined for starting the
project...

Climate is aiming to reserve both virtual (instances, stacks, etc.) and
physical (compute hosts) resources [2]. There is no overhead with
hypervisors, that's only matter of saying 'this node is for me, I'll be
booting anything I want on it", provided the hypervisor would be KVM,
Xen-API, or Baremetal (of course, with the Baremetal driver, you would then
only get a lock)


That said, I'm just talking about Climate for making sure we don't generate
silos and duplicate stuff. I still have to think about how we can plug
TripleO resources as native resources for Climate, so in that case TripleO
would just have to call Climate for provisioning the resources.

-Sylvain

[1] : https://launchpad.net/climate
[2] :
https://wiki.openstack.org/wiki/Blueprint-nova-planned-resource-reservation-api


I think this could potentially cover some of the TripleO stuff without
> needing an over/under cloud. For that use case, all the BareMetal nodes
> could be added to Nova as such, allocated by the "services" tenant as
> running a nested VM image type resource stack, and then made available to
> all tenants. Sys admins then could dynamically shift resources from VM
> providing nodes to BareMetal Nodes and back as needed.
>
> This allows a user to allocate some raw resources as a group, then
> schedule higher level services to run only in that group, all with the
> existing api.
>
> Just how crazy an idea is this?
>
> Thanks,
> Kevin
>
> ___
> OpenStack-dev mailing list
> OpenStack-dev@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

Re: [openstack-dev] [Nova] Support for Pecan in Nova

[Christopher Yeoh]

On Sat, Dec 14, 2013 at 8:48 AM, Doug Hellmann
wrote:

> That covers routes. What about the properties of the inputs and outputs?
>
>
I think the best way for me to describe it is that as the V3 API core and
all the extensions
are written, both the routes and input and output parameters are from a
client's perspective fixed at application
startup time. Its not an inherent restriction of the framework (an
extension could for
example dynamically load another extension at runtime if it really wanted
to), but we just don't do that.

Note that values of parameters returned can be changed by an extension
though. For example os-hide-server-addresses
can based on a runtime policy check and the vm_state of the server, filter
whether the values in the
addresses field are filtered out or not when returning information about a
server. This isn't a new thing in the
V3 API though, it already existed in the V2 API.

Chris


>
> On Fri, Dec 13, 2013 at 4:43 PM, Ryan Petrello <
> ryan.petre...@dreamhost.com> wrote:
>
>> Unless there’s some other trickiness going on that I’m unaware of, the
>> routes for the WSGI app are defined at application startup time (by methods
>> called in the WSGI app’s __init__).
>>
>> ---
>> Ryan Petrello
>> Senior Developer, DreamHost
>> ryan.petre...@dreamhost.com
>>
>> On Dec 13, 2013, at 12:56 PM, Doug Hellmann 
>> wrote:
>>
>> >
>> >
>> >
>> > On Thu, Dec 12, 2013 at 9:22 PM, Christopher Yeoh 
>> wrote:
>> > On Fri, Dec 13, 2013 at 4:12 AM, Jay Pipes  wrote:
>> > On 12/11/2013 11:47 PM, Mike Perez wrote:
>> > On 10:06 Thu 12 Dec , Christopher Yeoh wrote:
>> > On Thu, Dec 12, 2013 at 8:59 AM, Doug Hellmann
>> > > > >wrote:
>> >
>> >
>> >
>> >
>> > On Wed, Dec 11, 2013 at 3:41 PM, Ryan Petrello <
>> > ryan.petre...@dreamhost.com
>> > >
>> > wrote:
>> >
>> > Hello,
>> >
>> > I’ve spent the past week experimenting with using Pecan for
>> > Nova’s
>> > API
>> > and have opened an experimental review:
>> >
>> >
>> > https://review.openstack.org/#/c/61303/6
>> >
>> > …which implements the `versions` v3 endpoint using pecan (and
>> > paves the
>> > way for other extensions to use pecan).  This is a *potential*
>> >
>> > approach
>> > I've considered for gradually moving the V3 API, but I’m open
>> > to other suggestions (and feedback on this approach).  I’ve
>> > also got a few open questions/general observations:
>> >
>> > 1.  It looks like the Nova v3 API is composed *entirely* of
>> > extensions (including “core” API calls), and that extensions
>> > and their routes are discoverable and extensible via installed
>> > software that registers
>> > itself
>> > via stevedore.  This seems to lead to an API that’s composed of
>> >
>> > installed
>> > software, which in my opinion, makes it fairly hard to map out
>> > the
>> > API (as
>> > opposed to how routes are manually defined in other WSGI
>> > frameworks).  I
>> > assume at this time, this design decision has already been
>> > solidified for
>> > v3?
>> >
>> >
>> > Yeah, I brought this up at the summit. I am still having some
>> > trouble understanding how we are going to express a stable core
>> > API for compatibility testing if the behavior of the API can be
>> > varied so significantly by deployment decisions. Will we just
>> > list each
>> > "required"
>> > extension, and forbid any extras for a compliant cloud?
>> >
>> >
>> > Maybe the issue is caused by me misunderstanding the term
>> > "extension," which (to me) implies an optional component but is
>> > perhaps reflecting a technical implementation detail instead?
>> >
>> >
>> > Yes and no :-) As Ryan mentions, all API code is a plugin in the V3
>> > API. However, some must be loaded or the V3 API refuses to start
>> > up. In nova/api/openstack/__init__.py we have
>> > API_V3_CORE_EXTENSIONS which hard codes which extensions must be
>> > loaded and there is no config option to override this (blacklisting
>> > a core plugin will result in the V3 API not starting up).
>> >
>> > So for compatibility testing I think what will probably happen is
>> > that we'll be defining a minimum set (API_V3_CORE_EXTENSIONS) that
>> > must be implemented and clients can rely on that always being
>> > present
>> > on a compliant cloud. But clients can also then query through
>> > /extensions what other functionality (which is backwards compatible
>> > with respect to core) may also be present on that specific cloud.
>> >
>> > This really seems similar to the idea of having a router class, some
>> > controllers and you map them. From my observation at the summit,
>> > calling everything an extension creates confusion. An extension
>> > "extends" something. For example, Chrome has extensions, and they
>> > extend the idea of the core features of a browser. If you want to do
>> > more than back/forward, go to an address, stop, etc, that's an
>> > extension. If you want it to play an audio clip "stop, hammer time"
>> > after clicking the 

[openstack-dev] [Tempest][Neutron] Network client and API tests refactoring.

[Eugene Nikanorov]

Hi tempest folks,

While adding new API tests for Neutron I found out that network client used
in tempest could be improved in at least two ways:
1) code duplication between xml and json versions could be removed
2) client API could be made more flexible and similar to real neutron
client API.
E.g. instead of accepting fixed subset of properties of the resource,
methods could accept full dict with resource description.

Implementing this two points will also require some changes in the tests:
for example, instead of calling
 network_client.create_network(name)
test should call:
 network_data = {'name': name, 'property2': property, ...}
 network_client.create_network(network_data)

Those both items require quite a bit of work and I'm working on such
improvement right now.
My personal motivation for this is that I hate to add new client methods by
copy-pasting existing and
hardcoding stuff in them (and btw, I need to do it for both xml and json
versions)

So before committing more than two hours on this, I wanted to ask if such
kind of change is desirable?
If so, I'm going to present a sequence of patches that will implement this
plan.

Thanks,
Eugene.
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev