Re: [openstack-dev] [all][OSC] OpenStack Interpreter: A useful tool python interpreter tool for the OpenStack client libraries.
On 04/05/17 03:32, Dean Troyer wrote:
> On Tue, May 2, 2017 at 7:32 PM, Adrian Turjak wrote:
>> Shade in this context is both really easy, and harder since I can't just
>> give it the same session so it can reuse the same token. I've tried
>> seeing if I can pilfer the OpenStackCloudConfig from OSC but passing
>> that to shade seemed to break. If you run that interpreter command you
>> can explore the OSC objects too. "self.app.cloud_config" or
>> "self.app.cloud" appears to be close to what we want, but I can't get it
>> to play nice with shade as it appears to be a OSC extension of the
>> os-client-config class.
> With last week's release of os-client-config 1.27.0 and osc-lib 1.5.1
> and the addition of these to global-requirements minimums for today's
> OSC 3.10.0 release we are in a place to move most of the special-case
> code back in to osc-lib and os-client-config. Much of this is due to
> backward-compatibility issues that OSC needed to handle.
>
>> If the interpreter is started from envvars you can do "cloud =
>> openstack_cloud()" and shade does the right thing. If it was started
>> with --os-cloud then you can also do "cloud =
>> openstack_cloud(cloud=self.app.cloud.config.get('cloud'))". The latter
>> also works with envvars as "self.app.cloud.config.get('cloud')" returns
>> an empty string so shade looks at envvars. I can easily make a function
>> that just returns shade with a new token, just kind of sucks that there
>> is no way to pass it a valid session/token for it to reuse. Would be
>> fantastic if you can take a gander at that as I don't know that much
>> about Shade. I'd prefer to have this thing use a single shared session
>> or token as much as possible.
> Both OSC and Shade use ksa Sessions, that bit should be directly
> sharable and includes the token handling. OSC's auth stuff changed a
> bit in the 3.10.0 release in the first step of simplifying it and
> actually being able to use os-client-config as intended. There are a
> bunch of timing issues regarding _when_ auth plugins get loaded that
> differ between Shade and OSC's usage. once we complete resolving that
> the level of reuse should be able to go up substantially.
>
> dt
With the newest release, the following seems to work without errors:
In [1]: import shade
In [2]: cloud = shade.openstack_cloud(config=self.app.cloud_config)
In [3]: cloud.list_servers()
I'm not sure it reuses the same session, but *shrug* at least it is
easily piping through the same auth and profile values you used to setup
the openstackclient to shade. I'll add a basic factory for shade (not
that it really needs much of one), and then one for OpenStackSDK soon
and publish a new version of the interpreter command.
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][OSC] OpenStack Interpreter: A useful tool python interpreter tool for the OpenStack client libraries.
On 04/05/17 03:20, Dean Troyer wrote: > On Mon, May 1, 2017 at 11:11 PM, Adrian Turjak > wrote: >> As part of my dev work I recently put together a cool little tool which >> lets me have much easier access to the various OpenStack python clients >> in the scope of a python interpreter session. The first version was a >> little rough and without os-client-config support. The current version >> is now a plugin for the openstackclient and introduces a command that >> simply authenticates you, sets up the environment and helper tools, and >> then drops you into an ipython interactive session. The helper stuff is >> fairly simple, but combined with the features of ipython it really lets >> you start playing with the tools quickly, and by piggybacking onto >> openstackclient I get access to a lot of the niceties and inbuilt auth >> mechanisms. >> >> It is useful for learning, testing, or development against the various >> openstack client libraries, and even as an ops tool to quickly run some >> basic actions without having to resort to weird or silly bash command >> combinations. > This is at the other end of the spectrum of client usage from the CLI > in that it takes a very developer-like view of the APIs. OSC has its > interactive mode (using cmd2) that is just a command loop without even > the ability to change global settings. My hunch is that something in > between would be really useful, but I am not interested in creating > yet another DSL for this to include flow control and branching. > > You mentioned Shade, which falls somewhere in between as it implements > a lot of logic to hide differences in clouds and ease dealing with > some of our API warts. What would you think about producing a Python > mirror of the CLI interface with resource names, option names, > everything, but in Python. I have not thought all the way through > this yet, but think we could take advantage of using the cliff command > classes and be able to re-use all of the bits already written. > > This also magnifies the things we need to add to (or factor out of) > OSC to make it truly useful, such as manipulation of the global > options, maintaining multiple client contexts, etc. > > Thanks for sharing! > dt A new DSL is probably a terrible idea for obvious reasons: https://xkcd.com/927/ A good interpreter like ipython combined with a reasonably easy way to access the existing myriad of client libraries gets us most of the way. While making yet another wrapper layer to simplify everything could work, I feel that we have enough libraries already, and no one really knows them all that well, and a lot of them are seriously lacking good docs outside of the code itself. While the idea of making a interpreter specific layer for the APIs would be interesting, I feel like it would be wasted effort that could go towards promoting better consistency and quality among the various existing python tools. The project specific python libraries have a fairly similar command structure to OSC as is, but it is interesting to compare them all: OSC: openstack vs clients: .. vs SDK: .._ vs shade: ._ There are of course differences between some of the python-clients structurally but a lot of them tend to follow the same pattern. Then there is minor but annoying stuff like some list() commands return lists, while others return generators. I don't think writing yet another python library is a good idea even if specific to python interpreter interactions. I think we just need to document and educate about all the existing client options really well, and have tools like this that promote them. Including non-python libraries, because often those feel like second class citizen, and often we forget about them entirely. I had to recommend a good java library to a client recently and after testing a bunch I found that only openstack4j was actually easy to use, and had proper identity v3 support. The most recommended javaSDK is jclouds and it was so very confusing and badly documented at a glance and from a new user perspective. My goal with this tool was mainly to promote better literacy for the existing python libraries among our team here at Catalyst and give our cloud customers a place to start prototyping interactions when building applications that talk to the APIs. Well... and build something I myself wanted to use! Less new libraries and standards unless we actually deprecate old stuff! More docs, polish for existing libraries, and better ways to use them! :P __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][OSC] OpenStack Interpreter: A useful tool python interpreter tool for the OpenStack client libraries.
On Tue, May 2, 2017 at 7:32 PM, Adrian Turjak wrote:
> Shade in this context is both really easy, and harder since I can't just
> give it the same session so it can reuse the same token. I've tried
> seeing if I can pilfer the OpenStackCloudConfig from OSC but passing
> that to shade seemed to break. If you run that interpreter command you
> can explore the OSC objects too. "self.app.cloud_config" or
> "self.app.cloud" appears to be close to what we want, but I can't get it
> to play nice with shade as it appears to be a OSC extension of the
> os-client-config class.
With last week's release of os-client-config 1.27.0 and osc-lib 1.5.1
and the addition of these to global-requirements minimums for today's
OSC 3.10.0 release we are in a place to move most of the special-case
code back in to osc-lib and os-client-config. Much of this is due to
backward-compatibility issues that OSC needed to handle.
> If the interpreter is started from envvars you can do "cloud =
> openstack_cloud()" and shade does the right thing. If it was started
> with --os-cloud then you can also do "cloud =
> openstack_cloud(cloud=self.app.cloud.config.get('cloud'))". The latter
> also works with envvars as "self.app.cloud.config.get('cloud')" returns
> an empty string so shade looks at envvars. I can easily make a function
> that just returns shade with a new token, just kind of sucks that there
> is no way to pass it a valid session/token for it to reuse. Would be
> fantastic if you can take a gander at that as I don't know that much
> about Shade. I'd prefer to have this thing use a single shared session
> or token as much as possible.
Both OSC and Shade use ksa Sessions, that bit should be directly
sharable and includes the token handling. OSC's auth stuff changed a
bit in the 3.10.0 release in the first step of simplifying it and
actually being able to use os-client-config as intended. There are a
bunch of timing issues regarding _when_ auth plugins get loaded that
differ between Shade and OSC's usage. once we complete resolving that
the level of reuse should be able to go up substantially.
dt
--
Dean Troyer
dtro...@gmail.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][OSC] OpenStack Interpreter: A useful tool python interpreter tool for the OpenStack client libraries.
On Mon, May 1, 2017 at 11:11 PM, Adrian Turjak wrote: > As part of my dev work I recently put together a cool little tool which > lets me have much easier access to the various OpenStack python clients > in the scope of a python interpreter session. The first version was a > little rough and without os-client-config support. The current version > is now a plugin for the openstackclient and introduces a command that > simply authenticates you, sets up the environment and helper tools, and > then drops you into an ipython interactive session. The helper stuff is > fairly simple, but combined with the features of ipython it really lets > you start playing with the tools quickly, and by piggybacking onto > openstackclient I get access to a lot of the niceties and inbuilt auth > mechanisms. > > It is useful for learning, testing, or development against the various > openstack client libraries, and even as an ops tool to quickly run some > basic actions without having to resort to weird or silly bash command > combinations. This is at the other end of the spectrum of client usage from the CLI in that it takes a very developer-like view of the APIs. OSC has its interactive mode (using cmd2) that is just a command loop without even the ability to change global settings. My hunch is that something in between would be really useful, but I am not interested in creating yet another DSL for this to include flow control and branching. You mentioned Shade, which falls somewhere in between as it implements a lot of logic to hide differences in clouds and ease dealing with some of our API warts. What would you think about producing a Python mirror of the CLI interface with resource names, option names, everything, but in Python. I have not thought all the way through this yet, but think we could take advantage of using the cliff command classes and be able to re-use all of the bits already written. This also magnifies the things we need to add to (or factor out of) OSC to make it truly useful, such as manipulation of the global options, maintaining multiple client contexts, etc. Thanks for sharing! dt -- Dean Troyer dtro...@gmail.com __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][OSC] OpenStack Interpreter: A useful tool python interpreter tool for the OpenStack client libraries.
On 03/05/17 01:23, Monty Taylor wrote:
> On 05/02/2017 12:11 AM, Adrian Turjak wrote:
>> Hello OpenStack folks,
>>
>> As part of my dev work I recently put together a cool little tool which
>> lets me have much easier access to the various OpenStack python clients
>> in the scope of a python interpreter session. The first version was a
>> little rough and without os-client-config support. The current version
>> is now a plugin for the openstackclient and introduces a command that
>> simply authenticates you, sets up the environment and helper tools, and
>> then drops you into an ipython interactive session. The helper stuff is
>> fairly simple, but combined with the features of ipython it really lets
>> you start playing with the tools quickly, and by piggybacking onto
>> openstackclient I get access to a lot of the niceties and inbuilt auth
>> mechanisms.
>>
>> It is useful for learning, testing, or development against the various
>> openstack client libraries, and even as an ops tool to quickly run some
>> basic actions without having to resort to weird or silly bash command
>> combinations.
>>
>> I personally use it to test out commands or libraries I'm not familiar
>> with, or if I just need to work out what the output from something is.
>> Often even doing once off admin actions that require parsing through and
>> comparing different values and resources, but isn't worth writing a
>> script for.
>>
>> My goal was to make something easy to use, and help almost anyone pick
>> up and start using the various python clients without needing to dig
>> through too much docs.
>>
>> https://pypi.python.org/pypi/openstack-interpreter
>
> Nice work! I'd love to help you connect the shade bits - as well as
> raw REST clients. (I frequently just open a python REPL and type:
>
> import shade
> shade.simple_logging(http_debug=True)
> c = shade.openstack_cloud(cloud='vexxhost')
>
> and then do a bunch of stuff with c. In fact, that's my most common
> form of interaction with OpenStack these days. :)
Much the same here, although swap out shade for
python-client. I've been doing a lot in an ipython
interpreter lately, so anything to get me up and running quickly helps,
hence this silly little project.
>
> So since you did the nice work to connect it to OSC, I'd love to help
> you connect a few more options in there. (shade's constructor takes an
> os-client-config OpenStackCloudConfig object and uses the session from
> it, so if you've got occ support already, plumbing in shade with
> shared session should be easy. If it's not, I'm happy to help sort
> that out.)
>
> Monty
I've got support for the various python-client libraries,
and adding new ones is easy, even if they have slightly different
constructors (*shakes fist at SwiftClient*). I tend to use these the
most, and prefer them over the openstackSDK so they were the first to be
implemented in this.
OpenstackSDK is also reasonably easy, but I want to create a nice little
factory for configuring it because at first glance the Profile objects
it uses are a little confusing, and it would be nice to abstract all of
that away into a factory function that just spits out a connection
object based on the params you give it but reusing the existing
keystoneauth session.
I should point out though that by os-client-config support I mainly
mean, I'm letting OSC handle that, and then just pilfering the
keystoneauth session it builds. :P
Shade in this context is both really easy, and harder since I can't just
give it the same session so it can reuse the same token. I've tried
seeing if I can pilfer the OpenStackCloudConfig from OSC but passing
that to shade seemed to break. If you run that interpreter command you
can explore the OSC objects too. "self.app.cloud_config" or
"self.app.cloud" appears to be close to what we want, but I can't get it
to play nice with shade as it appears to be a OSC extension of the
os-client-config class.
If the interpreter is started from envvars you can do "cloud =
openstack_cloud()" and shade does the right thing. If it was started
with --os-cloud then you can also do "cloud =
openstack_cloud(cloud=self.app.cloud.config.get('cloud'))". The latter
also works with envvars as "self.app.cloud.config.get('cloud')" returns
an empty string so shade looks at envvars. I can easily make a function
that just returns shade with a new token, just kind of sucks that there
is no way to pass it a valid session/token for it to reuse. Would be
fantastic if you can take a gander at that as I don't know that much
about Shade. I'd prefer to have this thing use a single shared session
or token as much as possible.
Cheers,
Adrian
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][OSC] OpenStack Interpreter: A useful tool python interpreter tool for the OpenStack client libraries.
On 05/02/2017 12:11 AM, Adrian Turjak wrote: Hello OpenStack folks, As part of my dev work I recently put together a cool little tool which lets me have much easier access to the various OpenStack python clients in the scope of a python interpreter session. The first version was a little rough and without os-client-config support. The current version is now a plugin for the openstackclient and introduces a command that simply authenticates you, sets up the environment and helper tools, and then drops you into an ipython interactive session. The helper stuff is fairly simple, but combined with the features of ipython it really lets you start playing with the tools quickly, and by piggybacking onto openstackclient I get access to a lot of the niceties and inbuilt auth mechanisms. It is useful for learning, testing, or development against the various openstack client libraries, and even as an ops tool to quickly run some basic actions without having to resort to weird or silly bash command combinations. I personally use it to test out commands or libraries I'm not familiar with, or if I just need to work out what the output from something is. Often even doing once off admin actions that require parsing through and comparing different values and resources, but isn't worth writing a script for. My goal was to make something easy to use, and help almost anyone pick up and start using the various python clients without needing to dig through too much docs. https://pypi.python.org/pypi/openstack-interpreter Nice work! I'd love to help you connect the shade bits - as well as raw REST clients. (I frequently just open a python REPL and type: import shade shade.simple_logging(http_debug=True) c = shade.openstack_cloud(cloud='vexxhost') and then do a bunch of stuff with c. In fact, that's my most common form of interaction with OpenStack these days. :) So since you did the nice work to connect it to OSC, I'd love to help you connect a few more options in there. (shade's constructor takes an os-client-config OpenStackCloudConfig object and uses the session from it, so if you've got occ support already, plumbing in shade with shared session should be easy. If it's not, I'm happy to help sort that out.) Monty __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [all][OSC] OpenStack Interpreter: A useful tool python interpreter tool for the OpenStack client libraries.
Cool, thanks for sharing Adrian. On Tue, May 2, 2017 at 12:11 AM, Adrian Turjak wrote: > Hello OpenStack folks, > > As part of my dev work I recently put together a cool little tool which > lets me have much easier access to the various OpenStack python clients > in the scope of a python interpreter session. The first version was a > little rough and without os-client-config support. The current version > is now a plugin for the openstackclient and introduces a command that > simply authenticates you, sets up the environment and helper tools, and > then drops you into an ipython interactive session. The helper stuff is > fairly simple, but combined with the features of ipython it really lets > you start playing with the tools quickly, and by piggybacking onto > openstackclient I get access to a lot of the niceties and inbuilt auth > mechanisms. > > It is useful for learning, testing, or development against the various > openstack client libraries, and even as an ops tool to quickly run some > basic actions without having to resort to weird or silly bash command > combinations. > > I personally use it to test out commands or libraries I'm not familiar > with, or if I just need to work out what the output from something is. > Often even doing once off admin actions that require parsing through and > comparing different values and resources, but isn't worth writing a > script for. > > My goal was to make something easy to use, and help almost anyone pick > up and start using the various python clients without needing to dig > through too much docs. > > https://pypi.python.org/pypi/openstack-interpreter > > Feedback is welcome! > > Cheers, > Adrian Turjak > > > __ > OpenStack Development Mailing List (not for usage questions) > Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev > __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [all][OSC] OpenStack Interpreter: A useful tool python interpreter tool for the OpenStack client libraries.
Hello OpenStack folks, As part of my dev work I recently put together a cool little tool which lets me have much easier access to the various OpenStack python clients in the scope of a python interpreter session. The first version was a little rough and without os-client-config support. The current version is now a plugin for the openstackclient and introduces a command that simply authenticates you, sets up the environment and helper tools, and then drops you into an ipython interactive session. The helper stuff is fairly simple, but combined with the features of ipython it really lets you start playing with the tools quickly, and by piggybacking onto openstackclient I get access to a lot of the niceties and inbuilt auth mechanisms. It is useful for learning, testing, or development against the various openstack client libraries, and even as an ops tool to quickly run some basic actions without having to resort to weird or silly bash command combinations. I personally use it to test out commands or libraries I'm not familiar with, or if I just need to work out what the output from something is. Often even doing once off admin actions that require parsing through and comparing different values and resources, but isn't worth writing a script for. My goal was to make something easy to use, and help almost anyone pick up and start using the various python clients without needing to dig through too much docs. https://pypi.python.org/pypi/openstack-interpreter Feedback is welcome! Cheers, Adrian Turjak __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
