[Yahoo-eng-team] [Bug 1705755] Re: [RFE] Plugin support for API resource attribute extensions
On November 30th we decided to continue support of neutron python client so we can support API resource attribute extension ** Changed in: neutron Status: Triaged => Won't Fix -- You received this bug notification because you are a member of Yahoo! Engineering Team, which is subscribed to neutron. https://bugs.launchpad.net/bugs/1705755 Title: [RFE] Plugin support for API resource attribute extensions Status in neutron: Won't Fix Status in python-openstackclient: New Bug description: NOTE: On 11/28/2017 I removed the old description from this bug and replaced with a simpler more consolidated version based on an example. It seems the old description's verbose nature may cause elongated understanding of the issue. PROBLEM: Neutron plugin API extensions that add attributes to existing API resources have no proper way to extend the python-openstackclient (OSC). EXAMPLE: a) We have a plugin that extends an existing API resource [1]; in this example we add a 'logging' attribute to the existing 'security_groups' API resource. b) Now we want to add CLI support for the 'logging' attribute. But there's not clean way with OSC today: ..1) To get the new 'logging' attribute to even show up (parse from API and display on output), we need to update the sdk [2]. Traditionally this has not been something done for "non-stadium projects". ..2) The python-openstackclient (OSC) already defines the logic for 'security_groups'; we want to be able to extend it to handle our 'logging' attribute, but there's no current way [3] to extend an OSC networking command. ..3) As a result we need to duplicate the OSC code [3] for security_groups to add our custom logic [4]. Now we have to try and keep [4] in sync with [3] which is less than optimal. Some existing plugins that have resource attribute extensions today: [1][5][6][7][8] MITIGATION: Today we can still use the python-neutronclient to interface with these types of extensions. This support is provided by the "looseness" of the neutronclient and it's support for arbitrary kwargs. For example with neutron client you can do something like: neutron security-group-create --logging=blah and the client will pass logging=blah to the API allowing the extension to function properly. In addition the neutronclient will display the value(s) of 'logging' in the CLI output without and code changes to the python-neutronclient. That said the current mitigation is to use the neutronclient. CONSIDERATIONS: - If the longer term goal is to move away from API extensions in neutron (like nova did [9]), then maybe we just support neutronclient until that time. - While the current support we have for these extensions in neutronclient maybe a "side effect" (e.g. loose kwarg passing and CLI input/output), I know we have existing deployments relying on this support. So removing such support would leave them with no options. - There maybe some middle ground to get support in OSC in the shorter-term, but in general it's my understanding that the SDK/OSC has few reviewers and maybe undergoing a "restructure" [10]. [1] https://github.com/openstack/vmware-nsx/blob/master/vmware_nsx/extensions/securitygrouplogging.py [2] https://github.com/openstack/python-openstacksdk/blob/master/openstack/network/v2/security_group.py [3] https://github.com/openstack/python-openstackclient/blob/master/openstackclient/network/v2/security_group.py#L117 [4] https://review.openstack.org/#/c/393322/4/vmware_nsx/osc/v2/security_group.py [5] http://git.openstack.org/cgit/openstack/networking-h3c/tree/networking_h3c/extensions/portextensions.py [6] http://git.openstack.org/cgit/openstack/astara-neutron/tree/astara_neutron/extensions/routerstatus.py [7] http://git.openstack.org/cgit/openstack/networking-cisco/tree/networking_cisco/plugins/cisco/extensions/routertypeawarescheduler.py [8] http://git.openstack.org/cgit/openstack/group-based-policy/tree/gbpservice/neutron/extensions/implicit_subnetpools.py [9] https://blueprints.launchpad.net/nova/+spec/api-no-more-extensions-pike [10] http://lists.openstack.org/pipermail/openstack-dev/2017-August/120673.html To manage notifications about this bug go to: https://bugs.launchpad.net/neutron/+bug/1705755/+subscriptions -- Mailing list: https://launchpad.net/~yahoo-eng-team Post to : yahoo-eng-team@lists.launchpad.net Unsubscribe : https://launchpad.net/~yahoo-eng-team More help : https://help.launchpad.net/ListHelp
[Yahoo-eng-team] [Bug 1705755] Re: [RFE] Plugin support for API resource attribute extensions
** Changed in: python-openstacksdk
Status: New => Won't Fix
** No longer affects: python-openstacksdk
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1705755
Title:
[RFE] Plugin support for API resource attribute extensions
Status in neutron:
Triaged
Status in python-openstackclient:
New
Bug description:
FORWARD
---
The purpose of this bug is to facilitate a discussion. While I'm willing to
do all the work to implement the feature request herein, I'd first like some
agreement that:
- We want to solve this problem.
- Core(s) on the respective projects (python-openstackclient and
python-openstacksdk) are willing to provide some guidance on the best
high-level approach.
BACKGROUND
--
Some OpenStack REST APIs such as neutron and nova support API extensions.
While the implementation details for API extensions may differ from project
to project, the basic extension support across all projects includes the
ability for pluggable extensions to augment the REST API in the following ways:
(a) Introducing new RESTful resources (APIs). These may be new top-level
resources (e.g. /v1/theapi/{extension_resource}) or sub-resources (e.g.
/v1/theapi/existing_resource/{extension_subresource}).
(b) Adding new attributes (fields) to existing RESTful resources. For example
neutron's net-mtu extension adds a 'mtu' attribute to networks [1] and nova's
extended status adds attributes to servers [2].
While some API extensions may exist in-tree (e.g. right in neutron or
nova), they can also live in out-of-tree projects that implement
plugins/drivers/etc..
Hopefully we can all agree that a CLI should encompass the means to
support both #a and #b above in some form or another in order to
obtain the pluggability our consumers expect with OpenStack.
PROBLEM DESCRIPTION
For sake of argument lets consider #a and #b as use cases from an OpenStack
CLI (python-openstackclient and python-openstacksdk) perspective. Albeit we are
talking from a contributor perspective here; but the ability to contribute
drive features users can consume.
(a) This case should already be supported via the existing OSC plugin
model. For example [3].
(b) This case is covered for stadium projects; they can add logic
right into python-openstackclient[sdk] as needed. However for non-
stadium projects, this case is not covered; today there's no easy way
to extend existing OSC resources/actions/options/etc. in a reusable
manner. For example, suppose a non-stadium neutron (plugin) project
implements the network API, extends the network resource with
additional attribute(s) and wants to provide CLI support for these
attributes. Ideally they should be able to reuse the existing neutron
network OSC logic in a pluggable manner such that they need not
reimplement the OSC "actions" and associated logic.
From a technical perspective, case #b roughly requires the ability for
non-stadium projects (with such attribute extensions) to:
(1) Add attributes to python-openstacksdk resource(s). This allows the
extension attributes to be "collected" from an API response body.
(2) Extend existing python-openstackclient commands by;
a) Adding parser options if needed to support the attribute extension(s).
b) Adding custom "take action" logic enabling the extension to process
attribute extension options if needed.
c) Displaying the attribute extension(s).
In neutron alone, the following out-of-tree projects implement extensions
falling under case #b:
- vmware-nsx
- gbpservice
- quark
- networking-cisco
- networking-bigswitch
- networking-h3c
This support was provided in an ad-hoc fashion with the classic python
clients (e.g. python-neutronclient) via it's ability to
pass/handle/display arbitrary key/values. For example an API extension
that adds a 'meta' boolean attribute to networks can be handled with
the neutron client without any changes. The sample output snippets
below show GET and POST (notice the 'meta' attribute that's added by
the sample extension).
-->
stack@bvm2:~/devstack$ neutron --debug net-show my-meta-net
...
RESP BODY:
{"network":{"provider:physical_network":null,"ipv6_address_scope":null,"meta":true,
...
+---+--+
| Field | Value|
+---+--+
| admin_state_up| True |
| availability_zone_hints | |
| availability_zones| nova |
| created_at| 2017-07-21T17:31:23Z |
| description |
[Yahoo-eng-team] [Bug 1705755] Re: [RFE] Plugin support for API resource attribute extensions
** Also affects: neutron
Importance: Undecided
Status: New
--
You received this bug notification because you are a member of Yahoo!
Engineering Team, which is subscribed to neutron.
https://bugs.launchpad.net/bugs/1705755
Title:
[RFE] Plugin support for API resource attribute extensions
Status in neutron:
New
Status in python-openstackclient:
New
Status in OpenStack SDK:
New
Bug description:
FORWARD
---
The purpose of this bug is to facilitate a discussion. While I'm willing to
do all the work to implement the feature request herein, I'd first like some
agreement that:
- We want to solve this problem.
- Core(s) on the respective projects (python-openstackclient and
python-openstacksdk) are willing to provide some guidance on the best
high-level approach.
BACKGROUND
--
Some OpenStack REST APIs such as neutron and nova support API extensions.
While the implementation details for API extensions may differ from project
to project, the basic extension support across all projects includes the
ability for pluggable extensions to augment the REST API in the following ways:
(a) Introducing new RESTful resources (APIs). These may be new top-level
resources (e.g. /v1/theapi/{extension_resource}) or sub-resources (e.g.
/v1/theapi/existing_resource/{extension_subresource}).
(b) Adding new attributes (fields) to existing RESTful resources. For example
neutron's net-mtu extension adds a 'mtu' attribute to networks [1] and nova's
extended status adds attributes to servers [2].
While some API extensions may exist in-tree (e.g. right in neutron or
nova), they can also live in out-of-tree projects that implement
plugins/drivers/etc..
Hopefully we can all agree that a CLI should encompass the means to
support both #a and #b above in some form or another in order to
obtain the pluggability our consumers expect with OpenStack.
PROBLEM DESCRIPTION
For sake of argument lets consider #a and #b as use cases from an OpenStack
CLI (python-openstackclient and python-openstacksdk) perspective. Albeit we are
talking from a contributor perspective here; but the ability to contribute
drive features users can consume.
(a) This case should already be supported via the existing OSC plugin
model. For example [3].
(b) This case is covered for stadium projects; they can add logic
right into python-openstackclient[sdk] as needed. However for non-
stadium projects, this case is not covered; today there's no easy way
to extend existing OSC resources/actions/options/etc. in a reusable
manner. For example, suppose a non-stadium neutron (plugin) project
implements the network API, extends the network resource with
additional attribute(s) and wants to provide CLI support for these
attributes. Ideally they should be able to reuse the existing neutron
network OSC logic in a pluggable manner such that they need not
reimplement the OSC "actions" and associated logic.
From a technical perspective, case #b roughly requires the ability for
non-stadium projects (with such attribute extensions) to:
(1) Add attributes to python-openstacksdk resource(s). This allows the
extension attributes to be "collected" from an API response body.
(2) Extend existing python-openstackclient commands by;
a) Adding parser options if needed to support the attribute extension(s).
b) Adding custom "take action" logic enabling the extension to process
attribute extension options if needed.
c) Displaying the attribute extension(s).
In neutron alone, the following out-of-tree projects implement extensions
falling under case #b:
- vmware-nsx
- gbpservice
- quark
- networking-cisco
- networking-bigswitch
- networking-h3c
This support was provided in an ad-hoc fashion with the classic python
clients (e.g. python-neutronclient) via it's ability to
pass/handle/display arbitrary key/values. For example an API extension
that adds a 'meta' boolean attribute to networks can be handled with
the neutron client without any changes. The sample output snippets
below show GET and POST (notice the 'meta' attribute that's added by
the sample extension).
-->
stack@bvm2:~/devstack$ neutron --debug net-show my-meta-net
...
RESP BODY:
{"network":{"provider:physical_network":null,"ipv6_address_scope":null,"meta":true,
...
+---+--+
| Field | Value|
+---+--+
| admin_state_up| True |
| availability_zone_hints | |
| availability_zones| nova |
| created_at| 2017-07-21T17:31:23Z |
| description |
