Re: [openstack-dev] [qa][tempest] "kwargs" of service clients for POST/PUT methods

2015-08-01 Thread Ken'ichi Ohmichi
Hi Jordan,

Thanks for pointing this up:-)

Your point is right for current nova situation.
Nova API continues changing with small steps. The kwargs changes of tempest
will help us to avoid a lot of changes in long-term.

I also am happy if getting opinions from the others.
Thanks for your help and reviews anyway.

Thanks
Ken Ohmichi

2015年7月31日(金) 21:33 Jordan Pittier :

> Hi,
> So after I took a lot at Ken'ichi's recent proposed changes, I think this
> is the good approach. The kwargs approach has the good benefit of being
> generic so that if a consumer (say Nova) of the client class wants to add a
> new parameter to one of its API, it can do it without the need of updating
> the client class. This makes the client class more lightweight and should
> ease its adoption.
>
> But I'd like to hear what other Tempest developers thing about that ? (
> the topic has been mentioned in yesterday's QA meeting).
>
> Jordan
> On Fri, Jul 10, 2015 at 9:02 AM, Ken'ichi Ohmichi 
> wrote:
>
>> Hi Anne,
>>
>> 2015-07-09 12:22 GMT+09:00 Anne Gentle :
>> > On Wed, Jul 8, 2015 at 9:48 PM, GHANSHYAM MANN > >
>> > wrote:
>> >> On Thu, Jul 9, 2015 at 9:39 AM, Ken'ichi Ohmichi <
>> ken1ohmi...@gmail.com>
>> >> wrote:
>> >> > 2015-07-08 16:42 GMT+09:00 Ken'ichi Ohmichi :
>> >> >> 2015-07-08 14:07 GMT+09:00 GHANSHYAM MANN > >:
>> >> >>> On Wed, Jul 8, 2015 at 12:27 PM, Ken'ichi Ohmichi
>> >> >>>  wrote:
>> >> 
>> >>  By defining all parameters on each method like
>> update_quota_set(), it
>> >>  is easy to know what parameters are available from
>> caller/programer
>> >>  viewpoint.
>> >> >>>
>> >> >>> I think this can be achieved with former approach also by defining
>> all
>> >> >>> expected param in doc string properly.
>> >> >>
>> >> >> You are exactly right.
>> >> >> But current service clients contain 187 methods *only for Nova* and
>> >> >> most methods don't contain enough doc string.
>> >> >> So my previous hope which was implied was we could avoid writing doc
>> >> >> string with later approach.
>> >> >
>> >> > I am thinking it is very difficult to maintain doc string of REST
>> APIs
>> >> > in tempest-lib because APIs continue changing.
>> >> > So instead of doing it, how about putting the link of official API
>> >> > document[1] in tempest-lib and concentrating on maintaining official
>> >> > API document?
>> >> > OpenStack APIs are huge now and It doesn't seem smart to maintain
>> >> > these docs at different places.
>> >> >
>> >>
>> >> ++, this will be great. Even API links can be provided in both class
>> >> doc string as well as each method doc string (link to specific API).
>> >> This will improve API ref docs quality and maintainability.
>> >
>> >
>> > Agreed, though I also want to point you to this doc specification. We
>> hope
>> > it will help with the maintenance of the API docs.
>> >
>> > https://review.openstack.org/#/c/177934/
>> >
>> > I also want Tempest maintainers to start thinking about how a diff
>> > comparison can help with reviews of any changes to the API itself. We
>> have a
>> > proof of concept and need to do additional work to ensure it works for
>> > multiple OpenStack APIs.
>>
>> Thanks for your feedback,
>> That will be a big step for improving the API docs, I also like to
>> join for working together.
>>
>> Thanks
>> Ken Ohmichi
>>
> __
>> OpenStack Development Mailing List (not for usage questions)
>> Unsubscribe:
>> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [qa][tempest] "kwargs" of service clients for POST/PUT methods

2015-07-31 Thread Jordan Pittier
Hi,
So after I took a lot at Ken'ichi's recent proposed changes, I think this
is the good approach. The kwargs approach has the good benefit of being
generic so that if a consumer (say Nova) of the client class wants to add a
new parameter to one of its API, it can do it without the need of updating
the client class. This makes the client class more lightweight and should
ease its adoption.

But I'd like to hear what other Tempest developers thing about that ? ( the
topic has been mentioned in yesterday's QA meeting).

Jordan

On Fri, Jul 10, 2015 at 9:02 AM, Ken'ichi Ohmichi 
wrote:

> Hi Anne,
>
> 2015-07-09 12:22 GMT+09:00 Anne Gentle :
> > On Wed, Jul 8, 2015 at 9:48 PM, GHANSHYAM MANN 
> > wrote:
> >> On Thu, Jul 9, 2015 at 9:39 AM, Ken'ichi Ohmichi  >
> >> wrote:
> >> > 2015-07-08 16:42 GMT+09:00 Ken'ichi Ohmichi :
> >> >> 2015-07-08 14:07 GMT+09:00 GHANSHYAM MANN :
> >> >>> On Wed, Jul 8, 2015 at 12:27 PM, Ken'ichi Ohmichi
> >> >>>  wrote:
> >> 
> >>  By defining all parameters on each method like update_quota_set(),
> it
> >>  is easy to know what parameters are available from caller/programer
> >>  viewpoint.
> >> >>>
> >> >>> I think this can be achieved with former approach also by defining
> all
> >> >>> expected param in doc string properly.
> >> >>
> >> >> You are exactly right.
> >> >> But current service clients contain 187 methods *only for Nova* and
> >> >> most methods don't contain enough doc string.
> >> >> So my previous hope which was implied was we could avoid writing doc
> >> >> string with later approach.
> >> >
> >> > I am thinking it is very difficult to maintain doc string of REST APIs
> >> > in tempest-lib because APIs continue changing.
> >> > So instead of doing it, how about putting the link of official API
> >> > document[1] in tempest-lib and concentrating on maintaining official
> >> > API document?
> >> > OpenStack APIs are huge now and It doesn't seem smart to maintain
> >> > these docs at different places.
> >> >
> >>
> >> ++, this will be great. Even API links can be provided in both class
> >> doc string as well as each method doc string (link to specific API).
> >> This will improve API ref docs quality and maintainability.
> >
> >
> > Agreed, though I also want to point you to this doc specification. We
> hope
> > it will help with the maintenance of the API docs.
> >
> > https://review.openstack.org/#/c/177934/
> >
> > I also want Tempest maintainers to start thinking about how a diff
> > comparison can help with reviews of any changes to the API itself. We
> have a
> > proof of concept and need to do additional work to ensure it works for
> > multiple OpenStack APIs.
>
> Thanks for your feedback,
> That will be a big step for improving the API docs, I also like to
> join for working together.
>
> Thanks
> Ken Ohmichi
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [qa][tempest] "kwargs" of service clients for POST/PUT methods

2015-07-10 Thread Ken'ichi Ohmichi
Hi Anne,

2015-07-09 12:22 GMT+09:00 Anne Gentle :
> On Wed, Jul 8, 2015 at 9:48 PM, GHANSHYAM MANN 
> wrote:
>> On Thu, Jul 9, 2015 at 9:39 AM, Ken'ichi Ohmichi 
>> wrote:
>> > 2015-07-08 16:42 GMT+09:00 Ken'ichi Ohmichi :
>> >> 2015-07-08 14:07 GMT+09:00 GHANSHYAM MANN :
>> >>> On Wed, Jul 8, 2015 at 12:27 PM, Ken'ichi Ohmichi
>> >>>  wrote:
>> 
>>  By defining all parameters on each method like update_quota_set(), it
>>  is easy to know what parameters are available from caller/programer
>>  viewpoint.
>> >>>
>> >>> I think this can be achieved with former approach also by defining all
>> >>> expected param in doc string properly.
>> >>
>> >> You are exactly right.
>> >> But current service clients contain 187 methods *only for Nova* and
>> >> most methods don't contain enough doc string.
>> >> So my previous hope which was implied was we could avoid writing doc
>> >> string with later approach.
>> >
>> > I am thinking it is very difficult to maintain doc string of REST APIs
>> > in tempest-lib because APIs continue changing.
>> > So instead of doing it, how about putting the link of official API
>> > document[1] in tempest-lib and concentrating on maintaining official
>> > API document?
>> > OpenStack APIs are huge now and It doesn't seem smart to maintain
>> > these docs at different places.
>> >
>>
>> ++, this will be great. Even API links can be provided in both class
>> doc string as well as each method doc string (link to specific API).
>> This will improve API ref docs quality and maintainability.
>
>
> Agreed, though I also want to point you to this doc specification. We hope
> it will help with the maintenance of the API docs.
>
> https://review.openstack.org/#/c/177934/
>
> I also want Tempest maintainers to start thinking about how a diff
> comparison can help with reviews of any changes to the API itself. We have a
> proof of concept and need to do additional work to ensure it works for
> multiple OpenStack APIs.

Thanks for your feedback,
That will be a big step for improving the API docs, I also like to
join for working together.

Thanks
Ken Ohmichi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [qa][tempest] "kwargs" of service clients for POST/PUT methods

2015-07-08 Thread Anne Gentle
On Wed, Jul 8, 2015 at 9:48 PM, GHANSHYAM MANN 
wrote:

> On Thu, Jul 9, 2015 at 9:39 AM, Ken'ichi Ohmichi 
> wrote:
> > 2015-07-08 16:42 GMT+09:00 Ken'ichi Ohmichi :
> >> 2015-07-08 14:07 GMT+09:00 GHANSHYAM MANN :
> >>> On Wed, Jul 8, 2015 at 12:27 PM, Ken'ichi Ohmichi <
> ken1ohmi...@gmail.com> wrote:
> 
>  By defining all parameters on each method like update_quota_set(), it
>  is easy to know what parameters are available from caller/programer
>  viewpoint.
> >>>
> >>> I think this can be achieved with former approach also by defining all
> >>> expected param in doc string properly.
> >>
> >> You are exactly right.
> >> But current service clients contain 187 methods *only for Nova* and
> >> most methods don't contain enough doc string.
> >> So my previous hope which was implied was we could avoid writing doc
> >> string with later approach.
> >
> > I am thinking it is very difficult to maintain doc string of REST APIs
> > in tempest-lib because APIs continue changing.
> > So instead of doing it, how about putting the link of official API
> > document[1] in tempest-lib and concentrating on maintaining official
> > API document?
> > OpenStack APIs are huge now and It doesn't seem smart to maintain
> > these docs at different places.
> >
>
> ++, this will be great. Even API links can be provided in both class
> doc string as well as each method doc string (link to specific API).
> This will improve API ref docs quality and maintainability.
>

Agreed, though I also want to point you to this doc specification. We hope
it will help with the maintenance of the API docs.

https://review.openstack.org/#/c/177934/

I also want Tempest maintainers to start thinking about how a diff
comparison can help with reviews of any changes to the API itself. We have
a proof of concept and need to do additional work to ensure it works for
multiple OpenStack APIs.
Thanks,
Anne


>
> > Thanks
> > Ken Ohmichi
> >
> > ---
> > [1]: http://developer.openstack.org/api-ref.html
> >
> >
> __
> > OpenStack Development Mailing List (not for usage questions)
> > Unsubscribe:
> openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>
>
>
> --
> Thanks & Regards
> Ghanshyam Mann
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
>



-- 
Anne Gentle
Rackspace
Principal Engineer
www.justwriteclick.com
__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [qa][tempest] "kwargs" of service clients for POST/PUT methods

2015-07-08 Thread GHANSHYAM MANN
On Thu, Jul 9, 2015 at 9:39 AM, Ken'ichi Ohmichi  wrote:
> 2015-07-08 16:42 GMT+09:00 Ken'ichi Ohmichi :
>> 2015-07-08 14:07 GMT+09:00 GHANSHYAM MANN :
>>> On Wed, Jul 8, 2015 at 12:27 PM, Ken'ichi Ohmichi  
>>> wrote:

 By defining all parameters on each method like update_quota_set(), it
 is easy to know what parameters are available from caller/programer
 viewpoint.
>>>
>>> I think this can be achieved with former approach also by defining all
>>> expected param in doc string properly.
>>
>> You are exactly right.
>> But current service clients contain 187 methods *only for Nova* and
>> most methods don't contain enough doc string.
>> So my previous hope which was implied was we could avoid writing doc
>> string with later approach.
>
> I am thinking it is very difficult to maintain doc string of REST APIs
> in tempest-lib because APIs continue changing.
> So instead of doing it, how about putting the link of official API
> document[1] in tempest-lib and concentrating on maintaining official
> API document?
> OpenStack APIs are huge now and It doesn't seem smart to maintain
> these docs at different places.
>

++, this will be great. Even API links can be provided in both class
doc string as well as each method doc string (link to specific API).
This will improve API ref docs quality and maintainability.

> Thanks
> Ken Ohmichi
>
> ---
> [1]: http://developer.openstack.org/api-ref.html
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Thanks & Regards
Ghanshyam Mann

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [qa][tempest] "kwargs" of service clients for POST/PUT methods

2015-07-08 Thread Rochelle Grober


Ken'ichi Ohmichi wrote:

2015-07-08 16:42 GMT+09:00 Ken'ichi Ohmichi :
> 2015-07-08 14:07 GMT+09:00 GHANSHYAM MANN :
>> On Wed, Jul 8, 2015 at 12:27 PM, Ken'ichi Ohmichi  
>> wrote:
>>>
>>> By defining all parameters on each method like update_quota_set(), it
>>> is easy to know what parameters are available from caller/programer
>>> viewpoint.
>>
>> I think this can be achieved with former approach also by defining all
>> expected param in doc string properly.
>
> You are exactly right.
> But current service clients contain 187 methods *only for Nova* and
> most methods don't contain enough doc string.
> So my previous hope which was implied was we could avoid writing doc
> string with later approach.

I am thinking it is very difficult to maintain doc string of REST APIs
in tempest-lib because APIs continue changing.
So instead of doing it, how about putting the link of official API
document[1] in tempest-lib and concentrating on maintaining official
API document?
OpenStack APIs are huge now and It doesn't seem smart to maintain
these docs at different places.

[Rockyg] +100 Code documentation should be maintained closest to its 
implementation and extracted from there, not replicated all over.  Great idea!


Thanks
Ken Ohmichi

---
[1]: http://developer.openstack.org/api-ref.html

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [qa][tempest] "kwargs" of service clients for POST/PUT methods

2015-07-08 Thread Ken'ichi Ohmichi
2015-07-08 16:42 GMT+09:00 Ken'ichi Ohmichi :
> 2015-07-08 14:07 GMT+09:00 GHANSHYAM MANN :
>> On Wed, Jul 8, 2015 at 12:27 PM, Ken'ichi Ohmichi  
>> wrote:
>>>
>>> By defining all parameters on each method like update_quota_set(), it
>>> is easy to know what parameters are available from caller/programer
>>> viewpoint.
>>
>> I think this can be achieved with former approach also by defining all
>> expected param in doc string properly.
>
> You are exactly right.
> But current service clients contain 187 methods *only for Nova* and
> most methods don't contain enough doc string.
> So my previous hope which was implied was we could avoid writing doc
> string with later approach.

I am thinking it is very difficult to maintain doc string of REST APIs
in tempest-lib because APIs continue changing.
So instead of doing it, how about putting the link of official API
document[1] in tempest-lib and concentrating on maintaining official
API document?
OpenStack APIs are huge now and It doesn't seem smart to maintain
these docs at different places.

Thanks
Ken Ohmichi

---
[1]: http://developer.openstack.org/api-ref.html

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [qa][tempest] "kwargs" of service clients for POST/PUT methods

2015-07-08 Thread Ken'ichi Ohmichi
Hi Ghansyham,

Thank you for your comment.

2015-07-08 14:07 GMT+09:00 GHANSHYAM MANN :
> On Wed, Jul 8, 2015 at 12:27 PM, Ken'ichi Ohmichi  
> wrote:
>>
>> By defining all parameters on each method like update_quota_set(), it
>> is easy to know what parameters are available from caller/programer
>> viewpoint.
>
> I think this can be achieved with former approach also by defining all
> expected param in doc string properly.

You are exactly right.
But current service clients contain 187 methods *only for Nova* and
most methods don't contain enough doc string.
So my previous hope which was implied was we could avoid writing doc
string with later approach.

>> So I feel the later seems easy to use them as library methods.
>> But as the demerit, a caller cannot specify *undefined" parameters for
>> fuzzing tests.
>> Nova v2.1 API should deny a request which includes undefined
>> parameters, and Tempest will be able to test the behavior if the
>> former style.
>> So this is my concern point now.
>>
>> I tend to prefer the later(all parameters need to be defined on each
>> method) because that will be useful for callers as the above.
>> In addition, tempest-lib should be a library for implementing
>> integration tests, then fuzzing tests can be out of scope.
>> If fuzzing tests are necessary, we will be able to implement them with
>> RestClient of tempest-lib without service clients.
>
> Second approach also looks good but I prefer the first one. As you
> mentioned that fuzzy testing will not be possible with second
> approach, we should not restrict our lib functions for the same. Any
> project/test suits should be able to do API/Fuzzy testing with
> tempest-lib.
> Good example is nova v2.1 as you mentioned. Using service clients of
> lib, project (Nova) or test suits (Tempest) should be able to perform
> fuzzy testing.
>
> IMO, lib's service clients should not have any restriction on param
> list (type or number etc) to be passed from its users. Whatever
> (random param etc) is being passed that should be passed to
> corresponding projects APIs and let APIs through error or success.
>
> This can be useful when project adds new request attributes (with
> current API change guidelines like with microversion), in that case we
> do not need to change service clients. But microversion testing is not
> yet decided so this should/may not be considered a strong point?

That is a nice point.
We can avoid changing service client code with the former approach
when introducing a new attributes via a new microversion as you said.
That will be a merit when implementing tests for microversions on Tempest.

If we take the former approach, most service clients' modules contain
resource URIs and HTTP methods(PUT, POST, etc.) only as the specific
information, that seems raw methods like base RestClient class.
But as library methods, flexibility is important and the former
approach will be nice I feel now.

Thanks
Ken Ohmichi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [qa][tempest] "kwargs" of service clients for POST/PUT methods

2015-07-07 Thread GHANSHYAM MANN
On Wed, Jul 8, 2015 at 12:27 PM, Ken'ichi Ohmichi  wrote:
> Hi tempest team,
>
> We are working for refactoring service clients in tempest for
> preparing to migrate them to tempest-lib, and I found an inconsistency
> between them.
> So I'd like to ask opinions for considering which is the best as
> library methods.
>
> On some methods, a caller can specify whole body for sending a request
> to http POST/PUT methods.
> But on the others a caller can specify only part of body.
>
> For example, a caller can specify any parameters to
> update_quota_class_set() like:
>
> tempest/services/compute/json/quota_classes_client.py
>  34 def update_quota_class_set(self, quota_class_id, **kwargs):
>  35 """
>  36 Updates the quota class's limits for one or more resources.
>  37 """
>  38 post_body = json.dumps({'quota_class_set': kwargs})
>  39
>  40 resp, body = self.put('os-quota-class-sets/%s' % quota_class_id,
>  41   post_body)
>
> but update_quota_set method has each argument and a caller cannot
> specify the other parameters like:
>
> tempest/services/compute/json/quotas_client.py
>  44 def update_quota_set(self, tenant_id, user_id=None,
>  45  force=None, injected_file_content_bytes=None,
>  46  metadata_items=None, ram=None, floating_ips=None,
>  47  fixed_ips=None, key_pairs=None, instances=None,
>  48  security_group_rules=None, injected_files=None,
>  49  cores=None, injected_file_path_bytes=None,
>  50  security_groups=None):
>  51 """
>  52 Updates the tenant's quota limits for one or more resources
>  53 """
>  54 post_body = {}
>  55
>  56 if force is not None:
>  57 post_body['force'] = force
>  58
>  59 if injected_file_content_bytes is not None:
>  60 post_body['injected_file_content_bytes'] = \
>  61 injected_file_content_bytes
>  [..]
>  96 post_body = json.dumps({'quota_set': post_body})
>  97
>  98 if user_id:
>  99 resp, body = self.put('os-quota-sets/%s?user_id=%s' %
> 100   (tenant_id, user_id), post_body)
> 101 else:
> 102 resp, body = self.put('os-quota-sets/%s' % tenant_id,
> 103   post_body)
>
> By defining all parameters on each method like update_quota_set(), it
> is easy to know what parameters are available from caller/programer
> viewpoint.

I think this can be achieved with former approach also by defining all
expected param in doc string properly.

> So I feel the later seems easy to use them as library methods.
> But as the demerit, a caller cannot specify *undefined" parameters for
> fuzzing tests.
> Nova v2.1 API should deny a request which includes undefined
> parameters, and Tempest will be able to test the behavior if the
> former style.
> So this is my concern point now.
>
> I tend to prefer the later(all parameters need to be defined on each
> method) because that will be useful for callers as the above.
> In addition, tempest-lib should be a library for implementing
> integration tests, then fuzzing tests can be out of scope.
> If fuzzing tests are necessary, we will be able to implement them with
> RestClient of tempest-lib without service clients.

Second approach also looks good but I prefer the first one. As you
mentioned that fuzzy testing will not be possible with second
approach, we should not restrict our lib functions for the same. Any
project/test suits should be able to do API/Fuzzy testing with
tempest-lib.
Good example is nova v2.1 as you mentioned. Using service clients of
lib, project (Nova) or test suits (Tempest) should be able to perform
fuzzy testing.

IMO, lib's service clients should not have any restriction on param
list (type or number etc) to be passed from its users. Whatever
(random param etc) is being passed that should be passed to
corresponding projects APIs and let APIs through error or success.

This can be useful when project adds new request attributes (with
current API change guidelines like with microversion), in that case we
do not need to change service clients. But microversion testing is not
yet decided so this should/may not be considered aa strong point?

>
> I am on between both now, and I'd like to get feedback about this
> before changing them.

Thanks a lot for driving service clients towards good interface :)

> Any comments are welcome :)
>
> Thanks
> Ken ohmichi
>
> __
> OpenStack Development Mailing List (not for usage questions)
> Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



-- 
Thanks & Regards
Ghanshyam Mann


[openstack-dev] [qa][tempest] "kwargs" of service clients for POST/PUT methods

2015-07-07 Thread Ken'ichi Ohmichi
Hi tempest team,

We are working for refactoring service clients in tempest for
preparing to migrate them to tempest-lib, and I found an inconsistency
between them.
So I'd like to ask opinions for considering which is the best as
library methods.

On some methods, a caller can specify whole body for sending a request
to http POST/PUT methods.
But on the others a caller can specify only part of body.

For example, a caller can specify any parameters to
update_quota_class_set() like:

tempest/services/compute/json/quota_classes_client.py
 34 def update_quota_class_set(self, quota_class_id, **kwargs):
 35 """
 36 Updates the quota class's limits for one or more resources.
 37 """
 38 post_body = json.dumps({'quota_class_set': kwargs})
 39
 40 resp, body = self.put('os-quota-class-sets/%s' % quota_class_id,
 41   post_body)

but update_quota_set method has each argument and a caller cannot
specify the other parameters like:

tempest/services/compute/json/quotas_client.py
 44 def update_quota_set(self, tenant_id, user_id=None,
 45  force=None, injected_file_content_bytes=None,
 46  metadata_items=None, ram=None, floating_ips=None,
 47  fixed_ips=None, key_pairs=None, instances=None,
 48  security_group_rules=None, injected_files=None,
 49  cores=None, injected_file_path_bytes=None,
 50  security_groups=None):
 51 """
 52 Updates the tenant's quota limits for one or more resources
 53 """
 54 post_body = {}
 55
 56 if force is not None:
 57 post_body['force'] = force
 58
 59 if injected_file_content_bytes is not None:
 60 post_body['injected_file_content_bytes'] = \
 61 injected_file_content_bytes
 [..]
 96 post_body = json.dumps({'quota_set': post_body})
 97
 98 if user_id:
 99 resp, body = self.put('os-quota-sets/%s?user_id=%s' %
100   (tenant_id, user_id), post_body)
101 else:
102 resp, body = self.put('os-quota-sets/%s' % tenant_id,
103   post_body)

By defining all parameters on each method like update_quota_set(), it
is easy to know what parameters are available from caller/programer
viewpoint.
So I feel the later seems easy to use them as library methods.
But as the demerit, a caller cannot specify *undefined" parameters for
fuzzing tests.
Nova v2.1 API should deny a request which includes undefined
parameters, and Tempest will be able to test the behavior if the
former style.
So this is my concern point now.

I tend to prefer the later(all parameters need to be defined on each
method) because that will be useful for callers as the above.
In addition, tempest-lib should be a library for implementing
integration tests, then fuzzing tests can be out of scope.
If fuzzing tests are necessary, we will be able to implement them with
RestClient of tempest-lib without service clients.

I am on between both now, and I'd like to get feedback about this
before changing them.
Any comments are welcome :)

Thanks
Ken ohmichi

__
OpenStack Development Mailing List (not for usage questions)
Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev