Re: [openstack-dev] [api] API Definition Formats
On Thu, 29 Jan 2015, michael mccune wrote: in a similar vein, i started to work on marking up the sahara and barbican code bases to produce swagger. for sahara this was a little easier as flask makes it simple to query the paths. for barbican i started a pecan-swagger[1] project to aid in marking up the code. it's still in infancy but i have a few ideas. pecan-swagger looks cool but presumably pecan has most of the info you're putting in the decorators in itself already? So, given an undecorated pecan app, would it be possible to provide it to a function and have that function output all the paths? -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] API Definition Formats
On 02/02/2015 10:26 AM, Chris Dent wrote: pecan-swagger looks cool but presumably pecan has most of the info you're putting in the decorators in itself already? So, given an undecorated pecan app, would it be possible to provide it to a function and have that function output all the paths? you are correct, pecan is storing most of the information we want in it's controller metadata. i am working on the next version of pecan-swagger now that will reduce the need for so many decorators, and instead pull the endpoint information out of the pecan based controller classes. in terms of having a completely undecorated pecan app, i'm not sure that's possible just yet due to the object-dispatch routing used by pecan. in the next version of pecan-swagger i'm going to reduce the deocrators to only be needed on controller classes, but i'm not sure that it will be possible to reduce further as there will need to be some way to learn the route path hierarchy. i suppose in the future it might be advantageous to create a pecan controller base class that could help inform the routing structure, but this would still need to be added to current pecan projects. mike __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] API Definition Formats
On 01/28/2015 12:56 PM, Max Lincoln wrote: tl;dr: I wanted to be able to see what OpenStack APIs might look like in Swagger and starting experimenting with Swagger in projects for things like stubbing services, API test coverage, and code generation. In order to do that I created wadl2swagger [1]. I've published copies [2] of what the converted documents look like. If you follow the Open on Swagger-Editor links at http://rackerlabs.github.io/wadl2swagger/openstack.html you can open the Swagger-equivalent of any WADL file and see both the source and a preview of what Swagger-generated documentation would look like. this is awesome, very cool to look at the converted WADL. in a similar vein, i started to work on marking up the sahara and barbican code bases to produce swagger. for sahara this was a little easier as flask makes it simple to query the paths. for barbican i started a pecan-swagger[1] project to aid in marking up the code. it's still in infancy but i have a few ideas. also, i've collected my efforts so far here[2]. mike [1]: https://github.com/elmiko/pecan-swagger [2]: https://github.com/elmiko/os-swagger-docs __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] API Definition Formats
On Jan 18, 2015, at 9:25 PM, Jay Pipes jaypi...@gmail.com wrote: On 01/13/2015 07:41 AM, Sean Dague wrote: On 01/09/2015 04:17 PM, Everett Toews wrote: One thing that has come up in the past couple of API WG meetings [1] is just how useful a proper API definition would be for the OpenStack projects. By API definition I mean a format like Swagger, RAML, API Blueprint, etc. These formats are a machine/human readable way of describing your API. Ideally they drive the implementation of both the service and the client, rather than treating the format like documentation where it’s produced as a by product of the implementation. I think this blog post [2] does an excellent job of summarizing the role of API definition formats. Some of the other benefits include validation of requests/responses, easier review of API design/changes, more consideration given to client design, generating some portion of your client code, generating documentation, mock testing, etc. If you have experience with an API definition format, how has it benefitted your prior projects? Do you think it would benefit your current OpenStack project? It would hugely benefit OpenStack to have this clear some where that was readable. I don't specifically have experience with these, my only feedback would be make sure whatever format supports having multiple examples per API call referenced or embedded. My experience is that API specs aren't typically fully read and injested. Instead examples are used to get some minimum working code, then bits are spot referenced and evolved until the client code looks like it does what was expected. So providing multiple examples per API will help more people wrap their head around the interface in question. In my experience developing client tools for OpenStack for the past 2 years, the examples are useful but, sooner rather than later, I find myself needing the full API definition (I’m avoiding the word spec because it’s overloaded in OpenStack land already). The reason the OpenStack API examples are so useful is because they include details that really should be part of the definition. So often I find myself looking at an example and wondering where a param came from and its possible range of values only to find no mention of that param in the definition. I’m definitely +1 to multiple examples per API but those examples need to be built on top of proper API definitions. I’d like to see us move away from an API documentation second mindset to an API definition first mindset. Having the API definition drive the implementation of the client and the service. This is spot-on, Sean. I would support making Swagger the API definition format for OpenStack APIs. I think it's by far the best of the bunch, in my experience, and I've used API Blueprint, Swagger, and RAML. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] API Definition Formats
tl;dr: I wanted to be able to see what OpenStack APIs might look like in Swagger and starting experimenting with Swagger in projects for things like stubbing services, API test coverage, and code generation. In order to do that I created wadl2swagger [1]. I've published copies [2] of what the converted documents look like. If you follow the Open on Swagger-Editor links at http://rackerlabs.github.io/wadl2swagger/openstack.html you can open the Swagger-equivalent of any WADL file and see both the source and a preview of what Swagger-generated documentation would look like. I definitely agree that moving to a better API definition would be very beneficial. Technically WADL *is* an API definition format, it's just not one that's very usable or that has an active community (with the exception of documentation, and even that is probably mostly only OpenStack). I think it's partly because WADL was created during a transitional period - services were tending towards REST but not yet towards JSON - but I think that Apigee summed it up nicely when they created the Swagger Editor [3]. WSDL and WADL failed to cross languages and therefore developer communities, but Swagger has thrived, with community contributions easily outnumbering those from the team at Reverb. Check out the list of related projects for Swagger [4] to verify that for yourself. Not only are there *actively developed* projects in many different programming languages, the projects also target a wide range of problems, from extracting documentation from code to testing or code generation. I think that's especially true of Swagger, though you could say the same thing about RAML. However, the Swagger/RAML gap is much smaller than the WADL/Swagger gap, and there's already a swagger2raml project underway, so converting data from WADL to Swagger seems like a good move even if RAML is still being considered. So since WADL is an API definition format, there were three possible ways to try to get a Swagger copy of the OpenStack APIs: - Convert WADL to Swagger - Extract Swagger from the OpenStack code or tests - Write Swagger by hand I focused on the first approach, merging in a little bit of hand-written Swagger from the third approach to cover data that I wanted in Swagger but that isn't available in the WADL. The second approach may also be worth pursuing, especially if the data we can get from WADL and the data we can get from code/tests are complimentary. The WADL is good at converting the routing, parameters, request/response examples, general documentation and expected response codes, but it isn't easy to get request/response body schemas. I hope this project can help in one of two ways: - A preview of what OpenStack Swagger would look like to start discussions about best practices or APIs that are difficult to model in Swagger. - As an actually tool to enable a WADL-Swagger transition or to compare to Swagger from other sources to ensure the documentation/implementation are in sync. If you just take these documents as a draft than there's no need to look at the code, but if you want to use and improve the tool then I need some help maintaining the project. It probably makes sense to clean up the project before doing any more enhancements, because it started as a spike and is my first non-trivial Python project, so the code is undoubtedly sloppy. However it's fairly small (543 lines of Python) so cleaning it up probably isn't a huge undertaking. Please let me know or check out the issues on GitHub [5] if you're interested in helping clean with the project. [1]: https://github.com/rackerlabs/wadl2swagger [2]: http://rackerlabs.github.io/wadl2swagger/openstack.html [3]: https://blog.apigee.com/detail/a_design_first_approach_to_building_apis_with_swagger [4]: https://github.com/swagger-api/swagger-spec#additional-libraries [5]: https://github.com/rackerlabs/wadl2swagger/issues On Wed, Jan 28, 2015 at 11:41 AM, Everett Toews everett.to...@rackspace.com wrote: On Jan 18, 2015, at 9:25 PM, Jay Pipes jaypi...@gmail.com wrote: On 01/13/2015 07:41 AM, Sean Dague wrote: On 01/09/2015 04:17 PM, Everett Toews wrote: One thing that has come up in the past couple of API WG meetings [1] is just how useful a proper API definition would be for the OpenStack projects. By API definition I mean a format like Swagger, RAML, API Blueprint, etc. These formats are a machine/human readable way of describing your API. Ideally they drive the implementation of both the service and the client, rather than treating the format like documentation where it’s produced as a by product of the implementation. I think this blog post [2] does an excellent job of summarizing the role of API definition formats. Some of the other benefits include validation of requests/responses, easier review of API design/changes, more consideration given to client design, generating some portion of your client code, generating documentation, mock
Re: [openstack-dev] [api] API Definition Formats
On 01/13/2015 07:41 AM, Sean Dague wrote: On 01/09/2015 04:17 PM, Everett Toews wrote: One thing that has come up in the past couple of API WG meetings [1] is just how useful a proper API definition would be for the OpenStack projects. By API definition I mean a format like Swagger, RAML, API Blueprint, etc. These formats are a machine/human readable way of describing your API. Ideally they drive the implementation of both the service and the client, rather than treating the format like documentation where it’s produced as a by product of the implementation. I think this blog post [2] does an excellent job of summarizing the role of API definition formats. Some of the other benefits include validation of requests/responses, easier review of API design/changes, more consideration given to client design, generating some portion of your client code, generating documentation, mock testing, etc. If you have experience with an API definition format, how has it benefitted your prior projects? Do you think it would benefit your current OpenStack project? It would hugely benefit OpenStack to have this clear some where that was readable. I don't specifically have experience with these, my only feedback would be make sure whatever format supports having multiple examples per API call referenced or embedded. My experience is that API specs aren't typically fully read and injested. Instead examples are used to get some minimum working code, then bits are spot referenced and evolved until the client code looks like it does what was expected. So providing multiple examples per API will help more people wrap their head around the interface in question. This is spot-on, Sean. I would support making Swagger the API definition format for OpenStack APIs. I think it's by far the best of the bunch, in my experience, and I've used API Blueprint, Swagger, and RAML. Best, -jay __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] API Definition Formats
On 1/13/15, 04:16, Chris Dent chd...@redhat.com wrote: On Tue, 13 Jan 2015, Ian Cordasco wrote: On 1/12/15, 17:21, Chris Dent chd...@redhat.com wrote: On Mon, 12 Jan 2015, Ian Cordasco wrote: This worked extremely well in my experience and helped improve development time for new endpoints and new endpoint versions. The documentation was also heavily used for the multiple internal clients for that API. This idea of definition formats seems like a reasonable idea (see my response to Anne over on the gabbi thread[1]) but I worry about a few things: Your response below suggests that I didn't make the point above about how I think this is a reasonable idea strongly enough. My comments were not to disparage the idea but rather to start applying some flesh on the bare bones of some concerns that might arise as people try to apply the idea. Yes, I misunderstood you. Sorry about that. * Unless you're auto generating the code from the formal defition you run into a lot of opportunities for truth to get out of sync between the definition and the implementation. The /documentation/ was used by /developers/ to build the internal clients. It was also used by the front-end developers who built the user-facing interface that consumed these APIs. Yes, that restates the problem nicely: all code always has the problem I stated: documentation (in whatever form) is highly likely to get out of sync with implementations. You made some motions towards ways to guard against this problem when you described how tests and documentation can be interlinked. Sounds great! But it doesn't entirely ameliorate the concern. So if the schemas are part of the test suite (enforcing them and ensuring that the endpoints return responses compliant with it) then we’ll catch changes that aren’t documented by the schema. Using the schema (with its examples) to document the endpoint’s request/response cycle means that the documentation of what the endpoint expects and what it returns is at the very least technically correct. Good descriptions of the endpoints and the data in the schema will only improve the generated documentation. I’m not sure how that doesn’t reduce the overhead of writing documentation for the API. * Specifying every single endpoint or many endpoints is just about as anti-REST as you can get if you're a HATEOAS believer. I suspect this line of concern is well-trod ground and not worth bringing back up, but all this stuff about versioning is meh and death to client diversity. Except that we don’t even try to achieve HATEOAS (or at least the OpenStack APIs I’ve seen don’t). If we’re being practical about it, then the idea that we have a contract between the API consumer (also read: user) and the server makes for a drastic simplification. The fact that the documentation is auto-generated means that writing tests with gabbi would be so much simpler for you (than waiting for people familiar with it to help you). We don't try to achieve HATEOAS _now_ but why should it be off the radar for things we do in the future? There's a frequent tension in the API discussions between describing and validating the existing APIs and describing a target for future improvements. I think we should be doing both. There could be room for discussion about increased hypermedia. It is _one_ of the ways to allow robust longevity of APIs. I don't personally want to open that can of worms if it has been opened many times before, but it is worth noting the option as it provides a much different technique for addressing the concerns about versioning and documentation. All that said, what you describe in the following would be nice if it can be made true and work well. I suspect I'm still scarred from WSDL and company but I'm not optimistic that culturally it can be made to work. Simple HTTP APIs wins over SOAP and pragmatic HTTP wins over true REST and JSON wins over XML because all of the latter have a flavor of flexibility and easy to diddle that does not exist in the former. The problem is social, not technical. Well I’ve only seen it used with JSON, so I’m not sure where you got XML from (or SOAP for that matter). Besides, this is a tool that will help the API developers more than it will hurt them. In-tree definitions in a (fairly) human readable format that clearly states what is accepted and generated by an endpoint means that scrutinizing Pecan and WSME isn’t necessary (until you start writing the endpoint itself). What I was expressing in that paragraph was a bit of allegoric thinking: This thing you're suggesting smells a bit like these other things that although they sounded like good ideas failed to have long-lived success all for much the same reason. I guess I should have made the implicit questions explicit: How is this formalism that you are suggesting different from those? How can we guard against the social tendency of devs who have freedom of choice to choose tools that
Re: [openstack-dev] [api] API Definition Formats
On Tue, 13 Jan 2015, Ian Cordasco wrote: On 1/12/15, 17:21, Chris Dent chd...@redhat.com wrote: On Mon, 12 Jan 2015, Ian Cordasco wrote: This worked extremely well in my experience and helped improve development time for new endpoints and new endpoint versions. The documentation was also heavily used for the multiple internal clients for that API. This idea of definition formats seems like a reasonable idea (see my response to Anne over on the gabbi thread[1]) but I worry about a few things: Your response below suggests that I didn't make the point above about how I think this is a reasonable idea strongly enough. My comments were not to disparage the idea but rather to start applying some flesh on the bare bones of some concerns that might arise as people try to apply the idea. * Unless you're auto generating the code from the formal defition you run into a lot of opportunities for truth to get out of sync between the definition and the implementation. The /documentation/ was used by /developers/ to build the internal clients. It was also used by the front-end developers who built the user-facing interface that consumed these APIs. Yes, that restates the problem nicely: all code always has the problem I stated: documentation (in whatever form) is highly likely to get out of sync with implementations. You made some motions towards ways to guard against this problem when you described how tests and documentation can be interlinked. Sounds great! But it doesn't entirely ameliorate the concern. * Specifying every single endpoint or many endpoints is just about as anti-REST as you can get if you're a HATEOAS believer. I suspect this line of concern is well-trod ground and not worth bringing back up, but all this stuff about versioning is meh and death to client diversity. Except that we don’t even try to achieve HATEOAS (or at least the OpenStack APIs I’ve seen don’t). If we’re being practical about it, then the idea that we have a contract between the API consumer (also read: user) and the server makes for a drastic simplification. The fact that the documentation is auto-generated means that writing tests with gabbi would be so much simpler for you (than waiting for people familiar with it to help you). We don't try to achieve HATEOAS _now_ but why should it be off the radar for things we do in the future? There's a frequent tension in the API discussions between describing and validating the existing APIs and describing a target for future improvements. I think we should be doing both. There could be room for discussion about increased hypermedia. It is _one_ of the ways to allow robust longevity of APIs. I don't personally want to open that can of worms if it has been opened many times before, but it is worth noting the option as it provides a much different technique for addressing the concerns about versioning and documentation. All that said, what you describe in the following would be nice if it can be made true and work well. I suspect I'm still scarred from WSDL and company but I'm not optimistic that culturally it can be made to work. Simple HTTP APIs wins over SOAP and pragmatic HTTP wins over true REST and JSON wins over XML because all of the latter have a flavor of flexibility and easy to diddle that does not exist in the former. The problem is social, not technical. Well I’ve only seen it used with JSON, so I’m not sure where you got XML from (or SOAP for that matter). Besides, this is a tool that will help the API developers more than it will hurt them. In-tree definitions in a (fairly) human readable format that clearly states what is accepted and generated by an endpoint means that scrutinizing Pecan and WSME isn’t necessary (until you start writing the endpoint itself). What I was expressing in that paragraph was a bit of allegoric thinking: This thing you're suggesting smells a bit like these other things that although they sounded like good ideas failed to have long-lived success all for much the same reason. I guess I should have made the implicit questions explicit: How is this formalism that you are suggesting different from those? How can we guard against the social tendency of devs who have freedom of choice to choose tools that are embedded in the ethic of easy to diddle? How can we ensure that the tools we create fall on that side of things? -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] API Definition Formats
On 01/09/2015 04:17 PM, Everett Toews wrote: One thing that has come up in the past couple of API WG meetings [1] is just how useful a proper API definition would be for the OpenStack projects. By API definition I mean a format like Swagger, RAML, API Blueprint, etc. These formats are a machine/human readable way of describing your API. Ideally they drive the implementation of both the service and the client, rather than treating the format like documentation where it’s produced as a by product of the implementation. I think this blog post [2] does an excellent job of summarizing the role of API definition formats. Some of the other benefits include validation of requests/responses, easier review of API design/changes, more consideration given to client design, generating some portion of your client code, generating documentation, mock testing, etc. If you have experience with an API definition format, how has it benefitted your prior projects? Do you think it would benefit your current OpenStack project? It would hugely benefit OpenStack to have this clear some where that was readable. I don't specifically have experience with these, my only feedback would be make sure whatever format supports having multiple examples per API call referenced or embedded. My experience is that API specs aren't typically fully read and injested. Instead examples are used to get some minimum working code, then bits are spot referenced and evolved until the client code looks like it does what was expected. So providing multiple examples per API will help more people wrap their head around the interface in question. -Sean Thanks, Everett [1] https://wiki.openstack.org/wiki/Meetings/API-WG [2] http://apievangelist.com/2014/12/21/making-sure-the-most-important-layers-of-api-space-stay-open/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev -- Sean Dague http://dague.net __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] API Definition Formats
On 1/9/15, 15:17, Everett Toews everett.to...@rackspace.com wrote: One thing that has come up in the past couple of API WG meetings [1] is just how useful a proper API definition would be for the OpenStack projects. By API definition I mean a format like Swagger, RAML, API Blueprint, etc. These formats are a machine/human readable way of describing your API. Ideally they drive the implementation of both the service and the client, rather than treating the format like documentation where it’s produced as a by product of the implementation. I think this blog post [2] does an excellent job of summarizing the role of API definition formats. Some of the other benefits include validation of requests/responses, easier review of API design/changes, more consideration given to client design, generating some portion of your client code, generating documentation, mock testing, etc. If you have experience with an API definition format, how has it benefitted your prior projects? Do you think it would benefit your current OpenStack project? Thanks, Everett [1] https://wiki.openstack.org/wiki/Meetings/API-WG [2] http://apievangelist.com/2014/12/21/making-sure-the-most-important-layers- of-api-space-stay-open/ Hey Everett, As we discussed in the meeting, I have some experience with a library called Interpol [1] and using it in a massive API service. The idea behind that service was re-written as an open source case study in a project called Caravan [2]. In short, each and every endpoint used JSON Schema to validate the request and response for each version of the endpoint. (Yes, endpoints were versioned individually and that’s a topic for a different discussion.) The files used by Interpol (which is what applied the defined JSON Schema to the request/response cycle via Rack middleware) looked something like https://github.com/bendyworks/caravan/blob/master/lib/endpoint_definitions/ users/user_by_id.yml. If you read it closely, you’ll notice that path parameters are part of the schema [3] and status codes are required [4]. Each part of the schema also has the ability to be described [5]. This allows for Interpol to automatically document the API for you. Finally, you can define example responses [6] so you can prop up a stub application for other services/applications to use. Finally, Interpol has a way of testing the endpoint definitions (as they’re referred to) to ensure that the example data actually does follow the schema provided. As far as I know, there’s nothing similar to Interpol in Python … yet. I’m fairly confident that the middleware would take a weekend or two of sprinting to complete. Further, we could allow for more formats than YAML but I think this could tie in well with the gabbi testing discussion taking place. The rest might take a bit longer to complete. In short, using schemas in test and in production allowed the integration/acceptance tests to remain far more succinct. If you have something enforcing your request and response formats then you can simply test that you did get a status code 200 because something else has validated the contents. If you want to validate that there’s items in the array, you can skip validating the other properties because if there’s at least one, the objects inside have been validated by the middleware (so you can assert at least one came back and be confident). This worked extremely well in my experience and helped improve development time for new endpoints and new endpoint versions. The documentation was also heavily used for the multiple internal clients for that API. The company that used this used the validation in production (as well as in testing) had no problems with scaling or performance. The problem with building something like this /might/ be tying it in to the different frameworks used by each of the services but on the whole could be delegated to each service as it looks to integrate. From my personal perspective, YAML is a nice way to document all of this data, especially since it’s a format that most any language can parse. We used these endpoint definitions to simply how we wrote clients for the API we were developing and I suspect we could do something similar with the existing clients. It would also definitely help any new clients that people are currently writing. The biggest win for us would be having our documentation mostly auto-generated for us and having a whole suite of tests that would check that a real response matches the schema. If it doesn’t, we know the schema needs to be updated and then the docs would be automatically updated as a consequence. It’s a nice way of enforcing that the response changes are documented as they’re changed. Cheers, Ian [1] https://github.com/seomoz/interpol [2] https://github.com/bendyworks/caravan [3] https://github.com/bendyworks/caravan/blob/aa05fb345ad346b85fa989e857478491 2104570b/lib/endpoint_definitions/users/user_by_id.yml#L8..L12 [4]
Re: [openstack-dev] [api] API Definition Formats
On Mon, 12 Jan 2015, Ian Cordasco wrote: This worked extremely well in my experience and helped improve development time for new endpoints and new endpoint versions. The documentation was also heavily used for the multiple internal clients for that API. This idea of definition formats seems like a reasonable idea (see my response to Anne over on the gabbi thread[1]) but I worry about a few things: * Unless you're auto generating the code from the formal defition you run into a lot of opportunities for truth to get out of sync between the definition and the implementation. * Ugh, auto generated code. Magic. Ew. This is Python by golly! * Specifying every single endpoint or many endpoints is just about as anti-REST as you can get if you're a HATEOAS believer. I suspect this line of concern is well-trod ground and not worth bringing back up, but all this stuff about versioning is meh and death to client diversity. * Yes to this: The problem with building something like this /might/ be tying it in to the different frameworks used by each of the services but on the whole could be delegated to each service as it looks to integrate. All that said, what you describe in the following would be nice if it can be made true and work well. I suspect I'm still scarred from WSDL and company but I'm not optimistic that culturally it can be made to work. Simple HTTP APIs wins over SOAP and pragmatic HTTP wins over true REST and JSON wins over XML because all of the latter have a flavor of flexibility and easy to diddle that does not exist in the former. The problem is social, not technical. From my personal perspective, YAML is a nice way to document all of this data, especially since it’s a format that most any language can parse. We used these endpoint definitions to simply how we wrote clients for the API we were developing and I suspect we could do something similar with the existing clients. It would also definitely help any new clients that people are currently writing. The biggest win for us would be having our documentation mostly auto-generated for us and having a whole suite of tests that would check that a real response matches the schema. If it doesn’t, we know the schema needs to be updated and then the docs would be automatically updated as a consequence. It’s a nice way of enforcing that the response changes are documented as they’re changed. [1] http://lists.openstack.org/pipermail/openstack-dev/2015-January/054287.html -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent__ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [api] API Definition Formats
On 1/12/15, 17:21, Chris Dent chd...@redhat.com wrote: On Mon, 12 Jan 2015, Ian Cordasco wrote: This worked extremely well in my experience and helped improve development time for new endpoints and new endpoint versions. The documentation was also heavily used for the multiple internal clients for that API. This idea of definition formats seems like a reasonable idea (see my response to Anne over on the gabbi thread[1]) but I worry about a few things: * Unless you're auto generating the code from the formal defition you run into a lot of opportunities for truth to get out of sync between the definition and the implementation. The /documentation/ was used by /developers/ to build the internal clients. It was also used by the front-end developers who built the user-facing interface that consumed these APIs. * Ugh, auto generated code. Magic. Ew. This is Python by golly! I’m not suggesting auto-generated code (although that’s always a *possibility*). * Specifying every single endpoint or many endpoints is just about as anti-REST as you can get if you're a HATEOAS believer. I suspect this line of concern is well-trod ground and not worth bringing back up, but all this stuff about versioning is meh and death to client diversity. Except that we don’t even try to achieve HATEOAS (or at least the OpenStack APIs I’ve seen don’t). If we’re being practical about it, then the idea that we have a contract between the API consumer (also read: user) and the server makes for a drastic simplification. The fact that the documentation is auto-generated means that writing tests with gabbi would be so much simpler for you (than waiting for people familiar with it to help you). * Yes to this: The problem with building something like this /might/ be tying it in to the different frameworks used by each of the services but on the whole could be delegated to each service as it looks to integrate. All that said, what you describe in the following would be nice if it can be made true and work well. I suspect I'm still scarred from WSDL and company but I'm not optimistic that culturally it can be made to work. Simple HTTP APIs wins over SOAP and pragmatic HTTP wins over true REST and JSON wins over XML because all of the latter have a flavor of flexibility and easy to diddle that does not exist in the former. The problem is social, not technical. Well I’ve only seen it used with JSON, so I’m not sure where you got XML from (or SOAP for that matter). Besides, this is a tool that will help the API developers more than it will hurt them. In-tree definitions in a (fairly) human readable format that clearly states what is accepted and generated by an endpoint means that scrutinizing Pecan and WSME isn’t necessary (until you start writing the endpoint itself). From my personal perspective, YAML is a nice way to document all of this data, especially since it’s a format that most any language can parse. We used these endpoint definitions to simply how we wrote clients for the API we were developing and I suspect we could do something similar with the existing clients. It would also definitely help any new clients that people are currently writing. The biggest win for us would be having our documentation mostly auto-generated for us and having a whole suite of tests that would check that a real response matches the schema. If it doesn’t, we know the schema needs to be updated and then the docs would be automatically updated as a consequence. It’s a nice way of enforcing that the response changes are documented as they’re changed. [1] http://lists.openstack.org/pipermail/openstack-dev/2015-January/054287.htm l -- Chris Dent tw:@anticdent freenode:cdent https://tank.peermore.com/tanks/cdent __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
[openstack-dev] [api] API Definition Formats
One thing that has come up in the past couple of API WG meetings [1] is just how useful a proper API definition would be for the OpenStack projects. By API definition I mean a format like Swagger, RAML, API Blueprint, etc. These formats are a machine/human readable way of describing your API. Ideally they drive the implementation of both the service and the client, rather than treating the format like documentation where it’s produced as a by product of the implementation. I think this blog post [2] does an excellent job of summarizing the role of API definition formats. Some of the other benefits include validation of requests/responses, easier review of API design/changes, more consideration given to client design, generating some portion of your client code, generating documentation, mock testing, etc. If you have experience with an API definition format, how has it benefitted your prior projects? Do you think it would benefit your current OpenStack project? Thanks, Everett [1] https://wiki.openstack.org/wiki/Meetings/API-WG [2] http://apievangelist.com/2014/12/21/making-sure-the-most-important-layers-of-api-space-stay-open/ __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
