Hey y'all! Great conversation! This is something I've been working in/on for a few years now, here and there. I started by working on github.com/apigee-127/swagger-test-templates , a unit test scaffolding generator (the concept of which I think was mentioned a bit in conversation here earlier?). It parses a swagger spec and generates unit tests for every path/operation/status code defined, while exposing configurations for supplying parameters and expected response bodies. The main goal is to accelerate API development by enabling developers to create test code for their APIs early and often, thus strengthening the contract between implementation and specification. It was meant to be integrated with swagger-node, but has gotten a decent amount of use as an individual module.
I've since been working on an unofficial rewrite built on github.com/apigee-127/sway . The rewrite can be found at github.com/noahdietz/oatts and I'd love feedback/contributors as it is in the early stages. @Cody, I love the problems you're solving and bringing up. One thing I have found is valuable is if you can't assert against actual values in the expected responses, to merely validate the schema of the response body (for JSON, at least) is definitely a win (unsure if you Assertible is actually doing that just after a quick glance). WRT populating auth values, that's a tough one. With the aforementioned swagger-test-templates, because it was designed to generate test code locally, we added support for loading values from a config file or the environment for such sensitive things, rather than hard code them. This is tougher when all done in the cloud for them. Maybe you can expose a hook that they configure to pull some creds at runtime, that are not stored and they can change whenever? Lets chat if you're interested in sharing ideas! I'm @no_d_here on Twitter if thats easier than this or email. On Monday, May 8, 2017 at 4:14:57 PM UTC-7, Cody Reichert wrote: > > I've been really interested in this problem for a while now, and your > assumption is correct that it's definitely possible. We've been working on > solving this at Assertible <https://assertible.com>, and a couple of the > harder parts we've hit are: > > Route parameters and other variables. Required "setup" for PUT/DELETE/etc > requests. Setting up auth. In a lot of cases there just *has* to be some > additional configuration. > > That said, we've built what I this is a very good starting point to > automatically generate a test suite from a Swagger spec. > > We put together a post on Testing an API using Swagger > <https://assertible.com/blog/testing-an-api-using-swagger>. > > Basically, when you import a Swagger spec, a test is generated for each > unique endpoint/method combination (like *GET* /users and *POST* /users). > Route parameters are automatically filled in as "*variables*" and auth is > stubbed out from the "*securityDefinitions*" found in the swagger spec. > The actual tests/validations are done by creating "*assertions*" that are > created for the expected status code response. > > For example, importing the Twitter Swagger spec would give you something > like this: > > > <https://lh3.googleusercontent.com/-oIsCAXZYBAw/WRD6ZB5aRII/AAAAAAAAAU8/WwTJhE9fsuESsNPdWjhaz3Bl7UI5zq0kQCLcB/s1600/assertible-twitter-swagger-import.png> > What are your (or anyone elses) thoughts on this approach? We've got a lot > of plans/ideas for how we can dynamically generate even more tests and > assertions from a Swagger spec. The idea is that, you simply have to import > your spec once, and you're set up with automated tests, monitoring, failure > alerts, etc, right out of the box. > > Let me know what you think - I'm really interested in others people's > thoughts about testing from a spec! > > - Cody Reichert > [email protected] <javascript:> > https://assertible.com > > On Friday, August 16, 2013 at 6:38:35 AM UTC-5, chrokh wrote: >> >> I'm new to Swagger but I believe I'm getting a clearer picture of the >> eco-system, and it looks fantastic. >> >> However, I'm wondering whether there's a module that allows me to run >> automated tests against my actual API to see whether it conforms with my >> Swagger spec. *Without* server integration. >> >> Assuming you have a Swagger spec-file and a running API it sounds >> reasonable to me that this should be possible? Basically, black-box >> testing. Is there something like this? >> >> Thanks in advance! >> > -- You received this message because you are subscribed to the Google Groups "Swagger" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/d/optout.
