Swagger would be a good tool for this, but the amount of work involved in Swagger-izing the existing TO API would mean that we most likely wouldn't be able to get a Mock TO for quite a long time, and we could benefit from something like this in the meantime.
I agree that we should probably integrate Swagger into TO API 2.0 from the start, but I'm not sure we should really spend much effort on Swagger-izing the existing TO API since that would mean less capacity to work on a better version of the API with Swagger built-in from the start. - Rawlin On Mon, Feb 4, 2019 at 1:31 PM Fieck, Brennan <[email protected]> wrote: > > Swagger is meant to be used to design the API which can then spit out a spec > (as far as I can tell). I don't know how well that integrates with our > existing API, or even our existing documentation. But supposing it's fairly > easy to do, it means creating a separate model for every API response we > have, and that means maintaining all of those models twice. If we're going to > integrate Swagger I think it's better to wait until API v2 so that we can use > it to make the model and maintain it exactly once. > ________________________________________ > From: Gray, Jonathan <[email protected]> > Sent: Monday, February 4, 2019 1:09 PM > To: [email protected] > Subject: Re: [EXTERNAL] Traffic Ops Mock for testing > > What do you think about instead of rolling a separate tool we just finish the > swagger documentation that's partially integrated. Then you can just feed > that swagger catalog into any languages stub code generator. Bonus is that > you get client stubs too for free. > > Jonathan G > > > On 2/4/19, 1:03 PM, "Fieck, Brennan" <[email protected]> wrote: > > Just to be perfectly clear: there should be no database, just a single > service - an HTTP server - that responds to requests for the Traffic Ops API > request paths with responses that are structurally and syntactically valid > for the respective request path. > ________________________________________ > From: Rawlin Peters <[email protected]> > Sent: Monday, February 4, 2019 10:35 AM > To: [email protected] > Subject: Re: [EXTERNAL] Traffic Ops Mock for testing > > The idea is that Mock TO will use the same exact Go structs as TO, and > by default Mock TO will dynamically generate a response from the given > endpoint using reflection on the endpoint's corresponding Go struct. > No dataset would need to be maintained for Mock TO unless you wanted > to write specific test cases in which you provide a predefined static > response for Mock TO to return (be it via predefined Go structs or > just JSON files). The mechanism for choosing dynamic vs static > responses would have to be designed still, but that's the general > idea. > > - Rawlin > > On Mon, Feb 4, 2019 at 10:25 AM Gray, Jonathan > <[email protected]> wrote: > > > > How do you plan to build/maintain the dataset behind such a tool? Do > you intend to use a fixed TO dataset backed by postgres, or just fixed > responses to various combinations of parameters. > > > > Jonathan G > > > > On 2/4/19, 9:41 AM, "Rawlin Peters" <[email protected]> wrote: > > > > Hey all, > > > > @ocket8888 and I would like to propose implementing a minimal > Traffic > > Ops "mock" that could be used for integration testing things like > > Traffic Ops clients (basically, anything that interacts with the > > Traffic Ops API but doesn't need to verify the actual Traffic Ops > > implementation, just the external interface) without having to stand > > up a real instance of Traffic Ops. > > > > This Mock TO would basically just be a minimal Go HTTP server that > > receives requests against the TO API endpoints, unmarshalls JSON > > request data into the endpoint's corresponding Go TC struct, > marshalls > > the Go TC struct back into JSON, and returns it back to the client. > > > > So for most POST/PUT endpoints, the Mock TO would basically just be > an > > echo server, and for DELETEs it could just return some > > statically-defined response that matches the API implementation. > > Ideally, the DELETE could just reuse the corresponding DELETE > success > > response from TO, but we don't know how tightly we should integrate > > this Mock TO with the Real TO just yet. For GET endpoints, we could > > have Mock TO return some statically-defined JSON response for the > > endpoint (be it via literal JSON files read off the filesystem or > just > > predefined TC Go structs that would marshal into the expected JSON), > > but it would be nice to also have the Mock TO be able to dynamically > > generate a GET response using reflection against the Go struct. With > > dynamic generation of the response, we wouldn't have to maintain > > static JSON files against the current version of the API for > > everything, but we'd still be able to define static JSON responses > for > > specific test scenarios. > > > > For example, in order to test ORT, you might want to hit the Mock TO > > but define a static response for a configfile endpoint in order to > > verify that ORT can create the file from a given response. > > > > In order to test the Python and Java TO clients, we don't > necessarily > > need to define static JSON responses from Mock TO, since for those > we > > don't care about the actual values but more so about the structure > of > > the request and response through the various stages between the > client > > and the server (marshalling, unmarshalling, etc). > > > > We're currently getting great test coverage of the Go TO client due > to > > being used for the Go API integration tests, but the Python and Java > > TO clients are getting left behind. It would be nice to at least be > > able to verify that there haven't been any breaking-API changes that > > would actually break TO clients in the repo. For example, if there > is > > an API regression in a Go-rewrite of a Perl endpoint that breaks the > > Python TO client, we can easily find and fix that regression before > > the PR is merged. Currently, the only way we can find out if the > > Python or Java TO clients have been broken is through normal usage > > after breaking-PRs have been merged, which is not great. > > > > A Mock TO would enable us to test all these things, keep the in-repo > > TO clients from breaking, and help us prevent breaking the Traffic > Ops > > API contract accidentally. Mock TO would use the same Go structs as > > the Real TO, so Mock TO wouldn't have to maintain its own separate > > structs and would automatically stay in sync with changes to the > > official TO structs. It would essentially always stay up to date > with > > the latest version of the structs, which we think is important for > > something like this. Otherwise, it could quickly become out of sync > > with what it's supposed to mock, and its value as a test tool would > be > > degraded. > > > > A major advantage of this Mock TO is that it could be used for easy > > integration testing where you don't necessarily want to set up and > > configure a real instance of Traffic Ops, which is currently a major > > pain and honestly a bit overkill for things like just verifying > that a > > TO client can properly handle requests/responses using the current > > version of the API. > > > > Some goals we would have for Mock TO: > > - be integrated enough with Real TO so that it "automatically" stays > > up to date with Real TO's interface as much as possible > > - require very little maintenance (i.e. as new endpoints are added > or > > existing endpoints are modified, those changes would be picked up > > "automatically" in Mock TO) > > - virtually no required configuration (probably just a port to > listen on) > > - require no prerequisite data in order to run and serve mock > requests > > > > Please let us know if there's something specific you'd also like to > > see in a Mock TO or have other ideas for mocking TO that you'd like > to > > share. We'd like for Mock TO to be as pain-free and useful as > > possible. > > > > - @ocket8888 and Rawlin > > > > > >
