Note: A day late, due to odd schedules. What did I get done this week?
This week I switched focus from cgimap coding to integration testing. To do this, I had to develop a way to test API responses. I spent awhile looking at different frameworks for testing that would work on a RESTful API and allow me to check the contents of XML responses. Because multiple XML documents may parse the same but be different on a byte-by-byte level I couldn't do exact comparisons with files, and because there are multiple ways to represent the same data in OSM XML (e.g. reordering the tag elements for an object) I could not simply compare XML trees. I settled on Gatling, a tool originally designed for performance testing. It has a DSL which is rich enough to test API responses and includes Xpath 1.0 support. Xpath allows me to test conditions like the <osm/> element has one child, and that child is a node element with id=1234 and that node has two tags which are a=foo and b=bar. I am also writing an OSM XML file which contains the test cases which gets loaded into the API software of choice, and an OSC XML file which then gets applied. These allow me to check stuff like deleted nodes, moved nodes, etc. The repository I am using is https://github.com/pnorman/openstreetmap-api-testsuite I believe I have completed an adequate test suite for node and nodes + diffs Next week: More API tests! I also need to follow-up on some issues found with apidb, rails port, existing cgimap code, osmosis, and gatling. Still outstanding: pgsnapshot changes, raising a plea for a backport of 9.3 postgres index fudge factor changes Problems I'm not really sure how much to test in each test. As an example, I have the "one tag", "multiple tags" and "different tag order" tests at https://github.com/pnorman/openstreetmap-api-testsuite/blob/master/Node.scal a#L152-207 which are checking the visible, uid, user, lon, lat, etc of the nodes the API is emitting. On the other hand, I've already tested that the API emits those attributes successfully in the earlier "attributes" and "different attributes" tests. Feedback from someone more experienced at specifying tests would be nice. I'm also not sure how closely to define the API responses. For example, the API wiki page specifies a 410 response for a deleted node. The rails port will always emit an empty response body with Content-Length: 0 and Cache-Control: no-cache. Should I be testing these other headers? _______________________________________________ dev mailing list [email protected] http://lists.openstreetmap.org/listinfo/dev
