Comment #6 on issue 688 by bryan.oakley: would like to be able to associate metadata with a test case
http://code.google.com/p/robotframework/issues/detail?id=688

I understand your concern, but I disagree this feature can already be done. It can be simulated, it can be worked around, but it can't be done in a practical way IMO.

Lets look at a couple concrete examples. Let's say I want the version of the file from the revision control system actually in the file for each test case, so a later post-mortem can look at the test file and find out what version it was when a test case was added.

If I were to use tags, I'd end up with literally hundreds of tags, maybe thousands over time: version-1, version-2, version-1.1.1, etc. Not only that, but tag 'version-1' in file A has no relation to tag "version-1" in file B, so reporting on failures, or including or excluding tests based on that tag make no sense.

For our immediate need, we want to track which testing team created each test case. We can't do this at the file level because multiple teams may alter a test case file at some point or another. We don't care at all about teams during the running of the test -- we don't want results broken out by team, we won't ever include or exclude tests based on a team. We do, however, have nightly jobs that report on the number of tests written by a team. So, using tags like team-foo, team-bar ends up creating dozens of spurious tags in reports. Yes, we could use a comment, but that's sloppy IMO. Comments are for describing testing logic, not for identification.

I think the documentation concerns are not that big of a deal. The distinction between metadata and tags is easy: use tags to organize and categorize tests. Tags let you include and exclude tests in a test run, and you get results organized by tag in the reports. Metadata is for associating custom bits of data in a test case, typically for internal tracking and specialized tools. Tags potentially affect the running and reporting of a test, metadata does not.



Reply via email to