On Wed, Aug 18, 2010 at 10:05 AM, Roberto Longobardi <secc...@gmail.com> wrote: > Hi Olemis,
:o) > I have introduced the "Test Plans" feature in the plugin. Cool ! It's very nice to realize that you're really enthusiastic about developing the plugin . Hopefully I will be able to adapt it to my own needs sooner or later (I already bought the hammer ;o) > The scenario you describe below would now look like as follows. >> - TCs are *discovered* (generated ?) from >> source code (e.g. ) committed to a repos >> and they are using static code analysis, >> abstract models , XML specs ... >> - Test execution , etc , is performed by an external >> CI tool (currently Hudson + Bitten ...) >> >> In this scenario , I'd like to be able to >> >> - Retrieve test results from CI tool using an API >> - Trigger build jobs on changing TC states or when >> aforementioned test campaigns (which will be >> mapped to in Hudson builds) are started , >> explicitly executed . >> - ... well some other , but have no time to say it all >> > > You can create test catalogs and cases programmatically by means of the > following http requests: [...] > Create a root test catalog: > [...] > Create a sub-catalog: > [...] > Create a Test Case: > [...] > Create a Test Plan from a specific catalog: [...] > > Set a Test Case execution verdict, in the context of a Test Plan: > [...] This is really good news !!! However I'd really like to have an RPC interface to perform these actions (because in my deployment environment there will be external tools e.g. Hudson, ... and a BPMS that will be in «touch» with the test case manager). I can dedicate some time to get this done, and maintain the RPC handlers. I'd really like to join | contribute to the project . The major difficulty to do it (from my perspective ;o) is that the VCS is SVN (and I cannot interact with SVN repos since long time ago :-( ... ) . If the switch to a DVCS like bzr, hg, git is not an option , as a workaround , I'll try to create (somehow) an HgSVN repos and publish it as well as a patch queue @ Bitbucket . I'll develop patches and send them back to you . Is it ok ? PS: Could you please register the plugin in PyPI ? > The supported statuses are currently: > - TO_BE_TESTED > - SUCCESSFUL > - FAILED > Support for custom test results may be an important addition for future versions (there are lots of threads about this at TiP ML ;o) . An approach to get this done would be to consider test statuses as dumb plain text identifiers, and give them some meaning inside Trac using workflow actions (in a similar way to tickets actions). >> - There are traceability relationships between TCs and >> tickets (more technically speaking, requirements >> tracked by tickets ;o) so that it's possible to let's say >> know what TCs should be run on changing a ticket >> state manually | by workflow or whatever . > > The traceability is currently only from a Ticket top the corresponding Test > Case, but not the other way around. Ok, but ideally it would be nice to have some kind of traceability matrices (probably implemented as sparse matrices ;o) and custom traceability relations. The traceability model in use (in this concrete scenario) is based on Requisite Pro traceability matrices, but it would be nice to have something like that inside Trac ;o). Of course , this could be handled at a higher level of abstraction ;o) > You can open a Ticket and have a traceback to the (e.g. failed) Test Case as > follows: > Open a Ticket on a Test Case: > Whether you deploy TracTicketTemplatePlugin or not, you can get the > following URL, where testCaseNumber is the Test Case complete path, planid > is the Test Plan ID and planName is its name > [...] Do you use custom ticket fields ? >> - Tickets , test cases , build jobs and everything participate >> in a workflow , even if there are multiple tools >> involved. > > I get the idea, but maybe orchestrating different tools > in a workflow should > be the task of a different tool (a process server?). > Well as I mentioned before in this particular scenario there are two concurrent workflows installed . Trac built-in workflow orchestrates *EVERYTHING* related to software dev process and part of QA. Another BPEL-WS enabled tool considers Trac (and it's workflow ;o) as a block of a wider enterprise process. It encloses other areas (e.g. CRM, HRM, ...) and interacts with Trac via RPC . >> - How extensible are TC management components ? > I haven't currently defined any extension points... do you guys have any > idea what could be useful here (besides custom test case execution > verdicts)? > I'll take a look and suggest concrete extension points. I just need to prepare the DVCS stuff ;o) >> - Wouldn't it be nice to have workflow actions >> related to TC management ? > > Being test cases (and catalogs) actually Wiki pages gives us for free the > versioning and history capabilities. > I have also added with this release custom security permissions for test > case and test plan management. > What else may be in a workflow then, approval processes? Notifications? > E.g. consider the case when you want to mark as suspicious all test cases related to a ticket (well assuming that 1 to many relations are possible, which what really hapens in real life ;o) once you set ticket state to 'reopened' (or whatever) or OTOH consider the case when you check in code in VCS, hooks update ticket state (e.g. close it), then it's prepared for testing, so external CI-tool starts build jobs (a Trac plugin triggers the build according to workflow ;o), and when done, test case status is updated according to test reports published by CI-tool, on failure ticket is reopened and a new test set (you call it test plan?) containing failed testcases is prepared to retry the build later if changes are detected ... and all this orchestrated by Trac workflow ;o). >> - Wouldn't it be nice to provide RPC handlers to >> perform remote actions on TC specs (e.g. from >> inside VCS hooks ;o) > > I'd like to stick to the restful API as the primary remote interface. > Having to POST Wiki pages to submit them is annoying anyway, so I was > planning to add an additional parameter to the /testcreate API to directly > save the Wiki pages, with no need to edit them eventually. Quite unlikely . All such POST requests are protected by RequestDispacher against XSRF attacks (@see trac.web.main.RequestDispacher.dispatch). > If any VCS supports making http calls, then this should work in your > scenario. > Well I don't get it very well but TracGViz provides read-only access to VCS (any VCS suported by plugins implementing Trac VCS API ;o) items via RPC ... > What do you think? More or less what's been said above ;o) -- Regards, Olemis. Blog ES: http://simelo-es.blogspot.com/ Blog EN: http://simelo-en.blogspot.com/ Featured article: -- You received this message because you are subscribed to the Google Groups "Trac Users" group. To post to this group, send email to trac-us...@googlegroups.com. To unsubscribe from this group, send email to trac-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/trac-users?hl=en.
