Not sure what to think of those examples.

The theory sounds nice and it probably produce nice result when
talking about one method with a few inputs and an output but for
anything more complex (like anything involving components) I feel you
end up with a generated test which is a lot harder to read than simply
looking at the code.

* https://jira.xwiki.org/browse/XWIKI-13031: does not produce the same
error at all. Looks like evocash just try to produce any
ClassCastException but does not care about the actual types in it
which is quite useless (plus reading that test requires a lot of
effort just to conclude that the type of the field in the document is
not String...)

* https://jira.xwiki.org/browse/XWIKI-14475: totally unreadable
* https://jira.xwiki.org/browse/XRENDERING-422: not really useful but
I doubt evocrash can do much about this kind of issue

* https://jira.xwiki.org/browse/XWIKI-13196: this one could help a bit
I guess but not sure it's really faster to execute evocash than
looking at the code :)
* https://jira.xwiki.org/browse/XWIKI-13916: that's a more complex way
than it should to express that we call addTextAreaField for an already
existing field yes it's useful

On Fri, Apr 6, 2018 at 10:44 AM, Vincent Massol <vinc...@massol.net> wrote:
> Hi devs,
>
> Context
> ======
>
> Evocrash is a tool developed as part of the STAMP research project 
> (https://www.stamp-project.eu/). Its goal is to take a stack trace (the use 
> case is production stack traces) and generate a test that, when executed, 
> reproduces the stack trace ;)
>
> It’s using Guided Genetic Algorithm to simplify the search space (which is 
> pretty cool, isn’t it? ;)). More info at 
> https://github.com/STAMP-project/EvoCrash and http://www.evocrash.org/
>
> Results
> ======
>
> So far evocrash was able to generate tests for the following issues (I’ve 
> pasted the generated test code as comment):
> * https://jira.xwiki.org/browse/XRENDERING-422
> * https://jira.xwiki.org/browse/XWIKI-14475
> * https://jira.xwiki.org/browse/XWIKI-13031
> * https://jira.xwiki.org/browse/XWIKI-13196
> * https://jira.xwiki.org/browse/XWIKI-13916
>
> Using
> =====
>
> The use case I see is the following:
> * A user reports a problem when using XWiki and create a jira issue with a 
> stack trace
> * An XWiki dev finds it and runs evocrash to generate a test reproducing the 
> problem
> * The XWiki dev uses the test to understand the reason (breakpoints can be 
> put in the IDE) of the problem and writes a better test (which can be 
> inspired by the test generated by evocrash or be completely different).
>
> Questions
> ========
>
> The question we need to answer is whether we think it coud be useful or not. 
> Are there cases where it can be useful?
>
> For example if we imagine someone not knowing the xwiki code base well, maybe 
> this can help him/her understand the bug in a simpler way than just having to 
> read/find where the problem is in the code base?
>
> I’ve started the work of trying to evaluate how useful it could be on the 
> XWiki project by analyzing the examples above. I've only analyzed 2 ATM (see 
> the links in the jira comments) and I’d be interested to have your analysis 
> too.
>
> WDYT?
>
> Thanks
> -Vincent



-- 
Thomas Mortagne

Reply via email to