On Apr 29, 2010, at 11:38 AM, Ojan Vafai wrote:
I don't really follow the argument against supporting this test.
What is the maintenance overhead? Isn't this just like having a
layout test with expected results? It's a small isolated test
instead of testing everything. That seems like a good thing.
More importantly, it lets you be sure that every feature of the code
generator has some testing. In the real IDLs, a feature might stop
getting used temporarily and then changes to the code generator
would not be readily apparent.
I don't think your comments above are responsive to my paragraph below:
On Thu, Apr 29, 2010 at 11:05 AM, Maciej Stachowiak <m...@apple.com>
wrote:
It also strikes me as odd to do testing by doing exact comparison of
the generated source. But I can also see side benefits. I think the
real issue here may be one of naming. If the use model is that you
fully regenerate the results every time you make a change to the
bindings generator, then it's not really a regression test. The
purpose is not to catch unintentional changes but rather to be able
to observe changes to generated code, and new kinds of generated
code, while working on a change and when reviewing code. Perhaps the
tool should have a name that reflects that, instead of implying that
the purpose is to catch bugs accidentally introduced by changes. It
doesn't seem like an efficient or effective way to do the latter.
To repeat, I think this is a useful tool, but not necessarily as a test.
The mode of operation is that when you run this test, if the generated
bindings for the text IDL file change in any way, it reports a
failure. Then you must manually take the step of regenerating the
golden master file. It doesn't seem like the failure report itself
will result in a decision. It doesn't provide interesting information
until you regenerate the test file and look at the diffs, except in
the highly unusual case of changing the output of the codegen script
unintentially. Most changes to the codegen script are to intentionally
change the output.
It seems to me a better model would be to regenerate the bindings test
file automatically as part of the build. Then the changes can still be
reviewed by you, or as part of a diff, but there would be no extra
manual steps involved. And people making behaviorally transparent
changes to codegen output would perhaps feel a little less burdened.
That makes more sense to me than treating it as a regression test. It
also seems like this model would still have all the benefits you cite
above.
Regards,
Maciej
_______________________________________________
webkit-dev mailing list
webkit-dev@lists.webkit.org
http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
