On Jun 8, 2012, at 1:04 PM, Ryosuke Niwa wrote: > On Fri, Jun 8, 2012 at 12:50 PM, Filip Pizlo <[email protected]> wrote: > On Jun 8, 2012, at 12:31 PM, Dirk Pranke wrote: > > On Fri, Jun 8, 2012 at 10:56 AM, Filip Pizlo <[email protected]> wrote: > >> > >> It's a lot harder to dive into, a lot more cumbersome to improve, and not > >> any easier to maintain. > >> > > > > I definitely agree that NRWT is more complicated than it seems like it > > should be; it got contorted as we added all the features we needed to > > add, and I have been in a "simplify" mode over the past few months. I > > would welcome any feedback where you think things are overly complex. > > This is a difficult question - it's unfortunately easier to observe that > something is complex, than it is to pinpoint why it is complex. But I will > try. > > 1) Code locality. I can open Tools/Scripts/old-run-webkit-tests and pretty > rapidly discover (a) how options are parsed, (b) how platform differences are > handled, (c) how tests are found, and (d) how tests are run. I can hack all > of this code because it's all in one place. I don't have to be a domain > expert to do it. Hell, I don't even have to be good a Perl to find my way > around. > > 2) Code size: > > [pizlo@wartooth OpenSource] wc Tools/Scripts/old-run-webkit-tests > 2796 10316 98733 Tools/Scripts/old-run-webkit-tests > > [pizlo@wartooth OpenSource] wc Tools/Scripts/new-run-webkit-tests > Tools/Scripts/webkitpy/layout_tests/*.py > Tools/Scripts/webkitpy/layout_tests/*/*.py > .... bunch of stuff > 23197 91897 1049914 total > > That's a *HUGE* difference. Consider that NRWT just adds only one thing that > most people care about: parallelism. Is an 8x increase in code size > justified? > > I know that LoC metrics are evil in most cases. But this is not most cases. > This is an order-of-magnitude difference. That's 8x more code I have to look > at to find what I want. That's 8x more code that I potentially have to edit > to add a feature. That's 8x more code that could have a bug. And so on. > Badness! > > Part of that comes from the fact layout_tests directory also includes port > classes, which knows various aspects of each port, and they're also used in > webkit-patch and other Python tools. > > But the act of running layout tests is in itself a regression test of the > layout test harness just as much as it is a regression test of WebKit. > > So the unit tests are superfluous. In particular, if I had to pick between > only having unit tests or only having regression tests, I might pick unit > tests. But if I already have regression tests then I'm unlikely to want to > incur technical debt to build unit tests, particularly since unit tests > requiring changing the infrastructure to make the code more testable, which > then leads to the problems listed above. > > I agree that we've accumulated way too many unit tests in webkitpy and some > of our unit test code is hideous but I think it's quite unrealistic not to > have any unit tests. We've had many regressions in the past, and it's hard to > make sure your code works on all platforms without tests. So it's the balance > that's important.
I think you've had regressions because the code is too complex. > > I am intrigued by the notion of using Erlang, but worry that it would reduce > effective hackability due to there being less Erlang experience in the > universe. I also don't want the porting of Erlang's runtime to be a gating > factor for porting WebKit itself. Hence, I fear that we should stick to > broadly accepted languages, like Python, or Perl, or if need be, C++. > > If I were to write a new test runner, I'd use C++ without a question. It has > solid threading support in almost all operating systems, is statically typed, > and virtually everyone who contributes to WebKit can read/write well. I would try to avoid using C++ unless we really had to. The goal here is simplicity, and C++ is not the best for that. > > - Ryosuke >
_______________________________________________ webkit-dev mailing list [email protected] http://lists.webkit.org/mailman/listinfo.cgi/webkit-dev
