On May 22, Jacob Matthews wrote:
> I don't mean to argue with you about this, but I don't really find
> these points convincing. After all, we manage to make it work even
> though I'm pretty sure we've got way more code than PLT Scheme. (To
> quote Kevin Bourrillion, Google's Java library curator: "If code
> were ice cream, we would have ... a whole lot of ice cream.")
re (a) I'm sure that you also are running things a little faster, and
utilizing a few more cpus on the way. Last time (only time) I tried
to compile all of plt in errortrace mode, it took around 2 hours;
extrapolating from the current build times, this means that the
solaris run will take more than 8 hours not counting running the code.
re (b) -- yes, you have much more code, probably enough that keeping
it all in memory is impractical anyway. This is another possible
improvement to errortrace: keep the coverage points on disk. It won't
help with the current system though, and in any case we don't have a
server farm. Not even a server garden. More like a window box in my
office and sneaking into other people's parties at night. This is
also related to my (c) point.
> I think what you're really saying is that it's impractical to test
> all of PLT Scheme with one central testing service that tries to do
> the whole thing at once. I can definitely believe that. I think the
> key might be in making the process more modular --- a continuous
> build with "all of PLT Scheme" as its target is orders of magnitude
> too big to be very useful IMO. One way to chop things up would be to
> have separate builds for different components: a mzscheme build, a
> mred build, a drscheme build (or maybe further break down DrS into
> smaller pieces and potentially several other components for
> different collects. Each sub-build runs the test suites appropriate
> to it, and just measures code coverage in the files it cares about.
> (In practice, even though a test suite written to cover, say, a mred
> component might incidentally cover some mzlib stuff, that coverage
> isn't very high-quality and probably shouldn't count as mzlib tests
> anyway.) Sub-builds can run independently on different machines,
> set up by individual groups. When I was working on planet, for
> instance, I could've set up my own sub-project just for it, and had
> the system run on my local machine. (I do effectively the same thing
> on my current project.)
Yes, all of that makes sense. Care to donate a small farm? IOW, it
will require work to make this possible -- not much work, but
currently there is no visible gains from it.
> One way to think about it is: suppose you're monitoring the the
> output of tests, and you get a message saying some tests have
> failed. Do you care? If you don't, you need to think about making
> better targets, and only monitoring the targets for which you can
> unhesitatingly say yes. This will incidentally make it a lot easier
> for smaller projects to get up and running.
That's a good point too. Had there been more tests, I'd arrange for
failure alerts to be sent to the responsible parties. Currently, I
get an email when tests fail, and I nag the responsible party.
--
((lambda (x) (x x)) (lambda (x) (x x))) Eli Barzilay:
http://www.barzilay.org/ Maze is Life!
_________________________________________________
For list-related administrative tasks:
http://list.cs.brown.edu/mailman/listinfo/plt-dev
