That sounds great! You will find that nearly all test suites print something, tho. I'm sure it won't be hard to change them to be mroe kind to a simple predicate like that, tho.
I like the idea of loading each file to see if it just runs without input (esp. now that we have a real bug that it would have caught). Are we worried about external effects here, tho? Also, I recently sped up test-coverage in the teaching languages by a large factor -- this speedup assumes that the code is only run in a single thread, which seems like a fine approximation for this job; perhaps something to think about eventually including? Robby On Fri, May 22, 2009 at 2:26 PM, Matthias Felleisen <matth...@ccs.neu.edu> wrote: > > I like your idea a lot. (I am sure you realize that some files need to be > loaded in mred.) Keep us posted about the progress -- Matthias > > > > > On May 22, 2009, at 3:09 PM, Jay McCarthy wrote: > >> On Fri, May 22, 2009 at 2:02 PM, Matthias Felleisen >> <matth...@ccs.neu.edu> wrote: >>> >>> The one thing I'd like to add to Eli's nightly build is some way of just >>> evaluating each file once. Period. If it doesn't load, send a message and >>> nag. Just like for the test cases. (I do have a separate test suite for >>> /htdp and /2htdp, but they aren't automated.) >> >> It seems like Eli's build is doing two things: producing the binaries >> for download and doing the automated tests linked in tests/. It makes >> sense to the first only nightly, but the second every commit. >> >> I can buy a machine and have it do the testing part basically 24/7 at >> BYU. I'll have it re-compile mzscheme only when the C code changes and >> the byte-code always. Then I'll load each file (which will include the >> tests, of course) and send blame to the most recent committer and the >> owner (via plt:responsible) of the file. >> >> I'll have a web page that shows the success of each file's loading and >> any messages spit out when it runs. At the beginning I can be testing >> library agnostic if I can assume any output to stderr is an error. >> >> I think that will be a lot better than what we have now and then we >> can experiment with better ways of communicating errors and coverage >> to my setup. For example, Eli is referring to errortrace as if it is >> the only option. I reckon if we just want to know whether some line of >> a file has been run we could get by with less information than is >> provided by errortrace. >> >> Anyways... that's my plan. Before I turn it live and you start getting >> obnoxious emails, I'll get more permission. >> >> Jay >> >> -- >> Jay McCarthy <j...@cs.byu.edu> >> Assistant Professor / Brigham Young University >> http://teammccarthy.org/jay >> >> "The glory of God is Intelligence" - D&C 93 > > _________________________________________________ > For list-related administrative tasks: > http://list.cs.brown.edu/mailman/listinfo/plt-dev > _________________________________________________ For list-related administrative tasks: http://list.cs.brown.edu/mailman/listinfo/plt-dev
