On Wed, Nov 25, 2009 at 8:29 PM, Erick Erickson <erickerick...@gmail.com> wrote: > IMHO there are other reasons to upgrade to junit4 besides > parallelization, there are some nice new capabilities. I > suppose the analogous question is "why upgrade to > Lucene 2.9?"
OK then I think we should do the upgrade, and separately think about how to take advantage of parallelization... I applied the patch for 1084, and also carried it to the back-compat branch, and upgraded the junit jar in the lib dir to 4.7, and all tests run fine. I also separately tested 2037, and it ran fine as well. > Especially since it's not a matter of upgrading. Junit3 tests run > just fine under junit4. I've tested after removing the > junit3 jar from lib, no problem. It even seems to run > slightly faster, which makes me wonder... > > So really, we have the best of both worlds. No work involved in > using Junit4 with the current tests, but the ability to use the > new features of Junit4. Although I'm sure there'll be > *something* that bites us, I have great faith in Murphy. > > Kinda reminds me of the Lucene drop-in replacement policy > <G>....... Heh ;) > But on the topic of parallelization: I'm not at all sure > it's worth the effort. As far as I can tell, it really only gets significant > gains when you have more cores to run with. It's not at all clear > to me how much time we spend doing I/O in the tests... very little > I suspect (although I confess I don't know for sure). And if we're > CPU bound anyway, parallelization doesn't help. Anybody know > for sure? > > And say we did all the work to parallelize all the tests. And say that > instead of taking 25 minutes on my 3 year old MacBook Pro, we > got it down to 10 minutes. Who cares? 10 minutes is still too long > according to the eXtreme Programming (XP) folks, and I sympathize > with their point of view. Even though I did spend some time trying > to trim some time. If you have enough cores, you can get decent speedups. I have a scary python wrapper that runs the tests in parallel (runs 5 threads on an 8 core machine... any more threads seems not to help), by letting each thread run a separate package, and sub-dividing the big (index, search) packages w/ a temporary patch. Still it takes ~6.5 minutes to run the tests on my machine these days, which is still too long for interactive testing... > The XP approach to unit testing is to run it almost every time you > change a line of code. OK, I'm exaggerating, but not by too > much with the die-hard XP folks. Even at 10 minutes, we can't > do that. > > So, I think the value for Lucene/SOLR comes *not* from running the > tests 15 times an hour. I think the real value comes from not > letting errors hide for days/weeks/months/releases. So I'm quite willing > to let the automated builds catch the unit test failures in unexpected > places in those instances where I don't run all of the tests before > a patch is committed. As long as we fix them as soon as they're > found. Kinda like what's the point of a cell phone having better battery life when it's already plenty for my daily usage yet not enough for 2 days usage, ie, I still have to charge it up, nightly.... it's quantized. Slightly faster tests won't really make a fundamental change in our development approach. But, still, while this speedup won't make the tests fully interactive, it's still an important improvement. I run the unit tests an insane number times, and bringing the time down even "just" by 2 minutes will make a real difference. > OK, I'm rambling. I'm off for Thanksgiving, and my daughter is > at her in-laws until tomorrow (they're visiting from CA). So sue > me <G>. Happy Thanksgiving!! Mike --------------------------------------------------------------------- To unsubscribe, e-mail: java-dev-unsubscr...@lucene.apache.org For additional commands, e-mail: java-dev-h...@lucene.apache.org
