Re: Possible test framework improvement

2012-06-13 Thread Sami Siren
hmm, actually the html report (and generate-test-reports) displays this information nicely so i don't know if this is necessary. On Wed, Jun 13, 2012 at 9:55 AM, Sami Siren ssi...@gmail.com wrote: Hi, While looking at some of the test failures it occurred to me that it would be great to have

Re: Possible test framework improvement

2012-06-13 Thread Dawid Weiss
  [junit4] Suite: org.apache.solr.analysis.TestKeepFilterFactory   [junit4] Completed on J0 in 0.22s, 1 test If that also had a time stamp when the test started it would be, in some cases, helpful to see what other tests were running at the same The event log contains this information, it's

Re: Possible test framework improvement

2012-06-13 Thread Sami Siren
On Wed, Jun 13, 2012 at 10:45 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote:   [junit4] Suite: org.apache.solr.analysis.TestKeepFilterFactory   [junit4] Completed on J0 in 0.22s, 1 test If that also had a time stamp when the test started it would be, in some cases, helpful to see what

Re: Possible test framework improvement

2012-06-13 Thread Sami Siren
On Wed, Jun 13, 2012 at 11:41 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: So how about if something similar was printed from different aspect: NOTE: All tests that were running at the same time: [TestA, TestB, TestC] I think this would be noisy -- if you have multiple jvms it's normal

Re: Possible test framework improvement

2012-06-13 Thread Dawid Weiss
This would be visible only for failed tests (no point in printing it for anything else) Yes, but it'd require higher-level analysis (cross-jvm). Another element of difficulty is that I also had a plan to distribute suites physically to different machines at some point (and the overlap wouldn't

RE: Possible test framework improvement

2012-06-13 Thread Uwe Schindler
Message- From: dawid.we...@gmail.com [mailto:dawid.we...@gmail.com] On Behalf Of Dawid Weiss Sent: Wednesday, June 13, 2012 11:11 AM To: dev@lucene.apache.org Subject: Re: Possible test framework improvement This would be visible only for failed tests (no point in printing it for anything

Re: Possible test framework improvement

2012-06-13 Thread Robert Muir
: dawid.we...@gmail.com [mailto:dawid.we...@gmail.com] On Behalf Of Dawid Weiss Sent: Wednesday, June 13, 2012 11:11 AM To: dev@lucene.apache.org Subject: Re: Possible test framework improvement This would be visible only for failed tests (no point in printing it for anything else) Yes

Re: Possible test framework improvement

2012-06-13 Thread Dawid Weiss
So I think the only likely possibilities for test interference across JVMS are things like using same temp directory outside of the source tree or under some svn-ignore'd part of the source tree, or using the same network port numbers, etc. Temp directories shouldn't be a problem if they're

Re: Possible test framework improvement

2012-06-13 Thread Robert Muir
On Wed, Jun 13, 2012 at 10:13 AM, Dawid Weiss dawid.we...@cs.put.poznan.pl wrote: So I think the only likely possibilities for test interference across JVMS are things like using same temp directory outside of the source tree or under some svn-ignore'd part of the source tree, or using the

Re: Possible test framework improvement

2012-06-13 Thread Dawid Weiss
foo-dependency-1.0.jar does stuff with java.io.tmpdir it could create an issue correct? or does the test runner also set this sysprop to the sandbox dir for slave jvms (if it doesn't already but can, that would be very cool, more safety). True, we could alter java.io.tmpdir. There is actually

RE: Possible test framework improvement

2012-06-13 Thread Chris Hostetter
: In my opinion, the separate JVMs should not produce test failures or : affect each other, because every JVM gets its own temporary directory : for running tests and creating indexes. I don't think anyone would disagree with that opinion -- but having a common opinion doesn't magically make

Re: Possible test framework improvement

2012-06-13 Thread Dawid Weiss
ie: tests in seperate JVMs shouldn't affect eachother, but in spite of best intentions they might, and having timestamps would help discover that. They do -- look at the html report, it contains timestamps. I'll add them to the txt report in the next version. Dawid