Hi Tim, > Are you doing automated smoke testing or reporting manual testing? > And are you using a virtual machine?
I'm reporting manual testing with CPAN::Reporter, and I'm using a real machine(which I think is the opposite of a virtual machine). > Are you saying that the tests pass when run manually using "prove", > but fail when run using "make test"? Yes. Provided that I use "make test", the tests would almost certainly fail. Worse, the failures would never uneven. Best regards, Taro Nishino On Sun, 19 Sep 2010 13:06:55 +0100 Tim Bunce <[email protected]> wrote: > > > Thank you for testing Devel-NYTProf-4.05! > > > > > > I'm puzzled by the test failures though. They don't seem to be consistent. > > > > > > Are the failures repeatable? > > > Is there sufficient disk space? > > > Is there sufficient memory? > > > The failures aren't repeatable in the least. They strongly depend upon > > much of situations when the tests are executed under at least my > > circumstance. In fact, on inspection, it is almost impossible for the > > tests to pass at a stretch by means of "make test", much less using > > CPAN.pm. In short, you can't count on the CPAN Testers reports. > > Are you doing automated smoke testing or reporting manual testing? > > And are you using a virtual machine? > > > So, I executed the dubious part of the tests separately by using "prove" > > as follows: > > > > C:\Perl\cpan\build\Devel-NYTProf-4.05-oF8sEy>prove t/test0*.t > [...] > > All tests successful. > > C:\Perl\cpan\build\Devel-NYTProf-4.05-oF8sEy>prove t/test1*.t > [...] > > All tests successful. > [...] > > > > As you can see, all the dubious parts pass. > > Are you saying that the tests pass when run manually using "prove", > but fail when run using "make test"? > > > I'm sorry to have caused you so much trouble. > > No trouble at all Taro. I'm just trying to understand if there's a > problem with NYTProf or something else, and what the causes are. > > > > Is there sufficient disk space? > > > Is there sufficient memory? > > > > The disk space is sufficient, but the memory has no more than 500M. > > While I'm not sure it's a cause of the trouble above, I feel like the > > tests themselves are too sensitive to be executed under the 500M memory. > > IMHO the test in general should be executed as small memory as possible. > > The way NYTProf tests work does run an extra perl process. > Some of the tests that failed use very little memory themselves > (e.g., t/test13.p) and certainly less that some that passed. > > 500M sounds like a reasonable amount of memory but perhaps that extra > perl process, plus the "make" process, is what's pushing the memory > usage too high. But maybe memory is not the problem, or not the only one. > > Two of the five failing tests start with "Failed to open output '...': > No error". That open happens very early in NYTProf, before the perl code > being profiled has even compiled. > > There are also several instances of "Unable to open ... for writing: > Permission denied" plus one "rename(...): Permission denied". > All very odd. > > > Finally, to be honest, I'm losing confidence in all my reports sent out > > before. > > Questions for cpantesters-discuss: > - Are there any guidelines for minimum available memory for cpan-testing? > - Could the ENVIRONMENT AND OTHER CONTEXT section show available memory? > (I presume there's some command to find it on windows.) > - Any thoughts about possible causes of the failures in this report? > > Tim.
