Barbie wrote:
Hi Apocalyse,
I'm trying to figure out the root cause for the problems, but it's
hard to dig through the data when all the reports are blacklisted and
not showing up anywhere.
You can still see your reports, as they are still on the NNTP server. If
you go to a page in Jan, e.g. [1], you'll be able to find the original
reports still. If you need them I can get the actual list of fail report
links for you (4281 of them ;)).
[1]
http://www.nntp.perl.org/group/perl.cpan.testers/2010/01/page600.html
After the modules are installed on the perls, I configure the system
perl so I can run the smoker script. Again, Chris has done an excellent
job of making POE::Component::SmokeBox and I just fire it up and let the
reports overwhelm you all! :) What this means is I have almost 15 perls
configured per VM, and once a new distribution hits PAUSE, I should send
out 60 reports for it. Seeing your preliminary statistics verified my
assumption that it was one VM that went haywire, or there would be a lot
more FAILs for each distribution.
It might be worth using a different email address for each VM. Currently
I have 'apo...@cpan.org' blacklisted. I'll lift that ban now that you've
stopped the smokers and can investigate the problem. However, if it
happens again, being able to blacklist based on an email address (or a
Metabase userid in the future), means we can reduce the impact of a
specific runaway smoker, rather than them all.
If you're able to tell me which perl/platforms were working/not working,
I can work through the database to ensure the entries for valid smokers
are unhidden from the eco-system.
Currently 18744 reports are marked as invalid, and appear to be a
mixture of all grades. Should all PASS reports be considered valid?
Cheers,
Barbie.
Hello,
Thanks again for investigating this! Ah, I thought the blacklist
applied at the email server, and my reports went down the toilet drain.
So they were processed by the system, but do not show up in the various
websites? That's great! Yes, please do and send me the 4281 FAILs so I
can run analyses on them and see if there is a common failure mode. Is
it possible for me to use the new analysis.cpantesters.org website to do
the digging around or is it approaching the problem from the wrong
perspective? It would be great if I can find a perfect correlation
across all FAILs!
After you send me the data I'll let you know which of them is "real"
and which is fake, so you can see if the DB is sane or not, thanks! I
already have a hunch, but would like to verify it first before telling
you all what an idiot I am :)
Thanks for lifting the ban, I will not be running my smokers en
masse but selectively smoke a few dists to test my changes and to try
and find the root cause. I will not be sending a blizzard of reports, at
least until after CT2.0 goes live and is stable :)
I believe all PASS reports should be fine, as what seems to be the
common problem in my setup is "missing second-level deps" which was
caused by an unknown thing. If it was missing and the dist merrily
reported a PASS, then it's the author's fault for not adding enough
tests, eh? /me grins! I'm not sure if the same logic can be applied to
N/A or UNKNOWN reports, so please leave them blacklisted until I do more
digging around, thanks.
~Apocalypse
