Testing on blead
We all love the CPAN testers and the results publised at http://testers.cpan.org. But recently I suddenly saw a big load of failures in one of my packages. All these failures were for 5.11.0. Now, 5.11.0 is a development version, and a moving target. A test failure with blead reveals often more about blead than about the tested package. So, on one hand, it is a good thing that packages are being tested with blead versions. On the other hand, when users want to find out if a package is usable they're only interested in released versions of perl. What are your ideas about this? Should blead test results be separated from the other results? -- Johan
Re: Testing on blead
On Mon, Jul 28, 2008 at 8:11 AM, Johan Vromans [EMAIL PROTECTED] wrote: What are your ideas about this? Should blead test results be separated from the other results? Wholeheartedly, yes. It's useful to get the reports on blead, but probably not so useful for people who just want to cruise through reports to see whether the module will work for them - they may not even usually be aware that they're looking at a blead test. -Ken
Re: Testing on blead
On 28 Jul 2008, at 15:28, Ken Williams wrote: On Mon, Jul 28, 2008 at 8:11 AM, Johan Vromans [EMAIL PROTECTED] wrote: What are your ideas about this? Should blead test results be separated from the other results? Wholeheartedly, yes. It's useful to get the reports on blead, but probably not so useful for people who just want to cruise through reports to see whether the module will work for them - they may not even usually be aware that they're looking at a blead test. I'd expect that the main audience for tests run against blead are Perl 5 core developers and the developers of the module itself. There seems to me to be little need to pollute the public test results with blead tests at all. -- Andy Armstrong, Hexten
Re: Testing on blead
Johan Vromans wrote: We all love the CPAN testers and the results publised at http://testers.cpan.org. Except when failure reports reflect problems in a particular tester's environment, and not problems with the modules being tested. (But that goes without saying, right?) But recently I suddenly saw a big load of failures in one of my packages. All these failures were for 5.11.0. Now, 5.11.0 is a development version, and a moving target. A test failure with blead reveals often more about blead than about the tested package. I routinely report tests against blead. I don't see failures often, and when I do, they usually reflect some new change in Perl that requires a subsequent change in the module. When I encounter such cases, I try to figure out a patch and then report it as bug (along with the patch) against the affected module. So, on one hand, it is a good thing that packages are being tested with blead versions. On the other hand, when users want to find out if a package is usable they're only interested in released versions of perl. What are your ideas about this? Should blead test results be separated from the other results? I agree they should be segregated. However, they should also be reported to the module's maintainers so they can fix things appropriately.
Re: Testing on blead
On Mon, Jul 28, 2008 at 03:31:17PM +0100, Andy Armstrong wrote: On 28 Jul 2008, at 15:28, Ken Williams wrote: On Mon, Jul 28, 2008 at 8:11 AM, Johan Vromans [EMAIL PROTECTED] wrote: What are your ideas about this? Should blead test results be separated from the other results? Wholeheartedly, yes. It's useful to get the reports on blead, but probably not so useful for people who just want to cruise through reports to see whether the module will work for them - they may not even usually be aware that they're looking at a blead test. I'd expect that the main audience for tests run against blead are Perl 5 core developers and the developers of the module itself. There seems to me to be little need to pollute the public test results with blead tests at all. At best publicising those results is harmless. I'm always happy to receive smoke reports against blead (and any other version except those that are below the minimum specified version) but I can't imagine them being of any use to anyone other than me, and possibly p5p when we are coming up to a release. -- Paul Johnson - [EMAIL PROTECTED] http://www.pjcj.net
