I would *love* a way for the cpanminus (cpanm) client to be able to use
distroprefs. The #1 perl issue plaguing me at work right now is being able
to control how each module in our dependency chain gets installed -- e.g.
some of them need to skip tests (because of various failures), some of them
need to be installed with PERL_USE_UNSAFE_INC=1, some of them need develop
prereqs installed as well, etc. etc....

Also, I think that the distroprefs feature is underdocumented in general --
I've found that few people are aware of its existence. If it was used more
widely, I suspect there would be better toolchain support for it.

Pursuing this subject further is on my todo-list at QAH-Oslo this April. It
would be awesome if others were interested as well, and we could figure out
a list of action items and set a plan in motion.

-ether


On Sun, Jan 21, 2018 at 1:18 PM, Slaven Rezic <sla...@rezic.de> wrote:

> Alceu Rodrigues de Freitas Junior via cpan-testers-discuss
> <cpan-testers-discuss@perl.org> writes:
>
> > Hello folks,
> >
> > In the past I already sent this for discussion, but didn't get much
> > attention. I hope I can get more feedback this time.
> >
> > We all create distroprefs to block distributions to run on our
> > smokers.
>
> Well, I cannot agree on this one. My primary goal for using distroprefs
> is to *enable* distributions to build and pass tests. Granted, I have
> one ugly big file called "01.DISABLED.yml" for blocking distributions,
> but this is mostly a short term scratchbook for misbehaving
> distributions.
>
> To enable distributions means that most of my distroprefs files try to
> set some knobs to make a distribution build & pass its tests. I do this
> for distributions which I need personally (my distroprefs files are not
> a smoker-only thing), or distributions which have reverse dependencies,
> so I am able to smoke these, too (especially important in times of
> larger upstream breakages, e.g. incompatible bleadperl changes).
>
> (Another tool for "enabling" is the CPAN.pm plugin
> CPAN::Plugin::Sysdeps, which installs system dependencies for CPAN
> distributions. Nothing fancy, this is just a hand-compiled mapping
> distribution -> package names.)
>
> > Although we can always publish those on something like
> > Github, it is not simple to keep sync with them.
> >
> > That's bad for several reasons:
> >
> > 1. We are not taking advantage of each other work of smoking test
> >    distributions. It took me several months of work to go over the ~37K
> >    distributions available on CPAN. This is a huge amount of resources
> >    wasted considering that we don't have a "watchdog" or something like
> >    to kill tests that were running for more than a hour, for instance.
> >    It is almost funny that for many days I left the Smoker from running
> >    unattended after hours just to find out the tests stopped after 15
> >    minutes I left.
>
> I guess that some of us indeed have watchdogs. A simple one is to setup
> a CPU limit (my smokers use one hour). If you're lucky and the
> misbehaving distribution has a busy endless loop, then the test will be
> automatically killed after an hour and a proper fail report is sent out.
> It's even quiet easy to recognize this, like in this report:
> http://www.cpantesters.org/cpan/report/3556e864-4490-
> 11e7-a21f-d75903976bbd
> Typically, in such reports CPU usage is somewhat larger than 3600s, and
> exit code of one of the tests is 9 (SIGKILL).
>
> However, the just hanging ones without eating CPU need a real watchdog.
> I have an unpublished script which looks for inactivity on the tty. If
> there's no output for 200s, then the watchdog notices the process id and
> possible distribution name+version, but does not do anything. Only if
> the distribution is in a list of known problematic
> distributions+versions, then the watchdog actually kills the test
> script. This watchdog is not really ready for the public (data & code
> live in the same file, and there's only support for linux, freebsd, and
> windows), but if somebody is interested, I can share.
>
> To be killed by a watchdog means that typically a fail report is
> generated, which is quite OK. This way the author has a chance to see
> that something is problematic.
>
> As this watchdog does not really notifies me, I have another one which
> checks if a smoker had created recently a report, and if there was none
> for 2h (and the recent page on MetaCPAN says that there are newer
> distributions available), then I get an error message on my display.
>
> > 2. We cannot classify easily the distroprefs (for instance, I want to
> >    use only distroprefs created for Linux or FreeBSD, or even for a
> >    specific perl version) to download and use in our smokers.
>
> Actually it's possible to restrict distroprefs documents to operating
> systems and perl versions. You can specify match.perlconfig.osname and
> match.perlconfig.version (and actually more, everything listed in
> Config.pm may be matched against).
>
> > 3. We don't have any simple way to contact the distribution author to
> >    let him/her know their respective distribution is blocked for some
> >    reason.
>
> This can go the normal way: write a bug report. I made a habbit to note
> the ticket number in the files I used for blocks or automatic kills.
> This way I can also see if a distribution is lacking a bug report ---
> sometimes the problems are many and time is sparse, and I don't write
> immediately for every noticed problem a bug report.
>
> > 4. Kwalitee reports don't take into consideration that a distribution
> >    is blocked.
>
> No opinion on this one.
>
> > 5. We can't easily implement policies like blocking distributions that
> >    don't include any test files at all (we shouldn't have to download,
> >    unpack and check that there aren't any test files).
>
> I don't think this is a big problem --- downloaded distributions are
> usually cached, so download has to be done only once, and the time to
> uselessly unpack such a distribution is probably much less than the time
> to write an entry in my block file.
>
> > 6. Not every CPAN contributor writes tests that allow execution of
> >    tests in parallel, or even to run in parallel with another perl
> >    executing the same tests at the same time.
>
> This would be an interesting thing to have, but I always thought that
> the specification should go into the distribution's META.* files (wasn't
> there already a proposal?). And CPAN.pm may inject changes into the META
> information via distroprefs files (at least this is possible for
> dependency information).
>
> > 7. Not every CPAN contributor writes tests that skips installation when
> >    there aren't requirements in place like a specific OS. Why would we
> >    want to smoke test a distribution that will only run on Win32 from a
> >    UNIX-like based smoker?
>
> Actually I have in 01.DISABLED.yml some YAML documents for blocking
> distributions on alien operating systems, but these were mostly created
> in the early days of this file. Nowadays most authors do a check and
> emit an "OS unsupported" notice if needed. This reduces this problem to
> point 5 --- the distribution is needlessly downloaded und unpacked, but
> not a big problem for me.
>
> >
> > Due my experiments with OpenBSD running a CPAN::Reporter::Smoker, I
> > was able to fetch a set of 183 distroprefs and the vast majority of
> > them have the following comment:
> >
> > comment: Tests hang smoker
> >
> > This "default" comment is due a limitation of the program dblock (see
> > CPAN-Reporter-Smoker-OpenBSD) that I made available on CPAN. For some
> > cases, I tried to add something more meaningful:
> >
> >     GRAVATTJ.Backup-EZ.yml:
> >
> >     comment: Requires root access to test/install
> >
> > But at the end I did it manually. That could be easily fixed adding a
> > command line option, but that's just an example. Even if I implement
> > that, the author will not get anything automatically.
> >
> > There are some other cases that the distribution used a huge amount of
> > /tmp space
>
> Taking a lot of resources may or may not be a problem. Limited disk
> space is a constant problem for me, and some distroprefs are indeed
> blocking for this reason, but only on some of my machines ---
> match.perlconfig.myuname is used as a "proxy" for identifying the
> systems with limited resources. I think this is a problem: at which
> point this should be considered a problem? Maybe only the amount of disk
> space (in the build dir, in /tmp, in the final installation) should be
> noted, and smokers may decide if blocking is necessary.
>
> > and other that created directories on CPAN build_dir
> > parameter with permissions that didn't allowed removal after the
> > tests.
>
> This could be fixed with another cleanup script running as root. But
> yes, I also know this problem.
>
> >
> > I'm considering starting a project to share this kind of information
> > from a website with the following features:
> >
> > 1. Front end to allow a developer to search for anything blocked based
> >    on this CPAN ID.
> > 2. REST interface to allow the same search (anonymous access) and
> >    submission of distroprefs (authenticated) in JSON format.
> > 3. Send and e-mail to the author when something is received for his/her
> >    distributions.
> >
> > For that, I reviewed the distroprefs specification (from the CPAN
> > module POD) and there aren't fields that I think would be required to
> > include into the YAML:
> >
> >  * The OS name and version
>
> Which is already there, at least as regexp matches:
> match.perlconfig.osname (or archname) and match.perlconfig.osvers.
>
> >  * The reporter ID
>
> Theoretically may be found via the source of the distroprefs file (i.e.
> git repository).
>
> >  * Reason (we might define some standard values)
> >  * perl details
>
> This is also possible: match.perlconfig.version, and everything else
> found in Config.pm
>
> >  * creation date of the distroprefs
>
> Theoretically may be found in the git log.
>
> >  * the version of the distribution that was being tested when the
> >    distroprefs was created
>
> It's already possible to specify a distribution match with the version,
> at least using a regexp. My (conservative) approach is to never use
> matches without versions, unless it's clear that later distribution
> releases won't change anything about the problem.
>
> >
> > I guess I could declare multiple "comment" entries with that, but that
> > would be a hack at best.
> >
> > Since I'm not sure if anybody is working on something like that, I'm
> > hoping to get some feedback like suggestions, complains and maybe, who
> > knows, some code. :-)
>
> As I outlined, my perspective on distroprefs is different. What would be
> interesting for me is to attach some kind of "tags" to the files so
> other possibly interested parties could decide if the files are usable
> for them, possibly automatically. Possible tag values would be
> "private" vs. "public", "fixing" vs. "blocking", maybe others...
>
> Also interesting would be some kind of database where other testers
> observed hanging builds and tests --- this could be used for watchdog
> implementations. This could be a distroprefs-like system or something
> completely different.
>
> > Thanks!
> >
> > Alceu
> >
>
> Regards,
>     Slaven
>
> --
> Slaven Rezic - slaven <at> rezic <dot> de
>   BBBike - route planner for cyclists in Berlin
>   WWW version:                           http://www.bbbike.de
>   Perl/Tk version for Unix and Windows:  http://bbbike.sourceforge.net
>
  • sharing distrop... Alceu Rodrigues de Freitas Junior via cpan-testers-discuss
    • Re: sharin... Slaven Rezic
      • Re: sh... Karen Etheridge
        • Re... Alceu Rodrigues de Freitas Junior via cpan-testers-discuss
      • Re: sh... Andreas Koenig
        • Re... Karen Etheridge
        • Re... Alceu Rodrigues de Freitas Junior via cpan-testers-discuss
        • Re... Slaven Rezic
      • Re: sh... Alceu Rodrigues de Freitas Junior via cpan-testers-discuss

Reply via email to