Hi Slaven,
Thanks for all the comments. I made myself some below:
Em 21/01/2018 19:18, Slaven Rezic escreveu:
Alceu Rodrigues de Freitas Junior via cpan-testers-discuss
<cpan-testers-discuss@perl.org> writes:
Well, I cannot agree on this one. My primary goal for using distroprefs
is to *enable* distributions to build and pass tests. Granted, I have
one ugly big file called "01.DISABLED.yml" for blocking distributions,
but this is mostly a short term scratchbook for misbehaving
distributions.
Never used distroprefs for that. For some distributions that failed due
some additional configuration requirements, I added that directly into
the VMs provisioning steps. Might consider that in the future.
(Another tool for "enabling" is the CPAN.pm plugin
CPAN::Plugin::Sysdeps, which installs system dependencies for CPAN
distributions. Nothing fancy, this is just a hand-compiled mapping
distribution -> package names.)
Looks cool... I'll make sure to take a look at it.
I guess that some of us indeed have watchdogs. A simple one is to setup
a CPU limit (my smokers use one hour). If you're lucky and the
misbehaving distribution has a busy endless loop, then the test will be
automatically killed after an hour and a proper fail report is sent out.
It's even quiet easy to recognize this, like in this report:
http://www.cpantesters.org/cpan/report/3556e864-4490-11e7-a21f-d75903976bbd
Typically, in such reports CPU usage is somewhat larger than 3600s, and
exit code of one of the tests is 9 (SIGKILL).
I'm wondering how you implemented that... for Linux there is ulimits and
cgroups, but I don't know how to do the same with OpenBSD.
However, the just hanging ones without eating CPU need a real watchdog.
I have an unpublished script which looks for inactivity on the tty. If
there's no output for 200s, then the watchdog notices the process id and
possible distribution name+version, but does not do anything. Only if
the distribution is in a list of known problematic
distributions+versions, then the watchdog actually kills the test
script. This watchdog is not really ready for the public (data & code
live in the same file, and there's only support for linux, freebsd, and
windows), but if somebody is interested, I can share.
If I may, I would suggest you to release it on Github as is and see what
comes around.
2. We cannot classify easily the distroprefs (for instance, I want to
use only distroprefs created for Linux or FreeBSD, or even for a
specific perl version) to download and use in our smokers.
Actually it's possible to restrict distroprefs documents to operating
systems and perl versions. You can specify match.perlconfig.osname and
match.perlconfig.version (and actually more, everything listed in
Config.pm may be matched against).
Got it. In fact, I was thinking about a scenario like "I want to
download all the distroprefs specific for Linux".
3. We don't have any simple way to contact the distribution author to
let him/her know their respective distribution is blocked for some
reason.
This can go the normal way: write a bug report. I made a habbit to note
the ticket number in the files I used for blocks or automatic kills.
This way I can also see if a distribution is lacking a bug report ---
sometimes the problems are many and time is sparse, and I don't write
immediately for every noticed problem a bug report.
Well, I guess time is not on my side in this one.
But it just occurred to me that the same test report could be created
and sent to the author in the case his/her distro was blocked for some
reason. That should give enough information for the author fix anything
on that matter.
5. We can't easily implement policies like blocking distributions that
don't include any test files at all (we shouldn't have to download,
unpack and check that there aren't any test files).
I don't think this is a big problem --- downloaded distributions are
usually cached, so download has to be done only once, and the time to
uselessly unpack such a distribution is probably much less than the time
to write an entry in my block file.
Indeed, but the time taken is exactly the same (but tests execution) of
a distribution that has tests included. I mean, negligible individually,
but not if all considered.
6. Not every CPAN contributor writes tests that allow execution of
tests in parallel, or even to run in parallel with another perl
executing the same tests at the same time.
This would be an interesting thing to have, but I always thought that
the specification should go into the distribution's META.* files (wasn't
there already a proposal?). And CPAN.pm may inject changes into the META
information via distroprefs files (at least this is possible for
dependency information).
I agree with that. A default value of "disabling" parallel tests should
be assumed if there is no information stating the contrary.
There are some other cases that the distribution used a huge amount of
/tmp space
Taking a lot of resources may or may not be a problem. Limited disk
space is a constant problem for me, and some distroprefs are indeed
blocking for this reason, but only on some of my machines ---
match.perlconfig.myuname is used as a "proxy" for identifying the
systems with limited resources. I think this is a problem: at which
point this should be considered a problem? Maybe only the amount of disk
space (in the build dir, in /tmp, in the final installation) should be
noted, and smokers may decide if blocking is necessary.
In the few cases (probably one or two), the whole smoker was blocked
because 100% of /tmp was taken. I could try to fix that by using disk
quota by users, but figuring out what is considered "acceptable" in
terms of disk space would be the issue.
If memory serves me well, I gave 1GB for /tmp on my smokers.
and other that created directories on CPAN build_dir
parameter with permissions that didn't allowed removal after the
tests.
This could be fixed with another cleanup script running as root. But
yes, I also know this problem.
I try to avoid running anything with root access on my smokers for
security reasons.
For that, I reviewed the distroprefs specification (from the CPAN
module POD) and there aren't fields that I think would be required to
include into the YAML:
* The OS name and version
I meant to provide the information from where the distroprefs were
created, so it could help the distro author. The same thing that the
test reports already does, basically.
I guess that even thought all this information could be recovered from
other sources, the author would be grateful to have everything at hand.
As I outlined, my perspective on distroprefs is different. What would be
interesting for me is to attach some kind of "tags" to the files so
other possibly interested parties could decide if the files are usable
for them, possibly automatically. Possible tag values would be
"private" vs. "public", "fixing" vs. "blocking", maybe others...
That could work too...
Also interesting would be some kind of database where other testers
observed hanging builds and tests --- this could be used for watchdog
implementations. This could be a distroprefs-like system or something
completely different.
Indeed! Let's see if I can get my hand over a watchdog first! * hint hint *
Regards,
Alceu
