from:"James E Keenan"

For the Perl Toolchain Summit Agenda

2024-04-22 Thread James E Keenan

By posting here I am hoping to attract the attention of people who will 
be attending the Perl Toolchain Summit 
(https://perltoolchainsummit.org/pts2024/) to be held in Lisbon from 
April 25 to 28.  (I originally posted this message to perl5-porters on 
March 28.)  We are experiencing problems with infrastructure that plays 
a vital role in our ongoing work on the Perl core distribution -- 
problems which may benefit from the attention of PTS attendees.


I. CPANtesters not reporting names of distributions tested

I originally reported this problem on the cpan-testers-discuss mailing 
list/newsgroup on January 9 2024 
(https://www.nntp.perl.org/group/perl.cpan.testers.discuss/2024/01/msg4599.html). 
 New versions of certain distributions have been uploaded to CPAN and 
have been tested by various CPANtesters rigs -- but the names of those 
distributions are not being logged and, as an apparent consequence, no 
reports for those versions are being presented in the web interfaces.


Case in point:  In my original posting I reported that Net-SSLeay-1.94 
was uploaded on January 8 (which was off by one day; the upload was on 
January 7) but no page displaying results for that version had yet to 
appear either at the matrix 
(http://matrix.cpantesters.org/?dist=Net-SSLeay) or the fast-matrix 
(http://fast-matrix.cpantesters.org/?dist=Net-SSLeay).  On both pages the

*displayed* version is 1.93_05, but you can also see:

#
Other versions
NOTE: no report for latest version 1.94
#

So the system is aware that 1.94 has been released to CPAN but believes 
that no reports have yet been uploaded.  That means that we at Perl 5 
Porters have no way of knowing whether changes we have been making in 
the core distribution may have broken Net-SSLeay -- a distribution for 
which there has even been discussion of shipping it with core.


This problem persists.  Another CPAN distribution whose latest version 
is not being reported is MIME-tools-5.514, uploaded to CPAN on February 
6.  The most recent version reported on at the matrix or the fast-matrix 
(http://fast-matrix.cpantesters.org/?dist=MIME-tools) is 5.513, the last 
report for which was received on February 6.


Slaven Rezić reported this problem as an issue in the 
cpantesters-backend bug queue on February 21 
(https://github.com/cpan-testers/cpantesters-backend/issues/27).  He 
suggested a possible diagnosis and correction.  While there has been 
some discussion in that ticket, it does not appear to have attracted the 
attention of people who to my knowledge were involved with the 
CPANtesters infrastructure in the past.


Implication:  Blead may be "breaking" CPAN, but, less than a month going 
into our next production release, we are not finding out about it.


II.  Delayed mail forwarding through cpan.org

On February 10 I reported to P5P 
(https://www.nntp.perl.org/group/perl.perl5.porters/2024/02/msg267842.html) 
that I was experiencing delays in receiving email notifications about 
postings being made to our GitHub issues and pull request queues.  At 
the time I speculated that the problem might have been on GH's side. 
This was incorrect.  Additional data and discussion with the perl.org 
system administrators led the admins to post 
(https://log.perl.org/2024/02/cpanorg-email-deliverability-issues.html) 
this was affecting all cpan.org email addresses, was a result of some 
backend changes and that "addressing this issue will require significant 
technical adjustments."


My hunch is that these problems are as much human (insufficient 
volunteer resources; recruiting the next generation of people to 
maintain Perl's infrastructure) as they are technical.  I hope they can 
be placed on the agenda for the PTS.


Thank you very much.
Jim Keenan

Re: Could not upload tarball to PAUSE from server

2023-10-16 Thread James E Keenan


On 10/8/23 18:38, Tony Cook wrote:

On Sun, Oct 08, 2023 at 09:59:10AM -0400, James E Keenan wrote:

It might be the host performing the fetch is missing the root
certificate needed for your LetsEncrypt certificate, but ssltest is
the place to start.



Is there any indication in the data above to support that hypothesis? If
not, where do we go from here?


Not that I can see.

I suspect this is something Andreas would need to resolve.

It's possible it was a temporary problem, so you should try again
before contacting Andreas.



I did try again, with the same results.  I then contacted Andreas, who 
said that (a) this is a known problem, i.e., PAUSE is currently unable 
to fetch tarballs for upload from https://; (b) it won't be fixed until 
the new PAUSE server gets installed.

Re: Could not upload tarball to PAUSE from server

2023-10-08 Thread James E Keenan

On 10/7/23 23:33, Tony Cook wrote:
> On Sat, Oct 07, 2023 at 10:37:36PM -0400, James E Keenan wrote:
>> The only thing which I think is different from my last CPAN upload 
is that
>> about a month ago I upgraded my server from http:// to https://.  I 
have not
>> encountered any problems with the server since then.  The file 
permissions

>> on the tarballs I was trying to upload are 0644 -- same as all the other
>> dozens of tarballs I've uploaded from that server.
>>
>> Any ideas as to why I could not upload from my server (for the first 
time in

>> 18 years!)?
>
> For some reason it can't verify the certificate:
>
> 2023-10-08 02:47:59 $$1354 v1049: Alert: nosuccesscount[10] 
error[Can't connect to thenceforward.net:443 (certificate verify 
failed)] (paused:708)

>
> You might try testing your site with:
>
> https://www.ssllabs.com/ssltest/
>
> which will detect problems that might not show in a browser.

When I switched from http:// to https:// in late September, I performed 
that ssltest on each of the three hosts I run off this machine/IP 
address.  Two of the three hosts were graded 'A'; one (which is not as 
important) was graded 'A' but only for IPv4.

https://www.ssllabs.com/ssltest/analyze.html?d=thenceforward.net  A
https://www.ssllabs.com/ssltest/analyze.html?d=jamesekeenan.com   A
https://www.ssllabs.com/ssltest/analyze.html?d=lerner-minsky.org  A only 
on ipv4

I re-performed these tests this morning.  In my first pass for 
thenceforward.net (which is the hostname PAUSE would have been looking 
for), the test hung indefinitely showing an ever spinning spike-wheel 
and "Please wait...Testing NPN"

My first pass for jamesekeenan.com got farther in the process.  After 
about 10 minutes it was still at:  "Please wait... 95% complete 
Simulating handshakes".  The process then appeared to hang indefinitely.

I hit the Clear Cache process and started anew testing 
thenceforward.net.  This time the process gave me an Overall Rating of 
'A' (better than google.com!) along with a wealth of other data, most of 
which did not seem anomalous.  In what follows I'm only pointing out 
those anomalies:

#
Certificate #1: EC 256 bits (SHA256withRSA): Only thing which looks 
anomalous is: "DNS CAA  No (more info)"

Subject thenceforward.net
Fingerprint SHA256: 
d71a64c0cb54787620cf4341ae28041b7e1b1d173785443337bbb2eb4d2923cb

Pin SHA256: P5vJZqpxQIYZPF3D8iMCtr5q/3eE6XlQyzK/IWnm60U=
Common namesthenceforward.net
Alternative names   thenceforward.net www.thenceforward.net
Serial Number   041ea3bd1884e0f09546ccd19e8b061c6588

Additional certificates (if supplied):  R3 no anomalies.  ISRG Root X1 
no anomalies.

Certification Paths:

Mozilla / Apple / Java

Path #1: Trusted: no anomalies

Path #2: Not trusted (path does not chain to a trusted anchor):  DST 
Root CA X3   Self-signed
* This is described as Extra download;  Not in trust store. RSA 2048 
bits (e 65537) / SHA1withRSA

Valid until: Thu, 30 Sep 2021 14:01:15 UTC
EXPIRED
Weak or insecure signature, but no impact on root certificate

Android / Windows

RSA 2048 bits is in trust store but has expired

Certificate #2: EC 256 bits (SHA256withRSA) No SNI

Subject jamesekeenan.com
Fingerprint SHA256: 
e64efa87924731fb4e7e1e893e61b46e1745299f249d5186c6dfe638f6d75df3

Pin SHA256: OonVyaTHdqBFC4MU4xN3fEko/1BdMoqGNswfbXx8DGI=
Common namesjamesekeenan.com
Alternative names   jamesekeenan.com www.jamesekeenan.com   MISMATCH
#   

>
> It might be the host performing the fetch is missing the root
> certificate needed for your LetsEncrypt certificate, but ssltest is
> the place to start.
>

Is there any indication in the data above to support that hypothesis? 
If not, where do we go from here?

> Tony

Thank you very much for taking the time to review this problem.

Jim Keenan

Could not upload tarball to PAUSE from server

2023-10-07 Thread James E Keenan

Tonight I attempted to upload to PAUSE a trial version of a CPAN 
distribution I maintain.  I built and tested a tarball on my desktop, 
then scp-ed the tarball to my Linode server.  I then signed in to PAUSE 
and put the URL to the tarball in the appropriate box.  I usually expect 
to get an upload notification within minutes and, soon thereafter, see 
it entering the metabase log file.  I have done CPAN uploads like this 
many dozens of times.


Tonight, however, I did not get any upload notification email and did 
not see the file entering metabase.  Instead, after several hours, I got 
an email from "PAUSE " with this content:


#
The URL 
https://thenceforward.net/perl/modules/Devel-NYTProf/Devel-NYTProf-6.12_001.tar.gz, 
requested for upload as J/JK/JKEENAN/Devel-NYTProf-6.12_001.tar.gz has 
problems[.] I have retried to fetch it 8 times to no avail. I'll 
continue to try until the maximum of 16

retries is reached. Then I'll give up to give room for a new trial.
#

I incremented $VERSION and tried again.  This did not appear to work.

I incremented $VERSION again and asked PAUSE to fetch the tarball 
directly from my desktop -- i.e., not from my server.  This succeeded.


The only thing which I think is different from my last CPAN upload is 
that about a month ago I upgraded my server from http:// to https://.  I 
have not encountered any problems with the server since then.  The file 
permissions on the tarballs I was trying to upload are 0644 -- same as 
all the other dozens of tarballs I've uploaded from that server.


Any ideas as to why I could not upload from my server (for the first 
time in 18 years!)?


Thank you very much.
Jim Keenan

www.cpan.org cert expired?

2022-03-12 Thread James E Keenan

When attempting to download tarballs from CPAN, I am getting the 
following errors:


$ wget https://www.cpan.org/authors/id/A/AN/ANDK/CPAN-2.33-TRIAL.tar.gz
--2022-03-12 17:58:01-- 
https://www.cpan.org/authors/id/A/AN/ANDK/CPAN-2.33-TRIAL.tar.gz

Resolving www.cpan.org (www.cpan.org)... 2a04:4e42:c::644, 151.101.50.132
Connecting to www.cpan.org (www.cpan.org)|2a04:4e42:c::644|:443... 
connected.

ERROR: The certificate of ‘www.cpan.org’ is not trusted.
ERROR: The certificate of ‘www.cpan.org’ has expired.

$ wget 
https://www.cpan.org/authors/id/J/JK/JKEENAN/ExtUtils-ModuleMaker-0.63.tar.gz
--2022-03-12 17:59:03-- 
https://www.cpan.org/authors/id/J/JK/JKEENAN/ExtUtils-ModuleMaker-0.63.tar.gz

Resolving www.cpan.org (www.cpan.org)... 2a04:4e42:c::644, 151.101.50.132
Connecting to www.cpan.org (www.cpan.org)|2a04:4e42:c::644|:443... 
connected.

ERROR: The certificate of ‘www.cpan.org’ is not trusted.
ERROR: The certificate of ‘www.cpan.org’ has expired.

I sent email to 'webmas...@cpan.org', which was rejected.  I then sent 
email to 'webmas...@perl.org'.


Does anyone else have suggestions?

Thank you very much.
Jim Keenan

Re: Unable to install via 'cpan' due to 'cpan_path' missing from CHECKSUMS

2022-03-01 Thread James E Keenan

On 3/1/22 13:48, Andreas Koenig wrote:

On Tue, 1 Mar 2022 10:23:12 -0500, James E Keenan  said:

   > Warning: checksum file
   > '/root/.cpan/sources/authors/id/N/NE/NEILB/CHECKSUMS' not conforming.

Please upgrade to CPAN-2.33-TRIAL.tar.gz which you find on CPAN in my
directory https://www.cpan.org/authors/id/A/AN/ANDK/

Thanks, that appears to work.  I see that a version 2.33 has already 
been synched into Perl 5 blead.  Do you have a timeline for a separate 
release of 2.33 to CPAN?

Unable to install via 'cpan' due to 'cpan_path' missing from CHECKSUMS

2022-03-01 Thread James E Keenan

On a machine where I have root privileges, I wish to test (then later 
install) Carp::Assert using the `cpan` client.  I encounter this failure:


#
$ sudo cpan -t Carp::Assert
Password:
Loading internal logger. Log::Log4perl recommended for better logging
Reading '/root/.cpan/Metadata'
  Database was generated on Tue, 01 Mar 2022 13:17:03 GMT
Running test for module 'Carp::Assert'
CPAN: Digest::SHA loaded ok (v6.02)

Warning: checksum file 
'/root/.cpan/sources/authors/id/N/NE/NEILB/CHECKSUMS' not conforming.


The cksum does not contain the key 'cpan_path' for 
'Carp-Assert-0.21.tar.gz'.

Proceed nonetheless? [no] no
Aborted.
#

When I examine the CHECKSUMS files beneath 
/root/.cpan/sources/authors/id, I see that only a handful of them have 
the string 'cpan_path' in them.


#
$ find /root/.cpan/sources/authors/id -type f -name 'CHECKSUMS' | xargs 
grep -l cpan_path |sort | xargs ls -l
-rw-r--r--  1 root  wheel   33696 Nov 23 20:30 
/root/.cpan/sources/authors/id/A/AN/ANDK/CHECKSUMS
-rw-r--r--  1 root  wheel   29147 Mar  1 14:48 
/root/.cpan/sources/authors/id/A/AR/ARC/CHECKSUMS
-rw-r--r--  1 root  wheel   22123 Mar  1 14:48 
/root/.cpan/sources/authors/id/A/AS/ASB/CHECKSUMS
-rw-r--r--  1 root  wheel   57924 Feb 20 23:08 
/root/.cpan/sources/authors/id/E/EX/EXODIST/CHECKSUMS
-rw-r--r--  1 root  wheel9931 Mar  1 14:48 
/root/.cpan/sources/authors/id/M/MI/MICKEY/CHECKSUMS
-rw-r--r--  1 root  wheel   38818 Feb 20 23:08 
/root/.cpan/sources/authors/id/N/NE/NEZUMI/CHECKSUMS
-rw-r--r--  1 root  wheel  310938 Mar  1 14:48 
/root/.cpan/sources/authors/id/O/OA/OALDERS/CHECKSUMS
-rw-r--r--  1 root  wheel  154626 Dec  1 18:12 
/root/.cpan/sources/authors/id/R/RU/RURBAN/CHECKSUMS
-rw-r--r--  1 root  wheel  200948 Dec  1 18:07 
/root/.cpan/sources/authors/id/Y/YV/YVES/CHECKSUMS

#

This means that I would be unable to install even my own CPAN modules!

I believe I have followed all the guidance provided by Neil Bowers in 
his blog post 
http://blogs.perl.org/users/neilb/2021/11/addressing-cpan-vulnerabilities-related-to-checksums.html. 
 So I would like to know how to update these CHECKSUMS files or 
otherwise cope with this problem.


Thank you very much.
Jim Keenan

Should URL in 02packages.details.txt be updated?

2022-01-15 Thread James E Keenan

Today I had occasion to go into my 'minicpan' repository and look at 
modules/02packages.details.txt.gz.  The file begins:


#
File: 02packages.details.txt
URL:  http://www.perl.com/CPAN/modules/02packages.details.txt
Description:  Package names found in directory $CPAN/authors/id/
Columns:  package name, version, path
Intended-For: Automated fetch routines, namespace documentation.
Written-By:   PAUSE version 1.005
Line-Count:   259003
Last-Updated: Sat, 08 Jan 2022 20:29:03 GMT
#

When I click on the link for the URL entry, I am redirected to:

#
https://www.cpan.org/modules/02packages.details.txt
#

Should the URL entry in '02packages.details.txt' be changed to that of 
the redirect?


That might prevent breakage in the future if the directory structure at 
www.perl.com changes.


If this should be changed, where should we file a ticket?

Thank you very much.
Jim Keenan

perlbrew clone-modules option needs work

2021-06-19 Thread James E Keenan

perlbrew's 'clone-module' functionality is suffering from a problem for 
which I suspect there already exist solutions.


This functionality is intended to enable a user to install all modules 
to a new perlbrew-installed perl that she installed in an older one.


Which means we face the problem that some module names don't correspond 
to the names of the CPAN distributions in which they are found.  E.g., 
LWP versus libwww-perl.


If you have a non-hack solution to this problem and can say something 
helpful, please read:


https://github.com/gugod/App-perlbrew/issues/722

Thank you very much.
Jim Keenan

Is fast-matrix.cpantesters.org dead?

2021-01-22 Thread James E Keenan


I have not been able to reach it for quite a few weeks now.

I know that the problem was raised on #cpantesters-discuss and that 
someone was going to contact the maintainer, but I haven't seen any 
report on what happened.


Thank you very much.
Jim Keenan

Thanks for the work on File-Slurp

2019-02-21 Thread James E Keenan

I want to commend Chase Whitener for his work on bringing File-Slurp 
into perl-5.30-readiness.


perl-5.29.8 hit the net this morning and this afternoon I ran my "cpan 
river 3000" program for that release.  Although there were problems with 
that run (which is why I haven't yet posted results to perl5-porters), 
one thing is clear:  Many distributions where were not being reached for 
building and testing during the run of this program in earlier months in 
this perl dev cycle were reached this month because, for the first time, 
File::Slurp passed its tests against blead, thereby enabling its revdeps 
to be reached during testing.


$ ack '(grade|PASS)$' changes-5.29.7-to-5.29.8.csv | grep ',x,x,x,x'
Bot-Training,x,x,x,x,AVAR,Bot-Training-0.07,0.07,PASS
Bot-Training-MegaHAL,x,x,x,x,AVAR,Bot-Training-MegaHAL-0.03,0.03,PASS
Bot-Training-StarCraft,x,x,x,x,AVAR,Bot-Training-StarCraft-0.03,0.03,PASS
CPAN-Mini-Inject,x,x,x,x,MITHALDU,CPAN-Mini-Inject-0.35,0.35,PASS
DBIx-Admin-DSNManager,x,x,x,x,RSAVAGE,DBIx-Admin-DSNManager-2.01,2.01,PASS
DBIx-Admin-TableInfo,x,x,x,x,RSAVAGE,DBIx-Admin-TableInfo-3.03,3.03,PASS
Daemon-Generic,x,x,x,x,MUIR,Daemon-Generic-0.85,0.85,PASS
Email-MIME-CreateHTML,x,x,x,x,VANSTYN,Email-MIME-CreateHTML-1.042,1.042,PASS
File-Flock,x,x,x,x,MUIR,File-Flock-2014.01,2014.01,PASS
File-Policy,x,x,x,x,BBC,File-Policy-1.005,1.005,PASS
GraphViz2,x,x,x,x,RSAVAGE,GraphViz2-2.47,2.47,PASS
IO-Any,x,x,x,x,JKUTEJ,IO-Any-0.09,0.09,PASS
JSON-Util,x,x,x,x,JKUTEJ,JSON-Util-0.06,0.06,PASS
Kelp,x,x,x,x,MINIMAL,Kelp-1.02,1.02,PASS
Lingua-PT-PLNbase,x,x,x,x,AMBS,Lingua-PT-PLNbase-0.27,0.27,PASS
Locale-PO,x,x,x,x,COSIMO,Locale-PO-0.27,0.27,PASS
Locales,x,x,x,x,DMUEY,Locales-0.34,0.34,PASS
Module-Build-SysPath,x,x,x,x,JKUTEJ,Module-Build-SysPath-0.18,0.18,PASS
MooseX-Getopt-Usage,x,x,x,x,PITCHLESS,MooseX-Getopt-Usage-0.24,0.24,PASS
Parse-CPAN-Packages,x,x,x,x,MITHALDU,Parse-CPAN-Packages-2.40,2.40,PASS
ParseLex,x,x,x,x,PSCUST,ParseLex-2.21,2.21,PASS
Pod-Readme,x,x,x,x,RRWO,Pod-Readme-v1.2.3,v1.2.3,PASS
Portable-Dist,x,x,x,x,KMX,Portable-Dist-1.06,1.06,PASS
String-BOM,x,x,x,x,DMUEY,String-BOM-0.3,0.3,PASS
Sys-Path,x,x,x,x,JKUTEJ,Sys-Path-0.16,0.16,PASS
TAP-Formatter-JUnit,x,x,x,x,GTERMARS,TAP-Formatter-JUnit-0.11,0.11,PASS
Test-Inline,x,x,x,x,ADAMK,Test-Inline-2.213,2.213,PASS
Test-Name-FromLine,x,x,x,x,SATOH,Test-Name-FromLine-0.13,0.13,PASS
WWW-Search,x,x,x,x,MTHURN,WWW-Search-2.518,2.518,PASS

I can't say for certain that these distros were reached and PASSed 
because of the resolution of the File-Slurp problems, but I suspect 
that's the case for most of them.


Thanks again, genio!

Jim Keenan

How avoid "argument isn't numeric" warning when evaluating $VERSION?

2019-01-03 Thread James E Keenan


Suppose that in package Someones::Module I have:

#
$Someones::Module::VERSION = '1.4417_001';
#

And then in a different module I have:

#
die "Need more recent version of Someones::Module" unless 
$Someones::Module::VERSION >= "1.40";

#

This will generate a warning like this:

#
Argument "1.4417_001" isn't numeric in numeric ge (>=) at ...
#

I would like to submit a patch for the relevant module.  What is the 
recommended way of numifying $Someones::Module::VERSION in the '>=' 
comparison?


Or is there a better approach?

Thank you very much.
Jim Keenan

Re: File::Copy::Recursive broken on FreeBSD and other OSes; workingon replacement

2018-04-20 Thread James E Keenan


On 04/20/2018 08:48 AM, James E Keenan wrote:

On 04/20/2018 08:30 AM, Karen Etheridge wrote:
A new File::Copy::Recursive has been released -- has the threat now 
passed?


[snip]



Karen, thanks for alerting me to that.  I am now checking the 
installation on FreeBSD-11.1 of modules that this was blocking, 
including Test-File-ShareDir, DateTime and Dist-Zilla.  I should have 
results in a couple of hours.




Yes, it appears that with DMUEY's release of version 0.41 of 
File::Copy::Recursive, the crisis has passed.


To give you a feel for the scope of the (now averted) crisis, I'm 
attaching a plain-text file which is an excerpt from the 'cpan' 
reports-sent.db for the run this morning where I requested that 'cpan' 
install File::Copy::Recursive, Test::File::ShareDir, Data::MessagePack, 
Inline::C, DateTime and Dist::Zilla.  None of these were automatically 
installable on FreeBSD before 0.41.  This opens up 6000+ distros to 
installation via 'cpan', 'cpanm' and to BBC investigation.


A couple of CPAN distros switched to using 
File::Copy::Recursive::Reduced.  I'll continue to support it but now 
consider it feature-complete.


Thank you very much.
Jim Keenan
test PASS File-Copy-Recursive-0.41
test PASS Test-File-ShareDir-1.001002
test PASS DateTime-Locale-1.18
test PASS DateTime-1.48
test PASS Dist-Zilla-6.011
test PASS MouseX-SimpleConfig-0.11
test PASS IO-stringy-2.111
test PASS CGI-Simple-1.15
test PASS CGI-Struct-1.21
test PASS Class-C3-Adopt-NEXT-0.14
test PASS HTTP-Body-1.22
test PASS Class-Accessor-0.51
test PASS HTTP-Request-AsCGI-1.2
test PASS Hash-MultiValue-0.16
test PASS MooseX-Emulate-Class-Accessor-Fast-0.00903
test PASS MooseX-Getopt-0.71
test PASS MooseX-Types-Stringlike-0.003
test PASS MooseX-Types-Path-Tiny-0.012
test PASS MooseX-ConfigFromFile-0.14
test PASS MooseX-SimpleConfig-0.11
test PASS MooseX-MethodAttributes-0.31
test PASS POSIX-strftime-Compiler-0.42
test PASS Test-MockTime-0.17
test PASS Apache-LogFormat-Compiler-0.35
test PASS Test-Time-0.05
test PASS Cookie-Baker-0.09
test PASS Devel-StackTrace-AsHTML-0.15
test PASS Test-SharedFork-0.35
test PASS Filesys-Notify-Simple-0.13
test PASS HTTP-MultiPartParser-0.02
test PASS Stream-Buffered-0.03
test PASS WWW-Form-UrlEncoded-0.24
test PASS HTTP-Entity-Parser-0.21
test PASS HTTP-Headers-Fast-0.21
test PASS Test-TCP-2.19
test PASS Plack-1.0047
test PASS Plack-Middleware-FixMissingBodyInRedirect-0.12
test PASS Plack-Middleware-MethodOverride-0.15
test PASS Plack-Middleware-RemoveRedundantBody-0.07
test PASS Plack-Middleware-ReverseProxy-0.15
test PASS Plack-Test-ExternalServer-0.02
test PASS Safe-Isa-1.08
test PASS Text-SimpleTable-2.04
test PASS Tree-Simple-1.33
test PASS Tree-Simple-VisitorFactory-0.15
test PASS URI-ws-0.03
test PASS Catalyst-Runtime-5.90117
test PASS Tie-ToObject-0.03
test PASS Data-Visitor-0.30
test PASS Catalyst-Action-RenderView-0.16
test PASS Catalyst-Plugin-ConfigLoader-0.34
test PASS MIME-Types-2.17
test PASS Catalyst-Plugin-Static-Simple-0.36
test PASS Config-General-2.63
test PASS Type-Tiny-1.002002
test PASS Type-Tiny-XS-0.012
test PASS Type-Tie-0.009
test PASS Hash-FieldHash-0.15
test PASS Regexp-Util-0.003
test PASS Ref-Util-XS-0.117
test PASS PadWalker-2.3
test PASS Devel-Caller-2.06
test PASS Devel-LexAlias-0.05
test PASS File-ChangeNotify-0.28
test PASS Data-Compare-1.25
test PASS Devel-CheckOS-1.81
test PASS MooseX-Types-Path-Class-0.09
test PASS MooseX-Daemonize-0.21
test PASS HTTP-Parser-XS-0.17
test PASS Net-Server-2.009
test PASS Starman-0.4014
test PASS AppConfig-1.71
test PASS Template-Toolkit-2.27
test PASS Catalyst-Devel-1.39

Re: File::Copy::Recursive broken on FreeBSD and other OSes; working on replacement

2018-04-20 Thread James E Keenan


On 04/20/2018 08:30 AM, Karen Etheridge wrote:

A new File::Copy::Recursive has been released -- has the threat now passed?

On Wed, Apr 18, 2018 at 7:25 PM, James E Keenan <jkee...@pobox.com 
<mailto:jkee...@pobox.com>> wrote:


This message is intended for people working on the Perl toolchain
and, in particular, for attendees at the Perl Toolchain Summit
starting tomorrow in Oslo.

The test-against-dev work I have been conducting for several months
has identified at least one CPAN distro, high up in the toolchain,
which is broken on non-Linux operating systems.  The reasons for
this breakage are sub-optimal code in modules or test suites, not
"Blead Breaks CPAN."  Such breakage prevents the distro from being
installed and means that its revdeps are never reached during the
following cases:

* Installation via 'cpan' shell:  Distro fails and is graded as FAIL
and reported to metabase.  Assuming on 'force install' is attempted
on distro, revdep is graded as DISCARD and no report is sent to
metabase or to revdep's maintainers.

* Installation via 'cpanm':  Distro fails and is graded as FAIL. 
Revdep is not attempted and is not graded anywhere.  You have to

examine build.log for messages reporting "Bailing out "  No
report is sent to metabase or to maintainers.

* Because test-against-dev relies on 'cpanm', if a distro in, say,
the top 1000 of the CPAN river fails, its revdeps are never reached,
meaning their code does not get exercised against the latest monthly
development release.

The CPAN distro of greatest concern right now is
File-Copy-Recursive(http://search.cpan.org/~dmuey/File-Copy-Recursive-0.40/
<http://search.cpan.org/~dmuey/File-Copy-Recursive-0.40/>),
currently ranked about #128 in the CPAN river.  On FreeBSD (and
perhaps other OSes), FCR's current version (0.40 as of Apr 17 2018)
has an unfixed bug
(https://rt.cpan.org/Ticket/Display.html?id=123964
<https://rt.cpan.org/Ticket/Display.html?id=123964>) leading to test
failures and failure to install.  Unless a user chooses to '--force'
install FCR, that prevents its revdeps from being installed in an
automated manner or tested against dev/blead.

Over 6000 CPAN distributions have direct or indirect dependencies on
FCR, including CPAN-Reporter, Test-File-ShareDir, DateTime,
Dist-Zilla and Catalyst-Devel.  So thousands of CPAN distros cannot
be installed on FreeBSD and other systems.

The author of FCR is aware of the bug ticket and Tom Hukins provided
a patch which I have tested successfully on FreeBSD.  But the patch
has not been applied as of yesterday.  As we are now in the
countdown to perl-5.28.0, we really need to be able to assess the
impact of release candidates on CPAN -- and not just on Linux.

Earlier this week I wanted to resume sending CPANtesters reports
from FreeBSD-11.1.  I tried to install CPAN-Reporter but could not,
due to the bug in its dependency, FCR.  To address that I have
uploaded to CPAN a new CPAN distro, File-Copy-Recursive-Reduced
(FCR2), which provides stripped-down versions of 2 FCR subroutines,
fcopy() and dircopy(), which are used in the CPAN-Reporter test
suite.  I then branched CPAN-Reporter to create a version which uses
the FCR2 versions of these functions rather than FCR.  That makes
CPAN-Reporter installable again. I have therefore filed:

https://github.com/cpan-testers/CPAN-Reporter/issues/88
<https://github.com/cpan-testers/CPAN-Reporter/issues/88>
and
https://github.com/cpan-testers/CPAN-Reporter/pull/90
<https://github.com/cpan-testers/CPAN-Reporter/pull/90> (a better
p.r. than my earlier /89)

I hope that during the PTS you can review and apply this p.r. and
upload a new version of CPAN-Reporter.  This will once again permit
automated installation on FreeBSD and other non-Linux operating
systems.  It will also help us to assess perl-5.28's possible impact
on that distro and its revdeps.



Karen, thanks for alerting me to that.  I am now checking the 
installation on FreeBSD-11.1 of modules that this was blocking, 
including Test-File-ShareDir, DateTime and Dist-Zilla.  I should have 
results in a couple of hours.


Thank you very much.
Jim Keenan

Re: CPAN-river: can graph calculation be modified? Neil Bowers <neil.bow...@cogendo.com>

2018-02-02 Thread James E Keenan


On 02/02/2018 11:08 AM, H.Merijn Brand wrote:

On Fri, 2 Feb 2018 15:51:32 +, Neil Bowers
 wrote:


For the 5.29.* development cycle starting in May of this year, I would like to 
be able to use a ranking of CPAN distros which goes beyond asking:

* "How many other distributions depend on this one?"

... to asking:

* "How many distributions by other authors/maintainers depend on this one?"

Would that be feasible?  Has anyone attempted this already?


When we were discussing the River model at QAH, and in discussions afterwards, 
this came up. In the end we decided to keep things simple and go with the 
current common definition. There are some tools in the CPAN ecosystem that only 
count dependencies written by others.

We’d need to agree which dists get ignored in this alternate scheme. Consider 
this example:



Here MARY has released a bunch of dists, but Foo-Bar is also relied
on by other dists written by MUNGO and MIDGE.

The river count for Foo-Bar would be 2 here (ignoring the whole
branch that contains only dists from MARY), but the Foo river count
should be 3, I think. Foo-Bar “counts”, because it in turn is
depended on by dists from other authors. Otherwise the river count
would be 2 for both Foo and Foo-Bar. Basically we’re starting at the
“bottom" of the dependency graph, and trimming sub-graphs all from
one author.




Also consider this example:

What’s the river count of Plant — 0, 1, or 3? I think it should be 1,
in this alternate measure.


1 or 3: 1 if module chains from the same author are "compressed" to 1,
3 if not

More interesting would be

  Thing - Plant - Fruit - Banana - Silver Banana - Distasteful stuff
  JOHNPAULRINGO   RINGORINGO   GEORGE

would plant now be 1, 2, or 4?


I.e. for sub-graphs by the same author, you only include the dist at
the head of the sub-graph.


I'd suggest to have an option to squeeze any unbranched chain of
modules from the same author to 1



I *think* that's what I'm aiming for.  Let's say I have a CPAN distro 
called Gamma on which nothing else depends.  I refactor code out of 
Gamma into Beta, such that Gamma now depends on Beta.  By the standard 
definition, Beta moves up-river, Gamma down-river.


Next I refactor code out of Beta into Alpha.  Alpha is now farther 
up-river than both Beta and Gamma.


Suppose that Alpha now falls into the "top 1000" of the CPAN river. 
When I then switch Perl community roles and start to play the role of 
"rapid BBC evaluator."  A certain portion of my BBC program is now taken 
up with testing Alpha.  But, assuming I confine my focus to the top 
1000, that means some *other* CPAN distribution -- perhaps one whose 
revdeps are from different authors -- has been pushed out of the top 
1000.  That means the data I generate for P5P has been skewed toward 
myself.  That's what I'd like to avert.




It would be useful to have both measures available: raw-river and
author-river.

When looking at a dist there are (at least) three figures that might
be of interest: the full river count (total number of direct and
indirect dependencies), the author-filtered river count (as above),
and the number of direct dependencies (which could be split in 2 as
well).

Neil




Thank you very much.
Jim Keenan

How to name a subclass of a distro in App namespace?

2017-11-29 Thread James E Keenan


This is a CPAN distribution naming question.

My understanding is that the 'App' top-level namespace on CPAN is 
intended for the use of distributions which someone will primarily use 
by running an executable program found in the distribution's 'bin/' 
directory rather than by constructing program using functions found in 
.pm files under the 'lib/' directory.


Examples:

DistributionExecutable
App-cpanminus   cpanm
App-cpanminus-reporter  cpanm-reporter

Now suppose that I want to write a library which subclasses an 'App' 
distribution but where I don't want to provide an "app", i.e., where I'm 
not going to write a script in the 'bin/' directory but am simply going 
to write methods in a module underneath 'lib/'.


Should such a library go into the 'App' namespace?  I wouldn't think so, 
because it no longer provides an app.


On the other hand, our ordinary practice when subclassing is to extend 
the name one directory deeper.  Dropping the 'App-' from the start of a 
distro's name when subclassing the distro deviates from this practice. 
That's potentially confusing to CPAN users.


Specifically, I want to create a distro which subclass garu's 
App-cpanminus-reporter but does not provide an installable in 'bin/'. 
Do I call it:


App-cpanminus-reporter-XXX

... or

Cpanminus-reporter-XXX

... or something else?

Thank you very much.
Jim Keenan

Re: Open source archives hosting malicious software packages

2017-09-20 Thread James E Keenan


On 09/20/2017 06:01 PM, Neil Bowers wrote:

http://www.theregister.co.uk/2017/09/15/pretend_python_packages_prey_on_poor_typing/Would
 CPAN be subject to the same problem as described in the article above?


Yes.

DBI::Class, for example, could be a typo for DBIx::Class or a
misremembered Class::DBI, and there's nothing stopping anyone from
uploading a DBI::Class package that does all kinds of dodgy stuff.


There are plenty of confusable (small edit distance) pairs of module names on 
CPAN.

For example,
Algorithm::SVM and Algorithm::VSM
AI::POS and AI::PSO
both pairs are from different dists. More likely with short acronyms.

One thing we could do is have a tool looking at newly registered package names 
and alert the PAUSE admins to have a look at any that are a short edit distance 
from an existing package name.



Would anyone know of any prior art for detection of "short edit 
distances"?  (Perhaps even already on CPAN?)


Thank you very much.
Jim Keenan

Open source archives hosting malicious software packages

2017-09-15 Thread James E Keenan


http://www.theregister.co.uk/2017/09/15/pretend_python_packages_prey_on_poor_typing/

Would CPAN be subject to the same problem as described in the article above?

Re: Making www.cpan.org TLS-only

2017-08-31 Thread James E Keenan


On 08/31/2017 09:10 PM, Ask Bjørn Hansen wrote:

Hi everyone,

We’re considering how/how-much we can make www.cpan.org TLS-only.
http://log.perl.org/2017/08/tls-only-for-wwwcpanorg.html

I expect that we can’t make the whole site TLS-only without breaking some CPAN 
clients, so the conservative version is to force TLS for

- any url ending in *.html
- any url not in matching some variation of
  (/authors/ | /MIRRORED.BY | ^/modules/[^/]+ )

Does that sound about right? Maybe /src/, too?

(Also - we will support TLS for www.cpan.org permanently now, so please update 
URLs where possible and appropriate).




To be honest, I had no idea what 'TLS' meant when I first read this 
message.  So I can't say anything one way or the other about your proposal.


I suspect I'm not alone in this.  I would encourage you to post in a 
location like blogs.perl.org as to what 'TLS' is, so that the census 
count of the ignorant can be reduced.


Thank you very much.
Jim Keenan

Re: How can I regenerate a .minicpanrc?

2017-03-16 Thread James E Keenan


On 03/16/2017 05:47 PM, Olaf Alders wrote:



On Mar 16, 2017, at 5:17 PM, James E Keenan <jk...@verizon.net> wrote:

Last month I was testing the latest version of CPAN module B::C.  The testing 
process caused major damage to my machine.  One of my FreeBSD VMs was rendered 
inaccessible.  All of the .xxx configuration files in my home directory 
disappeared.  I spent most of the next several days reinstalling the VM, 
restoring my .xxx files and backing them up in an off-site repository.

I thought I was completely recovered until today.  For the first time in about 
6 weeks, I decide to run 'minicpan'.  I typed the command by itself, as I have 
always done -- and only got the usage statement.  I attribute this to the 
disappearance of .minicpanrc config file.

I am now running 'minicpan -l .  AFAICT, the minicpan tree 
is itself undamaged.  Is there any way to simply regenerate .minicpanrc without 
triggering another minicpan update process?


Hi Jim,

Could you just copy/paste it?  For example, mine is currently:

local: ~/minicpan
remote: http://cpan.metacpan.org/
exact_mirror: 1



Thanks, Olaf.  I guess that's the "default" .minicpanrc.  But ISTR that 
at one point, on xdg's suggestion, I had something that pointed to a 
file whose name started with '06'.


jimk

Re: Supporting a perl without . in @INC

2017-01-18 Thread James E Keenan

On 01/16/2017 12:06 PM, Todd Rinaldo wrote:

All,

Currently in blead is a change that will begin breaking many CPAN
installs. This is a result of a non-default change to perl builds which
removes . from @INC. There is currently a separate
proposal ( https://rt.perl.org/Public/Bug/Display.html?id=130467 )being
discussed to remove . from @INC by default in 5.26.

More information on the impact of this can also be found
here.
http://blogs.perl.org/users/todd_rinaldo/2016/11/how-removing-from-inc-is-about-to-break-cpan.html

As I understand things, this is the closest thing to a mailing list for
the toolchain group, so I'm trying this list first.

In order to action RT 130467 without completely breaking CPAN, I propose
the following patches to CPAN install related modules to fix the problem:

* Inject PERL_USE_UNSAFE_INC=1 into the environment early in the
following clients. This assures that everything spawned by these clients
gets . in @INC during test/install.
CPAN
CPANPLUS
App::cpanminus

* Inject PERL_USE_UNSAFE_INC=1 into TAP::Harness to support ad-hoc use
of prove. (Leon is already working on this)

* Inject PERL_USE_UNSAFE_INC=1 into install modules to try to address as
many Makefile.PL missing . in @INC issues as possible:
ExtUtils::MakeMaker
Module::Build
Module::Build::Tiny

What at this point I feel is lacking is agreement and/or discussion that
the above is the correct approach to solving this problem.

If you are not for this plan and/or you are a maintainer of one of the
above mentioned packages, your response would be appreciated. We're
running out of time to complete this in time for perl 5.26.

Thanks,
Todd Rinaldo

Re: No . in @INC breaks CPAN

2016-11-15 Thread James E Keenan


On 11/14/2016 07:39 PM, Aristotle Pagaltzis wrote:

* Todd Rinaldo  [2016-11-14 15:12]:

Long Term

We need to fix the CPAN modules themselves.


I’m afraid that’s a pipe dream. You can fix the most popular part of
CPAN but not even close to everything. There are still distributions
containing broken Module::Install versions, years after the last
bugfixes. Any solution that sets out from “we fix everything on CPAN”
as a starting point is no solution at all.

So the reality is this: if we make a change that makes most of CPAN non-
installable then a quite significant fraction of it will be left behind
and will never work again.



Before we polarize ourselves into the camps of "we have to fix all of 
CPAN" and "it's hopeless to try to fix CPAN", it's important to realize 
that we now have the conceptual tools with which to assess the scope of 
the problem.


We now know that we can visualize CPAN as a river of dependencies.  If 
we can identify a critical part of the "upstream", we can set up a 
CPANtesters-like apparatus to see how much damage (flooding?) 
default_inc_excludes_dot will actually cause.  Discussion of courses of 
action will then be empirically informed.


This change is, after all, just a much larger version of the "blead 
breaks CPAN" problem we've been handling for years.


Thank you very much.
Jim Keenan

Re: No . in @INC breaks CPAN

2016-11-14 Thread James E Keenan

On 11/14/2016 09:11 AM, Todd Rinaldo wrote:
> Howdy,
>
> I've done a write up of a recent change to blead perl. In the future 
it will no longer be possible to count on . being in @INC. This will 
break many of the existing CPAN installs.

>
> It was suggested I send the RFC here:
>
> 
http://blogs.perl.org/users/todd_rinaldo/2016/11/how-removing-from-inc-is-about-to-break-cpan.html

>
> In Perl 5.26, it will no longer be a safe assumption to assume . is 
in @INC. This is a good move towards a more secure Perl, but will break 
the installation of many CPAN modules. For those of you wondering why 
this was done, see this post for more information.

>
> Many CPAN modules try to do things like: use inc::Module::Install; 
This depends on . being in @INC. If you invoke Makefile.PL without it, 
the script will not even run.

>

Here is a use case which I found very soon after building a perl at 
blead with the new option and then installing App::cpanminus against 
that perl.

#
Summary of my perl5 (revision 5 version 25 subversion 7) configuration:
  Commit id: dd4f16e4ff71ea6d0422d277f00fe430e1d93938
[snip]
config_args='-des -Dusedevel -Uversiononly 
-Dprefix=/home/jkeenan/testing/nodot -Dman1dir=none -Dman3dir=none 
-Ddefault_inc_excludes_dot'

[snip]
  @INC:
lib
/home/jkeenan/testing/nodot/lib/perl5/site_perl/5.25.7/x86_64-linux
/home/jkeenan/testing/nodot/lib/perl5/site_perl/5.25.7
/home/jkeenan/testing/nodot/lib/perl5/5.25.7/x86_64-linux
/home/jkeenan/testing/nodot/lib/perl5/5.25.7
#

I used ./bin/cpanm to install one of my CPAN distros that has a 
second-level dependency on another one of those distros, 
Perl-StringIdentifier.

#
Building and testing String-PerlIdentifier-0.05 ... FAIL
! Installing String::PerlIdentifier failed. See 
/home/jkeenan/.cpanm/work/1479138905.14485/build.log for details. Retry 
with --force to force install it.

#

Let's inspect that build log.

#
PERL_DL_NONLAZY=1 "/home/jkeenan/testing/nodot/bin/perl" 
"-MExtUtils::Command::MM" "-MTest::Harness" "-e" "undef 
*Test::Harness::Switches; test_harness(0, 'blib/lib', 'blib/arch')" t/*.t
Can't locate t/eligible_chars in @INC (@INC contains: t/ 
/home/jkeenan/.cpanm/work/1479138905.14485/String-PerlIdentifier-0.05/blib/lib 
/home/jkeenan/.cpanm/work/1479138905.14485/String-PerlIdentifier-0.05/blib/arch 
/home/jkeenan/testing/nodot/lib/perl5/site_perl/5.25.7/x86_64-linux 
/home/jkeenan/testing/nodot/lib/perl5/site_perl/5.25.7 
/home/jkeenan/testing/nodot/lib/perl5/5.25.7/x86_64-linux 
/home/jkeenan/testing/nodot/lib/perl5/5.25.7) at t/Auxiliary.pm line 14.

Compilation failed in require at t/01_basic.t line 8.
BEGIN failed--compilation aborted at t/01_basic.t line 8.
# Looks like your test exited with 2 just after 1.
t/01_basic.t ..
Dubious, test returned 2 (wstat 512, 0x200)
Failed 40/41 subtests
#

Here's what that distro has in/under its 't/' directory.

#
$ ls t |cat
01_basic.t
02_specified.t
03_min.t
04_max.t
05_default.t
06_no_scores.t
Auxiliary.pm
eligible_chars
#

#
$ head t/01_basic.t
# t/01_basic.t - four basic tests
use Test::More tests => 41;
use strict;
use warnings;

BEGIN { use_ok( 'String::PerlIdentifier' ); }
use lib ("t/");
use Auxiliary qw{ _first_and_subsequent };

four_basic_tests() for (1..10);
#
$ head -15 t/Auxiliary.pm
package Auxiliary;
use strict;
use warnings;
our ($VERSION, @ISA, @EXPORT_OK);
$VERSION = '0.05';
require Exporter;
@ISA = qw(Exporter);
@EXPORT_OK   = qw(
_first_and_subsequent
);
*ok = *Test::More::ok;
use lib ('.');

our (%eligibles, %chars);
require "t/eligible_chars";
#

The test file attempts to load Auxiliary.pm, which is located in the 
same directory as the test file.  Auxiliary.pm in turn 'require's a 
plain-text file, 'eligible_chars', which is also located in t/.  That 
'require' fails.

In this case, adding '.' to the distribution's Makefile.PL made no 
difference.  I had to add "use lib ('.');" to Auxiliary.pm to enable it 
to locate 'eligible_chars', after which 'make test' PASSed.

Based on this example and several other failures, my hunch is that many 
of the failures which we'll see on CPAN are failures of *tests* rather 
than failures of the modules themselves.

Thank you very much.
Jim Keenan

When cpanm fails to find any of the files in a kit

2016-09-09 Thread James E Keenan


Today my attention was drawn to this older post on Stack Overflow:

http://stackoverflow.com/questions/37009887/cpanm-perl-module-installation-failed-on-make-and-make-test

The original poster apparently found a work-around for his problem, but 
I felt there were still puzzling things about the case.  Here's the 
setup (as best I can tell):


1. User has installed App::cpanminus to work with the system perl.

2. User tries to use cpanm to install a module (in this case, Try::Tiny, 
but it doesn't really matter which one) from CPAN.  It appears that, at 
least on the user's first pass he/she did not use 'sudo' in any way -- 
but the actual command-line invocation is not provided.


3. cpanm first sounds warnings at:

#
Checking if your kit is complete...
Warning: the following files are missing in your kit:
lib/Try/Tiny.pm
maint/bench.pl
t/00-report-prereqs.dd
...
#

The process appears to be failing *all* the files in the kit; this is 
not a problem of a CPAN author screwing up one or two items in the MANIFEST.


That's what puzzling to me:  The fact that cpanm (or perhaps 'make' 
underneath cpanm) is missing the kit in its entirety.


Have others encountered this?

Do you know what could have caused this?  (The poster apparently got a 
better outcome when using 'sudo' with 'cpanm', but it's not clear to me 
why that would happen.)


Thank you very much.
Jim Keenan

Re: cpan-rsync.perl.org::CPAN ; unreachable

2016-08-21 Thread James E Keenan


On 08/21/2016 06:08 AM, Henk P. Penning wrote:

Hi,

  for the past few hours, this fails :

rsync cpan-rsync.perl.org::CPAN

rsync: failed to connect to cpan-rsync.perl.org: No route to host (113)

  No issues are mentioned here : http://log.perl.org/

  Groeten,

  HPP



Since I've never used this particular service, I don't know what 
behavior to expect.  I recommend you file a bug report by sending email to:


webmaster at perl.org

Thank you very much.
Jim Keenan

Where should Test::Smoke be installed?

2016-08-06 Thread James E Keenan

I'm not sure whether this is the best list to pose this question on, but 
since I haven't received any response to the following link, I'll proceed.


https://rt.cpan.org/Ticket/Display.html?id=116698#txn-1655515

http://search.cpan.org/~abeltje/Test-Smoke-1.70/README states:

"The Test::Smoke suite has been designed to be installed outside of the 
normal Perl library-tree."


I suspect that is why, when I today tried to install Test::Smoke with 
'cpanm Test::Smoke', 'cpanm' timed out. I examined the Makefile.PL and 
noticed that it explicitly prompts the user to determine where the 
distribution should be installed.


But that begs the question: "Where *should* Test::Smoke be installed?" 
Where is the best place "outside of the normal Perl library-tree" to put 
it? The docs are silent on this and should be clarified.


Here's my specific use case: I have installed FreeBSD 10.3 in a 
VirtualBox VM on a Linux x86-64 host. I have never previously sent smoke 
reports about Perl itself. (Linux is sufficiently well covered.) But now 
I am considering doing smoke-testing for Perl 5 blead (and perhaps for 
smoke-me branches) on this FreeBSD installation. I have not yet used 
something like 'perlbrew' or 'plenv' to re-set my "everyday perl, so my 
everyday perl is 5.20.3 found at /usr/local/bin/perl. I have a git 
checkout of perl underneath my home directory at ~/gitwork/perl/.


Let's assume that at first I only want to send smoke reports 
episodically, i.e., kicked off manually rather than by a cron job set to 
a particular time of day or upon detection of a new commit to blead. How 
and where should I install Test::Smoke for that purpose?


Thank you very much.
Jim Keenan

Re: Found rare bug in Pod::Simple

2016-03-05 Thread James E Keenan


On 03/05/2016 03:56 PM, Neil Bowers wrote:

Hi Marc, & CPAN Workers,

I’ve been looking into the final two CPAN Testers fails, and have finally got 
to the bottom of them.
The failing test is search50.t, and the problem is where it does the following:

- call survey() to get hash of name => path
- foreach name, then call find() and check it returns the same path

There are two CPAN Testers fails:

http://www.cpantesters.org/cpan/report/4ddcddb1-6c58-1014-bec3-a1032b7077ee 

http://www.cpantesters.org/cpan/report/39970866-dd9c-11e5-a3ee-89603848fe5a 


Basically the problem is that

- the pod directory has both perlpodstyle and perlpodstyle.pod in it 
(how come?!)


Is it possible that the testers have files left over from previous test 
runs?


jimk

Need dependency-free way of capturing verbose function output in a variable

2015-07-18 Thread James E Keenan

As part of the work which Richard Elberger and I are doing for 
File-Path, we would like to reduce the test suite's dependency on 
libraries not distributed with the Perl 5 core.  Specifically, that 
means eliminating use of Test::Output functions stdout_is(), stderr_is() 
and stderr_like().


I know how to use $SIG{__WARN__} and $SIG{__DIE__} to replace the last 
two functions.  That leaves only the problem of capturing STDOUT when a 
user has requested verbose output from mkpath, make_path, rmtree or 
remove_tree.


If we were not working on a library distributed with core, I would use 
any one of IO::Capture, IO::CaptureOutput or Capture::Tiny.  (I've even 
contributed to some of them!)  But even Capture::Tiny (a) is not (yet) 
core; and (b) has a dependency on File::Temp, which in turn has a 
dependency on File::Path.


ISTR that there's some magic you can perform with 'select' to obtain 
this objective, but I can't seem to locate what I have in mind on the 
network.  What I want is something that, with just core Perl will be 
callable like this:


my $captured_verbose_output =
_run_for_verbose( sub {@created = mkpath($dir, 1)} );
is(
$captured_verbose_output,
mkdir $base\nmkdir $dir\n,
'mkpath verbose (old style 1)'
);

... where _run_for_verbose() has this look:

sub _run_for_verbose {
my $code = shift;
my $captured_verbose_output;
... # filehandle magic; no non-core libraries
$code;
... # complete the magic
return $captured_verbose_output;
}

Suggestions?  Thanks in advance.

Jim Keenan

Re: Identify ways downstream distros use a given distro; include testin test suite

2015-06-12 Thread James E Keenan


On 06/11/2015 01:21 AM, David Golden wrote:

On Wed, Jun 10, 2015 at 10:54 PM, James E Keenan jk...@verizon.net wrote:



Is there any prior art for it?



https://metacpan.org/pod/Test::DependentModules

Though I think that just runs downriver tests, not scrapes their usage in
any way.



A good suggestion, but I cannot currently install Test::DependentModules 
(at least not via 'cpanm' and perl 5.22.0) due to a build failure in 
Net::SSLeay (https://rt.cpan.org/Ticket/Display.html?id=105189).


(A good demonstration of the effect which failures in distributions high 
upstream in the CPAN river.)


Thank you very much.
Jim Keenan

Re: File::Temp/File::Spec problems on Windows under taint mode – breaking change needed?

2015-04-24 Thread James E Keenan


On 04/24/2015 06:21 AM, David Golden wrote:

On Thu, Apr 23, 2015 at 11:37 PM, Aristotle Pagaltzis pagalt...@gmx.de
wrote:


Not sure it’s the right venue for this level of discussion. It seems to
me more governance/admin-level. Then again looking back at the archives,
there have been quite specific technical discussions here before, so I’m
not sure quite where it falls.



In Berlin we said that that the toolchain charter meant discussing
major/breaking changes in a publicly archived venue -- and we picked this
existing one to start.

I hope we'll see more technical discussions of this sort going forward.



Having such discussions on this list will help those of us who are on 
the periphery of toolchain maintenance and development to keep up and, 
possibly, be more effective contributors.  So I, too, hope this list is 
used more extensively in the future.

Potential hackathon themes

2014-12-15 Thread James E Keenan

No, I'm not talking about topic areas for the 2015 Perl QA Hackathon, 
which I believe is being held in Berlin this year.


I'm talking about local hackathons, such as the ones we had in St Louis 
in November 2012 or in New York City in March 2013.  We've been talking 
about having another NYC Perl Hackathon in spring 2015.  But so far we 
-- or, at least, I -- have not been able to come up with any compelling 
themes for a hackathon, and I'm writing to cpan-workers to gather 
suggestions.


More specifically, we need to identify areas where people who are 
attending a hackathon for the first or second time can make a meaningful 
contribution by attending a one-day event.  In the two events cited 
above, we were able to integrate first-timers by having them sign up 
before the hackathon to work on projects suggested by others.  For 
example, we were able to have some first-timers write descriptions for 
tests in the Perl 5 core distribution lacking them.


I follow Perl-Toolchain-Gang discussions, though not as closely as the 
main contributors.  If there are PTC-ish topics that would benefit from 
an infusion of efforts from less-experienced-than-PTC hackers, I would 
love to hear about them (on list or off), as they could be worked into 
themes for our (hopefully) upcoming event.


Thank you very much.
Jim Keenan

Re: Where is ExtUtils-Install maintained?

2014-09-16 Thread James E Keenan


On 09/15/2014 06:37 PM, Leon Timmermans wrote:

On Sun, Sep 14, 2014 at 2:22 PM, James E Keenan jk...@verizon.net wrote:


Last month I sent this message to the perl5-porters mailing list.  I got
no replies.  Does anyone on this list have thoughts on this?

#
Two recently filed RT tickets ...

https://rt.perl.org/Ticket/Display.html?id=121829
https://rt.perl.org/Ticket/Display.html?id=122596

... led me to look for the source code for ExtUtils::Installed and
ExtUtils::Packlist.  They're part of the ExtUtils-Install distribution
which is distributed with core.

In the core distro, ExtUtils-Install is found under 'cpan/', which
suggests that it is *primarily* maintained on CPAN by one or more
maintainers, and not maintained by P5P.

However, when I go to http://search.cpan.org/~
bingos/ExtUtils-Install-1.68/, I see that this distro's bug tracker is
listed as http://rt.perl.org/rt3/ and its repository is listed as
http://perl5.git.perl.org/perl.git/.  In other words, it's maintained in
core not on cpan/.

If so, then shouldn't ExtUtils-Install be under 'dist/' rather than
'cpan/' in the core distribution?

 From pod/perlsource.pod:

dist: This directory is for dual-life modules where the blead source is
canonical.

I'd like to clarify this before embarking on work on either of the two RTs
cited above.



AFAIK it has recently been decided to move it from core, but as it hasn't
seen a release yet rt.cpan.org still thinks the upstream bugtracker is
rt.perl.org.

Leon



Please ping me when this release happens so that I can close related 
tickets in rt.perl.org.


Thank you very much.
Jim Keenan

Re: All tickets at rt.cpan.org for ExtUtils-Manifest are closable

2014-08-02 Thread James E Keenan


On 08/01/2014 11:21 PM, James E Keenan wrote:





They've all been created in the last few weeks, so they're near the top
of either the New or Open queues when searched in reverse order of Last
Updated.  You can't miss 'em.

Or easier still: http://tinyurl.com/qb63sn9



Sorry, I think that was a link to rt.cpan.org.  Here are the rt.perl.org 
tickets and my action recommendations for each.


https://rt.perl.org/Ticket/Display.html?id=122415
whole name quoted filename
Review the patch I submitted:
https://rt.perl.org/Ticket/Attachment/1302699/690549/122415-0001-Have-maniread-properly-handle-whole-name-quoted-file.patch

https://rt.perl.org/Ticket/Display.html?id=122416
MANIFEST.SKIP exclusions
Note that OP has responded.  Maintainer should make a decision as to 
whether change is desired and work it out with OP.


https://rt.perl.org/Ticket/Display.html?id=122421
aborting make manifest
Maintainer (Toolchain Gang, I guess) should make a decision as to 
whether change is desired


https://rt.perl.org/Ticket/Display.html?id=122417
print STDERR vs warn
Maintainer should make a decision as to whether change is desired. Patch 
will be simple.


https://rt.perl.org/Ticket/Display.html?id=122420
maniread on VMS
Make friends with Craig Berry :-)

Thank you very much.
Jim Keenan

Re: All tickets at rt.cpan.org for ExtUtils-Manifest are closable

2014-08-02 Thread James E Keenan


On 08/01/2014 11:21 PM, James E Keenan wrote:

On 08/01/2014 07:48 PM, Karen Etheridge wrote:


I should have added this one:

https://rt.perl.org/Ticket/Display.html?id=122427
PERL_MM_MANIFEST_VERBOSE

It has been resolved.

Re: http://search.cpan.org/ does not answer

2014-07-19 Thread James E Keenan


On 07/17/2014 02:48 PM, Juergen Rose wrote:

root@caiman:/root(118)# ping search.cpan.org
PING cpansearch.perl.org (207.171.7.49) 56(84) bytes of data.
64 bytes from 207.171.7.49: icmp_seq=1 ttl=55 time=220 ms
64 bytes from 207.171.7.49: icmp_seq=2 ttl=55 time=216 ms
64 bytes from 207.171.7.49: icmp_seq=3 ttl=55 time=218 ms
^C
--- cpansearch.perl.org ping statistics ---
4 packets transmitted, 3 received, 25% packet loss, time 3000ms
rtt min/avg/max/mdev = 216.044/218.380/220.864/1.970 ms
root@caiman:/root(119)# wget http:// search.cpan.org
http://: Invalid host name.
--2014-07-17 20:45:33--  http://search.cpan.org/
Resolving search.cpan.org (search.cpan.org)... 207.171.7.49,
207.171.7.59
Connecting to search.cpan.org (search.cpan.org)|207.171.7.49|:80...
connected.
HTTP request sent, awaiting response... 503 Service Unavailable
2014-07-17 20:45:39 ERROR 503: Service Unavailable.





We're back in business.  Try your link again.

cpanm: Is it possible to install a localized version of cpanm for one-shot use?

2014-06-22 Thread James E Keenan


Q. Why would you want to do this?

A. Because I am trying to adapt/modularize a program written by another 
developer which installs cpanm to a previously-cpanm-free environment 
for use in testing.  I want to be able to use this program in 
environments where cpanm is already installed.


Q. Can you be more specific?  What's the program you're trying to 
modularize?


A. It's a program I call 'khwdebug.sh'; sanitized version attached.  It 
was written by Karl Williamson, an important contributor to the Perl 5 
core distribution and to the Perl 5 Porters mailing list, to address 
particular bug reports filed on rt.perl.org.


Q. Which bug reports?

A. Long-time Perl contributor Andreas Koenig tests CPAN modules against 
Perl 5 blead.  When a CPAN distribution known to work on a released 
version of Perl 5 suddenly starts to fail tests when run against blead, 
a 'perlbug' process generates an RT ticket.  The author of the CPAN 
distribution gets notified.


But it's often useful for people other than Andreas to try to confirm 
the test failure report.  Karl Williamson (khw) wrote a program to do 
that.  It is set up to run on the Perl 5 development server known as 
'dromedary' which is donated to us by booking.com.  I've started to run 
'khwdebug.sh' on dromedary as well.


Q. What good will modularizing this program do?

A. It will enable people who do not have access to the dromedary server 
to test CPAN distributions against blead more easily than they can do 
now.  And it will enable us to determine whether tests are failing on 
platforms other than the Linux/x86_64 found on dromedary.


Q. Where does 'cpanm' come into all this?

A. khwdebug.sh builds Perl 5 blead and installs it into a user-specified 
testing directory.  Example:


dir=/home/jkeenan/testing/blead

./Configure -des -Dusedevel -Uversiononly -Dprefix=$dir -Dman1dir=none 
-Dman3dir=none


make -j$test_jobs install

khwdebug.sh then downloads the fatpacked version of cpanm from the 
network and builds and installs it into the same directory as the blead 
'perl' executable.


wget --no-check-certificate -O- -q http://bit.ly/cpanm |$dir/bin/perl - 
-v App::cpanminus


Remember:  This is happening on a machine where 'cpanm' is not installed 
by default.


khwdebug.sh then uses that version of 'cpanm' to download a CPAN distro 
from a mirror and prepare all its dependencies.


module=Text::CSV::Hashify
$dir/bin/cpanm --installdeps $module

khwdebug.sh then makes use of the nifty '--look' option to cpanm to open 
a sub-shell, change to the directory where the CPAN distro has been 
unpacked, and prompt the user to run 'perl Makefile.PL' where the 'perl' 
in question is the recently installed blead:


echo About to cd a work space directory 2
echo From there, run '$dir/bin/perl Makefile.PL' 2
echo And then, probably 'make test' 2

$dir/bin/cpanm --look $module

The user then creates a Makefile and runs 'make test' (or similarly if 
the distro uses Module::Build).  The user can then confirm the test 
failure report and perform local debugging.  The more confirmations we 
get from more platforms, the more we can be sure that we have a real 
problem.


Q. Alright, so what's the problem with 'cpanm'?

A. When you try to do a bootstrapped installation of the fatpacked 
version of 'cpanm', it, not surprisingly, asks if App::cpanminus is 
already installed.  If cpanm is already installed, then no additional 
version is installed into $dir/bin/.  But at a certain point, 
khwdebug.sh expects to be able to find and use $dir/bin/cpanm.  When 
that expectation is not meant, khwdebug.sh cannot proceed.


Thank you very much.
Jim Keenan


sanitized_khwdebug.sh
Description: application/shellscript

perlbrew switch perl-5.18.2 opening subshell

2014-03-22 Thread James E Keenan


Apologies in advance if this is the wrong place to post.

I am setting up a new machine with Linux/Ubuntu v13.10 -- the first time 
in nearly two years when I am using Linux as a desktop.  I'm trying to 
make a clean start with respect to my Perl-related programming practices:


* I want to use perlbrew as much as possible
* I want to use cpanm rather than cpan as much as possible

But I am having a hell of a time trying to get perlbrew to work in the 
way I am accustomed to it working on my $job MacBook Pro.


I have (several times today) installed perlbrew with this command:

cpan
 install App::perlbrew

I now have:

#
which perlbrew
/usr/local/bin/perlbrew
#

And I have added: 'source ~/perl5/perlbrew/etc/bashrc' to the end of my 
~/.bash_profile file.  (AAMOF, it's the only entry yet in that file.)


And I have said: 'perlbrew init'

The following commands work as expected:

#
perlbrew available
perlbrew install perl-5.18.2
perlbrew list
#

However, whenever I say:

##
perlbrew switch perl-5.18.2
##

I get:

#
A sub-shell is launched with perl-5.18.2 as the activated perl. Run 
'exit' to finish it.


bash-4.2$
#

This is unexpected and undesired behavior.  On the basis of using 
perlbrew on my work laptop for  1 year, I would *not* expect a 
sub-shell to be opened.  I would expect that, from the moment of 
'perlbrew switch' forward, I would:


a) stay in the same shell
b) have 'perl -v' point to the 5.18.2 underneath my homedir.

On this new machine, when I am in the subshell, which perl -v gives 
5.18.2.  But when I exit, I'm back to the vendor perl.


'perldoc perlbrew' documents 'switch' as: Permanently use the specified 
per as default.


That's the behavior I want.  I don't want a subshell to be opened.  I 
want perlbrew switch perl-5.18.2 to make 5.18.2 my default perl.


Can anyone tell me what I'm doing wrong?

Thank you very much.
Jim Keenan

Re: perlbrew switch perl-5.18.2 opening subshell

2014-03-22 Thread James E Keenan


On 3/22/14 10:31 AM, James E Keenan wrote:
[snip]



And I have added: 'source ~/perl5/perlbrew/etc/bashrc' to the end of my
~/.bash_profile file. (AAMOF, it's the only entry yet in that file.)



Adding that 'source' line to my .bash_profile appeared to be 
ineffective.  However, when I added it to the end of ~/.bashrc, then 
called 'source ~/.bashrc', 'perlbrew switch perl-5.18.2' DWIMmed.


http://perlbrew.pl/Release-0.29.html appears to suggest that this is a 
recent modification to perlbrew's behavior.


If anyone can confirm that this diagnosis/correction is optimal, that 
would be appreciated.


Thank you very much.
Jim Keenan

Re: perlbrew switch perl-5.18.2 opening subshell

2014-03-22 Thread James E Keenan


On 3/22/14 1:04 PM, Kent Fredric wrote:

On 23 March 2014 04:31, James E Keenanjk...@verizon.net  wrote:


And I have added: 'source ~/perl5/perlbrew/etc/bashrc' to the end of my
~/.bash_profile file.  (AAMOF, it's the only entry yet in that file.)



This might be the cause, I have that stanza in ~/.bashrc instead

And this seems to make a difference.


Thanks, Kent.  I got your post just after I stumbled on the same thing 
and posted to the list.


Thank you very much.
Jim Keenan

Configuring a new machine in an up-to-date way

2014-01-19 Thread James E Keenan


Friends,

Assuming no significant interference from the polar vortex, later this 
week I will be installing either FreeBSD or OpenBSD on a recent model 
Asus ultrabook computer.  This will be the first time since ... well, 
first time in many years that I will be working with a new operating 
system on a machine that I own and am completely responsible for.  So it 
provides me with an opportunity to configure the machine with up-to-date 
(and, perhaps, best) practices.


I would like your guidance on how to proceed once the new OS is 
installed.  Here are my plans for this machine:


1.  I don't intend it to be my primary personal laptop -- at least not 
at first.  I don't intend it to be the box to which I download my email 
or from which I browse the web.  At least at first it will not be the 
machine on which I do my perl5.porters-related work or go on IRC.


2.  I do intend to use it to do smoke-testing of CPAN distributions. 
Specifically, those distributions which changes in Perl 5 blead break, 
as reported in the tickets Andreas files on rt.perl.org.  Andreas does 
his testing on Linux, so smoke-testing on a non-Linux system can be 
helpful in resolving an RT.  For example, if I were able to do a *BSD 
smoke test against blead for Devel-SizeMe, I would be confident in 
closing https://rt.perl.org/rt3/Ticket/Display.html?id=119295.


I don't currently do testing of CPAN distros *against blead* or *against 
dev releases* (e.g., 5.19.*) because, on dromedary at least, I haven't 
figured out how to install an easily disposable blead or dev against 
which I can test a CPAN distro.


(I don't intend to do general smoke-testing of all CPAN against blead 
or dev release with this machine.  It's just a laptop, after all.)


3.  I do intend to use this machine for testing of Perl 5 blead. 
Currently I test blead on Linux/x86_64 (the dromedary server) several 
times a week and on my 10-year-old iBook G4 (Darwin/PPC) several times a 
month.  I don't actually submit smoke reports; I merely file RTs when 
they occur.  I may or may not consider smoke tests against blead on this 
*BSD once it's up and running.


4.  I do intend to use this machine for learning a new language which 
sits on top of the JVM.  Both my current laptop and my Linode are not 
suitable candidates for installing a JVM.


5.  For the past several years, my approach when being presented with a 
new box on, say, $job, has been to compile the latest Perl 5 stable 
release from source and install it into /usr/local/.  I then use 'cpan' 
to install Devel::Cover, DateTime, LWP, Moose, etc.  I have had some 
limited experience with 'perlbrew' at $job, but not for the purpose of 
switching between a stable version of Perl and one intended to test 
things against the cutting edge.


So, given the above, what would you recommend once the new OS is 
installed and I can do 'perl -V'?


Specifically, how should I install and use tools like:

* perlbrew
* cpanm

Note that outside of the considerations listed above, I would like to 
rely on the OS's ports/packages system for all other software installation.


Thank you very much.
Jim Keenan

Re: Preserving git history across repositories

2013-12-29 Thread James E Keenan


On 12/29/13 5:30 AM, Aristotle Pagaltzis wrote:


I’ve put up the result:

 https://github.com/ap/opodhtml

Feel free to clone that as a basis for the rest of your work.



When I went to that page, I saw a box labelled HTTPS clone URL where I 
most often see SSH clone URL.  I tried the following and did not succeed.


$ git clone https://github.com/ap/opodhtml.git aristotle-opodhtml
Initialized empty Git repository in 
/Users/jimk/gitwork/aristotle-opodhtml/.git/
fatal: https://github.com/ap/opodhtml.git/info/refs download error - 
Protocol https not supported or disabled in libcurl


I have available:
$ git version
git version 1.6.3.2

Can you identify the problem -- or enable clone via SSH?

Thank you very much.
Jim Keenan

Re: Preserving git history across repositories

2013-12-29 Thread James E Keenan


On 12/29/13 9:27 AM, Zefram wrote:

James E Keenan wrote:

When I went to that page, I saw a box labelled HTTPS clone URL
where I most often see SSH clone URL.  I tried the following and
did not succeed.


Github also supports the Git protocol, for which you just need to change
the scheme part of the URL:

$ git clone git://github.com/ap/opodhtml.git
Cloning into 'opodhtml'...
remote: Counting objects: 1596, done.
remote: Compressing objects: 100% (632/632), done.
remote: Total 1596 (delta 474), reused 1596 (delta 474)
Receiving objects: 100% (1596/1596), 372.52 KiB | 445 KiB/s, done.
Resolving deltas: 100% (474/474), done.

-zefram



I tried this on my laptop after trying 'https' and before posting my 
last message and did not succeed.


Or, more precisely, I *thought* I tried this on my laptop.

But I see now that I was using the wrong syntax.  I was using:

  git clone g...@github.com/ap/opodhtml.git aristotle-opodhtml

instead of this based on yours above:

  git clone git://github.com/ap/opodhtml.git aristotle-opodhtml

And I see that this repo does indeed have history going back all the way 
to Larry's original commit.  I hope to pursue the rest of Aristotle's 
suggestions later today.


Thank you very much.
Jim Keenan

Re: Preserving git history across repositories

2013-12-29 Thread James E Keenan


On 12/29/13 5:30 AM, Aristotle Pagaltzis wrote:

* James E Keenanjk...@verizon.net  [2013-12-28 19:15]:

At this point, the top-level directory merely contained the
directories formerly found in ext/Pod-Html/. And 'git log' and 'git
blame' indicated that the history had been preserved.


Except, at one time it had lived in lib/ and you are completely missing
that history. Yours effectively starts at the point where Nick moved it:
https://github.com/jkeenan/opodhtml/commit/75e62e6c1c6203daf034df38b525a6428d419b19
But you then have a couple of commits before that, at the beginning of
your local history… which are completely empty.


git filter-branch --subdirectory-filter ext/Pod-Html/ -- --all


So first, make sure there are no useless commits:

 git filter-branch --prune-empty \
 --subdirectory-filter ext/Pod-Html/ -- --all

Next, you REALLY don’t want --all, which makes some 63 THOUSAND commits
that will take forEVER to process (hours). You want to look at only the
commits that touch relevant paths, which is some 530 total. On an SSD
that can be index-filtered in half a minute. MUCH better.

 git filter-branch --prune-empty \
 --subdirectory-filter ext/Pod-Html/ -- -- lib/Pod ext/Pod-Html

Note the double `--` – that is not a typo, the first `--` is for telling
git-filter-branch that the rest of the arguments are for git-rev-list,
so the second one gets passed through to git-rev-list, which takes it to
mean that only paths follow.

Next, since unfortunately --subdirectory-filter cannot extract multiple
directories at once, this job will need --index-filter.

 git filter-branch --prune-empty --index-filter '
 git rm --cached -r -q -- . ;
 git reset -q $GIT_COMMIT -- ext/Pod-Html/ lib/Pod/
 ' -- -- ext/Pod-Html/ lib/Pod/

The first line will clear out the index entirely. The next line restores
the relevant directories from the original commit undergoing rewriting.

Now comes the hard part, because lib/Pod/ alone is both too much (there
have been a number of other modules in there over time) as well as too
little (the relevant files used to be strewn all over the place before
there were consolidated into ext/Pod-Html/). This requires sleuthing.
I started with a full clone of perl5.git and did

 git log --name-status --full-diff -- ext/Pod-Html | egrep ^R

to find out all the files that were ever moved into ext/Pod-Html from
elsewhere:

 lib/Pod/Html.pm
 lib/Pod/t/eol.t
 lib/Pod/t/htmlescp.pod
 lib/Pod/t/htmlescp.t
 lib/Pod/t/htmllink.pod
 lib/Pod/t/htmllink.t
 lib/Pod/t/htmlview.pod
 lib/Pod/t/htmlview.t
 lib/Pod/t/pod2html-lib.pl
 pod/pod2html.PL

That’s not necessarily sufficient since those files may have chequered
histories of their own that may need tracking. E.g. it turns out that
pod/pod2html.PL had a predecessor called pod/pod2html.SH very early on,
which had been called pod/pod2html before even that. Are they relevant?
Maybe. In this case it turns out the answer is yes: they were not actual
shell scripts, but wrappers that generated a Perl pod2html script whose
code became the installed pod2html became the module plus stub script…
so you don’t want to miss them.

The other files turn out to be boring and obvious. (Thankfully!)

A potential complication is that Pod::Html used to load Pod::Functions.
But it turns out that it never actually used anything from that module…
as far as I can tell. So I’d take the easy way out: simply ignore that.

In the final analysis, you get this:

 export L='ext/Pod-Html/ lib/Pod/Html.pm ...' # all the files listed above
 git filter-branch --prune-empty --index-filter '
 git rm --cached -r -q -- . ;
 git reset -q $GIT_COMMIT -- $L
 ' -- -- $L

This extracts 287 commits from perl5.git, including the far beginning of
history. It leaves everything in the subdirectories it was in while it
was in perl5.git, but I figure that’s better here since the history of
splits and moves among files becomes unintelligible otherwise.

I’ve put up the result:

 https://github.com/ap/opodhtml

Feel free to clone that as a basis for the rest of your work.

I figure a commit that moves ext/Pod-Html/* to the root of the repo is
a clean cut to document “today begins the rest of life for this module”.
I have not done this, figuring I’ll leave it to you to do the honours.

(I want to get rid of that repo once it has served its purpose so please
let me know either way. If you do choose to use it, ideally, do not fork
it on GitHub (since that would record it as a fork), just git-clone it,
change the origin URL in to your existing GH repo, and force-push.)



Aristotle,

Thanks for the git work; it's years beyond my own git fu and that of 
almost everyone I know.


Can you check out that the 'master' branch at 
https://github.com/jkeenan/opodhtml has the correct history?


Thank you very much.
Jim Keenan

Re: Preserving git history across repositories

2013-12-28 Thread James E Keenan


On 12/22/13 11:02 AM, Mark Allen wrote:

I haven't tried this (yet) but it looks like a promising solution...

http://gbayer.com/development/moving-files-from-one-git-repository-to-another-preserving-history/




It turned out that for my purpose I only needed some of the instructions 
at that site.


It took me two tries, but on the second, these commands sufficed:

  540  git clone git://perl5.git.perl.org/perl.git perl4podhtml
  541  cd perl4podhtml/
  542  git remote rm origin
  543  vi .git/config
  544  git filter-branch --subdirectory-filter ext/Pod-Html/ -- --all

At this point, the top-level directory merely contained the directories 
formerly found in ext/Pod-Html/.  And 'git log' and 'git blame' 
indicated that the history had been preserved.  So I quit right there.


After the usual finagling and some renaming, I was able to put a 
repository on github:


https://github.com/jkeenan/opodhtml

It currently has two branches:  'blead', which should represent the 
state of ext/Pod-Html in Perl 5 blead as of today; 'master', which will 
be my master branch for refactoring, adding tests, giving Pod-Html the 
old Phalanx treatment.


Thank you very much.
Jim Keenan

Preserving git history across repositories

2013-12-22 Thread James E Keenan

Is it possible to preserve the commit history of a tree of files as it 
moves out of one repository and into another?


I have two use-cases to consider: one long-term, one short-term.

I.

Long-term:  I am considering trying to move Pod-Html from its current 
location within ext/ in the Perl 5 core distribution to cpan/, i.e., 
have CPAN be upstream.[1]  That really can't be done right away because, 
as the subject line of a META ticket opened by rjbs indicates, Pod-Html 
is a mess.  It's code is poorly structured.  Its test coverage is 
unsatisfactory.  It has many RTs open.  It would be shameful to put it 
out on CPAN in its current state.  I've made one sustained attempt at a 
cleanup[2]; I'm now considering another.  But once it's out on CPAN and 
its principal repository is presumably on github.com, I would like the 
code to retain as much of its history from the Perl 5 repository as 
possible.


How do we do that?

II.

Short-term:  In order to do a thorough cleanup on Pod-Html, I want to be 
free to poke and prod it to my heart's content.  Among other things, 
that's means constant checking of the test coverage with Devel-Cover. 
My preference would be to do that in a repo other than Perl 5, but if I 
did so I would want my changes to go back into Perl 5's Pod-Html so that 
people could get used to it before we moved it to CPAN.  And I would 
want to preserve the commit history in lib/Pod/Htmlm.pm and the test 
suite throughout that round trip.[3]


How do I do that?

Thank you very much.
Jim Keenan


[1] Personally, I don't think Pod-Html needs to be distributed with core 
at all.  But since I know there will be differences of opinion about 
this and since moving it to upstream CPAN is a prerequisite to 
expulsion, moving it to cpan/ is all we need to discuss for now.


[2] At the DuckDuckGo hackathon in Paoli in June/July 2012, rjbs 
suggested I look at Pod-Html and start cleaning it up so that it could 
eventually go to CPAN.  I spent about six weeks working on it.  I myself 
had not used Pod-Html in more than a decade.  I didn't know anyone who 
did.  So I ran out of motivation.  But since I've recently met one 
person who does use it, I'm considering resuming my efforts.


[3] In my 2012 attempts, I moved the existing codebase into a typical 
CPAN structure, then committed that to a new git repo, losing the 
existing history in the process.  I knew much less about git back then. 
 I discussed that with rjbs and he said we could worry about the commit 
history later.  If I embark on this project again, I'd like to get it 
right from the git-go.

RT #36539: Edge cases in find_perl algorithms

2013-09-07 Thread James E Keenan

I am writing to this list concerning a ticket in the Perl 5 bug queue, 
https://rt.perl.org/rt3//Ticket/Display.html?id=36539: Edge cases in 
find_perl algorithms.  This ticket was filed in 2005 by Michael 
Schwern and I have pasted its original post below.


There were several rounds of back-and-forth between Schwern and Ken 
Williams, but the conversation petered out years ago.  We would like to 
close this ticket.  In that RT ticket, I asked whether this issue would 
be better handled by the so-called Perl Toolchain Gang; Leon Timmermans 
agreed that it would.  So I'm now writing to see where this should be 
reported properly.


Is there a particular bug tracker to which I could transfer this 
discussion?  (Note: I have no opinion myself on these issues, so I'm 
only looking for a procedural answer, not a substantive discussion.)


Thank you very much.
Jim Keenan


###
Several places try to convert $^X to an absolute path. CPAN-perl,
ExtUtils::MM_Unix-find_perl and Module::Build all do this. CPANPLUS is
generally not affected as it lets either Module::Build or the shell figure
it out.

The search order is typically some variant on:

my @perls = ($^X, 'perl', 'perl5', perl$]);
my @paths = (File::Spec-path, $Config{binexp});

foreach my $perl (@perls) {
foreach my $path (@paths) {
...
}
}

In the case when $^X cannot be found this algorithm is likely to find not
just the wrong perl but the wrong version of Perl. This is because it
will look for perl before perl5 and perl5 before perl5.00504.
I've had it accidentally pick up perl1 before! The order should be
reversed, looking for the more version specific names before the generic
ones.

Additionally, since 5.6 perl binaries have been named perl5.6.1 and not
perl5.00601. This algorithm does not search for that.

So the @perls list should be:

@perls = ($^X);
push @perls, sprintf perl%vd, $^V if defined $^V;
push @perls, (perl$], perl5, perl);

Module::Build and CPANPLUS do not appear to be vulnerable to this as they
only check for $^X and do not try any fallback filenames. CPANPLUS doesn't
even try to make $^X absolute and leaves that up to the shell or
Module::Build. This is probably safer as if it cannot find your perl it
will yelp rather than silently risk running the wrong version. Changing
CPAN and MakeMaker's behaviors at this point isn't worth it, but a warning
wouldn't hurt.

Additionally, CPAN looks for $^X in the cwd but does it wrong. The
algorithm should be something like this:

# $^X is absolute
if( File::Spec-file_name_is_absolute($^X) ) {
push @perls, $^X;
}
else {
my $first_dir = File::Spec-splitdir($^X))[0];

# $^X is ./path/to/perl or ../path/to/perl. Make it
# absolute using the cwd.
if( $first_dir eq File::Spec-curdir or
$first_dir eq File::Spec-updir )
{
push @perls, File::Spec-rel2abs($^X);
}
}
# else leave $^X alone and do a PATH search

This most closely simulates how a shell finds perl. CPAN.pm makes the
mistake of always looking for $^X in the cwd. This means it can get the
following wrong:

$ ls ./perl
./perl
$ perl -MCPAN -e shell

$^X will be perl. It will look for $cwd/perl and find the one in the
cwd rather than perform a PATH search.

MakeMaker, CPANPLUS and Module::Build do not handle this case at all, 
they do

not look in the cwd. In this case MakeMaker and CPANPLUS will use the
relative $^X, which is dangerous because it will go wrong as soon as
something chdirs.

Module::Build has a heuristic which checks to see if the perl it has found
is the same as the perl it was run with so though it will not find the
perl in the cwd it will not be fooled by ones later in the search. The
heuristic is pretty simple: it compares the output of Config::myconfig.

To summarize:

Change the filename search order to look for the most specific
versions first. (Module::Build and CPANPLUS not affected)

Add perlX.Y.Z to the filename search. (MB and CP not affected)

Add a warning when we cannot find $^X and must fall back to
another filename. (MB and CP not affected)

Look for $cwd/$^X only when $^X is ./perl or ../perl.
(all affected)

Compare myconfig of the found perl and the perl we were run with
to better ensure we found the right Perl. (MB already does this.
CP not affected as it does not search for Perl)

I'll patch up MakeMaker to do this and provide a patch for CPAN.pm. If I'm
feeling gung-ho I might do CPANPLUS and Module::Build, too.

47 matches

Mail list logo