This Week on perl5-porters - 18-25 January 2009
A summary of last week's activity on perl5-porters, by David Landgren.
"Are these names considered to be a public non-changeable interface
by virtue of having been placed in these directories for a while, even
though they were never documented as far as I can tell? Or are these
files considered to be an internal implementation detail of Perl, not
for external use, and hence subject to change or deletion?" -- Karl
Williamson, waging war on two fronts simultaneously, Perl and Unicode.
Topics of Interest
Interesting git-based changelog workflow
Aristotle Pagaltzis mentioned a workflow he had heard about regarding
git. The aim was to ease the burden of producing the Changelog and
perldelta files, which are huge timesinks as it involves sifting
through thousands of commit messages to look for the big-ticket items.
The solution is to create a more incremental effort.
Dave Mitchell said he hoped to produce a one line summary for
perldelta for each day of reviewing and merging patches from blead to
maint.
A number of porters highlighted the difficulty of being able to know
precisely when a bug was fixed, since smokers of CPAN on blead tended
to uncover problems a long time after the "final" patch had landed.
David Golden suggested that a "add-perldelta" tool might be a step in
the right direction to keep track of the changes at an appropriately
high meta-level of changes.
navigating the delta
http://xrl.us/beeifz
disabling "suidperl" in Configure
Nicholas Clark put forward a patch to prevent "suidperl" being
configured in 5.11 and beyond and wondered if there was a better
approach. Andy Dougherty suggested another way, but it had the
disadvantage of increasing the workload for H.Merijn Brand, the
"Configure" pumpking, and would also not produce a useful error
message if someone attempted to try and build "suidperl" in 5.12
anyway.
As it was, H.Merijn applied Nicholas's patch.
it's dead jim
http://xrl.us/beeif3
Later on, Nicholas axed it altogether, reasoning that if there was
some brave maintainer who wanted to resurrect it, all they had to do
was to reverse out one change from their local git repository.
let sleeping dogs
http://xrl.us/beeif5
"mingw libperl" broken
Reini Urban reported a discrepancy between mingw and cygwin as to what
public functions were exported to the world at large. Nicholas Clark
thought that mingw (not exporting three particular functions) was
correct, and it was cygwin that was being lax. And further, modules
outside the core had no business in defining "PERL_CORE" in order to
call functions to which it had no right.
This problem is not limited to the Windows platform. AIX also has a
strict policy on what is and isn't exported.
Dmitry Karasik noted that the problem had been found a long time ago
and a patch had been produced, and wondered why it had never been
applied. This allowed Nicholas Clark to recycle his standard remark
about volunteer workers and companies using Perl not making resources
or money forward in order to improve the situation.
plus ça change
http://xrl.us/beeif7
Reini revived cpan bug #28912 to fix "B::Generate" and deal with its
breakage. He said that three functions, "Perl_pad_alloc",
"Perl_cv_clone" and "Perl_fold_constants" are exported anyway, and so
should be published in the API as such. Nicholas Clark was a lot more
reticent, saying that while it may indeed be useful for them to be
exported, were they documented, testable, stable and supportable.
a gentleman's agreement
http://xrl.us/beeif9
LSB testing issue with "Test::Harness"
Stew Benedict reported a problem with the LSB project's handling of
perl's test infrastructure. They currently use "Test::Harness" version
N-1, and run into problems when that is upgraded to the current
version N.
Yves Orton thought that the best way forward was to deal with the pain
now and upgrade to the current version, even if there were backward
compatible shims available for those who wished to keep working with
N-1.
duty now for the future
http://xrl.us/beeigb
Later on, Stew gave a status report on the second round of Perl
modules included in the LSB. He made a list of modules that had been
pulled out of their core distribution, a list of modules that might
yet be pulled, and a list of those modules that remain, along with
their counts as pre-requisites. Stew explained that it was easier to
pull things out now, possibly to put them in later, rather than the
other way around. Thus, it would pay to be as aggressive as possible.
Yves Orton thought that removing parts of the testing infrastructure
was definitely a bad idea. Nicholas Clark pointed out that while "CGI"
has been in core for a long time, FreeBSD removed it in a recent
release, apparently without much ill effect. He also responded with a
large list of modules that have achieved ubiquity (and thus must be
present) as well as other modules that also must be present even
though we all wish they would curl up in a corner and die.
freebsd decoring++
http://xrl.us/beeigd
Need advice on gotchas on upgrading unicode db to 5.1
Karl Williamson is currently working on bring the Unicode database in
Perl up to date with the latest and greatest, version 5.1.
The first problem he encountered was that the previous update didn't
upgrade the Property Value Aliases, which is a hassle, because
upgrading it now may introduce curious backward compatible bugs.
H.Merijn Brand thought that the problems Karl was raising were
sufficiently far down in the basement that the general public
shouldn't be aware of them, let alone rely on them, and that anything
that allowed Perl to track Unicode with a minimum of intermediate
layers and mappings was a worthy goal. He also put forward his idea of
the list of files in the distribution that the general public was
likely to be aware of, and using.
Karl went ahead after receiving additional clearance from Nicholas,
and CYAed himself with a "these files are subject to change without
notice" header.
"just change it" works for me
http://xrl.us/beeigf
As part of this work, Karl added the Unicode file NameAliases.txt,
which fixes up some of the problems that have arisen over time.
http://xrl.us/beeigh
"use bytes" and regular expressions
Karl is also continuing to wade through the crocodile-infested swamp
of bytes, characters and regular expressions and was wondering what
exactly should be happening when a regular expression was being
compiled when "use bytes" was effect. The main problem being a
complete lack of guidance insofar as the test suite makes no attempt
to specify what should be happening one way or another.
http://xrl.us/beeigj
"glob()" on VMS totally different to Unix
John Malmberg was stunned to learn that "glob('foo')" on VMS is the
equivalent to "glob('foo.*')", as it can lead to some pretty nasty
cross-platform side effects. And even more curiously, there was
explicit code to make things behave differently on VMS if 'foo' were
in fact a directory. He wondered what the appropriate fix would be.
The goal was to get "Module::Build" working correctly on VMS.
Craig Berry wasn't too sure either, since the observed "glob"
behaviour had been like that since time immemorial, and it also
happened to reflect the native behaviour in VMS anyway.
John countered that in actual fact it was the utilities that were
doing the expansion, not the shell.
Peter Prymmer voiced the opinion that things are working according to
spec already, and pointed out that "unlink" only removes the current
version of the file (since the RMS filesystem on VMS allows multiple
versions of the same file to exist simultaneously -- a concept quite
alien to Unix where Perl was initially developed).
Peter also thought that "Module::Build" really ought to be doing the
"opendir"/"readdir"/"closedir" dance, since differences in "glob"
behaviour had been observed in the past among different Unices.
After some more work, John produced a final patch that cleared up the
remaining "Module::Build" test errors. He then spotted a different
error, that Craig Berry identified as being fallout from the Y2038+
code that entered blead recently. Craig promised to get it sorted out
if no-one else wanted to.
it's a bird... it's a plane... it's a directory!
http://xrl.us/beeigm
John forwarded some patches to start clearing this up.
http://xrl.us/beeigo
http://xrl.us/beeigy
"lock()" on a code ref?
Jerry D. Hedden wondered why we check for code references when locking
scalars, since it is meaningless to do so. He wanted to remove it
since he felt it was superfluous. Dave Mitchell suggested putting an
assertion in its place.
next up, signed coderefs
http://xrl.us/beeigq
And thus it came to pass.
let my code free
http://xrl.us/beeigs
A plan for ext/
Nicholas Clark would like to clean up ext/ by hoisting anything that
pertains to the build process *per se*, such as make_ext.pl so that
all that is left are the extensions themselves.
He also wants to flatten the directory structure so that all module
build directories can be globbed with "ext/*".
The Windows and VMS platforms in particular need some refactoring done
so that everything can be driven off a generic "make_ext" build
process. Some modules will need to have a Makefile.PL synthesised as
well. Another problem to address is the dependency chain: ensuring a
modules's prerequisites are built before it is.
Craig A. Berry remembered his classes and noted that ext/ used to
represent modules that contained XS components, and lib/ those that
didn't. While he didn't have a problem with Nicholas's idea, he
pointed out it would be a good idea to look for cruft in %Config and
"installperl" that such a change would create.
And then dual-life modules can migrate from ./lib to ./ext.
Stay tuned next week.
sounds like a... oh wait
http://xrl.us/beeigu
A plan for pod/
Nicholas also had some good ideas about POD. For instance,
pod/perlapi.pod, pod/perlintern.pod and pod/perltoc.pod are currently
maintained under revision control, but could just as easily be
generated dynamically by "./miniperl" during the build. Similarly, the
"man" pages could be built before the install phase, thus avoiding
another step that currently performed in a task run with super-user
privileges.
One benefit is that it reduces the amount of data shipped in an perl
tarball, and helps reduce the speed at which the generated
documentation drifts out of date.
generate the pod bay doors please
http://xrl.us/beeigw
TODO of the week
Make Config.pm cope with differences between built and installed perl
Quite often vendors ship a perl binary compiled with their (pay-for)
compilers. People install a free compiler, such as gcc. To work out
how to build extensions, Perl interrogates %Config, so in this
situation %Config describes compilers that aren't there, and extension
building fails. This forces people into choosing between re-compiling
perl themselves using the compiler they have, or only using modules
that the vendor ships.
It would be good to find a way teach "Config.pm" about the
installation setup, possibly involving probing at install time or
later, so that the %Config in a binary distribution better describes
the installed machine, when the installed machine differs from the
build machine in some significant way.
Patches of Interest
Avoid confusing gcc with -2147483648
Robin Barker wanted to play by gcc's rules in relation to
"LOCALTIME_MIN" but there appears to be problems with having things
work correctly with C89-compliant compilers.
time is not on my side
http://xrl.us/beeig2
Run make_patchnum.pl before configpm
Jerry D. Hedden wanted to ensure that ib/Config_git.pl was available
before "configpm" was right, but this turned out to be surprisingly
difficult in the light of parallel make environments and stamping out
the possibility of race conditions that could arise therein.
first things first
http://xrl.us/beeig4
Fix "mktables" time stamp test
Karl Williamson tried to cut through sedimentary layers of code checks
designed to prevent "mktables" from being run repeatedly during the
build process. Just when he thought he had things solved he ran into
another problem. Rafaël suggested that the remaining problem was also
some sort of bogus optimisation and so maybe it too should be brute
forced.
one thing at a time
http://xrl.us/beeig6
"File::Copy" and permission bits
Charles Bailey stumbled across a pot of tuits and spent them on
learning git and cleaning up "File::Copy", specifically, the
management of permission bits. The result was to make "cp" more POSIXy
and "copy" more native.
Abigail added a few more tests to look at how the copy dealt with
setuid and sticky bits, and pointed out that there was now a potential
world visibility race condition in the code for anyone in a Unix
session with a 022 umask (all too common). Charles thought that
"File::Copy"'s security mandate didn't stretch that far (the paranoid
are invited to look at a hand-rolled solution involving "sysopen").
do you copy
http://xrl.us/beeig8
New and old bugs from RT
updated reproducible perlmodlib.PL with bugfix for "Test::Tutorial"'s
entry (#53000)
Richard Soderberg tossed out some code to attack a TODO, but Rafaël
Garcia-Suarez wasn't able to make use of it, and it prodded Nicholas
Clark to wonder whether our git-configure-fu was sufficiently strong.
Richard said he'd try again some day.
to do later
http://xrl.us/beeiha
"sqrt" without EXPR and "Math::Complex" always 0 (#62412)
Frank Wiegand found a problem with "sqrt" failing to operate on $_ if
"Math::Complex" was loaded. Abigail found a way to use "use
Scalar::Util 'set_prototype'" (suggested by Rafaël and have things
work correctly before and after 5.10.
the root of complexity
http://xrl.us/beeihc
Dies in recursion on some regexes (#62444)
Vany wondered why "(1 x $_) !~ /^(1{2,}?){2,}$/" blew up with a
segfault when $_ became large. Abigail had a look and replied that
"This is to be expected given the regexp."
EWONTFIX
http://xrl.us/beeihe
Fcntl exported constants become tainted under 5.8.9 (#62502)
Mark Martinec discovered that under perl 5.8.9, some O_* and other
constants as exported by module "Fcntl" (or "IO::File") become tainted
under certain circumstances. Both Nicholas Clark and Ben Morrow were
able to confirm seeing the same behaviour but were none the wiser as
to why it was happening.
and you don't make no sense
http://xrl.us/beeihg
Insecure dependency error in sprintf under -T (#62512)
Eric Promislow reported another easy bug to track down: problems with
a "sprintf" format string being tainted... when run inside the Komodo
debugger. Rafaël thought that if it couldn't be reproduced in a stock
perl there wasn't much chance of it being fixable.
heisentaint
http://xrl.us/beeihi
"warnings::register" regressions introduced in 5.10 (#62522)
Peter Rabbitson filed a report about a regression in 5.10 and backed
it up with comprehensive set of tests to circumscribe the problem.
Alas, it attracted no attention.
http://xrl.us/beeihk
Perl 5.8.8 (Tainting) vulnerable to CWE-732 attacks (#62526)
The attack in question is due to the fact that perl will happily allow
a program to "require" a file (and thus, run code) even if said file
happens to be world writable. There was some talk of making taint deal
with this (refuse to load the file, à la openssh) but Abigail asked
how one would go about untainting it.
Yves Orton thought that a new paranoia mode, activated by a new
command-line switch, would be a better alternative.
oh I give up, can I trust you?
http://xrl.us/beeihn
Method interpretation in "A::B->C" (#62584)
Peter Scott noted that what is called by "A::B->C" depends on whether
or "A::B" and/or "A::B::C" are defined. This then leads to program
behaviour that can be difficult to predict.
Eric Brine pointed out that "A::B::->C" is the preferred way of
disambiguating the situation, and Ben Morrow explained why it is
unlikely that a favourable resolution will made of the matter.
there's one right way to do it
http://xrl.us/beeihp
Maximum string length with "substr" (#62646)
skylar reported running into grief with "substr" being asked to chop
away at a string larger than 2147483648 bytes (give or take a byte).
Nicholas Clark explained that the 1998-vintage implementation uses a
32-bit quantity which is clearly starting to become a limit in the
21st century.
sign of the times
http://xrl.us/beeihr
Override documentation (#62648)
Jerrad Pierce would love to see "override" have its own documentation,
or at least be able to get to it easily from "overload", which does
have its own documentation.
overboard
http://xrl.us/beeiht
UTF8 failure with "sprintf ()" (#62666)
H.Merijn Brand has managed to get the UTF-8 cache confused and panic
with nothing more than a "sprintf".
doctor it hurts
http://xrl.us/beeihv
Perl5 Bug Summary
http://xrl.us/beeihx
http://rt.perl.org/rt3/NoAuth/perl5/Overview.html
New Core Modules
podlators 2.2.2
Russ Allbery pushed out a new version, the main improvement being
the ability to deal correctly with verbatim paragraphs that
contain lines with only whitespace.
http://xrl.us/beeihz
Encode 2.27
Dan Kogai released a new version of "Encode" that clears up a
number of reported bugs.
http://xrl.us/beeih3
Module::Load 0.14
Jos Boumans synced the latest version of "Module::Load" that
corrects a reported bug and streamlines the test suite.
"Module::Load::Conditional" was also upgraded to clear up some
bugs in the test suite itself, smoked out by Win32 and VMS.
sweet
http://xrl.us/beeih5
Archive::Tar 1.44
Jos also released a bug fix for "Archive::Tar".
http://xrl.us/beeih7
In Brief
Some consting goodness for time64 was applied.
http://xrl.us/beeih9
Renée Bäcker thought it would be great to show -E in error message
when called with -E, and on the surface it is. Except it breaks tests
like "$0 eq '-e'" and to Do The Right Thing in light of that was so
hard that all hope was abandoned.
http://xrl.us/beeiib
Slaven Rezic improved the warnings if forks fail in "Perl_my_popen".
http://xrl.us/beeiid
Karl Williamson is getting close to landing code to solve the problem
of the Unicode ordinal range 128-255 and so he delivered a skeleton
legacy.pm to bring lay the way for genuine Unicode goodness.
http://xrl.us/beeiif
Ben Morrow spotted some curious behaviour when calling "-t", "-T" on
IO refs and a variation on the theme that tripped an assertion
failure. He promised to write a patch after he had audited the rest of
the codebase for uses of "isGV()".
http://xrl.us/beeiih
Ricardo Signes wrote a rough patch to remove indirect method calls
from documentation. This was applied, but he admitted that other
examples no doubt lie around in obscure corners of the
documentation.(I thought chromatic wrote an exhaustive patch for this
a couple of years ago --ed).
http://xrl.us/beeiij
Reini Urban pulled off a gnarly trick to use MSVC6 Project files to
build perl.exe, which reduces the edit-compile-debug cycle on windows
down to keypresses "F7"-"F5".
faster debugging
http://xrl.us/beeiim
Last week's summary
There was none. I needed a break. As much as I enjoy doing these
summaries, it's a huge chunk of tuits each week.
About this summary
This summary was written by David Landgren.
Weekly summaries are published on http://use.perl.org/ and posted on a
mailing list, (subscription: perl5-summary-subscr...@perl.org). The
archive is at http://dev.perl.org/perl5/list-summaries/. Corrections
and comments are welcome.
If you found this summary useful, please consider contributing to the
Perl Foundation or attending a YAPC to help support the development of
Perl.
--
it's an anthem in a vacuum on a hyperstation, day dreaming days in a
daydream nation
