This Week on perl5-porters - 24-30 April 2006
Lots of work consting, compiling and testing perl this week.
Topics of Interest
Running hard just to stand still
Nicholas Clark commented on Jerry D. Hedden's cry for help concerning
the slow application of his "threads" patches, and wondered what do
other open source languages, such as Lua, Python, Ruby and Tcl cope
with the day-to-day drudgery of sifting through bug reports,
shepherding the changes to the code base, and generally making sure
that the project keeps moving forward.
Holding the fort
http://xrl.us/k7mm
New taint tests cause threaded FreeBSD to hang
Nicholas Clark saw that some new taint tests were causing perl to hang
on recent versions FreeBSD, and wondered what to do about it. Steve
Peters reproduced the problem on OpenBSD as well.
On the other hand, Dominic Dunlop gave Mac OS/X Darwin a clean bill of
health (despite the fact that it is derived from FreeBSD).
Anton Berezin (the man for Perl on FreeBSD) suggested linking with a
different threads library. Everything worked when Nicholas tried that,
leading him to conclude that hints/freebsd.sh could do with an update.
I'll give you a hint
http://xrl.us/k7mn
Rebuilding XS modules for threaded perl
Peter Scott wanted some "best practices" advice on how to build a
threaded and non-threaded perl, and have them coexist peacefully,
falling back on the unthreaded perl for all the ancillary tools such
as "perldoc" and "perlbug". Specifically, and hence the title of his
message, he wanted to know if he could get away with using the same
compilation of XS modules for both perl instances.
Jan Dubois clarified the situation, explaining that threaded and
unthreaded perls are binary incompatible, so modules with XS
components need to be recompiled, and need to be installed in separate
locations (so that the associated perl picks up the right libraries
for itself).
Andy Dougherty welcomed Peter's suggestion to patch the documentation
with his findings.
We build them here, we build them there, we build these modules
everywhere
http://xrl.us/k7mo
Using "/" and "X" in "unpack"
Peter Dintelmann had a question concerning the ability to walk back
the stream with "unpack". He was trying to understand how to do it but
was struggling with the paucity of information. Sadahiro Tomoyuki
explained exactly how it worked, even going so far as to examine the C
source and realise that the error checking could do with a little
tightening up.
http://xrl.us/k7mp
Tomoyuki's explanation was just what Peter needed.
http://xrl.us/k7mq
Refitting t/op/*.t files to use "test.pl"
David Landgren started to work on refitting t/op/pat.t to use
"test.pl", which would make it simpler and easier to add new tests. He
had a couple of questions, mainly concerning how to modernise some
fairly archaic tests.
Andy Lester was thrilled. Nicholas Clark recalled that one useful
feature that test.pl does not have but should, is the management of
temporary files. Being able to create them on demand, and clean them
up automatically at the end of the script. Apparently this sort of
make-work code is scattered throughout the test suite.
Into the third millenium we go
http://xrl.us/k7mr
David also had a look at a section in t/op/pat.t for patterns that
warn when compiled and wondered whether a particularly intricate
collection of closures was in fact a no-op. No takers.
The sound of chain-saw warming up
http://xrl.us/k7ms
The new version was finally delivered. It generated a lot of
discussion, but alas, remained unapplied.
http://xrl.us/k7mt
Use of "eval" in t/op/append.t
David then looked at t/op/append.t and saw that some of the test
results were "eval"'ed (rather than the thing being tested), and also
wondered why the tests were tested with regexps instead of comparison
operators. Sadahiro Tomoyuki explained that the tests were there to
see if concatenation stops on "\0" (binary zero), and regexps were
used instead of comparison operators, in case the comparisons also had
an equivalent bug which stopped them scanning on "\0".
Which led David to wonder if there wasn't a just as likely possibility
for regexps to also stop on "\0", in which case something like
"rindex" might be better, since it would never encounter any dubious
characters.
Tomoyuki thought that the "eval" was no longer necessary (although it
does mean that things happen at run-time rather than at compile-time,
which may have an influence).
http://xrl.us/k7mu
Elsewhere, David looked at the tests in t/op/grep.t and was surprised
to discover that some tests can pass, and print "ok", or they can
fail, and print... nothing.
As long as everything works
http://xrl.us/k7mv
Perl on Win98
David Bree asked for advice on compiling perl on Windows 98 using
MinGW. He had in fact compiled it, but was wondering whether it was
safe to install, since about five percent of the tests failed. Which
doesn't seem to shabby. Unfortunately, no-one was able to offer any
advice.
Maybe if we knew what had failed
http://xrl.us/k7mw
Merging "wince" and "win32" directories
Yves Orton started to tackle the long-talked-about much-awaited merge
of the "wince" and "win32" directories, that contain the
platform-specific extensions for the Windows CE platform on one hand,
and Windows in general on the other.
Vadim Vladimirovich Konovalov (hereafter referred to as Vadim)
promised to add some improvements to the WinCE side, and was pleased
to see that someone was taking the necessary time to clean up the
source tree. He also mentioned the existence of a "perlce.sf.net"
project, but that it is pretty calm at the moment.
Yves's approach was to create a new "win" directory, starting with the
contents of "win32", and then see what happened as he pushed and poked
bits of "wince" into it, until there is nothing left over.
Where do you want Perl to go today?
http://xrl.us/k7mx
t/op/stat.t failures on "z/OS"
Mohammad Yaseen reported the difficulties he was having with 5.8.7 on
z/OS (the IBM mainframe OS). In this instance, t/op/stat.t was failing
with messages about /dev/tty, depending on whether the test was run in
single-user or multi-user mode.
Jim Cromie asked for some specific details, to help understand what
single-user and multi-user means in the context of z/OS.
http://xrl.us/k7my
Clawing scarce bits back from the magic vtable structure
As Nicholas happily integrated a change from "blead" to "maint"
concerning bitmask flags on a field in the magic vtable structure, he
realised that there weren't a whole lot of spare bits that remained,
and even though code with the first definitions of those flags has
been out in the wild for some time, he wondered if some of the newer
bitmask flags could be reeled back in.
Dave Mitchell thought that three flags could be freed up, since they
were either marked as being wildly experimental, or not documented at
all.
Peak bits
http://xrl.us/k7mz
"taint" and "fork" on Win32
David Golden had been taking a VanillaPerl on Win32 for a spin, and
noticed that "Test::WWW::Mechanize" kept dying a horrible death. Carl
Franks and he narrowed it down to a delightfully simple snippet: "perl
-Te fork" and then searched the bug database but found nothing
similar. So he wanted to know whether to file a bug report, or whether
the failure mode was common knowledge. No takers.
Can't have your fork and taint it
http://xrl.us/k7m2
Is a "FileHandle" "IO::Seekable"?
Adam Kennedy related the problems he was having after having taken
over the maintenance of "Archive::Zip". In a nutshell, he needs to
have seekable file handles. And yet it turns out that at one point in
the test suite, a file handle winds up being connected to a pipe,
which definitely is not seekable. So Adam wanted to know if something
was truly seekable, as opposed to merely having a "seek" method.
The best Randy W. Sims could come up with was a kluge with file test
operators.
Seeking advice
http://xrl.us/k7m3
Patches of Interest
The continuing saga of the improvements to the regexp engine
Yves Orton repatched his trie work to simplify the sharing of the code
base between "blead" and "maint".
Nice trie
http://xrl.us/k7m4
5.8.0 could be supported as well, if there was some way to fake
"SAVEBOOL" via ppport.h.
http://xrl.us/k7m5
Reini Urban showed what needed to be done to get it to compile with
the ActivePerl source.
http://xrl.us/k7m6
http://xrl.us/k7m7
"const"ing goodness
Andy Lester took a crack at adding some "const"ing goodness to doop.c,
but Sadahiro Tomoyuki pointed out a mistake. So Andy took had another
go at it. Not applied, as far as I can see.
http://xrl.us/k7m8
He then killed a modal boolean that controlled the behaviour of
"S_glob_2inpuv" in sv.c, preferring to split the functionality into
two different routines. Applied.
http://xrl.us/k7m9
Andy then carried on, inlining a couple of static routines in perl.c.
Also applied.
http://xrl.us/k7na
Andy found reasons to use "NOOP" macro, part one.
http://xrl.us/k7nb
Moving right along, he removed some unused (or rather, useless) casts
in regcomp.c and regexec.c, which prompted Yves Orton to comment that
when he was working on adding trie support to the regexp engine, he
found those casts to be quite helpful in figuring out what was going
on.
http://xrl.us/k7nc
He had more luck removing unused context in sv.c.
http://xrl.us/k7nd
And another context parameter in pp_ctl.c, with some "const"ing
goodness thrown in for good measure.
http://xrl.us/k7ne
And some more places in which he could tweak the usage of
"SvREFCNT_inc" calls.
In mg.c
http://xrl.us/k7nf
and pp_ctl.c
http://xrl.us/k7ng
And some compiler warnings in perlio.c.
And that's what Andy did this week
http://xrl.us/k7nh
Using "v?snprintf"/"strlcpy"/"strlcat" when useful
Jarkko Hietaniemi patched a number of files to use the C run-time
equivalent string routines (that have been vetted as working correctly
at build time) rather than alternative, less safe routines. Jarkko
asked for eyeballs to look carefully at the patch in case there were
any glaring errors.
Steve Peters took the patch for a spin on the platforms he had lying
around. In the meantime, Jarkko found a section in pp_ctl.c that he
had missed.
H.Merijn Brand wondered what the status was concerning these routines,
since on some proprietary platforms, the C source to these routines
had some fairly fussy copyright noticed attached. Steve said he had
dug up the source to the Perl routines from an old copy of "inn", and
believed that they were in the public domain.
Russ Allbery said that he authored "inn", and that Steve was perfectly
entitled to having taken them for use in perl. This prompted H.Merijn
to suggest removing the "#ifdef" trickery surrounding the calls to
these routines, and instead use them everywhere, and bundle Russ's
public domain code to be used on platforms that don't provide them.
New uses for old
http://xrl.us/k7ni
Symbian, "dVAR" and "-DPERL_GLOBAL_STRUCT_PRIVATE"
Jarkko continued on his "dVAR" quest with
"-DPERL_GLOBAL_STRUCT_PRIVATE" to move Perl C run-time data to the
heap. As he marked a previously "const" variable as non-constant, Andy
Lester insisted on a full report. Jarkko explained what had happened,
and Andy thought of another approach. Jarkko urged Andy to proceed
cautiously, as it concerns only the Symbian port as is also rather
fragile. Any changes must be tested with "PERL_GLOBAL_STRUCT_PRIVATE"
and "PERL_GLOBAL_STRUCT"
Do not meddle in the affairs of Jarkko
http://xrl.us/k7nj
And then sent in another "dVAR" patch, this time to make perl's
"malloc" happy, plus some corrections to fix up some signed/unsigned
mismatches that had crept in when no-one was looking.
http://xrl.us/k7nk
"gcc -ansi -pedantic" noise reduction
The patches kept rolling in from way up north, this time cleaning up
the worst of the damage when one lets "gcc -ansi -pendantic" loose on
the source (it screams a lot). He recommended that porters build perl
from time to time using these two compiler options, and taking the
appropriate action based on the fallout. Steve Peters applied the
patch. Andy Lester was pleased.
I dare you
http://xrl.us/k7nm
Another handy tip for the code police
And wrapping up, Jarkko offered a handy tip for people who enjoyed
torture-testing the compilation of the codebase, to use the "-O" "gcc"
compiler switch in conjunction with "-Wall" (warn about everything).
It turns out that some code checks don't kick in until the optimizer
is brought to bear on the code. Andy Lester wanted to know if
different "-O" levels kicked in successive classes of warnings.
Cheap trick
http://xrl.us/k7nn
Andy posted the armada of compilation switches that he uses to compile
the source.
-Wextra is good for a laugh, too
http://xrl.us/k7no
Patching various t/op test files to use test.pl
David Landgren sent in a some patches to for files in t/op to use
test.pl, instead of their home-grown each-one-different methods. One
main benefit is that now the tests have names, so when test 43 fails,
it becomes trivial to locate it in the source file.
In the process, he also found an nice bug of a string comparison using
numeric equality in t/op/loopctl.t.
t/op/auto.pl
http://xrl.us/k7np
t/op/defins.pl
http://xrl.us/k7nq
t/op/delete.t
http://xrl.us/k7nr
t/op/glob.t
http://xrl.us/k7ns
t/op/hashwarn.t
http://xrl.us/k7nt
t/op/loopctl.t
http://xrl.us/k7nu
t/op/sort.pl (or what I did over my lunch hours this week)
http://xrl.us/k7nv
(Summariser's note: At first I wasn't sure how I was going to
summarise this, but Jim Cromie sort of forced my hand).
It's all Jim's fault
http://xrl.us/k7nw
Clarifying the "Fatal.pm" documentation
Tom Regner had a discussion with Ilya Zakharevich on
"comp.lang.perl.modules" about "Fatal", and the result of the
discussion was a POD patch. Unapplied.
http://xrl.us/k7nx
Watching the smoke signals
Smoke [5.9.4] 28020 FAIL(M) MSWin32 Win2000 SP4 (x86/1 cpu)
Abe Timmerman identified this failure as the result of the Windows
part of the code base not finding change #27992 to its taste.
http://xrl.us/k7ny
Smoke [5.9.4] 28009 FAIL(Xm) irix 6.2 (IP22/1 cpu)
In this report, it appears that something had gone wrong with the
"gcc" C run-time library during the configuration phase.
http://xrl.us/k7nz
New and old bugs from RT
Many bizarre bugs
Animator discovered that the bizarre code leads to bizarre ARRAY
assignment (bug #9374) had been fixed in 5.8.8 and 5.9.3. Andreas
Koenig identified change #25828 as being the one responsible for the
fix.
http://xrl.us/k7n2
In a similar bug report (#3112), Animator simplified the snippet that
causes a bizarre copy of ARRAY in "last". No-one attempted a fix for
the time being,
for (1) { push @a, last; }
http://xrl.us/k7n3
and bug #9540 still produced "Bizarre copy of HASH" errors,
@{my %self; %self}{1} = 1;
http://xrl.us/k7n4
while #22238 continued to give you a Bizarre copy of HASH in leave
@{ my %self; %self }{1} = 1;
http://xrl.us/k7n5
and for the last in a series of bizarre bug reports, Animator noted
that bug #36229 (Bizarre copy of IO) is a duplicate of an earlier bug
(#3314).
$a = ${*STDOUT{IO}}
http://xrl.us/k7n6
So don't do that.
"unpack" fails on utf-8 strings (#33734)
Nicholas Clark returned to the problem raised by Marc A. Lehmann back
in early 2005 and explained what he had done with Ton's code, which
fixed up the problem in a perhaps non-backwards-compatible manner.
Glenn Linderman wondered whether the new "pack"/"unpack" semantics
could not be bundled into a loadable module and made available via a
feature pragma.
Marc Lehmann thought that "unpack" should simply be fixed (the problem
is how "unpack" behaves in the face of UTF-8 data) once and for all.
If the fix is too major, then it will appear in 5.10. If it can be
arranged to not break anything already relying on the current
behaviour, then it can be fixed in 5.8.
http://xrl.us/k7n7
Perl build problem (failing "File::Basename") (#38891)
Dominic Dunlop nudged this bug report, requesting more information. No
response received as we went to the press.
http://xrl.us/k7n8
double free detected (#38943)
Ulrich Windl coaxed a "*** glibc detected *** double free or
corruption" out of the debugger. Neither Dave Mitchell nor Nicholas
Clark were unable to reproduce the problem on the perls and platforms
they had on hand.
Michael Shroeder remembered that Ulrich had already reported a similar
problem with "Term-ReadLine-Gnu", and wondered if the two problems
were related, and Ulrich concurred that it appeared to be a problem
concerning the C "libreadline" library in his SuSE distribution.
SEP
http://xrl.us/k7n9
debugger can't breakpoint until module loaded (#38966)
Albert Cahalan was debugging some unfamiliar code, and wanted to set a
breakpoint in some as-yet-unseen code, and wondered why the debugger
was refusing to do so, given that "gdb" will prompt for more
information when in similar circumstances when debugging C code.
Rafael wondered if "b postpone *subname*" would do the trick. Albert
agreed that that was just the ticket, but thought that the debugger
could be tweaked to ask whether one wished to postpone the breakpoint
if the routine was currently unknown to the debugger.
People missed the point, and started making fun of Albert, suggesting
that if he didn't know what he wanted to debug, he should connect the
debugger to "Symbol::Approx::Sub" in order to breakpoint on a routine
that most closely matched.
Albert replied calmly with an example involving "gdb" that made
perfect sense. The main obstacle to adding this functionality is that
everyone suddenly appears to be very busy Doing Other Stuff when it
comes to patching the debugger.
Wouldn't it be nice
http://xrl.us/k7oa
t/op/getppid.t fails in a Solaris Zone (#39010)
Mark Suter reported a failure in t/op/getppid.t, that is due to a
changed assumption concerning the owning pid of an orphaned process
(hint: it is not 1). It turns out that Sun had been kind enough to
loan the porters a Solaris machine with zones set up, and the bug had
been found and fixed four weeks ago.
Coming to a 5.8.9 near you
http://xrl.us/k7ob
Attempt to free unreferenced scalar (#39012)
Salvador Fandiño filed a bug report with a nice short snippet showing
how to provoke an "Attempt to free unreferenced scalar" error. After a
bit of thought, Dave Mitchell showed how it could be simplified even
further. And then after a bit more thought, patched scope.t to fix the
problem, and rolled the snippet into a test for t/op/local.t.
All in a day's work
http://xrl.us/k7oc
Segfault in functional streams/bug in closure allocation? (#39017)
"[EMAIL PROTECTED]" tried to create an infinite stream with closures
but the only thing he managed to create were core dumps. Dave Mitchell
explained that it wasn't actually closure-related, but yet another
manifestation of the recursive implementation used to free recursive
data structures.
Make a deep structure, eat up all your C stack
http://xrl.us/k7od
Christian returned with a follow-up, showing how he had managed to get
it to almost but not quite work (but at least it no longer crashed).
http://xrl.us/k7oe
The thread returned under the title of Garbage collection issues later
on, where Christian wondered about a couple of issues he was having
with his code. Dave Mitchell explained precisely what was happening,
and then asked Christian to explain what he was really trying to do.
Once Dave understood what Christian wanted, he realised the proposed
code was far too complex, and sketched out an alternative approach,
with negligible memory requirements to boot.
Yves thought that what Christian was trying to do was shoehorn Lisp
idioms into Perl, and suggested he take a look at Mark-Jason Dominus's
Higher Order Perl.
http://xrl.us/k7of
HOP to it
http://hop.plover.com/
"Pod::Html" generates incorrect href's in some html anchors (#39020)
"bgstewart" filed a problem with some incorrectly generated HTML from
POD. Steve Peters explained that it had already been fixed. Tels
thought there was a bug in the source POD, and backed up his assertion
with a report from "podcheck". Russ thought the POD was valid, just
somewhat unusual.
Merely plain, not simple
http://xrl.us/k7og
"Tie::Memoize::EXISTS" not caching the value (#39026)
This was reported as a bug, but no-one commented on it.
http://xrl.us/k7oh
Perl5 Bug Summary
Another decline in the number of tickets noted this week.
13 created + 27 closed (yay!) = 1525
http://xrl.us/k7oi
What outstanding bugs
http://rt.perl.org/rt3/NoAuth/perl5/Overview.html
New Core Modules
* The "dor" ("//") patch was refreshed by H.Merijn Brand as
dor-588-27985.diff, following some tweaks to perly.y.
http://xrl.us/k7oj
* "Module::Build" 0.28 released by Ken Williams.
http://xrl.us/k7ok
This followed a patch to "blead" from Steve Hay to fix up some
problems on Win32 following the upgrade of "Test-Harness".
http://xrl.us/k7om
In Brief
The "select((select(OUTPUT_HANDLE), $| = 1)[0])" thread evolved into a
discussion about sequence points in Perl.
http://xrl.us/k2zs
Rafael looked at Marcus Holland-Moritz's patch to clean up 212
warnings emitted by "gcc-4.2" and applied 4 out of 5 parts, the last
part seeming to be superfluous. Marcus commented in more detail on the
reasons behind the patches.
http://xrl.us/k2yk
Craig Berry made a belated followup to Ken Williams's "File::Spec"
changes for VMS, giving it a clean bill of health.
http://xrl.us/k2zr
Nicholas dropped some of recent tweaks to "blead"'s version of
"Test::Harness" in order to keep it perfectly synchronised with the
latest CPAN version.
Stamping out gratuitous differences
http://xrl.us/k2y4
The problem concerning multi-line attributes and "Attribute::Handlers"
noted by Bas van Sisseren (#38475), and its required fix to toke.c was
applied by Rafael, and the example was turned into a regression test.
Won't see that again
http://xrl.us/k7on
Rafael also applied the fix suggested in opening "|-" triggers
unjustified taint check. (#38709)
http://xrl.us/k7oo
Jerry D. Hedden continued to sync "threads" with "blead". Applied.
http://xrl.us/k7op
And then he consolidated some "XS" functions. Also applied.
http://xrl.us/k7oq
Here's one that was missed in "maint":
http://xrl.us/k7or
And another that was caught:
http://xrl.us/k7os
Nicholas Clark revived the "POSIX" build fails in "bleadperl" and
"RTMAX" patch bug (#36951). David Dyck confirmed that it is still
alive and kicking,
http://xrl.us/k7ot
and closed out the utf8 overload stringify bug (#34297), saying that
it was fixed in "blead" and that he hoped to backport it to 5.8.9.
http://xrl.us/k7ou
He then showed how "uc"/"lc"/"ucfirst"/"lcfirst" fail on typeglobs on
5.8.8 in bug #39019. The bug doesn't manifest itself on "blead", but
that is because it's typeglob implementation is quite different these
days.
http://xrl.us/k7ov
Joshua ben Jore wondered how to interpret the meanings of the MAD
keys.
No-one else knew, either
http://xrl.us/k7ow
Nicholas Clark wondered why "lc" and friends don't taint empty strings
when locales are in effect (bug #39028). Sadahiro Tomoyuki gave his
thoughts on the matter, explaining that perl was probably Doing The
Right Thing.
http://xrl.us/k7ox
Abe Timmerman fixed up "snsprintf" and "vsnsprintf" problems for MSVC
on the Windows platform.
http://xrl.us/k7oy
About this summary
This summary was written by David Landgren.
If you want a bookmarklet approach to viewing bugs and change reports,
there are a couple of bookmarklets that you might find useful on my
page of Perl stuff:
http://www.landgren.net/perl/
Weekly summaries are published on http://use.perl.org/ and posted on a
mailing list, (subscription: [EMAIL PROTECTED]). The
archive is at http://dev.perl.org/perl5/list-summaries/. Corrections
and comments are welcome.
If you found this summary useful or enjoyable, please consider
contributing to the Perl Foundation to help support the development of
Perl.
--
"It's overkill of course, but you can never have too much overkill."
