This Week on perl5-porters - 20-26 March 2006
Dave Mitchell converts the regular expression engine from recursive to
iterative.
Topics of Interest
More on "Module::Build" on VMS
Ken Williams got back to Craig A. Berry's patch from last week for
"Module::Build" on VMS, and implemented a new approach to deal with
backtick captures. John E. Malmberg and Craig batted it around for a
while until it looked ready. John wrapped it up as a new version of
"ExtUtils::CBuilder". John noted that there might be issues with older
VMS versions that limit command lines to 255 characters, but decided
to punt the issue for the time being.
Looking good
http://xrl.us/km2x
Building a threads-friendly debugger.
Dean Arnold wrote to say that he was in the process of hacking "ptkdb"
to make it easier to deal with debugging multi-threaded programs. He
had reached the point where it seemed that the most promising way
forward was to change the $DB::single variable to be globally shared
across all the threads.
After the usual admonishments ("You're mad!", "No-one who has ventured
there has ever come back alive!"), Dave Mitchell said that he thought
that it couldn't do much harm, except that it was likely to bring
about a significant loss in performance, as the threads fought amongst
themselves to acquire a lock on $DB::single to read it.
Dean ran a couple of benchmarks and saw that Dave was right, the
resulting performance curve was pretty atrocious (about two orders of
magnitude).
Where hackers fear to tread
http://xrl.us/km2y
Dynamic libraries on AIX 5.1
Last time we heard from John L. Allen, he had been busy doing battle
with 32/64 bit builds with Oracle on AIX. This week he was having
trouble with "Math::Pari", and he and Ilya Zakharevich, "Math::Pari"'s
author, were stuck.
The problem revolved around which libraries were being linked, which
meant that the wrong version of the C language "pow" function being
used. John wanted to understand what was happening and why. H.Merijn
Brand guided him through the twisty mazes of AIX linker techniques.
By the end of the thread John had managed to concoct a method for
making it work, and H.Merijn made a plea for an AIX maven to step in
and take over (and revise) the README.aix file.
Fear and loathing
http://xrl.us/km2z
New "Time::Local" failure
Rafael Garcia-Suarez attempted to upgrade "blead" with "Time::Local"
version 1.12, and saw that the test suite failed. Steve Hay recalled
that this was the result of a bug that he had encountered in LWP's
test suite. Gisle Aas isolated the problem with "Time::Local", and
Dave Mitchell came up with the patch.
Steve wondered whether that patch should be applied only to the Win32
platform. Dave Rolsky, author of the module, responded saying that
there were some problems with integer overflow that gets triggered
only in certain time zones. He said that it was all a bit of a mess,
but that he was going to get it sorted out and release 1.13.
It's about time
http://xrl.us/km22
Revamped UTF-8 caching code
Nicholas Clark checked in some code to rework how UTF-8 caching is
performed.
First, some background: finding the offset of an arbitrary character
in a UTF-8 string can be a difficult proposition, depending on the
number of wide characters encountered in the string. The brute force
method consists of starting from the beginning, and then counting
characters until the desired offset is reached. Depending on the
length of the string, this can be very time-consuming.
To lessen this cost, perl maintains a cache of where wide characters
appear in a string, to minimise the amount of linear scanning
required. A few weeks ago, a bug report revealed that there were some
problems with the existing cache management code.
So Nicholas reworked it a fair bit, adding a "${^UTF8CACHE}" variable
to allow the caching code to be enabled and disabled at will, as well
as a "PERL_UTF8_CACHE_ASSERT" build-time switch to force extra
checking (verifying that the cached and uncached results agree). He
also discovered that the code wasn't taking full benefit of the
gathered information, and tweaked the code to minimise the amount of
linear scanning required.
And accessible from the command-line too
http://xrl.us/km23
see also
http://xrl.us/j9pq
The regexp engine no longer uses recursion
Dave Mitchell announced that he had reworked the regular expression
engine to use an iterative technique rather than recursive. He
achieved this feat by making "S_regmatch()" save its match context on
the heap and restart the main loop, rather than on the stack by
calling itself.
Dave measured that the heap allocation induced a 3% slowdown, but that
this should be avoided by switching to an arena-based allocation
scheme or similar, further down the track.
Before you ask, yes, "/(??{$re}/)" still causes recursion. And Hugo
van der Sanden thinks undoing *that* would be hard.
No more nasty stack overflow bugs
http://xrl.us/km24
Patches of Interest
Upgrading to "threads" version 1.12
Jerry Hedden had delivered a patch to sync "blead" with "CPAN". Dave
Mitchell declined the patch, saying that a patch must never mix
functionality and whitespace formatting changes. If the whitespace is
to be changed (and in general the rule is: never), then that should be
delivered in a separate patch.
Dave also thought that the approach was back to front. The changes
should be applied to "blead" first, and then after the changes have
had time to settle, the "blead" version can be released to "CPAN".
Jan Dubois agreed that he too would prefer it this way around, since
each change is tracked in Perforce, the "perl5-changes" mailing list
gets to hear about it, and e-mail "Message-ID"s from the latter list
make it easier to cross-reference the changes with traffic on
"perl5-porters".
http://xrl.us/km25
Jerry also asked about the definition of "THREAD_RET_TYPE", in the
process of coming to grips with the "threads" code base but received
no answers.
http://xrl.us/km26
and finally got a patch accepted to sync "blead" with CPAN.
http://xrl.us/km27
Serialising closures via "Storable"
David Wheeler wanted to know whether "Storable" could be used to dump
out a closure, bring it back again, and have it work. For instance, to
be able to say
my $var = 1;
my $code = sub { $var };
print $code->();
$code = thaw(freeze($code));
print $code->();
And have it print out "1" twice, rather than once and a warning about
uninitialised values in "print". Yuval Kogman explained how it was
more or less possible, and the pitfalls one would encounter if one
were brave enough to insist on the approach.
Yves Orton, author of "Data::Dump::Streamer", showed how using that
module could probably provide something closer to what David was
after. Joshua realised that one only had to teach "Storable" to use
"DDS" instead of "B::Deparse" and it would Just Work.
Rafael noted that Storable is in the core, but "DDS" is not, although
it should be possible to teach "Storable" to use it if it were
available locally.
http://xrl.us/km28
Watching the smoke signals
Compress/IO/Zlib/t/050interop-gzip.t failure on OpenBSD
Steve Peters tracked down the smoke failures occurring on OpenBSD. It
turns out that OpenBSD's "gzip" behaves differently when gzipping a
zero-byte file:
# Cygwin, FreeBSD, Linux, NetBSD, Solaris, ...
touch /tmp/foo; gzip -c /tmp/foo > /tmp/foo.gz; echo $?
0
# OpenBSD
touch /tmp/foo; gzip -c /tmp/foo > /tmp/foo.gz; echo $?
1
Paul took that into account, but wondered all the same why the smoke
results mentioned "Inconsistent test results (between TEST and
harness)", when one should expect that both TEST and harness should
fail in exactly the same way.
Steve had a hunch that the problem on OpenBSD arose when the file to
be compressed is less than 10 bytes long. Which seems odd, to say the
least. Joshua ben Jore mentioned that he had seen similar problems on
a Ubuntu Linux but hadn't been paying close attention. He promised to
go back and look more closely to see if it was the same error, or
something else again.
One more reason...
http://xrl.us/km29
And the patch to fix it
http://xrl.us/km3a
Smoke [5.9.4] 27593 FAIL(F) MSWin32 WinXP/.Net SP2 (x86/2 cpu)
Steve Hay had a Windows build fail due to a problem with
"ExtUtils::MakeMaker" (that Rafael had recently integrated), and asked
Michael to integrate the patch he made into the "EU::MM" repository.
Earth to Schwern, do you read me?
http://xrl.us/km3b
New and old bugs from RT
"print (...) interpreted as function" occasionally (#4346)
Many moons ago, Abigail reported that the message "print (...)
interpreted as function" appears inconsistently, depending on a
peculiar combination of closing braces, whitespace and/or semicolons.
Steve Peters said that "say" has picked up a similar habit.
The more things change...
http://xrl.us/km3c
More on overloading and reblessing (#34925)
The thread about overloading and reblessing objects continued this
week. Nicholas Clark proposed a solution to scan all the references to
an object and fix them up. Yitzchak Scott-Thoennes pointed out that
such an approach would break the following code:
$a = $b = {};
bless $b, OverloadedClass;
# $a is not overloaded here
Yitzchak admitted that such a construct would probably be quite rare,
and wondered whether it wouldn't be better simply to document the fact
that the initial example doesn't work, with suggested work-arounds.
Nicholas implemented the scan approach in "maint" as change #27512.
http://xrl.us/km3d
B::Lint chokes on simple script (#38771)
Bart Lateur filed a bug report against "B::Lint" (on perl 5.8.7). The
interesting thing is that the program in question was
print for 1 .. 10
Joshua ben Jore, who has recently put a fair amount of work into the
"B::" namespace observed that the problem has been fixed in "blead",
but that it probably still exists in 5.8.8.
http://xrl.us/km3e
"NaN"s on Win32 (#38779)
Rob a.k.a Sisyphus posted a bug report concerning "NaN"s (Not a
Number) on Win32. It seems that there is a compiler issue, which is
that code compiled with VC7 is correct, but VC6 is not.
Dominic Dunlop noted sadly that the best way to fix this bug would be
to add a note to the README.win32 documentation to say that perl
should not be built with VC6. There's an article on the MSDN site that
goes into more detail about floating point comparison issues.
Yves Orton thought that that was hardly ideal, since VC6 has always
been the standard compiler that ActiveState uses for their builds.
Except that Dominic was talking about Microsoft's freely downloadable
compiler, which, is apparently a slightly different beast.
Jan Dubois came up with the best patch, one that works around
compilers that have brain-damaged "NaN" comparison routines. Looking
more closely at the code, Jan realised that perl's handing "NaN"
handling is somewhat uneven. "grok_number()" will set the
"IS_NUMBER_NAN" and "IS_NUMBER_INFINITY" bits as appropriate, but
"sv_2nv()" doesn't bother to check them; it ducks the issue and lets
"atof()" deal with it. He also saw that the cmp.t test that tests how
"<=>" deals with "NaN"s is probably not doing anything meaningful.
http://xrl.us/km3f
In a thread-split elsewhere on the same topic, Jan provided keen
insight into the subject of C run-time libraries on Windows.
http://xrl.us/km3g
Constants with "undef" value deliver arbitrary value at first call (#38783)
Markus Herber posted a bug report dealing with the XS code of "IO-Tty"
that creates constant subroutine with "undef" as a value. Nicholas
Clark understood what was going wrong and promptly supplied a patch
which solved the problem. The patch is a bit of a stop-gap measure,
but it will do for now.
http://xrl.us/km3h
Deep hash of hashes breaks garbage collector (#38786)
Reto Stamm uncovered a lovely bug in the garbage collector. He posted
a program (paraphrased for succinctness here):
my $root = {};
my $h = $root;
$h->{kid} = {} and $h = $h->{kid} for 1..250000
This runs just fine, until the program exits, the garbage collector is
run, the garbage collector exhausts the C stack due to recursion and
the program goes belly up with a segmentation fault.
chromatic thought that simply rewriting "S_hfreeentries",
"Perl_hv_undef", "Perl_sv_clear", "Perl_sv_free2", and
"Perl_hv_free_ent" for good measure to use iteration instead of
recursion would probably do the trick.
*crickets chirping*
http://xrl.us/km3i
"Fatal" doesn't like "readdir()" (#38790)
Tom Hukins filed a report that showed that "readdir" breaks when
"Fatal" is used. ("Fatal" upgrades warnings to to fatal errors).
The trouble is that "Fatal" gets mixed up between scalar and list
context (doesn't everyone?) and throws all the results away. Rafael
thought that a judiciously placed "wantarray" would solve that, but
that in turn would alter the behaviour of something as admittedly
bizarre as
my @useless = open my $fh, 'does.not.exist';
Yitzchak suggested hunting down the exceptions ("select" also seemed
to be a likely candidate) and document their limitations in
conjunction with "Fatal". Joshua thought that this was less than
ideal. If someone was going to go to the effort of hunting down all of
weird special-context builtins to document them (and there aren't a
whole lot), it would take about as much effort to code "Fatal" to make
it do The Right Thing all the time.
Rafael agreed, and kept looking at his inbox for the patch. Joshua
mumbled something about some patches to "B::Lint" he was working on,
and promised to do something about this first.
http://xrl.us/km3j
Joshua went looking at "Fatal", and stumbled across some "AUTOLOAD"
code, and wondered if and how it was used. Mark Jason Dominus
suggested that its purpose was to allow the construct
use Fatal;
Fatal::open();
to work in the same manner as
use Fatal 'open';
open();
Which is either pretty slick, or pretty sick.
Nice to know
http://xrl.us/km3k
Perl5 Bug Summary
1560 open tickets
http://xrl.us/km3m
Right here
http://rt.perl.org/rt3/NoAuth/perl5/Overview.html
In Brief
Dave Mitchell reminded us that "our" variables and package variables
are compiled to the same code internally and as such have identical
performance characteristics.
http://xrl.us/km3n
Philip M. Gollucci reported a bug that manifests itself using
"mod_perl" on FreeBSD. Apparently another one of those "this is the
second time it's broken" bugs. Robin Barker and Gisle Aas committed a
couple of patches, including adding a check in the test suite, so
hopefully we won't see the likes of it again.
Perl_croak and nullch
http://xrl.us/km3o
Jim Cromie reported that "bleadperl" was uncompilable, due to problems
with "Dynaloader" failing. Rafael traced it to the fact that he was
integrating CPAN's "ExtUtils::MakeMaker" 6.30_01 into "blead", and its
handling of "MAN3PODS" was broken. So he fixed that, and "bleadperl"
started compiling again.
Safe to go back in the water
http://xrl.us/km3p
Dan Kogai found an anomaly whilst playing with "YAML::Syck" and
developed an detailed hypothesis as to what was going wrong. As of
summary publishing time, no comments had been made.
How to mangle the SvTYPEs on arrays and hashes
http://xrl.us/km3q
Someone asked how to use Perl to run Visual Basic code and was
directed to Perlmonks.
http://xrl.us/km3r
About this summary
This summary was written by David Landgren.
Information concerning bugs referenced in this summary (as #nnnnn) may
be viewed at http://rt.perl.org/rt3/Ticket/Display.html?id=nnnnn
Information concerning patches to "maint" or "blead" referenced in
this summary (as #nnnnn) may be viewed at
http://public.activestate.com/cgi-bin/perlbrowse?patch=nnnnn
If you want a bookmarklet approach to viewing bugs and change reports,
there are a couple of bookmarklets that you might find useful on my
page of Perl stuff:
http://www.landgren.net/perl/
Weekly summaries are published on http://use.perl.org/ and posted on a
mailing list, (subscription: [EMAIL PROTECTED]). The
archive is at http://dev.perl.org/perl5/list-summaries/. Corrections
and comments are welcome.
If you found this summary useful or enjoyable, please consider
contributing to the Perl Foundation to help support the development of
Perl.
--
"It's overkill of course, but you can never have too much overkill."
