This Week on perl5-porters - 10-16 July 2006
"Kindly avoid inflicting upon this one the same fate shared by so
much other documentation produced by a patchwork of well-meaning but
disparate contributors seeing more the trees than the forest as they
continually apply peephole optimizations that inadvertently compromise
the overall integrity of the work, where the presentation eventually
more resembles an illegible patchwork quilt of no especial application
or intended purpose than it does a dedicated garment of one cloth
woven." -- Tom Christiansen, at the height of his form.
Topics of Interest
Adding examples to the core documentation
Gabor Szabo continued his quest to add examples to the core
documentation, based on feedback he had received giving training
lessons. First up, he wanted to rewrite "perlfunc" to use the modern
approach for opening files (using lexical file handles and the three
argument form).
While no-one could dispute that this was a good idea, Nicholas
objected to the fact that it meant that new stuff was being added but
no old stuff was being pruned away.
Other changes that Gabor wished to make involved adding more examples
to the documentation. Again, Nicholas wondered why people wouldn't
have the reflex to toss off a few one-liners to see how things worked.
H.Merijn Brand suggested creating an examples directory. Gabor liked
the idea, but regretted that the usual installer tools ("CPAN" and
"CPANPLUS") do not install the "eg/" directories, making them all but
invisible to mere mortals.
Opening gambit
http://xrl.us/o7qy
Gabor then continued with examples for "index"/"rindex". Given that
the debate about "open" was not resolved, this was not applied.
http://xrl.us/o7qz
He then added a brief discussion of "use strict" in "perlintro" and
replacep5psummd the use of $a and $b by some less-special variables.
H.Merijn
liked it enough to apply it.
http://xrl.us/o7q2
"perlintro" also received another two batches of changes, here:
http://xrl.us/o7q3
and here:
http://xrl.us/o7q4
Gabor was briefly confused as to whether he should be patching "blead"
or "maint". H.Merijn Brand and Yves Orton explained that he should
patch "blead", and that the "maint" pumpking (currently Nicholas
Clark, but will be Dave Mitchell in the future) will backport patches
to "maint" as appropriate.
http://xrl.us/o7q5
So Gabor patched "perlhack" in order to point future porters to the
right thing to hack.
X marks the spot
http://xrl.us/o7q6
Gabor then gave perlopentut.pod a work-over to begin recommending the
use of lexical file handles rather than typeglobs. H.Merijn Brand had
a number of issues with the proposed changes, and the thread continued
on for quite some time. People seemed to be in agreement that some
sort of improvement was needed, but the hard question was what was
truly essential for a tutorial, and in what order to present it.
Opening night
http://xrl.us/o7q7
After digesting the suggestions that everyone put forward, Gabor
produced a second cut. After an initial false start, Tom Christiansen
explained that from a tutorial perspective, omitting parentheses and
preferring "or" over "||" is incorrect. Having taught thousands of
people Perl, Tom has come to the conclusion that parens and
low-precedence "||" *helps* people come to grips with the language.
Omitting needless parentheses comes further up the curve of Perl
mastery.
http://xrl.us/o7q8
On the correct use of "my_strlcpy"
Steve Peters had tried to replace some uses of "strcpy", (which is
deemed "unsafe" in that it can copy without limit), with the safer
"my_strlcpy". Except that they don't return the same thing, which
caused Steve Hay's smoke machine to have a fit. The first Steve was
rather embarrassed and apologised.
On the plus side, this revealed a place where more tests are needed.
Copy right
http://xrl.us/o7q9
"Hash::Util::FieldHash" *versus* global destruction
Joshua ben Jore wanted to know whether "Hash::Util::FieldHash" makes
inside-out objects safe in the face of object that defines a "DESTROY"
method. Anno Siegel explained that everything should be fine during
the normal course of events, but wasn't too sure whether global
destruction (what happens when the interpreter is being taken down
just before the perl process expires) changes the rules so much that
this no longer holds true.
Joshua spelt out some of the potential problem scenarios he envisaged,
and how best to deal with them. Anno thought that the proposed
work-arounds were all rather bletcherous, and promised to think about
it to see whether he could come up with a good solution.
The underlying problem is that inside-out objects reveal an aspect of
global destruction that was never apparent before, and if you're
really unlucky, it could bite your code.
Unintended consequences
http://xrl.us/o7ra
Confusing "SvCUR" and "SvLEN" documentation
Yves Orton read the documentation concerning these two functions and
had difficulty in reconciling it with what "Devel::Peek" was telling
him. Jan thought it made sense to him, and so he wondered what it was
that Yves didn't understand. There ensued a long conversation between
Jan and Yves that discussed the matter in more detail.
Yves finally understood the issue, and suggested that Jan write up
some documentation to clarify the matter. Jan hoped that someone else
could summarise the issue instead (who, me?) and Yves said that he
would take a stab at it, and allow Jan to make fun of it until it all
made sense.
String theory
http://xrl.us/o7rb
CPAN contributor feedback
Responding to the itchy thread from last week, David Muir Sharnoff
explained that his number one gripe with the core was its less than
complete support for tied arrays, magic on tied variables, and the
lack of a "GETREF" method for ties. All of which would no doubt make
his life easier.
Not to mention closing a bug or two
http://xrl.us/o7rc
Code that is only executed once
David Nicol proposed adding a 'once' statement modifier, that, after
having been executed, would then delve into the optree and excise the
statement out of the optree by rethreading the prior opcode to the
next opcode.
Rafael agreed that Perl 6 has a similar construct, but that there were
no plans to add it to 5.10. He was happy that "state" variables
resolved the ugly "my $x if 0" hack, and thought that "UNITCHECK",
from the "perltodo" list was more worthy of burning up some tuits.
COBOL has an ALTER verb, after all
http://xrl.us/o7rd
Removing the succeed path from the super-linear cache code
This is an optimisation in the regular expression engine that, as far
as Dave Mitchell can tell, is never exercised by the test suite, and,
try as he might, was unable to concoct any example that could. So
after talking things over with Hugo van der Sanden, he decided to fire
up the chain-saw.
So if your fancy patterns now starting taking longer than the
scheduled moment of the heat death of the universe, you know who to
blame. Hugo had hoped that some sort of instrumentation under a
debugging build might be added to the engine to allow the measurements
of different optimisation strategies. Dave wasn't planning to,
although he figured it would probably be doable, even if he didn't
know right now what shape or form it would take.
Look for a patch tomorrow morning
http://xrl.us/o7re
Comments on recent API changes to "threads"
Artur Bergman, one of the original authors of the "threads" module,
discovered all the work that had been done recently, and objected to
the change that allowed "exit" to end a thread silently (and not the
entire process), saying that it blurred the distinction between
threads and processes.
He also spotted a section in the code that dealt with passing signals
to threads, noting that it was not thread-safe. Jerry wasn't sure
whether this was a true, and asked for a patch to help him understand
the situation.
Jerry D. Hedden regretted that some of the design decisions that Artur
mentioned, as a way of explaining why things were the way they were,
were not mentioned in the documentation.
Jan defended the design decision to allow a nested function to
terminate the thread, and not have to percolate the desire back up the
call stack to the main thread function.
Liz Mattijsen voiced the opinion that it was important for a thread
encountering some naughty library code, that contained evil "exit"
somewhere, to not be able to take out the whole application.
Artur explained that a technique already existed for dealing with this
situation.
Picking up the thread
http://xrl.us/o7rf
Threads signalling issue
Jerry D. Hedden was sufficiently worried by Artur's claim that the
signal dispatching code was not safe, that he broke the discussion out
into a separate thread. He stated that as far as he could see, the
code *was* thread-safe. Jan Dubois confirmed that he was certain that
it was fine, as well.
Artur described the problem he envisaged in more detail, which allowed
Jan to point out where the argument was flawed, since the code already
took the situation Artur outlined into account.
And so now Artur owes Jan a beer.
Nicholas Clark pointed out that even though the above code is
race-safe on a single-processor box, on an SMP machine it might be
possible to have a race, and he showed where it could happen. The jury
is still out on this matter.
http://xrl.us/o7rg
"Storable.pm" has poor version compatibility
Kari Pahula was shuffling binary images between two computers using
"RPC::PlServer" and "RPC::PlClient", with "Storable" acting as a
serialising mechanism, and shipping the result over the wire.
Things started to fall apart when the two computers in question were
running different versions of "Storable" (The 2.6 receiver *versus*
the 2.7 sender), which rendered the received data essentially
worthless. Kari wished that "Storable" had a mechanism to specify that
version "N" should produce a data stream compatible with version
"N-x", for some small value of "x". That is, Kari would quite happy to
go back to his 2.7 server, and explicitly add a "make this 2.6
compatible" flag, except that this is currently not possible.
http://xrl.us/o7rh
Patches of Interest
Test scripts for "I18N::Langinfo" and "POSIX"
Last week, Sébastien Aperghis-Tramoni wrote some tests to improve the
coverage of these two modules. In doing so, he discovered a couple of
bugs: fields documented as existing, in fact, don't. Steve Peters had
problems with some of the test results, and applied a partial patch.
Craig Berry looked at some of the skipped tests this week, and made a
few more changes.
http://xrl.us/o7ri
Here are the changes, concerning missing POSIX functionality, that
Craig realised was being dealt with appropriately.
http://xrl.us/o7rj
Sébastien came back later on with a way to improve
"POSIX::localeconv()"'s usefulness, by defining all keys in the
underlying hash, and making the undefined ones point to an empty
string. This would make the test suite easier to manage, and would
lead to less make-work code being required in client code.
http://xrl.us/o7rk
"perlhack": reflections on portability
Jarkko Hietaniemi is undergoing therapy to recover from doing combat
for far too long with C compilers that conspire to make the perl
codebase such a fun place to hang out.
The tao of portability
http://xrl.us/o7rm
And another batch of things to watch out for.
http://xrl.us/o7rn
Making sense of the z/OS patches
Jarkko then started picking away at the z/OS patches that had recently
been sent to the list from IBM, to see what he could salvage. First up
was some low-hanging fruit in the shape of hints/os390.sh and
"Makefile.SH".
http://xrl.us/o7ro
And then a number of EBCDIC fixes in the test suite.
http://xrl.us/o7rp
He carried on with some more library files and test. Stephen McCamant
wondered whether there wasn't an elegant way to factorise the
EBCDIC/ASCII differences so that less code wound up being duplicated.
http://xrl.us/o7rq
And something to do with "main" that H.Merijn was unable to apply,
despite his best intentions.
http://xrl.us/o7rr
Taking that, and Stephen's, remarks into account, Jarkko brewed a
fresh patch.
http://xrl.us/o7rs
At the end of the week, Jarkko was able to tease out a fix for
"MIME::Base64"'s "quoted-print.t" test. He wasn't precisely sure how
EBCDIC and "quoted-printable" are supposed to work together at the
best of times, but sent the patch on its way.
[EMAIL PROTECTED]@[EMAIL PROTECTED]@[EMAIL PROTECTED]
http://xrl.us/o7rt
Various additions to cflags.SH and sundry files
Jarkko discovered with delight that recent versions of "gcc" have a
"-std" to specify to which C standard the source code should be held
against. So he taught cflags.SH to specify that C89 is the only game
in town.
// is not a comment
http://xrl.us/o7ru
And in the interest of perfection, Jarkko also ensured that a number
of recent additions to the code base were cited in cflags.SH.
http://xrl.us/o7rv
In a grab-bag of assorted fixes, Jarkko patched dump.c, pp_sys.c, sv.c
and util.c and snuck in a few more additions to "perlhack".
http://xrl.us/o7rw
In a last glorious attack on "perlhack", Jarkko wrote about the
"Configure -Dgccansipedantic" switch, that forces the "gcc" compiler
to become extremely fussy about what it considers acceptable practices
in code.
Squeaky clean
http://xrl.us/o7rx
Updated escapes and debug output improvements
Yves Orton landed a fairly hefty patch of tweakery, in part to improve
the escaping code, making it Unicode aware. The second part was to
clean up the way debugging output is displayed by the regexp engine.
The latter work was not yet complete, but Yves felt it was time to
commit.
Dave Mitchell applied the changes, and noted a number of compiler
warnings about "char *"/"U8" differences, and mentioned that the debug
output is currently a bit of mess, and the direction in which he
wished to push it.
Yves explained what he was tinkering with at the moment, and asked for
guidance on dealing with the variables used in the code. He thought
that the de-recursion work that Dave had put into the engine offered a
suitable place for squirrelling away new variables, and thought that
the current way of dealing with variables was weird.
Dave Mitchell explained that the current state of affairs can be
traced back to the heritage of Henry Spencer's original regular
expression library, twisted beyond belief to work in a
multi-interpreter, multi-threaded environment.
*sounds of hammers, saws and arc-welders in the basement*
http://xrl.us/o7ry
Yves fixed up a number of warnings that the new code issued, in a
subsequent patch.
http://xrl.us/o7rz
Watching the smoke signals
Smoke [5.9.4] 28549 FAIL(F) linux 2.6.12-9-powerpc [debian] (ppc/1 cpu)
This failure, Rafael Garcia-Suarez believed, was due to the recent
work performed on dump.c whose aim was to improve the output by
escaping "invisible" characters. Yves promised to take a look (which
happened just above).
http://xrl.us/o7r2
New and old bugs from RT
5.7.2 and "perlPod::Text::Overstrike" (#7959)
Whatever the problem was, at some point in the past four years, Steve
Peters discovered that it had been fixed.
Counterstrike
http://xrl.us/o7r3
localtime(3) calls tzset(3), but localtime_r(3) may not. (#26136)
Jason Vas Dias posted an patch to fix a bug for Perl on Fedora,
following on from a revival of the patch by Benjamin Holzman. Benjamin
followed up on Jason's work and tweaked the patch to make it suitable
for both "maint" and "blead".
It's about time
http://xrl.us/o7r4
"defined"-ness of substrings disappear over repeated calls (#39247)
A few words of wisdom from Yitzchak Scott-Thoennes on the matter of XS
code dealing with magic.
http://xrl.us/o7r5
Bug in system calls when %ENV is very large (#39547)
Steve Hay traced this problem down to the way perl gets compiled on
the Win32 platform and thought that as things stand, the restriction
appears to be an intrinsic limitation of the platform, and doubted it
could be solved.
Randy W. Sims forwarded the message to Microsoft, who confirmed that
the environment block for a process is indeed limited to just shy of
32 kilobytes.
Their bug, not ours
http://xrl.us/o7r6
gcc 3.3 has problems with "__attribute__((unused))" (#39634)
Andy Dougherty tweaked the setting of the "HAS_ATTRIBUTE" define in a
way that he felt was the clearest to understand. Rafael applied it.
http://xrl.us/o7r7
filetest problem with STDIN/OUT on Windows (#39637)
Steve Hay wrote some more C code to analyse this problem and found
that indeed, the "-r" operator returns false for "STDIN". Trying it
out with various C compilers led him to conclude that this is also
some sort of Win32 limitation. Steve and Yves kicked around a few
ideas, but at the end of the week it looks like it will wind up
documented in "perlport" as something that doesn't work.
The tailor's face and hands
http://xrl.us/o7r8
"S_regmatch" produces a stack overflow (#39774)
A pattern that results in a stack overflow was reported. Steve Peters
explained that in "blead", the overflow no longer exists, although an
error message described another sort of exhaustion is emitted instead.
Yves Orton explained how the pattern was one of the sort that tend to
give NFAs such as perl's regexp engine fits, as it forces backtracking
information to be stashed away at every single point in the string. He
went on to show how the pattern could be rewritten in a much more
efficient manner.
He then expressed hope that some day, the optimiser might be able to
compose the transform all by itself, but reasoned that it might be
difficult.
When he says it's not easy
http://xrl.us/o7r9
Cygwin Perl bug -- "pod2usage(-verbose => 0)" and "pod2usage(-verbose =>
1)" (#39775)
David Christensen wanted to know what the status was on a patch that
fixes a problem with "pod2usage".
http://xrl.us/o7sa
"Switch" module bug (#39789)
Artyom Tseitlin, poor chap, was bitten by too-clever-by-half magic of
"Text::Balanced" (upon which "Switch" relies). Adriano Ferreira posted
a semi-evil hack that would allow Artyom to continue to use "Switch".
Divide and conquer
http://xrl.us/o7sb
Unable to build Perl under Irix with "-Duseshrplib" (#39797)
Philippe Schaffnit was going mad trying to discover what his mistake
was in trying to build Perl on Irix. (no, *using Irix* is not an
acceptable answer). The problem sounded familiar to Andy Dougherty,
who suggested an approach to narrow things down. At the end of the
week we still had not heard back from Philippe.
http://xrl.us/o7sc
Mortality of objects (*e.g.* %$_) passed as args (#39800)
Vaclav Ovsik ran afoul of the absence of refcounting to @_ and the
perl stack.
Don't do that then
http://xrl.us/o7sd
"bleadperl -Dm -e1" segfaults on win32 (#39806)
Rafael made an educated guess that a thread was trying to print
something using a data structure that had already been freed...
http://xrl.us/o7se
"eval" and hash access in subroutine (#39816)
Thomas Ziehmer composed a clever little ditty that shows how he was
forced to make a bogus "%hash = %hash" assignment to make things work
correctly. I shall have commented on this next week, since nobody did
this week.
Which was last week, anyway
http://xrl.us/o7sf
"PERL5SHELL" is not checked for tainted data (#39832)
Paul Fenwick showed how an evil hacker could make Perl inadvertently
launch Notepad through a "system" call, even in taint mode, since
"PERL5SHELL" is not subject to the usual tainting strictures.
Rafael took a stab at closing the hole, and asked for someone with
access to a Windows machine to validate the fix. Yves confirmed that
the problem was indeed resolved. Dr. Ruud suggested applying similar
checks upon $ENV{SystemRoot} and $ENV{windir}.
Rafael wanted to know what purpose those two variables served.
Could have been worse
http://xrl.us/o7sg
Update perlipc.pod TCP server example with respect to safe signals and
"accept()" (#39835)
Andy Wardley discovered that the introduction of Safe Signals, from
5.7.3 onwards had rendered the TCP server examples in "perlipc"
obsolete. So he updated the POD, in the hope that some kind soul would
apply it to "blead" (and even "maint") so that others may avoid
running into the same problem.
http://xrl.us/o7sh
Script with threads hang when there are nested subthreads (#39839)
Erland Sommarskog discovered that nested threads don't.
http://xrl.us/o7si
Rafael Garcia-Suarez noticed that a "Data::Bind" compilation failure
could be fixed by diddling with how "Carp" was used, and therefore
wondered if something hadn't been broken somewhere. The symptoms
reminded David Landgren that he had encountered similar problems
testing modules under "blead".
http://xrl.us/o7sj
Perl5 Bug Summary
An internal server error prevented the bug summary from being posted
this week, but I assume we are somewhere around 1500.
Check 'em out
http://rt.perl.org/rt3/NoAuth/perl5/Overview.html
New Core Modules
* "Term::ANSIColor" version 1.11 released by Russ Allbery.
Essentially a documentation and test suite enhancements release.
http://xrl.us/o7sk
* "Test::Builder/More/Simple" version 0.64 was rushed to the
delivery room in an emergency by Michael Schwern. A change in 0.63
to "test_fail()" broke "Test::Exception" (and possibly many more
modules as well).
Author and module are in good health
http://xrl.us/o7sm
In Brief
Marcus Holland-Moritz commented on the "state" variables progress,
saying that if the making iffy constructs results in a slowdown in the
general case, then those constructs should simply be outlawed as
errors.
But the slowdown is only wafer thin
http://xrl.us/o7sn
David Nicol suggested that "INIT" blocks could be used to initialise
"state" variables, and proposed a suitable error message to be used
when indicating that a state variable was not going to be initialised
as expected.
http://xrl.us/o7so
Ron Savage followed up on the followup from Jan Dubois regarding
Oracle on ActiveState, noting that he was able to install
"DBD::Oracle", but unable to do anything with it. Jan suggested a few
things to try out, and outlined what ActiveState would have to do to
get it to work in all cases.
http://xrl.us/o7sp
Steve Hay noted that a recent version of "ext/threads" doesn't build
with Borland on Win32, due to differences in header files. Jerry D.
Hedden found an appropriate "#define" upon which to hang the missing
definitions.
http://xrl.us/o7sq
And Jerry delivered the patch, applied by Rafael.
http://xrl.us/o7sr
He then bumped up the version with a fix to stop signalling terminated
threads from dumping core.
http://xrl.us/o7ss
Dr. Ruud wondered whether it was worth chasing down 0 and replacing it
with "NULL", in pp.c to begin with.
http://xrl.us/o7st
statefulness is a property of the code, not the variable, sayeth Dave
Mitchell.
http://xrl.us/o7su
Steve Hay encountered a strange problem with perly.h on Win32, that he
traced tentatively back to the switch-over from "byacc" to "bison". A
tweak to regen_perly.pl fixed the problem. So Steve applied the fix,
but regretted not being able to understand why the error was
occurring.
Action and reaction
http://xrl.us/o7sv
Jarkko tweaked hv.c and sv.c for "-DPERL_GLOBAL_STRUCT_PRIVATE" and
expressed disgust at the very existence of a global variable named
"done_sanity_check". After having stepped back to look at his
handiwork, he realised that recent work to bring DynaLoader.o into
"libperl" has rendered the Symbian port of Perl unusable.
http://xrl.us/o7sw
Steve Hay discovered that "mod_perl" 1.x and "apache" 1.x do not like
"blead", possibly due to the way constants have been rejigged.
http://xrl.us/o7sx
Sadahiro Tomoyuki produced a patch to update comments in "scan_const",
since much of it remains as it did at change #780 in the repository,
which, if the records are correct, is roughly about the same time that
the porters stopped using punched cards.
http://xrl.us/o7sy
He also edited "perlop" to clarify the parsing of quoted constructs,
so that people could apply some scientific method rather than just
guessing to see how things behaved.
http://xrl.us/o7sz
Yves Orton silenced a "system" warning during tests, although was only
able to do so for Win32, although he thought that it should be
possible to tweak it to work in a cross-platform manner.
http://xrl.us/o7s3
About this summary
This summary was written by David Landgren.
Last week's summary, here:
http://xrl.us/o7s4
elicited a response from H.Merijn who was concerned with some smoke
tests throwing out black smoke that was not reproducible with a test
run by hand.
Yves Orton also mentioned that he had a documentation patch for the
heredoc/escaping muddle that was in limbo, and was wondering whether
it needed improvement or was just pushed down on the stack of things
to apply.
If you want a bookmarklet approach to viewing bugs and change reports,
there are a couple of bookmarklets that you might find useful on my
page of Perl stuff:
http://www.landgren.net/perl/
Weekly summaries are published on http://use.perl.org/ and posted on a
mailing list, (subscription: [EMAIL PROTECTED]). The
archive is at http://dev.perl.org/perl5/list-summaries/. Corrections
and comments are welcome.
If you found this summary useful, please consider contributing to the
Perl Foundation to help support the development of Perl.
--
"It's overkill of course, but you can never have too much overkill."
