This Week on perl5-porters - 27 March-2 April 2006
Be the first kid on the block to have your very own pragma.
Topics of Interest
[Part of last week's summary vanished into a worm-hole of the
space-time continuum, and reappeared this week. The missing portion is
reproduced below.]
Fixing the "Your Makefile has been rebuilt" tedium
Dave Mitchell grew tired of the fact that the tiniest change to the
perl source causes all the extensions to be rebuilt as well. This is
because they all have a dependency on lib/Config.pm, which itself is
rebuilt each time "miniperl" is rebuilt.
So Dave changed things around to so that it (and lib/Config_heavy.pl)
are only updated if in fact they actually change during a rebuild.
Nicholas thought that this might break parallel makes, and that the
"mv-if-diff" hack was removed as it operated on a similar principle,
causing "make" to consider that the Unicode tables were perpetually
out of date, which caused "Encode" to be needlessly rebuilt many times
over in a single run.
Works on my box
http://xrl.us/kqps
Why "sv_mortalcopy(&sv_no)"?
Nicholas Clark saw that the source code uses the idiom
PUSHs(sv = sv_mortalcopy(&PL_sv_no));
which dates back as far as 5.003, and wondered why the more reasonable
construct
PUSHs(sv = sv_newmortal());
wasn't used instead. Dave Mitchell thought of a couple of reasons why,
such as the "getpw*" returning empty scalars for unsupported features,
but in the general case it was probably quite unnecessary. Yitzchak
Scott-Thoennes realised that there was a subtle difference between the
two constructs.
This all came about because of change #27612, in which Nicholas
changed when and where "sv_mortalcopy" was used in pp_sys.c.
Mortal peril
http://xrl.us/kqpt
"pthread_attr_setstacksize" failure
Jerry D. Hedden mentioned that he had received a couple of bug reports
concerning the new API for thread stack sizes, concerning threads
allocating a stack size of exactly 2Mb. Jerry wondered whether this
was some sort of 32/64-bit conversion failure and was otherwise stuck
as to figuring out where to go from here.
2147483648 bytes and counting
http://xrl.us/kqpu
Replacing "S_new_HE" with "Perl_new_body"
Jim Cromie delivered an exploratory patch to simplify "HE" allocations
in hv.c, to see what effect it would have, and asked for comments.
Something for hash-heads
http://xrl.us/kqpv
Arena-based allocations for "op"s
Nicholas Clark delivered a long thoughtful analysis of Jim Cromie's
other patch that allocated ops from arenas, saying that the complexity
it adds probably outweighs the benefits.
But there is a way forward
http://xrl.us/kqpw
Redundant $Config{d_sitearch} paths
Gisle Aas noted that if you configure perl and set $sitearch to be the
same as $archlib then the same directory appears twice in the @INC
path, which is silly.
Gisle wanted to patch this, but wasn't sure whether he could dive
straight in, or whether it required hacking on "metaconfig".
To make matters worse, he found that "SITELIB_STEM" could add yet a
third copy of the same directory to @INC, and came up with one patch
to rule them all.
One is enough
http://xrl.us/kqpx
A working "CLONE" for "Tie::RefHash"
Yuval Kogman patched "Tie::RefHash" so that it would work correctly
with threads. Rafael tidied it a bit as he put it into "blead".
It works!
http://xrl.us/kqpy
Multiple Perl version support quandary
John Peacock admitted to having been exceedingly naughty in releases
new versions of "version" and not testing them on older perls. Some of
the code contained 5.6-isms (notably dealing with warnings), and
wondered what to do to make it work again on 5.005.
John thought that the easiest way would be to fake up an *ersatz*
"warnings" module, but then, he wasn't sure how to pull off something
like "no warnings qw(redefine)".
Nick Ing-Simmons provided a elegantly devious lightweight solution
that please John no end. Yitzchak thought of a problem that John's
final implementation might have, and provided another improvement.
Revis(?:it)?ing compatibility
http://xrl.us/kqpz
Do "PERL_FLEXIBLE_EXCEPTIONS" work?
Nicholas was working (or not) with "PERL_FLEXIBLE_EXCEPTIONS" and came
to the conclusion that due to a compilation error, it didn't work,
couldn't work, and could never have worked in over six years and
twelve stable releases of Perl. Given that no bug reports have been
received to date, Nicholas concluded that they could be scrapped
without causing any harm.
Nick Ing-Simmons cautioned about being too hasty, saying that this
mechanism was there to allow Perl to be compiled natively by a C++
compiler, to map C's "longjmp" to C++'s "throw".
So Nicholas went off and tried to compile perl with a C++ compiler
("g++") on Linux and FreeBSD. Everything failed, usually due to
prototypes not being sufficiently precise (mainly "char *" *versus*
"const char *". From which one may conclude that it may be possible to
compile Perl with a C++ compiler out of the box, but certainly not
with a common C++ compiler on two of the most common Unix platforms.
Probably not
http://xrl.us/kqp2
Making the "IO::Socket" tests pass on Win32
Yves Orton sent in a patch to get "IO::Socket" to run its test suite
correctly on a threaded Windows build. He had a look at "IO::Pipe",
but couldn't think of a sane enough approach to make it work.
Steve Hay couldn't get it to smoke cleanly on a non-threaded build,
and Yves asked for advice on a better indicator to decide whether or
not to skip some of the tests (that deal with "fork"ing).
Steve Hay showed how various configure-time switches can be combined
in different ways to make all sorts of threadish behaviour in Windows.
Andy Dougherty said that the right way of seeing whether "fork" was
implemented was to look at $Config{d_fork}. And if that gave bogus
results, well by golly it ought to be fixed up so that it gave a
useful result.
Yves thought that this was a marvellous idea... except that it didn't
solve the problems for all the perls out there in the field today.
http://xrl.us/kqp3
Yves followed up with a patch that fixed just about everything, which
was applied by Steve Hay.
Socket to me
http://xrl.us/kqp4
Perl memory management and documentation
Tom Schindl thought that the documentation was unclear on the concept
of explicitly setting lexical variables to "undef" to release memory
inside a scope. Pointers were given to various book references, and
Yitzchak Scott-Thoennes summed up Perl's memory strategy nicely:
"allocate as early as possible and for as long as possible".
Sadahiro Tomoyuki observed that while there are techniques for
pre-extending arrays and hashes, no Perl-level technique is available
for scalars (although "SvGROW" can be used from XS code). This could
be useful for strings that are expected to become very large. He
suggested that making "length($string)" lvalue-able, as in
length($str) = 700000000
to tell perl to allocate that many bytes for a scalar could be very
helpful (notwithstanding the usual provisos about characters not equal
to bytes in Unicode strings).
Let my bytes go
http://xrl.us/kqp5
As usual, Dave Mitchell explained what was happening behind the scenes
in a clear and concise manner.
The algorithm
http://xrl.us/kqp6
How should "%^H" work with lexical pragmata
One of Robin Houston's many contribution to "blead" last year was a
somewhat arcane improvement that concerned "%^H" (the hints variable)
being made available to "eval" blocks.
Nicholas Clark used Robin's insight to help finish lexical pragmata,
and was running into conceptual difficulties over the state of
contents of "%^H" at compile time and run time, and when to make it
readable or writable.
The fundamental problem that needs to be addressed is that the state
of the hints get compiled into the op-tree, which means that changing
the setting of a hint has no effect (hence the "read-only" nature of
the beast) once the code has been compiled.
Rafael pointed out that that was just it: "%^H" and $^H are for
affecting compilation and, as they stand, are just not useful at run
time.
Something that does have an effect at run time, warnings.pm uses its
own variable, a bitfield, which is stored in the op-tree in its own
right, which is how warnings can come and go during run time.
Nicholas had a look at the definition of "struct cop", as that seemed
to be the logical place from which to hang hints, but then realised
that since op-trees are shared between threads there's no sane way to
make it read-write as well (since that would mean all threads would
inherit the change of hinting an any thread).
Nicholas finished up doing an extreme programming number: writing the
test to prove that lexical pragmata work. And then subsequently
committed a patch that made the test succeed.
Then Hugo admitted to being slightly confused. That's good. I was
confused all along.
I'll give you a hint
http://xrl.us/kqp7
Continued in the new month
http://xrl.us/kqp8
Rafael Garcia-Suarez played around with the pragma stuff and came up
with a user-level pragma example. David Nicol and Nicholas played
around with that, and the feedback from the exercise resulted in a
couple of other code tweaks.
(In case you're wondering, a pragma is a module with a lower case name
that can be turned on and off through the code. Two prime examples are
"use strict"/"no strict" and "use warnings"/"no warnings").
Your very own pragma
http://xrl.us/kqp9
At the same time, Rafael thought that it ought to be possible to make
encoding lexical as well, and set about trying to find out what was
still needed to get it to work. Nicholas and he thrashed out the
details.
http://xrl.us/kqqa
Combining UTF-16 output with :crlf is awkward
In a parallel universe (read: another mailing list), Jan Dubois
discovered that stacking the ":crlf" layer on top of a Unicode layer
causes "Wide character in print" warnings to be issued. The
work-around is the use the (in Jan's words) "non-intuitive"
":raw:encoding(UTF-16LE):crlf:utf8" layers together, to turn off the
"PERLIO_F_UTF8" bit in the ":crlf" layer.
Jan wondered whether it would be possible for "PerlIOCrlf_pushed()" to
inherit the flag from the previous layer, or whether "PerlIO_isutf8()"
should walk the layer stack in order to determine what it should do.
Nick Ing-Simmons preferred the first approach, going as far as saying
that that should actually be the default behaviour for a layer. The
second solution has the problem of a layer having to determine whether
some other arbitrary layer affects UTF-8 or not.
Layer upon layer upon layer
http://xrl.us/kqqb
Thread non-safety in "sv_setsv"
Nicholas was rather distressed to discover a problem with
"sv_setsv_flags" may put an end to a workable copy-on-write scheme in
threaded builds. This came about from looking at the hints
implementation, and the fact that threads share op-trees.
Nice ASCII art, Nick
http://xrl.us/kqqc
Patches of Interest
"Devel::DProf" "const"ing
Andy Lester had a look at "Devel::DProf" to see about the bugs Jarkko
Hietaniemi raised a while back. He wasn't able to fix anything yet,
but did clean the code up somewhat, and added some lovely "const"s in
the process.
It's better than nothing
http://xrl.us/kqqd
Poisoning memory
Following on from John Malmberg's plea to have allocated and
deallocated memory filled with garbage values (and thus poisoned, to
cause errant dereferences to be noticed earlier, Jarkko added a patch
to do just that.
Now allocations can be initialised with 0xAB (also known as
*strawberry cyanide*) and freed memory can be overwritten with 0xEF
(or *blueberry lithium*). Andy wondered how we'd gone for so long
without it.
Two exciting new flavours!
http://xrl.us/kqqe
VMS pool corruption fix for "_NLA0:"
In turn, John delivered a patch to make "stat" work correctly on
"NLA0:", which is very important if you're doing VMS work.
http://xrl.us/kqqf
Long file path support for VMS
Having finished with the preliminaries, John then got down to business
with a patch to long path support to all versions of and platforms of
OpeVMS that support them.
http://xrl.us/kqqg
And rejigged the "stat" structure used when "largefile" support was
enabled.
http://xrl.us/kqqh
Tidying up regexec.c
Andy Lester set his sight on regexec.c, now that Dave has finished
with it, and zapped numerous unused macros, inlined a couple of small
static functions and sprinkled the magic wand of "const"ness over the
lot.
http://xrl.us/kqqi
Random accumulated patches from Andy
Andy then shipped out all his patches that been piling up: consting
and "NULL" tweaks ("NULL" instead of 0 when dealing with pointers, and
removing casts on "NULL" assignments).
http://xrl.us/kqqj
and redid the "PERL_UNUSED_DECL" macro, eliminating a grumpy comment
at the same time. [News flash: this was eventually reverted, as there
is code Out There which relied on the previous behaviour].
http://xrl.us/kqqk
and removed some unnecessary pointer checks
http://xrl.us/kqqm
and found some more appropriate versions of the "SvREFCNT_inc" macro
to use.
http://xrl.us/kqqn
Add "V.pm" to the core
Abe Timmerman posted a patch to add V.pm, which was originally written
back in 2002 in answer to a question from Tels. John Peacock thought
that "Module::Info" would be more useful.
"Why?" said Profane. "Why not?" said Stencil.
http://xrl.us/kqqo
New and old bugs from RT
"$qr = qr/^a$/m; $x =~ $qr" fails (#3038)
Nicholas Clark beat everyone else in closing out this bug from 2004.
http://xrl.us/kqqp
He also fixed up the "hash assignment to a tied hash erroneously
stores data in the real hash too (#36267)" bug too.
http://xrl.us/kqqq
Perl segfaults; test case available (#32332)
http://xrl.us/kqqr
no "sendmsg"/"recvmsg" support (#38808)
Nicholas noted that neither the core, nor the "Socket" module provide
the "sendmsg" and "recvmsg" functions. Gise Aas thought that "POSIX"
would be a suitable place in which to have them.
http://xrl.us/kqqs
Bad return value from a block with variable localization (#38809)
Vincent Pit filed a bug that showed some code using "if(@_)", "do" and
"return" picking up Cundef> in an unexpected manner.
http://xrl.us/kqqt
Encoding error in UTF-8 locales (#38812)
Vincent Lefevre posted an encoding bug. Nicholas stripped down the
example code and highlighted the error. He wasn't sure whether it was
a problem of the documentation not being sufficiently clear, or the
core for not dealing with the issue adequately
Maybe a bit of both
http://xrl.us/kqqu
"local $h{$unicode}" doesn't work (#38815)
Nicholas Clark noticed that "local $a{"\x{100}"} = 1" doesn't behave
correctly (the way a non-Unicode key like "local $a{"N"} = 1" does),
and promised to come up with a way to fix it, and did.
All part of a day's work
http://xrl.us/kqqv
Segment fault when using "Sockets" (#38817)
http://xrl.us/kqqw
"use sort 'stable'" sorts backwards with perl5.9.3 (#38831)
Stefan Lidman discovered that stable sorting in "blead" sorts
descending instead of ascending by default. Rafael and Robin Houston
had it sorted out in a jiffy.
I hope they added a test case
http://xrl.us/kqqx
What Steve Peters did this week
After Dave Mitchell landed his impressive iterative pattern match
patch, Steve equally impressively trawled RT to resolve, like, a
jazillion bugs, each resulting in a new message to the list.
An interesting case of seeing how different people explain in their
own words what is in fact the same thing.
* Regexp causes "SIGSEGV" (stack overflow?) (#1760)
* Core dump using a Perl regular expression (#6844)
* Segmentation fault in "regmatch()" (#6987)
* Perl Segmentation Fault using "/((\w+ )+)/" on long strings
(#8685)
* Recently-introduced regex segfault (#8870)
* 5.6.0, 5.6.1, 5.8.0 regexp core on "([EMAIL PROTECTED]@|.)*" (#17611)
* perl 5.8.0 segfaults (#18489)
* perl "SIGSEGV" when applying regular expression to a long string
(#21298)
* Regexp segfault "--> ("X"x3529) =~ /( (?: \\. | [^\$] ){1,4000}
)/gx;" (#21333)
* Regexp segfault (#21922)
* Seg fault on long input to re (#21940)
* Segfault (deep recursion?) in regex match (#22051)
* Core dump on big regex (#23666)
* Segmentation fault caused by capturing regex (#24271)
* METABUG - regex stack overflow issues (#24274)
* Segmentation fault at "m//" regexp (#28999)
* Regexp "/^([^f]|f.)+/" Bus error (#31887)
* SEGV with complicated regexp and long string (#32041)
* Long strings causes segmentation fault (#32465)
* Regular expression segfaults perl (#32803)
* "SIGSEGV" in "S_regmatch" (#34349)
* Simple regexp causes segfault (#36020)
* Segfault in simple regular expression (#36999)
* Segfault when doing this regex (#38031)
* Segmentation fault for matching too long regexps (#38379)
* Silent self-termination of script using regex (#38470)
* Regexp Bus error (#38473)
* Perl Segfault in Regex Match (#38717)
When Dave said he thought his patch would allow a whole pile of bugs
to be closed out, he wasn't joking.
Steve attempted to close out "Regular expression causes segfault
(#36903)" but was having access permission problems in retrieving the
test code to be used for the bug. Milo Thurston provided another URL
to get at the code.
403 Forbidden
http://xrl.us/kqqy
Perl5 Bug Summary
1563 bugs (but wait until next time)
http://xrl.us/kqqz
Over here
http://rt.perl.org/rt3/NoAuth/perl5/Overview.html
New Core Modules
* "version" 0.59 by John Peacock,
http://xrl.us/kqq2
* "Module::Build" 0.27_10 by Ken Williams,
http://xrl.us/kqq3
* and "Time::Local" 1.12_01 from Dave Rolsky.
http://xrl.us/kqq4
In Brief
Alan Burlison forwarded a message about a new project that had been
formed to deal with programming language vulnerabilities.
http://xrl.us/kqq5
Hugo van der Sanden added a brief documentation patch to clarify the
fact that you cannot use "times()" to obtain the elapsed time consumed
by running child processes, only for finished processes that have been
"wait"ed upon.
http://xrl.us/kqq6
David Nicol proposed a "Tie::MaskedArray" technique for avoiding the
remotely tied global localized with a sigil exploit. I'm a little hazy
on the details of this particular exploit. David's techique proposes
to replace "local", more slowly, but also more safely.
http://xrl.us/kqq7
John L. Allen forwarded the current, best patch for "pow()" on AIX.
http://xrl.us/kqq8
Jim Cromie updated the documentation to make it more clear what
happens when one does something like "Configure -des
-DNoSuchConfigureFlag".
http://xrl.us/kqq9
Robin Barker found a bug in "Readonly" in 5.8.8 and so fixed the bug,
and added a test to t/op/tie.t to make sure the problem doesn't
return.
(I think I'm beginning to see a pattern here)
http://xrl.us/kqra
Paul Marquess provided a small patch to the zip test harness for
"IO::Compress::Zip".
http://xrl.us/kqrb
Sadahiro Tomoyuki looked at some of the recent changes to the source
and found some unmatching of parameters and types. Nicholas updated
embed.fnc to take that into account.
http://xrl.us/kqrc
Andy had a go at linting the source with Sun Studio's lint, and found
lots of things that need to be looked at, and wondered whether there
were any other lint-like tools freely available that could be applied.
Jarkko mentioned FlexeLint (a.k.a Gimpel Lint), which is very nice,
but not free.
http://xrl.us/kqrd
Yves Orton found a tainting oddity that should possibly be documented.
Yitzchak thought that some patches that remove the surprising
behaviour would also be well received.
http://xrl.us/kqre
H.Merijn Brand backported all of recent changes made by Nicholas in
"blead"'s Configure to that of "maint". At the end of the week he was
still busy filling in gaps in Porting/Glossary.
http://xrl.us/kqrf
Feeback from last week's summary
Craig Berry corrected my misreading of the "Module::Build" on VMS
thread, which is that the VMS port currently doesn't offer the list
form of piped "open".
http://xrl.us/kqrg
About this summary
This summary was written by David Landgren. I will be getting a
life^W^W^Wtaking a break next week so the next summary will be for the
fortnight 3-16 April.
If you want a bookmarklet approach to viewing bugs and change reports,
there are a couple of bookmarklets that you might find useful on my
page of Perl stuff:
http://www.landgren.net/perl/
Weekly summaries are published on http://use.perl.org/ and posted on a
mailing list, (subscription: [EMAIL PROTECTED]). The
archive is at http://dev.perl.org/perl5/list-summaries/. Corrections
and comments are welcome.
If you found this summary useful or enjoyable, please consider
contributing to the Perl Foundation to help support the development of
Perl.
--
"It's overkill of course, but you can never have too much overkill."
