In perl.git, the branch blead has been updated
<http://perl5.git.perl.org/perl.git/commitdiff/a1bbdff363ce08fa98da9f4043f11f13488da5b2?hp=824afba194f106ca99a3f04463fda8c3247fbc70>
- Log -----------------------------------------------------------------
commit a1bbdff363ce08fa98da9f4043f11f13488da5b2
Author: brian d foy <brian.d....@gmail.com>
Date: Tue Sep 14 12:16:49 2010 -0500
* Add multilevel hash exists() to perlfaq4
How can I check if a keys exists in a multilevel hash?
How is it we got this far without ever answering this
FAQ? :)
M pod/perlfaq.pod
M pod/perlfaq4.pod
commit 03c6e0f89140dee7d6b58e7370bc8e0f2bfaf3da
Author: brian d foy <brian.d....@gmail.com>
Date: Tue Sep 14 12:15:10 2010 -0500
* Added a smart match example to perlfaq6
How do I efficiently match many regular expressions at once?
It's almost trivial with smart matching. Barely worth
asking anymore.
M pod/perlfaq6.pod
commit b400a9bf6ba3e9dead0d32d90a8109915c24a744
Author: brian d foy <brian.d....@gmail.com>
Date: Sun Sep 12 22:56:58 2010 -0500
More whitespace cleanups
M pod/perlfaq6.pod
commit c93274ade4f05dd201a4080f0d4e9f99de9d552f
Author: brian d foy <brian.d....@gmail.com>
Date: Sun Sep 12 22:53:09 2010 -0500
Whitespace cleanups
M pod/perlfaq6.pod
commit 701f2f0135f1b9b1d48604a792c07efa5b2e810d
Author: brian d foy <brian.d....@gmail.com>
Date: Wed Sep 1 03:32:12 2010 -0400
perlfaq4: fix a minor grammar problem in the Y2K answer
M pod/perlfaq4.pod
commit 67126c1b100963b7895272979a13cd24947be080
Author: brian d foy <brian.d....@gmail.com>
Date: Sun Aug 22 13:53:59 2010 -0500
* Correct Higher Order Perl to Higher-Order Perl
M pod/perlfaq.pod
M pod/perlfaq2.pod
commit 84adb7243d1e8c527659f32895b8cc8e6ea27401
Author: brian d foy <brian.d....@gmail.com>
Date: Sat Aug 21 12:03:46 2010 -0500
Fix trailing whitespace
M pod/perlfaq4.pod
M pod/perlfaq5.pod
M pod/perlfaq8.pod
M pod/perlfaq9.pod
commit 820b2690a908738e3f3ed2d0610107e771c57cf8
Author: brian d foy <brian.d....@gmail.com>
Date: Sat Aug 21 12:01:02 2010 -0500
* Update Effective Perl Programming reference
M pod/perlfaq2.pod
commit 7a98bd7582df3d286ae2c6a25d16bdea7325962a
Author: brian d foy <brian.d....@gmail.com>
Date: Sat Aug 21 11:37:49 2010 -0500
* How can I read in an entire file all at once?
+ Incorporate a few suggestions from Uri
M pod/perlfaq5.pod
commit a9aaded0aea5c0c5ef088f1865cb781e330ea170
Author: brian d foy <brian.d....@gmail.com>
Date: Sat Aug 21 11:27:10 2010 -0500
* How can I read in an entire file all at once?
* Modulate my assertions about File::Map
M pod/perlfaq5.pod
commit eadf64ed1681f6b03eeff0e07d61ce8f45b1471a
Author: brian d foy <brian.d....@gmail.com>
Date: Sat Aug 21 11:23:28 2010 -0500
* How come when I open a file read-write it wipes it out?
+ adjust some code lines to come in under 80 columns
+ other minor formatting issues.
M pod/perlfaq5.pod
commit 159235edd1ba7c0316e67a71da4bbb871c086a8a
Author: brian d foy <brian.d....@gmail.com>
Date: Sat Aug 21 11:12:35 2010 -0500
* How do I handle linked lists?
+ Steve Lembark makes interesting cases for perly
linked lists, depending on the task.
M pod/perlfaq4.pod
commit 3cd7ab715194be8ea797d7f907f547c571ec12f2
Author: brian d foy <brian.d....@gmail.com>
Date: Sat Aug 21 10:47:37 2010 -0500
* How can I write() into a string?
+ Actually answer the question, now that we have
filehandles to strings.
+ The swrite in perlform is no good anyway.
M pod/perlfaq5.pod
commit b67b790e7eb84923c1239217745e9fb6a50a1104
Author: brian d foy <brian.d....@gmail.com>
Date: Fri Aug 20 19:10:21 2010 -0500
How do I make sure users can't enter values into a form that cause my CGI
script to do bad things?
+ The CGI Meta FAQ is gone, so this needs a new
answer
M pod/perlfaq9.pod
commit 39150746af1d3f386e2eaeda9f218e056645f150
Author: brian d foy <brian.d....@gmail.com>
Date: Fri Aug 20 19:00:02 2010 -0500
* My CGI script runs from the command line but not the browser.
+ Tom's troubleshooting guide has disappeared in
the perl.com move (I think).
+ The CGI Meta FAQ is not such a fresh resource
M pod/perlfaq9.pod
commit f61c8e376daff76f2cdd42cf45e1249be8e1d7f0
Author: brian d foy <brian.d....@gmail.com>
Date: Fri Aug 20 18:45:23 2010 -0500
* How do I fetch/put an FTP file?
+ Net::FTP has been in core a long time
l Please enter the commit message for your changes. Lines starting
M pod/perlfaq9.pod
-----------------------------------------------------------------------
Summary of changes:
pod/perlfaq.pod | 50 ++++++++--------
pod/perlfaq2.pod | 8 +-
pod/perlfaq4.pod | 128 ++++++++++++++++++++++++++--------------
pod/perlfaq5.pod | 117 ++++++++++++++++++++++++++----------
pod/perlfaq6.pod | 174 ++++++++++++++++++++++++++++--------------------------
pod/perlfaq8.pod | 12 ++--
pod/perlfaq9.pod | 56 ++++++++++-------
7 files changed, 326 insertions(+), 219 deletions(-)
diff --git a/pod/perlfaq.pod b/pod/perlfaq.pod
index 6eb7d25..f917d44 100644
--- a/pod/perlfaq.pod
+++ b/pod/perlfaq.pod
@@ -23,12 +23,11 @@ The perlfaq is an evolving document and you can read the
latest version
at http://faq.perl.org/ . The perlfaq-workers periodically post extracts
of the latest perlfaq to comp.lang.perl.misc.
-You can view the source tree at https://github.com/briandfoy/perlfaq
-(which is outside of the main Perl source tree). The git repository
-notes all changes to the FAQ and holds the latest version of the
-working documents and may vary significantly from the version
-distributed with the latest version of Perl. Check the repository
-before sending your corrections.
+The perlfaq documents are in the perl source tree (see
+L<perlrepository>). The git repository notes all changes to the FAQ and
+holds the latest version of the working documents and may vary
+significantly from the version distributed with the latest version of
+Perl. Check the repository before sending your corrections.
=head2 How to contribute to the perlfaq
@@ -38,16 +37,13 @@ address to coordinate their efforts and track the perlfaq
development.
They appreciate your contributions to the FAQ but do not have time to
provide individual help, so don't use this address to ask FAQs.
-The perlfaq server posts extracts of the perlfaq to that newsgroup every
-6 hours (or so), and the community of volunteers reviews and updates the
-answers. If you'd like to help review and update the answers, check out
-comp.lang.perl.misc.
+The perlfaq server posts extracts of the perlfaq to that newsgroup
+every 6 hours (or so), and the community of volunteers reviews and
+updates the answers. If you'd like to help review and update the
+answers, check out comp.lang.perl.misc.
-You can also fork the git repository for the perlfaq and send a pull
-request so the main repository can pull your changes. The repository
-is at:
-
- https://github.com/briandfoy/perlfaq
+You can also fork the perl repository, make your changes, and send them
+to Perl 5 Porters. See L<perlrepository>.
=head2 What will happen if you mail your Perl programming problems to the
authors?
@@ -67,13 +63,13 @@ it, try the resources in L<perlfaq2>.
Tom Christiansen wrote the original perlfaq then expanded it with the
help of Nat Torkington. The perlfaq-workers maintain current document
-and the dezinens of comp.lang.perl.misc regularly review and update the
+and the denizens of comp.lang.perl.misc regularly review and update the
FAQ. Several people have contributed answers, corrections, and comments,
and the perlfaq notes those contributions wherever appropriate.
=head1 AUTHOR AND COPYRIGHT
-Tom Christainsen wrote the original version of this document.
+Tom Christiansen wrote the original version of this document.
brian d foy C<< <bd...@cpan.org> >> wrote this version. See the
individual perlfaq documents for additional copyright information.
@@ -121,7 +117,7 @@ What is Perl?
=item *
-Who supports Perl? Who develops it? Why is it free?
+Who supports Perl? Who develops it? Why is it free?
=item *
@@ -311,7 +307,7 @@ Where can I get Perl macros for vi?
=item *
-Where can I get perl-mode for emacs?
+Where can I get perl-mode or cperl-mode for emacs?
=item *
@@ -676,6 +672,10 @@ How can I use a reference as a hash key?
=item *
+How can I check if a keys exists in a multilevel hash?
+
+=item *
+
How do I handle binary data correctly?
=item *
@@ -801,11 +801,11 @@ Why can't I just open(FH, "E<gt>file.lock")?
=item *
-I still don't get locking. I just want to increment the number in the file.
How can I do this?
+I still don't get locking. I just want to increment the number in the file.
How can I do this?
=item *
-All I want to do is append a small amount of text to the end of a file. Do I
still have to use locking?
+All I want to do is append a small amount of text to the end of a file. Do I
still have to use locking?
=item *
@@ -898,7 +898,7 @@ How can I hope to use regular expressions without creating
illegible and unmaint
=item *
-I'm having trouble matching over more than one line. What's wrong?
+I'm having trouble matching over more than one line. What's wrong?
=item *
@@ -942,7 +942,7 @@ Can I use Perl regular expressions to match balanced text?
=item *
-What does it mean that regexes are greedy? How can I get around it?
+What does it mean that regexes are greedy? How can I get around it?
=item *
@@ -974,7 +974,7 @@ What good is C<\G> in a regular expression?
=item *
-Are Perl regexes DFAs or NFAs? Are they POSIX compliant?
+Are Perl regexes DFAs or NFAs? Are they POSIX compliant?
=item *
@@ -1341,7 +1341,7 @@ What is the correct form of response from a CGI script?
=item *
-My CGI script runs from the command line but not the browser. (500 Server
Error)
+My CGI script runs from the command line but not the browser. (500 Server
Error)
=item *
diff --git a/pod/perlfaq2.pod b/pod/perlfaq2.pod
index 036df5e..a8646f5 100644
--- a/pod/perlfaq2.pod
+++ b/pod/perlfaq2.pod
@@ -333,9 +333,9 @@ Recommended books on (or mostly on) Perl follow.
http://www.oreilly.com/catalog/cookbook/
Effective Perl Programming
- by Joseph Hall
- ISBN 0-201-41975-0 [1st edition 1998]
- http://www.awl.com/
+ by Joseph N. Hall, Joshua A. McAdams, brian d foy
+ ISBN 0-321-49694-9 [2nd edition 2010]
+ http://www.effectiveperlprogramming.com/
Real World SQL Server Administration with Perl
by Linchi Shea
@@ -349,7 +349,7 @@ Recommended books on (or mostly on) Perl follow.
ISBN: 0-596-00173-8 [1st edition July 2005]
http://www.oreilly.com/catalog/perlbp/
- Higher Order Perl
+ Higher-Order Perl
by Mark-Jason Dominus
ISBN: 1558607013 [1st edition March 2005]
http://hop.perl.plover.com/
diff --git a/pod/perlfaq4.pod b/pod/perlfaq4.pod
index 2e35068..94fc873 100644
--- a/pod/perlfaq4.pod
+++ b/pod/perlfaq4.pod
@@ -559,7 +559,7 @@ running projects:
% perl5.11.2 -E 'say scalar gmtime( 0x9FFF_FFFFFFFF )'
Thu Nov 2 00:42:39 5576711
-You're still out of luck if you need to keep tracking of decaying protons
+You're still out of luck if you need to keep track of decaying protons
though.
=head1 Data: Strings
@@ -1562,53 +1562,23 @@ that satisfies the condition.
=head2 How do I handle linked lists?
-In general, you usually don't need a linked list in Perl, since with
-regular arrays, you can push and pop or shift and unshift at either
-end, or you can use splice to add and/or remove arbitrary number of
-elements at arbitrary points. Both pop and shift are O(1)
-operations on Perl's dynamic arrays. In the absence of shifts and
-pops, push in general needs to reallocate on the order every log(N)
-times, and unshift will need to copy pointers each time.
-
-If you really, really wanted, you could use structures as described in
-L<perldsc> or L<perltoot> and do just what the algorithm book tells
-you to do. For example, imagine a list node like this:
-
- $node = {
- VALUE => 42,
- LINK => undef,
- };
-
-You could walk the list this way:
-
- print "List: ";
- for ($node = $head; $node; $node = $node->{LINK}) {
- print $node->{VALUE}, " ";
- }
- print "\n";
-
-You could add to the list this way:
+(contributed by brian d foy)
- my ($head, $tail);
- $tail = append($head, 1); # grow a new head
- for $value ( 2 .. 10 ) {
- $tail = append($tail, $value);
- }
+Perl's arrays do not have a fixed size, so you don't need linked lists
+if you just want to add or remove items. You can use array operations
+such as C<push>, C<pop>, C<shift>, C<unshift>, or C<splice> to do
+that.
- sub append {
- my($list, $value) = @_;
- my $node = { VALUE => $value };
- if ($list) {
- $node->{LINK} = $list->{LINK};
- $list->{LINK} = $node;
- }
- else {
- $_[0] = $node; # replace caller's version
- }
- return $node;
- }
+Sometimes, however, linked lists can be useful in situations where you
+want to "shard" an array so you have have many small arrays instead of
+a single big array. You can keep arrays longer than Perl's largest
+array index, lock smaller arrays separately in threaded programs,
+reallocate less memory, or quickly insert elements in the middle of
+the chain.
-But again, Perl's built-in are virtually always good enough.
+Steve Lembark goes through the details in his YAPC::NA 2009 talk "Perly
+Linked Lists" ( http://www.slideshare.net/lembark/perly-linked-lists ),
+although you can just use his C<LinkedList::Single> module.
=head2 How do I handle circular lists?
X<circular> X<array> X<Tie::Cycle> X<Array::Iterator::Circular>
@@ -2451,6 +2421,74 @@ If you actually need to be able to get a real reference
back from
each hash entry, you can use the Tie::RefHash module, which does the
required work for you.
+=head2 How can I check if a keys exists in a multilevel hash?
+
+(contributed by brian d foy)
+
+The trick to this problem is avoiding accidental autovivification. If
+you want to check three keys deep, you might naïvely try this:
+
+ my %hash;
+ if( exists $hash{key1}{key2}{key3} ) {
+ ...;
+ }
+
+Even though you started with a completely empty hash, after that call to
+C<exists> you've created the structure you needed to check for C<key3>:
+
+ %hash = (
+ 'key1' => {
+ 'key2' => {}
+ }
+ );
+
+That's autovivification. You can get around this in a few ways. The
+easiest way is to just turn it off. The lexical C<autovivification>
+pragma is available on CPAN. Now you don't add to the hash:
+
+ {
+ no autovivification;
+ my %hash;
+ if( exists $hash{key1}{key2}{key3} ) {
+ ...;
+ }
+ }
+
+The C<Data::Diver> module on CPAN can do it for you too. Its C<Dive>
+subroutine can tell you not only if the keys exist but also get the
+value:
+
+ use Data::Diver qw(Dive);
+
+ my @exists = Dive( \%hash, qw(key1 key2 key3) );
+ if( ! @exists ) {
+ ...; # keys do not exist
+ }
+ elsif( ! defined $exists[0] ) {
+ ...; # keys exist but value is undef
+ }
+
+You can easily do this yourself too by checking each level of the hash
+before you move onto the next level. This is essentially what
+C<Data::Diver> does for you:
+
+ if( check_hash( \%hash, qw(key1 key2 key3) ) ) {
+ ...;
+ }
+
+ sub check_hash {
+ my( $hash, @keys ) = @_;
+
+ return unless @keys;
+
+ foreach my $key ( @keys ) {
+ return unless eval { exists $hash->{$key} };
+ $hash = $hash->{$key};
+ }
+
+ return 1;
+ }
+
=head1 Data: Misc
=head2 How do I handle binary data correctly?
diff --git a/pod/perlfaq5.pod b/pod/perlfaq5.pod
index bd969f4..4fcf337 100644
--- a/pod/perlfaq5.pod
+++ b/pod/perlfaq5.pod
@@ -644,7 +644,54 @@ techniques to make it possible for the intrepid hacker.
=head2 How can I write() into a string?
X<write, into a string>
-See L<perlform/"Accessing Formatting Internals"> for an C<swrite()> function.
+(contributed by brian d foy)
+
+If you want to C<write> into a string, you just have to <open> a
+filehandle to a string, which Perl has been able to do since Perl 5.6:
+
+ open FH, '>', \my $string;
+ write( FH );
+
+Since you want to be a good programmer, you probably want to use a lexical
+filehandle, even though formats are designed to work with bareword filehandles
+since the default format names take the filehandle name. However, you can
+control this with some Perl special per-filehandle variables: C<$^>, which
+names the top-of-page format, and C<$~> which shows the line format. You have
+to change the default filehandle to set these variables:
+
+ open my($fh), '>', \my $string;
+
+ { # set per-filehandle variables
+ my $old_fh = select( $fh );
+ $~ = 'ANIMAL';
+ $^ = 'ANIMAL_TOP';
+ select( $old_fh );
+ }
+
+ format ANIMAL_TOP =
+ ID Type Name
+ .
+
+ format ANIMAL =
+ @## @<<< @<<<<<<<<<<<<<<
+ $id, $type, $name
+ .
+
+Although write can work with lexical or package variables, whatever variables
+you use have to scope in the format. That most likely means you'll want to
+localize some package variables:
+
+ {
+ local( $id, $type, $name ) = qw( 12 cat Buster );
+ write( $fh );
+ }
+
+ print $string;
+
+There are also some tricks that you can play with C<formline> and the
+accumulator variable C<$^A>, but you lose a lot of the value of formats
+since C<formline> won't handle paging and so on. You end up reimplementing
+formats when you use them.
=head2 How can I open a filehandle to a string?
X<string> X<open> X<IO::String> X<filehandle>
@@ -727,64 +774,64 @@ Within Perl, you may use this directly:
=head2 How come when I open a file read-write it wipes it out?
X<clobber> X<read-write> X<clobbering> X<truncate> X<truncating>
-Because you're using something like this, which truncates the file and
+Because you're using something like this, which truncates the file
I<then> gives you read-write access:
open my $fh, '+>', '/path/name'; # WRONG (almost always)
Whoops. You should instead use this, which will fail if the file
-doesn't exist.
+doesn't exist:
open my $fh, '+<', '/path/name'; # open for update
Using ">" always clobbers or creates. Using "<" never does
either. The "+" doesn't change this.
-Here are examples of many kinds of file opens. Those using sysopen()
-all assume
+Here are examples of many kinds of file opens. Those using C<sysopen>
+all assume that you've pulled in the constants from C<Fcntl>:
use Fcntl;
To open file for reading:
- open my $fh, '<', $path or die $!;
- sysopen my $fh, $path, O_RDONLY or die $!;
+ open my $fh, '<', $path or die $!;
+ sysopen my $fh, $path, O_RDONLY or die $!;
To open file for writing, create new file if needed or else truncate old file:
- open my $fh, '>', $path or die $!;
- sysopen my $fh, $path, O_WRONLY|O_TRUNC|O_CREAT or die $!;
- sysopen my $fh, $path, O_WRONLY|O_TRUNC|O_CREAT, 0666 or die $!;
+ open my $fh, '>', $path or die $!;
+ sysopen my $fh, $path, O_WRONLY|O_TRUNC|O_CREAT or die $!;
+ sysopen my $fh, $path, O_WRONLY|O_TRUNC|O_CREAT, 0666 or die $!;
To open file for writing, create new file, file must not exist:
- sysopen my $fh, $path, O_WRONLY|O_EXCL|O_CREAT or die $!;
- sysopen my $fh, $path, O_WRONLY|O_EXCL|O_CREAT, 0666 or die $!;
+ sysopen my $fh, $path, O_WRONLY|O_EXCL|O_CREAT or die $!;
+ sysopen my $fh, $path, O_WRONLY|O_EXCL|O_CREAT, 0666 or die $!;
To open file for appending, create if necessary:
- open my $fh, '>>' $path or die $!;
- sysopen my $fh, $path, O_WRONLY|O_APPEND|O_CREAT or die $!;
- sysopen my $fh, $path, O_WRONLY|O_APPEND|O_CREAT, 0666 or die $!;
+ open my $fh, '>>' $path or die $!;
+ sysopen my $fh, $path, O_WRONLY|O_APPEND|O_CREAT or die $!;
+ sysopen my $fh, $path, O_WRONLY|O_APPEND|O_CREAT, 0666 or die $!;
To open file for appending, file must exist:
- sysopen my $fh, $path, O_WRONLY|O_APPEND or die $!;
+ sysopen my $fh, $path, O_WRONLY|O_APPEND or die $!;
To open file for update, file must exist:
- open my $fh, '+<', $path or die $!;
- sysopen my $fh, $path, O_RDWR or die $!;
+ open my $fh, '+<', $path or die $!;
+ sysopen my $fh, $path, O_RDWR or die $!;
To open file for update, create file if necessary:
- sysopen my $fh, $path, O_RDWR|O_CREAT or die $!;
- sysopen my $fh, $path, O_RDWR|O_CREAT, 0666 or die $!;
+ sysopen my $fh, $path, O_RDWR|O_CREAT or die $!;
+ sysopen my $fh, $path, O_RDWR|O_CREAT, 0666 or die $!;
To open file for update, file must not exist:
- sysopen my $fh, $path, O_RDWR|O_EXCL|O_CREAT or die $!;
- sysopen my $fh, $path, O_RDWR|O_EXCL|O_CREAT, 0666 or die $!;
+ sysopen my $fh, $path, O_RDWR|O_EXCL|O_CREAT or die $!;
+ sysopen my $fh, $path, O_RDWR|O_EXCL|O_CREAT, 0666 or die $!;
To open a file without blocking, creating if necessary:
@@ -1081,11 +1128,11 @@ string without actually storing it in memory:
map_file my $string, $filename;
Once mapped, you can treat C<$string> as you would any other string.
-Since you don't actually load the data, mmap-ing is very fast and does
-not increase your memory footprint.
+Since you don't necessarily have to load the data, mmap-ing can be
+very fast and may not increase your memory footprint.
-If you really want to load the entire file, you can use the
-C<File::Slurp> module to do it in one step.
+If you want to load the entire file, you can use the C<File::Slurp>
+module to do it in one one simple and efficient step:
use File::Slurp;
@@ -1111,13 +1158,14 @@ you see someone do this:
You should think long and hard about why you need everything loaded at
once. It's just not a scalable solution. You might also find it more
-fun to use the standard Tie::File module, or the DB_File module's
-$DB_RECNO bindings, which allow you to tie an array to a file so that
+fun to use the standard C<Tie::File> module, or the C<DB_File> module's
+C<$DB_RECNO> bindings, which allow you to tie an array to a file so that
accessing an element the array actually accesses the corresponding
line in the file.
You can read the entire filehandle contents into a scalar.
+ my $var;
{
local $/;
open my $fh, '<', $file or die "can't open $file: $!";
@@ -1127,14 +1175,19 @@ You can read the entire filehandle contents into a
scalar.
That temporarily undefs your record separator, and will automatically
close the file at block exit. If the file is already open, just use this:
- $var = do { local $/; <$fh> };
+ my $var = do { local $/; <$fh> };
+
+You can do that one better by using a localized C<@ARGV> so you can
+eliminate the C<open>:
+
+ my $var = do { local( @ARGV, $/ ) = $file; <> };
-For ordinary files you can also use the read function.
+For ordinary files you can also use the C<read> function.
read( $fh, $var, -s $fh );
-The third argument tests the byte size of the data on the INPUT filehandle
-and reads that many bytes into the buffer $var.
+That third argument tests the byte size of the data on the C<INPUT> filehandle
+and reads that many bytes into the buffer C<$var>.
=head2 How can I read in a file by paragraphs?
X<file, reading by paragraphs>
diff --git a/pod/perlfaq6.pod b/pod/perlfaq6.pod
index fe91933..0238c9a 100644
--- a/pod/perlfaq6.pod
+++ b/pod/perlfaq6.pod
@@ -5,7 +5,7 @@ perlfaq6 - Regular Expressions
=head1 DESCRIPTION
This section is surprisingly small because the rest of the FAQ is
-littered with answers involving regular expressions. For example,
+littered with answers involving regular expressions. For example,
decoding a URL and checking whether something is a number are handled
with regular expressions, but those answers are found elsewhere in
this document (in L<perlfaq9>: "How do I decode or create those %-encodings
@@ -34,7 +34,7 @@ comments.
The C</x> modifier causes whitespace to be ignored in a regex pattern
(except in a character class and a few other places), and also allows you to
-use normal comments there, too. As you can imagine, whitespace and comments
+use normal comments there, too. As you can imagine, whitespace and comments
help a lot.
C</x> lets you turn this:
@@ -60,9 +60,9 @@ describing the meaning of each part of the pattern.
=item Different Delimiters
While we normally think of patterns as being delimited with C</>
-characters, they can be delimited by almost any character. L<perlre>
-describes this. For example, the C<s///> above uses braces as
-delimiters. Selecting another delimiter can avoid quoting the
+characters, they can be delimited by almost any character. L<perlre>
+describes this. For example, the C<s///> above uses braces as
+delimiters. Selecting another delimiter can avoid quoting the
delimiter within the pattern:
s/\/usr\/local/\/usr\/share/g; # bad delimiter choice
@@ -70,14 +70,14 @@ delimiter within the pattern:
=back
-=head2 I'm having trouble matching over more than one line. What's wrong?
+=head2 I'm having trouble matching over more than one line. What's wrong?
X<regex, multiline> X<regexp, multiline> X<regular expression, multiline>
Either you don't have more than one line in the string you're looking
at (probably), or else you aren't using the correct modifier(s) on
your pattern (possibly).
-There are many ways to get multiline data into a string. If you want
+There are many ways to get multiline data into a string. If you want
it to happen automatically while reading input, you'll want to set $/
(probably to '' for paragraphs or C<undef> for the whole file) to
allow you to read more than one line at a time.
@@ -85,15 +85,15 @@ allow you to read more than one line at a time.
Read L<perlre> to help you decide which of C</s> and C</m> (or both)
you might want to use: C</s> allows dot to include newline, and C</m>
allows caret and dollar to match next to a newline, not just at the
-end of the string. You do need to make sure that you've actually
+end of the string. You do need to make sure that you've actually
got a multiline string in there.
For example, this program detects duplicate words, even when they span
-line breaks (but not paragraph ones). For this example, we don't need
+line breaks (but not paragraph ones). For this example, we don't need
C</s> because we aren't using dot in a regular expression that we want
-to cross line boundaries. Neither do we need C</m> because we aren't
+to cross line boundaries. Neither do we need C</m> because we aren't
wanting caret or dollar to match at any point inside the record next
-to newlines. But it's imperative that $/ be set to something other
+to newlines. But it's imperative that $/ be set to something other
than the default, or else we won't actually ever have a multiline
record read in.
@@ -194,7 +194,7 @@ Good luck!
X<$/, regexes in> X<$INPUT_RECORD_SEPARATOR, regexes in>
X<$RS, regexes in>
-$/ has to be a string. You can use these examples if you really need to
+$/ has to be a string. You can use these examples if you really need to
do this.
If you have File::Stream, this is easy.
@@ -211,7 +211,7 @@ If you have File::Stream, this is easy.
If you don't have File::Stream, you have to do a little more work.
You can use the four-argument form of sysread to continually add to
-a buffer. After you add to the buffer, you check if you have a
+a buffer. After you add to the buffer, you check if you have a
complete line (using your regular expression).
local $_ = "";
@@ -239,7 +239,7 @@ being in memory at the end.
X<replace, case preserving> X<substitute, case preserving>
X<substitution, case preserving> X<s, case preserving>
-Here's a lovely Perlish solution by Larry Rosler. It exploits
+Here's a lovely Perlish solution by Larry Rosler. It exploits
properties of bitwise xor on ASCII strings.
$_= "this is a TEsT case";
@@ -329,7 +329,7 @@ the case of the last character is used for the rest of the
substitution.
=head2 How can I make C<\w> match national character sets?
X<\w>
-Put C<use locale;> in your script. The \w character class is taken
+Put C<use locale;> in your script. The \w character class is taken
from the current locale.
See L<perllocale> for details.
@@ -342,7 +342,7 @@ documented in L<perlre>.
No matter which locale you are in, the alphabetic characters are
the characters in \w without the digits and the underscore.
-As a regex, that looks like C</[^\W\d_]/>. Its complement,
+As a regex, that looks like C</[^\W\d_]/>. Its complement,
the non-alphabetics, is then everything in \W along with
the digits and the underscore, or C</[\W\d_]/>.
@@ -350,11 +350,11 @@ the digits and the underscore, or C</[\W\d_]/>.
X<regex, escaping> X<regexp, escaping> X<regular expression, escaping>
The Perl parser will expand $variable and @variable references in
-regular expressions unless the delimiter is a single quote. Remember,
+regular expressions unless the delimiter is a single quote. Remember,
too, that the right-hand side of a C<s///> substitution is considered
-a double-quoted string (see L<perlop> for more details). Remember
+a double-quoted string (see L<perlop> for more details). Remember
also that any regex special characters will be acted on unless you
-precede the substitution with \Q. Here's an example:
+precede the substitution with \Q. Here's an example:
$string = "Placido P. Octopus";
$regex = "P.";
@@ -443,9 +443,9 @@ For example, this one-liner
perl -0777 -pe 's{/\*.*?\*/}{}gs' foo.c
-will work in many but not all cases. You see, it's too simple-minded for
+will work in many but not all cases. You see, it's too simple-minded for
certain kinds of C programs, in particular, those with what appear to be
-comments in quoted strings. For that, you'd need something like this,
+comments in quoted strings. For that, you'd need something like this,
created by Jeffrey Friedl and later modified by Fred Curtis.
$/ = undef;
@@ -454,7 +454,7 @@ created by Jeffrey Friedl and later modified by Fred Curtis.
print;
This could, of course, be more legibly written with the C</x> modifier, adding
-whitespace and comments. Here it is expanded, courtesy of Fred Curtis.
+whitespace and comments. Here it is expanded, courtesy of Fred Curtis.
s{
/\* ## Start of /* ... */ comment
@@ -524,7 +524,7 @@ nesting. There are five total groups in angle brackets:
<another group <nested once <nested twice> > >
and that's it.
-The regular expression to match the balanced text uses two new (to
+The regular expression to match the balanced text uses two new (to
Perl 5.10) regular expression features. These are covered in L<perlre>
and this example is a modified version of one in that documentation.
@@ -537,7 +537,7 @@ backtracking.
Second, the new C<(?PARNO)> refers to the sub-pattern in the
particular capture group given by C<PARNO>. In the following regex,
the first capture group finds (and remembers) the balanced text, and
-you need that same pattern within the first buffer to get past the
+you need that same pattern within the first buffer to get past the
nested text. That's the recursive part. The C<(?1)> uses the pattern
in the outer capture group as an independent part of the regex.
@@ -626,13 +626,13 @@ first and the nested matches so up later:
Found:
<nested twice>
-=head2 What does it mean that regexes are greedy? How can I get around it?
+=head2 What does it mean that regexes are greedy? How can I get around it?
X<greedy> X<greediness>
Most people mean that greedy regexes match as much as they can.
Technically speaking, it's actually the quantifiers (C<?>, C<*>, C<+>,
C<{}>) that are greedy rather than the whole pattern; Perl prefers local
-greed and immediate gratification to overall greed. To get non-greedy
+greed and immediate gratification to overall greed. To get non-greedy
versions of the same quantifiers, use (C<??>, C<*?>, C<+?>, C<{}?>).
An example:
@@ -642,7 +642,7 @@ An example:
$s2 =~ s/ve.*?y //; # I am very cold
Notice how the second substitution stopped matching as soon as it
-encountered "y ". The C<*?> quantifier effectively tells the regular
+encountered "y ". The C<*?> quantifier effectively tells the regular
expression engine to find a match as quickly as possible and pass
control on to whatever is next in line, like you would if you were
playing hot potato.
@@ -672,7 +672,7 @@ might consider
=head2 How can I print out a word-frequency or line-frequency summary?
-To do this, you have to parse out each word in the input stream. We'll
+To do this, you have to parse out each word in the input stream. We'll
pretend that by word you mean chunk of alphabetics, hyphens, or
apostrophes, rather than the non-whitespace chunk idea of a word given
in the previous question:
@@ -710,40 +710,47 @@ See the module String::Approx available from CPAN.
X<regex, efficiency> X<regexp, efficiency>
X<regular expression, efficiency>
-( contributed by brian d foy )
+(contributed by brian d foy)
-Avoid asking Perl to compile a regular expression every time
-you want to match it. In this example, perl must recompile
-the regular expression for every iteration of the C<foreach>
-loop since it has no way to know what $pattern will be.
+If you have Perl 5.10 or later, this is almost trivial. You just smart
+match against an array of regular expression objects:
- @patterns = qw( foo bar baz );
+ my @patterns = ( qr/Fr.d/, qr/B.rn.y/, qr/W.lm./ );
+
+ if( $string ~~ @patterns ) {
+ ...
+ };
- LINE: while( <DATA> )
- {
- foreach $pattern ( @patterns )
- {
- if( /\b$pattern\b/i )
- {
+The smart match stops when it finds a match, so it doesn't have to try
+every expression.
+
+Earlier than Perl 5.10, you have a bit of work to do. You want to
+avoid compiling a regular expression every time you want to match it.
+In this example, perl must recompile the regular expression for every
+iteration of the C<foreach> loop since it has no way to know what
+C<$pattern> will be:
+
+ my @patterns = qw( foo bar baz );
+
+ LINE: while( <DATA> ) {
+ foreach $pattern ( @patterns ) {
+ if( /\b$pattern\b/i ) {
print;
next LINE;
}
}
}
-The C<qr//> operator showed up in perl 5.005. It compiles a
-regular expression, but doesn't apply it. When you use the
-pre-compiled version of the regex, perl does less work. In
-this example, I inserted a C<map> to turn each pattern into
-its pre-compiled form. The rest of the script is the same,
-but faster.
+The C<qr//> operator showed up in perl 5.005. It compiles a regular
+expression, but doesn't apply it. When you use the pre-compiled
+version of the regex, perl does less work. In this example, I inserted
+a C<map> to turn each pattern into its pre-compiled form. The rest of
+the script is the same, but faster:
- @patterns = map { qr/\b$_\b/i } qw( foo bar baz );
+ my @patterns = map { qr/\b$_\b/i } qw( foo bar baz );
- LINE: while( <> )
- {
- foreach $pattern ( @patterns )
- {
+ LINE: while( <> ) {
+ foreach $pattern ( @patterns ) {
if( /$pattern/ )
{
print;
@@ -752,22 +759,21 @@ but faster.
}
}
-In some cases, you may be able to make several patterns into
-a single regular expression. Beware of situations that require
-backtracking though.
+In some cases, you may be able to make several patterns into a single
+regular expression. Beware of situations that require backtracking
+though.
- $regex = join '|', qw( foo bar baz );
+ my $regex = join '|', qw( foo bar baz );
- LINE: while( <> )
- {
+ LINE: while( <> ) {
print if /\b(?:$regex)\b/i;
}
For more details on regular expression efficiency, see I<Mastering
-Regular Expressions> by Jeffrey Freidl. He explains how regular
+Regular Expressions> by Jeffrey Freidl. He explains how regular
expressions engine work and why some patterns are surprisingly
-inefficient. Once you understand how perl applies regular
-expressions, you can tune them for individual situations.
+inefficient. Once you understand how perl applies regular expressions,
+you can tune them for individual situations.
=head2 Why don't word-boundary searches with C<\b> work for me?
X<\b>
@@ -787,7 +793,7 @@ meaning that it doesn't represent a character in the
string, but a
condition at a certain position.
For the regular expression, /\bPerl\b/, there has to be a word
-boundary before the "P" and after the "l". As long as something other
+boundary before the "P" and after the "l". As long as something other
than a word character precedes the "P" and succeeds the "l", the
pattern will match. These strings match /\bPerl\b/.
@@ -801,8 +807,8 @@ These strings do not match /\bPerl\b/.
"Perl_" # _ is a word char!
"Perler" # no word char before P, but one after l
-You don't have to use \b to match words though. You can look for
-non-word characters surrounded by word characters. These strings
+You don't have to use \b to match words though. You can look for
+non-word characters surrounded by word characters. These strings
match the pattern /\b'\b/.
"don't" # the ' char is surrounded by "n" and "t"
@@ -843,7 +849,7 @@ really appreciate them. As of the 5.005 release, the $&
variable is no
longer "expensive" the way the other two are.
Since Perl 5.6.1 the special variables @- and @+ can functionally replace
-$`, $& and $'. These arrays contain pointers to the beginning and end
+$`, $& and $'. These arrays contain pointers to the beginning and end
of each match (see perlvar for the full story), so they give you
essentially the same information, but without the risk of excessive
string copying.
@@ -857,12 +863,12 @@ regular expression with the C</p> modifier.
X<\G>
You use the C<\G> anchor to start the next match on the same
-string where the last match left off. The regular
+string where the last match left off. The regular
expression engine cannot skip over any characters to find
the next match with this anchor, so C<\G> is similar to the
-beginning of string anchor, C<^>. The C<\G> anchor is typically
-used with the C<g> flag. It uses the value of C<pos()>
-as the position to start the next match. As the match
+beginning of string anchor, C<^>. The C<\G> anchor is typically
+used with the C<g> flag. It uses the value of C<pos()>
+as the position to start the next match. As the match
operator makes successive matches, it updates C<pos()> with the
position of the next character past the last match (or the
first character of the next match, depending on how you like
@@ -870,7 +876,7 @@ to look at it). Each string has its own C<pos()> value.
Suppose you want to match all of consecutive pairs of digits
in a string like "1122a44" and stop matching when you
-encounter non-digits. You want to match C<11> and C<22> but
+encounter non-digits. You want to match C<11> and C<22> but
the letter <a> shows up between C<22> and C<44> and you want
to stop at C<a>. Simply matching pairs of digits skips over
the C<a> and still matches C<44>.
@@ -879,7 +885,7 @@ the C<a> and still matches C<44>.
my @pairs = m/(\d\d)/g; # qw( 11 22 44 )
If you use the C<\G> anchor, you force the match after C<22> to
-start with the C<a>. The regular expression cannot match
+start with the C<a>. The regular expression cannot match
there since it does not find a digit, so the next match
fails and the match operator returns the pairs it already
found.
@@ -939,7 +945,7 @@ which works in 5.004 or later.
}
For each line, the C<PARSER> loop first tries to match a series
-of digits followed by a word boundary. This match has to
+of digits followed by a word boundary. This match has to
start at the place the last match left off (or the beginning
of the string on the first match). Since C<m/ \G( \d+\b
)/gcx> uses the C<c> flag, if the string does not match that
@@ -947,16 +953,16 @@ regular expression, perl does not reset pos() and the next
match starts at the same position to try a different
pattern.
-=head2 Are Perl regexes DFAs or NFAs? Are they POSIX compliant?
+=head2 Are Perl regexes DFAs or NFAs? Are they POSIX compliant?
X<DFA> X<NFA> X<POSIX>
While it's true that Perl's regular expressions resemble the DFAs
(deterministic finite automata) of the egrep(1) program, they are in
fact implemented as NFAs (non-deterministic finite automata) to allow
-backtracking and backreferencing. And they aren't POSIX-style either,
-because those guarantee worst-case behavior for all cases. (It seems
+backtracking and backreferencing. And they aren't POSIX-style either,
+because those guarantee worst-case behavior for all cases. (It seems
that some people prefer guarantees of consistency, even when what's
-guaranteed is slowness.) See the book "Mastering Regular Expressions"
+guaranteed is slowness.) See the book "Mastering Regular Expressions"
(from O'Reilly) by Jeffrey Friedl for all the details you could ever
hope to know on these matters (a full citation appears in
L<perlfaq2>).
@@ -979,14 +985,14 @@ X<regex, and multibyte characters> X<regexp, and
multibyte characters>
X<regular expression, and multibyte characters> X<martian> X<encoding, Martian>
Starting from Perl 5.6 Perl has had some level of multibyte character
-support. Perl 5.8 or later is recommended. Supported multibyte
+support. Perl 5.8 or later is recommended. Supported multibyte
character repertoires include Unicode, and legacy encodings
-through the Encode module. See L<perluniintro>, L<perlunicode>,
+through the Encode module. See L<perluniintro>, L<perlunicode>,
and L<Encode>.
If you are stuck with older Perls, you can do Unicode with the
C<Unicode::String> module, and character conversions using the
-C<Unicode::Map8> and C<Unicode::Map> modules. If you are using
+C<Unicode::Map8> and C<Unicode::Map> modules. If you are using
Japanese encodings, you might try using the jperl 5.005_03.
Finally, the following set of approaches was offered by Jeffrey
@@ -1004,9 +1010,9 @@ nine characters 'I', ' ', 'a', 'm', ' ', 'CV', 'SG',
'XX', '!'.
Now, say you want to search for the single character C</GX/>. Perl
doesn't know about Martian, so it'll find the two bytes "GX" in the "I
-am CVSGXX!" string, even though that character isn't there: it just
+am CVSGXX!" string, even though that character isn't there: it just
looks like it is because "SG" is next to "XX", but there's no real
-"GX". This is a big problem.
+"GX". This is a big problem.
Here are a few ways, all painful, to deal with it:
@@ -1040,7 +1046,7 @@ Goldberg, who uses a zero-width negative look-behind
assertion.
/x;
This succeeds if the "martian" character GX is in the string, and fails
-otherwise. If you don't like using (?<!), a zero-width negative
+otherwise. If you don't like using (?<!), a zero-width negative
look-behind assertion, you can replace (?<![A-Z]) with (?:^|[^A-Z]).
It does have the drawback of putting the wrong thing in $-[0] and $+[0],
@@ -1099,7 +1105,7 @@ for more details).
if( $string =~ m/\Q$regex\E/ ) { ... }
Alternately, you can use C<qr//>, the regular expression quote operator (see
-L<perlop> for more details). It quotes and perhaps compiles the pattern,
+L<perlop> for more details). It quotes and perhaps compiles the pattern,
and you can apply regular expression flags to the pattern.
chomp( my $input = <STDIN> );
@@ -1137,7 +1143,7 @@ This documentation is free; you can redistribute it
and/or modify it
under the same terms as Perl itself.
Irrespective of its distribution, all code examples in this file
-are hereby placed into the public domain. You are permitted and
+are hereby placed into the public domain. You are permitted and
encouraged to use this code in your own programs for fun
-or for profit as you see fit. A simple comment in the code giving
+or for profit as you see fit. A simple comment in the code giving
credit would be courteous but is not required.
diff --git a/pod/perlfaq8.pod b/pod/perlfaq8.pod
index 08a5b03..99af392 100644
--- a/pod/perlfaq8.pod
+++ b/pod/perlfaq8.pod
@@ -545,7 +545,7 @@ details of this, see the section on signals, especially the
time-out
handler for a blocking C<flock()> in L<perlipc/"Signals"> or the
section on "Signals" in I<Programming Perl>.
-If exception handling is all you're interested in, use one of the
+If exception handling is all you're interested in, use one of the
many CPAN modules that handle exceptions, such as C<Try::Tiny>.
If you want the C<atexit()> syntax (and an C<rmexit()> as well), try the
@@ -1263,7 +1263,7 @@ See the entry for C<use> in L<perlfunc> for more details.
=head2 How do I keep my own module/library directory?
-When you build modules, tell Perl where to install the modules.
+When you build modules, tell Perl where to install the modules.
If you want to install modules for your own use, the easiest way might
be C<local::lib>, which you can download from CPAN. It sets various
@@ -1349,9 +1349,9 @@ directory of the currently running script and puts it in
C<$Bin>, which
you can then use to construct the right library path:
use FindBin qw($Bin);
-
+
You can also use C<local::lib> to do much of the same thing. Install
-modules using C<local::lib>'s settings then use the module in your
+modules using C<local::lib>'s settings then use the module in your
program:
use local::lib; # sets up a local lib at ~/perl5
@@ -1386,9 +1386,9 @@ environment variables, run-time switches, and in-code
statements:
=item the C<local::lib> module:
use local::lib;
-
+
use local::lib "~/myown_perllib";
-
+
=back
The last is particularly useful because it knows about machine
diff --git a/pod/perlfaq9.pod b/pod/perlfaq9.pod
index 7ea553a..1c62cef 100644
--- a/pod/perlfaq9.pod
+++ b/pod/perlfaq9.pod
@@ -19,8 +19,6 @@ comp.infosystems.www.authoring.cgi
The CGI specification is outlined in an informational RFC:
http://www.ietf.org/rfc/rfc3875
-Other relevant documentation listed in: http://www.perl.org/CGI_MetaFAQ.html
-
These Perl FAQs very selectively cover some CGI issues. However, Perl
programmers are strongly advised to use the C<CGI.pm> module, to take care
of the details for them.
@@ -45,25 +43,15 @@ Using C<CGI.pm> gives excellent platform independence,
including EBCDIC
systems. C<CGI.pm> selects an appropriate newline representation
(C<$CGI::CRLF>) and sets binmode as appropriate.
-=head2 My CGI script runs from the command line but not the browser. (500
Server Error)
-
-Several things could be wrong. You can go through the "Troubleshooting
-Perl CGI scripts" guide at
+=head2 My CGI script runs from the command line but not the browser. (500
Server Error)
- http://www.perl.org/troubleshooting_CGI.html
-
-If, after that, you can demonstrate that you've read the FAQs and that
-your problem isn't something simple that can be easily answered, you'll
-probably receive a courteous and useful reply to your question if you
-post it on comp.infosystems.www.authoring.cgi (if it's something to do
-with HTTP or the CGI protocols). Questions that appear to be Perl
-questions but are really CGI ones that are posted to comp.lang.perl.misc
-are not so well received.
+(contributed by brian d foy)
-The useful FAQs, related documents, and troubleshooting guides are
-listed in the CGI Meta FAQ:
+There are many things that might be wrong with your CGI program, and only
+some of them might be related to Perl. Try going through the troubleshooting
+guide on Perlmonks:
- http://www.perl.org/CGI_MetaFAQ.html
+ http://www.perlmonks.org/?node_id=380424
=head2 How can I get better error messages from a CGI program?
@@ -351,9 +339,20 @@ a DBI compatible driver. C<HTTPD::UserAdmin> supports
files used by the
=head2 How do I make sure users can't enter values into a form that cause my
CGI script to do bad things?
-See the security references listed in the CGI Meta FAQ
+(contributed by brian d foy)
+
+You can't really prevent people from sending your script bad data, at
+least not with Perl, which works on the server side. If you want to
+prevent data that try to use SQL injection or other sorts of attacks
+(and you should want to), you have to not trust any data that enter
+your program.
- http://www.perl.org/CGI_MetaFAQ.html
+The L<perlsec> documentation has general advice about data security.
+If you are using the C<DBI> module, use placeholder to fill in data.
+If you are running external programs with C<system> or C<exec>, use
+the list forms. There are many other precautions that you should take,
+too many to list here, and most of them fall under the category of not
+using any data that you don't intend to use. Trust no one.
=head2 How do I parse a mail header?
@@ -657,12 +656,23 @@ This can make tasks like fetching the newsgroup list as
simple as
=head2 How do I fetch/put an FTP file?
-C<LWP::Simple> (available from CPAN) can fetch but not put. C<Net::FTP> (also
-available from CPAN) is more complex but can put as well as fetch.
+(contributed by brian d foy)
+
+The C<LWP> family of modules (available on CPAN as the libwww-perl distibution)
+can work with FTP just like it can with many other protocols. C<LWP::Simple>
+makes it quite easy to fetch a file:
+
+ use LWP::Simple;
+
+ my $data = get( 'ftp://some.ftp.site/some/file.txt' );
+
+If you want more direct or low-level control of the FTP process, you can use
+the C<Net::FTP> module (in the Standard Library since Perl 5.8). It's
+documentation has examples showing you just how to do that.
=head2 How can I do RPC in Perl?
-(Contributed by brian d foy)
+(contributed by brian d foy)
Use one of the RPC modules you can find on CPAN (
http://search.cpan.org/search?query=RPC&mode=all ).
--
Perl5 Master Repository
