Processing large data files.

2007-04-06 Thread Bill Stephenson
Perl crashes when using the included script for large data files. How can I prevent this from happening? Kindest Regards, -- Bill Stephenson Sample data source is a Delimited text file with the following fields: FMT:STORE:TIKEY:TITLE:QTY:DC:COMPANY:ZONE DV:043712:20882903:DVD-ALL THE

Re: Processing large data files.

2007-04-06 Thread Mumia W.
On 04/06/2007 03:27 PM, Bill Stephenson wrote: Perl crashes when using the included script for large data files. How can I prevent this from happening? What do you mean by crashes? Does it abort with an error message placed in the web-server log file? If so, that's not a crash; it's just an

Text munging problem

2007-04-06 Thread Glenn Booth
Hi All, I'm a two-week perl newbie, trying to get my head around text handling. I keep getting sent badly formatted text files, which I have to 'repair' so that I can use them to feed an Oracle database. They are typically a few thousand lines long. The files generally arrive in a format like

Text parsing ?

2007-04-06 Thread Stanislav Nedelchev
Hi to all, I;m trying to parse one text file and insert it into database . Here is the example of Data CONTACT : Chemico Ltd. MANAGER : Michael Kor STATE: Blagoevgrad CODE : 4005 City: Blagoevgrad Address : Neupaoer Str. 1 Phone : 073/880755 Name : Valeri lItov lItov - Cadr Name : Nikoilai

Re: Text parsing ?

2007-04-06 Thread Jeff Pang
I;m trying to parse one text file and insert it into database . Here is the example of Data CONTACT : Chemico Ltd. MANAGER : Michael Kor STATE: Blagoevgrad CODE : 4005 City: Blagoevgrad Address : Neupaoer Str. 1 Phone : 073/880755 Name : Valeri lItov lItov - Cadr Name : Nikoilai Michev Michev

Re: Text munging problem

2007-04-06 Thread Seanie
Glenn Booth wrote: I need to sort out the cases of the text fields (BOOK TITLE) and ( a book about cats ) and render them to Title Case (first character upper case for each word). Anyone have an elegant way? ucfirst() does what you want: echo hello world | perl -ne 'print join , map

Re: Text munging problem

2007-04-06 Thread Chas Owens
On 4/5/07, Glenn Booth [EMAIL PROTECTED] wrote: snip My failed approach so far: While loop to read file line by line Split each line using delimiter (pipe in this case) Put the text fields into an array Shift each element out of the array Run a regex to upper case the first character Shift

Re: Text munging problem

2007-04-06 Thread D. Bolliger
Glenn Booth am Donnerstag, 5. April 2007 23:10: Hi All, Hi Glenn I'm a two-week perl newbie, trying to get my head around text handling. I keep getting sent badly formatted text files, which I have to 'repair' so that I can use them to feed an Oracle database. They are typically a few

Re: Text munging problem

2007-04-06 Thread Chas Owens
On 4/6/07, D. Bolliger [EMAIL PROTECTED] wrote: snip $ perl -nle 'print join , map ucfirst, map lc, split' in.txt out.txt snip There is no need to have multiple maps and the @ARGV/ trick handles files as well as stdin, so there is no need to use perl -lne 'print join , map { ucfirst lc }

Re: Text munging problem

2007-04-06 Thread Rob Dixon
Chas Owens wrote: On 4/5/07, Glenn Booth [EMAIL PROTECTED] wrote: snip My failed approach so far: While loop to read file line by line Split each line using delimiter (pipe in this case) Put the text fields into an array Shift each element out of the array Run a regex to upper case the first

Re: Text munging problem: question on while loop differences

2007-04-06 Thread Jeni Zundel
What is the difference between: a. while(defined(my $line = )) ... and b. while() On Apr 6, 2007, at 7:52 AM, Rob Dixon wrote: Why shift and unshift? Just use a for loop:

Re: Text munging problem: question on while loop differences

2007-04-06 Thread Jeff Pang
: Re: Text munging problem: question on while loop differences What is the difference between: a. while(defined(my $line = )) ... and b. while() Hello, When you say my $line = you read the content from input (maybe

Re: Text munging problem: question on while loop differences

2007-04-06 Thread Jeni Zundel
So, if I put a filehandle in the diamond, instead of empty diamond, does that mean that the first would operate line by line and the second would pull the whole file into memory? Thanks much, Jen On Apr 6, 2007, at 9:31 AM, Jeff Pang wrote: : Re: Text munging problem: question on while

Re: Text munging problem: question on while loop differences

2007-04-06 Thread Chas Owens
On 4/6/07, Jeni Zundel [EMAIL PROTECTED] wrote: What is the difference between: a. while(defined(my $line = )) ... and b. while() snip while () {} is shorthand for while (defined ($_ = )) {} The biggest difference is

Re: Text munging problem: question on while loop differences

2007-04-06 Thread Jeff Pang
So, if I put a filehandle in the diamond, instead of empty diamond, does that mean that the first would operate line by line and the second would pull the whole file into memory? When the diamond() is appeared with while(),it means you read the file line by line. ig,while(FILEHANDLE){ ..

Re: Text munging problem: question on while loop differences

2007-04-06 Thread Chas Owens
On 4/6/07, Jeni Zundel [EMAIL PROTECTED] wrote: So, if I put a filehandle in the diamond, instead of empty diamond, does that mean that the first would operate line by line and the second would pull the whole file into memory? No, both while loops read line by line. The difference is where

Re: Text munging problem: question on while loop differences

2007-04-06 Thread Chas Owens
On 4/6/07, Jeff Pang [EMAIL PROTECTED] wrote: So, if I put a filehandle in the diamond, instead of empty diamond, does that mean that the first would operate line by line and the second would pull the whole file into memory? When the diamond() is appeared with while(),it means you read the

Retrieving a web resource (GET) for parsing

2007-04-06 Thread yitzle
If I want to parse a few web pages, what's the best way to retrieve them? Should I just run `wget $url`?

Re: Text munging problem

2007-04-06 Thread oryann9
$field =~ s/\b(.)(.*?)\b/\u$1\L$2/g; $record .= $field|; ** Is this regex s/\b(.)(.*?)\b/ saying boundry between any character zero or more times in $1 up to everything else non-greedy end word boundry in $2 sort of confused since your end goal is to CAPS first letter in

Re: Text munging problem

2007-04-06 Thread Chas Owens
On 4/6/07, oryann9 [EMAIL PROTECTED] wrote: $field =~ s/\b(.)(.*?)\b/\u$1\L$2/g; $record .= $field|; ** Is this regex s/\b(.)(.*?)\b/ saying boundry between any character zero or more times in $1 up to everything else non-greedy end word boundry in $2 sort of confused

Parsing an HTML file - skipping to a line

2007-04-06 Thread yitzle
I got an HTML file I want to parse. I have it stored in an array or whatever. If I want to skip to a line and go from there, what's the best way? I'm thinking of doing something like: do { shift @input; } while ($input[0] =~ m/string-line to find/); (Or a while-do as needed.

Re: Parsing an HTML file - skipping to a line

2007-04-06 Thread Tom Phoenix
On 4/6/07, yitzle [EMAIL PROTECTED] wrote: I got an HTML file I want to parse. Then there's a module on CPAN to help you. HTML is very complex stuff. I have it stored in an array or whatever. If I want to skip to a line and go from there, what's the best way? I'm not sure what skip to a

Re: Parsing an HTML file - skipping to a line

2007-04-06 Thread yitzle
I distrust modules? Dunno. I'm a fan of C programming and like doing stuff myself. Granted HTML is free form, but I'm parsing a webpage whose format likely will not change anytime soon, and if it did, even if I was using a module I would still need to fix up the script. I know exactly was format

Re: Text parsing ?

2007-04-06 Thread Dr.Ruud
Jeff Pang schreef: $fname =~ s/^\s+|\s+$//g; Alternative: s/^\s+//, s/\s+$// for $fname; Test: perl -wle' $s =abc ; s/^\s+//, s/\s+$// for $s; print $s ' abc -- Affijn, Ruud Gewoon is een tijger. -- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands,

Re: Parsing an HTML file - skipping to a line

2007-04-06 Thread Rob Dixon
Tom Phoenix wrote: On 4/6/07, yitzle [EMAIL PROTECTED] wrote: I'm thinking of doing something like: do { shift @input; } while ($input[0] =~ m/string-line to find/); Real Perl programmers don't use indices. At least, not like that. I think real Perl programmers use

Re: Text parsing ?

2007-04-06 Thread Chas Owens
On 4/6/07, Dr.Ruud [EMAIL PROTECTED] wrote: Jeff Pang schreef: $fname =~ s/^\s+|\s+$//g; Alternative: s/^\s+//, s/\s+$// for $fname; snip Just because I like hard numbers when comparing methods: Rate one_regex two_regex one_regex 34.0/s-- -67% two_regex

Re: Parsing an HTML file - skipping to a line

2007-04-06 Thread Chas Owens
On 4/6/07, Rob Dixon [EMAIL PROTECTED] wrote: Tom Phoenix wrote: On 4/6/07, yitzle [EMAIL PROTECTED] wrote: I'm thinking of doing something like: do { shift @input; } while ($input[0] =~ m/string-line to find/); Real Perl programmers don't use indices. At least, not

Re: Parsing an HTML file - skipping to a line

2007-04-06 Thread Tom Phoenix
On 4/6/07, yitzle [EMAIL PROTECTED] wrote: I distrust modules? Dunno. I'm a fan of C programming and like doing stuff myself. I distrust the dairy industry. I've got a yard full of grass for cows to eat, and I've seen what milking a cow looks like on Animal Planet. I can figure out how to

Re: Text munging problem

2007-04-06 Thread D. Bolliger
Chas Owens am Freitag, 6. April 2007 13:27: On 4/6/07, D. Bolliger [EMAIL PROTECTED] wrote: $ perl -nle 'print join , map ucfirst, map lc, split' in.txt out.txt There is no need to have multiple maps and the @ARGV/ trick handles files as well as stdin, so there is no need to use

Re: Retrieving a web resource (GET) for parsing

2007-04-06 Thread Dave Gray
On 4/6/07, yitzle [EMAIL PROTECTED] wrote: If I want to parse a few web pages, what's the best way to retrieve them? Should I just run `wget $url`? LWP can do this for you. http://search.cpan.org/~gaas/libwww-perl-5.805/lib/LWP.pm If you want more complicated interactions, WWW::Mechanize is

Re: Text munging problem

2007-04-06 Thread John W. Krahn
D. Bolliger wrote: Chas Owens am Freitag, 6. April 2007 13:27: On 4/6/07, D. Bolliger [EMAIL PROTECTED] wrote: $ perl -nle 'print join , map ucfirst, map lc, split' in.txt out.txt There is no need to have multiple maps and the @ARGV/ trick handles files as well as stdin, so there is no need