Hi
Hello.
This scripts sucks in a 109mb file and i'm trying to do a search and replace on unxtime to the format from strftime. Which is working...
But I run this system call and it took allnight to run :(
So I killed it... Any other suggestions to reach my goal.....
You bet.
First, you're slurping the entire file into memory, but you only need one line at a time. We can do better.
Second, you're shelling out to another process to do a global find and replace. Unfortunately, you're doing that every single line. So if your file has 10,000 lines, you're reading each line 10,001 times. That's a LOT of reading and we aren't even talking about writing yet. We can do better there too.
A simple one-liner should do everything you need (quickly even):
perl -MPOSIX=strftime -pi.bak -e 's/^(\d+)/strftime "%Y-%m-%d %H:%M:%S", localtime $1/e' /tmp/ip.txt
Pretty simple, right?
Let's look at your code below.
#!/usr/bin/perl use strict; use POSIX 'strftime';
my $f = "/tmp/ip.txt";
open (FILE, "< /tmp/ip.txt") || die ("Open Failed: $!\n");
my @lines = <FILE>; # file into array
This is where you are slurping the file. When we only need one line at a time though, it's generally better to work that way:
while (<FILE>) {
# work with $_ here
}foreach (@lines) { next if /^S/; next if /^-/;
my @line = split(/\t/, $_);
my @time = split(/\n/, $line[0]);
$line[0] contains the first field of the line, a run of digits. No newlines. @time only gets one entry.
foreach (@time) {
Which means, there's no point in looping over one entry.
my $convtime = strftime('%Y-%m-%d %H:%M:%S', localtime($_));
system ("perl -pi -e 's/$_/$convtime/g' $f");
Avoid shelling out whenever possible. Here, you could have just written the results to an output file instead.
Hope that helps.
James
}
}
close FILE; #
ip.txt before script:
Start header header ----- ----- ----- 1074715516 rrr rrrr rrrr 1074715516 rrr rrrr rrrr 1074715516 rrr rrrr rrrr 1074715516 rrr rrrr rrrr
GOAL = ip.txt after script:
Start header header ----- ----- ----- 2004-01-21 12:05:16 rrr rrrr rrrr 2004-01-21 12:05:16 rrr rrrr rrrr 2004-01-21 12:05:16 rrr rrrr rrrr 2004-01-21 12:05:16 rrr rrrr rrrr
thanks upfront, Rob
-- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] <http://learn.perl.org/> <http://learn.perl.org/first-response>
-- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED] <http://learn.perl.org/> <http://learn.perl.org/first-response>
