Re: [SLUG] gzip from perl script
Voytek wrote: I just need to force a 'y' here, or pass to gzip ...screen snip--- gzip: /backup/mysql/postfix-20050922.sql.gz already exists; do you wish to overwrite (y or n)? ... I guess there's alway -f -f --force Force compression or decompression even if the file has multiple links or the corresponding file already exists, or if the com- pressed data is read from or written to a terminal. If the input data is not in a format recognized by gzip, and if the option --stdout is also given, copy the input data without change to the standard ouput: let zcat behave as cat. If -f is not given, and when not running in the background, gzip prompts to verify whether an existing file should be overwritten. if you have a copy of the man page available. ;) Jacinta -- (`-''-/).___..--''`-._ | Jacinta Richardson | `6_ 6 ) `-. ( ).`-.__.`) | Perl Training Australia| (_Y_.)' ._ ) `._ `. ``-..-' | +61 3 9354 6001| _..`--'_..-_/ /--'_.' ,' | [EMAIL PROTECTED] | (il),-'' (li),' ((!.-' | www.perltraining.com.au | -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
butfirst (was Re: [SLUG] gzip from perl script)
Erik de Castro Lopo wrote:
One thing thats been bugging me for a while about Perl is the
lack of a butfirst keyword. This would be *really* useful for
constructs like:
{
# Huge chunk of code
}
butfirst
{
# Second huge chunk of code to be executed
# before the one above.
} ;
G'day Erik,
I'm struggling to imagine any cases this would be useful excepting do-while
loops (which exist anyway). I'm also not entirely convinced the loss of
readability (if the first hunk of code was really big, I wouldn't know to look
for a butfirst block and might therefore misunderstand the code) would make it
practical.
Can you give some pared down examples of what kinds of things you'd put in each
block and in particular why you wouldn't just do the butfirst stuff before the
first block?
All the best,
Jainta
--
(`-''-/).___..--''`-._ | Jacinta Richardson |
`6_ 6 ) `-. ( ).`-.__.`) | Perl Training Australia|
(_Y_.)' ._ ) `._ `. ``-..-' | +61 3 9354 6001|
_..`--'_..-_/ /--'_.' ,' | [EMAIL PROTECTED] |
(il),-'' (li),' ((!.-' | www.perltraining.com.au |
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: butfirst (was Re: [SLUG] gzip from perl script)
On Thu, Sep 22, 2005 at 05:20:23PM +1000, Jacinta Richardson wrote:
One thing thats been bugging me for a while about Perl is the
lack of a butfirst keyword. This would be *really* useful for
constructs like:
{
# Huge chunk of code
}
butfirst
{
# Second huge chunk of code to be executed
# before the one above.
} ;
I'm struggling to imagine any cases this would be useful excepting do-while
loops (which exist anyway). I'm also not entirely convinced the loss of
readability (if the first hunk of code was really big, I wouldn't know to look
for a butfirst block and might therefore misunderstand the code) would make it
practical.
Erik has strong views on programming languages, and he's trolling you. At
least he's giving those of us who don't completely despise C++ a break for
once.
--
Mark
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: butfirst (was Re: [SLUG] gzip from perl script)
Erik has strong views on programming languages, and he's trolling you. At least he's giving those of us who don't completely despise C++ a break for once. I'm glad someone said that. For a while there I was worried. Still, everyone here at the office had a good laugh about it. -- Del -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: butfirst (was Re: [SLUG] gzip from perl script)
Mark Johnathan Greenaway wrote: Erik has strong views on programming languages, and he's trolling you. Guilty as charged :-) At least he's giving those of us who don't completely despise C++ a break for once. Yeah, I needed a five minute break :-). Erik -- +---+ Erik de Castro Lopo +---+ Any sufficiently complicated C or Fortran program contains an ad-hoc, informally-specified, bug-ridden, slow implementation of half of CommonLisp. -- Greenspuns Tenth Rule Of Programming -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: butfirst (was Re: [SLUG] gzip from perl script)
quote who=Erik de Castro Lopo Mark Johnathan Greenaway wrote: Erik has strong views on programming languages, and he's trolling you. Guilty as charged :-) Someday you'll catch someone unsuspecting, and they'll end up with intercal. Let me know when it happens - I'll buy the beer. - Jeff -- UbuntuBelowZero in Montreal! http://wiki.ubuntu.com/UbuntuBelowZero That whole 'you complete me' thing is just tragic and totally unrealistic. Go complete yourself. - Anon -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: butfirst (was Re: [SLUG] gzip from perl script)
Jeff Waugh wrote: Someday you'll catch someone unsuspecting, and they'll end up with intercal. Let me know when it happens - I'll buy the beer. shock=horror, disbelief http://www.catb.org/~esr/intercal/iperl.html /shock Erik -- +---+ Erik de Castro Lopo +---+ The day Microsoft makes something that doesn't suck is probably the day they start making vacuum cleaners. -- Ernst Jan Plugge -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] gzip from perl script
On Thu, Sep 22, 2005 at 02:50:13PM +1000, Erik de Castro Lopo wrote:
One thing thats been bugging me for a while about Perl is the
lack of a butfirst keyword. This would be *really* useful for
constructs like:
{
# Huge chunk of code
}
butfirst
{
# Second huge chunk of code to be executed
# before the one above.
} ;
Don't be silly, of course it does:
--
#!/usr/bin/perl -w
(eval{
print foo\n;
print baz\n;
}) if (eval{
print bar\n;
print bing\n;
print bong\n;
} and 1 );
--
or just define your own syntax:
--
#!/usr/bin/perl -w
sub later (@)
{
@_ = reverse @_;
my $x;
while( $x = shift ) { $x; }
}
sub butfirst (@) { @_ }
later
{
print foo\n;
print baz\n;
}
butfirst
{
print bar\n;
print bing\n;
print bong\n;
}
butfirst
{
print zap\n;
print atooey\n;
}
--
All of which I'm sure has been demonstrated elsewhere.
But then we all know that if a programming language can define what
is *really* useful then it might as well just define the actual
program (since at the end of the day, a running program that performs
a task is the only thing that is *really* useful). Once we have such
a programming language then we won't need programmers at all.
In the meantime, I'll settle for perl and make my own calls about
which features I find useful and which I don't.
- Tel
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
[SLUG] gzip from perl script
I'm using a Perl script for MySQL backup, I'd like to compress the
database dumps, the business end of the script runs like:
---
while (my @arr = $sth-fetchrow) {
print $arr[1]\n;
system(rm $backuppath/$arr[1]-$oldyear$oldmonth$oldday.sql);
system(mysqldump --opt $arr[1] -u $DB_User --password=$DB_Password
$backuppath/$arr[1]-$year$month$day.sql);
}
---
do I simply add like:
system(gzip $backuppath/$arr[1]-$year$month$day.sql);
(and change the 'rm' line extension?)
?
--
Voytek
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] gzip from perl script
On 9/22/05, Voytek [EMAIL PROTECTED] wrote: [snip] do I simply add like: system(gzip $backuppath/$arr[1]-$year$month$day.sql); (and change the 'rm' line extension?) ? http://search.cpan.org Look for the GZIP module, nicer way of doing it. Also, you probably want to use unlink instead of that system rm command. There's probably even a MySQL module to do the MySQL dump. Usually it's best using those as it allows to look at return codes and it generally looks much neater in the code. HTH. Cheers, Gonzalo -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] gzip from perl script
G'day Voytek,
In a spirit of defensive programming, I have an important question for you.
What happens if the mysqldump fails? Perhaps the database goes offline part way
through, perhaps the disk fills up... since you've just deleted your old backup,
what are you going to do?
I suggest the following changes to your code:
* Take the backup first
* Check that it didn't fail
* Then do compression
* Check that it didn't fail (find out the possible failure modes,
can it corrupt your file? What happens if all the disk space vanishes
half way through?
* Then, when you're fairly certain that things will be okay, delete the
old version.
while( my ($ignored, $database) = $sth-fetchrow() ) {
print $database\n;
# Make the backup
system(mysqldump --opt $database -u $DB_User --password=$DB_Password .
$backuppath/$database-$year$month$day.sql);
# Handle any errors
if($?) {
# something went wrong... try to guess what.
die some error as appropriate;
}
# Do the compression
system(gzip, $backuppath/$database-$year$month$day.sql);
# Handle any errors
if($?) {
# something went wrong... try to guess what.
die some error as appropriate;
}
# Now that you're fairly certain that the new backup has worked...
unless( unlink($backuppath/$database-$oldyear$oldmonth$oldday.sql.gz);
# something went wrong... try to guess what.
# This could just be that the file doesn't exist.
die if you think it's appropriate;
}
}
It might also be worth using chdir to change into $backuppath so that you don't
need to keep prepending that:
chdir $backuppath or die Failed to chdir to $backuppath $!;
You probably also want to wonder why you're pulling out a column from the
database and then ignoring it.
Whether or not it's better to use system or a module to do your compression
depends on a few things, like how easy it is to get modules installed, how easy
it is to use the module you select. If you do use a module you're more likely
to have intuitive error handling rather than having to check one of Perl's very
super extra special variables.
All the best,
Jacinta
--
(`-''-/).___..--''`-._ | Jacinta Richardson |
`6_ 6 ) `-. ( ).`-.__.`) | Perl Training Australia|
(_Y_.)' ._ ) `._ `. ``-..-' | +61 3 9354 6001|
_..`--'_..-_/ /--'_.' ,' | [EMAIL PROTECTED] |
(il),-'' (li),' ((!.-' | www.perltraining.com.au |
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] gzip from perl script
quote who=Jacinta Richardson
thanks, Jacinta
ahem, you assume I'm somewhat perl-literate, beyond knowing how to
paste'n'save... I'm not...
the script was kindly given to me in the past by a slugger
paste'n'save gave me an error on the last '}' in your additions, after
chopping it, I'm getting:
# perl mysqlbackup-day.pl
syntax error at mysqlbackup-day.pl line 61, near );
Execution of mysqlbackup-day.pl aborted due to compilation errors.
line 61 is:
unless( unlink($backuppath/$database-$oldyear$oldmonth$oldday.sql.gz);
script now is:
---
#!/usr/bin/perl
use DBI;
use Mysql;
use Date::Pcalc qw(:all);
$DB_Host = localhost;
$DB_Name = mysql;
$DB_User = backup;
$DB_Password = password;
$backuppath = /backup/mysql;
$myoffset = -3; # set the number of days back to delete. Basically -number
of days you wish to keep logs rotating for.
my $dbh =
Mysql-Connect($DB_Host;database=$DB_Name;,$DB_User,$DB_User,$DB_Password)
or die $Mysql::db_errstr;
($year,$month,$day) = Today();
if (length($day)==1) {
$day = 0$day;
}
if (length($month)==1) {
$month = 0$month;
}
($oldyear, $oldmonth, $oldday) = Add_Delta_YMD($year, $month, $day, 0, 0,
$myoffset);
if (length($oldday)==1) {
$oldday = 0$oldday;
}
if (length($oldmonth)==1) {
$oldmonth = 0$oldmonth;
}
$dbh-selectdb(mysql) or die $Mysql::db_errstr;
my $sth = Query $dbh SELECT * FROM db or die $Mysql::db_errstr;
while( my ($ignored, $database) = $sth-fetchrow() ) {
print $database\n;
# Make the backup
system(mysqldump --opt $database -u $DB_User --password=$DB_Password .
$backuppath/$database-$year$month$day.sql);
# Handle any errors
if($?) {
# something went wrong... try to guess what.
die some error as appropriate;
}
# Do the compression
system(gzip, $backuppath/$database-$year$month$day.sql);
# Handle any errors
if($?) {
# something went wrong... try to guess what.
die some error as appropriate;
}
# Now that you're fairly certain that the new backup has worked...
unless( unlink($backuppath/$database-$oldyear$oldmonth$oldday.sql.gz);
# something went wrong... try to guess what.
# This could just be that the file doesn't exist.
die if you think it's appropriate;
}
undef $sth;
undef $dbh;
---
--
Voytek
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] gzip from perl script
Voytek wrote:
quote who=Jacinta Richardson
thanks, Jacinta
ahem, you assume I'm somewhat perl-literate, beyond knowing how to
paste'n'save... I'm not...
My apologies. Although I do find that it's usually better to to assume that
people who are asking about an existing Perl program are more often Perl
literate than they are Perl newbies. :)
unless( unlink($backuppath/$database-$oldyear$oldmonth$oldday.sql.gz);
Ah. Remove the trailing ; and replace with a ) {
script now is:
---
#!/usr/bin/perl
use DBI;
use Mysql;
use Date::Pcalc qw(:all);
$DB_Host = localhost;
$DB_Name = mysql;
$DB_User = backup;
$DB_Password = password;
$backuppath = /backup/mysql;
$myoffset = -3; # set the number of days back to delete. Basically -number
of days you wish to keep logs rotating for.
my $dbh =
Mysql-Connect($DB_Host;database=$DB_Name;,$DB_User,$DB_User,$DB_Password)
or die $Mysql::db_errstr;
($year,$month,$day) = Today();
if (length($day)==1) {
$day = 0$day;
}
if (length($month)==1) {
$month = 0$month;
}
($oldyear, $oldmonth, $oldday) = Add_Delta_YMD($year, $month, $day, 0, 0,
$myoffset);
if (length($oldday)==1) {
$oldday = 0$oldday;
}
if (length($oldmonth)==1) {
$oldmonth = 0$oldmonth;
}
$dbh-selectdb(mysql) or die $Mysql::db_errstr;
my $sth = Query $dbh SELECT * FROM db or die $Mysql::db_errstr;
You probably don't need to to select *. But it doesn't hurt.
while( my ($ignored, $database) = $sth-fetchrow() ) {
print $database\n;
# Make the backup
system(mysqldump --opt $database -u $DB_User --password=$DB_Password .
$backuppath/$database-$year$month$day.sql);
# Handle any errors
if($?) {
# something went wrong... try to guess what.
die some error as appropriate;
}
You might want to make the above error somewhat more useful. Perhaps:
die Mysqldump failed for some reason.;
just so that you know... I've reindented. Hopefully you just lost the
indentation in your cut and paste.
# Do the compression
system(gzip, $backuppath/$database-$year$month$day.sql);
# Handle any errors
if($?) {
# something went wrong... try to guess what.
die some error as appropriate;
}
You probably want to change this error message as well.
die Failed to gzip file.;
# Now that you're fairly certain that the new backup has worked...
unless( unlink($backuppath/$database-$oldyear$oldmonth$oldday.sql.gz);
This should be:
unless( unlink($backuppath/$database-$oldyear$oldmonth$oldday.sql.gz)) {
# something went wrong... try to guess what.
# This could just be that the file doesn't exist.
die if you think it's appropriate;
}
On further thought you proably want to change this die to the following:
warn Failed to remove $database-$oldyear$oldmonth$oldday.sql.gz: $!;
as failing to delete might not be important enough to stop doing all the
backups.
All the best,
Jacinta
--
(`-''-/).___..--''`-._ | Jacinta Richardson |
`6_ 6 ) `-. ( ).`-.__.`) | Perl Training Australia|
(_Y_.)' ._ ) `._ `. ``-..-' | +61 3 9354 6001|
_..`--'_..-_/ /--'_.' ,' | [EMAIL PROTECTED] |
(il),-'' (li),' ((!.-' | www.perltraining.com.au |
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] gzip from perl script
On 9/21/05, Voytek [EMAIL PROTECTED] wrote:
...
do I simply add like:
system(gzip $backuppath/$arr[1]-$year$month$day.sql);
(and change the 'rm' line extension?)
Go for it. There's no reason to install the module when you can safely
do it via system(). In fact, don't be afraid of using it at all. You
would need a module in a system without the gzip CLI tool but equipped
with the library... which anyway seems quite unusual for a *nix.
If you're not using strict, there's no reason to use my before any
variables. But instead of removing them, go for strict. You'll
probably have a few problems getting the program to run, but mind the
error messages, correct them, and move on.
A quick solution for that would be, instead of using Perl (nothing
against it, I'm a Perl lover), do it in shell. Like in:
$ mysqldump -u user | gzip --best dbbackup.$(date +%s).gz
You won't be able to check for the gzipped file size *before* actually
writing it to disk (unless you take that gzip output and count the
number of bytes, then compare with a df output...). So you can trap a
failure by doing something like
#/bin/sh
if ! mysqldump -u user 2 ~./mybackuperrors.$$ | gzip --best ...
# then maybe
echo -n $(date +%s)-`cat .mybkperrror.$$` /var/log/mybackuperrors
# more error handling, log anything, bail out
And here's a MySQL backup system, with error logging. Cron it, and
you're off and running in no time.
I'm using a Perl script for MySQL backup, I'd like to compress the
database dumps, the business end of the script runs like:
---
while (my @arr = $sth-fetchrow) {
print $arr[1]\n;
system(rm $backuppath/$arr[1]-$oldyear$oldmonth$oldday.sql);
system(mysqldump --opt $arr[1] -u $DB_User --password=$DB_Password
$backuppath/$arr[1]-$year$month$day.sql);
}
---
do I simply add like:
system(gzip $backuppath/$arr[1]-$year$month$day.sql);
(and change the 'rm' line extension?)
?
--
Voytek
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
--
Julio C. Ody
http://rootshell.be/~julioody
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] gzip from perl script
quote who=Jacinta Richardson Voytek wrote: My apologies. Although I do find that it's usually better to to assume that people who are asking about an existing Perl program are more often Perl literate than they are Perl newbies. :) no need to apologize, I'm flattered On further thought you proably want to change this die to the following: warn Failed to remove $database-$oldyear$oldmonth$oldday.sql.gz: $!; as failing to delete might not be important enough to stop doing all the backups. yes, 'die' was a show stopper, 'warn' is better (especially as I'm deleting 3 generations prior, anyhow) I just need to force a 'y' here, or pass to gzip ...screen snip--- gzip: /backup/mysql/postfix-20050922.sql.gz already exists; do you wish to overwrite (y or n)? ... thanks for all the help, -- Voytek -- SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/ Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
Re: [SLUG] gzip from perl script
Jacinta Richardson wrote:
Voytek wrote:
quote who=Jacinta Richardson
unless( unlink($backuppath/$database-$oldyear$oldmonth$oldday.sql.gz);
Ah. Remove the trailing ; and replace with a ) {
Oh, cool, a Perl person.
One thing thats been bugging me for a while about Perl is the
lack of a butfirst keyword. This would be *really* useful for
constructs like:
{
# Huge chunk of code
}
butfirst
{
# Second huge chunk of code to be executed
# before the one above.
} ;
Sincerely,
Erik
--
+---+
Erik de Castro Lopo
+---+
Incompetence, like misery, seeks company. -- Erik Naggum
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
