Re: crontab last day of the month

2010-04-08 Thread Artur Grabowski
On Tue, Apr 6, 2010 at 12:29 PM, Alexander Hall ha...@openbsd.org wrote:
 frantisek holop wrote:
 hmm, on Tue, Apr 06, 2010 at 11:26:28AM +0200, Jan Stary said that
 On Apr 06 11:15:26, frantisek holop wrote:
 hi there,

 what happens if i specify a cronjob like this?

 23 59 31 * * $HOME/bin/whatever
 Cron will just do what it's told: run whatever at 31.*. 23:59

 so i could basically do 12 lines with the correct
 last day of the month rules :]


 No, you're forgetting about leap years... :-)

Don't forget leap seconds.

//art



Re: crontab last day of the month

2010-04-08 Thread Alexander Hall
Artur Grabowski wrote:
 On Tue, Apr 6, 2010 at 12:29 PM, Alexander Hall ha...@openbsd.org wrote:
 frantisek holop wrote:
 hmm, on Tue, Apr 06, 2010 at 11:26:28AM +0200, Jan Stary said that
 On Apr 06 11:15:26, frantisek holop wrote:
 hi there,

 what happens if i specify a cronjob like this?

 23 59 31 * * $HOME/bin/whatever
 Cron will just do what it's told: run whatever at 31.*. 23:59
 so i could basically do 12 lines with the correct
 last day of the month rules :]

 No, you're forgetting about leap years... :-)
 
 Don't forget leap seconds.

Heh, I did not care about them since cron's granularity is limited to
minutes... Hmmm... I wonder if they could potentially cause any grieve...



Re: crontab last day of the month

2010-04-07 Thread Jan Stary
On Apr 06 23:52:31, Antti Harri wrote:
 On Tue, 6 Apr 2010, Jan Stary wrote:

 No it's not. Unless you have some heavy traffic just around the midnight
 that is the border between months, the above newsyslog line makes
 the just-rotated access_log.0.gz contains the log of the 
 month-that-just-ended.

 OK. How do I reliably get month and year out of the file?

You already know that. If it's April 2010 now, then access_log.0.gz
contains the logs of March 2010. (Beware the logs of March!)

If you want to get the date _out_of_the_file_ (why would you,
you already know that), it's in each and every line of the access_log:

158.108.213.20 - - [07/Apr/2010:08:01:48 +0200] GET /hoeg/ HTTP/1.0 200 3771


 Month I can get from the newsyslog output it puts in the logs,
 getting year without doing some sort of nasty kludge is hard.

 m=$(zfgrep logfile turned over $file | tail -n 1 | cut -b 1-3)

This is the month (which you already know anyway).
The year is not even present in the turned over messages.
But again, it is present in each and every of the log records.

 I'm using www/visitors to produce something meaningful from those
 logs, and I would like to generate stats just once for a certain
 time period so I can later see stats further than one year that newsyslog
 keeps.

 webalizer

 I told I'm using visitors.

Which apparently cannot do it (or what exactly is your problem?).

Try webalizer, it doesn't depend on log files being particularly
named and/or sorted, juts reads the individual records, remembers
previous periods, what was already processed, etc.



Re: crontab last day of the month

2010-04-07 Thread Lars Nooden

On Tue, 6 Apr 2010, Antti Harri wrote:

OK. How do I reliably get month and year out of the file?


awk will work for that and for massaging archived logs.  That's also the 
kind of task perl is best at.


Another option is to reconfigure the log file format.  The regular Apache 
format

LogFormat %h %l %u %t \%r\ %s %b common

can be replaced with something much, much easier to parse.  For example:

LogFormat %h\t%l\t%u\t%t\t\%r\\t%s\t%b common

Because connections or transactions take a while to complete, tidying up 
will be necessary regardless of when the logs get rotated.


/Lars



Re: crontab last day of the month

2010-04-07 Thread Antti Harri

On Wed, 7 Apr 2010, Jan Stary wrote:


I told I'm using visitors.


Which apparently cannot do it (or what exactly is your problem?).

Try webalizer, it doesn't depend on log files being particularly
named and/or sorted, juts reads the individual records, remembers
previous periods, what was already processed, etc.


I don't want webalizer. Visitor parses the contents just fine
but the naming of the resulted files is the problem.

I stated in the initial mail that I want to generate stats based
on the rotated files just *once*. When a month goes by the logfile
with the same name doesn't contain the same contents that it
did one month ago.

It looks like nobody has an answer for this, so I'll leave it be.

--
Antti Harri



crontab last day of the month

2010-04-06 Thread frantisek holop
hi there,

what happens if i specify a cronjob like this?

23 59 31 * * $HOME/bin/whatever

does cron handle months that dont have 31 days?
i am looking for an alternative @monthly, not

0 0 1 * *

but the last minutes of the last day of the month.

-f
-- 
i have an exceptionally high q.i.



Re: crontab last day of the month

2010-04-06 Thread Jan Stary
On Apr 06 11:15:26, frantisek holop wrote:
 hi there,
 
 what happens if i specify a cronjob like this?
 
 23 59 31 * * $HOME/bin/whatever

Cron will just do what it's told: run whatever at 31.*. 23:59

 does cron handle months that dont have 31 days?
 i am looking for an alternative @monthly, not
 
 0 0 1 * *
 
 but the last minutes of the last day of the month.

Why?



Re: crontab last day of the month

2010-04-06 Thread Markus Hennecke

Am 06.04.2010 11:15, schrieb frantisek holop:

hi there,

what happens if i specify a cronjob like this?

23 59 31 * * $HOME/bin/whatever

does cron handle months that dont have 31 days?
i am looking for an alternative @monthly, not

0 0 1 * *

but the last minutes of the last day of the month.

   
Write a wrapper script which will only run if invoked on the last day of 
month and then use somehting like

23 59 28-31 * * run_on_last_day_of_month.sh $HOME/bin/whatever

Kind regards,
  Markus Hennecke



Re: crontab last day of the month

2010-04-06 Thread frantisek holop
hmm, on Tue, Apr 06, 2010 at 11:26:28AM +0200, Jan Stary said that
 On Apr 06 11:15:26, frantisek holop wrote:
  hi there,
  
  what happens if i specify a cronjob like this?
  
  23 59 31 * * $HOME/bin/whatever
 
 Cron will just do what it's told: run whatever at 31.*. 23:59

so i could basically do 12 lines with the correct
last day of the month rules :]

  i am looking for an alternative @monthly, not
  
  0 0 1 * *
  
  but the last minutes of the last day of the month.
 
 Why?

because for me the month ends at 23:59:59 on the last day
of month n, and not at 00:00:00 on the first day of month n+1...

-f
-- 
if you have to travel on a titanic, why not go first class?



Re: crontab last day of the month

2010-04-06 Thread Bret S. Lambert
On Tue, Apr 06, 2010 at 12:09:01PM +0200, frantisek holop wrote:
 hmm, on Tue, Apr 06, 2010 at 11:26:28AM +0200, Jan Stary said that
  On Apr 06 11:15:26, frantisek holop wrote:
   hi there,
   
   what happens if i specify a cronjob like this?
   
   23 59 31 * * $HOME/bin/whatever
  
  Cron will just do what it's told: run whatever at 31.*. 23:59
 
 so i could basically do 12 lines with the correct
 last day of the month rules :]
 
   i am looking for an alternative @monthly, not
   
   0 0 1 * *
   
   but the last minutes of the last day of the month.
  
  Why?
 
 because for me the month ends at 23:59:59 on the last day
 of month n, and not at 00:00:00 on the first day of month n+1...

Not to be a dick, but what does one second buy you, really?

 
 -f
 -- 
 if you have to travel on a titanic, why not go first class?



Re: crontab last day of the month

2010-04-06 Thread Alexander Hall
frantisek holop wrote:
 hmm, on Tue, Apr 06, 2010 at 11:26:28AM +0200, Jan Stary said that
 On Apr 06 11:15:26, frantisek holop wrote:
 hi there,

 what happens if i specify a cronjob like this?

 23 59 31 * * $HOME/bin/whatever
 Cron will just do what it's told: run whatever at 31.*. 23:59
 
 so i could basically do 12 lines with the correct
 last day of the month rules :]
 

No, you're forgetting about leap years... :-)



Re: crontab last day of the month

2010-04-06 Thread Paul de Weerd
On Tue, Apr 06, 2010 at 12:09:01PM +0200, frantisek holop wrote:
|  Cron will just do what it's told: run whatever at 31.*. 23:59
| 
| so i could basically do 12 lines with the correct
| last day of the month rules :]

12 ? Months come in 12 different lengths where you live ? You have
bigger issues then - base cron only knows the Gregorian calendar where
there's 3 possible lengths for months in a year.

| because for me the month ends at 23:59:59 on the last day
| of month n, and not at 00:00:00 on the first day of month n+1...

What you want can not be done with cron alone. The solution is to have
a cronjob at days 28-31 (as was already suggested) to verify if it is
indeed the last day of the month and then run the intended script. Or
have 2 entries in your crontab :

59 23 31 1,3,5,7,8,10,12 * /path/to/your/cronscript
59 23 30 4,6,9,11 * /path/to/your/cronscript

and then a third one for february that you change every leap year. Or
you could suck it up and always do the february one on the 28th :

59 23 28 2 * /path/to/your/cronscript

If you want to get really fancy, add a @yearly cronjob that changes
your crontab to suit the 2nd month of the year. It's just an s/28/29/
or s/29/28/, depending on `cal feb | grep -q 29` (make sure your box
is up and cron is running when @yearly time comes around .. maybe you
want to make this @reboot to check things).

Cron does not support 'last day of month' / 'last weekday of month'
setups.

Cheers,

Paul 'WEiRD' de Weer

-- 
[++-]+++.+++[---].+++[+
+++-].++[-]+.--.[-]
 http://www.weirdnet.nl/ 



Re: crontab last day of the month

2010-04-06 Thread Otto Moerbeek
On Tue, Apr 06, 2010 at 12:09:01PM +0200, frantisek holop wrote:

 hmm, on Tue, Apr 06, 2010 at 11:26:28AM +0200, Jan Stary said that
  On Apr 06 11:15:26, frantisek holop wrote:
   hi there,
   
   what happens if i specify a cronjob like this?
   
   23 59 31 * * $HOME/bin/whatever
  
  Cron will just do what it's told: run whatever at 31.*. 23:59
 
 so i could basically do 12 lines with the correct
 last day of the month rules :]
 
   i am looking for an alternative @monthly, not
   
   0 0 1 * *
   
   but the last minutes of the last day of the month.
  
  Why?
 
 because for me the month ends at 23:59:59 on the last day
 of month n, and not at 00:00:00 on the first day of month n+1...
 
 -f
 -- 
 if you have to travel on a titanic, why not go first class?

You have too much believe in the granularity of cron.

-Otto



Re: crontab last day of the month

2010-04-06 Thread Artur Grabowski
On Tue, Apr 6, 2010 at 12:09 PM, frantisek holop min...@obiit.org wrote:
  i am looking for an alternative @monthly, not
 
  0 0 1 * *
 
  but the last minutes of the last day of the month.

 Why?

 because for me the month ends at 23:59:59 on the last day
 of month n, and not at 00:00:00 on the first day of month n+1...

cron can't guarantee that it will run the job at exactly that second.
It will run the job some time after the time specified, but not
before. So if you're looking for second precision for your jobs, cron
is not the right tool (there aren't any tools that can guarantee this
unless you're running a real time operating system).

//art



Re: crontab last day of the month

2010-04-06 Thread Jan Stary
On Apr 06 12:09:01, frantisek holop wrote:
 hmm, on Tue, Apr 06, 2010 at 11:26:28AM +0200, Jan Stary said that
  On Apr 06 11:15:26, frantisek holop wrote:
   hi there,
   
   what happens if i specify a cronjob like this?
   
   23 59 31 * * $HOME/bin/whatever
  
  Cron will just do what it's told: run whatever at 31.*. 23:59
 
 so i could basically do 12 lines with the correct
 last day of the month rules :]

And correct the 59:23, of course.
Or you could deal with the last-day-of-the-month arithmetic
inside ~/bin/whatever (should I really run?), and do
59 23 28-31 * * whatever


   i am looking for an alternative @monthly, not
   
   0 0 1 * *
   
   but the last minutes of the last day of the month.
  
  Why?
 
 because for me the month ends at 23:59:59 on the last day
 of month n, and not at 00:00:00 on the first day of month n+1...

Why?



Re: crontab last day of the month

2010-04-06 Thread frantisek holop
hmm, on Tue, Apr 06, 2010 at 12:20:03PM +0200, Bret S. Lambert said that
 Not to be a dick, but what does one second buy you, really?

it's not really about that second.
actually, i dont mind losing some 5 minutes even
from the current month.


my goal is to have log files that end at a certain period.

e.g. an archived log file of march that doesn't contain
april entries (from the future), although it's all right
if it contains a couple of stray entries from february
(the casualties of log rotation).

now i have logfiles named after the current month containing
all the entries from the previous month.

getting the name of the previous month from the current month
is another can of worms i dont want to open.

-f
-- 
careful planning will never replace dumb luck.



Re: crontab last day of the month

2010-04-06 Thread Robert Bronsdon
On Tue, 06 Apr 2010 13:24:27 +0100, frantisek holop min...@obiit.org  
wrote:



e.g. an archived log file of march that doesn't contain
april entries (from the future), although it's all right
if it contains a couple of stray entries from february
(the casualties of log rotation).


Then depending on what is creating the log files you are almost certanly  
better off getting the software to create them with sane names and moving  
them with cron later.



--
Using Opera M2: http://www.opera.com/mail/



Re: crontab last day of the month

2010-04-06 Thread Paul de Weerd
On Tue, Apr 06, 2010 at 02:24:27PM +0200, frantisek holop wrote:
| my goal is to have log files that end at a certain period.
| 
| e.g. an archived log file of march that doesn't contain
| april entries (from the future), although it's all right
| if it contains a couple of stray entries from february
| (the casualties of log rotation).

Aren't those first few also the casualties of log rotation ? I don't
see what you gain here, honestly. Either you have the first log
entries of a month in the rotated logs of the previous month. Or you
have the last log entries of a month in the rotated logs of the next
month. Tomato vs tomato.

| now i have logfiles named after the current month containing
| all the entries from the previous month.

So rename your logs properly.

| getting the name of the previous month from the current month
| is another can of worms i dont want to open.

What, really ?!

The approach with a simple lookup table:

PREVMONTH[1]=dec; PREVMONTH[2]=jan; PREVMONTH[3]=feb
PREVMONTH[4]=mar; PREVMONTH[5]=apr; PREVMONTH[6]=may
PREVMONTH[7]=jun; PREVMONTH[8]=jul; PREVMONTH[9]=aug
PREVMONTH[10]=sep; PREVMONTH[11]=oct; PREVMONTH[12]=nov

echo ${PREVMONTH[`date +%m`]}

Or, if you are certain you're within the first day of the new month
(as for 0 0 1 * * crons), `date -r $(($(date +%s) - 24*3600)) +%m`
(adjust for 'first week of new month' etc .. or get fancy and find the
number of days you've spent in the current month and go back that many
plus one).

Come on man .. 'can of worms' ?! It's not even a real challenge.

Paul 'WEiRD' de Weerd

-- 
[++-]+++.+++[---].+++[+
+++-].++[-]+.--.[-]
 http://www.weirdnet.nl/ 



Re: crontab last day of the month

2010-04-06 Thread Bret S. Lambert
On Tue, Apr 06, 2010 at 02:24:27PM +0200, frantisek holop wrote:
 hmm, on Tue, Apr 06, 2010 at 12:20:03PM +0200, Bret S. Lambert said that
  Not to be a dick, but what does one second buy you, really?
 
 it's not really about that second.
 actually, i dont mind losing some 5 minutes even
 from the current month.
 
 
 my goal is to have log files that end at a certain period.
 
 e.g. an archived log file of march that doesn't contain
 april entries (from the future), although it's all right
 if it contains a couple of stray entries from february
 (the casualties of log rotation).

I'm still not seeing what you're really getting, here; you're
just pushing that spillover from one end to another, which
are just as easily rationalized as the casualties of log
rotation.

But, it's your bikeshed, you can build it how you want.

 
 now i have logfiles named after the current month containing
 all the entries from the previous month.
 
 getting the name of the previous month from the current month
 is another can of worms i dont want to open.

date +mm?

 
 -f
 -- 
 careful planning will never replace dumb luck.



Re: crontab last day of the month

2010-04-06 Thread Martin Schröder
2010/4/6 frantisek holop min...@obiit.org:
 my goal is to have log files that end at a certain period.

syslog-ng is in ports.

Best
Martin



Re: crontab last day of the month

2010-04-06 Thread Antti Harri

Hi all,

I want to hijack this thread a little because it sounds like
OP has similar problem to mine.

I'm rotating logs with newsyslog and for example for web logs
I keep 12 logs available (actually it looks like count=12 keeps 13
files available but anyway that's not my problem..), rotated monthly so I 
have logs from present moment to one year back like this:


# ls access_log* | sort -n -k 2 -t .
access_log
access_log.0.gz
access_log.1.gz
access_log.2.gz
access_log.3.gz
access_log.4.gz
access_log.5.gz
access_log.6.gz
access_log.7.gz
access_log.8.gz
access_log.9.gz
access_log.10.gz
access_log.11.gz
access_log.12.gz

The newsyslog.conf looks like this:

/path/access_log   vhost3:wheel640 12 * $M1D0 Z

My problem is that when I want to produce something based on those
files, it is very hard to know the time range of the actual contents.
Any pointers? I would rather keep using the newsyslog from base
than install anything from ports, as someone suggested syslog-ng already.

I'm using www/visitors to produce something meaningful from those
logs, and I would like to generate stats just once for a certain
time period so I can later see stats further than one year that newsyslog 
keeps.


--
Antti Harri



Re: crontab last day of the month

2010-04-06 Thread L. V. Lammert
On Tue, 6 Apr 2010, Antti Harri wrote:

 Hi all,

 I want to hijack this thread a little because it sounds like
 OP has similar problem to mine.

 I'm rotating logs with newsyslog and for example for web logs
 I keep 12 logs available (actually it looks like count=12 keeps 13
 files available but anyway that's not my problem..), rotated monthly so I
 have logs from present moment to one year back like this:

Instead of rotating logs, consider using cronolog - no restart of Apache
requierd.

It maintains a daemon for each virtual host, and automatically rolls files
over at the end of any defined period.

Lee



Re: crontab last day of the month

2010-04-06 Thread frantisek holop
hmm, on Tue, Apr 06, 2010 at 02:40:03PM +0200, Paul de Weerd said that
 What, really ?!
 
 The approach with a simple lookup table:
 
 PREVMONTH[1]=dec; PREVMONTH[2]=jan; PREVMONTH[3]=feb
 PREVMONTH[4]=mar; PREVMONTH[5]=apr; PREVMONTH[6]=may
 PREVMONTH[7]=jun; PREVMONTH[8]=jul; PREVMONTH[9]=aug
 PREVMONTH[10]=sep; PREVMONTH[11]=oct; PREVMONTH[12]=nov
 
 echo ${PREVMONTH[`date +%m`]}
 
 Or, if you are certain you're within the first day of the new month
 (as for 0 0 1 * * crons), `date -r $(($(date +%s) - 24*3600)) +%m`
 (adjust for 'first week of new month' etc .. or get fancy and find the
 number of days you've spent in the current month and go back that many
 plus one).
 
 Come on man .. 'can of worms' ?! It's not even a real challenge.

you left off the year...


i'd rather not bloat my 3 line scripts with lookup tables
and recursive date(1) calls, i prefer them easy and short.

here's the full picture of the current situation:

on 1. april i get this file through monthly.local:

logfile-2010-04.log.gz

containing all _march_ entries (plus some april if it creeped in).

i have 2 choices:
1. do the rename dance backwards one month
2. let the job actually run sometimes before midnight, last day of the month.

newsyslog can do it.  cron can't.

regarding the forward casualties of log rotation, yes it's the same.
while chronologically i still prefer the other way, i.e. have older
entries in the newer log file (and not newer entries in older logfiles),
but the main issue is, that i end up with the wrong file name.

-f
-- 
what follows 2 days of rain? -- monday



Re: crontab last day of the month

2010-04-06 Thread Sidarta Aguiar de Oliveira
Hello,

You can put in your crontab something like that:

0 0 * * * if [ $(date '+%b') != $(date -r $(expr $(date '+%s') + 86400 )
'+%b') ]; then echo 'do something'; else echo 'do nothing'; fi

This will execute everyday. This calculates the month of the next day and
compare with the actual month. If it's different, will execute your script ;]

See ya,


S idarta Oliveira



- Mensagem original -
De: frantisek holop min...@obiit.org
Para: misc@openbsd.org
Enviadas: TerC'a-feira, 6 de Abril de 2010 11:18:22 (GMT-0300) Auto-Detected
Assunto: Re: crontab last day of the month

hmm, on Tue, Apr 06, 2010 at 02:40:03PM +0200, Paul de Weerd said that
 What, really ?!

 The approach with a simple lookup table:

 PREVMONTH[1]=dec; PREVMONTH[2]=jan; PREVMONTH[3]=feb
 PREVMONTH[4]=mar; PREVMONTH[5]=apr; PREVMONTH[6]=may
 PREVMONTH[7]=jun; PREVMONTH[8]=jul; PREVMONTH[9]=aug
 PREVMONTH[10]=sep; PREVMONTH[11]=oct; PREVMONTH[12]=nov

 echo ${PREVMONTH[`date +%m`]}

 Or, if you are certain you're within the first day of the new month
 (as for 0 0 1 * * crons), `date -r $(($(date +%s) - 24*3600)) +%m`
 (adjust for 'first week of new month' etc .. or get fancy and find the
 number of days you've spent in the current month and go back that many
 plus one).

 Come on man .. 'can of worms' ?! It's not even a real challenge.

you left off the year...


i'd rather not bloat my 3 line scripts with lookup tables
and recursive date(1) calls, i prefer them easy and short.

here's the full picture of the current situation:

on 1. april i get this file through monthly.local:

logfile-2010-04.log.gz

containing all _march_ entries (plus some april if it creeped in).

i have 2 choices:
1. do the rename dance backwards one month
2. let the job actually run sometimes before midnight, last day of the month.

newsyslog can do it. cron can't.

regarding the forward casualties of log rotation, yes it's the same.
while chronologically i still prefer the other way, i.e. have older
entries in the newer log file (and not newer entries in older logfiles),
but the main issue is, that i end up with the wrong file name.

-f
--
what follows 2 days of rain? -- monday



Re: crontab last day of the month

2010-04-06 Thread Jan Stary
On Apr 06 16:39:51, Antti Harri wrote:
 Hi all,

 I want to hijack this thread a little because it sounds like
 OP has similar problem to mine.

 I'm rotating logs with newsyslog and for example for web logs
 I keep 12 logs available (actually it looks like count=12 keeps 13
 files available but anyway that's not my problem..), rotated monthly so I 
 have logs from present moment to one year back like this:

 # ls access_log* | sort -n -k 2 -t .
 access_log
 access_log.0.gz
 access_log.1.gz
 access_log.2.gz
 access_log.3.gz
 access_log.4.gz
 access_log.5.gz
 access_log.6.gz
 access_log.7.gz
 access_log.8.gz
 access_log.9.gz
 access_log.10.gz
 access_log.11.gz
 access_log.12.gz

 The newsyslog.conf looks like this:

 /path/access_log   vhost3:wheel640 12 * $M1D0 Z

 My problem is that when I want to produce something based on those
 files, it is very hard to know the time range of the actual contents.

No it's not. Unless you have some heavy traffic just around the midnight
that is the border between months, the above newsyslog line makes
the just-rotated access_log.0.gz contains the log of the month-that-just-ended.

 I'm using www/visitors to produce something meaningful from those
 logs, and I would like to generate stats just once for a certain
 time period so I can later see stats further than one year that newsyslog 
 keeps.

webalizer



Re: crontab last day of the month

2010-04-06 Thread Benny Lšöfgren

frantisek holop wrote:

Come on man .. 'can of worms' ?! It's not even a real challenge.

you left off the year...

i'd rather not bloat my 3 line scripts with lookup tables
and recursive date(1) calls, i prefer them easy and short.

here's the full picture of the current situation:

on 1. april i get this file through monthly.local:

logfile-2010-04.log.gz

containing all _march_ entries (plus some april if it creeped in).

i have 2 choices:
1. do the rename dance backwards one month
2. let the job actually run sometimes before midnight, last day of the month.


Not that it helps unbloating your scripts, but when I have that 
problem I usually cheat by running my script twice - once a few minutes 
before midnight (as cron doesn't guarantee execution at a specific time, 
only at or after, we need a margin) and once after.


The first time the script is called (with a switch set to indicate it's 
the pre-run) it creates a file containing the current date in a suitable 
format. The next time, it uses that file for the rotated log file name.


Here's an example I use to rotate my apache log files. First the crontab 
entries:


888888 (cut)
50  23  *   *   *   /var/www/logs/archivelogs -d
1   0   *   *   *   /var/www/logs/archivelogs
888888 (cut)

Then the script itself:

888888 (cut)
#!/bin/sh

cd /var/www/logs

if [ ! -f ./archivedate ]
then
if [ $1 = -d ]
then
date '+%y%m%d' ./archivedate
exit 0
fi

echo $0: error: no /var/www/logs/archivedate
exit 1
fi

ARCHDATE=`cat ./archivedate`

# Make archive directory if it doesn't exist already
mkdir -p ./Archive

if [ -f ./Archive/access-$ARCHDATE.log -o -f \
./Archive/access-$ARCHDATE.log.gz ]
then
echo $0: error: access.log archive file for $ARCHDATE already 
there

exit 2
fi

if [ -f ./Archive/error-$ARCHDATE.log -o -f \
./Archive/error-$ARCHDATE.log.gz ]
then
echo $0: error: error.log archive file for $ARCHDATE already 
there

exit 3
fi

mv ./access.log ./Archive/access-$ARCHDATE.log
mv ./error.log ./Archive/error-$ARCHDATE.log

# Hopefully there is a better way to do this...
#/usr/sbin/apachectl stop
#sleep 5
#/usr/bin/nice -8 /usr/sbin/httpd -u
apachectl graceful

sleep 180

gzip -q ./Archive/access-$ARCHDATE.log
gzip -q ./Archive/error-$ARCHDATE.log

rm -f ./archivedate

exit 0
888888 (cut)


Best regards,

/Benny

--
internetlabbet.se / work:   +46 8 551 124 80  / Words must
Benny LC6fgren/  mobile: +46 70 718 11 90 /   be weighed,
/   fax:+46 8 551 124 89/not counted.
   /email:  benny aa internetlabbet.se



Re: crontab last day of the month

2010-04-06 Thread Paul de Weerd
On Tue, Apr 06, 2010 at 04:18:22PM +0200, frantisek holop wrote:
|  Come on man .. 'can of worms' ?! It's not even a real challenge.
| 
| you left off the year...

So you think it's my problem ? I showed two possible options, both
very workable (and easily extensible to add something simple as a
year). Yet...

| i'd rather not bloat my 3 line scripts with lookup tables
| and recursive date(1) calls, i prefer them easy and short.

...you choose to complain about the options I provided. 

| here's the full picture of the current situation:
| 
| on 1. april i get this file through monthly.local:
| 
| logfile-2010-04.log.gz
| 
| containing all _march_ entries (plus some april if it creeped in).
| 
| i have 2 choices:
| 1. do the rename dance backwards one month
| 2. let the job actually run sometimes before midnight, last day of the month.

or 3. use recursive date(1) calls (how, exactly, are those
recursive ? you've got a weird definition of recursion)

gzip  ${LOG}  logfile-$(date -r $(($(date +%s) - 86400)) +%Y-%m).gz

There, I've put in the year for you, beacuse you couldn't think of
that yourself somehow. I don't really care much how you do it, but
that is not recursion nor very bloated, imo. 

You come to this list to ask for something. It's not possible and you
are given alternative solutions. All you can do is complain about
them. Why ?

| regarding the forward casualties of log rotation, yes it's the same.
| while chronologically i still prefer the other way, i.e. have older
| entries in the newer log file (and not newer entries in older logfiles),
| but the main issue is, that i end up with the wrong file name.

It isn't an issue - the solution has been presented to you. A few
lines up, I've spelled it out for you (one possible solution, that
is). You just choose to publicly dismiss it because you don't like it.
Why ?

Paul 'WEiRD' de Weerd

PS: Don't bother replying to the 'whys' in this e-mail, they're meant
for self-reflection.

-- 
[++-]+++.+++[---].+++[+
+++-].++[-]+.--.[-]
 http://www.weirdnet.nl/ 



Re: crontab last day of the month

2010-04-06 Thread Chris Bennett

Paul de Weerd wrote:

On Tue, Apr 06, 2010 at 04:18:22PM +0200, frantisek holop wrote:
|  Come on man .. 'can of worms' ?! It's not even a real challenge.
| 
| you left off the year...


So you think it's my problem ? I showed two possible options, both
very workable (and easily extensible to add something simple as a
year). Yet...

| i'd rather not bloat my 3 line scripts with lookup tables
| and recursive date(1) calls, i prefer them easy and short.

...you choose to complain about the options I provided. 


| here's the full picture of the current situation:
| 
| on 1. april i get this file through monthly.local:
| 
| logfile-2010-04.log.gz
| 
| containing all _march_ entries (plus some april if it creeped in).
| 
| i have 2 choices:

| 1. do the rename dance backwards one month
| 2. let the job actually run sometimes before midnight, last day of the month.

or 3. use recursive date(1) calls (how, exactly, are those
recursive ? you've got a weird definition of recursion)

gzip  ${LOG}  logfile-$(date -r $(($(date +%s) - 86400)) +%Y-%m).gz

There, I've put in the year for you, beacuse you couldn't think of
that yourself somehow. I don't really care much how you do it, but
that is not recursion nor very bloated, imo. 


You come to this list to ask for something. It's not possible and you
are given alternative solutions. All you can do is complain about
them. Why ?

| regarding the forward casualties of log rotation, yes it's the same.
| while chronologically i still prefer the other way, i.e. have older
| entries in the newer log file (and not newer entries in older logfiles),
| but the main issue is, that i end up with the wrong file name.

It isn't an issue - the solution has been presented to you. A few
lines up, I've spelled it out for you (one possible solution, that
is). You just choose to publicly dismiss it because you don't like it.
Why ?

Paul 'WEiRD' de Weerd

PS: Don't bother replying to the 'whys' in this e-mail, they're meant
for self-reflection.

  

I think I can read his mind here.
He is clearly stating that the existing software is not adequate for his 
needs. Let's face it, software written in perl, script, and C often 
bloat things up unnecessarily. Sure, C compilers are getting better, but 
small fast programs are often best produced by going to straight 
assembly language.


Some of the correct tools, if i386 is being used - nasm or fasm or yasm, 
ald plus a little study and Boom! small neat fast tools to do all of 
this work. Screw that piece of junk cron! :)


--
A human being should be able to change a diaper, plan an invasion,
butcher a hog, conn a ship, design a building, write a sonnet, balance
accounts, build a wall, set a bone, comfort the dying, take orders,
give orders, cooperate, act alone, solve equations, analyze a new
problem, pitch manure, program a computer, cook a tasty meal, fight
efficiently, die gallantly. Specialization is for insects.
  -- Robert Heinlein



Re: crontab last day of the month

2010-04-06 Thread Antti Harri

On Tue, 6 Apr 2010, Jan Stary wrote:


No it's not. Unless you have some heavy traffic just around the midnight
that is the border between months, the above newsyslog line makes
the just-rotated access_log.0.gz contains the log of the month-that-just-ended.


OK. How do I reliably get month and year out of the file?
Month I can get from the newsyslog output it puts in the logs,
getting year without doing some sort of nasty kludge is hard.

m=$(zfgrep logfile turned over $file | tail -n 1 | cut -b 1-3)


I'm using www/visitors to produce something meaningful from those
logs, and I would like to generate stats just once for a certain
time period so I can later see stats further than one year that newsyslog
keeps.


webalizer


I told I'm using visitors.

--
Antti Harri



Re: crontab last day of the month

2010-04-06 Thread frantisek holop
hmm, on Tue, Apr 06, 2010 at 09:51:29PM +0200, Paul de Weerd said that
 So you think it's my problem ? I showed two possible options, both
 very workable (and easily extensible to add something simple as a
 year). Yet...
 
 ...you choose to complain about the options I provided. 

i am humbled by the fact that you took some time and
tried to help me.  i am not complaining, i merely dislike
your solutions.

i tapped into the mailing list's collective wisdom to see
if someone has a solution my feeble mind would like.


 or 3. use recursive date(1) calls (how, exactly, are those
 recursive ?

geometrically.


 It isn't an issue - the solution has been presented to you. A few
 lines up, I've spelled it out for you (one possible solution, that
 is). You just choose to publicly dismiss it because you don't like it.
 Why ?

i dont like your solutions because they are hacks.
it is nothing personal.

i am off to self-reflect, sensei.

-f
-- 
one seventh of our lives is spent on mondays.



Re: crontab last day of the month

2010-04-06 Thread frantisek holop
hmm, on Tue, Apr 06, 2010 at 03:01:17PM +0200, Bret S. Lambert said that
 I'm still not seeing what you're really getting, here; you're
 just pushing that spillover from one end to another, which
 are just as easily rationalized as the casualties of log
 rotation.

i get the correct name/number of the month...

 date +mm?

$ date +mm
mm
$ date +mm?
mm?

-f
-- 
incipit vita nova.



Re: crontab last day of the month

2010-04-06 Thread Stuart Henderson
On 2010-04-06, frantisek holop min...@obiit.org wrote:
 hmm, on Tue, Apr 06, 2010 at 03:01:17PM +0200, Bret S. Lambert said that
 I'm still not seeing what you're really getting, here; you're
 just pushing that spillover from one end to another, which
 are just as easily rationalized as the casualties of log
 rotation.

 i get the correct name/number of the month...

mv logfile logfile.$(date -r $(($(date +%s)-600)) +%Y-%m-%d)