[PATCH] Apache 1.3 built in log rotation...
This patch adds the directives LogRotateDaily and LogRotateInterval to the mod_log_config modules. These directives allow all of the custom logs to be automatically rotated on either a daily basis or at a specific interval. This patch is based on a previous patch that was submitted by Bertrand Demiddelaer. One of the problems that we have had on NetWare is the lack of a way to automatically rotate the log files. NetWare is unable to use the RotateLog utility due to the fact that the OS does not support pipes. This patch is being submitted as a general patch rather than a NetWare specific patch so that other platforms can take advantage of it if they choose to. If there are objections to this patch I could submit it as a NetWare only fix. If there are no objections, I would like to go ahead and check it in. Brad Brad Nicholes Senior Software Engineer Novell, Inc., a leading provider of Net business solutions http://www.novell.com mod_log_config.c.patch Description: Binary data
Re: [PATCH] Apache 1.3 built in log rotation...
On Wed, 27 Feb 2002, Brad Nicholes wrote: This patch adds the directives LogRotateDaily and LogRotateInterval to the mod_log_config modules. These directives allow all of the custom logs to be automatically rotated on either a daily basis or at a specific interval. This patch is based on a previous patch that was submitted by Bertrand Demiddelaer. One of the problems that we have had on NetWare is the lack of a way to automatically rotate the log files. NetWare is unable to use the RotateLog utility due to the fact that the OS does not support pipes. This patch is being submitted as a general patch rather than a NetWare specific patch so that other platforms can take advantage of it if they choose to. If there are objections to this patch I could submit it as a NetWare only fix. If there are no objections, I would like to go ahead and check it in. This patch is a major security problem on Unix, since you should not have your log files writable by the user apache runs as. They should only be writable by the user that starts Apache (normally root). This means child processes can not reopen logs. BTW, please try to include patches in the body of the message instead of as binary attachments.
Re: [PATCH] Apache 1.3 built in log rotation...
Since I am not a Unix developer, can this security problem be overcome
somehow or does this mean that I should #ifdef the code as NETWARE
only?
Brad
Brad Nicholes
Senior Software Engineer
Novell, Inc., a leading provider of Net business solutions
http://www.novell.com
[EMAIL PROTECTED] Wednesday, February 27, 2002 1:34:46 PM
On Wed, 27 Feb 2002, Brad Nicholes wrote:
This patch adds the directives LogRotateDaily and
LogRotateInterval
to the mod_log_config modules. These directives allow all of the
custom
logs to be automatically rotated on either a daily basis or at a
specific interval. This patch is based on a previous patch that was
submitted by Bertrand Demiddelaer.
One of the problems that we have had on NetWare is the lack of
a
way to automatically rotate the log files. NetWare is unable to use
the
RotateLog utility due to the fact that the OS does not support pipes.
This patch is being submitted as a general patch rather than a
NetWare
specific patch so that other platforms can take advantage of it if
they
choose to. If there are objections to this patch I could submit it
as a
NetWare only fix. If there are no objections, I would like to go
ahead
and check it in.
This patch is a major security problem on Unix, since you should not
have
your log files writable by the user apache runs as. They should only
be writable by the user that starts Apache (normally root). This
means
child processes can not reopen logs.
BTW, please try to include patches in the body of the message instead
of
as binary attachments.
--- mod_log_config.c.orgWed Feb 27 12:59:20 2002
+++ mod_log_config.cWed Feb 27 12:52:57 2002
@@ -231,6 +231,8 @@
array_header *config_logs;
array_header *server_config_logs;
table *formats;
+int rotatedaily;
+int rotateinterval;
} multi_log_state;
/*
@@ -252,6 +254,7 @@
int outcnt;
char outbuf[LOG_BUFSIZE];
#endif
+time_t time_jump;
} config_log_state;
/*
@@ -803,6 +806,39 @@
int len = 0;
array_header *format;
char *envar;
+int log_fd;
+
+multi_log_state *mls =
ap_get_module_config(r-server-module_config,config_log_module);
+
+if ((mls-rotatedaily || mls-rotateinterval)
+(r-request_time=cls-time_jump)
+(*cls-fname!='|') (strcmp(cls-fname,/dev/null) != 0))
{
+char * fname;
+struct tm *time_tmp;
+
+if (mls-rotatedaily) {
+time_tmp=localtime((r-request_time));
+
cls-time_jump=r-request_time+((60-time_tmp-tm_sec)+60*(59-time_tmp-tm_min)+3600*(23-time_tmp-tm_hour));
+}
+else
+cls-time_jump = r-request_time +
(60*mls-rotateinterval);
+
+fname = ap_pstrcat(r-pool,
+ap_server_root_relative(r-pool, cls-fname),
+-,
+ap_ht_time(r-pool,r-request_time,%Y%m%d%H%M,0),
+NULL
+);
+
+if ((log_fd = open(fname, xfer_flags, xfer_mode)) 0) {
+ap_log_error(APLOG_MARK, APLOG_ERR, r-server,
+could not open transfer log file %s., fname);
+}
+else {
+dup2 (log_fd, cls-log_fd);
+close (log_fd);
+}
+}
if (cls-fname == NULL) {
return DECLINED;
@@ -926,6 +962,8 @@
mls-default_format = NULL;
mls-server_config_logs = NULL;
mls-formats = ap_make_table(p, 4);
+mls-rotatedaily = 0;
+mls-rotateinterval = 0;
ap_table_setn(mls-formats, CLF, DEFAULT_LOG_FORMAT);
return mls;
@@ -942,6 +980,13 @@
multi_log_state *base = (multi_log_state *) basev;
multi_log_state *add = (multi_log_state *) addv;
+if (add-rotatedaily==0) {
+ add-rotatedaily=base-rotatedaily;
+}
+if (add-rotateinterval==0) {
+ add-rotateinterval=base-rotateinterval;
+}
+
add-server_config_logs = base-config_logs;
if (!add-default_format) {
add-default_format_string = base-default_format_string;
@@ -1025,6 +1070,34 @@
return add_custom_log(cmd, dummy, fn, %{Cookie}n \%r\ %t,
NULL);
}
+static const char *set_rotate_log_daily(cmd_parms *cmd, void *dummy,
int arg)
+{
+multi_log_state *mls =
ap_get_module_config(cmd-server-module_config,
+ config_log_module);
+
+mls-rotatedaily = arg;
+if (mls-rotatedaily)
+mls-rotateinterval = 0;
+return NULL;
+}
+
+static const char *set_rotate_log_interval(cmd_parms *cmd, void
*dummy, char *arg)
+{
+multi_log_state *mls =
ap_get_module_config(cmd-server-module_config,
+ config_log_module);
+int interval = 0;
+
+if (arg)
+interval = atoi(arg);
+
+if (interval 0)
+return NULL;
+
+mls-rotatedaily = 0;
+mls-rotateinterval = interval;
+return NULL;
+}
+
static const command_rec config_log_cmds[] =
{
{CustomLog, add_custom_log, NULL, RSRC_CONF, TAKE23,
@@ -1036,6 +1109,10 @@
a log format string
RE: [PATCH] Apache 1.3 built in log rotation...
Do we really want Apache rotating logs? Apache is a web server it serves web pages really well. If you want log rotation, use either a piped log or a cron job that restarts the server. Ryan Since I am not a Unix developer, can this security problem be overcome somehow or does this mean that I should #ifdef the code as NETWARE only? Brad Brad Nicholes Senior Software Engineer Novell, Inc., a leading provider of Net business solutions http://www.novell.com [EMAIL PROTECTED] Wednesday, February 27, 2002 1:34:46 PM On Wed, 27 Feb 2002, Brad Nicholes wrote: This patch adds the directives LogRotateDaily and LogRotateInterval to the mod_log_config modules. These directives allow all of the custom logs to be automatically rotated on either a daily basis or at a specific interval. This patch is based on a previous patch that was submitted by Bertrand Demiddelaer. One of the problems that we have had on NetWare is the lack of a way to automatically rotate the log files. NetWare is unable to use the RotateLog utility due to the fact that the OS does not support pipes. This patch is being submitted as a general patch rather than a NetWare specific patch so that other platforms can take advantage of it if they choose to. If there are objections to this patch I could submit it as a NetWare only fix. If there are no objections, I would like to go ahead and check it in. This patch is a major security problem on Unix, since you should not have your log files writable by the user apache runs as. They should only be writable by the user that starts Apache (normally root). This means child processes can not reopen logs. BTW, please try to include patches in the body of the message instead of as binary attachments.
RE: [PATCH] Apache 1.3 built in log rotation...
Like I mentioned before, on NetWare we can't use a piped log because the NetWare OS doesn't support pipes. A cron job is also a problem because we don't have that either. Since Apache created the log file, writes to the log file, formats the output and closes the log file, is it that big of a leap to have it rotate the log file as well? Brad Brad Nicholes Senior Software Engineer Novell, Inc., a leading provider of Net business solutions http://www.novell.com [EMAIL PROTECTED] Wednesday, February 27, 2002 1:45:27 PM Do we really want Apache rotating logs? Apache is a web server it serves web pages really well. If you want log rotation, use either a piped log or a cron job that restarts the server. Ryan Since I am not a Unix developer, can this security problem be overcome somehow or does this mean that I should #ifdef the code as NETWARE only? Brad Brad Nicholes Senior Software Engineer Novell, Inc., a leading provider of Net business solutions http://www.novell.com [EMAIL PROTECTED] Wednesday, February 27, 2002 1:34:46 PM On Wed, 27 Feb 2002, Brad Nicholes wrote: This patch adds the directives LogRotateDaily and LogRotateInterval to the mod_log_config modules. These directives allow all of the custom logs to be automatically rotated on either a daily basis or at a specific interval. This patch is based on a previous patch that was submitted by Bertrand Demiddelaer. One of the problems that we have had on NetWare is the lack of a way to automatically rotate the log files. NetWare is unable to use the RotateLog utility due to the fact that the OS does not support pipes. This patch is being submitted as a general patch rather than a NetWare specific patch so that other platforms can take advantage of it if they choose to. If there are objections to this patch I could submit it as a NetWare only fix. If there are no objections, I would like to go ahead and check it in. This patch is a major security problem on Unix, since you should not have your log files writable by the user apache runs as. They should only be writable by the user that starts Apache (normally root). This means child processes can not reopen logs. BTW, please try to include patches in the body of the message instead of as binary attachments.
RE: [PATCH] Apache 1.3 built in log rotation...
Like I mentioned before, on NetWare we can't use a piped log because the NetWare OS doesn't support pipes. A cron job is also a problem because we don't have that either. Since Apache created the log file, writes to the log file, formats the output and closes the log file, is it that big of a leap to have it rotate the log file as well? It always has been in the past. The thing is that the web server should be serving pages, not mucking with log files. The other thing is that rotation is going to need to be different on different platforms. How does Windows handle it if you try to rotate a log file in the middle of writing to the log. There is definitely a race condition there, where you are writing a message in one thread, and another thread moves the file out of the way? I really think that if NetWare can't use the current mod_log_config then I think you need to write a NetWare specific logging module. Ryan
RE: [PATCH] Apache 1.3 built in log rotation...
Is is enough for us to simply #ifdef these changes in mod_log_config.c or would you rather see a separate logging module? Except for the addition of these two directives, everything else works for us. Brad Brad Nicholes Senior Software Engineer Novell, Inc., a leading provider of Net business solutions http://www.novell.com Ryan Bloom [EMAIL PROTECTED] Wednesday, February 27, 2002 3:28:25 PM Like I mentioned before, on NetWare we can't use a piped log because the NetWare OS doesn't support pipes. A cron job is also a problem because we don't have that either. Since Apache created the log file, writes to the log file, formats the output and closes the log file, is it that big of a leap to have it rotate the log file as well? It always has been in the past. The thing is that the web server should be serving pages, not mucking with log files. The other thing is that rotation is going to need to be different on different platforms. How does Windows handle it if you try to rotate a log file in the middle of writing to the log. There is definitely a race condition there, where you are writing a message in one thread, and another thread moves the file out of the way? I really think that if NetWare can't use the current mod_log_config then I think you need to write a NetWare specific logging module. Ryan
RE: [PATCH] Apache 1.3 built in log rotation...
Is is enough for us to simply #ifdef these changes in mod_log_config.c or would you rather see a separate logging module? Except for the addition of these two directives, everything else works for us. I would personally like to see a separate logging module, because it makes it less likely that somebody else would want to put log rotation into mod_log_config. Ryan Brad Brad Nicholes Senior Software Engineer Novell, Inc., a leading provider of Net business solutions http://www.novell.com Ryan Bloom [EMAIL PROTECTED] Wednesday, February 27, 2002 3:28:25 PM Like I mentioned before, on NetWare we can't use a piped log because the NetWare OS doesn't support pipes. A cron job is also a problem because we don't have that either. Since Apache created the log file, writes to the log file, formats the output and closes the log file, is it that big of a leap to have it rotate the log file as well? It always has been in the past. The thing is that the web server should be serving pages, not mucking with log files. The other thing is that rotation is going to need to be different on different platforms. How does Windows handle it if you try to rotate a log file in the middle of writing to the log. There is definitely a race condition there, where you are writing a message in one thread, and another thread moves the file out of the way? I really think that if NetWare can't use the current mod_log_config then I think you need to write a NetWare specific logging module. Ryan
RE: [PATCH] Apache 1.3 built in log rotation...
Isn't that the whole cross platform point here? If another platform decided that it was better for them to have log rotation in Apache, isn't it better to share code rather than reinvent or duplicate the wheel? I don't have a problem with separating the code into another module, but I would rather share than duplicate. Brad Brad Nicholes Senior Software Engineer Novell, Inc., a leading provider of Net business solutions http://www.novell.com Ryan Bloom [EMAIL PROTECTED] Wednesday, February 27, 2002 3:37:12 PM Is is enough for us to simply #ifdef these changes in mod_log_config.c or would you rather see a separate logging module? Except for the addition of these two directives, everything else works for us. I would personally like to see a separate logging module, because it makes it less likely that somebody else would want to put log rotation into mod_log_config. Ryan Brad Brad Nicholes Senior Software Engineer Novell, Inc., a leading provider of Net business solutions http://www.novell.com Ryan Bloom [EMAIL PROTECTED] Wednesday, February 27, 2002 3:28:25 PM Like I mentioned before, on NetWare we can't use a piped log because the NetWare OS doesn't support pipes. A cron job is also a problem because we don't have that either. Since Apache created the log file, writes to the log file, formats the output and closes the log file, is it that big of a leap to have it rotate the log file as well? It always has been in the past. The thing is that the web server should be serving pages, not mucking with log files. The other thing is that rotation is going to need to be different on different platforms. How does Windows handle it if you try to rotate a log file in the middle of writing to the log. There is definitely a race condition there, where you are writing a message in one thread, and another thread moves the file out of the way? I really think that if NetWare can't use the current mod_log_config then I think you need to write a NetWare specific logging module. Ryan
RE: [PATCH] Apache 1.3 built in log rotation...
Isn't that the whole cross platform point here? If another platform decided that it was better for them to have log rotation in Apache, isn't it better to share code rather than reinvent or duplicate the wheel? I don't have a problem with separating the code into another module, but I would rather share than duplicate. My point is that I don't think any other platform should have this logic. I don't think this is what a web server should be working on, and I don't think that this will work reliably on multiple platforms when you have to deal with the difficulties of threads or process based requests. Ryan
RE: [PATCH] Apache 1.3 built in log rotation...
At 04:56 PM 2/27/2002, you wrote: Isn't that the whole cross platform point here? If another platform decided that it was better for them to have log rotation in Apache, isn't it better to share code rather than reinvent or duplicate the wheel? I don't have a problem with separating the code into another module, but I would rather share than duplicate. For Apache 2.1 I agree with dirk's general sentiments; break up logging into several layers, each of which can be changed for appropriate purposes, and the uberlayer could either create one logging file, or handle log rotation. It is probably more efficient as a thread of the server than a seperate app, even in a model like worker, on at least some Unix platforms. But the answer has been [continues to be] that mod_log_config configures what goes into a log, and allows piped logs for extensibility. Proposals to change this are vetoed about four times a year. Consider this, today you want name format foo. Tommorow someone wants to introduce format bar. Now someone comes along and wants to roll in the logresolve style logic. Next we decide we want additional log filtering. The existing schema allows for extensibility via pipes/log children. What is brought up frequently is rescoping the logging in 2.1 to be more modular, so that these sort of things -could- fit into the server without crowbars. But think about one of two tacts. One - mod_log_netware probably makes the most sense for 1.3 - it impacts no other code. Also consider this for 2.0 and beyond - if the user could drop in a pipe you would be fine. But there are no pipes and child processes, correct? Look at creating an alternative schema for Netware that allows you to create faux-CGI type applications, such as log children or cgi apps. That would be far more useful going forward than hacking more cruft into mod_log_config.
RE: [PATCH] Apache 1.3 built in log rotation...
Hopefully for NetWare this will be an Apache 1.3 issue only. It is a long story, but Apache 1.3 was built on our old CLib libraries which do not support functionality like pipes. Apache 2.0 and beyond is based on LibC which is a ground up rewrite of the standard library functionality and brings us much closer to the rest of the world. With LibC, pipes as well as some other issues that we have had, should no longer be a problem. You are right, we still don't have child processes, but for the most part I have been able to fake it with threads. Our current plans for Apache 2.0 are to support the RotateLog utility just like everybody else. But we needed something to patch this hole in our 1.3 story. I will go ahead and check in a mod_log_nw module and switch our build to use that instead. Brad Brad Nicholes Senior Software Engineer Novell, Inc., a leading provider of Net business solutions http://www.novell.com [EMAIL PROTECTED] Wednesday, February 27, 2002 4:11:00 PM At 04:56 PM 2/27/2002, you wrote: Isn't that the whole cross platform point here? If another platform decided that it was better for them to have log rotation in Apache, isn't it better to share code rather than reinvent or duplicate the wheel? I don't have a problem with separating the code into another module, but I would rather share than duplicate. For Apache 2.1 I agree with dirk's general sentiments; break up logging into several layers, each of which can be changed for appropriate purposes, and the uberlayer could either create one logging file, or handle log rotation. It is probably more efficient as a thread of the server than a seperate app, even in a model like worker, on at least some Unix platforms. But the answer has been [continues to be] that mod_log_config configures what goes into a log, and allows piped logs for extensibility. Proposals to change this are vetoed about four times a year. Consider this, today you want name format foo. Tommorow someone wants to introduce format bar. Now someone comes along and wants to roll in the logresolve style logic. Next we decide we want additional log filtering. The existing schema allows for extensibility via pipes/log children. What is brought up frequently is rescoping the logging in 2.1 to be more modular, so that these sort of things -could- fit into the server without crowbars. But think about one of two tacts. One - mod_log_netware probably makes the most sense for 1.3 - it impacts no other code. Also consider this for 2.0 and beyond - if the user could drop in a pipe you would be fine. But there are no pipes and child processes, correct? Look at creating an alternative schema for Netware that allows you to create faux-CGI type applications, such as log children or cgi apps. That would be far more useful going forward than hacking more cruft into mod_log_config.
