Re: apr_token_* conclusions (was: Better casecmpstr[n]?)

[Mikhail T.]

On 25.11.2015 18:21, Bert Huijben wrote:
> That Turkish ‘I’ problem is the only case I know of where the
> collation actually changes behavior within the usual western alphabet
> of ASCII characters.
Argh, yes, I see now, what the problem would be... Thank you,

-mi

Re: apr_token_* conclusions (was: Better casecmpstr[n]?)

[Mikhail T.]

On 25.11.2015 12:42, William A Rowe Jr wrote:
> If the script switches setlocale to turkish, for example, our
> forced-lowercase content-type conversion 
> will cause "IMAGE/GIF" to become "ımage/gıf", clearly not what the
> specs intended.
I'm sorry, could you elaborate on this? Would not strtolower(3) convert
"IMAGE/GIF" to "image/gif" in /all/ locales -- including "C"? At least,
in all single-byte charsets -- such as the Turkish ISO 8859-9
? Yes, the function will
act differently on the strings containing octets above 127, but those
would occur neither in content-types nor in header-names...
> Adding unambiguous token handling functions would be good for the few
> case-insensitive string comparison, string folding, and search
> functions.  It allows the spec-consumer to trust their string processing.
Up until now, I thought, the thread was about coming up with a short-cut
-- an optimization for processing tokens, like request-headers, which
are known to be in US-ASCII anyway and where using locale-aware
functions is simply wasteful -- but not incorrect.

You seem to imply, the locale-aware functions might be doing the wrong
thing some times -- and this confuses me...

Yours,

-mi

Re: apr_token_* conclusions (was: Better casecmpstr[n]?)

[Mikhail T.]

On 25.11.2015 13:16, William A Rowe Jr wrote:
>
> Two variables, LC_CTYPE and LC_COLLATE control this text processing
> behavior.  The above is the correct lower case transliteration for
> Turkish.  In German, the upper case correspondence of sharp-S ß is
> 'SS', but multi-char translation is not provided by the simple
> tolower/toupper functions.
>
So, the concern is, some hypothetical header, such as X-ASSIGN-TO may,
after going through the locale-aware strtolower() unexpectedly become
x-aßign-to?

-mi

Re: apr_token_* conclusions (was: Better casecmpstr[n]?)

[Mikhail T.]

On 25.11.2015 14:10, Mikhail T. wrote:
>>
>> Two variables, LC_CTYPE and LC_COLLATE control this text processing
>> behavior.  The above is the correct lower case transliteration for
>> Turkish.  In German, the upper case correspondence of sharp-S ß is
>> 'SS', but multi-char translation is not provided by the simple
>> tolower/toupper functions.
>>
> So, the concern is, some hypothetical header, such as X-ASSIGN-TO may,
> after going through the locale-aware strtolower() unexpectedly become
> x-aßign-to?
I just tested the above on both FreeBSD and Linux, and the results are
encouraging:

% echo STRASSE | env LANG=de_DE.ISO8859 tr '[[:upper:]]' '[[:lower:]]'
strasse

Thus, I contend, using C-library will not cause invalid results, and the
only reason to have Apache's own implementation is performance, but not
correctness.

-mi

Re: Better ap_casecmpstr[n]?

[Mikhail T.]

On 24.11.2015 13:04, Yann Ylavic wrote:
> int ap_casecmpstr_2(const char *s1, const char *s2)
> {
> size_t i;
> const unsigned char *ps1 = (const unsigned char *) s1;
> const unsigned char *ps2 = (const unsigned char *) s2;
>
> for (i = 0; ; ++i) {
> const int c1 = ps1[i];
> const int c2 = ps2[i];
>
> if (c1 != c2) {
> return c1 - c2;
> }
> if (!c1) {
> break;
> }
> }
> return (0);
> }
Sorry, but would not the above declare "A" and "a" to be different?

-mi

Re: strncasecmp

[Mikhail T.]

On 24.11.2015 10:08, William A Rowe Jr wrote:
> As long as this function is promoted for fast ASCII-specific token
> recognition and has no other unexpected equalities, it serves a useful
> purpose.
Because of this, I'd suggest renaming it to something, that emphasizes
it being ASCII-only.

-mi

Re: strncasecmp

[Mikhail T.]

On 23.11.2015 23:14, William A Rowe Jr wrote:
> L1 cache and other direct effects of cpu internal optimization.
Just what I was thinking. Attached is the same program with one more
pair of functions added (and an easy way to add more "candidates" to the
main-driver). I changed the FOR-loop define to obtain repeatable results:

# Test 1 -- equal strings:
foreach m ( 0 1 2 )
foreach? ./strncasecmp $m 1 a A 7
foreach? end
string.h (nb=1, len=7)
time = 6.975845 : res = 0
optimized (nb=1, len=7)
time = 1.492197 : res = 0
'A' - 'a' (nb=1, len=7)
time = 1.787807 : res = 0

# Test 2 -- immediately-different strings
foreach m ( 0 1 2 )
foreach? ./strncasecmp $m 1 a x 7
foreach? end
string.h (nb=1, len=7)
time = 2.527727 : res = -23
optimized (nb=1, len=7)
time = 0.406867 : res = -23
'A' - 'a' (nb=1, len=7)
time = 0.440320 : res = -23

# Test 3 -- strings different at the very end
foreach m ( 0 1 2 )
foreach? ./strncasecmp $m 1 a x 0
foreach? end
string.h (nb=1, len=0)
time = 9.629660 : res = -23
optimized (nb=1, len=0)
time = 1.387208 : res = -23
'A' - 'a' (nb=1, len=0)
time = 1.754683 : res = -23

The new pair (method 2) does not use the static table, which is likely
to benefit from CPU-cache unfairly in repetitive benchmarks.  It is
slower than the table-using method 1 functions. But the two pairs might
be comparable -- or even faster -- in real life.

-mi

#include 
#include 
#include 

#define gettimeofday(X) gettimeofday(X, NULL)

#include 
#include 
#include 
#include 
#include 

#include 

/*
 * Provide our own known-fast implementation of str[n]casecmp()
 * NOTE: ASCII only!
 */
static const unsigned char ucharmap[] = {
0x0,  0x1,  0x2,  0x3,  0x4,  0x5,  0x6,  0x7,
0x8,  0x9,  0xa,  0xb,  0xc,  0xd,  0xe,  0xf,
0x10, 0x11, 0x12, 0x13, 0x14, 0x15, 0x16, 0x17,
0x18, 0x19, 0x1a, 0x1b, 0x1c, 0x1d, 0x1e, 0x1f,
0x20, 0x21, 0x22, 0x23, 0x24, 0x25, 0x26, 0x27,
0x28, 0x29, 0x2a, 0x2b, 0x2c, 0x2d, 0x2e, 0x2f,
0x30, 0x31, 0x32, 0x33, 0x34, 0x35, 0x36, 0x37,
0x38, 0x39, 0x3a, 0x3b, 0x3c, 0x3d, 0x3e, 0x3f,
0x40,  'a',  'b',  'c',  'd',  'e',  'f',  'g',
 'h',  'i',  'j',  'k',  'l',  'm',  'n',  'o',
 'p',  'q',  'r',  's',  't',  'u',  'v',  'w',
 'x',  'y',  'z', 0x5b, 0x5c, 0x5d, 0x5e, 0x5f,
0x60,  'a',  'b',  'c',  'd',  'e',  'f',  'g',
 'h',  'i',  'j',  'k',  'l',  'm',  'n',  'o',
 'p',  'q',  'r',  's',  't',  'u',  'v',  'w',
 'x',  'y',  'z', 0x7b, 0x7c, 0x7d, 0x7e, 0x7f,
0x80, 0x81, 0x82, 0x83, 0x84, 0x85, 0x86, 0x87,
0x88, 0x89, 0x8a, 0x8b, 0x8c, 0x8d, 0x8e, 0x8f,
0x90, 0x91, 0x92, 0x93, 0x94, 0x95, 0x96, 0x97,
0x98, 0x99, 0x9a, 0x9b, 0x9c, 0x9d, 0x9e, 0x9f,
0xa0, 0xa1, 0xa2, 0xa3, 0xa4, 0xa5, 0xa6, 0xa7,
0xa8, 0xa9, 0xaa, 0xab, 0xac, 0xad, 0xae, 0xaf,
0xb0, 0xb1, 0xb2, 0xb3, 0xb4, 0xb5, 0xb6, 0xb7,
0xb8, 0xb9, 0xba, 0xbb, 0xbc, 0xbd, 0xbe, 0xbf,
0xc0, 0xc1, 0xc2, 0xc3, 0xc4, 0xc5, 0xc6, 0xc7,
0xc8, 0xc9, 0xca, 0xcb, 0xcc, 0xcd, 0xce, 0xcf,
0xd0, 0xd1, 0xd2, 0xd3, 0xd4, 0xd5, 0xd6, 0xd7,
0xd8, 0xd9, 0xda, 0xdb, 0xdc, 0xdd, 0xde, 0xdf,
0xe0, 0xe1, 0xe2, 0xe3, 0xe4, 0xe5, 0xe6, 0xe7,
0xe8, 0xe9, 0xea, 0xeb, 0xec, 0xed, 0xee, 0xef,
0xf0, 0xf1, 0xf2, 0xf3, 0xf4, 0xf5, 0xf6, 0xf7,
0xf8, 0xf9, 0xfa, 0xfb, 0xfc, 0xfd, 0xfe, 0xff
};

int ap_strcasecmp(const char *s1, const char *s2)
{
const unsigned char *ps1 = (const unsigned char *) s1;
const unsigned char *ps2 = (const unsigned char *) s2;

while (*ps1 == *ps2 || ucharmap[*ps1] == ucharmap[*ps2]) {
if (*ps1 == '\0') {
return (0);
}
++ps1;
++ps2;
}
return (ucharmap[*ps1] - ucharmap[*ps2]);
}

int ap_strncasecmp(const char *s1, const char *s2, size_t n)
{
const unsigned char *ps1 = (const unsigned char *) s1;
const unsigned char *ps2 = (const unsigned char *) s2;
while (n--) {
if (*ps1 == *ps2 || ucharmap[*ps1] != ucharmap[*ps2]) {
return (ucharmap[*ps1] - ucharmap[*ps2]);
}
if (*ps1 == '\0') {
break;
}
++ps1;
++ps2;
}
return (0);
}

int mi_strcasecmp(const char *s1, const char *s2)
{
const unsigned char *ps1 = (const unsigned char *) s1;
const unsigned char *ps2 = (const unsigned char *) s2;

for (;;) {
int diff = *ps1 - *ps2;
switch (diff) {
case 0:
break;
case 'A' - 'a':
if (*ps1 <= 'Z' && *ps1 >= 'A')
break;
return 1;
case 'a' - 'A':
if (*ps2 <= 'Z' && *ps2 >= 'A')
 

Re: strncasecmp

[Mikhail T.]

On 23.11.2015 19:05, Yann Ylavic wrote:
> Here is the correct (new) test, along with the diff wrt the original
> (Christophe's) test.c.
BTW, if the program measures its own time, should it not use getrusage()
instead of gettimeofday()?

-mi

Re: strncasecmp

[Mikhail T.]

On 23.11.2015 17:43, Yann Ylavic wrote:
> with attachment...
There is a mistake somewhere in the optimized version:

./o 1 1  aa1a 0
Optimized (nb=1, len=0)
time = 0.611311 : res = 0

The result should not be zero. Indeed, the string.h version is correct:

./o 0 1  aa1a 0
 (nb=1, len=0)
time = 4.189366 : res = 48

Yours,

-mi

Re: strncasecmp

[Mikhail T.]

On 23.11.2015 19:05, Yann Ylavic wrote:
> while (ucharmap[*ps1] == ucharmap[*ps2++]) {
> if (*ps1++ == '\0') {
> return (0);
> }
> }
> return (ucharmap[*ps1] - ucharmap[*--ps2]);
Is there really a gain in inc- and decrementing this way? Would not it
be easier to read with the explicit increments -- and, incidentally, no
decrements at all?

while (ucharmap[*ps1] == ucharmap[*ps2]) {
if (*ps1 == '\0') {
return (0);
}
++ps1;
++ps2;
}
return (ucharmap[*ps1] - ucharmap[*ps2]);

> We don't care about the whole process time and other counters.
That's certainly true. But, then, why bother with building time-counter
into the test at all -- instead of simply relying on time(1)?

But something is still not right -- the result (for either of the
methods) can depend on the number of iterations (!!):

./strncasecmp 1 27 aCaa Ac 2
Optimized (nb=27, len=2)
time = 0.01 : res = 32
./strncasecmp 1 26 aCaa Ac 2
Optimized (nb=26, len=2)
time = 0.01 : res = 0
./strncasecmp 0 27 aCaa Ac 2
 (nb=27, len=2)
time = 0.03 : res = 32
./strncasecmp 0 26 aCaa Ac 2
 (nb=26, len=2)
time = 0.03 : res = 0

Using clang on FreeBSD/amd64 here. Yours,

-mi

Re: strncasecmp

[Mikhail T.]

On 23.11.2015 19:43, Yann Ylavic wrote:
> No measured difference in my tests, I guess it depends on likelyhood to 
> fail/succeed early in the string or not.
? I don't see, where it wins anything -- but I do see, where it loses a
little...
> That's expected (or at least no cared about in this test case). We
> simply want res to not be optimized out, so print it before leaving,
> without any particular relevance for its value (string.h and optimized
> versions should return the same res with the same args, ascii strings
> only, though). 
Yes, we do want the value printed -- such as to catch the kind of errors
I reported earlier. But my question was, why does the value change
depending on the number of iterations?

-mi

Re: Supporting non-TCP/IP transports

[Mikhail T.]

On 18.07.2015 15:43, Kean Johnston wrote:
 On 7/18/2015 9:25 PM, Phil Lello wrote:
 - UNIX-domain sockets
 I personally think this could be very useful.
Me too. In particular, it is fairly common these days for
Varnish-proxy to run on the same server as the actual Apache. Having the
two communicate via a socket would be a win...

-mi

Re: Older clients stopped working after server disabled SSLv3

[Mikhail T.]

On 29.10.2014 04:37, Yann Ylavic wrote:
 Forgot to mention the OP reproducer, that is with SSLProtocol ALL
 -SSLv3 (with or without the patch), both SSLv2Hello and SSLv3Hello
 (version SSLv3) are refused by httpd.
But if ALL is replaced with ANY, then the (patched) server will be
willing to advise the connecting clients to talk TLS, right?

That would solve our problem, though some may wonder about the subtle
differences between any and all :-) More seriously, it would also
make the config-files incompatible with earlier httpd-releases --
whereas the patch I linked to does not have this problem.

But if your patch is going to be part of the next release, I'll proceed
to building the (patched) 2.4.10 here ahead of time -- corporate
Information Security are quite nervous about us still allowing SSLv3...

Thanks! Yours,

-mi

Older clients stopped working after server disabled SSLv3

[Mikhail T.]

Hello!

After disabling SSLv3:

SSLOptions ALL -SSLv3

we noticed, that curl itself and libcurl-using programs (such as git) stopped
working on some of the (older) systems -- such as RHEL5 -- when invoked against
the https-URLs pointing at the reconfigured servers.

Invoking curl with the -1 option (a.k.a. --tlsv1) worked, but without the option
curl kept failing -- complaining about SSL protocol error. Unfortunately, there
is no way to propagate that option through git to the underlying libcurl...

On newer systems (RHEL6, FreeBSD9), things are fine, but we have a substantial
number of those old ones and need a solution...

I was able to find this question:


http://serverfault.com/questions/637880/disabling-sslv3-but-still-supporting-sslv2hello-in-apache/

and a patch linked to from one of the answers:

http://pastebin.com/Nvat7xTy

I can confirm, that the patch works -- curl and git started working after I
restarted the rebuilt httpd. And running sslscan against the patched server
continues to list the bad SSLv3 as disabled.

Could somebody, perhaps, begin reviewing it and/or comment even before it is
formally filed with Bugzilla? I searched there, but could not find anything
relevant... Thanks! Yours,

-mi

Is mod_substitute effective on mod_autoindex-generated HTML?

[Mikhail T.]

Hello!

I'm trying to make some substitutions on the HTML-output generated by 
mod_autoindex.

However, things quietly do not work... Even the simplest directive:

Substitute s/Parent/Father/

seems ignored -- and the browser is showing Parent Directory instead of
Father Directory.

Is this normal? I'm using Apache-2.4.6 currently... Thanks!

-mi

Use of HTML on mailing lists (Re: SO_REUSEPORT in the children processes)

[Mikhail T.]

On 07.03.2014 12:28, Yann Ylavic wrote:
 Sorry, this was posted from gmail...
Is it written anywhere in the bylaws of this mailing list, that use of HTML is
something to apologize for? With all due sympathies to Reindl's medical
condition, why must we -- in the second decade of the 21st century -- deny
ourselves the means of expression afforded by HTML on this list? He can -- ought
to be able to -- adjust his MUA to use his font-preferences over whatever may be
specified in the incoming email.

I'd ask this question on /any/ mailing list, but it seems especially ironic
among developers of Apache httpd -- software meant to distribute content, that
is most often written in HTML...

Granted, some HTML can be offensive even to the healthy eyes -- and ought to be
viewed as impolite (not that Yann's was anything of the kind). But to request,
as Reindl did, that /all/ postings -- from all participants -- be in plain text,
seems rather overbearing...
 Here is the plain text.
Wouldn't it be better to send such a duplication to Reindl /personally/?

To avoid this becoming a list-wide flame war, I ask for the powers running this
list to convene and hand us the law: either HTML is acceptable (my own
preference), or it is not (in which case it should be bounced back by the
mailing list software with an appropriate message).

-mi

Re: Behavior of Host: vs. SNI Hostname in proxy CONNECT requests

[Mikhail T.]

On 21.02.2014 04:08, Yann Ylavic wrote:
 Similarly, a new SSLProxyCheckPeerCN canon option could be handled
 so that admins needing ProxyPreserveHost on could still forward the
 client's Host but check the backend's CN against ServerName.
It remains my humble opinion, that this behavior should be the default
-- as the most natural one. The connection from proxy to the back-end is
completely distinct from that between user-agent and the proxy. Although
SSL can be used to assure the proxy, it reached the correct back-end, it
should not matter to this verification, which host the user-agent asked
for in the other connection.

Suppose, for example, you call a corporate office... You dial the number
and the pleasant voice answers, stating, you've reached Acme Industries
and would you, please, state your business.

You do so and the employee decides, the matter is to be handled by
executive N. He puts you on hold, while he dials the executive via
internal line.

Would it not be silly for the secretary to drop both connections (to you
and to N.), if N. answers his phone as N., rather than as Acme Industries?

Just in case I'm not sufficiently clear, in the above example you are
the user's agent (browser), Acme Industries is a web-site, secretary
is the Internet-facing proxy, and N. is the back-end server...

Yours,

-mi

mod_fastcgi and Apache 2.4 (Re: what to expect from fcgid)

[Mikhail T.]

On 21.02.2014 11:56, ? ?? wrote:
 But mod_fastcgi doesn't compile with Apache 2.4.
This is entirely possible, actually -- and has been for a while. Pathes exist
and are maintained:

https://github.com/ByteInternet/libapache-mod-fastcgi

Yours,

-mi

Re: mod_fastcgi and Apache 2.4 (Re: what to expect from fcgid)

[Mikhail T.]

On 21.02.2014 13:59, Anthony Pankov wrote:
 Thank you for suggestion.
 But this packet does not install from box in
 FreeBSD.  It  does  not  install  via  apxs -cia also.  Besides that
 there  is  a  patch for mod_fastcgi in FreeBSD ports that doesn't seem
 trivial.
Unless apa...@freebsd.org team comes up with a good reason to the contrary, the
mod_fastcgi port will be buildable with either 2.2 or 2.4 apache in a week or so
-- the PR addressing the problem was submitted last summer.

http://www.freebsd.org/cgi/query-pr.cgi?pr=ports/179710

Meanwhile, you can test the update offered on the above ticket (second diff).
Barring show-stopping feedback, that's what will be committed to the port.

For more FreeBSD-specific questions about Apache, please contact
apa...@freebsd.org (and/or myself) -- not this list. Yours,

-mi

Re: Is AuthnProviderAlias subtly broken in 2.4?

[Mikhail T.]

02.10.2013 09:59, tguadagno ???(??):
 hi, i am having the same issue, have you figured out a fix yet?
Nope... I rewrote the config replicating in multiple places the details,
that were neatly aliased in 2.2

-mi

Re: Resolved (sort of): Struggling with AuthMerging

[Mikhail T.]

03.08.2013 02:05, Ben Reser wrote:
 You don't seriously expect the auth system to know all of those intricacies?
Let me take a step back here. What I found about my particular situation
is -- using your own term -- absurd:

 1. The current behavior is not documented.
 2. The current behavior is not even known: neither you, nor anybody
else on this list of httpd developers (!) were able to recognize it,
when I first asked, nor explain, what's happening. Asking elsewhere

http://stackoverflow.com/questions/17974096/apache-2-4-how-to-close-entire-site-except-one-subdirectory
proved fruitless too.
 3. The current behavior suffers from an obvious performance penalty --
wasting CPU-cycles rerunning authz rules multiple times on each hit.

You agree with that -- the only mitigation offered was: a) it used to be
even worse in 2.2; b) fixing it would have to be done very carefully.

Need anything more be said or written for a consensus, that things do
need fixing? And, as per point 3. above, not only on the documentation
side... At the very least, I'd say, there should be a way to turn it off
per subconfig (Location, Directory, or vhost). I don't know, how to do
it. But it seems rather obvious, that it needs to be done.

03.08.2013 13:15, Stefan Fritsch wrote:
 Do you have mod_log_debug loaded? If yes, try if things change if you don't 
 load it.
No, there is no mod_log_debug loaded here. And my own -- mod_authnz_tiv
-- does not use the define either:

ap_register_auth_provider(p, AUTHN_PROVIDER_GROUP, tiv,
AUTHN_PROVIDER_VERSION, provider,
AP_AUTH_INTERNAL_PER_CONF);

Thanks. Yours,

-mi

Re: Resolved (sort of): Struggling with AuthMerging

[Mikhail T.]

03.08.2013 14:14, Eric Covener wrote:
 I don't agree re: necessity. As Ben said, httpd only knows that /tiv
 (where you tried to punch a hole) and the target of your Action
 directive have different per-directory configurations, so
 authorization is checked on the subrequest.   It's erring on the side
 of running authz checks, and I don't disagree that it could be
 enhanced/optimized.
Point is, it is /erring/. I asked Ben for possible use-cases and his two
examples were modules, which use the authorization rules to generate
different content depending on the result. Rather than to decide,
whether to authorize the request at all.

The situation would've made some sense, if I could configure things
separately. For example:

# Lock-out attempts to invoke php-fpm directly:
Location /php-fpm
Require none granted
/Location

# Allow any PHP script under DocumentRoot to be executed otherwise:
LocationMatch \.php$
Require all granted
/LocationMatch

But I can't -- all requests for foo.php would go through /both/ of the
above...

-mi

Re: Resolved (sort of): Struggling with AuthMerging

[Mikhail T.]

03.08.2013 15:19, Eric Covener ???(??):
 I didn't interpret his response that way. Those are modules that will
 create subrequests/internal redirects to new URIs that could have
 separate authz applied to them from the original URI --  you can't
 assume the server is any less interested in performing authz on them.
Ben's examples -- given in

CADkdwvRme0QObKdQVCjF+_h7St+CG8zDHhpnLXjup2V=kpq...@mail.gmail.com

-- were mod_autoindex and mod_dav_svn. Both -- as far as I understood --
used additional authz-checks when generating the /body/ of the response.
Not to decide, whether to authorize the request itself, but to decide,
what exactly to send back after the authorization already succeeded.

 The server can't tell the difference between that and
 your mod_actions internal redirect to a new URI -- they need to be
 checked.
Then, perhaps, there should be a way for me to tell the server, that
such a decision can be made for a particular Location (or Directory, or
vhost).

Yours,

-mi

Re: Resolved (sort of): Struggling with AuthMerging

[Mikhail T.]

02.08.2013 20:17, Ben Reser написав(ла):
 mod_autoindex automatically provides a directory listing of files
 under a path.  However, by default it doesn't display any paths that
 you don't have access to, e.g. .htaccess.  It does this by issuing
 subrequests for those other paths so that authz can run on them.
 (This behavior could be changed with IndexOptions ShowForbidden).

 mod_dav_svn.  Numerous commands in SVN impact other paths than the URI
 (e.g. `svn list` which is similar to the autoindex case above,
 commiting a copy or move which touch two paths one of which is on in
 the URI but rather in the headers).
The modules in your examples /deliberately/ use the authz mechanism to
generate different output based on the results. But what is doing it in
the case I describe -- where the generated content is exactly the same?

Why would anyone want to provide distinct authorization rules to /foo/
and /foo/index.html, for example? What is a possible use-case?

Likewise, why would anyone configure mod_fastcgi to hand off processing
of PHP-files to FPM:

FastCGIExternalServer   /usr/sbin/php-fpm -socket /var/run/fpm/main.sock
AddHandler  php-fastcgi .php
Action  php-fastcgi /php-fpm
ScriptAlias /php-fpm/usr/sbin/php-fpm

but need the /php-fpm/foo/ to have authorization rules different from
those of /foo/?

-mi

Resolved (sort of): Struggling with AuthMerging

[Mikhail T.]

30.07.2013 19:18, I wrote:

 authorization result of Require all granted: granted
 authorization result of RequireAny: granted
 authorization result of AuthMerging Any: granted
 authorization result of Require all granted: granted
 authorization result of RequireAny: granted
 authorization result of AuthMerging Any: granted
 authorization result of Require tiv ipaddress: denied (no
 authenticated user yet)
 authorization result of Require tiv expiration: denied (no
 authenticated user yet)
 authorization result of RequireAll: denied (no authenticated
 user yet)
 authorization result of RequireAny: denied (no authenticated
 user yet)

After instrumenting mod_authz_core.c to provide a little more
information about the actual request_rec being processed, I got the
following output, which explains, what's happening.

The problem -- and I do think, it is a bug -- is that the entire
authorization is invoked not just for the request, but for the internal
subrequests too. So, in my scenario, when I asked for /tiv/, the authz
core made the following checks (color-coded to match the above-quoted
log-entries):

 1. Check location /tiv/ -- granted, no problem.
 2. Check location /tiv/index.php -- granted as well, so far so good.
 3. We use mod_actions to hand-off processing of php-files to php-fpm,
so Apache also checks location */php-fpm/*tiv/index.php...
Because there is no explicit sublocation defined for /php-fpm/, the
rules for the Location / are invoked. Which leads to our tiv being
queried -- denying the request...

Items 1 and 2 explain, why I saw the Require all granted rule
processed twice in the log. Item 3 explain, why the rules of the top
Location got invoked at all...

I got things to work by changing the sub-location configuration to read
simply:

LocationMatch ^(/php-fpm)?/tiv/
Require all granted
DirectoryIndex  index.php
/LocationMatch

There is no need for AuthMerging even, after all. But I struggle to
understand, why the same HTTP-hit results in multiple processing of the
same authorization rules (some of which may, actually, be heavy...). Is
this really normal and expected behavior?

01.08.2013 21:05, Ben Reser wrote:
 If the resulting response is AUTHZ_DENIED_NO_USER then processing continues.

Is that so that if any of the subsequent children of the same RequireAll
say AUTHZ_DENIED, the server will not even bother figuring out the user?
Ok, makes sense, thank you. Turns out, this is not related to the main
problem, after all.

-mi

Re: Resolved (sort of): Struggling with AuthMerging

[Mikhail T.]

01.08.2013 22:47, Ben Reser ???(??):
  LocationMatch ^(/php-fpm)?/tiv/
  Require all granted
  DirectoryIndex  index.php
  /LocationMatch
 I'm guessing you're using AP_AUTH_INTERNAL_PER_CONF which should avoid
 the 3rd call with the above configuration.
Even without my module at all -- using purely Apache's own -- the above
configuration results in a browser's request for /tiv/ being checked
three times now:

 1. /tiv/
 2. /tiv/index.php
 3. /php-fpm/tiv/index.php

This does not seem right... Even if it may be necessary in some cases,
there's got to be a way to turn the behavior off...

 That's not a bug at all.  In some cases it may be necessary for
 authorization to run for sub-requests.
Could you give an example or two? Thanks,

-mi

Re: Struggling with AuthMerging

[Mikhail T.]

30.07.2013 19:27, Tim Bannister ???(??):
 The server might be working properly and it's just the documentation that has 
 fallen short.

As a minimum, testing the subsequent children of RequireAll after one of
them already responded with denied seems like a bug...

-mi

Struggling with AuthMerging

[Mikhail T.]

Hello!

I realize, configurations questions aren't meant for this list, but I'm
beginning to suspect a bug...

Here is the configuration:

Location /
AuthTypeform
AuthFormProvidertiv
Session On
SessionCookieName   ti2f
Include conf/sessionpw.conf
AuthNameTI
RequireAll
Require tiv ipaddress
Require tiv expiration
/RequireAll
/Location

Location /tiv
AuthMerging Or
Require  all granted
DirectoryIndex index.php
/Location

The idea is, the entire site is protected by our special authn/authz
module we wrote (named tiv), except for a subdirectory /tiv, which
everybody is supposed to be able to access without questions.

Unfortunately, access to /tiv is denied as well. This is, what to
authz_core:debug lists for each hit:

authorization result of Require all granted: granted
authorization result of RequireAny: granted
authorization result of AuthMerging Any: granted
authorization result of Require all granted: granted
authorization result of RequireAny: granted
authorization result of AuthMerging Any: granted
authorization result of Require tiv ipaddress: denied (no
authenticated user yet)
authorization result of Require tiv expiration: denied (no
authenticated user yet)
authorization result of RequireAll: denied (no authenticated user yet)
authorization result of RequireAny: denied (no authenticated user yet)

What makes me think, there is a bug:

 1. The Require all granted: granted and AuthMerging Any: granted
are mentioned multiple times -- instead of once per hit.
 2. AuthMerging settings -- and I tried all legal values (Off, And, Or)
-- does not seem to have an effect, I'm denied access to /tiv no
matter what.
 3. RequireAll -- from Location / -- is examined at all, despite
Location /tiv declaring AuthMerging Or (tried all values,
actually, just in case).
 4. Require tiv expiration is tested, even though its AND-connected
sibling Require tiv ipaddress has already failed.

How can I grant open access to a subdirectory (sublocation), while
keeping the rest of the server locked-up? Thanks!

-mi

Re: Decrypting mod_session-created cookie

[Mikhail T.]

On 09.07.2013 00:43, Yehuda Katz wrote:
Unfortunately not this week. Send me a reminder email next week and I should 
be able to look at it.
Although I was able to answer my own question last week — and have replicated 
Apache's default AES256 en/decryption in PHP 
http://aldan.algebra.com/%7Emi/mod_session_crypt.html, I still have another, 
related, question unanswered


Do I need to worry about the /integrity/ of the decrypted text? In other words, 
although I trust AES256 to protect the text from being decrypted by an attacker 
(as long as the passphrase is not known, of course), do I also trust it for 
protection against the text being tampered with?


If not, I'd have to implement my own signing of the contents — with some kind of 
HMAC_Foo, perhaps. But I'd rather not bother, if I don't have to... Do I? Thanks!


   -mi

Re: Decrypting mod_session-created cookie

[Mikhail T.]

09.07.2013 08:31, Eric Covener написав(ла):
 I am not surprised that separately documenting is not a priority for
 anyone,  given it's implemented in httpd for anyone to see.
Source code is not documentation, is it? In matters of encryption
especially the methods chosen should be documented better -- not only
for interoperability, but also to allow to better judge the strengths of
the method (not that I personally am capable of the latter).

Yours,

-mi

Re: Decrypting mod_session-created cookie

[Mikhail T.]

On 09.07.2013 09:59, Daniel Lescohier wrote:

So, it doesn't look like it's an quick and easy thing to document or 
re-implement.

Thanks for the pointers, Daniel. I managed to reimplement the decryption today:

   http://aldan.algebra.com/~mi/mod_session_crypt.html

Once I figure out the encryption, I'll update the above page. Yours,

   -mi

Decrypting mod_session-created cookie

[Mikhail T.]

From PHP I need to be able to set and read the session cookies created by 
mod_session_crypto.


Suppose, I know the SessionCryptoCipher (aes256, the default) and the 
SessionCryptoPassphrase, how can I decrypt the session-cookie? Its value is 
available to PHP as _REQUEST['session']...


I have both openssl and mcrypt PHP-extensions available on the server, but the 
straightforward approach of


   $decrypted = openssl_decrypt($_REQUEST['session'], 'aes256', $password);

is failing... Thank you! Yours,

   -mi

Re: Decrypting mod_session-created cookie

[Mikhail T.]

On 08.07.2013 19:11, Daniel Lescohier wrote:

https://httpd.apache.org/docs/2.4/mod/mod_session.html#sessionprivacy

The session will be automatically decrypted on load, and encrypted on save by 
Apache, the underlying application using the session need have no knowledge 
that encryption is taking place.


Thank you, Daniel, for providing a you don't need to know answer to a 
question.

I do, however, have this need -- in my application the cookie will need to be 
created by a server completely different from the one, that will be parsing it. 
I would also like to code-up a series of jmeter-tests to assure speed and 
correctness of the application -- so I'll also need to implement the same 
encryption in JavaScript or some other language available inside jmeter.


Is there, perhaps, a better answer available? Thanks,

   -mi

Re: Decrypting mod_session-created cookie

[Mikhail T.]

08.07.2013 19:35, Graham Leggett wrote:
 Like Daniel said, you don't need to know.
This is unhelpful. Do you /know/ the answer? If you do, could you share
it? If you are trying to avoid committing to a particular method --
because you foresee it changing in the future -- well, that does not
seem right either. The cookies may already be stored by the browsers and
invalidating them all by upgrading the server would not be proper.
 You can configure the decrypted session to be provided to php either via an 
 environment variable or a request header, your choice. You can then 
 optionally update the session with a response header. All encryption is 
 transparent to php.
Only if the php is running on the same -- or similarly configured Apache
server.

And then there is the other aspect I mentioned -- the tests, which would
require the session-cookie to be generated (correctly) inside JMeter.

08.07.2013 19:33, Daniel Lescohier wrote:
 Perhaps your decryption code isn't handling the salt?
Perhaps... But for now I'm just trying to decrypt the existing cookie
myself -- if only to better understand, how it is constructed. I'd
appreciate the description of the method used -- rather than a lecture
on how I don't need to worry my pretty little head...

I'm also curious, if the cookie is only encrypted, or if it is also
signed. As well as whether it is possible to just have it signed without
encrypting... Thanks,

-mi

Re: Decrypting mod_session-created cookie

[Mikhail T.]

08.07.2013 23:44, Yehuda Katz написав(ла):
 Ignoring the apache-specific configuration, it looks pretty standard
 to me (although I did not spend too long looking at it, but I did
 teach college-senior crypto last semester and it looks similar to a
 project we assigned).
Would you be able to translate the calls to APR's crypto API into PHP's
mcrypt http://www.php.net/manual/en/ref.mcrypt.php or openssl
http://www.php.net/manual/en/ref.openssl.php functions? For
simplicity, let's assume the cipher is always the default -- AES256.

Thank you very much. Yours,

-mi

Re: Decrypting mod_session-created cookie

[Mikhail T.]

08.07.2013 19:29, Graham Leggett wrote:
 See also the section on integrating with external applications.

 https://httpd.apache.org/docs/2.4/mod/mod_session.html#integration
Indeed, from this link (emphasis mine):

Standalone applications
Applications might choose to manipulate the session outside the
control of the Apache HTTP server. In this case, *it is the
responsibility of the application* to read the session from the
chosen storage mechanism, decrypt the session, update the
session, encrypt the session and write the session to the chosen
storage mechanism, as appropriate.

My question -- from the beginning of this thread -- was how to fulfill
the emphasized responsibility... I am surprised, this aspect is not
documented anywhere yet :-(

-mi

Can a module find out, whether another module is present?

[Mikhail T.]

Hello!

What is the official way for a module to check, whether another module (known by 
name) is loaded and, if so, whether its hooks (cleanup in particular) will be 
invoked before or after those of the inquirer?


I don't need to affect the order -- I just need to figure out, what it is... 
Thanks! Yours,


   -mi

Re: Can a module find out, whether another module is present?

[Mikhail T.]

On 05.02.2013 16:37, Nick Kew wrote:

But in general, querying another module, or knowing anything about
its cleanups, would be a violation of modularity.  If it's legitimate
for a module to expose its inner workings, it can do so by exporting
an API.

Why the questions?  Are you writing two modules that relate closely
to each other?
I'm not writing them -- they already exist. The two Tcl-modules (rivet and 
websh) both destroy the Tcl-interpreter at exit. The module, that gets to run 
the clean up last usually causes a crash: 
https://issues.apache.org/bugzilla/show_bug.cgi?id=54162


If each module could query, whether the other one is loaded too, the first one 
could skip destroying the interpreter -- leaving the task to the last one. This 
approach would work even if only one of them has been patched to do this.


The modularity is a great thing, of course, but when the modules use shared 
data-structures (from another library -- such as libtcl), they better cooperate, 
or else...


Yours,

   -mi

Re: Can a module find out, whether another module is present?

[Mikhail T.]

On 05.02.2013 17:14, Graham Dumpleton wrote:
In the next version of mod_wsgi though I am dropping support for coexistence. 
I want to flag that fact with a big error message and refuse to start up if 
both loaded.
I'm not sure, how Python-users will react, but, as a Tcl-user, I'd hate to be 
forced to choose one of the two modules. I'm hosting to completely unrelated 
vhosts, which use the two Tcl-using modules.


On 05.02.2013 17:20, Jeff Trawick wrote:

module *modp;
for (modp = ap_top_module; modp; modp = modp-next) {
   foo(modp-name);
}
Cool! I thought of relying on the fact, that server_rec's module_config is a an 
array of module-pointers, but the above seems more reliable. Thank you!


   -mi

Re: Can a module find out, whether another module is present?

[Mikhail T.]

On 05.02.2013 17:33, Nick Kew wrote:

Are you sure?  My recollection of Tcl is of creating an interpreter
when I want to use it, and destroying it after use.  Many could run
concurrently with a threaded MPM.
You are right. However, calling Tcl_Finalize -- which is what mod_rivet is doing 
-- would destroy /everything/ :)

The correct place to ensure calling library init and cleanuo
functons more than once doesn't hurt is in the library, and
if Tcl doesn't do that, you might want to report a bug.
Personally, I doubt, the Tcl_Finalize call is useful at all. It is only done (by 
mod_rivet), when httpd is exiting anyway.


Yours,

   -mi

Re: Can a module find out, whether another module is present?

[Mikhail T.]

On 05.02.2013 17:30, Mikhail T. wrote:

On 05.02.2013 17:20, Jeff Trawick wrote:

module *modp;
for (modp = ap_top_module; modp; modp = modp-next) {
   foo(modp-name);
}
Cool! I thought of relying on the fact, that server_rec's module_config is a 
an array of module-pointers, but the above seems more reliable. Thank you!
BTW, is modp-module_index a reliable indication of order in which modules are 
processed? In other words, of module1's index is smaller than that of module2, 
does that mean, module1's hooks will be invoked prior to module2's? Or must one 
process the link-list to establish order?


   -mi

Re: Can a module find out, whether another module is present?

[Mikhail T.]

On 05.02.2013 18:01, William A. Rowe Jr. wrote:

What if both attempt to register an identical apr_optional_fn for
tcl_destroy.  That way you will never have both optional functions
called.
My plan was for each of the modules to skip the destruction, if the OTHER module 
is registered to run clean-up AFTER it.


This way the last module in the list will always run the destructor.

FWIW I would call that function as a destructor of the process_pool,
which you can find by walking the config pool's parents.
That's an idea... But, I think, I found a Tcl-specific solution for this 
particular problem -- instead of calling Tcl_Finalize(), which ruins libtcl for 
everyone in the same process, mod_rivet should simply delete the Tcl-interpreter 
it created (websh does limit itself to exactly that already).


Let's see, what mod_rivet maintainers have to say 
(https://issues.apache.org/bugzilla/attachment.cgi?id=29923action=diff).


But this was a very educating thread nonetheless. Thank you, everyone. Yours,

   -mi

Re: Can a module find out, whether another module is present?

[Mikhail T.]

On 05.02.2013 18:25, Graham Dumpleton wrote:
If in the Apache parent process, you would still have to call Tcl_Finalize() 
at some point wouldn't you to ensure that all memory is reclaimed?
I don't think so. If only because after calling Tcl_Finalize(), any other calls 
into libtcl are undefined -- not supposed to happen. So, it can not be done on 
graceful restart anyway. From Tcl's man-page:


   Tcl_Finalize is similar to Tcl_Exit except that it does not  exit  from
   the  current  process.   It is useful for cleaning up when a process is
   finished using Tcl but wishes to continue executing, and when  Tcl  is
   used  in  a  dynamically loaded extension that is about to be unloaded.
   Your code should always invoke Tcl_Finalize when Tcl is being unloaded,
   to  ensure  proper cleanup. Tcl_Finalize can be safely called more than
   once.

One of the flaws early on in mod_python was that it didn't destroy the Python 
interpreter. When an Apache restart was done, mod_python and the Python 
library would be unloaded from memory. When the in process startup was done 
after rereading the configuration Apache would load them again. Because it was 
reloaded it was a completely clean set of static variables holding interpreter 
state and so interpreter had to be reinitialised.
websh is already doing just the Tcl_DeleteInterpreter -- for the interpreter /it 
created/. That seems like the right thing to do anyway.


If websh is wrong (and mod_rivet is right) in that an explicit call to 
Tcl_Finalize is needed for an exiting process, then we are coming back to my 
original question. Registering it as a clean-up call for the process' pool (as 
wrowe@ suggested) seems like the best approach to that.


Yours,

   -mi

Re: Can a module find out, whether another module is present?

[Mikhail T.]

On 05.02.2013 19:05, Graham Dumpleton wrote:
So the section of documentation you quote appears to support what I am saying 
that Tcl_Finalize() still needs to be called. After the module is loaded and 
initialised again, then Tcl_Init(), or whatever is used to create it again, 
would be called to start over and allow new instance of interpreter to be 
setup in parent process before new child processes are forked.
I do not think, Tcl_Init can (officially) be called after Tcl_Finalize. The 
function is meant only for situations, when libtcl (or a shared library using 
it) is itself being unloaded (think dlclose()).
As I asked before, is this being done in the Apache parent process or only in 
the child processes? If it is all only going on in the child processes, the 
point I am making is moot, but if the interpreter is being initialised in the 
Apache parent process before the fork, then it would be relevant.
I don't see, why Tcl would be initialized in the main process. If it is, that's, 
probably, a bug in itself.


But I'll await a response from mod_rivet maintainers.

   -mi

Re: Balancer persist and inherit stuff in trunk

[Mikhail T.]

On 05.01.2013 10:30, Jim Jagielski wrote:

I haven't created the patch yet... it seems backwards
to generate a backport patchset unless what is currently
in trunk is OK.
I can drop in a patch into our 2.4.3 build relatively easily. Building 
(and testing) from trunk is a more serious undertaking. Yours,


   -mi

Re: Balancer persist and inherit stuff in trunk

[Mikhail T.]

Sorry -- I meant to send the below just to Jim, but messed-up the headers in 
Thunderbird -- and it ended up looking, as if Jim wrote it :(


On 04.01.2013 14:06, j...@jagunet.com wrote:

On 04.01.2013 13:48, Jim Jagielski wrote:

Have people had a chance to test, review and try the balancer
persist and inheritance stuff in trunk? I want to make
sure that we have some level of verification and agreement
there before I work on the backports for 2.4;)
Could you give the link to the patch/issue? I'd be happy to test it with a 
custom-built 2.4.3 here. Thanks!


-mi

Use of global balancer inside a vhost (Re: [Bug 54319] New: ProxyPass has no effect in Location if rewrite occurs)

[Mikhail T.]

On 02.01.2013 08:28, Eric Covener wrote:

If mod_rewrite or your own custom module just changes the URI in place
after the configuration has been determined, it doesn't change the
per-request configuration.

mod_proxy in 2.2 doesn't properly use per-request configuration, and
crawls through every ProxyPass every time.

Your module could send an internal redirect if it wanted the
per-request configuration to reflect the new URL.
I wonder, if this is the reason, I could not refer to the globally-defined 
mod_proxy balancer from a vhost's set of mod_rewrite rules (with the P-flag 
http://wiki.apache.org/httpd/RewriteMisc#line-56):


   Proxy balancer80://mycluster
BalancerMember http://backend:80
   /Proxy
   .
   VirtualHost *:80
   .
RewriteRule .*balancer80://mycluster/%2
   ...


I ended up hard-coding the back-end for the time being, but do not like it, 
obviously, because in some cases we'd like to have multiple balancer-members:


   RewriteRule .* http://backend/%2 [DPI,P,QSA]

Are global balancer-definitions supposed to be usable by rewriterules inside 
vhosts?Thanks! Yours,


   -mi

Re: Use of global balancer inside a vhost (Re: [Bug 54319] New: ProxyPass has no effect in Location if rewrite occurs)

[Mikhail T.]

On 02.01.2013 15:27, Eric Covener wrote:

Are global balancer-definitions supposed to be usable by rewriterules inside
vhosts?Thanks! Yours,


I would have thought so, is it a 2.2/2.4 difference for you?
I would not know -- because one of our RewriteMaps is using a database (Sybase, 
in fact), we never attempted this with 2.2.


   -mi

Are lookups in mod_rewrite's maps cached for the request?

[Mikhail T.]

Suppose I have the following construct:

   ...
   RewriteCond${mymap:%1|NOT_FOUND}!=NOT_FOUND
   # Store the lookup-result in environment variable target:
   RewriteRule--[DP,E=target:${mymap:%1|NOT_FOUND}]

Note, that the key for lookup in map mymap is the same. How many map-lookups 
will mod_rewrite perform? I'm hoping, only one...


Now, I know, that there are maps and fastmaps -- the latter caching the 
lookup-results for ever. My question is about regular maps -- do they and, if 
not, should not they, cache the lookup results for the duration of a request?


Thanks! Yours,

   -mi

Re: What happens, when a CGI program is invoked?

[Mikhail T.]

On 05.12.2012 04:43, Graham Leggett wrote:

In my case, for example, the CGI-invocation belongs to a completely different 
vhost than mod_rivet and websh…

Are the configurations for these expensive modules defined server wide? This 
could be an issue where other configuration is leaking into your virtual hosts, 
and is running when not necessary.
No, the modules are only configured inside the vhosts, where they are 
used -- and these happen to be different from the vhost, where I invoke 
the CGI.


But, it would seem, the modules' hooks are invoked simply because they 
are LoadModule-ed in...


   -mi

What happens, when a CGI program is invoked?

[Mikhail T.]

Hello!

From my (limited) investigation, it would appear, that in order to 
invoke a vanilla CGI script, httpd is created and goes through all of 
the modules' initialization and then, immediately, clean-up functions...


Is that right? Some of these callbacks are fairly heavy and, in most 
cases, the init/destroy cycle is useless... Both websh and mod_rivet, 
for example, seem to initialize a Tcl-interpreter first only destroy it 
afterwards -- completely pointless, when a CGI-script is invoked.


Perhaps, it is possible to somehow bypass entering all/most of the 
modules, when processing a CGI request? Thanks!


   -mi

Re: What happens, when a CGI program is invoked?

[Mikhail T.]

On 05.12.2012 02:01, Nick Kew wrote:

Back to that maybe answer, I don't know how well mod_rivet or websh
are written, and whether they might impose a gratuitous overhead.
Perhaps you could raise your concerns with their developers?
It certainly may be a module-specific thing, but it seemed to me, there 
should be some generic way of bypassing a module's initialization 
completely, if a request will not be handled by a module.


In my case, for example, the CGI-invocation belongs to a completely 
different vhost than mod_rivet and websh...


   -mi

Re: Limitations of mod_dbd - single server per vhost

[Mikhail T.]

On 22.08.2012 07:24, Nick Kew wrote:

The implementation needs object persistence, of the kind we have
in the per-server configuration but not per-directory.  That was the
original design constraint.
I'd say, my complaint was a little misunderstood. It is not, that I wish 
to see separate DBD-configurations per location/directory, though it may 
be a good thing too.


It is that I want to be able to use more than one server per vhost. This 
collection of servers can be maintained in the persistent object you had 
in mind, where the single server is maintained now.


Perhaps, the DBD-statements should begin accepting one more (optional) 
argument -- the server's label:


   DBDriver first   freetds
   DBDriver second  mysql
   DBDParamsfirst   
   DBDParamssecond  ...

Then RewriteMaps can then changed to operate thus:

   RewriteMap   first   dbd:first:SELECT 
   RewriteMap   second  dbd:second:SELECT 

I don't think, I'm the best person to implement this -- whoever 
quick-hacked the dbd-maps in recently is, probably, better positioned 
to do this ;-) My employer's needs here are satisfied for now -- we 
decided to use text-maps for local cache and thus need only a single 
dbd-map for talking to database (if the local cache fails). But I can 
give it a try, if no one else does.


Yours,

   -mi

P.S. BTW, I fixed the FreeTDS/Sybase driver 
https://issues.apache.org/bugzilla/show_bug.cgi?id=53666 for APR/APU 
somewhat -- it has been absolutely broken and unusable for years, no 
kidding -- but don't see any movement on the ticket... Does anyone care?

Fixing apr_dbd_freetds (Re: Limitations of mod_dbd - single server per vhost)

[Mikhail T.]

Perhaps, this discussion should be happening on theticket  
https://issues.apache.org/bugzilla/show_bug.cgi?id=53666  itself? Oh, well :) 
Comments inline:

On 22.08.2012 10:47, Nick Kew wrote:

I think I've pointed a few people at the ODBC driver as an alternative.
Do you have a strong reason to use FreeTDS rather than ODBC?
Yes, ODBC is yet another layer of abstraction. I'd understand, if apr_dbd.c used 
purely ODBC, leaving the ODBC implementation to deal with the backends' native 
drivers. But if apr_dbd offers native drivers of its own, better to use those, 
if possible. That's my strong reason :-)

One question here may be, does anyone have the platforms to test-drive
your patch?
I would not know... But, in my opinion, even without getting tested by anyone 
else, my patch is still a vast improvement over the current situation...

It seems to do rather more than just fix a simple bug:
That's because the bug was not simple -- the driver remained broken ever since 
the apr_dbd.c was changed to do the parsing of query-templates centrally instead 
of delegating the job to each driver. Other drivers got updated back then, but 
not the FreeTDS one :-( It still tried to do its own parsing, and was failing...

you've removed the untainting code which was part of protecting against
injection attacks.
I don't think, the untainting was ever properly implemented (you can't do it 
right without prepared statements) and I don't believe, it is the driver's job 
to do it anyway. None of the other drivers do it either -- though they rely on 
the respective client-libraries. Sybase/FreeTDS client does not offer such 
checks -- so be it... The untainting was costly (done for each call) and not 
guaranteed -- better to leave it to the caller, IMHO.

Have you implemented prepared statements properly for all backends?
To the best of my knowledge, neither the Sybase's db-client library nor the 
FreeTDS reimplementation of it offer prepared statements 
http://lists.ibiblio.org/pipermail/freetds/2010q1/025493.html, unfortunately. 
The current implementation certainly does not use them. Sybase offers a newer 
client interface (ct-lib), that does have prepared statements (see ct_dynamic() 
http://infocenter.sybase.com/help/index.jsp?topic=/com.sybase.help.sdk_12.5.1.ctref/html/ctref/CHDIGFHG.htm), 
but FreeTDS does not provide it, so, for the driver to be compatible with both, 
it has to use the old db-interface... To me this seems like an acceptable 
caveat, as long as it is known and documented.


My patch does not touch any of the other backends.

Perhaps, some day I (or someone else) will implement apr_dbd_sybase.c -- using 
the ct-interface, which would give prepared statements and other improvements. 
But that would have to be maintained outside of Apache, because, foolishly, 
Sybase would not open-source the client... It would also be Sybase-only (no MS 
SQL Server).


For the time being, my patch offers a major improvement over the status quo -- 
the driver becomes usable. I am, in fact, preparing to use it with RewriteMaps 
to do SEO for a giant site, that still uses an old Sybase-backed CMS. The 
RewriteRules will ensure, no tainted keys are passed to the queries and the 
DB-user's credentials will be limited to only SELECTs and only for a particular 
table to mitigate the risk of any SQL-injection attack.


Yours,

   -mi

Re: Fixing apr_dbd_freetds

[Mikhail T.]

On 22.08.2012 12:55, Jeff Trawick wrote:

and/or...@apr.apache.org
You are right. Unfortunately, that list (like this one), requires one to 
subscribe before posting. I deeply resent such requirements and try to avoid 
such lists, when at all possible :-(


Yours,

   -mi

Limitations of mod_dbd - single server per vhost

[Mikhail T.]

From reading the module's documentation 
http://httpd.apache.org/docs/2.4/mod/mod_dbd.html, it seems, a single vhost 
can only use one DB-server (and only with one set of credentials): only a single 
DBDriver and DBDParams is possible, for example.


Also, the syntax of mod_rewrite's dbd/fastdbd maps 
http://httpd.apache.org/docs/2.4/rewrite/rewritemap.html#dbd doesn't even 
allow to specify the DB-server, implying, once again, there can only be one in a 
given context (entire server or virtual host). The missing capability could be 
useful to someone, who would want, for example, to check a local SQLite-database 
for data and, failing to find anything, attempt a query against a remote MySQL 
or Sybase server. Other enterprises may want to talk to multiple remote servers 
(potentially using different drivers).


I would've filed an enhancement-request, but, perhaps, this was already 
discussed here and there are some good reasons for the limitation? Thanks! Yours,


   -mi

Re: What is apreq?

[Mikhail T.]

On 28.04.2012 09:05, Joe Schaefer wrote:

libapreq2 (which is the stuff currently sitting in trunk/server)
depends entirely on libapr and libaprutil- there are no httpd-specific
parts involved in it.
To me this suggests, it may be generally useful and thus should have a 
life of its own -- outside of httpd source -- thus allowing other 
projects to begin using it as a 3rd-party library, that can be listed as 
a project's dependency.


Myself a porter of quite a few 3rd-party packages to my OS of choice 
(FreeBSD), I have a deep resentment towards software projects, which 
bundle multiple projects' code in their own sources. OpenOffice is the 
worst offender by far in this regard, bundling not only all of /its own/ 
code together, but also including (and fully recompiling as part of the 
build) the full sources of python, boost, libjpeg, expat, png, zlib, 
epm, and a bunch of other projects... Had it not been for licensing 
reasons, I'm pretty sure, they would've been bundling Java as well...


It is very rare (if ever), that such bundling is justified -- most of 
the time it is done for reasons like:


 * We wish to provide complete build tree that can be built independently
   or
 * We don't want to support interactions with other versions of the
   library, which you might have

Neither reason is valid, in my not so humble opinion:

 * The first reason leads to pessimizing well-managed computer systems
   (using well-manageable OSes), where recent versions of all
   dependencies are already independently available, for the benefit of
   the poorly-managed ones.
 * The second is completely superfluous, because no support is
   promised neither implicitly nor explicitly anyway.

So, in the same opinion, not just libapreq (if it is, indeed, generally 
useful), but also apr and aprutil ought to be available /only/ as 
standalone packages and used as such during build by default.


Yours,

   -mi

Re: Moving on

[Mikhail T.]

On 20.04.2012 09:46, Rich Bowen wrote:
My comment on this is that humorous comments can be good, and they can be 
intimidating and confusing - particularly for people who don't get the joke, 
and in particular for those whose first language is not English or other 
related languages or whose culture is not conducive to humor in a technical 
context. People who are in the know, and get the jokes, don't see this as a 
problem because, well, they get the joke.
In my opinion, this is a good argument to discourage such style from appearing 
in new comments. But it does not seem enough to rephrase the existing ones. 
Clarifications can be added, but the older comments should persist for as long 
as they remain technically correct.


Yours,

   -mi

Why aren't name-based vhosts not working properly under SSL?

[Mikhail T.]

If the SSL-certificate is the same for all named vhosts configured for the given 
IP-address/port-number combination, why can not the vhosts have different 
DocumentRoots and other settings?


Thank you. Yours,

   -mi

Re: Why aren't name-based vhosts not working properly under SSL?

[Mikhail T.]

On 16.04.2012 11:40, Tom Evans wrote:

They can. Excerpt from my httpd.conf:
Your excerpt does not show different DocumentRoots -- nor any other settings... 
Could you show more contents? What is the Apache version you are using? In all 
my attempts, Apache a) issues a pointless warning about multiple SSL vhosts on 
the same IP/port; b) uses the settings (including DocumentRoot) from the first 
vhost encountered for all of them.


On 16.04.2012 11:39, Reindl Harald wrote:
because SSL was misdesigned years ago and the Host-Header is also sent 
encrypted, so the server can not know for with hostname the ssl-handshake is 
and since he knows the Hostname AFTER handshake it is too late 
No, this does not answer my question. In my scenario the SSL-certificate is the 
same for all vhosts concerned. So Apache could use that certificate to establish 
the SSL connection, and then parse the Host:-header to determine, which group of 
other (non-SSL) settings to apply to the request. But Apache does not do that -- 
not in 2.2.22.


Is this an omission, that can and should be fixed, or am I missing something 
else? Thanks!


   -mi

Re: Why aren't name-based vhosts not working properly under SSL?

[Mikhail T.]

On 16.04.2012 11:55, Eric Covener wrote:

Got a pointer to your configuration?
Well, the real one I was designing now uses a work-around (single vhost with 
mod_rewrite examining the Host-header and picking the proper subdirectore). Here 
is a mock one, that I'd rather be using -- instead of messing with mod_rewrite:


   Listen: 443

   # Common settings for all:
   SSLCertificateFile  conf/ssl.crt/everywhere.cer
   SSLCertificateKeyFile   conf/ssl.key/everywhere.key
   SSLCertificateChainFile conf/ssl.crt/Comodo-intermediate.cer

   VirtualHost *:443

   ServerNamedrupal6
   ServerAlias   project1.example.com
   ServerAlias   project2.example.net
   DocumentRoot  /www/drupal6

   /VirtualHost

   VirtualHost *:443

   ServerNamedrupal7
   ServerAlias   project3.example.com
   ServerAlias   project4.example.net
   DocumentRoot  /www/drupal7

   /VirtualHost

Older projects 1 and 2 use Drupal-6, while the new projects 3 and 4 -- Drupal-7. 
Yours,


   -mi

Re: Why aren't name-based vhosts not working properly under SSL?

[Mikhail T.]

On 16.04.2012 12:24, Plüm, Rüdiger, Vodafone Group wrote:


Without a NameVirtualHost directive this cannot work as you intend.

Add NameVirtualHost *:433

I see... I thought, I'm already giving Apache all the information it needs, 
though (with ServerAlias directives)... But if spelling-out the NameVirtualHost 
is all I need to add, that works for me...


Thank you! Yours,

   -mi

Re: which apr to use, version numbering confusion

[Mikhail T.]

On 27.12.2011 02:12, Ruediger Pluem wrote:

You can still run httpd 2.3 with APR-UTIL 1.3.x, but you will miss some 
features then.


Will the features be missing from Apache itself? If so, is there a list 
of what's not available in the httpd?


We are using the 2.2.x branch and I prefer compiling against the 
OS-provided apr, apr-util, and pcre. Thanks! Yours,


   -mi

Re: which apr to use, version numbering confusion

[Mikhail T.]

On 27.12.2011 08:24, Ruediger Pluem wrote:

Some modules in 2.3 require the apr-util crypto API. These
won't work with older apr-util versions.
Oh, Ok -- so a module may just not be built, if apr(-util) is too old at 
compile time. But if it is available, it is fully-featured, right? In 
other words, there are no silent #ifs in the code like:


   #if APR_UTIL_VERSION  1.4
   disable some really cool feature or performance improvement
   #endif

Thanks for clarification! Yours,

   -mi

Re: Error codes

[Mikhail T.]

On 29.11.2011 23:30, William A. Rowe Jr. wrote:


But my point remains, that we allocate each module a block of some 50
codes, such that mod_aaa gets AHM-0049 and mod_aab gets 50-99, etc. 


How will 3rd-party modules be getting their blocks?

   -mi

Re: Proposal: error codes

[Mikhail T.]

On 27.11.2011 12:14, Stefan Fritsch wrote:

Yes, that would be a good idea and I agree with Daniel that we should
use a distinct prefix or format. We currently have around 2700 calls
to *_log_?error in trunk, so a 4-digit number should be ok. Together
with for example AH as prefix for Apache HTTPD this would result in
numbers like AH0815 which don't seem to cause many hits on google.
I'd like to suggest, the codes be 8-character (or shorter) words and declared as 
something like:


   union apr_error_code {
uint64_ti;
chars[8];
   };

This way the code itself can contain a meaningful short-string (printable and 
human-readable):

but different codes can still be assigned and compared without a special 
function:

   if (code.i) {
log(Attempt to foo returned: %.8s, code.s);
   }

The actual numbers may differ between platforms of different endianness, but who 
cares -- as long as the character strings themselves remain the same and 
meaningful... Yours,


   -mi

mod_ftp's handling of SITE UMASK command

[Mikhail T.]

Hello!

One of our scripts is using ncftpput to automatically upload a file. The script 
invokes ncftpput with a -U flag to explicitly set the umask.


Though the file is successfully uploaded (with the default umask) there is with 
a warning:


   % ncftpput -U 0 localhost / /COPYRIGHT
   ncftpput could not set umask: server said: Error (no message)

and the server logs the following:

   127.0.0.1 - - [18/Jul/2011:17:16:06 -0400] SITE UMASK 0 504 - - -

Looking inside the code (ftp_commands.c), I see, that the SITE-command is not 
implemented:


ftp_hook_cmd(SITE, *NULL*, FTP_HOOK_LAST,
 FTP_NEED_LOGIN | FTP_TAKE1 | FTP_EXTENSIBLE,
 sp site-cmd [ sp arguments ]);

I wonder, if that's intentional -- perhaps, for reasons of security -- or simply 
because nobody got to it yet. Please, advise. Thanks! Yours,


   -mi

passing RELATIVE filename to cronolog

[Mikhail T.]

 Hello!

For whatever reasons, we are using cronolog with Apache. From one of our 
httpd.conf files:


   CustomLog ||cronolog -p 1day -H
   /data/ti/servers/apache-/example/-prod/logs/access
   /data/ti/servers/apache-/example/-prod/logs/access.%Y%m%d-%H%M combined

Note, the /absolute/ paths for the logs... To minimize the differences 
between httpd.conf files of different Apache-servers (running on the 
same Unix server), we'd like -- if possible -- to use relative paths. 
Unfortunately, something like:


   CustomLog ||cronolog -p 1day -H logs/access
   logs/access.%Y%m%d-%H%M combined

does not work, because -- unlike Apache -- cronolog does not prepend 
server's directory to the relative path-names. Things work, if we 
/start/ Apache in the right directory, but, upon /restart/, the current 
directory is already / and cronolog simply creates /logs (at the very 
top) and puts log-files in there...


Putting environment variables into CustomLog-specified command like does 
not work -- they aren't interpreted by anything and we simply get a 
/$DIR/logs directory created upon Apache restart.


I'm wondering, if anybody has solved this somehow already... Thanks!

   -mi

Re: passing RELATIVE filename to cronolog

[Mikhail T.]

 24.09.2010 17:40, Eric Covener написав(ла):

work -- they aren't interpreted by anything and we simply get a /$DIR/logs

If it's set in the native environment when httpd is invoked, try ${DIR}
Awesome! That worked... Can you elaborate on why the plain $/var/ does 
not work, but ${/var/} does? Thank you very much!


   -mi

apr_vformatter and the %p-format

[Mikhail T.]

Why does the apr_vformatter -- and thus all its callers (apr_psprintf, 
logging-functions, etc.) reject the %p format (replacing it with words 
``bogus %p'')?


The format is for outputting pointers and is supported by the 
C-library's printf-family.


If this is a deliberate limitation, I'd like to know, what the 
motivations are. If it is simply an omission, I may be able to offer a 
patch...


Please, advise. Thanks! Yours,

   -mi

Re: apr_vformatter and the %p-format

[Mikhail T.]

08.06.2010 14:13, Mikhail T. ???(??):
Why does the apr_vformatter -- and thus all its callers (apr_psprintf, 
logging-functions, etc.) reject the %p format (replacing it with words 
``bogus %p'')?


The format is for outputting pointers and is supported by the 
C-library's printf-family.


If this is a deliberate limitation, I'd like to know, what the 
motivations are. If it is simply an omission, I may be able to offer a 
patch...
Ok, replying to myself... Apparently, one is supposed to use %pp instead 
of simple %p to output pointers. This is not well-implemented, however, 
because real printf would left-pad pointers with zeros, which is not 
even possible to request with the current apr_vformatter implementation. 
The 0x-prefix is not output either, but that one can request by 
explicitly putting it into format.


Also, trying to use %pt (for the thread ID) -- gets flagged by the gcc's 
-Wformat, because these functions are, somewhat misleadingly, declared 
printf-like.


   -mi

Qs on the post_config hook, restarts

[Mikhail T.]

Hello!

Various sources suggest, the hook can be called several times -- could 
someone summarize those times for the record?


For example, it appears, that upon start-up the hook is called once to 
check the syntax and then again -- for real. Mod-developers can check 
by recording previous invocations with apr_pool_userdata_set() 
(although, really, there should be some other mechanism).


Ok. Now, what about the graceful restarts? The hook is called again, 
understandably, but only once, right?


Now, when should a module free() its malloc()-ed memory and do other 
clean-up between restarts? It would seem, that the same process (the 
master httpd) is doing the dlopen() of the module upon every restart and 
thus its memory-use should be continuously growing...


My module, at least, is using some external libraries, so I can't rely 
on the apr_pools for clean-ups. How do I know, when to free-up the 
resources I've allocated?


Thanks!

   -mi

Re: Qs on the post_config hook, restarts

[Mikhail T.]

04.06.2010 16:00, Nick Kew ???(??):

The answer is, yes you can and should use APR pools.
To translate this answer: no, there is no hook invoked, when a module 
should clean-up... Thanks.


What about the rest of my question -- about the sequence of post_config 
hook invocations?


   -mi

mod_dav vs. mod_rewrite

[Mikhail T.]

Hello!

I'm working on adding WebDAV access to the directories already 
accessible via FTP. The layout is fairly simple: the /data/ftp tree 
houses the home directories of FTP's (virtual) users. I wanted to give 
the HTTP users the same experience they get via FTP -- they must login, 
and then their top-level directory will be /data/ftp//USERNAME/. No user 
should see the full listing of users, nor be able to get into another 
user's subtree even by guessing the other username.


To this end I set up the configuration as follows:

   DocumentRoot /data/ftp

   Directory /data/ftp

   AuthType Basic
   AuthName Foo File-Server
   Requirevalid-user
   AuthUserFile /somewhere/passwd

   RewriteEngineon
   # Without the below condition, the rewrite rule keeps rewriting
   # until Internal Error. A mystery...
   RewriteCond%{ENV:REDIRECT_STATUS}!200
   RewriteRule(.*)%{REMOTE_USER}/$1[L] 


   /Directory

   Directory  /data/ftp/*

   DAV on 

   /Directory 

This almost works -- upon going to the site (say, 
https://ftp.example.com/), the user is asked to login. He is then served 
the content of /data/ftp//USERNAME/, which appears as the top-level 
directory in the browser... Great...


However, for some reason, when a DAV-capable client is used (checked 
with cadaver and konqueror), a user wallaby sees not only the content 
of /data/ftp/wallaby, but also an entry (a collection) named 
wallaby, which is not really there on the server at all (there is no 
/data/ftp/wallaby/wallaby). Trying to access that phantom collection 
results in an error...


The questions:

  1. How do I eliminate the phantom entry from the otherwise perfect
 listing? Is its appearance a manifestation of a bug in mod_dav?
  2. Is it possible to achieve the same effect (DocumentRoot based on
 REMOTE_USER) without mod_rewrite?
  3. If the answer to 2. is no, is there a better way to express what
 I want, than the two statements I came up with? In particular,
 without the check for REDIRECT_STATUS, I see (in mod_rewrite's
 log) continuing rewrites to
 /data/ftp/wallaby/wallaby/wallaby//wallaby -- until it gives
 up and throws a 500...

Thank you very much for any hints. Yours,

   -mi

mod_reopen (Re: reopening of logs without restarting)

[Mikhail T.]

22.01.2010 15:28, Akins, Brian ???(??):

Each process (not thread) has an open filehandle with mod_log_config,
correct?
   
Having almost finished the necessary changes to mod_log_config, I 
realized, that there are/may be modules, that keep their own logging. 
Being able to include those in my reopening is not required for my 
purposes, but it would be nice...


Going through the process' table of opened files, probably, can not be 
implemented in a cross-platform fashion, but as long as the arp-API was 
used to open the log-files, I thought, I could achieve it...


Here is the (Unix-only) plan -- please, shoot me down, before I waste 
too much time going down the wrong path:


  1. On start-up, my mod_reopen will:
 * install its own signal-handler for the user-configurable
   reopen signal;
 * memorize the values of the first server_rec (server_zero)
   and arp_pool_t (pool_zero), that are passed to the hook
   registered via ap_hook_child_init();
  2. Upon receiving a signal, I will:
 * go through all servers:
  for (server = server_zero; server; server = server-next)
   re-opening their respective error_logs (each one has its own
   mutex already).
 * I will then go through all cleanup_t entries of the
   pool_zero, looking for those, that have
   apr_unix_file_cleanup listed in their plain_cleanup_fn field
   and treating their data records as the apr_file_t they are.

The file_reopen function would compare the st_dev and st_ino numbers of 
the stat structures filled by fstat-ing the current apr_file_t's filedes 
and stat-ing its fname. If a change is detected, it will open a new 
file-descriptor and, upon success, assign its value to filedes, then 
closing the old value. This will all happen while holding the existing 
thlock of the apr_file_t, if necessary.


I'm going to ignore the pipes for now, on the assumption, that people 
using them are happy with their own log-rotations. I will also, of 
course, ignore files opened for anything, but writing (in append-mode).


The questions:

  1. Am I right thinking, that the pool_zero pool will contain all
 interesting files?
  2. What is the right hook for the reoping? Safely the
 signal-handler can only set a flag, which must be examined later
 -- where? Perhaps, I can -- from the signal-handler -- inject
 myself with ap_hook_log_transaction to be called upon the next
 transaction, perform the reopening and remove myself then?

Or should I just stick to changing the mod_log_config as I initially 
planned?


   -mi

reopening of logs without restarting

[Mikhail T.]

Hello!

Some of our web-servers take a while to restart (because some custom
modules need to login to database backends, etc.) This makes it
undesirable for us to use the SIGUSR1 (for graceful restart) and we
currently log to stdin of an easier to restart command-line utility.

How hard would it be to implement a separate signal-handler, which
would -- upon receiving, say, SIGUSR2 -- reopen the log-files without
performing a full restart of each worker? That would provide for a
possibility to log straight into a file and rotate that once in a while
without a full restart of the httpds.

Currently there are ap_run_open_logs and worker_open_logs. Can my
hypothetical signal-handler simply go through the list of opened
descriptors, set them to new values and close the old?

Or is this a hairy task, that some have tried, but nobody succeeded
in implementing?

Thanks! Yours,

-mi

Re: reopening of logs without restarting

[Mikhail T.]

22.01.2010 14:03, Jeff Trawick написав(ла):

  Currently there are ap_run_open_logs and worker_open_logs. Can my
  hypothetical signal-handler simply go through the list of opened
  descriptors, set them to new values and close the old?
 

You'd need to do that in every child process, without interfering with
current log file use.
   
Yes, of course -- I'll open up a new file descriptor (or FILE* ?) first, 
and then assign the new value to the field in the relevant structure -- 
up until that assignment, the current log-file use will continue using 
the earlier descriptor.


Is not worker_open_logs() running in every child process?

  Or is this a hairy task, that some have tried, but nobody succeeded
  in implementing?
 

I'm not sure hairy captures the complexity adequately.
   

Thanks for the fair warning!.. Yours,

   -mi

Re: reopening of logs without restarting

[Mikhail T.]

22.01.2010 14:18, Graham Leggett написав(ла):

What you're probably after are piped logs:

http://httpd.apache.org/docs/2.2/logs.html#piped

If you pipe the log to something like cronolog, cronolog is able to 
rotate logs for you without needing to restart httpd. 
Thanks for the pointer, sir. We are using cronolog now. But it is, quite 
obviously, a hack -- a work-around for Apache's inability to handle 
log-rotation on its own.


Being able to pipe log messages into stdin is great, but it should not 
be necessary to achieve something basic as log-rotation. When logging 
directly to files, it ought to be possible to reopen such files without 
a full restart, however graceful.


Don't you think? Yours,

   -mi

Re: reopening of logs without restarting

[Mikhail T.]

22.01.2010 14:33, Jeff Trawick написав(ла):

the open_logs hook runs only in the parent

I see... And the children simply inherit it after forks? Thank you,

   -mi

Re: reopening of logs without restarting

[Mikhail T.]

22.01.2010 14:50, Brian J. France написав(ла):

Funny you should bring this up, at work we have been look at out patches to apr 
and httpd that we can give back to the up stream public source and one of those 
patches is to apr to allow hupless log rotation in httpd 2.x.

The patch adds APR_FOPEN_ROTATING to the flags when opening a file (which we 
patch httpd to add it), then open will save extra data after opening (flags, 
perm, apr_finfo_t, last check and timeout).  If APR_FOPEN_ROTATING is set then 
before every write call it will check to see if it needs to re-stat the file to 
see if anything has change (device/inode) and if it has it will re-open the 
file automatically.

This allows us to move log files to a new name, wait X seconds (our default 
timeout is 60, so we wait 90 to be safe) and then we can do what ever we want 
to the log file because all children will have either re-opened it or will 
re-opened it before the next write call.

Is that what you are looking for?

I was going to bring up the patch during the Monday/Tuesday hack days along 
with a few others.
   
It is very close, thank you very much. If you publish it, I will be able 
to improve it to my taste -- instead of stat-ing every time (a 
potentially expensive thing to do), it would only perform the reopening 
upon receiving a signal (such as SIGUSR2).


With that you'll have to rename the current files, send all 
httpd-processes the signal and wait for the new files to appear before 
compressing the renamed ones, etc.


BTW, if you are on Irix, Linux, Solaris-10, or a BSD system, you can 
improve your work yourself to avoid that 90 second wait too. The OSes I 
listed all have mechanisms, whereby a program can ask the kernel to 
notify it of certain events (such as new files appearing in a given 
directory). Unfortunately, the mechanisms are quite different on all 
OSes (dnotify on Linux, kqueue on BSD, etc.), but you only need to 
implemented it for /yours/ :-)


And finally, should not a module be able to perform all the reopening on 
its own -- without patching the server code? Or the modules don't have 
access to all the necessary functions/values?


Thanks! Yours,

   -mi

Re: reopening of logs without restarting

[Mikhail T.]

22.01.2010 15:28, Akins, Brian ???(??):

Each process (not thread) has an open filehandle with mod_log_config,
correct?
   
Except for the ErrorLog... Perhaps, that handle is reachable from a 
module somehow too?


   -mi