Re: New ServerUID directive

[Jim Jagielski]

But we already HAVE an "automatic" ID... that's what the Vhost's
server_rec is. The issue is that there is no "one-size-fits-all" definition
of what such an automatic ID means or how it should be defined.

Is it just Vhost name + Port? That might be good for some modules,
but NOT good for others. What if the definition/config of the
Vhost changes between restarts? Should the 2 Vhosts have
the same ID (ie: are they the "same" Vhosts?) Again, for some
modules maybe yes, others maybe no. It is up to each module
to know and use what makes sense for itself.

An automatic ID is just unnecessary fluff... It's an API data field
that provides no real useful information beyond that which we
already provide.

> On Feb 6, 2018, at 5:54 AM, Stefan Eissing  
> wrote:
> 
> 
> 
>> Am 06.02.2018 um 11:45 schrieb Helmut K. C. Tessarek :
>> 
>> On 2018-02-06 05:13, Yann Ylavic wrote:
>>> Sorry for what is probably (my) bad english, "fixed" meant "the same
>>> after restart (or stop/start)".
>> 
>> Right, but isn't the virtual host's server name/port config after the
>> restart the same as well? Why do you need a new separate unique identifier?
>> 
>> And should you ever change the port number and/or the virtual host's
>> server name, then this virtual host won't be the same after a restart
>> anyway.
>> 
>> Either I'm missing something here, but I still don't understand the
>> reason for a unique identifier, when you already have one.
> 
> You are missing that Yann exactly wants to do that. 
> 
> Only as consideration for people who prefer otherwise, he considered to
> introduce a ServerUID directive.
> 
> Now, he tried several times to get the discussion back to what a good
> *automatic* id for the load balancer is, but everyone keeps discussing
> directives...
> 
> *Waves Jedi Hand*: "Forget the directive..."
> 
>> 
>> Or at least one that can be used from a combination of several fields in
>> the server struct.
>> 
>> What am I missing?
>> 
>> -- 
>> regards Helmut K. C. Tessarek  KeyID 0x172380A011EF4944
>> Key fingerprint = 8A55 70C1 BD85 D34E ADBC 386C 1723 80A0 11EF 4944
>> 
>> /*
>>  Thou shalt not follow the NULL pointer for chaos and madness
>>  await thee at its end.
>> */

Re: New ServerUID directive

[Yann Ylavic]

On Tue, Feb 6, 2018 at 11:54 AM, Stefan Eissing
 wrote:
>
>> Am 06.02.2018 um 11:45 schrieb Helmut K. C. Tessarek :
>>
>> On 2018-02-06 05:13, Yann Ylavic wrote:
>>> Sorry for what is probably (my) bad english, "fixed" meant "the same
>>> after restart (or stop/start)".
>>
>> And should you ever change the port number and/or the virtual host's
>> server name, then this virtual host won't be the same after a restart
>> anyway.

Which may be irrelevent for, say, a module which cares only about
backend connections.
Why would adding another incoming IP:port or ServerAlias matter there?

>>
>> Either I'm missing something here, but I still don't understand the
>> reason for a unique identifier, when you already have one.
>
> You are missing that Yann exactly wants to do that.
>
> Only as consideration for people who prefer otherwise, he considered to
> introduce a ServerUID directive.
>
> Now, he tried several times to get the discussion back to what a good
> *automatic* id for the load balancer is, but everyone keeps discussing
> directives...

Thanks Stefan, I couldn't have said better!

>
> *Waves Jedi Hand*: "Forget the directive..."

May the force be with you ;)

Re: New ServerUID directive

[Michael]

On 06/02/2018 11:54, Stefan Eissing wrote:



Am 06.02.2018 um 11:45 schrieb Helmut K. C. Tessarek :

On 2018-02-06 05:13, Yann Ylavic wrote:

Sorry for what is probably (my) bad english, "fixed" meant "the same
after restart (or stop/start)".

Right, but isn't the virtual host's server name/port config after the
restart the same as well? Why do you need a new separate unique identifier?

And should you ever change the port number and/or the virtual host's
server name, then this virtual host won't be the same after a restart
anyway.

Either I'm missing something here, but I still don't understand the
reason for a unique identifier, when you already have one.

You are missing that Yann exactly wants to do that.

Only as consideration for people who prefer otherwise, he considered to
introduce a ServerUID directive.

Now, he tried several times to get the discussion back to what a good
*automatic* id for the load balancer is,


Ah, for the fortunate that have so much traffic they need the 'lb'. And 
I imagine, for that 'automatic' is fine. Never had to use one though - 
so no idea how hard they are to configure/manage. However, I expect I 
would rather "not care" how the internals work for giving me a vhost 
ServerID. Why should I care - after a restart whether the value 
generated is the same or not.


That said - what could I do with a ServerID (forget the unique for the 
moment).


Again, my first thoughts are with regard to 'security' aka 'access 
control'. Could I use (or is there already something I am unaware of) a 
ServerID in  blocks, e.g., with  - so that I can 
specify access control in terms of the  rather than as attributes 
of clients. Might all be nonsense - asin - this is just me brainstorming.


I guess my question is closer to: are there ways to manage 'access 
control' based on the server configuration and the physical resources 
(mainly thinking files). What is more manageable? What is easier to 
report on/with (to a non-httpd specialist). What is easier to audit/log, 
perhaps in separate logs?



  but everyone keeps discussing
directives...

*Waves Jedi Hand*: "Forget the directive..."

(* Michael blinks - what were we talking about? *)



Or at least one that can be used from a combination of several fields in
the server struct.

What am I missing?

--
regards Helmut K. C. Tessarek  KeyID 0x172380A011EF4944
Key fingerprint = 8A55 70C1 BD85 D34E ADBC 386C 1723 80A0 11EF 4944

/*
   Thou shalt not follow the NULL pointer for chaos and madness
   await thee at its end.
*/

Re: New ServerUID directive

[Yann Ylavic]

On Tue, Feb 6, 2018 at 11:27 AM, Michael  wrote:
> On 06/02/2018 11:13, Yann Ylavic wrote:
>>
>> And the U in UID meant Unique, not User :/
>
> Then I think UUID - except you want it remembered over restarts - rather
> than for a session.

The first U there is Universal (Univeral Unique ID), and I didn't want
to impose a format either...
Simple ID would have done it probably (implies unique anyway).

>
> However, if it is not for 'external use' - how does it help me? What does it
> solve (ignorance is bliss!)? I got rather lost during the internal focused
> discussion.

It's both for "external" (e.g. %{SERVER_ID} anywhere you want) and
internal use where appropriate since this ID is garanteed to be unique
for the lifetime of the vhost (per admin choice), it could be used to
associate resources with the same (per admin choice) lifetime.
We can do things to compute this ID automatically when none is given,
but I think we can't do so *and* please everyone (doing configuration
changes), hence this long discussion...

Re: New ServerUID directive

[Stefan Eissing]

> Am 06.02.2018 um 11:45 schrieb Helmut K. C. Tessarek :
> 
> On 2018-02-06 05:13, Yann Ylavic wrote:
>> Sorry for what is probably (my) bad english, "fixed" meant "the same
>> after restart (or stop/start)".
> 
> Right, but isn't the virtual host's server name/port config after the
> restart the same as well? Why do you need a new separate unique identifier?
> 
> And should you ever change the port number and/or the virtual host's
> server name, then this virtual host won't be the same after a restart
> anyway.
> 
> Either I'm missing something here, but I still don't understand the
> reason for a unique identifier, when you already have one.

You are missing that Yann exactly wants to do that. 

Only as consideration for people who prefer otherwise, he considered to
introduce a ServerUID directive.

Now, he tried several times to get the discussion back to what a good
*automatic* id for the load balancer is, but everyone keeps discussing
directives...

*Waves Jedi Hand*: "Forget the directive..."

> 
> Or at least one that can be used from a combination of several fields in
> the server struct.
> 
> What am I missing?
> 
> -- 
> regards Helmut K. C. Tessarek  KeyID 0x172380A011EF4944
> Key fingerprint = 8A55 70C1 BD85 D34E ADBC 386C 1723 80A0 11EF 4944
> 
> /*
>   Thou shalt not follow the NULL pointer for chaos and madness
>   await thee at its end.
> */
> 

Re: New ServerUID directive

[Helmut K. C. Tessarek]

On 2018-02-06 05:13, Yann Ylavic wrote:
> Sorry for what is probably (my) bad english, "fixed" meant "the same
> after restart (or stop/start)".

Right, but isn't the virtual host's server name/port config after the
restart the same as well? Why do you need a new separate unique identifier?

And should you ever change the port number and/or the virtual host's
server name, then this virtual host won't be the same after a restart
anyway.

Either I'm missing something here, but I still don't understand the
reason for a unique identifier, when you already have one.

Or at least one that can be used from a combination of several fields in
the server struct.

What am I missing?

-- 
regards Helmut K. C. Tessarek  KeyID 0x172380A011EF4944
Key fingerprint = 8A55 70C1 BD85 D34E ADBC 386C 1723 80A0 11EF 4944

/*
   Thou shalt not follow the NULL pointer for chaos and madness
   await thee at its end.
*/



signature.asc
Description: OpenPGP digital signature

Re: New ServerUID directive

[Michael]

On 06/02/2018 11:13, Yann Ylavic wrote:

On Tue, Feb 6, 2018 at 10:21 AM, Michael  wrote:

On 01/02/2018 18:54, Yann Ylavic wrote:

I have this patch (attached) floating around that allows users to
configure a *fixed* UID for each vhost.

When I read the above sentence - with the emphasis on *fixed* my first
thoughts were not on httpd internals on how a (semi-)fixed ID is used
internally.

Sorry for what is probably (my) bad english, "fixed" meant "the same
after restart (or stop/start)".
I 'translated' it to mean - 'settable' (as a Directive) - and permanence 
(restarts aka multiple sessions) and bound to the vhost.



Falls in the category of "don't want to know. Instead, my
initial reaction - this might be useful for having multiple UID - aka
"UserNames" or other sort of "external", UID.

And the U in UID meant Unique, not User :/


Then I think UUID - except you want it remembered over restarts - rather 
than for a session.


I 'strongly dislike' naming things (i.e., I have a terrible coming up 
with names I like). PID (Permanant) ID is also confusing. Maybe VHID 
(Virtual Host ID), or UVID or longer UVHID).


However, if it is not for 'external use' - how does it help me? What 
does it solve (ignorance is bliss!)? I got rather lost during the 
internal focused discussion.




So I may not have choosen the right terms in the first place...
Maybe - but great new ideas come from confusion about what was initially 
meant :)



Regards,
Yann.

Re: New ServerUID directive

[Yann Ylavic]

On Tue, Feb 6, 2018 at 10:21 AM, Michael  wrote:
> On 01/02/2018 18:54, Yann Ylavic wrote:
>
> I have this patch (attached) floating around that allows users to
> configure a *fixed* UID for each vhost.
>
> When I read the above sentence - with the emphasis on *fixed* my first
> thoughts were not on httpd internals on how a (semi-)fixed ID is used
> internally.

Sorry for what is probably (my) bad english, "fixed" meant "the same
after restart (or stop/start)".

> Falls in the category of "don't want to know. Instead, my
> initial reaction - this might be useful for having multiple UID - aka
> "UserNames" or other sort of "external", UID.

And the U in UID meant Unique, not User :/

So I may not have choosen the right terms in the first place...


Regards,
Yann.

Re: New ServerUID directive

[Michael]

On 01/02/2018 18:54, Yann Ylavic wrote:

I have this patch (attached) floating around that allows users to
configure a*fixed*  UID for each vhost.


I am not an httpd expert. And I probably already know more than I want to.

When I read the above sentence - with the emphasis on *fixed* my first 
thoughts were not on httpd internals on how a (semi-)fixed ID is used 
internally. Falls in the category of "don't want to know. Instead, my 
initial reaction - this might be useful for having multiple UID - aka 
"UserNames" or other sort of "external", UID.


As I read the discussions I realized my first thought was off - but 
still I continued thinking about - are there ways to make a vhost UID 
external, e.g., add to my logs for accountability. And I found myself 
"dreaming" - how about a different UID (aka Username) that would 
setuid() per vhost. Could be a nice way to separate data permissions - 
per vhost.


So, maybe you do not really need it for something internal - per your 
own discussion.


However, as a new "Directive" and all - what are ways this could be 
applied to improve/enhance security and/or accountability. Is, perhaps, 
the concept of a ServerUID as a new directive something that could be 
useful to a complex website - and much more than config-file fluff?


I hope this helps - and is "out of the box" thinking. If not, well I 
tried. :)


Good day all!

Michael

Re: New ServerUID directive

[Jim Jagielski]

> On Feb 5, 2018, at 7:38 AM, Stefan Eissing  
> wrote:
> 
> 
> 2. Does httpd core expose a VirtualHost Identifier in its API and
>   what would the semantic properties of such an identifier be?
> 

Yes, it does. It's the server_rec. That contains all the info required
to determine any and all fashion of "unique" Vhost IDs.

Re: New ServerUID directive

[Stefan Eissing]

> Am 04.02.2018 um 15:32 schrieb Jim Jagielski :
> 
> 
> 
>> On Feb 2, 2018, at 8:05 PM, Yann Ylavic  wrote:
>> 
>> That's a new directive too, very specialized (though it's your point).
>> Wouldn't it end up being declined to other modules one day or the other?
>> But mainly, rather than ignoring config changes, I think mod_proxy_lb
>> should ignore non-changes.
>> And it already knows how to do that provided the vhost's balancer id
>> does not change unnecessarily...
> 
> The main point is that modules are free to determine what they
> consider a unique Vhost; they are free to make whatever assumptions
> are required. Inevitably, it depends on the module and WHY it needs
> to know if a Vhost is "unique" and that will vary. Some will be more
> stringent, others more free. Creating a user land directive which is
> basically for internal module use doesn't help at all.

No one wants to restrict any module's freedom to choose meaningful 
identifier for the data it manages. I think we are all in agreement
to that.

There seem to be two things worth discussion:

1. How can the default behaviour of mod_proxy_lb be changed, so
   that site owner with frequent reloads no longer suffer the current
   problems.

I think we can get a solution without introducing a new directive.
That would be good for our users.

2. Does httpd core expose a VirtualHost Identifier in its API and
   what would the semantic properties of such an identifier be?

The second part is what I and Yann have both argued for. I do not
know if we are in 100% agreement to what it should be, but we both
seem to see use cases where an identifier (for me, an id always is 
unique) for a virtual host is useful.
Again, I think that such a server id can be handed out by httpd
without using a directive. That id does not have to satisfy all
uses cases, since modules - as stated above - remain free to use 
it or come up with their own.

Cheers,

Stefan

> 
> The only issue is that *in this particular case* there is a
> weird use case where the assumption on what makes a
> Vhost unique doesn't match with what they want. One
> option, of course, is to say "Well, you need to figure out
> a way such that when you adjust the config file, the line
> number doesn't change". For such an extremely rare edge
> case, this seems a valid suggestion, rather than directly
> jumping into code and adding fluff and cruft that 99.%
> of httpd users will never want and see.
> 
> But sometimes, such edge cases are not-so-rare enough that
> we provide code workaround for the normal case, ala the
> singular workaround directive suggested. It's specific, targeted
> and low touch for the vast majority of httpd users and admins.
> 
> For example, consider the above. You say "I think mod_proxy_lb
> should ignore non-changes.". What do we do for the admin
> who doesn't feel that way? Or that we want ServerUID to be
> one thing for module foo and another for module bar? Isn't
> it up to the *module* itself to determine what makes sense?
> The module itself is (hopefully!! :) ) designed and architected
> and "logic" with its own specific assumption and requirement
> on what a unique UID needs to be. I would guess that 99.999%
> of the time, any change to that would cause really weird behavior.
> 
> This just seems like a heavy-weight "fix" for an extremely
> weird on one-off edge case which is more suitably handled
> by, if an actual code change is really *required*, a small
> targeted patch.

Re: New ServerUID directive

[Yann Ylavic]

On Sun, Feb 4, 2018 at 3:32 PM, Jim Jagielski  wrote:
>
>
>> On Feb 2, 2018, at 8:05 PM, Yann Ylavic  wrote:
>>
>> That's a new directive too, very specialized (though it's your point).
>> Wouldn't it end up being declined to other modules one day or the other?
>> But mainly, rather than ignoring config changes, I think mod_proxy_lb
>> should ignore non-changes.
>> And it already knows how to do that provided the vhost's balancer id
>> does not change unnecessarily...
>
> The main point is that modules are free to determine what they
> consider a unique Vhost; they are free to make whatever assumptions
> are required. Inevitably, it depends on the module and WHY it needs
> to know if a Vhost is "unique" and that will vary. Some will be more
> stringent, others more free. Creating a user land directive which is
> basically for internal module use doesn't help at all.

I agree with that in general, can we agree that mod_proxy_lb wants a
fixed server id on restart and that it'd rather create new SHMs solely
based on its own configuration changes (i.e. lbmethod, # members,
members' loadfactor/status/...)?
This is clearly not what happens currently (besides maybe the number
of members which could make the SHM's size change and cause a reset).

>
> The only issue is that *in this particular case* there is a
> weird use case where the assumption on what makes a
> Vhost unique doesn't match with what they want. One
> option, of course, is to say "Well, you need to figure out
> a way such that when you adjust the config file, the line
> number doesn't change". For such an extremely rare edge
> case, this seems a valid suggestion, rather than directly
> jumping into code and adding fluff and cruft that 99.%
> of httpd users will never want and see.

I don't understand why you state it's a weird/edge case.
Start httpd, add a blank line anywhere before the vhost which runs the
balancer, gracefuly restart httpd and see how new SHMs are created for
the new generation, losing all the previous state (load distribution,
errors, stats...).
IMHO the old and new generations should share and preserve the state here.

>
> For example, consider the above. You say "I think mod_proxy_lb
> should ignore non-changes.". What do we do for the admin
> who doesn't feel that way? Or that we want ServerUID to be
> one thing for module foo and another for module bar? Isn't
> it up to the *module* itself to determine what makes sense?
> The module itself is (hopefully!! :) ) designed and architected
> and "logic" with its own specific assumption and requirement
> on what a unique UID needs to be. I would guess that 99.999%
> of the time, any change to that would cause really weird behavior.

If modules want to build this uid for their own use they can already
do that (with whatever they find relevent in the server_rec).
But if they want to reassociate their resources to the same vhost
regardless of changes (irrelevent to them) done on restart, how would
they without the help of the only who knows (the webmaster)? This is
where any automatic thing can't please everyone.
Of course this does not avoid the module to check that the
configuration they care about is still compatible with the resources
they want to reuse, and to do the proper action otherwise, but
anything else is irrevelevent to them.
For instance, for the mod_proxy_lb case, if you add a listening IP or
a ServerAlias I don't see why the balancer should suddenly clear
everything, or refuse to start, should it? Isn't it about backend
connections only?
A quick look at "mod_bmx_vhost.c" (mentioned in the other thread)
shows that it uses "BMXVHostDBMFilename" to identify vhosts, i.e. by
their dbm path. If you want a simple and targeted solution for
mod_proxy_lb, just let the admin configure the SHM file path, that's
the UID. Simple.
Friendly? hmm. Prone to directives proliferation? sure.

>
> This just seems like a heavy-weight "fix" for an extremely
> weird on one-off edge case which is more suitably handled
> by, if an actual code change is really *required*, a small
> targeted patch.

Again, the ServerUID is not a fix from my POV, it's not targeted at
mod_proxy_lb.
And the mod_proxy_lb case is not weird, it's a bug which we should
fix, with or without ServerUID.

But I don't want to push ServerUid against all odds, let's forget
about it and talk about an internal/autmatic "server uid" which
uniquely identifies a vhost and can serve all users (no that weird)
needs.
What would it be for mod_proxy_lb since we are at it? :)

Re: New ServerUID directive

[Jim Jagielski]

> On Feb 2, 2018, at 8:05 PM, Yann Ylavic  wrote:
> 
> That's a new directive too, very specialized (though it's your point).
> Wouldn't it end up being declined to other modules one day or the other?
> But mainly, rather than ignoring config changes, I think mod_proxy_lb
> should ignore non-changes.
> And it already knows how to do that provided the vhost's balancer id
> does not change unnecessarily...

The main point is that modules are free to determine what they
consider a unique Vhost; they are free to make whatever assumptions
are required. Inevitably, it depends on the module and WHY it needs
to know if a Vhost is "unique" and that will vary. Some will be more
stringent, others more free. Creating a user land directive which is
basically for internal module use doesn't help at all.

The only issue is that *in this particular case* there is a
weird use case where the assumption on what makes a
Vhost unique doesn't match with what they want. One
option, of course, is to say "Well, you need to figure out
a way such that when you adjust the config file, the line
number doesn't change". For such an extremely rare edge
case, this seems a valid suggestion, rather than directly
jumping into code and adding fluff and cruft that 99.%
of httpd users will never want and see.

But sometimes, such edge cases are not-so-rare enough that
we provide code workaround for the normal case, ala the
singular workaround directive suggested. It's specific, targeted
and low touch for the vast majority of httpd users and admins.

For example, consider the above. You say "I think mod_proxy_lb
should ignore non-changes.". What do we do for the admin
who doesn't feel that way? Or that we want ServerUID to be
one thing for module foo and another for module bar? Isn't
it up to the *module* itself to determine what makes sense?
The module itself is (hopefully!! :) ) designed and architected
and "logic" with its own specific assumption and requirement
on what a unique UID needs to be. I would guess that 99.999%
of the time, any change to that would cause really weird behavior.

This just seems like a heavy-weight "fix" for an extremely
weird on one-off edge case which is more suitably handled
by, if an actual code change is really *required*, a small
targeted patch.

Re: New ServerUID directive

[Jim Jagielski]

> On Feb 4, 2018, at 5:49 AM, Stefan Eissing  
> wrote:
> 
> 
> Initially, ServerUID sounded like a tweak, but now we are talking
> about the identity of a server_rec across reloads and how that
> can be useful.
> 

But does it need to be a user Directive? I still say it is not needed
whether or not.

My point is that each module is free to determine what makes
a Vhost "unique", all by itself, across graceful restarts, across
restarts with config file changes, etc... It is all based on
the assumption that, TO THAT MODULE, what "is" a unique
Vhost. And that assumption is not absolute. It depends on
the use case. And modules, via post_config and determining
which post config # it is, can create it's own internal ServerUID
for whatever it needs to use it for. There is no need for this
to be some external ID for the user land.

So even if we create this directive, we will undoubtedly run into
cases where people say "Well, it doesn't do exactly what
I want I want ServerUID to change when I do 'foo' or remain
the same when I do 'bar'" and we will add additional fluff
and cruft.

It's a solution looking for a problem.

Re: New ServerUID directive

[Stefan Eissing]

> Am 03.02.2018 um 02:05 schrieb Yann Ylavic :
> 
> On Fri, Feb 2, 2018 at 5:52 PM, Jim Jagielski  wrote:
>> But we already have a unique ID in the form of
>> the actual server struct itself, which contains
>> all the data required to make a vhost "unique"
> 
> Sure, my patch or any other way to compute an automatic UID for a
> vhost don't need anything else than the server_rec.
> More exactly the proposed patch needs all the servers because it
> doesn't take the line number into account, and nothing prevents
> multiple exact same vhosts to be configured (totally useless, so an
> implementation detail and not the point here).
> 
>> 
>> The only "issue" is what makes a host "unique"... IMO,
>> in an environ where the definition line of the vhost changes
>> between restarts is a "new" vhost and that was the assumption
>> that the implementation was based on.
> 
> I don't deny it can be a valid assumption for some cases, but I
> observe that it breaks others like "automatic vhost UID should remain
> the same when some totally unrelated line is added before it".

Code exists that wants to persist (and be it shared mem) across
reloads or even restarts for the same (set of) virtual hosts. 

Code that wants to re-init on every reload has the generation 
and/or timestamps to work with, I assume.


>> *But that assumption can more easily be changed by providing
>> a work-around for that specific use case*
> 
> I'm not proposing this directive to address the mod_proxy_lb case, and
> just gave it as an example for an in-tree case where it would be
> useful (at least a new automatic UID is needed I think).
> That case just reminded me that I had a patch somewhere for the
> ServerUID, which is actually part of a larger one for a per-vhost
> scoreboard/logio stuff which I already talked about some times ago
> (with no apparent feedback/interest).

On my ever growing list of needful things is status information about
managed domains. It would be interesting to also show how much traffic
and redirects are being observed on individual server_rec* belonging
to a domain.

> This larger patch really needs a vhost UID for stats to be relevent
> days/month/years later, regardless of minor configuration changes,
> even for things like vhost IP or port (which does not matter too much
> in a local network, not to talk about line numbers...) or
> ServerName/Alias (nothing mandates them for a vhost to work, e.g. IPs
> only and ProxyPreserveHost).

I always think of ServerName as identifier, although the server itself
does not make that assertion. What would be the use of identifying vhosts
*not* based on their ServerName? (Curious)

> Anyway, if you already have an UID in an User Interface to identify a
> vhost, put it in ServerUID for httpd to use it where needed internally
> and/or to "print" that UID anywhere you configured it to (e.g.
> %{SERVER_UID} in access_log, headers, expr, ...).
> At the end of the day, the webmaster really talks about the ServerName
> or this "UID" to refer to a vhost (they can be and even usually are
> the same modulo ports 80/443), but never about the IP or line number
> it once had.

+1

> Sooo :) instead of proposing the change on how we can uniquely
> identify a vhost for mod_proxy_lb only (in an automatic way or not), I
> thought ServerUID could start its upstream life.

+1.

>> instead of creating a
>> whole new Directive for a "generic" server UID which:
>> 
>>  1. Implies that the webmaster must know when and how that UID
>> is used internally

I think key is to define a concept of a server identifier that
has clear semantics without knowing the internals.

> How is it different than now, or than anything we could do
> automatically/unconditionaly based on the server_rec?
> This argument sounds to me like we really need the ServerUID, if the
> webmaster really wants httpd to use an ID which does not depend on
> whatever future change to the configuration, (s)he has to provide that
> ID or know the internals.
> The only requirement on the httpd side is that an given ServerUID is
> unique (that should be documented, though the directive name not
> really ambiguous). It shouldn't be too hard for the user, UUID or
> alike.
> 
>>  2. Add more Directive cruft to an already bloated collection
>> of directives.
> 
> That's a very valid concern! But I think my proposal is quite
> orthogonal to the mod_proxy_lb issue, and it doesn't avoid us to find
> one (or Stefan's multiple) good way to uniquely identify a vhost when
> no ServerUID is specified.

There are directives which introduce "things" in the server, e.g.
"VirtualHost" and there are directives which peek/poke at highly
specific aspects of something, tweaking the defaults. There are
few of the first and a myriad of the latter ones.

Initially, ServerUID sounded like a tweak, but now we are talking
about the identity of a server_rec across reloads and how that
can be useful.

I would like to come 

Re: New ServerUID directive

[Yann Ylavic]

On Fri, Feb 2, 2018 at 5:52 PM, Jim Jagielski  wrote:
> But we already have a unique ID in the form of
> the actual server struct itself, which contains
> all the data required to make a vhost "unique"

Sure, my patch or any other way to compute an automatic UID for a
vhost don't need anything else than the server_rec.
More exactly the proposed patch needs all the servers because it
doesn't take the line number into account, and nothing prevents
multiple exact same vhosts to be configured (totally useless, so an
implementation detail and not the point here).

>
> The only "issue" is what makes a host "unique"... IMO,
> in an environ where the definition line of the vhost changes
> between restarts is a "new" vhost and that was the assumption
> that the implementation was based on.

I don't deny it can be a valid assumption for some cases, but I
observe that it breaks others like "automatic vhost UID should remain
the same when some totally unrelated line is added before it".

>
> *But that assumption can more easily be changed by providing
> a work-around for that specific use case*

I'm not proposing this directive to address the mod_proxy_lb case, and
just gave it as an example for an in-tree case where it would be
useful (at least a new automatic UID is needed I think).
That case just reminded me that I had a patch somewhere for the
ServerUID, which is actually part of a larger one for a per-vhost
scoreboard/logio stuff which I already talked about some times ago
(with no apparent feedback/interest).
This larger patch really needs a vhost UID for stats to be relevent
days/month/years later, regardless of minor configuration changes,
even for things like vhost IP or port (which does not matter too much
in a local network, not to talk about line numbers...) or
ServerName/Alias (nothing mandates them for a vhost to work, e.g. IPs
only and ProxyPreserveHost).
Anyway, if you already have an UID in an User Interface to identify a
vhost, put it in ServerUID for httpd to use it where needed internally
and/or to "print" that UID anywhere you configured it to (e.g.
%{SERVER_UID} in access_log, headers, expr, ...).
At the end of the day, the webmaster really talks about the ServerName
or this "UID" to refer to a vhost (they can be and even usually are
the same modulo ports 80/443), but never about the IP or line number
it once had.

Sooo :) instead of proposing the change on how we can uniquely
identify a vhost for mod_proxy_lb only (in an automatic way or not), I
thought ServerUID could start its upstream life.

> instead of creating a
> whole new Directive for a "generic" server UID which:
>
>   1. Implies that the webmaster must know when and how that UID
>  is used internally

How is it different than now, or than anything we could do
automatically/unconditionaly based on the server_rec?
This argument sounds to me like we really need the ServerUID, if the
webmaster really wants httpd to use an ID which does not depend on
whatever future change to the configuration, (s)he has to provide that
ID or know the internals.
The only requirement on the httpd side is that an given ServerUID is
unique (that should be documented, though the directive name not
really ambiguous). It shouldn't be too hard for the user, UUID or
alike.

>   2. Add more Directive cruft to an already bloated collection
>  of directives.

That's a very valid concern! But I think my proposal is quite
orthogonal to the mod_proxy_lb issue, and it doesn't avoid us to find
one (or Stefan's multiple) good way to uniquely identify a vhost when
no ServerUID is specified.

>
> Instead, as I said, we should have a specific balancer directive
> or option, complementing the Persist-related ones, that either
> have the config line number included or not in the hash, ala
> "IgnoreConfigFileChange" (or whatever)

That's a new directive too, very specialized (though it's your point).
Wouldn't it end up being declined to other modules one day or the other?
But mainly, rather than ignoring config changes, I think mod_proxy_lb
should ignore non-changes.
And it already knows how to do that provided the vhost's balancer id
does not change unnecessarily...

It really matters for (graceful) restarts only, because otherwise we
don't need to reuse names or slotmems (besides Mark's unusual issue,
see [1] below).
Why would one restart httpd if not to apply some configuration change?
Whether this change relates to balancers or not is something the
webmaster shouldn't care about (and figure out if
IgnoreConfigFileChange should be on/off, depending on the kind of
change, before the next restart).


Regards,
Yann.

[1] AIUI, the issue Mark encounters with balancers is that the main
process crashes because of third party modules, leaking IPC-SysV SHMs
to the system (many balancers thus many leaks).
Since his configuration file changes often, the SHMs are not cleaned
up on the next startup either (their names changed with line numbers).
So I proposed, 

Re: New ServerUID directive

[Helmut K. C. Tessarek]

On 2018-02-01 12:54, Yann Ylavic wrote:
> I have this patch (attached) floating around that allows users to
> configure a *fixed* UID for each vhost.

What do you mean by *fixed*?

I thought the virtual host itself already has to be unique. As far as I
know, I can't have two virtual hosts with the same hostname and port.

So why is it necessary to create another unique id?

Cheers,
  K. C.

-- 
regards Helmut K. C. Tessarek  KeyID 0x172380A011EF4944
Key fingerprint = 8A55 70C1 BD85 D34E ADBC 386C 1723 80A0 11EF 4944

/*
   Thou shalt not follow the NULL pointer for chaos and madness
   await thee at its end.
*/



signature.asc
Description: OpenPGP digital signature

Re: New ServerUID directive

[Jim Jagielski]

But we already have a unique ID in the form of
the actual server struct itself, which contains
all the data required to make a vhost "unique"

The only "issue" is what makes a host "unique"... IMO,
in an environ where the definition line of the vhost changes
between restarts is a "new" vhost and that was the assumption
that the implementation was based on.

*But that assumption can more easily be changed by providing
a work-around for that specific use case* instead of creating a
whole new Directive for a "generic" server UID which:

  1. Implies that the webmaster must know when and how that UID
 is used internally
  2. Add more Directive cruft to an already bloated collection
 of directives.

Instead, as I said, we should have a specific balancer directive
or option, complementing the Persist-related ones, that either
have the config line number included or not in the hash, ala
"IgnoreConfigFileChange" (or whatever)

> On Feb 2, 2018, at 9:52 AM, Yann Ylavic <ylavic@gmail.com> wrote:
> 
> On Fri, Feb 2, 2018 at 3:50 PM, Yann Ylavic <ylavic@gmail.com> wrote:
>> On Fri, Feb 2, 2018 at 3:44 PM, Stefan Eissing
>> <stefan.eiss...@greenbytes.de> wrote:
>>> 
>>> 
>>>> Am 02.02.2018 um 15:42 schrieb Yann Ylavic <ylavic@gmail.com>:
>>>> 
>>>> On Fri, Feb 2, 2018 at 3:25 PM, Plüm, Rüdiger, Vodafone Group
>>>> <ruediger.pl...@vodafone.com> wrote:>
>>>>> 
>>>>>> -Ursprüngliche Nachricht- Von: Jim Jagielski
>>>>>> [mailto:j...@jagunet.com] Gesendet: Freitag, 2. Februar 2018 15:15
>>>>>> An: httpd <dev@httpd.apache.org> Betreff: Re: New ServerUID
>>>>>> directive
>>>>>> 
>>>>>> Why? If it is designed to not change between restarts then there
>>>>>> are much easier ways to be unique, which it kinda already is,
>>>>>> considering the actual structs being used.
>>>> 
>>>> I don't know what "easier ways" you are thinking about, the one
>>>> proposed here is not that complicated IMO.
>>>> In any case the method we are currently using in mod_proxy_lb *is*
>>>> changing accross restarts, mainly because of the line number.
>>>> What if you add/remove a line before the ?
>>>> At least for graceful restarts, I think it shall not change, SHMs
>>>> should be reused.
>>> 
>>> Is it a hash across the config record of a server what would give
>>> the desired behaviour?
>> 
>> Yes, a hash using the minimal (IP[:port])* + ServerName, which is what
>> matters to select a vhost in the first place (maybe it's missing
>> ServerAlias), and what's done in this patch.
>> Should two vhosts have the same hash, only the first one will ever
>> handle requests.
> 
> But I still think that the ServerUID directive is useful because the
> above are not immutable for the lifetime of a vhost either.
> That would address all needs, IMO.

Re: New ServerUID directive

[Jim Jagielski]

> On Feb 2, 2018, at 9:42 AM, Stefan Eissing <stefan.eiss...@greenbytes.de> 
> wrote:
> 
>> Am 02.02.2018 um 15:25 schrieb Plüm, Rüdiger, Vodafone Group 
>> <ruediger.pl...@vodafone.com>:
>> 
>> 
>> 
>>> -Ursprüngliche Nachricht-
>>> Von: Jim Jagielski [mailto:j...@jagunet.com]
>>> Gesendet: Freitag, 2. Februar 2018 15:15
>>> An: httpd <dev@httpd.apache.org>
>>> Betreff: Re: New ServerUID directive
>>> 
>>> Why? If it is designed to not change between restarts then
>>> there are much easier ways to be unique, which it kinda
>>> already is, considering the actual structs being used.
>>> 
>>> Also, this seems like unnecessary admin overhead for the
>>> webmaster... if there is a need for such an ID, httpd should
>>> provide for it automagically and not require users to have
>>> to config one. It seems like config-file fluff to me, IMO.
>> 
>> +1. If the current ID used is not meeting our requirements, my first thought 
>> would be if we could improve it
>> to meet them.
> 
> Totally agree that inventing a new Directive is the last resort if we cannot 
> find a better solution.
> 
> If I understand then concrete case here correctly, the admin makes frequent 
> config changes and *wants* the shared memory id to stay the same, so the shm 
> is re-used and not cleaned up or re-initialized. And our current 
> implementation generates new identifiers much to frequently bc the 
> line-number cheat was made to detect config changes?

I think that is correct, which implies, to me, the solution would
be some mod_proxy_balancer Directive like "IgnoreConfigFileChanges"
or something specific like that.

But I am still confused if the above is actually what the issue
is, hence my "resistance" for any modifications at all until
the problem is crystal clear. If we can check, in a one-off
test, that removing that line-number hack solves the "problem"
then that is more easily handled by a specific use-case for
balancer/slotmem.

My 2c

Re: New ServerUID directive

[Stefan Eissing]

> Am 02.02.2018 um 16:43 schrieb Yann Ylavic :
> 
> On Fri, Feb 2, 2018 at 4:28 PM, Stefan Eissing
>  wrote:
>> 
>>> Am 02.02.2018 um 15:50 schrieb Yann Ylavic :
>>> 
 Is it a hash across the config record of a server what would give
 the desired behaviour?
>>> 
>>> Yes, a hash using the minimal (IP[:port])* + ServerName, which is what
>>> matters to select a vhost in the first place (maybe it's missing
>>> ServerAlias), and what's done in this patch.
>>> Should two vhosts have the same hash, only the first one will ever
>>> handle requests.
>> 
>> So, for my understanding, what we are looking for it:
>> 
>> const char * ap_get_server_id(server_rec *s);
> 
> Yes, mainly, which:
> {
>return r->server_id;
> }
> :)
> 
>> 
>> that gives a string usable as a file name which does not change
>> unless the given server has "changed"? And the distinction of
>> what exactly is the "change", we probably can argue millenia
>> about?
> 
> Precisely, the admin knows..
> That doesn't prevent us from having a beer and talk about when he/she
> doesn't care and simply wants it to work ;)

I need to ponder this with a nice bottle of wine this evening...maybe.

My gut feeling is that, by pure numbers, most users want Apache to
serve domains one next to the other. They do not want overlaps, they
want http: and https: for the same domain to show the same resource (or
have the former redirect to the later).

I think they would like Apache to refuse a configuration where this
is not somehow true (e.g. ServerAlias/Name overlapping, a Location
slipped into one vhost of a domain, but not the other). The would
like Apache to answer 404 for any domain that is not configured.

Then we have fewer, but possibly larger and "more interesting" setups
where people use the config possibilities up to the max, know which
virtual host a fallback for unknown domains is, etc. etc. etc.

How far would come in satisfying all by having a core directive

   DomainScope  global | ports | sockets | free

with meaning:
free:as is now
sockets: domain settings are unique for each address+port (can be same for 
different addresses),
 e.g. ServerName/-Alias props can differ on each listening socket
ports:   domain settings are unique for each port number (irregardless of 
listening address)
 e.g. ServerName/-Alias props can differ on other ports
global:  domain settings are unique for all ports and addresses, 
 e.g. ServerName/-Alias props are the same everywhere in this server
?

Re: New ServerUID directive

[Yann Ylavic]

On Fri, Feb 2, 2018 at 4:28 PM, Stefan Eissing
 wrote:
>
>> Am 02.02.2018 um 15:50 schrieb Yann Ylavic :
>>
>>> Is it a hash across the config record of a server what would give
>>> the desired behaviour?
>>
>> Yes, a hash using the minimal (IP[:port])* + ServerName, which is what
>> matters to select a vhost in the first place (maybe it's missing
>> ServerAlias), and what's done in this patch.
>> Should two vhosts have the same hash, only the first one will ever
>> handle requests.
>
> So, for my understanding, what we are looking for it:
>
> const char * ap_get_server_id(server_rec *s);

Yes, mainly, which:
{
return r->server_id;
}
:)

>
> that gives a string usable as a file name which does not change
> unless the given server has "changed"? And the distinction of
> what exactly is the "change", we probably can argue millenia
> about?

Precisely, the admin knows..
That doesn't prevent us from having a beer and talk about when he/she
doesn't care and simply wants it to work ;)

Re: New ServerUID directive

[Stefan Eissing]

> Am 02.02.2018 um 15:50 schrieb Yann Ylavic <ylavic@gmail.com>:
> 
> On Fri, Feb 2, 2018 at 3:44 PM, Stefan Eissing
> <stefan.eiss...@greenbytes.de> wrote:
>> 
>> 
>>> Am 02.02.2018 um 15:42 schrieb Yann Ylavic <ylavic@gmail.com>:
>>> 
>>> On Fri, Feb 2, 2018 at 3:25 PM, Plüm, Rüdiger, Vodafone Group
>>> <ruediger.pl...@vodafone.com> wrote:>
>>>> 
>>>>> -Ursprüngliche Nachricht- Von: Jim Jagielski
>>>>> [mailto:j...@jagunet.com] Gesendet: Freitag, 2. Februar 2018 15:15
>>>>> An: httpd <dev@httpd.apache.org> Betreff: Re: New ServerUID
>>>>> directive
>>>>> 
>>>>> Why? If it is designed to not change between restarts then there
>>>>> are much easier ways to be unique, which it kinda already is,
>>>>> considering the actual structs being used.
>>> 
>>> I don't know what "easier ways" you are thinking about, the one
>>> proposed here is not that complicated IMO.
>>> In any case the method we are currently using in mod_proxy_lb *is*
>>> changing accross restarts, mainly because of the line number.
>>> What if you add/remove a line before the ?
>>> At least for graceful restarts, I think it shall not change, SHMs
>>> should be reused.
>> 
>> Is it a hash across the config record of a server what would give
>> the desired behaviour?
> 
> Yes, a hash using the minimal (IP[:port])* + ServerName, which is what
> matters to select a vhost in the first place (maybe it's missing
> ServerAlias), and what's done in this patch.
> Should two vhosts have the same hash, only the first one will ever
> handle requests.

So, for my understanding, what we are looking for it:

const char * ap_get_server_id(server_rec *s);

that gives a string usable as a file name which does not change
unless the given server has "changed"? And the distinction of 
what exactly is the "change", we probably can argue millenia
about?

-Stefan

Re: New ServerUID directive

[Dirk-Willem van Gulik]

> On 2 Feb 2018, at 15:44, Stefan Eissing <stefan.eiss...@greenbytes.de> wrote:
> 
> 
> 
>> Am 02.02.2018 um 15:42 schrieb Yann Ylavic <ylavic@gmail.com>:
>> 
>> On Fri, Feb 2, 2018 at 3:25 PM, Plüm, Rüdiger, Vodafone Group
>> <ruediger.pl...@vodafone.com> wrote:>
>>> 
>>>> -Ursprüngliche Nachricht- Von: Jim Jagielski
>>>> [mailto:j...@jagunet.com] Gesendet: Freitag, 2. Februar 2018 15:15
>>>> An: httpd <dev@httpd.apache.org> Betreff: Re: New ServerUID
>>>> directive
>>>> 
>>>> Why? If it is designed to not change between restarts then there
>>>> are much easier ways to be unique, which it kinda already is,
>>>> considering the actual structs being used.
>> 
>> I don't know what "easier ways" you are thinking about, the one
>> proposed here is not that complicated IMO.
>> In any case the method we are currently using in mod_proxy_lb *is*
>> changing accross restarts, mainly because of the line number.
>> What if you add/remove a line before the ?
>> At least for graceful restarts, I think it shall not change, SHMs
>> should be reused.
> 
> Is it a hash across the config record of a server what would give
> the desired behaviour?

We have several internal modules which sort of need this. In each case we 
generate a  sha256 and a UUID based on the FQDN or IP address and the (first) 
port number it listens on. 

And we then log this with ‘info’ during startup; thus allowing an admin to at 
some later point add a ServerUUID to the list of ServerName, ServerAdmin, etc. 
as he or she sees fit.

As in practice - we found that in the few times when you need to change that 
UUID - you are in effect doing that as part of switching to a new origin server 
or something like that at protocol level with respect to things like locks, 
caches and other short-transactions/connection RESTy things surviving stuff.

That said - having this in the core and being able to routinely seed this extra 
into things like cookies is goodness. We do that a lot now - and it helps with 
simplifying compliance a lot.

Dw.

Re: New ServerUID directive

[Yann Ylavic]

On Fri, Feb 2, 2018 at 3:50 PM, Yann Ylavic <ylavic@gmail.com> wrote:
> On Fri, Feb 2, 2018 at 3:44 PM, Stefan Eissing
> <stefan.eiss...@greenbytes.de> wrote:
>>
>>
>>> Am 02.02.2018 um 15:42 schrieb Yann Ylavic <ylavic@gmail.com>:
>>>
>>> On Fri, Feb 2, 2018 at 3:25 PM, Plüm, Rüdiger, Vodafone Group
>>> <ruediger.pl...@vodafone.com> wrote:>
>>>>
>>>>> -Ursprüngliche Nachricht- Von: Jim Jagielski
>>>>> [mailto:j...@jagunet.com] Gesendet: Freitag, 2. Februar 2018 15:15
>>>>> An: httpd <dev@httpd.apache.org> Betreff: Re: New ServerUID
>>>>> directive
>>>>>
>>>>> Why? If it is designed to not change between restarts then there
>>>>> are much easier ways to be unique, which it kinda already is,
>>>>> considering the actual structs being used.
>>>
>>> I don't know what "easier ways" you are thinking about, the one
>>> proposed here is not that complicated IMO.
>>> In any case the method we are currently using in mod_proxy_lb *is*
>>> changing accross restarts, mainly because of the line number.
>>> What if you add/remove a line before the ?
>>> At least for graceful restarts, I think it shall not change, SHMs
>>> should be reused.
>>
>> Is it a hash across the config record of a server what would give
>> the desired behaviour?
>
> Yes, a hash using the minimal (IP[:port])* + ServerName, which is what
> matters to select a vhost in the first place (maybe it's missing
> ServerAlias), and what's done in this patch.
> Should two vhosts have the same hash, only the first one will ever
> handle requests.

But I still think that the ServerUID directive is useful because the
above are not immutable for the lifetime of a vhost either.
That would address all needs, IMO.

Re: New ServerUID directive

[Yann Ylavic]

On Fri, Feb 2, 2018 at 3:44 PM, Stefan Eissing
<stefan.eiss...@greenbytes.de> wrote:
>
>
>> Am 02.02.2018 um 15:42 schrieb Yann Ylavic <ylavic@gmail.com>:
>>
>> On Fri, Feb 2, 2018 at 3:25 PM, Plüm, Rüdiger, Vodafone Group
>> <ruediger.pl...@vodafone.com> wrote:>
>>>
>>>> -Ursprüngliche Nachricht- Von: Jim Jagielski
>>>> [mailto:j...@jagunet.com] Gesendet: Freitag, 2. Februar 2018 15:15
>>>> An: httpd <dev@httpd.apache.org> Betreff: Re: New ServerUID
>>>> directive
>>>>
>>>> Why? If it is designed to not change between restarts then there
>>>> are much easier ways to be unique, which it kinda already is,
>>>> considering the actual structs being used.
>>
>> I don't know what "easier ways" you are thinking about, the one
>> proposed here is not that complicated IMO.
>> In any case the method we are currently using in mod_proxy_lb *is*
>> changing accross restarts, mainly because of the line number.
>> What if you add/remove a line before the ?
>> At least for graceful restarts, I think it shall not change, SHMs
>> should be reused.
>
> Is it a hash across the config record of a server what would give
> the desired behaviour?

Yes, a hash using the minimal (IP[:port])* + ServerName, which is what
matters to select a vhost in the first place (maybe it's missing
ServerAlias), and what's done in this patch.
Should two vhosts have the same hash, only the first one will ever
handle requests.

Re: New ServerUID directive

[Stefan Eissing]

> Am 02.02.2018 um 15:42 schrieb Yann Ylavic <ylavic@gmail.com>:
> 
> On Fri, Feb 2, 2018 at 3:25 PM, Plüm, Rüdiger, Vodafone Group
> <ruediger.pl...@vodafone.com> wrote:>
>> 
>>> -Ursprüngliche Nachricht- Von: Jim Jagielski
>>> [mailto:j...@jagunet.com] Gesendet: Freitag, 2. Februar 2018 15:15
>>> An: httpd <dev@httpd.apache.org> Betreff: Re: New ServerUID
>>> directive
>>> 
>>> Why? If it is designed to not change between restarts then there
>>> are much easier ways to be unique, which it kinda already is,
>>> considering the actual structs being used.
> 
> I don't know what "easier ways" you are thinking about, the one
> proposed here is not that complicated IMO.
> In any case the method we are currently using in mod_proxy_lb *is*
> changing accross restarts, mainly because of the line number.
> What if you add/remove a line before the ?
> At least for graceful restarts, I think it shall not change, SHMs
> should be reused.

Is it a hash across the config record of a server what would give
the desired behaviour?

Re: New ServerUID directive

[Stefan Eissing]

> Am 02.02.2018 um 15:25 schrieb Plüm, Rüdiger, Vodafone Group 
> <ruediger.pl...@vodafone.com>:
> 
> 
> 
>> -Ursprüngliche Nachricht-
>> Von: Jim Jagielski [mailto:j...@jagunet.com]
>> Gesendet: Freitag, 2. Februar 2018 15:15
>> An: httpd <dev@httpd.apache.org>
>> Betreff: Re: New ServerUID directive
>> 
>> Why? If it is designed to not change between restarts then
>> there are much easier ways to be unique, which it kinda
>> already is, considering the actual structs being used.
>> 
>> Also, this seems like unnecessary admin overhead for the
>> webmaster... if there is a need for such an ID, httpd should
>> provide for it automagically and not require users to have
>> to config one. It seems like config-file fluff to me, IMO.
> 
> +1. If the current ID used is not meeting our requirements, my first thought 
> would be if we could improve it
> to meet them.

Totally agree that inventing a new Directive is the last resort if we cannot 
find a better solution.

If I understand then concrete case here correctly, the admin makes frequent 
config changes and *wants* the shared memory id to stay the same, so the shm is 
re-used and not cleaned up or re-initialized. And our current implementation 
generates new identifiers much to frequently bc the line-number cheat was made 
to detect config changes?

-Stefan

Re: New ServerUID directive

[Yann Ylavic]

On Fri, Feb 2, 2018 at 3:25 PM, Plüm, Rüdiger, Vodafone Group
<ruediger.pl...@vodafone.com> wrote:>
>
>> -Ursprüngliche Nachricht- Von: Jim Jagielski
>> [mailto:j...@jagunet.com] Gesendet: Freitag, 2. Februar 2018 15:15
>> An: httpd <dev@httpd.apache.org> Betreff: Re: New ServerUID
>> directive
>>
>> Why? If it is designed to not change between restarts then there
>> are much easier ways to be unique, which it kinda already is,
>> considering the actual structs being used.

I don't know what "easier ways" you are thinking about, the one
proposed here is not that complicated IMO.
In any case the method we are currently using in mod_proxy_lb *is*
changing accross restarts, mainly because of the line number.
What if you add/remove a line before the ?
At least for graceful restarts, I think it shall not change, SHMs
should be reused.

>>
>> Also, this seems like unnecessary admin overhead for the
>> webmaster... if there is a need for such an ID, httpd should
>> provide for it automagically and not require users to have to
>> config one. It seems like config-file fluff to me, IMO.
>
> +1. If the current ID used is not meeting our requirements, my first
> thought would be if we could improve it to meet them.

I'm fine with removing the directive and let the automatic UID be
generated at startup, the thing is already there in this patch.
The directive simply helps for the case when the admin knows what to
do, and for instance wants the vhosts to remain the same even if an IP
address or a ServerName is changed.


Regards,
Yann.

Re: New ServerUID directive

[Jim Jagielski]

Why? If it is designed to not change between restarts then
there are much easier ways to be unique, which it kinda
already is, considering the actual structs being used.

Also, this seems like unnecessary admin overhead for the
webmaster... if there is a need for such an ID, httpd should
provide for it automagically and not require users to have
to config one. It seems like config-file fluff to me, IMO.

> On Feb 1, 2018, at 12:54 PM, Yann Ylavic  wrote:
> 
> I have this patch (attached) floating around that allows users to
> configure a *fixed* UID for each vhost.
> 
> There are several places where we need an ID for vhosts, and where we
> compute one based on ServerName/Port, addresses, configuration path
> and line numbers, ...
> This UID could be used instead, moreover since it has the property to
> not change on (re)starts we can also use it for things bound to a
> vhost regardless of the startup and unrelated configuration changes
> (the attached patch uses it for SHMs in mod_proxy_balancer, as an
> example).
> 
> If no ServerUID is configured, I _think_ we can compute one too,
> better than the one usually computed in our code since it won't change
> unless address(es)/port(s) or ServerName of the vhost changes (which
> is not a "light" change anyway).
> In any case the patch also handles collisions, if ever...
> So for this what the patch does is (with rationale in comment):
> 
> +int i, *num;
> +apr_hash_t *servers_uids = apr_hash_make(p);
> []
> +if (!s->server_uid) {
> +server_addr_rec *addr;
> +apr_md5_ctx_t md5_ctx;
> +unsigned char md5[APR_MD5_DIGESTSIZE];
> +
> +/* Assumes the unique identifier of a vhost is its address(es)
> + * plus the ServerName:Port. Should two or more vhosts have this
> + * same identifier, the first one would always be elected to
> + * handle the requests, so this shouldn't be an issue...
> + */
> +apr_md5_init(_ctx);
> +for (addr = s->addrs; addr; addr = addr->next) {
> +char host_ip[64]; /* for any IPv[46] string */
> +apr_sockaddr_ip_getbuf(host_ip, sizeof host_ip,
> +   addr->host_addr);
> +apr_md5_update(_ctx, (unsigned char *)host_ip,
> + strlen(host_ip));
> +apr_md5_update(_ctx, (unsigned char *)>host_port,
> + sizeof(addr->host_port));
> +}
> +apr_md5_update(_ctx, (unsigned char *)s->server_hostname,
> + strlen(s->server_hostname));
> +apr_md5_update(_ctx, (unsigned char *)>port,
> + sizeof(s->port));
> +apr_md5_final(md5, _ctx);
> +
> +s->server_uid = apr_pescape_hex(p, md5, sizeof md5, 0);
> +}
> +/* Handle collisions, that's Unique ID! */
> +num = apr_hash_get(servers_uids, s->server_uid,
> +   APR_HASH_KEY_STRING);
> +if (num) {
> +++*num;
> +s->server_uid = apr_psprintf(p, "%s_%i", s->server_uid, *num);
> +}
> +else {
> +num = apr_pcalloc(p, sizeof *num);
> +apr_hash_set(servers_uids, s->server_uid,
> + APR_HASH_KEY_STRING, num);
> +}
> (Rest attached)
> 
> WDYT?
> 

Re: New ServerUID directive

[Exonetric]

On 1 Feb 2018, at 17:54, Yann Ylavic  wrote:
> 
> I have this patch (attached) floating around that allows users to
> configure a *fixed* UID for each vhost.
> 
> There are several places where we need an ID for vhosts, and where we
> compute one based on ServerName/Port, addresses, configuration path
> and line numbers, ...
> This UID could be used instead, moreover since it has the property to
> not change on (re)starts we can also use it for things bound to a
> vhost regardless of the startup and unrelated configuration changes
> (the attached patch uses it for SHMs in mod_proxy_balancer, as an
> example).
> 
> If no ServerUID is configured, I _think_ we can compute one too,
> better than the one usually computed in our code since it won't change
> unless address(es)/port(s) or ServerName of the vhost changes (which
> is not a "light" change anyway).
> In any case the patch also handles collisions, if ever...
> So for this what the patch does is (with rationale in comment):
> 
> +int i, *num;
> +apr_hash_t *servers_uids = apr_hash_make(p);
> []
> +if (!s->server_uid) {
> +server_addr_rec *addr;
> +apr_md5_ctx_t md5_ctx;
> +unsigned char md5[APR_MD5_DIGESTSIZE];
> +
> +/* Assumes the unique identifier of a vhost is its address(es)
> + * plus the ServerName:Port. Should two or more vhosts have this
> + * same identifier, the first one would always be elected to
> + * handle the requests, so this shouldn't be an issue...
> + */
> +apr_md5_init(_ctx);
> +for (addr = s->addrs; addr; addr = addr->next) {
> +char host_ip[64]; /* for any IPv[46] string */
> +apr_sockaddr_ip_getbuf(host_ip, sizeof host_ip,
> +   addr->host_addr);
> +apr_md5_update(_ctx, (unsigned char *)host_ip,
> + strlen(host_ip));
> +apr_md5_update(_ctx, (unsigned char *)>host_port,
> + sizeof(addr->host_port));
> +}
> +apr_md5_update(_ctx, (unsigned char *)s->server_hostname,
> + strlen(s->server_hostname));
> +apr_md5_update(_ctx, (unsigned char *)>port,
> + sizeof(s->port));
> +apr_md5_final(md5, _ctx);
> +
> +s->server_uid = apr_pescape_hex(p, md5, sizeof md5, 0);
> +}
> +/* Handle collisions, that's Unique ID! */
> +num = apr_hash_get(servers_uids, s->server_uid,
> +   APR_HASH_KEY_STRING);
> +if (num) {
> +++*num;
> +s->server_uid = apr_psprintf(p, "%s_%i", s->server_uid, *num);
> +}
> +else {
> +num = apr_pcalloc(p, sizeof *num);
> +apr_hash_set(servers_uids, s->server_uid,
> + APR_HASH_KEY_STRING, num);
> +}
> (Rest attached)
> 
> WDYT?

I am surprised httpd doesn’t already have this, considering the fundamental 
role of the virtualhost container.  

- Mark