Over the years spent on and off with mod_proxy_cluster [1]
it has always been a problem and a bottleneck to maintain
fresh shared information among httpd's processes and threads
about joining _and leaving_ workers. Especially when the number
of workers is in many hundreds and the joining/leaving fluctuation
is a regular ongoing activity.

The implementation is basically shm/slotmem with all its
boons and banes.


1) Would it make sense to get rid of all shm and offload the information
sharing for these highly dynamic load balancing scenarios to LMDB [2]?

I learned about LMDB while working with Knot Resolver [3]. Knot Resolver
is a modern high-performance resolver (CZ-NIC uses it in our Czech TLD
It scales with processes using event loop - and all its processes share a cache 
resolved domains [4]. Over the years LMDB emerged as the winning backend for
that cache.

2) Given its license [5], would it ever be even possible for httpd trunk
to depend on it?

3) Is there any similar ongoing effort? I remember Jim's notes on nng
for messaging between workers and balancers. But I don't think he meant
it to also serve for httpd processes to exchange information about workers...?
Did you? :)

4) Does it even make sense to be spending any time with anything like this
or is it a solved problem in some mod_proxy* library I am missing?


[1] http://modcluster.io/
[2] https://en.wikipedia.org/wiki/Lightning_Memory-Mapped_Database
[3] https://github.com/CZ-NIC/knot-resolver
[4] https://knot-resolver.readthedocs.io/en/stable/daemon.html
[5] http://www.openldap.org/software/release/license.html
[6] https://github.com/nanomsg/nng

Michal Karm Babacek

Sent from my Hosaka Ono-Sendai Cyberspace 7

Reply via email to