Hi Apollon,
On Mon, Jul 28, 2014 at 12:54:07PM +0300, Apollon Oikonomopoulos wrote:
> > - multi-process : better synchronization of stats and health checks,
> > and find a way to support peers in this mode. I'm still thinking a
> > lot that due to the arrival of latency monsters that are SSL and
> > compression, we could benefit from having a thread-based architecture
> > so that we could migrate tasks to another CPU when they're going to
> > take a lot of time. The issue I'm seeing with threads is that
> > currently the code is highly dependent on being alone to modify any
> > data. Eg: a server state is consistent between entering and leaving
> > a health check function. We don't want to start adding huge mutexes
> > everywhere.
>
> How about using shared memory segments for stats, health checks and
> peers?
I thought I wrote about it and re-read my e-mail and it's not there :-)
In short :
- stats: should definitely be in a shared memory segment, and we can
get accurate stats using atomic operations. There are a few tricky
points (eg: concurrent conns) which are used both as stats and for
the process's decisions (queue or not to queue) but I think it can
be addressed ;
- peers: no problem on output, but not easy on input since the incoming
data must be distributed to all consumers, and we must prevent ACKs
from being sent before all of them get the information. Instead, I'd
like to have the stick tables shared between processes, and have only
one process handle the synchronization (eg: abitrarily the first one).
There are still issues caused by the independant per-process contexts.
For example, when a process updates a stick-table contents, it has no
way of waking up the other process to detect the change and program an
update propagation. Still I think that could be manageable.
- health checks: definitely not suited for shared memory right now, as
all the code assumes that the state is stable along a whole function,
so instead we need to broadcast event change notification.
It's all these apparently easy but practically hard to synchronize issues
that make me think we'd rather go down the thread route than the process
route, because at least any thread which detects a change can do all the
relevant job by itself.
> > If anyone has any comment / question / suggestion, as usual feel free to
> > keep the discussion going on.
>
> Could I also add shared SSL session cache over multiple boxes (like
> stud), to aid SSL scalability behind LVS directors? It has been asked
> for before in the mailing list if I recall correctly.
Good point, and Emeric initially wrote the code for Stud as a PoC so
that we could get it afterwards. There's not much that needs to be done,
mosly implement a better UDP protocol handling and UDP receivers. However
as Dirkjan has pointed out, it is possible that by the time this is
achieved, TLS tickets will have obsoleted this feature. Finding a way
to intelligently exchange a seed for TLS tickets can be interesting. For
instance, we could have a large cyclic pseudo-random generator implemented
in all nodes seeded with a shared secret. Each node could randomly decide
to skip entries and regularly broadcast the number of entries skipped, so
that all other nodes have a way to jump to the new key. I don't know how
that would resist to restarts though.
Regards,
Willy
