On Fri, Oct 02, 2009 at 10:19:47PM +0200, Aleksandar Lazic wrote:
> Hm so if I want to add ajp,fcgi, ... protocol I think I will need a
> buffer for the 'header' and for the content.
Yes, the principle is to chain two series of buffers like this :
b1 b4
,---->[....]----v ,----->[....]----v
(SI1) (SI2)(SI3) (SI4)
^-----[....]----' ^------[....]----'
b2 b3
The client's connection attaches to SI1 (mode HTTP) and sends its
received data to buffer B1, and sends data to client from B2.
HTTP contents are then transfered to interface SI2 which is
handled as an autonomous task. So basically the task is woken up
and that's all. When this task wakes up, it performs http->fcgi
transform between SI2 and SI3 and sends the transformed data into
B4, to reach the server via a stream sock SI4. Etc...
That way you have two independant tasks, one for the client's
connection, one for the fcgi "session", which know each other
and are able to wake each other up when there is some work to
do.
This is how we want the SSL transformations to be performed too.
A colleague of mine already tried to implement FCGI last year
around 1.3.15 and almost succeeded, but he agreed this was
awful due to the lack of infrastructure help from haproxy's
lower layers. But it was very useful as a proof-of-concept
to find out what was wrong with that architecture and what
needed to evolve. Now it should be a lot easier, but since
the same colleague also intends to work on SSL again, I'll
not hurry him too much :-)
> Do you think it is the right time to add it to haproxy or should I wait
> until the 'interface' is more ready?
both :-)
To be quite honnest, it is a bit early but it is also what helps
the design evolve towards the right direction. If you start to
implement something now, I'm absolutely certain that you will
regularly block because of missing feature X or Y and that sometimes
I won't even have any immediate response. This is sometimes very
frustrating for everyone. But if you don't waste your time on it
and just start it as a PoC, it should be an interesting experience
which can serve both of us.
> Do you plan to make it possible as dynamic librarys?
I was planning to do so but am hesitating now :
- the internal API is still changing too fast, so it is very
risky for the people who'd use those libs (even for those
who develop them)
- even if we implement a means of loading a dyn lib just before
the release, it brings a major stability issue. When I see
how many times I break something during the development, I
realize how much the code is sensible to minor mistakes. It
seems like few people are able to think "event-driven" and
that by encouraging external devs, we'd lose some reviewing,
causing more lockups, busy loops, crashes and memory leaks
to happen.
Most likely we'd go for a small out-of-tree patch which would
help developers test their code without rebuilding everything.
But in the short term, I'm not that sure anymore about loadble
libs for the end user.
> >Right now, a huge stickiness rework is being funded and performed by
> >Exceliance and Loadbalancer.org. Two steps are planned, the first one
> >consisting in rearchitecturing the internals, and the second one,
> >consisting in finding how to transfer data between processes (old to
> >new at first, then possibly permanent sync).
> >
> >I think we should wait for the first step to be complete and discuss
> >the ideas then. After some time thinking on the matter, I feel like we
> >never need to sync large amounts of LBs together and that having
> >full-mesh communications would be more than enough and would ensure
> >synchronization without the overhead of something like TT. Also, the TT
> >server would have itself to be backed up and synchronized, resulting in
> >a more complex setup for small deployments. But right now these are
> >just feelings, neither arguments nor directions to take.
>
> Ok,I will wait for this.
> Is there a 'open' discussion about this issue?!
Not particularly since I have discussed it with a coworker with
pencil and paper, but there's nothing closed. I'll try to take
some time to explain here what we intend to build. It's basically
what has already been discussed here months (years?) ago :
extract, store, match.
> >>Which balance methode is the most used one?
> >
> >From what I have seen in configs I receive in private or find on the
> >net, the roundrobin is the clear winner, with the other ones used only
> >when there is a real reason for this.
>
> So no stickyness on the loadbalancer, so the most application use the
> cookies and shared storage for the sessions, right.
Oh sorry if I was not clear in my response. Since you only asked
about the LB method, I thought you were only interested in the
algo. Stickiness based on cookie insertion is almost always used
with HTTP, since it's the easiest one to deploy.
> >>Does anybody use the appsession feature?
> >
> >I know that some people use it, as I have received some configs where
> >it was enabled and people had good reasons for this. I remember that
> >the main reason was the same you had when you developed it : support
> >for some very specific browsers which don't support cookies at all (eg:
> >WAP or things like this).
>
> This was my first 'session based methode' to make it HA ;-).
> If so less user use it I'am not sure if it is worth until the 'global'
> handling is designed.
The new global handling should cover it as well (we discussed it BTW,
explaining why we need multiple matches, cookies+url for instance).
Then once the new design is ready, we can convert the appsession code
to rely on it and all the ongoing work for shared sessions will be
usable for appsessions too.
Regards,
Willy
